Commit Graph

271 Commits

Author SHA1 Message Date
Tony Murray
c22c879983 fix: Remove faulty memcached code (not related to distributed polling) (#7881) 2017-12-10 20:40:45 +00:00
Tony Murray
b620692426 refactor: Update all applications to store metrics (#7853)
* Update all applications to store metrics

* fix db schema

* Add glue test
2017-12-06 22:13:10 +00:00
Tony Murray
5405498ed6 alerts: Adde generic alerts for state sensors (#7812)
* feature: Generic alerts for state sensors
A bit of a hack, but gets the job done. Perhaps someone can refactor the ResolveGlues() function sometime to make it better.

* Fix mistakes

* Add docs and collection rules, critical enabled by default

* moved schema file to 221.sql
2017-12-02 23:03:36 +00:00
Tony Murray
4a03e7838e feature: Save application metrics to db for alerting (#7828)
* feature: save application metrics to db for alerting
However, alerting will not work because ResolveGlues() is broken.
Can add workaround after state_translations alerting is merged

Does not update all applications yet, not sure if that should be done here or in another PR.

Introduces two handy functions dbDeleteOrphans() and array_by_column().  Will replace those in other locations after this is merged or separate them out if this is not merged.

* remove accidental inclusions

* Add db schema
2017-12-01 07:53:26 +00:00
Tony Murray
257a314fca feature: Validate session directory is writable (#7784) 2017-11-23 08:29:10 +00:00
Tony Murray
0dee809607 fix: Discovery os changes weren't properly reflected if os changed (#7779) 2017-11-22 21:48:28 +00:00
Jonathan Lassoff
fe8d011dce refactor: Allow setting the target shorthost length. (#7453)
* Create a target shorthost length. Addresses #7452.

* Update defaults.inc.php
2017-11-09 20:47:52 +00:00
Tony Murray
cbe6cb532d Fix: Discovery validation with ping-only devices (#7665)
* fix: validation check cli php extensions
Fix fatal errors when mysql is not loaded / can't connect

* fix: discovery validation incorrectly detects on ping only devices.
2017-11-07 20:09:49 +00:00
Tony Murray
54b476c769 fix: devices detected as ibmtl or generic (#7618)
* fix: devices detected as ibmtl
Guess at the sysObjectID, please report if you have devices that should be ibmtl and are no longer detected.
Add output of os in discovery.

* Always load all os for discovery.  Should be cached most of the time.
2017-11-03 16:10:24 -05:00
Neil Lathwood
3e94077a71 Revert "Update RFC1123 Hostname Check" (#7577)
* Revert "Revert "feature: Allow snmpget in os discovery yaml (#7558)" (#7576)"

This reverts commit 426916dbe4.

* Revert "fix: lldp find_port_id finds not the most correct port (#7564)"

This reverts commit 7e05fff615.

* Revert "fix: Update RFC1123 Hostname Check (#7572)"

This reverts commit ebcfa45a93.
2017-10-29 20:05:59 +00:00
Parth Laxmikant Kolekar
ebcfa45a93 fix: Update RFC1123 Hostname Check (#7572)
* Update RFC1123 Hostname Check

The regex was wrong. It failed to match devices like `cisco-3750x` even though it is a valid hostname. 
I picked this regex from the RFC1123 documentation, and updated it.

* Update CommonFunctionsTest.php

Update tests

* Updated to allow domains ending with a dot
2017-10-29 11:12:32 +01:00
Zmegolaz
f8d7ccfe0d feature: Support for up/down detection of ping only devices.
* Added support for ping only devices.

* Renamed sql-schema/206 to 207.

* Discovery tried to detect the OS when it shouldn't.

* Updated db_schema.yaml

* Added ping icon.

* Don't show unused graphs and tabs when SNMP is disabled.

* Allow the user to specify OS of ping only devices.

* Removing custom OS now changes it to 'ping'.

* Removed unnecessary use of mres().

* UI select box for SNMP enable/disable.

* Hide device_ping_perf if ping is disable on device.

* Fixed SNMP settings update status messages.

* Added functionality to add ping only devices via the web ui.

* Added ping only option to addhost.php

* Added ping only support to snmp-scan.py

* Moved sql 208 to 211

* Fixed scrutinizer issues.

* Fixed broken ossearch.

* Added ping only support to the API.

* Updated API doc.

* Added (optional) to OS and hardware description. Hid Port Settings, Applications, Modules, Storage, Processors, Memory and Components from the edit menu

* Style fix.

* Updated ping icon.

* clean() instead of mres(). More escaping. Better help in snmp-scan.py and addhost.php

* Fixed scrutinizer issue.

* Always try SNMP in snmp-scan.py, new option for it in addhost.php. Slice instead of chunk in ajax_ossuggest.php. Other minor style changes.

* Updated sql modifications to insert the new column in the same place as in db_schema.yaml.
2017-10-27 22:59:25 -05:00
Tony Murray
8a051aa295 fix: Fix a couple of OS cache invalidation issues (#7567)
* fix: fix a couple of OS cache invalidation issues
Fixes issues where we might be using outdated os definitions.

* use Config and fix style issue

* only pull in settings from config.php (otherwise will possibly contain populated os config)
2017-10-27 21:03:58 +01:00
Tony Murray
51ba934e11 feature: Added web validation (#7474)
* refactor: convert validations to modules to prep for gui integration

* accidentally dropped, schema update

* fix accidental output to webui

* mention discovery-wrapper.py and re-arrange.

* refine some fix statements

* rename the Config validation group to Configuration.

* fix some scrutinizer issues
remove as many local functions from validator.php as possible
move extensions from pre-check
remove duplicate timezone check
looks like there is some db schema differences between mariadb 10.1 and 10.2, investigating

* mariadb version diff take2

* Check schema version first for database.
Remove stop to go back to command line for install docs.
Add helpful link when there is no devices added to /addhost

* fix incorrect validation for empty string defaults

* Fix style

* Add additional file permissions checks

* catch exception and fail for invalid timezone
Change visuals a bit.

* add php version warning

* fix space
2017-10-26 07:56:09 +01:00
Tony Murray
a2b9342f05 fix: improve accuracy of is_valid_hostname() (#7435)
* fix: improve accuracy of is_valid_hostname()
fixes discovery code attempting to add discover invalid dns names
the old code allowed some invalid hostnames, this is more thorough
add tests

* fix merge error
2017-10-05 16:04:17 -05:00
Tony Murray
c0b267b49c refactor: Discovery protocols re-write (#7380)
* refactor: discovery protocols re-write
extract more code to make it easier to read
use snmpwalk_group
walk lldpRemTable instead of lldpRemoteSystemsData
Optimize discover_device to reduce dns lookups of bad dns names
Make discovery_by_ip behavior consistent between cdp and lldp
move can_skip_discovery to discovery functions

Tested lldp, cdp, and ospf.

* always discover links even if the remote device isn't in system
Fix discover_link always updating

* fix style

* fix update, not fetching id

* increase platform length.  Most devices use sysDescr here, which is max length 255

* remove orphaned links

* update db_schema.yaml
2017-10-04 08:16:23 +01:00
Neil Lathwood
f7e14df90e newdevice: Updated Arista entity-physical support to use high/low values from device (#7156)
* fix: Fixed the high/low thresholds for Arista devices

* final updates

* push threshold changes

* more updates

* another update

* updated entity-sensor for current value calc

* hoping final update
2017-09-07 08:49:32 -05:00
Tony Murray
c8436d2a4f fix: issues with Geist Watchdog
miss-named variable in sensor pre-caching
internal humidity and temperature was discovered twice
humidity was mis-spelled in yaml discovery
temperature and current had incorrect divisor in yaml

remove redundant fahrenheit to celsius function

fixes #7236
2017-08-28 13:00:29 -05:00
Neil Lathwood
25d79ce9d7 newdevice: Added further sensor support for Geist Watchdog (#7143)
* newdevice: Added further sensor support for Geist Watchdog

* relaxed if check
2017-08-19 20:22:23 +01:00
Neil Lathwood
96b010a735 feature: Forced OS Cache rebuild for unit tests (#7163) 2017-08-16 13:58:10 -05:00
Tony Murray
1d777693e1 feature: New IP parsing classes. Removes usage of Pear Net_IPv4 and Net_IPv6. (#7106)
* Refactor remove use of pear Net_IPv4 and Net_IPv6

Caution, I added tests to test the functionality of the new functions, but I'm not 100% certain the replaced usages match 100%.  Please test.

Left usage in generate-iplist.php because I'm not sure the use of that script, so I didn't want to implement additional functionality to support it.

Add support for ipv6 fo ipInNetwork() function

Add phpdocs to functions
Remove accidental inclusion

Use binary operations instead of string for networKFromIp()
Tidy up binary operations in ipInNetwork()

tidy parsing cidr network

Change to a class based implementation.

Update phpdocs.
Fully catch exceptions for hex parsing.

Fix mistake.

support older phpunit versions

Fix php shifting to negative with php 5

* Fix graph authorization. Uncaught exception and checking an ipv4 against an ipv6 network == fail.

* Also remove colons from hex strings.

* Fix typo in bgp polling
Two passes of removing invalid characters from hex strings. (That way we don't mangle valid IPs)
2017-08-08 14:14:58 -05:00
Tony Murray
1e77d4b3ea refactor: validate.php improvements (#6973)
Always output header (may be missing mysql data)
Improve version display, now looks like '1.28-129-g74e6c3edf' <tag>-<commits since tag>-<shortag>
Add mysqli to the list of required extensions, check extensions before init
Add dbIsConnected() function, used to check if we have db in version_info()
Do not die on IPv4 and IPv6 so validate can print errors.
Move git checks so they are output together
fix rrdtool 1.7.0 reports version as 1.7.01.7.0
2017-07-17 19:02:28 +01:00
Aldemir Akpinar
5581c40e19 webui: Order the list of devices in get_all_devices() (#6797) 2017-06-07 14:20:59 +01:00
Tony Murray
b6e18db766 fix: page/graph load speed: part 1 (#6611)
* fix: page/graph load speed: part 1
Speed up load_all_os() by caching the full $config['os'] array
Function time goes from 1.13648s -> 0.00073s
The cache is updated during discovery, but only if it is >22hrs old.
Additionally, the cache is not used and a full yaml load is forced if the cache is >24hrs old.
The cache time is controlled by $config['os_def_cache_time'], I don't see any reason anyone would change that time.

* Delete Session.php

* Include the cache directory.
2017-05-10 08:57:10 +01:00
Neil Lathwood
34c35f3627 webui: Updated remaining display options where we do not show sysName if hostname is IP (#6585)
* webui: Updated remaining display options where we do not show sysName if hostname is IP

* renamed function + simplified
2017-05-05 06:25:58 -05:00
Tony Murray
4dbab407bb refactor: OS discovery tests are now dynamic (#6555)
* refactor: OS discovery tests are now dynamic
All that needs to be done now is supply snmprec files with the proper name. os.snmprec or os_description.snmprec
I also tweaked the loading of yaml files to speed up the tests substantially.
Verifies all os have at least one test and all snmprec files are tested.

* fix typo
2017-05-03 15:52:53 +01:00
Tony Murray
4c0412b14d feature: Wireless Sensors Overhaul (#6471)
* feature: Wireless Sensors
Includes client counts for ios and unifi
Graphing could use some improvement.
Alerting and threshold ui not implemented

WIP: starting OO based wireless sensors.

Class based functionality working
remove old functional files
add schema file

discovery needs to be enabled, not polling

fix up schema

fix Unifi discovery not returning an array

Add some debug when discovering a sensor.
Fix style.

Add missing semicolin

Add a null object (Generic) for OS.
Fill out some phpdocs

Re-organized code
Each sensor type now has it's own discovery and polling interface
Custom polling tested with Unifi CCQ

Left to do:
Implement UI (Graphs and Custom thresholds)
Alerting
Testing

Fix event message text

Remove runDiscovery and runPolling from OS, they are unused and don't belong there.

Cleanups/docs

Missed this file.

Remove the requirement to fetch the current value to check validity.
Do that automatically if current is not specified
A few cleanups here and there

First pass at graphing.
device_ and wireless_ graphs added.

Add RouterOS support

Singleton OS instance isn't required right now.
Remove that to allow some memory to be freed.

Add wireless to the device list metrics.
Make all metrics clickable

Tweak graphs a bit

Implement limit configuration page.
Use sensors page as common code instead of duplicating.
Clean up some javascript interactions:  Allow enter on values to save. Cancel if update is not needed. Enable the clear custom button after setting a custom value.
Add some wireless alert rules to the library.

Add documentation.

Add unifi client counts by ssid in addition to radio.
Optimize Sensor polling a bit.

Add HP MSM clients support (for full controller)
Fix function accessibility

Formalize the discovery and poller interfaces.

Add Xirrus clients and noise floor
move module interfaces to a more appropriate place.
push caching code up to os, unsure about this do to the limitations

No point in selectively enabling wireless discovery.  We only discover if the device supports something.

Add RSSI, Power, and Rate.
Add these sensors for Ubnt Airos.
Clean up some copyrights.

Reduce the amount of files need to add new types.
Leave graph files for consistency and to allow customization.

Remove the old wifi clients graph completely.
ciscowlc should have improved counts (total and per-ssid)

Schema didn't get added.

Impelement the rest of the AirOS sensors
Reformat and re-organize the Airos.php class.

Add several UBNT AirFiber sensors

A few fixes add links to the section headers

Add HP MSM mibs.

* Schema file got dropped in rebase.

* Add wireless menu to view sensors across all devices.
Icons in the menu need help :/

* Add HeliOS, Mimosa, and Siklu support
Sensors added SNR + Noise

* Add power and utilization to Unifi

* Update polling to prefetch all sensor data in a few snmp requests as possible

* Add Extendair: tx+rx power, aggregate rate, frequency

* Add a check for duplicate sensors in discovery.  Just print an error for now.

* Add Bit Error Ratio (named error-ratio to allow for bit error rate to be added if needed)
Fix an incorrect link in the wireless sensors table

* Add error rate and change all bps and Hz to use si units

* Fixes to limits and frequency display

* Fix overview graph frequency display
A few decimal place tweaks

* Don't allow switching sensor and wireless-sensor graphs, it doesn't work.
Change individual distance graphs to use si units

* Go through the OS and make sure I got all the sensors I can (probably missed some still)
Because pollWirelessChannelAsFrequency() is generic and a little complex, so pull it up to OS.
Message to help developers adding supports that don't return an array from discover functions.

* Fix some issues

* Remove noise and signal for now at least
A couple more fixes
Add a notification

* Oopsie

* Bonus AirFiber sensors
2017-05-01 23:49:11 -05:00
Neil Lathwood
056ddbac05 webui: Fixed the display date for the current version (#6474)
* webui: Fixed the display date for the current version

* changed to use exec

* changed >> to >
2017-04-22 18:03:14 -05:00
Tony Murray
72ce39cf9b refactor: schema updates (#6370)
Test the schema files with STRICT_TRANS_TABLES and fix any issues
2017-04-13 10:18:12 +01:00
Neil Lathwood
1bbbaff58b refactor: Moved ifLabel -> cleanPort and updated the usage (#6288)
* refactor: Moved ifLabel -> cleanPort and updated the usage

* added list of tags allowed

* few updates as per comments from @murrant

* Added simple unit tests

* double display for javascript popover

* Remove ifNameDescr() function
Fix realtime port page

* remove accidental temp code

* Remove double display calls in ajax tables
2017-04-04 08:08:23 +01:00
Tony Murray
4b9f3f37d7 fix: move user preferences dashboard and twofactor out of users table (#6286)
* fix: move user preferences dashboard and twofactor out of users table
This allows them to work with any authentication method
Add set_user_pref() and get_user_pref() helper functions

* fix edit users for other users

* Fix updated_at default timestamp

* Update and rename 183.sql to 184.sql

* removed commented out debug
2017-04-01 22:18:00 +01:00
Aldemir Akpinar
2dd0ab870f Do not show unintended debug messages (#6296)
* Do not show unintended debug messages

* Fix the silly mistake
2017-03-29 14:35:44 +01:00
Tony Murray
1ea7af424b feature: bind user for active_directory auth (#6255)
* feature: bind user for active_directory auth
Optional, allows the use of "remember me", API, and alerting.

* missing global (but still may not be working)

* always return a value from reauthenticate()

* Make sure the ldapbind credentials are correct on reauth.
Do not send output if they are incorrect (use d_echo) this breaks ajax calls, etc.
Add scripts/auth_test.php, to make it easier to debug authentication.

* Refine auth_test.php a bit more
A few small cleanups in other places of the auth

* Add auth_test.php to docs
Some more improvements in the auth_test.php output.

* Update Authentication.md
2017-03-29 08:22:02 -05:00
Neil Lathwood
d00a2d6215 fix: Updated set_null() function to be more useful #6030 (#6049)
* fix: Updated set_null() function to be more useful #6030

* drop isset

* Added unit test

* change to assertNull
2017-03-07 22:15:22 -06:00
Neil Lathwood
ed02e39347 webui: Update device overview to not show hostname when certain conditions match #5984 (#6091) 2017-03-04 21:25:06 -06:00
Neil Lathwood
46198b2caa feature: Added ability to show links to fixes for validate (#6054)
* feature: Added ability to show links to fixes for validate

* Finish this PR up and add a few more messages
Move collation checks to print_list()
2017-03-02 08:04:29 +00:00
Richard Mayhew
5bfd23e7eb webui: New eventlog severity classification (#5830)
* New event log severity model

* Revert "New event log severity model"

This reverts commit 0783527af8.

* New event log severity model

* Fixed spaces vs tabs

* Learning the format requirements

* Learning the format requirements

* Moved the colours in to the stylesheet

* change SQL file to stop conflicting

* refactor log_event so severity isn't last, but reference is.

* keep having to move the file due to new PR conflicts

* Revert "keep having to move the file due to new PR conflicts"

This reverts commit f3030e7218.

* extended support to dashboard and eventlog page

* formatting
2017-02-12 22:41:05 +00:00
Søren Rosiak
414efb3539 refactor: Various Code Cleanup (#5777)
* refactor: Various Code Cleanup

* moar
2017-02-05 22:37:08 +00:00
Neil Lathwood
a19b2614dc fix: added some timeout safeguards for curl calls (#5743) 2017-02-03 08:12:42 -06:00
Neil Lathwood
981e974a70 fix: Check lat/lng are numeric rather than !empty #5585 (#5657) 2017-01-31 08:07:27 +00:00
Neil Lathwood
483c40eddd api: Added parent_id to VMs that have a host #3523 (#5621) 2017-01-26 22:38:43 +00:00
Tony Murray
2f1095cb8d fix: html purify init wasn't done always when it was used (#5626) 2017-01-26 19:23:03 +00:00
Tony Murray
59706194bf refactor: move HTMLPurifier init to init.php so we only create one object. (#5601) 2017-01-25 20:56:59 +00:00
Neil Lathwood
599e1c4b25 fix: HTML Purifier would create tmp caches within the vendor folder, moved to users tmp dir #5561 (#5596) 2017-01-25 14:17:03 -06:00
Tony Murray
af0446f37f fix: PHP 7.1 function usages with too few parameters (#5588) 2017-01-25 17:43:45 +00:00
Neil Lathwood
32da1bceea fix: Changing device type now is persistant (#5529) 2017-01-22 09:49:13 +00:00
Richard Mayhew
47397d0083 webui: Updated icons to use Font Awesome (#5468) 2017-01-21 11:24:05 +00:00
Neil Lathwood
01e4ac3d58 newdevice: Added additional sensor support for Huawei VRP #4279 2017-01-19 16:42:57 +00:00
Neil Lathwood
8557f2df41 newdevice: APC Environmental monitoring units #5140 2017-01-18 08:48:33 +00:00
David Bell
7fc2834eb4 fix: Stop flattening config options added in config.php (#5493) 2017-01-17 22:26:35 +00:00