DO NOT DELETE THIS TEXT
#### Please note
> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.
- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)
#### Testers
If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
* Initial start on new templating support
* further updates
* more updates
* More working version
* Last fixes
* Small updates
* modified test to look for dbname.
* Schema update
* Added fix for not being able to disable Philips Hue transport
* Updated sql file
* Updated db_schema
* Set myclabs/deep-copy to be 1.7.x version in composer
* Fixes from murrant
* Forced nikic/php-parser to version 3.1.x in composer
* Updated composer to use custom fork of string-blade-compiler + fixed transport use
* Updated to always use correct template
* Merged legacy and blade templating engines
* Removed template type + fixed test-alert.php
* Added more template placeholders
* Added ability to reference data in templates as $alert->X
* Updated docs for templates
* Updated db_schema.yaml
* Added $alert->builder
* Clarify the use of $alert->builder
* Fixed the use of $alert->transport
* renamed schema file
* Added template validation
* Small update to fix travis issue
* Add Docs text to title bar
* Updated amqp to new of version
* Consistency in alert rule and template title bars
Use Log facility when Laravel is booted.
Update init.php so we can easily boot Laravel for CLI scripts. (and just Eloquent, but that may go away)
Move all debug setup into set_debug() function and use that across all scripts.
Log Laravel database queries.
Send debug output to librenms log file when enabling debug in the webui.
Allow for colorized Log CLI output. (currently will leave % tags in log file output)
** Needs testing and perhaps tweaking still.
DO NOT DELETE THIS TEXT
#### Please note
> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.
- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)
#### Testers
If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
* corrected display of minigraph when using sysName as hostname
* Check to see if its an IP or hostname. Make sure all 3 scenarios work.
* removed test rrd symlink
* removed test rrd symlink
* reverted old change
* Improve and add tests.
* 'lat' and 'lng' columns added on list_devices()
* 'lat' and 'lng' columns added on device fetch
Helped by https://github.com/laf
* Commit user.name changed
* feature: Added new alert rule builder UI
* Updated to export sql queries
* More updates
* more changes
* removed debug
* fix scrut
* Updated to include import options + various other fixes
* fix rule
* Populate name from collection rules.
* Fix default rule import
Allow new and old style rules in the collection.
Don't add new yet as I'm not sure GenSQL() is working.
* Fix GenSQL call
* Extract filter building to class so it is nicely contained in one place
* moved schema
* some fixes and tweaks
* travis fixes
* Some more features / updates
* Fix up my mistakes when adding default rules
* Use a modal for new alert (Incomplete)
Larger dialog!!
Remove page loading stuff.
Working:
Loading rules, resetting dialog, importing from collection.
Not working yet:
select width
device limited rule access? don't know what this is...
Lots of unused stuff to delete...
* reload "table" after save
* fixed editing rule
* Auto select2 width
* Reload window on save
* Restore per-device alert. Remove debug.
* Small cleanups. Rule Name first.
* Restore button to button type. Rename schema.
* Fixes: wrong command to reload window, remove extra attributes, rule is never passed
* Fixed old rule editing
* some small updates for old imports
* travis update to use trusty
* maybe travis fix
* Ability to set alert rule mappings on the rule edit screen
* pip installs one line, no quiet for deploy
* update schema def
* Fix style and some copyright headers
* fix docs missing file
* Allow new versions of snmpsim and libraries
* Parser WIP
* Fix default rules insert
* reorganize
* Legacy import first draft done
* Implement saving
Skip translation to sql for now
* Working on glues
* small rule collection fix
* Working on glues
* Working on glues
* Docs updates + small UI changes
* Parser WIP
* reorganize
* Legacy import first draft done
* Implement saving
Skip translation to sql for now
* Working on glues
* Working on glues
* Working on glues
* Add table mapping, should move to it's own class
* WIP
* Glue working!!
* Extract Schema class
* Some final touches.
revert alerts_rules.json for now.
* Finish up initial implementation
Needs more tests
* Fix a few places
* small doc updates
* Fix finding tables in grouped rules.
* remove unused code
* code format fixes
* Some quick tests for Schema
Simplified output for findRelationshipPath. Always includes start and target in the result.
This simplifies a lot of code in QueryBuilderParser.php
This also always loads the target table data now (which we want)
* Make bill_id the PRIMARY index for the bills table
* Load macros from a json file in misc instead of the database.
* Fix whitespace and wrong key for collection.
* Handle IN properly when generating SQL
* Fix glue (devices.device_id = ports.port_id) is incorrect :D
Show ALL tables we can resolve relationships for in the query builder filter.
* Remove all macros from the database
Remove insert statements, leave updates to update user's existing rules.
* tests: Added testing fro Arista EOS BGP
* fixing tests
* small updates
* last one
* more updates
* Account for no space after = in snmp data
always load_os() when a new device is freshly loaded by device_by_id_cache()
* don't be fatal in load_os()
* Update json to correct data
* Processor Tests!
* Capture data from live devices easily.
* fix up some stuff, remove powerconnect things as they seem to be just broken.
* formatting, fix missing assignment
add netonix processor data
* fix multi-line, always add sysDescr and sysObjectID
ios cpm test file
* revert composer change and fix whitespace issues
* add help text
* missed help text
* tighter debug output
* handle empty strings properly and mibs with numbers
* use keys for sorting as intended
* fix type with empty data
* oops :)
* whitespace fix
* try installing fping
* Fix TestCase collision + cleanup
* mark TestCase as abstract
don't run two instances of snmpsim
* better database dumps, improved capture
* style fixes
* fix quotes add a few more tables
* add --prefer-new, properly merge data
* Support separate discovery and poller data. But don't waste space if they aren't needed.
* refactor to use class
collects all code in one place for reusability
* reorganize
* Print out when not saving.
* Support for running multiple (or all) modules at once.
* tidy
* Change unit test to a generic name and test all modules we have data for.
* Add documentation and a few more tidies
* whitespace fixes
* Fix tests, add a couple more modules, more docs
* More docs updates
* Fix scrutinizer issues
* add bgp-peers
* Junos dwdm interface sensor support …
four new sensors
Chromatic Dispersion ps/nm
Delay s (seconds)
Q_factor (dB)
preFEC_BER (ratio)
reuse of
SNR, dbm, temperature, current
* new dwdm sensors more generic, ber and quality_factor
* new sensors updated
* renamed html graphs new sensors
* fixed ber polling
* Change to base SI Hz and s.
* update sensor divisors to bigint
* rename sql-schema/216.sql to 218.sql
* YAML junos dwmd sensor definitions, except BER
* bigint divisor fixed
* deafult 1 divisior
* small fixes including schema change for another PR
* schema fix
* feature: Generic alerts for state sensors
A bit of a hack, but gets the job done. Perhaps someone can refactor the ResolveGlues() function sometime to make it better.
* Fix mistakes
* Add docs and collection rules, critical enabled by default
* moved schema file to 221.sql
* feature: save application metrics to db for alerting
However, alerting will not work because ResolveGlues() is broken.
Can add workaround after state_translations alerting is merged
Does not update all applications yet, not sure if that should be done here or in another PR.
Introduces two handy functions dbDeleteOrphans() and array_by_column(). Will replace those in other locations after this is merged or separate them out if this is not merged.
* remove accidental inclusions
* Add db schema
* fix: validation check cli php extensions
Fix fatal errors when mysql is not loaded / can't connect
* fix: discovery validation incorrectly detects on ping only devices.
* fix: devices detected as ibmtl
Guess at the sysObjectID, please report if you have devices that should be ibmtl and are no longer detected.
Add output of os in discovery.
* Always load all os for discovery. Should be cached most of the time.
* Revert "Revert "feature: Allow snmpget in os discovery yaml (#7558)" (#7576)"
This reverts commit 426916dbe4.
* Revert "fix: lldp find_port_id finds not the most correct port (#7564)"
This reverts commit 7e05fff615.
* Revert "fix: Update RFC1123 Hostname Check (#7572)"
This reverts commit ebcfa45a93.
* Update RFC1123 Hostname Check
The regex was wrong. It failed to match devices like `cisco-3750x` even though it is a valid hostname.
I picked this regex from the RFC1123 documentation, and updated it.
* Update CommonFunctionsTest.php
Update tests
* Updated to allow domains ending with a dot
* Added support for ping only devices.
* Renamed sql-schema/206 to 207.
* Discovery tried to detect the OS when it shouldn't.
* Updated db_schema.yaml
* Added ping icon.
* Don't show unused graphs and tabs when SNMP is disabled.
* Allow the user to specify OS of ping only devices.
* Removing custom OS now changes it to 'ping'.
* Removed unnecessary use of mres().
* UI select box for SNMP enable/disable.
* Hide device_ping_perf if ping is disable on device.
* Fixed SNMP settings update status messages.
* Added functionality to add ping only devices via the web ui.
* Added ping only option to addhost.php
* Added ping only support to snmp-scan.py
* Moved sql 208 to 211
* Fixed scrutinizer issues.
* Fixed broken ossearch.
* Added ping only support to the API.
* Updated API doc.
* Added (optional) to OS and hardware description. Hid Port Settings, Applications, Modules, Storage, Processors, Memory and Components from the edit menu
* Style fix.
* Updated ping icon.
* clean() instead of mres(). More escaping. Better help in snmp-scan.py and addhost.php
* Fixed scrutinizer issue.
* Always try SNMP in snmp-scan.py, new option for it in addhost.php. Slice instead of chunk in ajax_ossuggest.php. Other minor style changes.
* Updated sql modifications to insert the new column in the same place as in db_schema.yaml.
* fix: fix a couple of OS cache invalidation issues
Fixes issues where we might be using outdated os definitions.
* use Config and fix style issue
* only pull in settings from config.php (otherwise will possibly contain populated os config)
* refactor: convert validations to modules to prep for gui integration
* accidentally dropped, schema update
* fix accidental output to webui
* mention discovery-wrapper.py and re-arrange.
* refine some fix statements
* rename the Config validation group to Configuration.
* fix some scrutinizer issues
remove as many local functions from validator.php as possible
move extensions from pre-check
remove duplicate timezone check
looks like there is some db schema differences between mariadb 10.1 and 10.2, investigating
* mariadb version diff take2
* Check schema version first for database.
Remove stop to go back to command line for install docs.
Add helpful link when there is no devices added to /addhost
* fix incorrect validation for empty string defaults
* Fix style
* Add additional file permissions checks
* catch exception and fail for invalid timezone
Change visuals a bit.
* add php version warning
* fix space
* fix: improve accuracy of is_valid_hostname()
fixes discovery code attempting to add discover invalid dns names
the old code allowed some invalid hostnames, this is more thorough
add tests
* fix merge error
* refactor: discovery protocols re-write
extract more code to make it easier to read
use snmpwalk_group
walk lldpRemTable instead of lldpRemoteSystemsData
Optimize discover_device to reduce dns lookups of bad dns names
Make discovery_by_ip behavior consistent between cdp and lldp
move can_skip_discovery to discovery functions
Tested lldp, cdp, and ospf.
* always discover links even if the remote device isn't in system
Fix discover_link always updating
* fix style
* fix update, not fetching id
* increase platform length. Most devices use sysDescr here, which is max length 255
* remove orphaned links
* update db_schema.yaml
* fix: Fixed the high/low thresholds for Arista devices
* final updates
* push threshold changes
* more updates
* another update
* updated entity-sensor for current value calc
* hoping final update
miss-named variable in sensor pre-caching
internal humidity and temperature was discovered twice
humidity was mis-spelled in yaml discovery
temperature and current had incorrect divisor in yaml
remove redundant fahrenheit to celsius function
fixes#7236
* Refactor remove use of pear Net_IPv4 and Net_IPv6
Caution, I added tests to test the functionality of the new functions, but I'm not 100% certain the replaced usages match 100%. Please test.
Left usage in generate-iplist.php because I'm not sure the use of that script, so I didn't want to implement additional functionality to support it.
Add support for ipv6 fo ipInNetwork() function
Add phpdocs to functions
Remove accidental inclusion
Use binary operations instead of string for networKFromIp()
Tidy up binary operations in ipInNetwork()
tidy parsing cidr network
Change to a class based implementation.
Update phpdocs.
Fully catch exceptions for hex parsing.
Fix mistake.
support older phpunit versions
Fix php shifting to negative with php 5
* Fix graph authorization. Uncaught exception and checking an ipv4 against an ipv6 network == fail.
* Also remove colons from hex strings.
* Fix typo in bgp polling
Two passes of removing invalid characters from hex strings. (That way we don't mangle valid IPs)
Always output header (may be missing mysql data)
Improve version display, now looks like '1.28-129-g74e6c3edf' <tag>-<commits since tag>-<shortag>
Add mysqli to the list of required extensions, check extensions before init
Add dbIsConnected() function, used to check if we have db in version_info()
Do not die on IPv4 and IPv6 so validate can print errors.
Move git checks so they are output together
fix rrdtool 1.7.0 reports version as 1.7.01.7.0
* fix: page/graph load speed: part 1
Speed up load_all_os() by caching the full $config['os'] array
Function time goes from 1.13648s -> 0.00073s
The cache is updated during discovery, but only if it is >22hrs old.
Additionally, the cache is not used and a full yaml load is forced if the cache is >24hrs old.
The cache time is controlled by $config['os_def_cache_time'], I don't see any reason anyone would change that time.
* Delete Session.php
* Include the cache directory.
* refactor: OS discovery tests are now dynamic
All that needs to be done now is supply snmprec files with the proper name. os.snmprec or os_description.snmprec
I also tweaked the loading of yaml files to speed up the tests substantially.
Verifies all os have at least one test and all snmprec files are tested.
* fix typo
* feature: Wireless Sensors
Includes client counts for ios and unifi
Graphing could use some improvement.
Alerting and threshold ui not implemented
WIP: starting OO based wireless sensors.
Class based functionality working
remove old functional files
add schema file
discovery needs to be enabled, not polling
fix up schema
fix Unifi discovery not returning an array
Add some debug when discovering a sensor.
Fix style.
Add missing semicolin
Add a null object (Generic) for OS.
Fill out some phpdocs
Re-organized code
Each sensor type now has it's own discovery and polling interface
Custom polling tested with Unifi CCQ
Left to do:
Implement UI (Graphs and Custom thresholds)
Alerting
Testing
Fix event message text
Remove runDiscovery and runPolling from OS, they are unused and don't belong there.
Cleanups/docs
Missed this file.
Remove the requirement to fetch the current value to check validity.
Do that automatically if current is not specified
A few cleanups here and there
First pass at graphing.
device_ and wireless_ graphs added.
Add RouterOS support
Singleton OS instance isn't required right now.
Remove that to allow some memory to be freed.
Add wireless to the device list metrics.
Make all metrics clickable
Tweak graphs a bit
Implement limit configuration page.
Use sensors page as common code instead of duplicating.
Clean up some javascript interactions: Allow enter on values to save. Cancel if update is not needed. Enable the clear custom button after setting a custom value.
Add some wireless alert rules to the library.
Add documentation.
Add unifi client counts by ssid in addition to radio.
Optimize Sensor polling a bit.
Add HP MSM clients support (for full controller)
Fix function accessibility
Formalize the discovery and poller interfaces.
Add Xirrus clients and noise floor
move module interfaces to a more appropriate place.
push caching code up to os, unsure about this do to the limitations
No point in selectively enabling wireless discovery. We only discover if the device supports something.
Add RSSI, Power, and Rate.
Add these sensors for Ubnt Airos.
Clean up some copyrights.
Reduce the amount of files need to add new types.
Leave graph files for consistency and to allow customization.
Remove the old wifi clients graph completely.
ciscowlc should have improved counts (total and per-ssid)
Schema didn't get added.
Impelement the rest of the AirOS sensors
Reformat and re-organize the Airos.php class.
Add several UBNT AirFiber sensors
A few fixes add links to the section headers
Add HP MSM mibs.
* Schema file got dropped in rebase.
* Add wireless menu to view sensors across all devices.
Icons in the menu need help :/
* Add HeliOS, Mimosa, and Siklu support
Sensors added SNR + Noise
* Add power and utilization to Unifi
* Update polling to prefetch all sensor data in a few snmp requests as possible
* Add Extendair: tx+rx power, aggregate rate, frequency
* Add a check for duplicate sensors in discovery. Just print an error for now.
* Add Bit Error Ratio (named error-ratio to allow for bit error rate to be added if needed)
Fix an incorrect link in the wireless sensors table
* Add error rate and change all bps and Hz to use si units
* Fixes to limits and frequency display
* Fix overview graph frequency display
A few decimal place tweaks
* Don't allow switching sensor and wireless-sensor graphs, it doesn't work.
Change individual distance graphs to use si units
* Go through the OS and make sure I got all the sensors I can (probably missed some still)
Because pollWirelessChannelAsFrequency() is generic and a little complex, so pull it up to OS.
Message to help developers adding supports that don't return an array from discover functions.
* Fix some issues
* Remove noise and signal for now at least
A couple more fixes
Add a notification
* Oopsie
* Bonus AirFiber sensors
* refactor: Moved ifLabel -> cleanPort and updated the usage
* added list of tags allowed
* few updates as per comments from @murrant
* Added simple unit tests
* double display for javascript popover
* Remove ifNameDescr() function
Fix realtime port page
* remove accidental temp code
* Remove double display calls in ajax tables
