* Dispatcher option to log output
-o --log-output Log output into various files in the log directory
wire up -d option to be passed into scheduled commands
Caution, can fill your disk.
* style fixes
* more silly style fixes (and a typo accidentally added)
* final lint maybe?
* more lint...
* believe it or not, more lint
Previously, if the device was ping only, it wasn't marked as discovered.
Now we always run discovery, but basically all it does is update last_discovered.
and increase default to 20s from 10s
20s should still be fast enough to prevent gaps, but larger installs can take longer than 10s (or even 20s) to do dispatch work.
* Implement ACL support for redis (and sentinel)
Currently, sentinel only works with anonymous connections.
Some parameters are passed when using sentinel, however these are
dropped on the floor.
This encapsulates them as py-redis expects, and passes them correctly.
* Pass username
* Differentiate duplicate error messages
* Actually pass var
* Docs and requirement bump
* Lint
* Consistency
* More lint
* Lint harder
* Doc Updates
* Allow configuration of the SSL/TLS operating mode when connecting to a mysql database
* Support SSL/TLS DB connections in the dispatcher service as well
* Apply black formatting standards to Python files
* Suppress pylint errors as redis module is not installed when linting
* More pylint fixes
* Correct typo in logging output
* Refactor SSL/TLS changes into DBConfig class instead of ServiceConfig
* Define DB config variables as class vars instead of instance vars
* Break circular import
* Dispatcher bugfix queues not being disabled properly
Introduced in #13355
Affected queues: Alerting, Discovery, Services, Ping
Adjust log level of several messages
* better formatting? looks like garbage python black
* Dispatcher: Don't update workers/frequency when updating stats
* Fix the right thing
* Don't set poller groups on the cluster entry, this way config.php can override if it hasn't been set by the webui
* New service/discovery/poller wrapper
* Convert old wrapper scripts to bootstrap loaders for wrapper.py
* Move wrapper.py to LibreNMS module directory
* Reformat files
* File reformatting
* bootstrap files reformatting
* Fusion service and wrapper database connections and get_config_data functions
* Moved subprocess calls to command_runner
* LibreNMS library and __init__ fusion
* Reformat files
* Normalize logging use
* Reformatting code
* Fix missing argument for error log
* Fix refactor typo in DBConfig class
* Add default timeout for config.php data fetching
* distributed discovery should finish with a timestamp instead of an epoch
* Fix docstring inside dict prevents service key to work
* Fix poller insert statement
* Fix service wrapper typo
* Update docstring since we changed function behavior
* Normalize SQL statements
* Convert optparse to argparse
* Revert discovery thread number
* Handle debug logging
* Fix file option typo
* Reformat code
* Add credits to source package
* Rename logs depending on the wrapper type
* Cap max logfile size to 10MB
* Reformat code
* Add exception for Redis < 5.0
* Make sure we always log something from service
* Fix bogus description
* Add an error message on missing config file
* Improve error message when .env file cannot be loaded
* Improve wrapper logging
* Fix cron run may fail when environment path is not set
* Add missing -wrapper suffix for logs
* Conform to prior naming scheme
* Linter fix
* Add inline copy of command_runner
* Another linter fix
* Raise exception after logging
* Updated inline command_runner
* Add command_runner to requirements
* I guess I love linter fixes ;)
* Don't spawn more threads than devices
* Fix typo in log call
* Add exit codes to log on error, add command line to debug log
* Add thread name to error message
* Log errors in end message for easier debugging
* Typo fix
* In love of linting
* Add inline command_runner library
* New service/discovery/poller wrapper
* Convert old wrapper scripts to bootstrap loaders for wrapper.py
* Add command_runner to current requirements
* Move wrapper.py to LibreNMS module directory
* Reformat files
* File reformatting
* bootstrap files reformatting
* Fusion service and wrapper database connections and get_config_data functions
* Moved subprocess calls to command_runner
* LibreNMS library and __init__ fusion
* Reformat files
* Normalize logging use
* Reformatting code
* Fix missing argument for error log
* Fix refactor typo in DBConfig class
* Add default timeout for config.php data fetching
* distributed discovery should finish with a timestamp instead of an epoch
* Fix docstring inside dict prevents service key to work
* Fix poller insert statement
* Fix service wrapper typo
* Update docstring since we changed function behavior
* Normalize SQL statements
* Convert optparse to argparse
* Revert discovery thread number
* Handle debug logging
* Fix file option typo
* Reformat code
* Add credits to source package
* Rename logs depending on the wrapper type
* Cap max logfile size to 10MB
* Reformat code
* Add exception for Redis < 5.0
* Make sure we always log something from service
* Fix bogus description
* Handle more signals
* Flush buffers before exiting process
This ensures log messages aren't lost
* Restart process before jobs have finished
If there is a very log running job it can cause service restart to
take over 5 minutes.
We tweak the order of things to make sure that running processes
continue, but nothing more is scheduled.
The worst case impact is that a pollling/discovery job gets
scheduled twice, but this should not be a big issue - this should
only occur at most once per day.
* Remove python 3.8 feature
* Ensure that processes from the previous invocation are reaped
* Correct typo's
* Attach subprocess descriptors to /dev/null
Occasionally, PHP would throw a fit and crash when its stdout went
away. To avoid this, we attach stdout to devnull.
This means we lost output of daily.sh - but this is already recorded
in $LOGDIR/daily.log
* Don't immediately schedule long running jobs
To avoid the situation where the maintenance reload happens or a sighup,
then a second long running job is immediately started, we wait
(`last_[poll/discovery]_timetaken` * 1.25) seconds before scheduling
any jobs.
* Add `psutil` to requirements
* Add support for "systemctl reload" to the unit files
* Add a fallback for systems that don't have psutil
* Reduce CPU load when psutil is not installed
* Don't avoid double polling by extending the timeout
This shouldn't happen due to locks
* Remove fallback option
* Remove extra variable
* Fix issue introduced during rebase
* Fix issue introduced when fixing issue introduced during rebase
* Make psutil optional
If daily.sh exited with non-zero it would kill the maintenance thread, stopping daily.sh
The maintenance lock was never released, this wouldn't cause an issue in normal operation as it should expire.
* Poller settings WIP
* Poller settings WIP2
* working on SettingMultiple
* setting multiple working
* settings sent with all required info
* fix translation
* Fix keys
* fix groups setting
* Apply settings to service
fixes and validations for setting
* don't error when no poller_cluster entry exists
* hid tab when no poller cluster entries
* Authorization
* make prod
* daily maintenance toggle should be advanced
* Update schema def
service_alerting_enable no will properly disable alerting (may be set globally or per node)
service_alerting_frequency will no properly control frequency (set globally the same, via db is best)
* Refactor LibreNMS service
add ping
* services ported
remote legacy stats collection
* alerting
* implement unique queues
* update discovery queue manager
* remove message
* more cleanup
* Don't shuffle queue
* clean up imports
* don't try to discover ping only devices
* Fix for discovery not running timer
* Update docs a bit and and add some additional config options.
Intentionally undocumented.
* Wait until the device is marked up by the poller before discovering
* Handle loosing connection to db gracefully
* Attempt to release master after 5 db failures
* Sleep to give other nodes a chance to acquire
* Update docs and rename the doc to Dispatcher Service to more accurately reflect its function.
* add local notification
Currently has a file handle leak (and will eventually run out of handles) related to the self update process.
Either need to fix that or rip out self-update and leave that up to cron or something.
DO NOT DELETE THIS TEXT
#### Please note
> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.
- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)
#### Testers
If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
