* Implement ACL support for redis (and sentinel)
Currently, sentinel only works with anonymous connections.
Some parameters are passed when using sentinel, however these are
dropped on the floor.
This encapsulates them as py-redis expects, and passes them correctly.
* Pass username
* Differentiate duplicate error messages
* Actually pass var
* Docs and requirement bump
* Lint
* Consistency
* More lint
* Lint harder
* Doc Updates
* Allow configuration of the SSL/TLS operating mode when connecting to a mysql database
* Support SSL/TLS DB connections in the dispatcher service as well
* Apply black formatting standards to Python files
* Suppress pylint errors as redis module is not installed when linting
* More pylint fixes
* Correct typo in logging output
* Refactor SSL/TLS changes into DBConfig class instead of ServiceConfig
* Define DB config variables as class vars instead of instance vars
* Break circular import
* Dispatcher bugfix queues not being disabled properly
Introduced in #13355
Affected queues: Alerting, Discovery, Services, Ping
Adjust log level of several messages
* better formatting? looks like garbage python black
* Fix python config fetch disrupted by stderr output
I have no idea how to make command_runner not include debug output, so just use subprocess.check_output
* restore import
* New service/discovery/poller wrapper
* Convert old wrapper scripts to bootstrap loaders for wrapper.py
* Move wrapper.py to LibreNMS module directory
* Reformat files
* File reformatting
* bootstrap files reformatting
* Fusion service and wrapper database connections and get_config_data functions
* Moved subprocess calls to command_runner
* LibreNMS library and __init__ fusion
* Reformat files
* Normalize logging use
* Reformatting code
* Fix missing argument for error log
* Fix refactor typo in DBConfig class
* Add default timeout for config.php data fetching
* distributed discovery should finish with a timestamp instead of an epoch
* Fix docstring inside dict prevents service key to work
* Fix poller insert statement
* Fix service wrapper typo
* Update docstring since we changed function behavior
* Normalize SQL statements
* Convert optparse to argparse
* Revert discovery thread number
* Handle debug logging
* Fix file option typo
* Reformat code
* Add credits to source package
* Rename logs depending on the wrapper type
* Cap max logfile size to 10MB
* Reformat code
* Add exception for Redis < 5.0
* Make sure we always log something from service
* Fix bogus description
* Add an error message on missing config file
* Improve error message when .env file cannot be loaded
* Improve wrapper logging
* Fix cron run may fail when environment path is not set
* Add missing -wrapper suffix for logs
* Conform to prior naming scheme
* Linter fix
* Add inline copy of command_runner
* Another linter fix
* Raise exception after logging
* Updated inline command_runner
* Add command_runner to requirements
* I guess I love linter fixes ;)
* Don't spawn more threads than devices
* Fix typo in log call
* Add exit codes to log on error, add command line to debug log
* Add thread name to error message
* Log errors in end message for easier debugging
* Typo fix
* In love of linting
* Add inline command_runner library
* New service/discovery/poller wrapper
* Convert old wrapper scripts to bootstrap loaders for wrapper.py
* Add command_runner to current requirements
* Move wrapper.py to LibreNMS module directory
* Reformat files
* File reformatting
* bootstrap files reformatting
* Fusion service and wrapper database connections and get_config_data functions
* Moved subprocess calls to command_runner
* LibreNMS library and __init__ fusion
* Reformat files
* Normalize logging use
* Reformatting code
* Fix missing argument for error log
* Fix refactor typo in DBConfig class
* Add default timeout for config.php data fetching
* distributed discovery should finish with a timestamp instead of an epoch
* Fix docstring inside dict prevents service key to work
* Fix poller insert statement
* Fix service wrapper typo
* Update docstring since we changed function behavior
* Normalize SQL statements
* Convert optparse to argparse
* Revert discovery thread number
* Handle debug logging
* Fix file option typo
* Reformat code
* Add credits to source package
* Rename logs depending on the wrapper type
* Cap max logfile size to 10MB
* Reformat code
* Add exception for Redis < 5.0
* Make sure we always log something from service
* Fix bogus description
* Handle more signals
* Flush buffers before exiting process
This ensures log messages aren't lost
* Restart process before jobs have finished
If there is a very log running job it can cause service restart to
take over 5 minutes.
We tweak the order of things to make sure that running processes
continue, but nothing more is scheduled.
The worst case impact is that a pollling/discovery job gets
scheduled twice, but this should not be a big issue - this should
only occur at most once per day.
* Remove python 3.8 feature
* Ensure that processes from the previous invocation are reaped
* Correct typo's
* Attach subprocess descriptors to /dev/null
Occasionally, PHP would throw a fit and crash when its stdout went
away. To avoid this, we attach stdout to devnull.
This means we lost output of daily.sh - but this is already recorded
in $LOGDIR/daily.log
* Don't immediately schedule long running jobs
To avoid the situation where the maintenance reload happens or a sighup,
then a second long running job is immediately started, we wait
(`last_[poll/discovery]_timetaken` * 1.25) seconds before scheduling
any jobs.
* Add `psutil` to requirements
* Add support for "systemctl reload" to the unit files
* Add a fallback for systems that don't have psutil
* Reduce CPU load when psutil is not installed
* Don't avoid double polling by extending the timeout
This shouldn't happen due to locks
* Remove fallback option
* Remove extra variable
* Fix issue introduced during rebase
* Fix issue introduced when fixing issue introduced during rebase
* Make psutil optional
* Refactor pre-commit to class
* docs build
* dusk check
* ci mode for checks
* full checks
* other mysql
* make other lint checks actually work
fix pylint finding
* ci is a long opt
* fix undefined index
* dusk fully working
* ask for forgiveness, not permission
* fix whitespace
* skip dusk sometimes
* Handle 3com and other os with digits
* flags instead of if else spaghetti
* convert to command
* cleanup
* missed check
* fixes
* case
* self-check :D
* argument now
* fix bugs from refactors
* another fix
* adjust file change parsing
* refactor execut a bit
* fallback to global quiet when unknown type.
* allow quiet override for specific commands
* output cleanup
* check flow
* start of tests
* file categorizer tests and fixes
* fixes and cleanup
* skipable not implemented...
* more tests, fix bugs
* more tests and cleanup
* wrong command
* fix canCheck and set env properly
* full env fix
* don't allow dusk on user's run as it will erase their db.
* fix os option
* fix whitespace
* don't need to start server
* ci doesn't like that
* Refactor LibreNMS service
add ping
* services ported
remote legacy stats collection
* alerting
* implement unique queues
* update discovery queue manager
* remove message
* more cleanup
* Don't shuffle queue
* clean up imports
* don't try to discover ping only devices
* Fix for discovery not running timer
* Update docs a bit and and add some additional config options.
Intentionally undocumented.
* Wait until the device is marked up by the poller before discovering
* Handle loosing connection to db gracefully
* Attempt to release master after 5 db failures
* Sleep to give other nodes a chance to acquire
* Update docs and rename the doc to Dispatcher Service to more accurately reflect its function.
* add local notification
Currently has a file handle leak (and will eventually run out of handles) related to the self update process.
Either need to fix that or rip out self-update and leave that up to cron or something.
DO NOT DELETE THIS TEXT
#### Please note
> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.
- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)
#### Testers
If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
