Commit Graph

17 Commits

Author SHA1 Message Date
Tony Murray
e0444bffcf Deprecate poller.php (#15370)
* Deprecate poller.php

* Apply fixes from StyleCI

* Remove references to poller.php
Leave wrapper...

* fix python style

* update wraper to call lnms device:poll

* Quiet output too

* make pb happy

---------

Co-authored-by: StyleCI Bot <bot@styleci.io>
2023-10-04 16:32:02 -05:00
Tony Murray
723600751c Dispatcher option to log output (#15230)
* Dispatcher option to log output
-o --log-output Log output into various files in the log directory
wire up -d option to be passed into scheduled commands
Caution, can fill your disk.

* style fixes

* more silly style fixes (and a typo accidentally added)

* final lint maybe?

* more lint...

* believe it or not, more lint
2023-08-21 18:28:07 -05:00
AdamB
55b167562e Implement authentication for Redis/Sentinel (#14805)
* Implement ACL support for redis (and sentinel)

Currently, sentinel only works with anonymous connections.
Some parameters are passed when using sentinel, however these are
dropped on the floor.
This encapsulates them as py-redis expects, and passes them correctly.

* Pass username

* Differentiate duplicate error messages

* Actually pass var

* Docs and requirement bump

* Lint

* Consistency

* More lint

* Lint harder

* Doc Updates
2023-04-14 07:11:44 -05:00
Tony Murray
6e5ec4a0b8 Dispatcher: services and ping not polling default groups (#13403) 2021-10-21 10:12:36 -05:00
Tony Murray
6b8b9fdb31 Fix Dispatcher service not discovering poller groups (#13377)
The discovery workers were not monitoring non-default poller group queues
2021-10-18 17:30:30 -05:00
Tony Murray
eb653619a3 Dispatcher bugfix queues not being disabled properly (#13364)
* Dispatcher bugfix queues not being disabled properly
Introduced in #13355
Affected queues: Alerting, Discovery, Services, Ping

Adjust log level of several messages

* better formatting? looks like garbage python black
2021-10-14 23:39:08 -05:00
Tony Murray
436487f5f2 Dispatch Service: always start queue managers (#13355)
* dispatch service always start queue managers
Only start workers if they are enabled for this node

* style

* please stop mr ide, sir
2021-10-13 21:49:43 -05:00
Orsiris de Jong
bfa200f3f7 Full Python code fusion / refactor and hardening 2nd edition (#13188)
* New service/discovery/poller wrapper

* Convert old wrapper scripts to bootstrap loaders for wrapper.py

* Move wrapper.py to LibreNMS module directory

* Reformat files

* File reformatting

* bootstrap files reformatting

* Fusion service and wrapper database connections and get_config_data functions

* Moved subprocess calls to command_runner

* LibreNMS library and __init__ fusion

* Reformat files

* Normalize logging use

* Reformatting code

* Fix missing argument for error log

* Fix refactor typo in DBConfig class

* Add default timeout for config.php data fetching

* distributed discovery should finish with a timestamp instead of an epoch

* Fix docstring inside dict prevents service key to work

* Fix poller insert statement

* Fix service wrapper typo

* Update docstring since we changed function behavior

* Normalize SQL statements

* Convert optparse to argparse

* Revert discovery thread number

* Handle debug logging

* Fix file option typo

* Reformat code

* Add credits to source package

* Rename logs depending on the wrapper type

* Cap max logfile size to 10MB

* Reformat code

* Add exception for Redis < 5.0

* Make sure we always log something from service

* Fix bogus description

* Add an error message on missing config file

* Improve error message when .env file cannot be loaded

* Improve wrapper logging

* Fix cron run may fail when environment path is not set

* Add missing -wrapper suffix for logs

* Conform to prior naming scheme

* Linter fix

* Add inline copy of command_runner

* Another linter fix

* Raise exception after logging

* Updated inline command_runner

* Add command_runner to requirements

* I guess I love linter fixes ;)

* Don't spawn more threads than devices

* Fix typo in log call

* Add exit codes to log on error, add command line to debug log

* Add thread name to error message

* Log errors in end message for easier debugging

* Typo fix

* In love of linting
2021-09-27 14:24:25 -05:00
Tony Murray
31246c6ba6 Revert "Full Python code fusion / refactor and hardening (#13094)" (#13123)
This reverts commit 9c534a1a90.
2021-08-10 15:13:05 -05:00
Orsiris de Jong
9c534a1a90 Full Python code fusion / refactor and hardening (#13094)
* Add inline command_runner library

* New service/discovery/poller wrapper

* Convert old wrapper scripts to bootstrap loaders for wrapper.py

* Add command_runner to current requirements

* Move wrapper.py to LibreNMS module directory

* Reformat files

* File reformatting

* bootstrap files reformatting

* Fusion service and wrapper database connections and get_config_data functions

* Moved subprocess calls to command_runner

* LibreNMS library and __init__ fusion

* Reformat files

* Normalize logging use

* Reformatting code

* Fix missing argument for error log

* Fix refactor typo in DBConfig class

* Add default timeout for config.php data fetching

* distributed discovery should finish with a timestamp instead of an epoch

* Fix docstring inside dict prevents service key to work

* Fix poller insert statement

* Fix service wrapper typo

* Update docstring since we changed function behavior

* Normalize SQL statements

* Convert optparse to argparse

* Revert discovery thread number

* Handle debug logging

* Fix file option typo

* Reformat code

* Add credits to source package

* Rename logs depending on the wrapper type

* Cap max logfile size to 10MB

* Reformat code

* Add exception for Redis < 5.0

* Make sure we always log something from service

* Fix bogus description
2021-08-09 18:49:29 -05:00
Jellyfrog
9946fe8b15 Format python code with Black (#12663) 2021-03-28 11:02:33 -05:00
Hayden
cdb6a74dc8 implement watchdog to librenms-service (#11353)
* add watchdog to librenms-service to check log file
add Redis timeout to librenms-service

* updated docs

* fixed logfile_watchdog() indentation in service.py

* indentation fix

* code climate patch

* updated default redis timeout if alerting frequency is 0
2020-03-31 23:10:45 -05:00
bewing
74724a4618 Add redis sentinel support to dispatcher service (#10598)
* Add redis sentinel support to dispatcher service

* Update docs for redis sentinel support

* Don't re-raise python exception in service
2019-10-01 06:51:07 +00:00
Tony Murray
15d72c23b0 Dispatcher Service: Remove duplicate polling complete message (#10290) 2019-05-31 20:04:29 -05:00
Tony Murray
aff2ac49e7 Dispatcher Service: ping and services did not respect assigned poller groups (#10259)
* Dispatcher Service: alerting and ping did not respect assigned poller groups

* Fix issue (missing comma)
add log entry for mysql errors
2019-05-25 19:08:04 -05:00
Tony Murray
604a200891 Python dispatcher service v2 (#10050)
* Refactor LibreNMS service
add ping

* services ported
remote legacy stats collection

* alerting

* implement unique queues

* update discovery queue manager

* remove message

* more cleanup

* Don't shuffle queue

* clean up imports

* don't try to discover ping only devices

* Fix for discovery not running timer

* Update docs a bit and and add some additional config options.
Intentionally undocumented.

* Wait until the device is marked up by the poller before discovering

* Handle loosing connection to db gracefully

* Attempt to release master after 5 db failures

* Sleep to give other nodes a chance to acquire

* Update docs and rename the doc to Dispatcher Service to more accurately reflect its function.

* add local notification
2019-05-20 11:35:47 -05:00
Tony Murray
0ba76e6d62 New python service for poller, discovery + more (#8455)
Currently has a file handle leak (and will eventually run out of handles) related to the self update process.

Either need to fix that or rip out self-update and leave that up to cron or something.


DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-06-30 12:19:49 +01:00