librenms-librenms/doc/Extensions/Dispatcher-Service.md

source: Extensions/Dispatcher-Service.md
path: blob/master/doc/

# Dispatcher Service

> Status: Release Candidate

The new LibreNMS dispatcher service (`librenms-service.py`) replaces
the old poller service (`poller-service.py`), improving its
reliability. It's mostly a drop in replacement for the old service,
but testing is recommended before switching over.

If you are currently using the old poller service, it's strongly
recommended that you migrate away - it has a serious defect under
certain versions of mysql/mariadb, and may be inadvertently DoS'ing
your devices. The new service does not have this issue,

Make sure you uninstall the old poller service before deploying the new service.

The dispatcher does not replace the php scripts, but the cron entries
running them.  It attempts to do a better job than simple time based
scheduling.

# External Requirements

## A recent version of Python

The LibreNMS service requires Python 3 and some features require
behaviour only found in Python3.4+.

## Python modules

- PyMySQL is recommended as it requires no C compiler to
  install. MySQLclient can also be used, but does require compilation.
- python-dotenv .env loader
- redis-py 3.0+ and Redis 5.0+ (if using distributed polling)

These can be obtained from your OS package manager, or from PyPI with the below commands.

```bash
pip3 install -r requirements.txt
```

## Redis (distributed polling)

If you want to use distributed polling, you'll need a Redis instance
to coordinate the nodes. It's recommended that you do not share the
Redis database with any other system - by default, Redis supports up
to 16 databases (numbered 0-15). You can also use Redis on a single host if you want

It's strongly recommended that you deploy a resilient cluster of redis
systems, and use redis-sentinel.

You should not rely on the password for the security of your
system. See <https://redis.io/topics/security>

## Memcached (distributed polling)

LibreNMS still uses memcached for locking daily update processes when using
distributed polling.  So you will still need to configure memcached
unless you have updates disabled.

## MySQL

You should already have this, but the pollers do need access to the
SQL database. The LibreNMS service runs faster and more aggressively
than the standard poller, so keep an eye on the number of open
connections and other important health metrics.

# Configuration

Connection settings are required in `.env`. The `.env` file is
generated after composer install and APP_KEY and NODE_ID are set.

```dotenv
#APP_KEY=   #Required, generated by composer install
#NODE_ID=   #Required, generated by composer install

DB_HOST=localhost
DB_DATABASE=librenms
DB_USERNAME=librenms
DB_PASSWORD=
```

## Distributed Polling Configuration

Once you have your Redis database set up, configure it in the .env file on each node.

```dotenv
REDIS_HOST=127.0.0.1
REDIS_PORT=6379
# OR
REDIS_SENTINEL=192.0.2.1:26379
REDIS_SENTINEL_SERVICE=myservice

REDIS_DB=0
#REDIS_PASSWORD=
#REDIS_TIMEOUT=60
```

## Basic Configuration

Additional configuration settings can be set in `config.php` or
directly into the database.

The defaults are shown here - it's recommended that you at least tune
the number of workers.

```php
$config['service_poller_workers']              = 24;     # Processes spawned for polling
$config['service_services_workers']            = 8;      # Processes spawned for service polling
$config['service_discovery_workers']           = 16;     # Processes spawned for discovery


//Optional Settings
$config['service_poller_frequency']            = 300;    # Seconds between polling attempts
$config['service_services_frequency']          = 300;    # Seconds between service polling attempts
$config['service_discovery_frequency']         = 21600;  # Seconds between discovery runs
$config['service_billing_frequency']           = 300;    # Seconds between billing calculations
$config['service_billing_calculate_frequency'] = 60;     # Billing interval
$config['service_poller_down_retry']           = 60;     # Seconds between failed polling attempts
$config['service_loglevel']                    = 'INFO'; # Must be one of 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'
$config['service_update_frequency']            = 86400;  # Seconds between LibreNMS update checks
```

There are also some SQL options, but these should be inherited from
your LibreNMS web UI configuration.

Logs are sent to the system logging service (usually `journald` or
`rsyslog`) - see
<https://docs.python.org/3/library/logging.html#logging-levels> for the
options available.

```php
distributed_poller                             = true;  # Set to true to enable distributed polling
distributed_poller_name                        = null;  # Uniquely identifies the poller instance
distributed_poller_group                       = 0;     # Which group to poll
```
### Tuning the number of workers

See https://your_librenms_install/poller 

You want to keep Consumed Worker Seconds comfortably below Maximum Worker Seconds. The closer the values are to each other, the flatter the CPU graph of the poller machine. Meaning that you are utilizing your CPU resources well. As long as Consumed WS stays below Maximum WS and Devices Pending is 0, you should be ok.

If Consumed WS is below Maximum WS and Devices Pending is > 0, your hardware is not up to the task.

Maximum WS equals the number of workers multiplied with the number of seconds in the polling period. (default 300)

# Fast Ping

The [fast ping](Fast-Ping-Check.md) scheduler is disabled by default.
You can enable it by setting the following:

```php
$config['service_ping_enabled'] = true;
```

# Watchdog

The watchdog scheduler is disabled by default. You can enable it by setting the following:

```php
$config['service_watchdog_enabled'] = true;
```

The watchdog scheduler will check that the poller log file has been written to within the last poll period. If there is no change to the log file since, the watchdog will restart the polling service. The poller log file is set by `$config['log_file']` and defaults to `./logs/librenms.log`

# Cron Scripts

Once the LibreNMS service is installed, the cron scripts used by
LibreNMS are no longer required and must be removed.

# Service Installation

A systemd unit file is provided - the sysv and upstart init scripts
could also be used with a little modification.

## systemd

A systemd unit file can be found in `misc/librenms.service`. To
install run `cp /opt/librenms/misc/librenms.service
/etc/systemd/system/librenms.service && systemctl enable --now
librenms.service`

## OS-Specific Instructions

### RHEL/CentOS

To get the LibreNMS service running under python3.4+ on
RHEL-derivatives with minimal fuss, you can use the software
collections build:

First, enable SCL's on your system:

#### CentOS 7

```
# yum install centos-release-scl
```

#### RHEL 7

```
# subscription-manager repos --enable rhel-server-rhscl-7-rpms
```

Then install and configure the runtime and service:

```
# yum install rh-python36 epel-release
# yum install redis
# vi /opt/librenms/config.php
# vi /etc/redis.conf
# systemctl enable --now redis.service
# scl enable rh-python36 bash
# pip install pymysql redis
# cp /opt/librenms/misc/librenms.service.scl /etc/systemd/system/librenms.service
# systemctl enable --now librenms.service
```

If you want to use another version of python 3, change `rh-python36`
in the unit file and the commands above to match the name of the
replacement scl.

### Debian/Ubuntu

#### Debian 9 (stretch)

install python3 and python-mysqldb. python-dotenv is not yet
available, but the testing package is working fine, you can grab it on
<https://packages.debian.org/fr/buster/all/python3-dotenv/download> (the
package may be updated and have a new version number).

```
apt install python3 python-mysqldb
cd /tmp
wget http://ftp.fr.debian.org/debian/pool/main/p/python-dotenv/python3-dotenv_0.9.1-1_all.deb
dpkg -i python3-dotenv_0.9.1-1_all.deb
```