librenms-librenms/doc/Alerting.md

Table of Content:
- [About](#about)
- [Rules](#rules)
 - [Syntax](#rules-syntax)
 - [Examples](#rules-examples)
- [Templates](#templates)
 - [Syntax](#templates-syntax)
 - [Examples](#templates-examples)
- [Transports](#transports)
 - [E-Mail](#transports-email)
 - [API](#transports-api)
 - [Nagios-Compatible](#transports-nagios)
 - [IRC](#transports-irc)

# <a name="about">About</a>

LibreNMS includes a highly customizable alerting system.  
The system requires a set of user-defined rules to evaluate the situation of each device, port, service or any other entity.

This document only covers the usage of it. See the [DEVELOPMENT.md](https://github.com/f0o/glowing-tyrion/blob/master/DEVELOPMENT.md) for code-documentation.

# <a name="rules">Rules</a>

Rules are defined using a logical language.  
The GUI provides a simple way of creating basic as well as complex Rules in a self-describing manner.  
More complex rules can be written manually.

## <a name="rules-syntax">Syntax</a>

Rules must consist of at least 3 elements: An __Entity__, a __Condition__ and a __Value__.  
Rules can contain braces and __Glues__.  
__Entities__ are provided as `%`-Noted pair of Table and Field. For Example: `%ports.ifOperStatus`.  
__Conditions__ can be any of:
- Equals `=`
- Not Equals `!=`
- Matches `~`
- Not Matches `!~`
- Greater `>`
- Greater or Equal `>=`
- Smaller `<`
- Smaller or Equal `<=`

__Values__ can be Entities or any single-quoted data.  
__Glues__ can be either `&&` for `AND` or `||` for `OR`.

__Note__: The difference between `Equals` and `Matches` (and it's negation) is that `Equals` does a strict comparison and `Matches` allows the usage of the placeholder `@`. The placeholder `@` is comparable with `.*` in RegExp.  
Arithmetics are allowed as well.

## <a name="rules-examples">Examples</a>

Alert when:
- Device goes down: `%devices.status != '1'`
- Any port changes: `%ports.ifOperStatus != 'up'`
- Root-directory gets too full: `%storage.storage_descr = '/' && %storage.storage_perc >= '75'`
- Any storage gets fuller than the 'warning': `%storage.storage_perc >= %storage_perc_warn`

# <a name="templates">Templates</a>

Templates can be assigned to a single or a group of rules.  
They can contain any kind of text.  
The template-parser understands `if` and `foreach` controls and replaces certain placeholders with information gathered about the alert.  

## <a name="templates-syntax">Syntax</a>

Controls:
- if-else (Else can be omitted):  
`{if %placeholder == 'value'}Some Text{else}Other Text{/if}`
- foreach-loop:  
`{foreach %placeholder}Key: %key<br/>Value: %value{/foreach}`

__Limitations__: Currently it is not possible to have nested `if` controls, so `if` inside an `if`. It is also not possible to have an `if` inside a `foreach` control. These limitations are going to be resolved in (near?) future.

Placeholders:
- Hostname of the Device: `%hostname`
- Title for the Alert: `%title`
- Time Elapsed, Only available on recovery (`%state == 0`): `%elapsed`
- Alert-ID: `%id`
- Unique-ID: `%uid`
- Faults, Only available on alert (`%state == 1`), must be iterated in a foreach: `%faults`
- State: `%state`
- Severity: `%severity`
- Rule: `%rule`
- Timestamp: `%timestamp`
- Contacts, must be iterated in a foreach, `%key` holds email and `%value` holds name: `%contacts`

## <a name="templates-examples">Examples</a>

Default Template:  
```text
%title\r\n
Severity: %severity\r\n
{if %state == 0}Time elapsed: %elapsed\r\n{/if}
Timestamp: %timestamp\r\n
Unique-ID: %uid\r\n
Rule: %rule\r\n
{if %faults}Faults:\r\n
{foreach %faults}  #%key: %value\r\n{/foreach}{/if}
Alert sent to: {foreach %contacts}%value <%key> {/foreach}
```

# <a name="transports">Transports</a>

Transports are located within `$config['install_dir']/includes/alerts/transports.*.php` and defined as well as configured via `$config['alert']['transports']['Example'] = 'Some Options'`.  

Contacts will be gathered automatically and passed to the configured transports.  
The contacts will always include the `SysContact` defined in the Device's SNMP configuration and also every LibreNMS-User that has at least `read`-permissions on the entity that is to be alerted.  
At the moment LibreNMS only supports Port or Device permissions.  
To include users that have `Global-Read` or `Administrator` permissions it is required to add these additions to the `config.php` respectively:
```php
$config['alert']['globals'] = true; //Include Global-Read into alert-contacts
$config['alert']['admins']  = true; //Include Adminstrators into alert-contacts
```

## <a name="transports-email">E-Mail</a>

E-Mail transport is enabled with adding the following to your `config.php`:  
```php
$config['alert']['transports']['mail'] = true;
```

The E-Mail transports uses the same email-configuration like the rest of LibreNMS.  
As a small reminder, here is it's configuration directives including defaults:
```php
$config['email_backend']              = 'mail';               // Mail backend. Allowed: "mail" (PHP's built-in), "sendmail", "smtp".
$config['email_from']                 = NULL;                 // Mail from. Default: "ProjectName" <projectid@`hostname`>
$config['email_user']                 = $config['project_id'];
$config['email_sendmail_path']        = '/usr/sbin/sendmail'; // The location of the sendmail program.
$config['email_smtp_host']            = 'localhost';          // Outgoing SMTP server name.
$config['email_smtp_port']            = 25;                   // The port to connect.
$config['email_smtp_timeout']         = 10;                   // SMTP connection timeout in seconds.
$config['email_smtp_secure']          = NULL;                 // Enable encryption. Use 'tls' or 'ssl'
$config['email_smtp_auth']            = FALSE;                // Whether or not to use SMTP authentication.
$config['email_smtp_username']        = NULL;                 // SMTP username.
$config['email_smtp_password']        = NULL;                 // Password for SMTP authentication.
```

## <a name="transports-api">API</a>

API transports definitions are a bit more complex than the E-Mail configuration.  
The basis for configuration is `$config['alert']['transports']['api'][METHOD]` where `METHOD` can be `get`,`post` or `put`.  
This basis has to contain an array with URLs of each API to call.  
The URL can have the same placeholders as defined in the [Template-Syntax](#templates-syntax).  
If the `METHOD` is `get`, all placeholders will be URL-Encoded.  
The API transport uses cURL to call the APIs, therefore you might need to install `php5-curl` or similar in order to make it work.  
__Note__: it is highly recommended to define own [Templates](#templates) when you want to use the API transport. The default template might exceed URL-length for GET requests and therefore cause all sorts of errors.  

Example:
```php
$config['alert']['transports']['api']['get'][] = "https://api.thirdparti.es/issue?apikey=abcdefg&subject=%title";
```

## <a name="transports-nagios">Nagios Compatible</a>

The nagios transport will feed a FIFO at the defined location with the same format that nagios would.  
This allows you to use other Alerting-Systems to work with LibreNMS, for example [Flapjack](http://flapjack.io).
```php
$config['alert']['transports']['nagios'] = "/path/to/my.fifo"; //Flapjack expects it to be at '/var/cache/nagios3/event_stream.fifo'
```

## <a name="transports-irc">IRC</a>

The IRC transports only works together with the LibreNMS IRC-Bot.  
Configuration of the LibreNMS IRC-Bot is described [here](https://github.com/librenms/librenms/blob/master/doc/IRC-Bot.md).  
```php
$config['alert']['transports']['irc'] = true;
```