mirror/nttgin-BGPalerter
1
0
Fork 0
mirror of https://github.com/nttgin/BGPalerter.git synced 2024-05-19 06:50:08 +00:00
Code Issues Releases Wiki Activity

improved documentation, needs revision

Browse Source
This commit is contained in:
Massimo Candela
2019-10-05 19:18:11 +02:00
parent 7a0c0581c0
commit 54ca735669
5 changed files with 263 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files

114
README.md
View File

@@ -2,6 +2,8 @@
# BGPalerter
Real-time BGP monitoring tool, pre-configured for visibility loss and hijacks detection.
You just run it. You don't need to provide any data source or connect it to anything in your network since it connects to publiic repos.
It can deliver alerts on files, by email, on slack, and more.
![BGPalerter](https://massimocandela.com/img/bgpalerter_github_image.png)
@@ -12,109 +14,33 @@ If you want to know more about the source code (which is completely open) please
1. Download the binary [here](https://github.com/nttgin/BGPalerter/releases) (be sure to select the one for your OS)
2. Download [`config.yml.example`](https://raw.githubusercontent.com/nttgin/BGPalerter/master/config.yml.example) as `config.yaml` and [`prefixes.yml.example`](https://raw.githubusercontent.com/nttgin/BGPalerter/master/prefixes.yml.example) as `prefixes.yml`, and place them in the same directory of the executable (if you skip this step, some default configuration files will be generated during the first execution)
2. Download [`config.yml.example`](https://raw.githubusercontent.com/nttgin/BGPalerter/master/config.yml.example) as `config.yml` and [`prefixes.yml.example`](https://raw.githubusercontent.com/nttgin/BGPalerter/master/prefixes.yml.example) as `prefixes.yml`, and place them in the same directory of the executable (if you skip this step, some default configuration files will be generated during the first execution)
3. Modify `prefixes.yml` and add the prefixes you want to monitor (or see below how to auto generate this file)
4. Run the executable
4. Run the executable (e.g. `chmod 700 bgpalerter-linux-x64 && nohup ./bgpalerter-linux-x64 &`)
5. See the alerts in `logs/reports-YYYY-MM-DD.log`
5. See the alerts in `logs/reports-YYYY-MM-DD.log` (e.g. `tail -f logs/reports*`)
In `config.yml` you can find other reporting mechanisms (e.g. email and slack) in addition to logging to files.
In `config.yml.example` you can find other reporting mechanisms (e.g. email and slack) in addition to logging to files.
Please uncomment the related section and configure according to your needs.
If you enable email reporting, download also the directory `reports/email_templates` in the same directory of the executable.
## More information for users
- [Monitored prefixes List](docs/prefixes.md#prefixes)
- [Auto-generate](docs/prefixes.md#generate)
- [Fields and meaning](docs/prefixes.md#prefixes-fields)
### Composition
You can compose the tool with 3 main components: connectors, monitors, and reports.
Connectors retrieve/listen to the data from different sources and transform them to a common format.
Monitors analyze the data flow and produce alerts. Different monitors try to detect different issues.
Reports send/store the alerts, e.g. by email or to a file.
##### connectors
Possible connectors are:
* _connectorRIS_, for real-time data from RIPE RIS (https://ris-live.ripe.net/)
* _connectorTest_, for testing purposes, it provokes all types of alerting
##### monitors
Possible monitors are:
* _monitorHijack_, for monitoring hijacks
* _monitorVisibility_, for monitoring prefixes' visibility (you will get notified when withdrawals make monitored routes disappear). A threshold can be specified in config.yml to trigger an alert only if the issue is visible from a certain amount of peers.
* _monitorNewPrefix_, for monitoring if new more specifics (of the monitored prefixes) start to be announced
##### reports
Possible reports are:
* _reportEmail_, to send alerts by email. SMTP configurations are in config.yml
* _reportFile_, to log the alerts to files. File directory, format, and log rotation configurations are in config.yml
* _reportSlack_, to send alerts in Slack. Hook URL is configurable in config.yml
## More information for developers
To start development:
1. git clone this repo
2. install Node.js (version >= 10.16) and npm ([installing node and npm](https://nodejs.org/en/download/))
3. execute `npm install` or `yarn` to install all dependencies ([installing yarn](https://yarnpkg.com/lang/en/docs/install))
4. run `npm run watch-and-serve` to run the application. At every file change it will self-reload.
### All npm commands
* `npm run watch-and-serve` to run the application from source code and monitor for file changes
* `npm run serve` to run the application from the source
* `npm run start|stop|restart` to run the application as a service in background
* `npm run status` to see the status of the service
* `npm run test` to run the tests
* `npm run build` to compile and build OS native applications
* `npm run generate-prefixes -- --a ASN(S) --o OUTPUT_FILE` to generate the monitored prefixes file
### Composition
You can compose the tool with 3 main components: connectors, monitors, and reports.
> **Important:**
All connectors MUST extend the class Connector. Monitors extend the class Monitor. Reports extend the class Report.
From the superclass they will inherit various generic methods while some have to be implemented.
Reports don't receive only alerts but also the data that provoked such alerts (so you can store the data and replay the accident later).
In `config.yml`, for each collection of components:
* `file` - refers to the file name which contains the class
* `channels` - refer to what channel(s) will be used to send or receive messages
* `params` - whatever parameters may be needed by the component at creation time
## Documentation
- [Installation](docs/installation.md)
- [Binary installation](docs/installation.md#running-bgpalerter-from-binaries)
- [Source code installation](docs/installation.md#running-bgpalerter-from-the-source-code)
- [Monitored prefixes list](docs/prefixes.md#prefixes)
- [Generate prefix list](docs/prefixes.md#generate)
- [Prefix attributes description](docs/prefixes.md#prefixes-fields)
- [Configuration](docs/configuration.md)
- [Composition](docs/configuration.md#composition)
- [Connectors](docs/configuration.md#connectors)
- [Monitors](docs/configuration.md#monitors)
- [Reports](docs/configuration.md#reports)
- [More information for developers](docs/develop.md)
- [All npm commands](docs/develop.md#all-npm-commands)

181
docs/configuration.md Normal file
View File

@@ -0,0 +1,181 @@
# Configuration
The entire configuration is contained in `config.yml`.
The following are common parameters which it is possible to specify in the configuration.
| Parameter | Description | Expected format | Example | Required |
|---|---|---|---|---|
|environment| You can specify various environments. The values "production" (not verbose) and "development" (verbose) will affect the verbosity of the error/debug logs. Other values don't affect the functionalities, they will be used to identify from which environment the log is coming from. | A string | production | Yes |
|notificationIntervalSeconds| The amount of seconds before the same alert can be repeated. An alert is repeated only if the cause of it has not being solved. | An integer | 1800 | Yes |
|clearNotificationQueueAfterSeconds| If the cause of an alert is resolved, then stop waiting for more information about the issue. | An integer (greater than notificationIntervalSeconds) | 1900 | Yes |
|checkStaleNotificationsSeconds| The amount of seconds between a check on stale alerts. A stale alert happens when the cause of an alert is resolved before the next notification round, in such a case send it anyway. | An integer | 60 | Yes |
|monitoredPrefixesFiles| The [list](docs/prefixes.md#array) of files containing the prefixes to monitor. See [here](docs/prefixes.md#prefixes) for more informations. | A list of strings (valid .yml files) | -prefixes.yml | Yes |
|logging| A dictionary of parameters containing the configuration for the file logging. | || Yes|
|logging.directory| The directory where the log files will be generated. The directory will be created if not existent. | A string | logs | Yes |
|logging.logRotatePattern| A pattern with date placeholders indicating the name of the file. This pattern will also indicate when a log file is rotated. | A string with date placeholders (YYYY, MM, DD, ss, hh) | YYYY-MM-DD | Yes |
|logging.zippedArchive| Indicates if when a file gets rotates it has to be zipped or not. | A boolean | true | Yes |
|logging.maxSize| Indicates the maximum file size allowed before to be rotated (by adding .number ad the end). This allows to rotate files when logRotatePattern still the same but the file is too big | A string (indicating an amount and a unit of measure) | 20m | Yes |
|logger.maxFiles| Indicates the maximum amount of files or the maximum amount of days the files are retained. When this threshold is passed, files get deleted. | A string (a number or an amount of days ending with "d") | 14d | Yes |
|checkForUpdatesAtBoot| Indicates if at each booth the application should check for updates. If an update is available, a notification will be sent to the default group. If you restart the process often (e.g. debugging, experimenting etc.) set this to false to avoid notifications. Anyway, BGPalerter checks for updates every 10 days.| A boolean | true | Yes |
## Composition
You can compose the tool with 3 main components: connectors, monitors, and reports.
* Connectors retrieve/listen to the data from different sources and transform them to a common format.
* Monitors analyze the data flow and produce alerts. Different monitors try to detect different issues.
* Reports send/store the alerts, e.g. by email or to a file.
```yaml
connectors:
- file: connectorRIS
name: ris
monitors:
- file: monitorHijack
channel: hijack
name: basic-hijack-detection
- file: monitorPath
channel: path
name: path-matching
params:
thresholdMinPeers: 10
reports:
- file: reportFile
channels:
- hijack
- path
```
Each monitor declaration is composed of:
| Attribute | Description |
|---|---|
| file | Name of the file containing the monitor implementation. Monitor implementations are in the `moniors` directory. |
| channel | The name of the channel that will be used by the monitor to dispatch messages. If the inserted name doesn't correspond to an already existent channel, a new channel is created.|
|name| The name associated to the monitor. Multiple monitors with the same implementation can be loaded with different names. This name will be used to annotate messages in order track from where they are coming from.|
|params| A dictionary of parameters that can be useful for the functioning of the monitor. Different monitors with the same implementation can be initialized with different parameters. |
Each report declaration is composed of:
| Attribute | Description |
|---|---|
| file | Name of the file containing the report implementation. Report implementations are in the `reports` directory. |
|params| A dictionary of parameters that can be useful for the functioning of the report. It is common to have group declarations among the parameters. Different reports with the same implementation can be initialized with different parameters. |
|channels| A [list](docs/prefixes.md#array) of channels the monitor will listen (never write). Different reports with the same implementation can be initialized with a different list of channels to listen.|
Each connector is composed of:
| Attribute | Description |
|---|---|
|file|Name of the file containing the connector implementation. Report implementations are in the `connectors` directory. |
|params| A dictionary of parameters that can be useful for the functioning of the connector. E.g. the data source url, password, socket options|
|name| The name that will be used to identify this connector and to annotate logs and messages. |
> Connectors will always send the BGP updates to all the channels. The BGP updates have all the same format.
### Connectors
#### connectorRIS
It connects to RIPE RIS https://ris-live.ripe.net/ and receives BGP updates coming from 600+ peers.
Parameters for this connector module:
|Parameter| Description|
|---|---|
|url| WebSocket end-point of RIS, which currently is `wss://ris-live.ripe.net/v1/ws/` |
|subscription| Dictionary containing the parameters required by RIS. Refer to the [official documentation](https://ris-live.ripe.net/) for details.|
#### connectorTest
Connector used for testing purposes, it provokes all types of alerting. Needed to run the tests (`npm run test`) .
### Monitors
#### monitorHijack
This monitor has the logic to detect basic hijack.
In particular, it will monitor for all the declared prefixes and will trigger an alert when:
* The origin AS of the prefix is different from the ones specified in the configuration.
* A more specific of the prefix has been announced by an AS which is different from the ones specified.
* The BGP update declares an AS_SET as origin and at least one of the AS in the AS_SET is not specified in the configuration.
Parameters for this monitor module:
|Parameter| Description|
|---|---|
|thresholdMinPeers| Minimum number of peers that need to see the BGP update before to trigger an alert. |
#### monitorVisibility
This monitor has the logic to detect loss of visibility.
In particular, it will monitor for all the declared prefixes and will trigger an alert when:
* The prefix is not visible anymore from at least `thresholdMinPeers` peers.
Parameters for this monitor module:
|Parameter| Description|
|---|---|
|thresholdMinPeers| Minimum number of peers that need to see the BGP update before to trigger an alert. |
#### monitorNewPrefix
This monitor has the logic to detect unexpected change of configuration in the form of new prefixes announced by the correct AS.
In particular, it will monitor for all the declared prefixes and will trigger an alert when:
* A sub-prefix of the monitored prefix starts to be announced by the same AS declared for the prefix.
Parameters for this monitor module:
|Parameter| Description|
|---|---|
|thresholdMinPeers| Minimum number of peers that need to see the BGP update before to trigger an alert. |
### Reports
Possible reports are:
#### reportFile
This report module is the default one. It sends the alerts as verbose logs.
To configure the logs see the [configuration introduction](configuration.md).
#### reportEmail
This report module sends the alerts by email.
Parameters for this report module:
|Parameter| Description|
|---|---|
|senderEmail| The email address that will be used as sender for the alerts. |
|smtp| A dictionary containing the SMTP configuration. Some parameters are described in `config.yml.example`. For all the options refer to the [nodemailer documentation](https://nodemailer.com/smtp/). |
|notifiedEmails| A dictionary containing email addresses grouped by user groups. (key: group, value: list of emails)|
|notifiedEmails.default| The default user group. Each user group is a [list](prefixes.md#array) of emails. This group should contain at least the admin. |
#### reportSlack
This report module sends alerts on Slack.
Parameters for this report module:
|Parameter| Description|
|---|---|
|colors| A dictionary having as key the event channel and as value a hex color (string). These colors will be used to make messages in Slack distinguishable. |
|hooks| A dictionary containing Slack WebHooks grouped by user group (key: group, value: WebHook).|
|hooks.default| The default user group. Each user group is a WebHook (url). |

30
docs/develop.md Normal file
View File

@@ -0,0 +1,30 @@
# More information for developers
To start development see how to install the source [here](installation.md#running-bgpalerter-from-the-source-code).
## All npm commands
* `npm run watch-and-serve` to run the application from source code and monitor for file changes
* `npm run serve` to run the application from the source
* `npm run start|stop|restart` to run the application as a service in background
* `npm run status` to see the status of the service
* `npm run test` to run the tests
* `npm run build` to compile and build OS native applications
* `npm run generate-prefixes -- --a ASN(S) --o OUTPUT_FILE` to generate the monitored prefixes file
## Composition notes
You can compose the tool with 3 main components: connectors, monitors, and reports.
> **Important:**
All connectors MUST extend the class Connector. Monitors extend the class Monitor. Reports extend the class Report.
From the superclass they will inherit various generic methods while some have to be implemented.
Reports don't receive only alerts but also the data that provoked such alerts (so you can store the data and replay the accident later).

31
docs/installation.md Normal file
View File

@@ -0,0 +1,31 @@
# Installation
## Running BGPalerter from binaries
1. Download the binary [here](https://github.com/nttgin/BGPalerter/releases) (be sure to select the one for your OS)
2. Download [`config.yml.example`](https://raw.githubusercontent.com/nttgin/BGPalerter/master/config.yml.example) as `config.yml` and [`prefixes.yml.example`](https://raw.githubusercontent.com/nttgin/BGPalerter/master/prefixes.yml.example) as `prefixes.yml`, and place them in the same directory of the executable (if you skip this step, some default configuration files will be generated during the first execution)
3. Modify `prefixes.yml` and add the prefixes you want to monitor (or see below how to auto generate this file)
4. Run the executable (e.g. `chmod 700 bgpalerter-linux-x64 && nohup ./bgpalerter-linux-x64 &`)
5. See the alerts in `logs/reports-YYYY-MM-DD.log` (e.g. `tail -f logs/reports*`)
In `config.yml.example` you can find other reporting mechanisms (e.g. email and slack) in addition to logging to files.
Please uncomment the related section and configure according to your needs.
If you enable email reporting, download also the directory `reports/email_templates` in the same directory of the executable.
## Running BGPalerter from the source code
1. git clone this repo
2. install Node.js (version >= 10.16) and npm ([installing node and npm](https://nodejs.org/en/download/))
3. execute `npm install` or `yarn` to install all dependencies ([installing yarn](https://yarnpkg.com/lang/en/docs/install))
4. run `npm run watch-and-serve` to run the application. At every file change it will self-reload.

1
docs/prefixes.md
View File

@@ -41,6 +41,7 @@ The prefix list is a file containing a series of blocks like the one below, one
```
###### <a name="array"></a>
> Tip: In yml, arrays of values are described with dashes, like below:
```
asn: