mirror/nttgin-BGPalerter
1
0
Fork 0
mirror of https://github.com/nttgin/BGPalerter.git synced 2024-05-19 06:50:08 +00:00
Code Issues Releases Wiki Activity

merged dev into release

Browse Source
This commit is contained in:
Massimo Candela
2023-02-23 17:38:44 +01:00
parent 6cd5e3b66d 80711b4f82
commit ca814aeeff
22 changed files with 1555 additions and 7027 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.github/workflows/main.yml vendored
View File

@@ -15,7 +15,7 @@ jobs:
- name: Set up Javascript/Node
uses: actions/setup-node@v3
with:
node-version: '14'
node-version: 18.14.0
- name: Check out code
uses: actions/checkout@v3
@@ -48,10 +48,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up Javascript/Node
- name: Set up nodejs
uses: actions/setup-node@v3
with:
node-version: '14'
node-version: 18.14.0
- name: Check out code
uses: actions/checkout@v3

2
Dockerfile
View File

@@ -1,5 +1,5 @@
# -- trivial container for BGPalerter
FROM node:14.20.0-alpine as build
FROM node:18.14.0-alpine as build
WORKDIR /opt/bgpalerter
COPY . .

6
build.sh
View File

@@ -7,11 +7,11 @@ npm ci --silent
npm run compile
./node_modules/.bin/pkg ./dist/package.json --targets node14-win-x64 --output bin/bgpalerter-win-x64 --loglevel=error
./node_modules/.bin/pkg ./dist/package.json --targets node18-win-x64 --output bin/bgpalerter-win-x64 --loglevel=error
./node_modules/.bin/pkg ./dist/package.json --targets node14-linux-x64 --output bin/bgpalerter-linux-x64 --loglevel=error
./node_modules/.bin/pkg ./dist/package.json --targets node18-linux-x64 --output bin/bgpalerter-linux-x64 --loglevel=error
./node_modules/.bin/pkg ./dist/package.json --targets node14-macos-x64 --output bin/bgpalerter-macos-x64 --loglevel=error
./node_modules/.bin/pkg ./dist/package.json --targets node18-macos-x64 --output bin/bgpalerter-macos-x64 --loglevel=error
echo "--> BGPalerter compiled in bin/ (ignore the warnings about files that cannot be resolved)."

21
config.yml.example
View File

@@ -64,7 +64,7 @@ monitors:
enableExpirationAlerts: true
enableExpirationCheckTA: true
enableDeletedCheckTA: true
enableAdvancedRpkiStats: true
enableAdvancedRpkiStats: false
roaExpirationAlertHours: 2
checkOnlyASns: true
toleranceDeletedRoasTA: 20
@@ -229,6 +229,7 @@ reports:
# - rpki
# - roa
# params:
# method: post
# templates: # See here how to write a template https://github.com/nttgin/BGPalerter/blob/main/docs/context.md
# default: '{"text": "${summary}"}'
# headers:
@@ -266,6 +267,24 @@ reports:
# params:
# maxAlertsAmount: 25
# - file: reportMatrix
# channels:
# - hijack
# - newprefix
# - visibility
# - path
# - misconfiguration
# - rpki
# - roa
# params:
# showPaths: 0 # Amount of AS_PATHs to report in the alert
# homeserverUrl: https://matrix.org
# accessToken: _ACCESS_TOKEN_
# roomIds:
# default: "_ROOM_ID_"
# noc: "_ROOM_ID_"
############################
# Notification settings:
# - notificationIntervalSeconds

2
docs/friends.md
View File

@@ -56,3 +56,5 @@ Please, let me know so I can add your company name here.
* Rechenzentrum Haßfurt GmbH (AS44973)
* obe.net (AS3399)
* IT-Total (AS8769)
* Speakup (AS49627)
* Nick Bouwhuis (AS202585)

4
docs/installation.md
View File

@@ -1,7 +1,7 @@
# Installation
## Requirements
BGPalerter requires a machine with *minimum* 2GB RAM and swap enabled. However, I recommend 4GB.
BGPalerter requires a machine with 4GB of RAM and swap enabled.
If you are using small AWS EC2 instances (or similar), remember to activate the swap.
## Running BGPalerter from binaries - Quick Setup
@@ -15,7 +15,7 @@ The first time you run it, the auto-configuration will start.
#### Linux
1. Download the binary: `wget https://github.com/nttgin/BGPalerter/releases/latest/download/bgpalerter-linux-x64`
1. Download the binary: `wget https://github.com/nttgin/BGPalerter/releases/latest/download/bgpalerter-linux-x64`
2. Download [`config.yml.example`](https://raw.githubusercontent.com/nttgin/BGPalerter/main/config.yml.example) as `config.yml` (in the same directory of the binary)

6
docs/node.md
View File

@@ -5,7 +5,7 @@ Some fast commands are below. The complete documentation (including other platfo
## With apt (e.g., debian)
```bash
curl -sL https://deb.nodesource.com/setup_14.x | sudo bash -
curl -sL https://deb.nodesource.com/setup_18.x | sudo bash -
sudo apt install nodejs
```
@@ -14,12 +14,12 @@ sudo apt install nodejs
```bash
brew update
brew install node@14
brew install node@18
```
## With yum (e.g., centos)
```bash
curl -sL https://deb.nodesource.com/setup_14.x | sudo bash -
curl -sL https://deb.nodesource.com/setup_18.x | sudo bash -
sudo yum install nodejs
```

10
docs/report-http.md
View File

@@ -1,11 +1,11 @@
# Send alerts with POST requests
# Send alerts with HTTP requests
BGPalerter can send alerts by means of POST requests to a provided URL.
This can be done by configuring the module reportHTTP. Read [here](configuration.md#reporthttp) to understand how.
BGPalerter can send alerts by means of HTTP requests to a provided URL.
This can be done by configuring the module reportHTTP. Read [here](reports.md#reportHTTP) to understand how.
For configuring reportHTTP, essentially you need to specify two things:
* The URL
* A template of the POST request.
* A template of the request.
If you are using [user groups](usergroups.md), you can specify a URL for every user group. This can be done inside `hooks`, a dictionary containing API URLs grouped by user group (key: group, value: URL).
The default user group is mandatory.
@@ -256,4 +256,4 @@ reports:
default: _API_URL_
```
Thanks [@momorientes](https://github.com/nttgin/BGPalerter/pull/923) and [@PacketVis](https://github.com/nttgin/BGPalerter/pull/1026).
Thanks [@momorientes](https://github.com/nttgin/BGPalerter/pull/923) and [@PacketVis](https://github.com/nttgin/BGPalerter/pull/1026).

43
docs/reports.md
View File

@@ -20,6 +20,7 @@ This will generate fake alerts. [Read more here](installation.md#bgpalerter-para
- [reportHTTP](reports.md#reportHTTP)
- [reportTelegram](reports.md#reportTelegram)
- [reportPullAPI](reports.md#reportPullAPI)
- [reportMatrix](reports.md#reportMatrix)
## reportFile
@@ -130,15 +131,16 @@ This report module sends alerts on a generic HTTP end-point.
Parameters for this report module:
|Parameter| Description|
|---|---|
|hooks| A dictionary containing API URLs grouped by user group (key: group, value: URL).|
|hooks.default| The URL of the default user group.|
|templates| A dictionary containing string templates for each channels. If a channel doesn't have a template defined, the `default` template will be used (see `config.yml.example` for more details). Read [here](context.md) how to write a template. |
|noProxy| If there is a global proxy configuration (see [here](http-proxy.md)), this parameter if set to true allows the single module to bypass the proxy. |
|isTemplateJSON| A boolean defining if the template provided above are JSON or plain string |
|headers| Additional headers to use in the GET request. For example for authentication.|
|showPaths| Amount of AS_PATHs to report in the alert (0 to disable). |
| Parameter | Description|
|----------------|---|
| hooks | A dictionary containing API URLs grouped by user group (key: group, value: URL).|
| hooks.default | The URL of the default user group.|
| templates | A dictionary containing string templates for each channels. If a channel doesn't have a template defined, the `default` template will be used (see `config.yml.example` for more details). Read [here](context.md) how to write a template. |
| noProxy | If there is a global proxy configuration (see [here](http-proxy.md)), this parameter if set to true allows the single module to bypass the proxy. |
| isTemplateJSON | A boolean defining if the template provided above are JSON or plain string |
| headers | Additional headers to use in the GET request. For example for authentication.|
| showPaths | Amount of AS_PATHs to report in the alert (0 to disable). |
| method | One of `post`, `put`, `patch`, `delete`. Default to `post`. |
[See here some examples of how to adapt reportHTTP to some common applications.](report-http.md)
@@ -171,10 +173,31 @@ This report module creates a REST API reachable at `http://host:port/alerts/`. T
The REST API uses the generic `rest` configuration in `config.yml`. Read [here](configuration.md) or see `config.yml.example` for more information.
Parameters for this report module:
|Parameter| Description|
|---|---|
|maxAlertsAmount| The maximum amount of alerts the API will return. By default set to 100. Don't exagerate with the number, the greater this value is the more memory BGPalerter will use. |
|noProxy| If there is a global proxy configuration (see [here](http-proxy.md)), this parameter if set to true allows the single module to bypass the proxy. |
## reportMatrix
This report module sends alerts directly to a specific Matrix room.
To send alert to Matrix you need an access token and a room ID.
You can find your access token in the Element client by going to All Settings > Help & About > Advanced. Read more about access tokens [here](https://spec.matrix.org/v1.6/client-server-api/#using-access-tokens).
You can find the room ID in the Element client by going to Room info > Room Settings > Advanced. Read more about Room ID [here](https://spec.matrix.org/latest/#room-structure).
Parameters for this report module:
|Parameter| Description|
|---|---|
|showPaths| Amount of AS_PATHs to report in the alert (0 to disable). |
|homeserverUrl| URL of your Matrix homeserver (for example: `https://matrix.org`. |
|noProxy| If there is a global proxy configuration (see [here](http-proxy.md)), this parameter if set to true allows the single module to bypass the proxy. |
|accessToken| The access token for authentication yourself to the Matrix API. |
|roomIds| A dictionary containing chat IDs grouped by user group (key: group, value: room ID). |
|roomIds.default| The room ID of the default room.|
Thanks to [@nickbouwhuis](https://github.com/nttgin/BGPalerter/pull/1043).

8294
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

20
package.json
View File

@@ -1,6 +1,6 @@
{
"name": "bgpalerter",
"version": "1.31.2",
"version": "1.32.0",
"description": "BGP and RPKI monitoring tool. Pre-configured for real-time detection of visibility loss, RPKI invalid announcements, hijacks, ROA misconfiguration, and more.",
"author": {
"name": "Massimo Candela",
@@ -44,8 +44,8 @@
"measurements"
],
"devDependencies": {
"@babel/cli": "^7.20.7",
"@babel/core": "^7.20.12",
"@babel/cli": "^7.21.0",
"@babel/core": "^7.21.0",
"@babel/node": "^7.20.7",
"@babel/plugin-proposal-class-properties": "^7.18.6",
"@babel/plugin-proposal-object-rest-spread": "^7.20.7",
@@ -58,7 +58,7 @@
"syslogd": "^1.1.2"
},
"dependencies": {
"@sentry/node": "^7.36.0",
"@sentry/node": "^7.38.0",
"axios": "=0.27.1",
"batch-promises": "^0.0.3",
"brembo": "^2.1.0",
@@ -66,7 +66,7 @@
"fast-file-logger": "^1.1.5",
"https-proxy-agent": "^5.0.1",
"inquirer": "=8.2.5",
"ip-sub": "^1.3.5",
"ip-sub": "^1.3.6",
"js-yaml": "^4.1.0",
"kafkajs": "^2.2.3",
"md5": "^2.3.0",
@@ -74,13 +74,13 @@
"node-cleanup": "^2.1.2",
"nodemailer": "^6.9.1",
"path": "^0.12.7",
"restify": "^8.6.1",
"rpki-validator": "^2.11.7",
"restify": "github:massimocandela/node-restify",
"rpki-validator": "^2.12.2",
"semver": "^7.3.8",
"syslog-client": "^1.1.1",
"uuid": "^9.0.0",
"ws": "^8.12.0",
"yargs": "^17.6.2"
"ws": "^8.12.1",
"yargs": "^17.7.1"
},
"pkg": {
"scripts": [
@@ -94,7 +94,7 @@
"./bin/config.yml"
],
"targets": [
"node14"
"node18"
]
},
"optionalDependencies": {

2
src/config/config.js
View File

@@ -90,7 +90,7 @@ export default class Config {
enableExpirationAlerts: true,
enableExpirationCheckTA: true,
enableDeletedCheckTA: true,
enableAdvancedRpkiStats: true,
enableAdvancedRpkiStats: false,
roaExpirationAlertHours: 2,
checkOnlyASns: true,
toleranceDeletedRoasTA: 20,

2
src/monitors/monitorNewPrefix.js
View File

@@ -56,7 +56,7 @@ export default class MonitorNewPrefix extends Monitor {
const message = alerts[0].matchedMessage;
const matchedRule = alerts[0].matchedRule;
return `Possible change of configuration. A new prefix ${message.prefix} is announced by ${message.originAS}. It is a more specific of ${matchedRule.prefix} (${matchedRule.description})`;
return `A new prefix ${message.prefix} is announced by ${message.originAS}. It is a more specific of ${matchedRule.prefix} (${matchedRule.description}). Maybe you need to update your BGPalerter prefix list.`;
}
return false;

12
src/monitors/monitorROAS.js
View File

@@ -35,16 +35,24 @@ export default class MonitorROAS extends Monitor {
};
if (this.enableDiffAlerts || this.enableDeletedCheckTA) {
setInterval(this._diffVrps, this.diffEverySeconds * 1000);
setInterval(() => {
this._skipIfStaleVrps(this._diffVrps);
}, this.diffEverySeconds * 1000);
}
if (this.enableExpirationAlerts || this.enableExpirationCheckTA) {
setInterval(() => {
this._verifyExpiration(this.roaExpirationAlertHours);
this._skipIfStaleVrps(() => this._verifyExpiration(this.roaExpirationAlertHours));
}, global.EXTERNAL_ROA_EXPIRATION_TEST || 600000);
}
};
_skipIfStaleVrps = (callback) => {
if (!this.rpki.getStatus().stale) {
callback();
}
}
_calculateSizes = (vrps) => {
const times = {};

2
src/monitors/monitorRPKI.js
View File

@@ -105,7 +105,7 @@ export default class MonitorRPKI extends Monitor {
_validate = (result, message, matchedRule) => {
const prefix = result.prefix;
const origin = result.origin.getValue();
if (result) {
if (result && !this.rpki.getStatus().stale) {
const cacheKey = "a" + [prefix, origin]
.join("-")

2
src/reports/reportAlerta.js
View File

@@ -42,7 +42,7 @@ export default class ReportAlerta extends Report {
if (!this.getUserGroup("default")) {
this.logger.log({
level: 'error',
message: "Alerta reporting is not enabled: no default group defined"
message: "Alerta is not enabled: no default group defined"
});
this.enabled = false;
}

13
src/reports/reportHTTP.js
View File

@@ -39,11 +39,20 @@ export default class ReportHTTP extends Report {
this.name = "reportHTTP" || this.params.name;
this.enabled = true;
this.method = (this.params?.method ?? "post").toLowerCase();
if (!["post", "put", "patch", "delete"].includes(this.method)) {
this.logger.log({
level: 'error',
message: `${this.name} is not enabled: the configured HTTP method is not valid`
});
this.enabled = false;
}
if (!this.getUserGroup("default")) {
this.logger.log({
level: 'error',
message: `${this.name} reporting is not enabled: no default group defined`
message: `${this.name} is not enabled: no default group defined`
});
this.enabled = false;
}
@@ -81,7 +90,7 @@ export default class ReportHTTP extends Report {
this.axios({
url,
method: "POST",
method: this.method,
headers: this.headers,
data: (this.params.isTemplateJSON) ? JSON.parse(blob) : blob
})

96
src/reports/reportMatrix.js Normal file
View File

@@ -0,0 +1,96 @@
/*
* BSD 3-Clause License
*
* Copyright (c) 2019, NTT Ltd.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* Redistributions of source code must retain the above copyright notice, this
* list of conditions and the following disclaimer.
*
* Redistributions in binary form must reproduce the above copyright notice,
* this list of conditions and the following disclaimer in the documentation
* and/or other materials provided with the distribution.
*
* Neither the name of the copyright holder nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
import ReportHTTP from "./reportHTTP";
import { v4 as uuidv4 } from 'uuid';
import brembo from "brembo";
export default class reportMatrix extends ReportHTTP {
constructor(channels, params, env) {
const hooks = {};
for (let userGroup in params?.roomIds ?? []) {
hooks[userGroup] = brembo.build(params?.homeserverUrl, {
path: ["_matrix", "client", "v3", "rooms", encodeURIComponent(params?.roomIds[userGroup]), "send", "m.room.message"]
});
}
const matrixParams = {
headers: {
"Content-Type": "application/json",
"Authorization": "Bearer " + params.accessToken
},
isTemplateJSON: true,
showPaths: params.showPaths,
hooks: hooks,
name: "reportMatrix",
method: "put",
templates: {}
};
super(channels, matrixParams, env);
this.roomIds = params.roomIds;
if (!params.homeserverUrl || !params.accessToken) {
this.logger.log({
level: 'error',
message: `${this.name} reporting is not enabled: homeserverUrl and accessToken are required`
});
this.enabled = false;
}
if (!params.roomIds || !params.roomIds["default"]) {
this.logger.log({
level: 'error',
message: `${this.name} reporting is not enabled: no default room id provided`
});
this.enabled = false;
}
};
getUserGroup = (group) => {
const transactionId = uuidv4();
const groups = this.params.hooks || this.params.userGroups || {};
const baseUrl = groups[group] || groups["default"];
return brembo.build(baseUrl, {path: [transactionId]})
};
getTemplate = (group, channel, content) => {
return JSON.stringify({
"msgtype": "m.text",
"body": "${summary}${markDownUrl}",
});
};
}

4
src/reports/reportTelegram.js
View File

@@ -57,7 +57,7 @@ export default class reportTelegram extends ReportHTTP {
if (!params.botUrl) {
this.logger.log({
level: 'error',
message: `${this.name} reporting is not enabled: no botUrl provided`
message: `${this.name} is not enabled: no botUrl provided`
});
this.enabled = false;
}
@@ -65,7 +65,7 @@ export default class reportTelegram extends ReportHTTP {
if (!params.chatIds || !params.chatIds["default"]) {
this.logger.log({
level: 'error',
message: `${this.name} reporting is not enabled: no default chat id provided`
message: `${this.name} is not enabled: no default chat id provided`
});
this.enabled = false;
}

2
src/reports/reportWebex.js
View File

@@ -41,7 +41,7 @@ export default class ReportWebex extends Report {
if (!this.getUserGroup("default")) {
this.logger.log({
level: 'error',
message: `Webex reporting is not enabled: no default group defined`
message: `Webex is not enabled: no default group defined`
});
this.enabled = false;
}

25
src/utils/rpkiUtils.js
View File

@@ -66,6 +66,7 @@ export default class RpkiUtils {
this._loadRpkiValidator();
if (this.params.markDataAsStaleAfterMinutes > 0) {
this._markAsStale();
setInterval(this._markAsStale, this.params.markDataAsStaleAfterMinutes * 60 * 1000);
}
@@ -174,7 +175,6 @@ export default class RpkiUtils {
.preCache(this.params.refreshVrpListMinutes)
.then(data => {
this.status.data = true;
this.status.stale = false;
return data;
})
@@ -189,7 +189,6 @@ export default class RpkiUtils {
})
} else {
this.status.data = true;
this.status.stale = false;
return Promise.resolve();
}
};
@@ -213,7 +212,7 @@ export default class RpkiUtils {
origin: message.originAS
};
}))
.then(results => {
.then((results=[]) => {
for (let result of results) {
const key = result.origin.getId() + "-" + result.prefix;
for (let { message, matchedRule, callback } of batch[key]) {
@@ -247,7 +246,7 @@ export default class RpkiUtils {
return Promise
.all(origins.map(asn => this.rpki.validate(prefix, asn, true))) // Validate each origin
.then(results => {
.then((results=[]) => {
if (results.length === 1) { // Only one result = only one origin, just return
return { ...results[0], prefix, origin };
} else { // Multiple origin
@@ -301,7 +300,23 @@ export default class RpkiUtils {
if (!!this.params.preCacheROAs) {
const digest = md5(JSON.stringify(this.getVRPs()));
if (this.oldDigest) {
this.status.stale = this.oldDigest === digest;
const stale = this.oldDigest === digest;
if (this.status.stale !== stale) {
if (stale) {
this.logger.log({
level: 'error',
message: "The VRP file is stale"
});
} else {
this.logger.log({
level: 'info',
message: "The VRP file is back to normal"
});
}
}
this.status.stale = stale;
}
this.oldDigest = digest;

8
tests/2_alerting.js
View File

@@ -228,7 +228,7 @@ describe("Alerting", function () {
id: '1234-175.254.205.0/25',
origin: 'prefix-detection',
affected: 1234,
message: 'Possible change of configuration. A new prefix 175.254.205.0/25 is announced by AS1234. It is a more specific of 175.254.205.0/24 (include exclude test)',
message: 'A new prefix 175.254.205.0/25 is announced by AS1234. It is a more specific of 175.254.205.0/24 (include exclude test). Maybe you need to update your BGPalerter prefix list.',
data: [
{
extra: {},
@@ -256,7 +256,7 @@ describe("Alerting", function () {
id: '1234-170.254.205.0/25',
origin: 'prefix-detection',
affected: 1234,
message: 'Possible change of configuration. A new prefix 170.254.205.0/25 is announced by AS1234. It is a more specific of 170.254.205.0/24 (include exclude test)',
message: 'A new prefix 170.254.205.0/25 is announced by AS1234. It is a more specific of 170.254.205.0/24 (include exclude test). Maybe you need to update your BGPalerter prefix list.',
data: [
{
extra: {},
@@ -285,7 +285,7 @@ describe("Alerting", function () {
id: '15562-165.254.255.0/25',
origin: 'prefix-detection',
affected: 15562,
message: 'Possible change of configuration. A new prefix 165.254.255.0/25 is announced by AS15562. It is a more specific of 165.254.255.0/24 (description 2)',
message: 'A new prefix 165.254.255.0/25 is announced by AS15562. It is a more specific of 165.254.255.0/24 (description 2). Maybe you need to update your BGPalerter prefix list.',
data: [
{
extra: {},
@@ -311,7 +311,7 @@ describe("Alerting", function () {
id: '204092-2a00:5884:ffff::/48',
origin: 'prefix-detection',
affected: "204092-45",
message: 'Possible change of configuration. A new prefix 2a00:5884:ffff::/48 is announced by AS204092. It is a more specific of 2a00:5884::/32 (alarig fix test)',
message: 'A new prefix 2a00:5884:ffff::/48 is announced by AS204092. It is a more specific of 2a00:5884::/32 (alarig fix test). Maybe you need to update your BGPalerter prefix list.',
data: [
{
extra: {},