mirror/nlnetlabs-rtrtr
1
0
Fork 0
mirror of https://github.com/NLnetLabs/rtrtr.git synced 2024-05-11 05:55:07 +00:00
Code Issues Releases Activity

Fix endless growth of history of "rtr" and "rtr-tls" targets. (#65)

Browse Source 
The "rtr" and "rtr-tls" targets both keep a list of changes in order to
process RTR serial queries. Due to a bug in the code, this maximum size
of that list was not limited as intended and keeps growing during runtime.

This change fixes this infinite growth and introduces a new config
parameter to the "rtr" and "rtr-tls" targets named "history-size" that
allows setting the maximum number of updates kept. When not present, it
defaults to 10.
This commit is contained in:
Martin Hoffmann
2022-03-10 12:25:31 +01:00
committed by GitHub
parent 55c05e6239
commit a8b1fb3731
5 changed files with 101 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
doc/manual/source/manual-page.rst
View File

@ -279,6 +279,14 @@ unit
A string value specifying the name of the unit that provides the data
set for the RTR target to offer.
history-size
An integer value specifying the number of diffs the target should keep
in order to process RTR serial queries, i.e., the number of updates to
the data set a client may fall behind before having to fetch the full
data set again.
If this value is missing, it defaults to 10.
The ``"rtr-tls"`` target has the following *additional* configuration
options:

100
doc/rtrtr.1
View File

@ -27,30 +27,29 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
..
.TH "RTRTR" "1" "Jan 19, 2022" "0.2.0" "RTRTR"
.TH "RTRTR" "1" "Mar 08, 2022" "0.2.1-dev" "RTRTR"
.SH NAME
rtrtr \- RPKI data proxy
.SH SYNOPSIS
.sp
\fBrtrtr\fP [options]
\fBrtrtr\fP [\fBoptions\fP]
.SH DESCRIPTION
.sp
RTRTR is an RPKI data proxy, designed to collect Validated ROA Payloads
from one or more sources in multiple formats and dispatch it onwards. It
provides the means to implement multiple distribution architectures for
RPKI such as centralised RPKI validators that dispatch data to local caching
RTR servers.
RTRTR is an RPKI data proxy, designed to collect Validated ROA Payloads from
one or more sources in multiple formats and dispatch it onwards. It provides
the means to implement multiple distribution architectures for RPKI such as
centralised RPKI validators that dispatch data to local caching RTR servers.
.sp
RTRTR can read RPKI data from multiple RPKI Relying Party packages via RTR
and JSON and, in turn, provide an RTR service for routers to connect to.
The HTTP server provides the validated data set in JSON format, as well as
a monitoring endpoint in plain text and Prometheus format.
and JSON and, in turn, provide an RTR service for routers to connect to. The
HTTP server provides the validated data set in JSON format, as well as a
monitoring endpoint in plain text and Prometheus format.
.SH OPTIONS
.INDENT 0.0
.TP
.B \-c path, \-\-config=path
Provides the path to a file containing the configuration for RTRTR. See
\fI\%Configuration File\fP below for more information on the format and
\fI\%CONFIGURATION FILE\fP below for more information on the format and
contents of the file.
.sp
This option is required.
@ -58,14 +57,15 @@ This option is required.
.INDENT 0.0
.TP
.B \-v, \-\-verbose
Print more information. If given twice, even more information is printed.
Print more information. If given twice, even more information is
printed.
.sp
More specifically, a single \fI\%\-v\fP increases the log level from the
default of warn to \fIinfo\fP, specifying it more than once increases it to
\fIdebug\fP\&.
More specifically, a single \fI\%\-v\fP increases the log level from
the default of warn to \fIinfo\fP, specifying it more than once increases
it to \fIdebug\fP\&.
.sp
See \fI\%Logging\fP below for more information on what information is logged at
the different levels.
See \fI\%LOGGING\fP below for more information on what information is logged
at the different levels.
.UNINDENT
.INDENT 0.0
.TP
@ -80,8 +80,8 @@ A single \fI\%\-q\fP will drop the log level to \fIerror\fP\&. Repeating
.B \-\-syslog
Redirect logging output to syslog.
.sp
This option is implied if a command is used that causes Routinator to run
in daemon mode.
This option is implied if a command is used that causes Routinator to
run in daemon mode.
.UNINDENT
.INDENT 0.0
.TP
@ -110,10 +110,10 @@ The configuration file describes how and from where RTRTR is collecting data,
how it processes it and how it should provide access to the resulting data
set or data sets.
.sp
The configuration file is a file in TOML format. It consists of a
sequence of key\-value pairs, each on its own line. Strings are to be enclosed in
double quotes. Lists can be given by enclosing a comma\-separated list of values
in square brackets. The file contains multiple sections, each started with a
The configuration file is a file in TOML format. It consists of a sequence of
key\-value pairs, each on its own line. Strings are to be enclosed in double
quotes. Lists can be given by enclosing a comma\-separated list of values in
square brackets. The file contains multiple sections, each started with a
name enclosed in square brackets.
.sp
The first section without a name at the beginning of the file provides
@ -134,11 +134,11 @@ its class. I.e., a unit named \fBfoo\fP would have a section name of
\fB[unit.foo]\fP while a target \fBbar\fP would have a section name of
\fB[target.bar]\fP\&.
.sp
The following reference lists all configuration options for the global section
as well as all options for each currently defined unit and target type. For
each option it states the name, type, and purpose. Any relative path given as
a configuration value is interpreted relative to the directory the
configuration file is located in.
The following reference lists all configuration options for the global
section as well as all options for each currently defined unit and target
type. For each option it states the name, type, and purpose. Any relative
path given as a configuration value is interpreted relative to the directory
the configuration file is located in.
.SH GLOBAL OPTIONS
.INDENT 0.0
.TP
@ -178,9 +178,9 @@ the log\-file configuration file entry.
The default if this value is missing is, unsurprisingly, default.
.TP
.B log\-file
A string value containing the path to a file to which log messages will be
appended if the log configuration value is set to file. In this case, the
value is mandatory.
A string value containing the path to a file to which log messages will
be appended if the log configuration value is set to file. In this
case, the value is mandatory.
.TP
.B syslog\-facility
A string value specifying the syslog facility to use for logging to
@ -214,14 +214,14 @@ If this option is missing, the default of 60 seconds is used.
.TP
.B cacerts
Only used with the \fB"rtr\-tls"\fP type, a list of paths to files that
contain one or more PEM encoded certificates that should be trusted when
verifying a TLS server certificate.
contain one or more PEM encoded certificates that should be trusted
when verifying a TLS server certificate.
.sp
The \fB"rtr\-tls"\fP unit also uses the usual set of web trust anchors, so
this option is only necessary when the RTR server doesn’t use a server
certificate that would be trusted by web browser. This is, for instance,
the case if the server uses a self\-signed certificate in which case this
certificate needs to be added via this option.
certificate that would be trusted by web browser. This is, for
instance, the case if the server uses a self\-signed certificate in
which case this certificate needs to be added via this option.
.UNINDENT
.SH JSON UNIT
.sp
@ -260,7 +260,8 @@ The \fB"any"\fP unit has the following configuration options:
.INDENT 0.0
.TP
.B sources
A list of strings each containing the name of a unit to use as a source.
A list of strings each containing the name of a unit to use as a
source.
.TP
.B random
A boolean value specifying whether the unit should pick a source unit
@ -270,9 +271,9 @@ picked in the order given.
.SH SLURM UNIT
.sp
A unit of type \fB"slurm"\fP will apply local exception rules to a data set
provided by another unit. These rules are defined through local JSON files
as described in \fI\%RFC 8416\fP\&. They allow to both filter out existing entries
in a data set as well as add new entries.
provided by another unit. These rules are defined through local JSON files as
described in \fI\%RFC 8416\fP\&. They allow to both filter out existing entries in a
data set as well as add new entries.
.sp
The \fB"slurm"\fP unit has the following configuration options:
.INDENT 0.0
@ -304,9 +305,18 @@ colon. IPv6 address should be enclosed in square brackets.
.B unit
A string value specifying the name of the unit that provides the data
set for the RTR target to offer.
.TP
.B history\-size
An integer value specifying the number of diffs the target should keep
in order to process RTR serial queries, i.e., the number of updates to
the data set a client may fall behind before having to fetch the full
data set again.
.sp
If this value is missing, it defaults to 10.
.UNINDENT
.sp
The \fB"rtr\-tls"\fP target has the following \fIadditional\fP configuration options:
The \fB"rtr\-tls"\fP target has the following \fIadditional\fP configuration
options:
.INDENT 0.0
.TP
.B certificate
@ -353,8 +363,8 @@ The log levels represent the following information:
.TP
.B error
Information related to events that prevent RTRTR from continuing to
operate at all as well as all issues related to local configuration even
if RTRTR will continue to run.
operate at all as well as all issues related to local configuration
even if RTRTR will continue to run.
.TP
.B warn
Information about events and data that influences the data sets
@ -370,8 +380,8 @@ Information about the internal state of RTRTR that may be useful for
debugging.
.UNINDENT
.SH AUTHOR
NLnet Labs (CC-BY 3.0)
NLnet Labs
.SH COPYRIGHT
2018-2021, NLnet Labs
2019–2022, NLnet Labs
.\" Generated by docutils manpage writer.
.

2
src/config.rs
View File

@ -322,7 +322,7 @@ impl ConfigFile {
fn resolve_pos(&self, pos: usize) -> LineCol {
let line = self.line_starts.iter().find(|&&start|
start < pos
).copied().unwrap_or_else(|| self.line_starts.len());
).copied().unwrap_or(self.line_starts.len());
let line = line - 1;
let col = self.line_starts[line] - pos;
LineCol { line, col }

45
src/targets/rtr.rs
View File

@ -33,15 +33,28 @@ use crate::manager::Component;
/// An RTR server atop unencrypted, plain TCP.
#[derive(Debug, Deserialize)]
pub struct Tcp {
/// The socket addresses to listen on.
listen: Vec<SocketAddr>,
/// The unit whose data set we should serve.
unit: Link,
/// The maximum number of deltas we should keep.
#[serde(default = "Tcp::default_history_size")]
#[serde(rename = "history-size")]
history_size: usize,
}
impl Tcp {
/// The default for the `history_size` value.
const fn default_history_size() -> usize {
10
}
/// Runs the target.
pub async fn run(mut self, component: Component) -> Result<(), ExitError> {
let mut notify = NotifySender::new();
let target = Source::default();
let target = Source::new(self.history_size);
for &addr in &self.listen {
self.spawn_listener(addr, target.clone(), notify.clone())?;
}
@ -101,9 +114,14 @@ impl Tcp {
/// An RTR server atop TLS.
#[derive(Debug, Deserialize)]
pub struct Tls {
listen: Vec<SocketAddr>,
unit: Link,
/// The configuration values shared with [`Tcp`].
#[serde(flatten)]
tcp: Tcp,
/// The path to the server certificate to present to clients.
certificate: ConfigPath,
/// The path to the private key to use for encryption.
key: ConfigPath,
}
@ -112,15 +130,15 @@ impl Tls {
pub async fn run(mut self, component: Component) -> Result<(), ExitError> {
let acceptor = TlsAcceptor::from(Arc::new(self.create_tls_config()?));
let mut notify = NotifySender::new();
let target = Source::default();
for &addr in &self.listen {
let target = Source::new(self.tcp.history_size);
for &addr in &self.tcp.listen {
self.spawn_listener(
addr, acceptor.clone(), target.clone(), notify.clone()
)?;
}
loop {
if let Ok(update) = self.unit.query().await {
if let Ok(update) = self.tcp.unit.query().await {
debug!(
"Target {}: Got update ({} entries)",
component.name(), update.set().len()
@ -242,13 +260,20 @@ impl Tls {
//------------ Source --------------------------------------------------------
#[derive(Clone, Default)]
#[derive(Clone)]
struct Source {
data: Arc<ArcSwap<SourceData>>,
diff_num: usize,
history_size: usize,
}
impl Source {
fn new(history_size: usize) -> Self {
Source {
data: Default::default(),
history_size
}
}
fn update(&self, update: payload::Update) {
let data = self.data.load();
@ -272,11 +297,11 @@ impl Source {
return
}
let mut diffs = Vec::with_capacity(
cmp::min(data.diffs.len() + 1, self.diff_num)
cmp::min(data.diffs.len() + 1, self.history_size)
);
diffs.push((data.state.serial(), diff.clone()));
for (serial, old_diff) in &data.diffs {
if diffs.len() == self.diff_num {
if diffs.len() >= self.history_size {
break
}
diffs.push((

2
test-data/rtrtr.conf
View File

@ -71,6 +71,7 @@ files = [ "exceptions.json" ]
type = "rtr"
listen = [ "127.0.0.1:3325" ]
unit = "any-any-exceptions"
history-size = 12
[targets.rtr-tls-9325]
type = "rtr-tls"
@ -85,6 +86,7 @@ listen = [ "127.0.0.1:9326" ]
unit = "rtr-tls-loop"
certificate = "localhost.crt"
key = "localhost.key"
history-size = 12
[targets.http-json]
type = "http"