mirror/librenms-librenms
1
0
Fork 0
mirror of https://github.com/librenms/librenms.git synced 2024-10-07 16:52:45 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #2458 from laf/rrdtune

Browse Source 
Fix for crazy traffic values seen in graphs
This commit is contained in:
Daniel Preussker
2015-12-08 16:12:22 +00:00
parent d13e86579c 48fab36592
commit a2d97f4283
11 changed files with 168 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
doc/Extensions/RRDTune.md Normal file
View File

@@ -0,0 +1,21 @@
# RRDTune?
When we create rrd files for ports, we currently do so with a max value of 12500000000 (100G). Because of this if a device sends us bad data back then it can appear as though
a 100M port is doing 40G+ which is impossible. To counter this you can enable the rrdtool tune option which will fix the max value to the interfaces physical speed (minimum of 10M).
To enable this you can do so in three ways!
- Globally under Global Settings -> External Settings -> RRDTool Setup
- For the actual device, Edit Device -> Misc
- For each port, Edit Device -> Port Settings
Now when a port interface speed changes (this can happen because of a physical change or just because the device has mis-reported) the max value is set. If you don't want to wait until
a port speed changes then you can run the included script:
script/tune_port.php -h <hostname> -p <ifName>
Wildcards are supported using *, i.e:
script/tune_port.php -h local* -p eth*
This script will then perform the rrdtool tune on each port found using the provided ifSpeed for that port.

8
doc/Support/FAQ.md
View File

@@ -12,6 +12,7 @@
- [How do I debug the discovery process?](#faq11)
- [How do I debug the poller process?](#faq12)
- [Why do I get a lot apache or rrdtool zombies in my process list?](#faq14)
- [Why do I see traffic spikes in my graphs?](#faq15)
### Developing
- [How do I add support for a new OS?](#faq8)
@@ -94,6 +95,13 @@ Please see the [Poller Support](http://docs.librenms.org/Support/Poller Support)
If this is related to your web service for LibreNMS then this has been tracked down to an issue within php which the developers aren't fixing. We have implemented a work around which means you
shouldn't be seeing this. If you are, please report this in [issue 443](https://github.com/librenms/librenms/issues/443).
#### <a name="faq15"> Why do I see traffic spikes in my graphs?</a>
This occurs either when a counter resets or the device sends back bogus data making it look like a counter reset. We have enabled support for setting a maximum value for rrd files for ports.
Before this all rrd files were set to 100G max values, now you can enable support to limit this to the actual port speed.
rrdtool tune will change the max value when the interface speed is detected as being changed (min value will be set for anything 10M or over) or when you run the included script (scripts/tune_port.php).
#### <a name="faq8"> How do I add support for a new OS?</a>
The easiest way to show you how to do that is to link to an existing pull request that has been merged in on [GitHub](https://github.com/librenms/librenms/pull/352/files)

23
html/includes/table/edit-ports.inc.php
View File

@@ -54,17 +54,22 @@ foreach (dbFetchRows($sql, $param) as $port) {
$isportbad = ($port['ifOperStatus'] == 'down' && $port['ifAdminStatus'] != 'down') ? 1 : 0;
$dowecare = ($port['ignore'] == 0 && $port['disabled'] == 0) ? $isportbad : !$isportbad;
$outofsync = $dowecare ? " class='red'" : '';
$checked = '';
if (get_dev_attrib($device_id, 'ifName_tune:'.$port['ifName']) == "true") {
$checked = 'checked';
}
$response[] = array(
'ifIndex' => $port['ifIndex'],
'ifName' => $port['label'],
'ifAdminStatus' => $port['ifAdminStatus'],
'ifOperStatus' => '<span name="operstatus_'.$port['port_id'].'"'.$outofsync.'>'.$port['ifOperStatus'].'</span>',
'disabled' => '<input type="checkbox" class="disable-check" name="disabled_'.$port['port_id'].'"'.($port['disabled'] ? 'checked' : '').'>
<input type="hidden" name="olddis_'.$port['port_id'].'" value="'.($port['disabled'] ? 1 : 0).'"">',
'ignore' => '<input type="checkbox" class="ignore-check" name="ignore_'.$port['port_id'].'"'.($port['ignore'] ? 'checked' : '').'>
<input type="hidden" name="oldign_'.$port['port_id'].'" value="'.($port['ignore'] ? 1 : 0).'"">',
'ifAlias' => '<div class="form-group"><input class="form-control input-sm" id="if-alias" name="if-alias" data-device_id="'.$port['device_id'].'" data-port_id="'.$port['port_id'].'" data-ifName="'.$port['ifName'].'" value="'.$port['ifAlias'].'"><span class="glyphicon form-control-feedback" aria-hidden="true"></span></div>'
'ifIndex' => $port['ifIndex'],
'ifName' => $port['label'],
'ifAdminStatus' => $port['ifAdminStatus'],
'ifOperStatus' => '<span name="operstatus_'.$port['port_id'].'"'.$outofsync.'>'.$port['ifOperStatus'].'</span>',
'disabled' => '<input type="checkbox" class="disable-check" name="disabled_'.$port['port_id'].'"'.($port['disabled'] ? 'checked' : '').'>
<input type="hidden" name="olddis_'.$port['port_id'].'" value="'.($port['disabled'] ? 1 : 0).'"">',
'ignore' => '<input type="checkbox" class="ignore-check" name="ignore_'.$port['port_id'].'"'.($port['ignore'] ? 'checked' : '').'>
<input type="hidden" name="oldign_'.$port['port_id'].'" value="'.($port['ignore'] ? 1 : 0).'"">',
'port_tune' => '<input type="checkbox" id="override_config" name="override_config" data-attrib="ifName_tune:'.$port['ifName'].'" data-device_id="'.$port['device_id'].'" data-size="small" '.$checked.'>',
'ifAlias' => '<div class="form-group"><input class="form-control input-sm" id="if-alias" name="if-alias" data-device_id="'.$port['device_id'].'" data-port_id="'.$port['port_id'].'" data-ifName="'.$port['ifName'].'" value="'.$port['ifAlias'].'"><span class="glyphicon form-control-feedback" aria-hidden="true"></span></div>',
);
}//end foreach

46
html/js/librenms.js
View File

@@ -1,30 +1,34 @@
function override_config(event, state, tmp_this) {
event.preventDefault();
var $this = tmp_this;
var attrib = $this.data('attrib');
var device_id = $this.data('device_id');
$.ajax({
type: 'POST',
url: 'ajax_form.php',
data: { type: 'override-config', device_id: device_id, attrib: attrib, state: state },
dataType: 'json',
success: function(data) {
if (data.status == 'ok') {
toastr.success(data.message);
}
else {
toastr.error(data.message);
}
},
error: function() {
toastr.error('Could not set this override');
}
});
}
var oldH;
var oldW;
$(document).ready(function() {
// Device override ajax calls
$("[name='override_config']").bootstrapSwitch('offColor','danger');
$('input[name="override_config"]').on('switchChange.bootstrapSwitch', function(event, state) {
event.preventDefault();
var $this = $(this);
var attrib = $this.data('attrib');
var device_id = $this.data('device_id');
$.ajax({
type: 'POST',
url: 'ajax_form.php',
data: { type: 'override-config', device_id: device_id, attrib: attrib, state: state },
dataType: 'json',
success: function(data) {
if (data.status == 'ok') {
toastr.success(data.message);
}
else {
toastr.error(data.message);
}
},
error: function() {
toastr.error('Could not set this override');
}
});
override_config(event,state,$(this));
});
// Device override for text inputs

18
html/pages/device/edit/misc.inc.php
View File

@@ -3,23 +3,29 @@
echo '
<form class="form-horizontal">
<div class="form-group">
<label for="icmp" class="col-sm-2 control-label">Disable ICMP Test?</label>
<div class="col-sm-10">
<label for="icmp" class="col-sm-4 control-label">Disable ICMP Test?</label>
<div class="col-sm-8">
'.dynamic_override_config('checkbox','override_icmp_disable', $device).'
</div>
</div>
<div class="form-group">
<label for="oxidized" class="col-sm-2 control-label">Exclude from Oxidized?</label>
<div class="col-sm-10">
<label for="oxidized" class="col-sm-4 control-label">Exclude from Oxidized?</label>
<div class="col-sm-8">
'.dynamic_override_config('checkbox','override_Oxidized_disable', $device).'
</div>
</div>
<div class="form-group">
<label for="unixagent" class="col-sm-2 control-label">Unix agent port</label>
<div class="col-sm-10">
<label for="unixagent" class="col-sm-4 control-label">Unix agent port</label>
<div class="col-sm-8">
'.dynamic_override_config('text','override_Unixagent_port', $device).'
</div>
</div>
<div class="form-group">
<label for="unixagent" class="col-sm-4 control-label">Enable RRD Tune for all ports?</label>
<div class="col-sm-8">
'.dynamic_override_config('checkbox','override_rrdtool_tune', $device).'
</div>
</div>
</form>
';

9
html/pages/device/edit/ports.inc.php
View File

@@ -5,7 +5,7 @@
<input type='hidden' name='ignoreport' value='yes'>
<input type='hidden' name='type' value='update-ports'>
<input type='hidden' name='device' value='<?php echo $device['device_id'];?>'>
<div class='table-responsibe'>
<div class='table-responsive'>
<table id='edit-ports' class='table table-striped'>
<thead>
<tr>
@@ -15,6 +15,7 @@
<th data-column-id='ifOperStatus'>Oper</th>
<th data-column-id='disabled' data-sortable='false'>Disable</th>
<th data-column-id='ignore' data-sortable='false'>Ignore</th>
<th data-column-id='port_tune' data-sortable='false' data-searchable='false'>RRD Tune</th>
<th data-column-id='ifAlias'>Description</th>
</tr>
</thead>
@@ -23,6 +24,7 @@
</form>
<script>
//$("[name='override_config']").bootstrapSwitch('offColor','danger');
$(document).on('blur', "[name='if-alias']", function (){
var $this = $(this);
var descr = $this.val();
@@ -153,5 +155,10 @@
};
},
url: "ajax_table.php"
}).on("loaded.rs.jquery.bootgrid", function() {
$("[name='override_config']").bootstrapSwitch('offColor','danger');
$('input[name="override_config"]').on('switchChange.bootstrapSwitch', function(event, state) {
override_config(event,state,$(this));
});
});
</script>

12
html/pages/settings/external.inc.php
View File

@@ -34,6 +34,17 @@ $unixagent_conf = array(
),
);
$rrdtool_conf = array(
array('name' => 'rrdtool',
'descr' => 'Path to rrdtool binary',
'type' => 'text',
),
array('name' => 'rrdtool_tune',
'descr' => 'Tune all rrd port files to use max values',
'type' => 'checkbox',
),
);
echo '
<div class="panel-group" id="accordion">
<form class="form-horizontal" role="form" action="" method="post">
@@ -41,6 +52,7 @@ echo '
echo generate_dynamic_config_panel('Oxidized integration',true,$config_groups,$oxidized_conf);
echo generate_dynamic_config_panel('Unix-agent integration',true,$config_groups,$unixagent_conf);
echo generate_dynamic_config_panel('RRDTool Setup',true,$config_groups,$rrdtool_conf);
echo '
</form>

13
includes/polling/ports.inc.php
View File

@@ -339,6 +339,7 @@ foreach ($ports as $port) {
}
// Update IF-MIB data
$tune_port = false;
foreach ($data_oids as $oid) {
if ($oid == 'ifAlias') {
@@ -358,6 +359,15 @@ foreach ($ports as $port) {
}
}
else if ($port[$oid] != $this_port[$oid]) {
$port_tune = get_dev_attrib($device, 'ifName_tune:'.$port['ifName']);
$device_tune = get_dev_attrib($device,'override_rrdtool_tune');
if ($port_tune == "true" ||
($device_tune == "true" && $port_tune != 'false') ||
($config['rrdtool_tune'] == "true" && $port_tune != 'false' && $device_tune != 'false')) {
if ($oid == 'ifSpeed') {
$tune_port = true;
}
}
$port['update'][$oid] = $this_port[$oid];
log_event($oid.': '.$port[$oid].' -> '.$this_port[$oid], $device, 'interface', $port['port_id']);
if ($debug) {
@@ -501,6 +511,9 @@ foreach ($ports as $port) {
'OUTMULTICASTPKTS' => $this_port['ifOutMulticastPkts'],
);
if ($tune_port === true) {
rrdtool_tune('port',$rrdfile,$this_port['ifSpeed']);
}
rrdtool_update("$rrdfile", $fields);
// End Update IF-MIB
// Update PAgP

23
includes/rrdtool.inc.php
View File

@@ -166,7 +166,11 @@ function rrdtool_graph($graph_file, $options) {
function rrdtool($command, $filename, $options) {
global $config, $debug, $rrd_pipes, $console_color;
if ($config['rrdcached'] && ($config['rrdtool_version'] >= 1.5 || $command != "create")) {
if ($config['rrdcached'] &&
(version_compare($config['rrdtool_version'], '1.5.5', '>=') ||
(version_compare($config['rrdtool_version'], '1.5', '>=') && $command != "tune") ||
($command != "create" && $command != "tune"))
) {
if (isset($config['rrdcached_dir']) && $config['rrdcached_dir'] !== false) {
$filename = str_replace($config['rrd_dir'].'/', './'.$config['rrdcached_dir'].'/', $filename);
$filename = str_replace($config['rrd_dir'], './'.$config['rrdcached_dir'].'/', $filename);
@@ -294,3 +298,20 @@ function rrdtool_escape($string, $maxlength=null){
return $result.' ';
}
function rrdtool_tune($type, $filename, $max) {
$fields = array();
if ($type === 'port') {
if ($max < 10000000) {
return false;
}
$max = $max / 8;
$fields = array(
'INOCTETS','OUTOCTETS','INERRORS','OUTERRORS','INUCASTPKTS','OUTUCASTPKTS','INNUCASTPKTS','OUTNUCASTPKTS','INDISCARDS','OUTDISCARDS','INUNKNOWNPROTOS','INBROADCASTPKTS','OUTBROADCASTPKTS','INMULTICASTPKTS','OUTMULTICASTPKTS'
);
}
if (count($fields) > 0) {
$options = "--maximum " . implode(":$max --maximum ", $fields). ":$max";
rrdtool('tune', $filename, $options);
}
}

32
scripts/tune_port.php Executable file
View File

@@ -0,0 +1,32 @@
#!/usr/bin/env php
<?php
require 'includes/defaults.inc.php';
require 'config.php';
require 'includes/definitions.inc.php';
require 'includes/functions.php';
rrdtool_pipe_open($rrd_process, $rrd_pipes);
$options = getopt('h:p:');
$hosts = str_replace('*', '%', mres($options['h']));
$ports = str_replace('*', '%', mres($options['p']));
if (empty($hosts) && empty($ports)) {
echo "-h <device hostname wildcard> Device(s) to match\n";
echo "-p <ifName widcard> Port(s) to match using ifName\n";
echo "\n";
}
foreach (dbFetchRows("SELECT `device_id`,`hostname` FROM `devices` WHERE `hostname` LIKE ?", array('%'.$hosts.'%')) as $device) {
echo "Found hostname " . $device['hostname'].".......\n";
foreach (dbFetchRows("SELECT `ifIndex`,`ifName`,`ifSpeed` FROM `ports` WHERE `ifName` LIKE ? AND `device_id` = ?", array('%'.$ports.'%',$device['device_id'])) as $port) {
echo "Tuning port " . $port['ifName'].".......\n";
$host_rrd = $config['rrd_dir'].'/'.$device['hostname'];
$rrdfile = $host_rrd.'/port-'.safename($port['ifIndex'].'.rrd');
rrdtool_tune('port',$rrdfile,$port['ifSpeed']);
}
}

1
sql-schema/079.sql Normal file
View File

@@ -0,0 +1 @@
INSERT INTO config (config_name,config_value,config_default,config_descr,config_group,config_group_order,config_sub_group,config_sub_group_order,config_hidden,config_disabled) values ('rrdtool','/usr/bin/rrdtool','/usr/bin/rrdtool','Path to rrdtool','external',0,'rrdtool',0,'0','0'), ('rrdtool_tune','false','false','Auto tune maximum value for rrd port files','external',0,'rrdtool',0,'0','0');