Files
librenms-librenms/scripts/json-app-tool.php

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

220 lines
6.7 KiB
PHP
Raw Permalink Normal View History

#!/usr/bin/env php
<?php
require realpath(__DIR__ . '/..') . '/includes/init.php';
// Pulled from includes/polling/functions.inc.php
function data_flatten($array, $prefix = '', $joiner = '_')
{
$return = [];
foreach ($array as $key => $value) {
if (is_array($value)) {
if (strcmp($prefix, '')) {
$key = $prefix . $joiner . $key;
}
$return = array_merge($return, data_flatten($value, $key, $joiner));
} else {
if (strcmp($prefix, '')) {
$key = $prefix . $joiner . $key;
}
$return[$key] = $value;
}
}
return $return;
}
// Pulled from includes/snmp.inc.php
function string_to_oid($string)
{
$oid = strlen($string);
for ($i = 0; $i != strlen($string); $i++) {
$oid .= '.' . ord($string[$i]);
}
return $oid;
}//end string_to_oid()
// Options!
$short_opts = 'S:sktmlhj:a:d:';
$options = getopt($short_opts);
// print the help
if (isset($options['h'])) {
echo 'LibreNMS JSON App tool
-j The file containing the JSON to use for the test.
-s Print the SNMPrec data.
-t Print the JSON test data file.
-l Just load and lint the JSON file.
-m Extract and print metric variables from the JSON file.
-k If m is specified, just print the keys in tested order.
-a The application name for use with -s and -t.
-S SNMP extend name. Defaults to the same as -a.
-d JSON file to use for app data.
-h Show this help text.
-j must always be specified.
-a must always be given with -s and -t.
-l if specified will override any others. If none of the others are
specified and just -j is given, then the file is loaded and then
linted if needed. For linting jsonlint needs to be installed.
-m is handy if you want to grab a list of metrics a JSON app is returning
and how they data is flattned as well for when writing alert rules. -k
prints it in a slightly neater manner and in a manner and in tested
order.
';
exit;
}
// make sure we have a JSON file to work with
if (! isset($options['j'])) {
echo "Nothing JSON file specified via -j.\n";
exit(1);
}
//read in the file
$raw_json = file_get_contents($options['j']);
if ($raw_json === false) {
exit(2);
}
// parse the read file
$json = json_decode(stripslashes($raw_json), true);
// check json_decode() for any errors
if (json_last_error() !== JSON_ERROR_NONE) {
echo "Parsing '" . $options['j'] . "' failed. Running jsonlint...\n\n";
system('jsonlint ' . escapeshellarg($options['j']));
exit(3);
}
//make sure the JSON actually contains something
if (empty($json)) {
echo "'" . $options['j'] . "' is a blank JSON file.\n";
exit(4);
}
//make sure it has all the required keys
if (! isset($json['error'], $json['data'], $json['errorString'], $json['version'])) {
echo "'" . $options['j'] . "' is missing one or more of the keys 'error', 'errorString', 'version', or 'data'.\n";
exit(5);
}
//successfully loaded and tested the file, just exit now if asked to
if ((isset($options['l'])) || (
(! isset($options['t'])) &&
(! isset($options['s'])) &&
(! isset($options['m']))
)) {
exit(0);
}
//pulls out the metrics
$data = $json['data'];
$metrics = data_flatten($data);
$metrics_keys = array_keys($metrics);
usort($metrics_keys, 'strcasecmp'); //orders them in the manner in which the test script compares them
//print metrics if needed
if (isset($options['m'])) {
if (isset($options['k'])) {
foreach ($metrics_keys as $key) {
echo $key . "\n";
}
} else {
foreach ($metrics_keys as $key) {
echo $key . '=' . $metrics[$key] . "\n";
}
}
exit(0);
}
// exit if -s or -t is not requested
if ((! isset($options['s'])) && (! isset($options['t']))) {
exit(0);
}
// For anything past here, we need -a given
if (! isset($options['a'])) {
echo "Nothing specified via -a\n";
exit(1);
}
// -S defaults to -a if not set
if (! isset($options['S'])) {
$options['S'] = $options['a'];
}
// Output snmprec data for snmpsim for use with testing.
if (isset($options['s'])) {
$oid = \LibreNMS\Util\Oid::ofString($options['S']);
echo "1.3.6.1.2.1.1.1.0|4|Linux server 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64\n" .
"1.3.6.1.2.1.1.2.0|6|1.3.6.1.4.1.8072.3.2.10\n" .
"1.3.6.1.2.1.1.3.0|67|77550514\n" .
"1.3.6.1.2.1.1.4.0|4|<private>\n" .
"1.3.6.1.2.1.1.5.0|4|<private>\n" .
"1.3.6.1.2.1.1.6.0|4|<private>\n" .
"1.3.6.1.2.1.25.1.1.0|67|77552962\n" .
"1.3.6.1.4.1.8072.1.3.2.2.1.21.6.100.105.115.116.114.111|2|1\n" .
'1.3.6.1.4.1.8072.1.3.2.2.1.21.' . $oid . "|2|1\n" .
'1.3.6.1.4.1.8072.1.3.2.3.1.2.' . $oid . '|4x|' . bin2hex($raw_json) . "\n";
exit(0);
}
// prints the json test data file if asked to
if (isset($options['t'])) {
$test_data = [
'applications' => [
'discovery' => [
'applications' => [[
'app_type' => $options['a'],
'app_state' => 'UNKNOWN',
'discovered' => '1',
'app_state_prev' => null,
'app_status' => '',
'app_instance' => '',
add the ability for storing app data to prevent spamming of the event log via via component usage (#14087) * initial work on add the ability to save/fetch app data * update to use get_app_data for ZFS * update the poller for the new app_data stuff * ZFS now logs changes to pools * add schema update for app_data stuff * small formatting fix * add a missing \ * now adds a column * sql-schema is no longer used, so remove the file that was added here * misc cleanups * rename the method in database/migrations/2022_07_03_1947_add_app_data.php * hopefully fix the migration bit * add the column to misc/db_schema.yaml * more misc small DB fixes * update the test as the json column uses collat of utf8mb4_bin * revert the last change and try manually setting it to what is expected * remove a extra ; * update suricata as well * correct the instance -> instances in one location to prevent the old instance list from being stomped * remove a extra ; * update fail2ban to use it as well * remove two unused functions as suricata and fail2ban no longer use components * style cleanup * postgres poller updated to use it * update html side of the postgres bits * chronyd now uses app data bits now as well * portactivity now uses it as well * style fix * sort the returned arrays from app_data * correct log message for port activity * collocation change * try re-ordering it * add in the new data column to the tests * remove a extra , * hmm... ->collate('utf8mb4_unicode_ci') is not usable as apparently collate does not exist * change the column type from json to longtext * mv chronyd stuff while I sort out the rest of the tests... damn thing is always buggy * hmm... fix a missing line then likely move stuff back * style fix * add fillable * add the expexcted data for fail2ban json * escape a " I missed * add data for portactivity * add suricata app data * add app data to zfs legacy test * put the moved tests back into place and update zfs-v1 test * add app data for chronyd test * add app data for fail2ban legacy test * update zfs v1 app data * add some notes on application dev work * add Developing/Application-Notes.md to mkdocs.yml * add data column to it * added various suggestions from bennet-esyoil * convert from isset to sizeof * type fix * fully remove the old save app data function and move it into a helper function... the other still needs cleaned up prior to removal * update docs * get_app_data is fully removed now as well * a few style fixes * add $casts * update chronyd test * attempt to fix the data * more doc cleanup and try changing the cast * style fix * revert the changes to the chronyd test * apply a few of murrant's suggestions * document working with ->data as json and non-josn * remove two no-longer used in this PR exceptions * ->data now operates transparently * style fix * update data tests * fix json * test fix * update the app notes to reflect how app data now works * app test fix * app data fix for linux_lsi * json fix * minor doc cleanup * remove duplicate querty and use json_decode instead * style fix * modelize the app poller * use a anon func instead of foreach * test update * style cleanup * style cleanup * another test cleanup * more test cleanup * reverse the test changes and add in some more glue code * revert one of the test changes * another small test fix * Make things use models Left some array access, but those will still work just fine. * missed chronyd and portactivity * rename poll to avoid make it any confusion * Remove extra save and fix timestamp * save any changes made to app->data * nope, that was not it * What are magic methods and how do they work? * fix two typos * update linux_lsi test * change quote type Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-07-22 16:01:55 -05:00
'data' => null,
]],
],
'poller' => [
'applications' => [[
'app_type' => $options['a'],
'app_state' => 'OK',
'discovered' => '1',
'app_state_prev' => 'UNKNOWN',
'app_status' => '',
'app_instance' => '',
add the ability for storing app data to prevent spamming of the event log via via component usage (#14087) * initial work on add the ability to save/fetch app data * update to use get_app_data for ZFS * update the poller for the new app_data stuff * ZFS now logs changes to pools * add schema update for app_data stuff * small formatting fix * add a missing \ * now adds a column * sql-schema is no longer used, so remove the file that was added here * misc cleanups * rename the method in database/migrations/2022_07_03_1947_add_app_data.php * hopefully fix the migration bit * add the column to misc/db_schema.yaml * more misc small DB fixes * update the test as the json column uses collat of utf8mb4_bin * revert the last change and try manually setting it to what is expected * remove a extra ; * update suricata as well * correct the instance -> instances in one location to prevent the old instance list from being stomped * remove a extra ; * update fail2ban to use it as well * remove two unused functions as suricata and fail2ban no longer use components * style cleanup * postgres poller updated to use it * update html side of the postgres bits * chronyd now uses app data bits now as well * portactivity now uses it as well * style fix * sort the returned arrays from app_data * correct log message for port activity * collocation change * try re-ordering it * add in the new data column to the tests * remove a extra , * hmm... ->collate('utf8mb4_unicode_ci') is not usable as apparently collate does not exist * change the column type from json to longtext * mv chronyd stuff while I sort out the rest of the tests... damn thing is always buggy * hmm... fix a missing line then likely move stuff back * style fix * add fillable * add the expexcted data for fail2ban json * escape a " I missed * add data for portactivity * add suricata app data * add app data to zfs legacy test * put the moved tests back into place and update zfs-v1 test * add app data for chronyd test * add app data for fail2ban legacy test * update zfs v1 app data * add some notes on application dev work * add Developing/Application-Notes.md to mkdocs.yml * add data column to it * added various suggestions from bennet-esyoil * convert from isset to sizeof * type fix * fully remove the old save app data function and move it into a helper function... the other still needs cleaned up prior to removal * update docs * get_app_data is fully removed now as well * a few style fixes * add $casts * update chronyd test * attempt to fix the data * more doc cleanup and try changing the cast * style fix * revert the changes to the chronyd test * apply a few of murrant's suggestions * document working with ->data as json and non-josn * remove two no-longer used in this PR exceptions * ->data now operates transparently * style fix * update data tests * fix json * test fix * update the app notes to reflect how app data now works * app test fix * app data fix for linux_lsi * json fix * minor doc cleanup * remove duplicate querty and use json_decode instead * style fix * modelize the app poller * use a anon func instead of foreach * test update * style cleanup * style cleanup * another test cleanup * more test cleanup * reverse the test changes and add in some more glue code * revert one of the test changes * another small test fix * Make things use models Left some array access, but those will still work just fine. * missed chronyd and portactivity * rename poll to avoid make it any confusion * Remove extra save and fix timestamp * save any changes made to app->data * nope, that was not it * What are magic methods and how do they work? * fix two typos * update linux_lsi test * change quote type Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-07-22 16:01:55 -05:00
'data' => null,
]],
'application_metrics' => [],
],
],
];
foreach ($metrics_keys as $key) {
$test_data['applications']['poller']['application_metrics'][] = [
'metric' => $key,
'value' => $metrics[$key],
'value_prev' => null,
'app_type' => $options['a'],
];
}
// if d is specified, try to read it in and add it
if (isset($options['d'])) {
$raw_app_data = file_get_contents($options['d']);
if ($raw_json === false) {
exit(2);
}
$app_data = json_decode(stripslashes($raw_app_data), true);
if (json_last_error() !== JSON_ERROR_NONE) {
echo "Parsing '" . $options['d'] . "' failed. Running jsonlint...\n\n";
system('jsonlint ' . escapeshellarg($options['j']));
exit(3);
}
$test_data['applications']['poller']['applications']['0']['data'] = json_encode($app_data);
}
echo json_encode($test_data, JSON_PRETTY_PRINT) . "\n";
exit(0);
}