Files
librenms-librenms/doc/Developing/Application-Notes.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

92 lines
2.9 KiB
Markdown
Raw Normal View History

add the ability for storing app data to prevent spamming of the event log via via component usage (#14087) * initial work on add the ability to save/fetch app data * update to use get_app_data for ZFS * update the poller for the new app_data stuff * ZFS now logs changes to pools * add schema update for app_data stuff * small formatting fix * add a missing \ * now adds a column * sql-schema is no longer used, so remove the file that was added here * misc cleanups * rename the method in database/migrations/2022_07_03_1947_add_app_data.php * hopefully fix the migration bit * add the column to misc/db_schema.yaml * more misc small DB fixes * update the test as the json column uses collat of utf8mb4_bin * revert the last change and try manually setting it to what is expected * remove a extra ; * update suricata as well * correct the instance -> instances in one location to prevent the old instance list from being stomped * remove a extra ; * update fail2ban to use it as well * remove two unused functions as suricata and fail2ban no longer use components * style cleanup * postgres poller updated to use it * update html side of the postgres bits * chronyd now uses app data bits now as well * portactivity now uses it as well * style fix * sort the returned arrays from app_data * correct log message for port activity * collocation change * try re-ordering it * add in the new data column to the tests * remove a extra , * hmm... ->collate('utf8mb4_unicode_ci') is not usable as apparently collate does not exist * change the column type from json to longtext * mv chronyd stuff while I sort out the rest of the tests... damn thing is always buggy * hmm... fix a missing line then likely move stuff back * style fix * add fillable * add the expexcted data for fail2ban json * escape a " I missed * add data for portactivity * add suricata app data * add app data to zfs legacy test * put the moved tests back into place and update zfs-v1 test * add app data for chronyd test * add app data for fail2ban legacy test * update zfs v1 app data * add some notes on application dev work * add Developing/Application-Notes.md to mkdocs.yml * add data column to it * added various suggestions from bennet-esyoil * convert from isset to sizeof * type fix * fully remove the old save app data function and move it into a helper function... the other still needs cleaned up prior to removal * update docs * get_app_data is fully removed now as well * a few style fixes * add $casts * update chronyd test * attempt to fix the data * more doc cleanup and try changing the cast * style fix * revert the changes to the chronyd test * apply a few of murrant's suggestions * document working with ->data as json and non-josn * remove two no-longer used in this PR exceptions * ->data now operates transparently * style fix * update data tests * fix json * test fix * update the app notes to reflect how app data now works * app test fix * app data fix for linux_lsi * json fix * minor doc cleanup * remove duplicate querty and use json_decode instead * style fix * modelize the app poller * use a anon func instead of foreach * test update * style cleanup * style cleanup * another test cleanup * more test cleanup * reverse the test changes and add in some more glue code * revert one of the test changes * another small test fix * Make things use models Left some array access, but those will still work just fine. * missed chronyd and portactivity * rename poll to avoid make it any confusion * Remove extra save and fix timestamp * save any changes made to app->data * nope, that was not it * What are magic methods and how do they work? * fix two typos * update linux_lsi test * change quote type Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-07-22 16:01:55 -05:00
# Notes On Application Development
## LibreNMS JSON SNMP Extends
The polling function `json_app_get` makes it easy to poll complex data
using SNMP extends and JSON.
The following exceptions are provided by it.
It takes three parameters, in order in the list below.
- Integer :: Device ID to fetch it for.
- String :: The extend name. For example, if 'zfs' is passed it will
be converted to 'nsExtendOutputFull.3.122.102.115'.
- Integer :: Minimum expected version of the JSON return.
The required keys for the returned JSON are as below.
- version :: The version of the snmp extend script. Should be numeric
and at least 1.
- error :: Error code from the snmp extend script. Should be > 0
(0 will be ignored and negatives are reserved)
- errorString :: Text to describe the error.
- data :: An key with an array with the data to be used.
The supported exceptions are as below.
- JsonAppPollingFailedException :: Empty return from SNMP.
- JsonAppParsingFailedException :: Could not parse the JSON
- JsonAppBlankJsonException :: Blank JSON.
- JsonAppMissingKeysException :: Missing required keys.
- JsonAppWrongVersionException :: Older version than supported.
- JsonAppExtendErroredException :: Polling and parsing was good, but
the returned data has an error set. This may be checked via
$e->getParsedJson() and then checking the keys error and
errorString.
The error value can be accessed via $e->getCode(). The output can be
accessed via $->getOutput() Only returned
JsonAppParsingFailedException. The parsed JSON can be access via
$e->getParsedJson().
An example below from `includes/polling/applications/zfs.inc.php`...
```php
try {
$zfs = json_app_get($device, $name, 1)['data'];
} catch (JsonAppMissingKeysException $e) {
//old version with out the data key
$zfs = $e->getParsedJson();
} catch (JsonAppException $e) {
echo PHP_EOL . $name . ':' . $e->getCode() . ':' . $e->getMessage() . PHP_EOL;
update_application($app, $e->getCode() . ':' . $e->getMessage(), []);
return;
}
```
### Compression
Also worth noting that `json_app_get` supports compressed data via
base64 encoded gzip. If base64 encoding is detected on the the SNMP
return, it will be gunzipped and then parsed.
`https://github.com/librenms/librenms-agent/blob/master/utils/librenms_return_optimizer`
may be used to optimize JSON returns.
add the ability for storing app data to prevent spamming of the event log via via component usage (#14087) * initial work on add the ability to save/fetch app data * update to use get_app_data for ZFS * update the poller for the new app_data stuff * ZFS now logs changes to pools * add schema update for app_data stuff * small formatting fix * add a missing \ * now adds a column * sql-schema is no longer used, so remove the file that was added here * misc cleanups * rename the method in database/migrations/2022_07_03_1947_add_app_data.php * hopefully fix the migration bit * add the column to misc/db_schema.yaml * more misc small DB fixes * update the test as the json column uses collat of utf8mb4_bin * revert the last change and try manually setting it to what is expected * remove a extra ; * update suricata as well * correct the instance -> instances in one location to prevent the old instance list from being stomped * remove a extra ; * update fail2ban to use it as well * remove two unused functions as suricata and fail2ban no longer use components * style cleanup * postgres poller updated to use it * update html side of the postgres bits * chronyd now uses app data bits now as well * portactivity now uses it as well * style fix * sort the returned arrays from app_data * correct log message for port activity * collocation change * try re-ordering it * add in the new data column to the tests * remove a extra , * hmm... ->collate('utf8mb4_unicode_ci') is not usable as apparently collate does not exist * change the column type from json to longtext * mv chronyd stuff while I sort out the rest of the tests... damn thing is always buggy * hmm... fix a missing line then likely move stuff back * style fix * add fillable * add the expexcted data for fail2ban json * escape a " I missed * add data for portactivity * add suricata app data * add app data to zfs legacy test * put the moved tests back into place and update zfs-v1 test * add app data for chronyd test * add app data for fail2ban legacy test * update zfs v1 app data * add some notes on application dev work * add Developing/Application-Notes.md to mkdocs.yml * add data column to it * added various suggestions from bennet-esyoil * convert from isset to sizeof * type fix * fully remove the old save app data function and move it into a helper function... the other still needs cleaned up prior to removal * update docs * get_app_data is fully removed now as well * a few style fixes * add $casts * update chronyd test * attempt to fix the data * more doc cleanup and try changing the cast * style fix * revert the changes to the chronyd test * apply a few of murrant's suggestions * document working with ->data as json and non-josn * remove two no-longer used in this PR exceptions * ->data now operates transparently * style fix * update data tests * fix json * test fix * update the app notes to reflect how app data now works * app test fix * app data fix for linux_lsi * json fix * minor doc cleanup * remove duplicate querty and use json_decode instead * style fix * modelize the app poller * use a anon func instead of foreach * test update * style cleanup * style cleanup * another test cleanup * more test cleanup * reverse the test changes and add in some more glue code * revert one of the test changes * another small test fix * Make things use models Left some array access, but those will still work just fine. * missed chronyd and portactivity * rename poll to avoid make it any confusion * Remove extra save and fix timestamp * save any changes made to app->data * nope, that was not it * What are magic methods and how do they work? * fix two typos * update linux_lsi test * change quote type Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-07-22 16:01:55 -05:00
## Application Data Storage
The `$app` model is supplied for each application poller and graph.
You may access and update the `$app->data` field to store arrays of data
the Application model.
When you call update_application() the `$app` model will be saved along with
any changes to the data field.
```
// set the varaible data to $foo
$app->data = [
'item_A' => 123,
'item_B' => 4.5,
'type' => 'foo',
'other_items' => [ 'a', 'b', 'c' ],
];
// save the change
$app->save();
// var_dump the contents of the variable
var_dump($app->data);
```