Commit Graph

133 Commits

Author SHA1 Message Date
Tony Murray
f2f169ae78 Modernize mempools (#12282)
* mempools to modern module
quick hacky hrstorage port

* port ucd-snmp-mib to Mempools

* Populate DB for ucd
Prep for yaml

* initial yaml attempt

* more complex conversions
fixes to YamlDiscovery, make leading $ optional and allow mib::oid format

* walk full tables and skip values
normalize percentages above 100

* handle precent only ones (specify total as 100)

* Move default polling out of YamlMempoolsDiscovery

* fixes

* Update test data hrstorage should be correct.

* perc_warn for hrstorage

* Host Resources, record buffer, cached, and shared

* Host Resources is always better, don't do both HR and UCD

* fix unix, include warning levels

* variable size

* consolidate skip_values

* define mempools schema

* number instead of integer

* more schema refactor

* one more skip_values reference

* throw error for invalid oid translation
aos6

* a*  and Cisco

* updated test data

* update almost all hrstorage data files

* b*

* c* with test data
use standard cache for hrStorage

* use cache for storage module too

* hand bsnmp properly

* bdcom

* exclude total oid from yaml so it is not polled
May add a way to ignore this behavior and poll it, but I don't know if that is needed.

* automatically handle percent only values

* ciscowlc

* only poll used or free if we have used, free, and total.

* fix skipping

* the dlinkoning
fix find value when value "name" is numeric

* support numeric oids

* dnos/ftos attempt

* I guess we can't filter on total > 0

* edgecos

* e*

* f WIP

* f*

* gwd (aka g*)

* h* + procurve

* i*

* j*

* m*

* support 0% used memory (however unlikely)

* n*

* CISCO-PROCESS-MIB memory, share cache with processors module

* ignore mempools with invalid total

* p*

* quanta

* r*
fix raisecom mibs terribly broken

* s-z

* style fixes

* Move VRP back to PHP and make it actually work

* fix zynos

* update schema

* Update Cisco processor data for description bug fixes

* fix comware processors

* comware mempools with memory size
default precision to 1

* sophos-xg updated data

* hrstorage use ram size for buffers, cache, and shared

* Show memory available instead of free in device overview

* UCD, use same rrd format, store available instead of free in the db.

* Calculate availability for HOST-RESOURCES-MIB

* Convert UCD to standard polling

* rename old rrd files

* initial graph work

* graph WIP

* Graph looking decent

* Graph looking decent for hr

* remove old ucd_graph code

* handle availability mempool
more graph cleanup

* color adjustments

* remove accidental free calculation

* Update test data and fix corner cases

* fis pfsense

* update schema

* add default value for mempool_class

* fix whitespace

* update schema

* update schema correctly

* one more time

* fortigate_1500d-sensors missing oids

* Update docs.

* fix indent

* add implements MempoolsDiscovery explicitly to OS

* remove ucd_memory graph references
remove unused device_memory graph

* remove unused functions

* set devices with mempools to rediscover to prevent/minimize gaps

* use a subquery

* add overview graph

* port health mempools table

* Update device mempool

* only show overview if multiple

* Don't override user set warn percentages in discovery

* fix missed usage

* fix style

* Safety check to not rename rrd files incorrectly if migration has not been run.

* Fix overview percent bar and represent available and used on the bar

* missed an item to convert to mempool_class

* percent on the wrong side
2020-11-23 15:35:35 -06:00
Jellyfrog
cfd9dce620 Convert Virtual Machine pages to Laravel (#12287)
* Convert Virtual Machine pages to Laravel

* wip

* wip

* wip

* wip

* wip

* wip

* delete

* wip

* wip

* move powerStateLabel
2020-11-11 01:15:20 +01:00
Hayden
e34b6877fd Add OSPF cost (TOS) (#11929)
* add OspfTos model to include ospfIfMetricEntry data (ospf interface cost)

* remove WIP comment

* add new model and db migration

* update OspfTos relationship to OspfPort and correct cleanup call

* add "Cost" to device\routing\ospf web UI interface

* updated db-schema test unit

* updated db_schema to resolve conflicts

* removed tos metrics from rrd ospf stats due to rrdcached errors

* reorder db_schema.yaml to match upstream

* readd ospf_tos to db_schema

* styleCI changes

* Capture OSPF test data

* add ospf unit tests for iosxr

* ospfAuthType

* update db_schema

* remove extra migrations

* add ospf tests for iosxe

* add ospf test to ios

* merge ospf_tos into ospf_ports

* update db_schema

* update ospf module unit tests for ios, iosxe, iosxr

* fix ospf_ports ospfIfMetricStatus column type

* update db_schema

* more efficient dropColumn in down() migration

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-07 09:27:25 -06:00
Hans Erasmus
6c4596d1b9 Added new authalgo support for SNMPv3 (#11966)
* Added new authalgo support for SNMPv3

* Update 2020_07_27_00522_alter_authalgo_column.php

Fixed case

* Update addhost.inc.php

Fixed case

* Update snmp.inc.php

Fixed case

* Update db_schema.yaml

Fixed case

* Fixed SHA-* order to ascending

* Update 2020_07_27_00522_alter_authalgo_column.php

Fixed space

* Update db_schema.yaml

Fixed line to be of type string

* Update config_definitions.json

Fixed sorting

* Fixed test file case sensitivity

* Fixed rollback function

* SourceDoctor code for checks

* Fixed indentation issues on Travis test

* Update to re-trigger testing stuck on CLA step

* Added crypto-algo updates

* Incorporated suggestions from Jellyfrog

* Added documentation entry

* Fixed travis spacing error

* Fixed validations

* Delete snmpv3_version_compare.patch

Sorry, this one was not supposed to be there.

* Added poller evaluation

* Added poller comparison

* Fixed merge-conflicts

* Fixed typo in test unit

* code refinements
only can check on webui that it supports the algorithms, hopefully, the eventlog is enough to notify users of distributed pollers.

* style fixes and remove openssl setting

* fix device access

* fix missing migration column

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-10-29 13:02:26 -05:00
Hayden
774d95c118 allow nullable ospf auth (#12249)
* ospfAuthType

* update db_schema

* remove extra migrations
2020-10-21 21:07:12 -05:00
Richard Kojedzinszky
2ca6f2f8d4 schema: add missing primary keys (#12106)
* schema: add missing primary keys

* Exclude ipv4_mac.id

* Made primary key migrations SQLite compatible

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-10-14 20:37:40 -05:00
Tony Murray
1e4702fa4f Support multiple daily process locking backends with distributed polling (#11896)
* Implement locks in the file cache

* Replace custom locks

* implement restore lock
Used when re-hydrating

* remove legacy use statements

* Add class descriptions

* Fix style

* Default to database cache driver

* missed cache_locks table
prevent chicken-egg issue

* style fixes

* Remove custom file lock implementation

* missed items from file cache

* Update schema definition
hmm, other schema noise must be from manual modification as this is generated from a freshly migrated db.

* require predis, it is pure python, so no harm in adding

* and set predis as the default client
2020-10-07 07:36:35 -05:00
Negatifff
444b9544a0 new index in alert_log table (#12143)
* new index in alert_log table

* codefix1

* codefix2

* added second index

* codefix

* Apply suggestions from code review

Co-authored-by: Jellyfrog <Jellyfrog@users.noreply.github.com>
2020-09-24 17:05:21 +02:00
SourceDoctor
427878dcaa Availability Calculation for all Devices (#12004)
* Availability Calculation for all Devices

* remove not more need uptime column
2020-08-28 13:01:54 +02:00
Jellyfrog
cf13e92f05 Add missing index for device_perf (#11974)
Makes the latency page load faster in case of a large device_perf table
2020-07-31 16:07:00 +02:00
Tony Murray
020c5fd7e1 Remove legacy code and fix missing device graphs (#11950)
* removing $graphs global

* remove unused things

* fix some additional graphs

* Fix graphs persisting too soon

* correct name for poller module performance graph

* only one type of graph is used here
2020-07-23 09:57:22 -05:00
SourceDoctor
e2e082af10 . 2020-06-23 19:42:39 +02:00
SourceDoctor
8f115bdd01 . 2020-06-23 19:16:58 +02:00
SourceDoctor
3f2c9386bf partial double 2020-06-23 16:51:31 +02:00
SourceDoctor
bfcec4dc16 change column type to float 2020-06-23 16:45:08 +02:00
SourceDoctor
ab80f7697b . 2020-06-23 02:21:16 +02:00
SourceDoctor
cf7f45fe53 . 2020-06-23 01:30:09 +02:00
SourceDoctor
afa26d66b5 Fix datatype for Availability_perc Column 2020-06-23 01:06:49 +02:00
SourceDoctor
4e6a7291a2 Device Availability Calculation (#11784)
* Device Availability Calculation

* Travis fix

* .

* schema corrections

* flexible duration

* travis

* .

* .

* .

* .

* remove not needed code

* line Text to RRD

* update humantime

* .

* only set up again if device was marked as down

* set RRD area transparency

* save uptime also, to keep last availability as good as possible

* file description correction

* look for outages even if uptime > duration
2020-06-22 22:57:30 +02:00
Tony Murray
867aa132c1 Merge pull request #11661 from SourceDoctor/app_dhcp_enhancement
Application DHCP Upgrade
2020-06-16 17:03:45 -05:00
Tony Murray
13ade9ab6c Merge branch 'master' into alerts-carbon 2020-06-12 07:48:51 -05:00
SourceDoctor
13ffc702ba . 2020-06-09 21:45:25 +02:00
SourceDoctor
1093315700 Merge branch 'master' into app_dhcp_enhancement 2020-06-08 20:51:41 +02:00
Tony Murray
300645388f Dispatcher Service settings (#11760)
* Poller settings WIP

* Poller settings WIP2

* working on SettingMultiple

* setting multiple working

* settings sent with all required info

* fix translation

* Fix keys

* fix groups setting

* Apply settings to service
fixes and validations for setting

* don't error when no poller_cluster entry exists

* hid tab when no poller cluster entries

* Authorization

* make prod

* daily maintenance toggle should be advanced

* Update schema def
2020-06-08 08:27:03 -05:00
arrmo
67cb7b4dc3 Remove int width from db schema (MySQL 8) (#11725)
* Remove width from db schema

* Update to address DEFAULT_GENERATED validation, and push new schema

* Only remove width from integer types

* Build from fresh db

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-05-30 17:06:53 -05:00
SourceDoctor
97e2ab4c04 Merge branch 'master' into app_dhcp_enhancement 2020-05-23 00:12:14 +02:00
Tony Murray
5fc3fcb9e8 Make migrations work in SQLite (#11643)
* WIP Sqlite

Down methods don't work either, avoid them with refresh for now.

WIP persistent support

WIP db_schema

WIP db_schema 2

Update new migrations...

revert dump_db_schema changes for now, too much difference.

fix migrations on mysql

fix up some more items, this should be our target schema
lots of index renames and a two misc changes

index rename WIP

index rename WIP

another round

fix up new schema changes

try case insensitive fix

Trying tests WIP

fix down methods

DBSetupTest working (uses mysql)

Test sqlite migrations work

* Properly validate sqlite output

* revert glue changes, should be separate PR

* remove dusk workaround

* remove unused variables

* import

* sqlite capitalization

* Revert some refresh tests

* testing_mysql back to testing
2020-05-22 16:49:21 -05:00
SourceDoctor
cbe572f959 Application DHCP Upgrade 2020-05-22 23:02:19 +02:00
Mathieu Poussin
fc391090e1 BGP Polling: Add error code management (#11424)
* BGP Polling: Add error code polling

* Rework describe_bgp_error_code and fix bgp error fields migration

* Add real test data for IOS-XR and Arista EOS
2020-04-24 22:47:20 -05:00
Vitali Kari
ce4ad2a09f Fix for Nokia 7705 SAR (#11021)
* Fix for Nokia 7705 SAR

* fix copy and paste typo

* Update 2020_01_10_075852_alter_mpls_lsp_paths_table.php

just trigger tests

* Update 2020_01_10_075852_alter_mpls_lsp_paths_table.php

* Update 2020_01_10_075852_alter_mpls_lsp_paths_table.php

Co-authored-by: Neil Lathwood <gh+n@laf.io>
2020-04-12 01:08:15 +02:00
Tony Murray
eac7838de9 we only need start end... we already have start/end 2020-04-10 12:32:54 -05:00
Tony Murray
6347687d54 Convert existing schedules to UTC 2020-04-10 11:25:21 -05:00
SourceDoctor
7dfb4ef1df Adding Column ifSpeed_prev, ifHighSpeed_prev to Ports Table (#11348)
* adding Column ifSpeed_prev, ifHighSpeed_prev to Ports table

* grammar fix

* rename migration class

* tests

* .

* .
2020-04-01 10:50:33 +02:00
Richard Kojedzinszky
676a120510 Database: add primary key to device_graphs (#11331)
* Database: add primary key to device_graphs

Percona Xtradb refused to INSERT IGNORE into a table
without a primary key.

* Update 2020_03_24_0844_add_primary_key_to_device_graphs.php

* Update db_schema.yaml

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-03-24 12:07:10 -05:00
SourceDoctor
5bfc44f812 Location based Alert Rule (#11128)
* Location base Alert Rule

* travis fix
2020-02-12 12:53:26 -06:00
Joseph Tingiris
6285caba52 Hide devices added for devices that existed before #11104 (#11107)
* Migrate devices.inserted to allow null values and set existng row values to null

* Hide Device Added for devices that existed before https://github.com/librenms/librenms/pull/11104

* Don't display any date that stats with a zero

* make schema match migration
2020-02-06 05:44:04 -06:00
Joseph Tingiris
79bdbe91b5 Show when a device was added and last discovered (#11104)
* Add inserted column to devices table

* Added schema for devices.inserted

* Show when a device was added and last discovered

* Added collection rule for "Device added within the last 60 minutes"
2020-02-05 14:06:43 -06:00
SourceDoctor
7950893cd1 Alternate Poller IP instead of Hostname (#10981)
* optional alternative poller ip

* add missing schema update

* .

* enhance API Call 'add_device' with overwrite_ip key

* .

* .

* updating docs
2020-01-30 06:20:30 -06:00
louis-oui
a384fc2830 Do not update alert timestamp when updating a triggered alert (#10907)
* Do not update alert timestamp when updating a triggered alert

* Only update alerts timestamp if alert is triggered or recovered

* db schema

* db schema

* cleanup timestamp update
2020-01-22 15:01:25 -06:00
louis-oui
e235c59d2f Add alert rule option to invert devices and groups "map to" list (#11038)
* Add alert rule option to invert map to
* Update 2019_12_17_151314_add_invert_map_to_alert_rules.php
* fix invert map to does not work if several groups are set in map to
* Clarify labels
* fix invert map to at rule creation
* clarify invert label
* Fix device in map to - not group
* dummy commit
2020-01-19 14:55:51 +01:00
Kevin Krumm
10b42137e9 Revert "Add alert rule option to invert devices and groups "map to" list (#10954)" (#11036)
This reverts commit 7a8d0b9e09.
2020-01-16 08:42:53 +01:00
louis-oui
7a8d0b9e09 Add alert rule option to invert devices and groups "map to" list (#10954)
* Add alert rule option to invert map to

* Add alert rule option to invert map to

* Update 2019_12_17_151314_add_invert_map_to_alert_rules.php

* fix invert map to does not work if several groups are set in map to

* fix invert map to does not work if several groups are set in map to

* fix invert map to does not work if several groups are set in map to

* Clarify labels

* Clarify labels

* Clarify labels

* Clarify labels

* fix invert map to at rule creation

* clarify invert label
2020-01-15 11:40:01 -06:00
louis-oui
0e3c472018 "Disable alerting" on device disables alert rules check (not just alert transport) (#11015)
* Fix disable notify

* Fix disable notify

* Disable all alerting

* fix db migration

* fix db migration

* fix db migration

* Add autoclearing active alerts if disable alerting set

* Add autoclearing active alerts if disable alerting set

* fix check

* Device in gray if disable_notify is set - not ignore tag
2020-01-09 18:46:52 -06:00
Vitali Kari
532ab1127a Mpls Path Visualization (#10936)
* update nokia mibs

* database migrations

* add code

* html code

* update db schema

* add test data

* add FIXME

* move mibs/nokia/MPLS-TE-MIB to mibs/MPLS-TE-MIB
2020-01-01 23:45:06 +01:00
Jellyfrog
b361710148 Device group based access (#10568)
* Device group based access

* Use Permissions class to resolve permissions

Also give port access based on device access

* Convert more pages to use Permissions class

* shorten config setting name
use Eloquent relationships in several places
alphabetize config_definitions.json

* Change Models and Permissions

* Clean up ajax_search LIMIT sql

* Convert more pages to use Permissions class

Co-authored-by: Tony Murray <murraytony@gmail.com>
2019-12-30 12:11:26 +01:00
louis-oui
934260cc75 Feature: Custom OID polling and graphing (#10945)
* merge

* fix db migration

* fix new auth

* fix new auth

* fix new auth

* fix new auth

* fix db schema tests

* fix polling customoid

* fix polling customoid

* fix graph

* fix graph

* fix graph

* fix CI

* fix CI

* always update prev value

* typo
2019-12-18 18:17:21 -06:00
PipoCanaja
bf181b9dc2 Added support for routing table collection in discovery (#10182)
* Clean broken VRF lite code
* Change DB table for route discovery
* Add VRF simple support
* add port_id to db and discovery
* static-fy the translation arrays
* sort and search cleaning
* Sorting refactor and validation
* formatItem shortened
* Handle ifIndex==0 meaning no next hop defined (MPLS)
* Sync all create/updates
* purge in daily
* remove old route table
* get rid of inetCidrRouteNextHop_device_id
* fix wonky column orders
* add route snmprec
* fix sorting by interface
* Move to new config
* rename to route the new table
* Properly display ipv6 compressed addresses
* Translation before merge ./lnms translation:generate
* Update manifest
2019-11-17 16:30:43 +01:00
SourceDoctor
ce628399a9 Allow user activation/deactivation (MySQL auth) (#10511)
* user deactivation feature

* update db_schema.yaml

* travis fix

* readd sqlfile with alter statement

* ..

* revert force push

* combine all queries

* fix query

* user enable/disable only visible on mysql authorization

* Update form.blade.php

* Update index.blade.php

* disable 'enabled' on own profile

* bootstraping checkboxes
2019-11-08 06:32:57 +00:00
Jellyfrog
8e1a7d75a4 Always sort indexes in dump_db_schema (#10732)
This avoids unnecessary diffs when MySQL outputs in different order
2019-10-22 13:50:41 +00:00
Tony Murray
699aa8a042 Consolidate configuration settings and implement dynamic webui settings (#9809)
* initial db/defaults dump

* numeric values

* Remove $config['time']

* Use config_definitions.json

* try setting definition as a complete array

* a little more

* fix format

* WIP converting dynamic setting pages

* rewriting the webui

* remove legacy and add translations

* finish email section

* improve navigation
update js a bit

* Import the remaining existing settings

* Update backend still some wip
migration

* fix config loading (db not overriding defaults)

* some cleanup

* more array collapsing

* update settings

* Settings search

* add purge settings to ui, order groups and sections

* collapse more arrays

* Auth definitions WIP

* remove needless slash escapes

* add warning to json

* moving settings from defaults.inc

* remove slow_statistics

* Move rrdtool settings to the poller section

* Use translations more consistently

* removing more settings from defaults.inc.php

* show setting name in tooltip

* allow help text to be shown on mobile

* disable settings that are set in config.php

* Implement undo and reset to default.

* Vue.js functional

* Vue.js WIP

* Implement tabs component

* accordion WIP

* lodash ver update

* two items

* Accordion somewhat working

* hash navigation

* Refine Accordion

* Fix up tab styling a bit

* Cleaner tab selected property
Hide html while loading

* spinner?

* Icon support
property for accordion to active

* WIP

* Tabs/Sections from ajax

* Setting Component skeletons

* Dynamic Component resolution

* Basic functionality

* toggle class

* Refactor components

* translate tabs and accordions

* simple array attempt

* improve readonly tooltip

* array styling

* array value editing

* organize snmp info

* Handle initial tab/section in url

* Use Laravel to parse tab/section, dump old

* Draggable array entries

* v-tooltip, for clickable (and touch) help tooltips
disable draggable

* Navigation WIP

* Navigation WIP

* groups computed

* filter settings

* fix event reference

* vue.js i18n initial

* missing description = just setting name

* en fallback

* tidy up the language support and js generation

* persist value to db

* fix issue with 0

* Delete settings from DB instead of setting them to default

* ldap-groups
fixup style

* Default dashboard selection

* fix array of options instead of object

* allow custom validation for settings

* translate options in SettingSelect

* SNMP v3 WIP

* fix setting arrays

* Split persist out of set

* Hook up events for SNMP v3 Auth
correct Config::persist behaviour with arrays

* dependent settings (primitive for now)
actually update the settings values in the LibrenmsSettings component

* more complex "when" behaviour

* remove un-needed seeder

* add poller_modules.cisco-qfp

* remove en.json (disable warning)

* don't set default for log_dir or log_file, otherwise it won't be processed correctly

* Fix module order
add some missing settings

* more config corrections

* correct graphs
correct loading values set to null (although it should have no difference)
remove project_name_version

* Add nfsen settings.  Docs are very confusing, so might have flubbed something
remove option for array definition of select option as numeric indexes messes it up

* Correct more upstream config differences

* Config cleanup after a bunch of merges.

* Fixes

* add version tags to js/css files
remove old js

* Print out full settings list read-only

* Add http_proxy setting
fix indents in config_definitions.json

* repeaters default is 0 (aka 20)

* cleanups

* rewrite the dynamic config docs

* add language docs

* Don't show snmp v3 auth add/remove if disabled by config.php
2019-10-16 21:22:05 +00:00