18059 Commits

Author SHA1 Message Date
Tony Murray
f2f169ae78 Modernize mempools (#12282)
* mempools to modern module
quick hacky hrstorage port

* port ucd-snmp-mib to Mempools

* Populate DB for ucd
Prep for yaml

* initial yaml attempt

* more complex conversions
fixes to YamlDiscovery, make leading $ optional and allow mib::oid format

* walk full tables and skip values
normalize percentages above 100

* handle precent only ones (specify total as 100)

* Move default polling out of YamlMempoolsDiscovery

* fixes

* Update test data hrstorage should be correct.

* perc_warn for hrstorage

* Host Resources, record buffer, cached, and shared

* Host Resources is always better, don't do both HR and UCD

* fix unix, include warning levels

* variable size

* consolidate skip_values

* define mempools schema

* number instead of integer

* more schema refactor

* one more skip_values reference

* throw error for invalid oid translation
aos6

* a*  and Cisco

* updated test data

* update almost all hrstorage data files

* b*

* c* with test data
use standard cache for hrStorage

* use cache for storage module too

* hand bsnmp properly

* bdcom

* exclude total oid from yaml so it is not polled
May add a way to ignore this behavior and poll it, but I don't know if that is needed.

* automatically handle percent only values

* ciscowlc

* only poll used or free if we have used, free, and total.

* fix skipping

* the dlinkoning
fix find value when value "name" is numeric

* support numeric oids

* dnos/ftos attempt

* I guess we can't filter on total > 0

* edgecos

* e*

* f WIP

* f*

* gwd (aka g*)

* h* + procurve

* i*

* j*

* m*

* support 0% used memory (however unlikely)

* n*

* CISCO-PROCESS-MIB memory, share cache with processors module

* ignore mempools with invalid total

* p*

* quanta

* r*
fix raisecom mibs terribly broken

* s-z

* style fixes

* Move VRP back to PHP and make it actually work

* fix zynos

* update schema

* Update Cisco processor data for description bug fixes

* fix comware processors

* comware mempools with memory size
default precision to 1

* sophos-xg updated data

* hrstorage use ram size for buffers, cache, and shared

* Show memory available instead of free in device overview

* UCD, use same rrd format, store available instead of free in the db.

* Calculate availability for HOST-RESOURCES-MIB

* Convert UCD to standard polling

* rename old rrd files

* initial graph work

* graph WIP

* Graph looking decent

* Graph looking decent for hr

* remove old ucd_graph code

* handle availability mempool
more graph cleanup

* color adjustments

* remove accidental free calculation

* Update test data and fix corner cases

* fis pfsense

* update schema

* add default value for mempool_class

* fix whitespace

* update schema

* update schema correctly

* one more time

* fortigate_1500d-sensors missing oids

* Update docs.

* fix indent

* add implements MempoolsDiscovery explicitly to OS

* remove ucd_memory graph references
remove unused device_memory graph

* remove unused functions

* set devices with mempools to rediscover to prevent/minimize gaps

* use a subquery

* add overview graph

* port health mempools table

* Update device mempool

* only show overview if multiple

* Don't override user set warn percentages in discovery

* fix missed usage

* fix style

* Safety check to not rename rrd files incorrectly if migration has not been run.

* Fix overview percent bar and represent available and used on the bar

* missed an item to convert to mempool_class

* percent on the wrong side
2020-11-23 15:35:35 -06:00
Jellyfrog
b022a30a5b Apply fixes from StyleCI (#12343) 2020-11-23 19:56:23 +01:00
Tony Murray
2b59ff5fda Add system validation (#12337)
validate lnms link and completion
validate log rotation file is installed
2020-11-23 09:46:22 -06:00
Orsiris de Jong
c149ce0b3a Python log mysql connection errors and harden config retrieval (#12058)
* Make usage of logger, also harden exception logging

* Make usage of hardened command_runner

This is a hardened version of subprocess.call which handles timeout, encoding and various errors that may happen during execution

* Remove unused exception logging

* Fix bogus exception logging

* Rename exception to match log line

* Make command_runner pylint friendly under Unix

* Add copyrights to my parts of code
2020-11-23 00:35:49 -06:00
Olen
4bdcb7d88e IRC Adding floodcontrol. Better alerts (#12141)
* Adding floodcontrol. Better alerts

* Code cleanup

* Code cleanup take 2

* Deduplicate code for sending alerts

* Keeping PRs in sync

* this->sendAlert...

* Allow filtering by hostname in ".log" command

* StyleCI

* StyleCI
2020-11-23 00:35:19 -06:00
Tony Murray
7ed34c889e Distributed Poller improved validation (#12269) 2020-11-23 00:33:56 -06:00
SourceDoctor
7550ef1d8c Show associated Alert Rules on Alert Templates (#12259)
* Show associated Alert Rules on Alert Templates

* rewrite to Eloquent

* eager load

* style fix

* style fix

* update query

* change query
2020-11-23 00:32:17 -06:00
Jellyfrog
1220d583e9 Apply fixes from StyleCI (#12336) 2020-11-21 22:52:58 -06:00
Tony Murray
1afc6abdb5 Always return json for api requests (#12335)
even if the client does not properly request it
2020-11-22 03:21:47 +01:00
SourceDoctor
b162eff5ee Global Settings - Discovery Network (#12334)
Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-21 18:08:46 +01:00
SourceDoctor
fde1b1ab43 Interface description types to Global Settings (#12291)
* Special Ports to Global Settings

* customers_descr type fix

* add mix-manifests - move section

* move to webui :P

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-21 11:44:33 +01:00
Meh
c53dec796e Added devices unpolled alert rule template (#12321)
* Update alert_rules.json

Added devices unpolled alert template

* Update alert_rules.json

* Use coalesce to provide default value

* Update alert_rules.json

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-20 22:22:45 -06:00
ottorei
a7e65be09d Bugfix: SQL error with alerts search bar (#12329)
* Fix SQL-error on search bar

* restore escapes

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-20 22:13:59 -06:00
hartred
c66d6493fa SNR-ERD add support RSensor-H/T/P (#12328)
* SNR-ERD add support RSensor H/T/P

yaml

* SNR-ERD add support RSensor H/T/P

test json

* SNR-ERD add support RSensor H/T/P

snmpsim
2020-11-20 21:25:46 -06:00
SourceDoctor
a3470bb06e Custom_descr Ports - configurable Icons (#12331)
* Custom_descr Ports - configurable Icons

* fixes

* fixes
2020-11-20 20:59:54 -06:00
SourceDoctor
04f29c2717 Fix Device Dependency Count Limitation (#12332) 2020-11-20 20:58:08 -06:00
Jellyfrog
820cf1dcca Fix disabling of built-in commands when using ./lnms (#12308)
* Fix disabling of built-in commands when using ./lnms in production

It wasn't working correctly after Laravel upstream changes

* Use nunomaduro/laravel-console-summary

* fix command name

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-19 22:51:20 +01:00
Tony Murray
aded238299 Do not run validate.php as root (#12327) 2020-11-19 22:41:32 +01:00
SourceDoctor
0ede8a2c1d API - Allow Hostname on add/remove Device Dependencies (#12319)
* API - Allow Hostname on add/remove Device Dependencies

* style fix

* change device placeholder name
2020-11-18 20:23:07 -06:00
Tony Murray
8ee9d0cfb7 Fix model observer registering multiple times (#12323)
* Fix model observer registering multiple times

* remove unused import
2020-11-18 20:21:04 -06:00
SourceDoctor
1652d17ff4 Global setting ad domain (#12326)
* API - Allow Hostname on add/remove Device Dependencies

* style fix

* change device placeholder name

* Global Settings - add missing Active Directory Domain field

* cleanup

* cleanup

* cleanup

* cleanup
2020-11-18 20:18:11 -06:00
SourceDoctor
3daacc5e1a Sort Alert Transport by Name (#12318) 2020-11-17 23:08:48 -06:00
SourceDoctor
deae23cca7 Mark acknowledged Alerts in Alert Rule List (#12320)
* Mark acknowledged Alerts in Alert Rule List

* style fix
2020-11-17 22:28:57 -06:00
HalianElf
498e0be319 Update regex for vCenter 7 (#12316)
It looks like VMware dropped the Appliance for vCenter 7 since it only comes as an appliance now. Example: `VMware vCenter Server 7.0.1.00100 embedded build 17004997 VMware, Inc x86_64`
2020-11-17 00:53:46 +01:00
MarlinMr
a95f098ee8 More secure example configuration (#12317) 2020-11-17 00:52:15 +01:00
hartred
3ffb797705 Add support new os SNR-ERD (#12315)
* Add support new os SNR-ERD

mib files

* Add support new os SNR-ERD

discovery os

* Add support new os SNR-ERD

support sensors

* tests files

* tests json
2020-11-14 20:24:50 +01:00
Tony Murray
f4ea49e38e Update Distributed-Poller.md 2020-11-12 07:40:02 -06:00
RobJE
e1497ecf0b shellcheck (#12258)
* remove unneeded semi-colons at end of line

* fix SC2206: Quote to prevent word splitting/globbing

* fix SC2164: Use 'cd ... || exit' in case cd fails

* fix SC2004: $/${} is unnecessary on arithmetic variables.

* fix SC2155: Declare and assign separately to avoid masking return values.

* fix SC2124: Assigning an array to a string! Assign as array, or use * instead of @ to concatenate.

* fix SC2059: Don't use variables in the printf format string. Use printf "..%s.." "$foo".

* fix SC2076: Don't quote rhs of =~, it'll match literally rather than as a regex.

* fix SC1090: Can't follow non-constant source. Use a directive to specify location.

* fix SC2166: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined.

* SC2223: This default assignment may cause DoS due to globbing. Quote it.

* fix SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

* fix SC1117: Backslash is literal in "\n". Prefer explicit escaping: "\\n"

* better handling of branch comapre

* add missing local ver_73 from merge

* remove duplicate definition
2020-11-11 23:21:15 +01:00
Keryazmi
565dc3c445 New device support named BTI SA-800 Series (#12264)
* New device named bti800

Support new device for model bti sa-805

* Create bti800.yaml

* Create BTI8xx-MIB

* Create BTI8xx-SYSTEM-MIB

* Create BTI8xx-TC-MIB

* Create BTI8xx-INTERFACE-MIB

* Create BTI8xx-SFP-MIB

* Create bti800.inc.php

* Create bti800.inc.php

* Create bti800.inc.php

* Create bti800.snmprec

* Create bti800.json

* OS icon for BTI800

* Update bti800.yaml

Change mib folder name 'bti800' to 'bti'.

* Rename mibs/bti800/BTI8xx-MIB to mibs/bti/BTI8xx-MIB

Change directory named 'bti800' to 'bti'.

* Rename mibs/bti800/BTI8xx-INTERFACE-MIB to mibs/bti/BTI8xx-INTERFACE-MIB

Change directory named from 'bti800' to 'bti'

* Rename mibs/bti800/BTI8xx-SFP-MIB to mibs/bti/BTI8xx-SFP-MIB

Change directory named 'bti800' to 'bti'.

* Rename mibs/bti800/BTI8xx-SYSTEM-MIB to mibs/bti/BTI8xx-SYSTEM-MIB

Change directory named 'bti800' to 'bti'.

* Rename mibs/bti800/BTI8xx-TC-MIB to mibs/bti/BTI8xx-TC-MIB

Change directory named 'bti800' to 'bti'.

* Create bti800.yaml

Added system temperature and transceiver temperature for BTI800.

* Delete bti800.inc.php

System temperature and transceiver temperature code convert to yaml file locate at /includes/definitions/discovery/bti800.yaml

* Update bti800.inc.php

Add missing code at line 20.

* Update bti800.inc.php

* Update bti800.yaml

* Delete bti.svg

* Replace bti.svg to bti.png

* Update bti800.snmprec

* Update bti800.json

* Update bti800.json

* Update bti800.json

* Update bti800.yaml

* Update bti800.inc.php

Change call for each current Tx and Rx value by using pre_cache() function instead using snmp_get() function for each value to return.
2020-11-11 23:13:55 +01:00
Denny Friebe
9977fc46df Rittal CMC III: Add device support for PU and PU Compact (#12268)
* Rittal CMC III: Add device support for PU and PU Compact

* Rittal CMC III: Readd accidentally removed sysObjectID

* Rittal CMC III: Some small fixes

* Rittal CMC III: Use device serial number instead of the internal device id for state index naming
- version, hardware and serial is now read via yaml
- update test files

* Rittal CMC III: Check if the serial number is available

* Rittal CMC III: Update test data

* Rittal CMC III: Update unset

* Rittal CMC III: Remove space in yaml
2020-11-11 23:11:50 +01:00
ospfbgp
ef7fd466c3 Add support for new Extreme VSP/VOSS models (#12273)
* Update voss.yaml

* Update xos.yaml

* Update voss.yaml

* Update xos.yaml

* Update voss.yaml

* Update voss.yaml

* Update voss.yaml

* Update xos.yaml

* Update boss.yaml

* Update voss.yaml

* Update boss.yaml

* Update voss.yaml
2020-11-11 23:09:53 +01:00
Hans Erasmus
904667def8 Update distributed poller documentation (#12312)
Because of https://github.com/librenms/librenms/pull/11896 I thought it would be a good idea to update the docs as well.
2020-11-11 23:05:02 +01:00
ottorei
f261abac59 Update junos MIB-files (#12313) 2020-11-11 23:01:24 +01:00
Jellyfrog
cfd9dce620 Convert Virtual Machine pages to Laravel (#12287)
* Convert Virtual Machine pages to Laravel

* wip

* wip

* wip

* wip

* wip

* wip

* delete

* wip

* wip

* move powerStateLabel
2020-11-11 01:15:20 +01:00
SourceDoctor
ac5920f45c OSPF View Open/Close Button (#12311)
* OSPF View Open/Close Button

* style fix

* style fix
2020-11-10 10:28:39 -06:00
SourceDoctor
be3853f885 Fix Network Map Device Group Highlighting (#12290)
* Fix Network Map Device Group Highlighting

* style fix

* change parameter fallback

* change if clause
2020-11-09 07:48:03 -06:00
Tony Murray
8f3b08721a fix customers_descr config definition (#12310) 2020-11-09 10:30:25 +01:00
Wheel
475c079402 Ubiquiti Edgepower battery values (#12247)
* UBNT-EdgeMAX mib update

* Ubiquiti Edgepower new SNMP values

* Fix typo

* Guess test data

* Update edgeos-ep.json

* more favorable rounding

* Update edgeos-ep.json

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-09 00:51:22 -06:00
Jellyfrog
b7676e4cc8 Update to Composer 2 (#12263)
* Update to Composer 2

No need to roll our own checks anymore since this is now built in.
Also limit the update check to once a week.

* update

* Make Composer wrapper report correct exit code

* Update composer_wrapper.php

* re-enable extension check

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-09 00:28:07 -06:00
Patrik Jonsson
591384f321 Adding Fortigate HA checks (#12300)
* Adding Fortigate HA checks

* Fixed issues the linter had

* Moving sync checks to the poller since standalone units was marked as unsynchronized

* Whitespace lint corrections

* More whitespace linting fixes

* Removing haState since it only showed the state of one device in the cluster.

* Adding a third state fo sync where it will show a warning if a device that is part of a cluster does not have a peer

* Updating snmp test data for fortigate 1500d

* Changing numerical oids to names

* Restoring the original 1500d json file

* Adding cleaned test data

* Removing tags to see if that makes Travis happier

* Removing duplicate rows in SNMP data

* Take this Travis!

* Pretty please Travis?

* Fourth time's a charm?
2020-11-09 00:07:45 -06:00
SourceDoctor
76ab814647 only add new outage entry if Device status changed (#12309)
* only add new outage entry if Device status changed

* .
2020-11-08 22:15:29 -06:00
PipoCanaja
2b5da1ef05 VRP NAC polling optimisation (#12279)
* Only poll the OIDs needed

* style
2020-11-07 22:31:04 -06:00
Jellyfrog
8977bb50ad Remove some unused files (#12307) 2020-11-07 22:00:47 -06:00
Jellyfrog
75230d344b Remove incorrect mib_dir (#12306) 2020-11-08 00:55:42 +01:00
Hayden
e34b6877fd Add OSPF cost (TOS) (#11929)
* add OspfTos model to include ospfIfMetricEntry data (ospf interface cost)

* remove WIP comment

* add new model and db migration

* update OspfTos relationship to OspfPort and correct cleanup call

* add "Cost" to device\routing\ospf web UI interface

* updated db-schema test unit

* updated db_schema to resolve conflicts

* removed tos metrics from rrd ospf stats due to rrdcached errors

* reorder db_schema.yaml to match upstream

* readd ospf_tos to db_schema

* styleCI changes

* Capture OSPF test data

* add ospf unit tests for iosxr

* ospfAuthType

* update db_schema

* remove extra migrations

* add ospf tests for iosxe

* add ospf test to ios

* merge ospf_tos into ospf_ports

* update db_schema

* update ospf module unit tests for ios, iosxe, iosxr

* fix ospf_ports ospfIfMetricStatus column type

* update db_schema

* more efficient dropColumn in down() migration

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-07 09:27:25 -06:00
averzicco
f7f2688125 Fix bgp polling for BGP4-MIB devices (#12301)
* Fix bgp polling

* Change polling logic for BGP4-MIB devices

* make int rollover easier to read

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-07 09:12:09 -06:00
Walter Sonius
ee739e21fe Extend Debian instructions with Raspberry Pi OS (#12302)
* Extend Debian instructions with Raspberry Pi OS

The Debian 10 part of the install instructions for LibreNMS just need a different bash location to be compatible on Raspberry Pi OS, formerly known as Raspbian which is based on Debian 10 Buster.

* Update Install-LibreNMS.md

Co-authored-by: Jellyfrog <Jellyfrog@users.noreply.github.com>
2020-11-07 02:01:56 +01:00
joseUPV
e781537b56 Ignore aos6 phantom fan (#12303) 2020-11-07 01:56:25 +01:00
Tony Murray
25722bbbd5 Faster initial database creation (#12297)
* Faster initial database creation
Add db dumps to improve initial db migration
On my test system 13s -> 7s, a 46% reduction.
Should help out a lot on systems like RPi

* lnms schema:dump command

--snapshots saves db snapshots, otherwise it saves the yaml

* fix style

* fix timezones...

* not working with in-memory db yet
remove build.sql
2020-11-06 01:19:15 +01:00
ottorei
03c950af8c Update F5 MIB-files (#12296) 2020-11-05 17:11:41 +01:00