Commit Graph

46 Commits

Author SHA1 Message Date
Tony Murray
957ecc2441 Locations UI and editing (#9480)
* Better handling of errors
Mapquest seems to return the center of the US on error.......

* Editable locations WIP

* Change to bootgrid ajax table WIP

* Graphs working, using handlebars
update js libs
add current location button

* remove sql query, change icon

* Add the map to the device view, only when gps is expanded.
Allow edit on device page, share js code

* fix chevron rotation, improve click area

* extra warning

* fix overview layout (remove containers)

* fix style

* fix html divs, change collapse ui a bit
move css, update css/js versions

* start zoomed out on new locations

* don't double load scripts, zoom to 17

* fix php-md errors, remove unused use statement

* improve non-admin experience

* Move locations page to Laravel
More functions in Url and Html util classes
reduce code duplication

* translation buttons too

* fix whitespace

* move formatters to the frontend

* small changes

* disable traffic for locations with no devices

* change down 0 to green from gray

* missing "

* Fix paginate all

* working fix for paginate all

* allow sort by counts

* fix down sort

* a little safety

* Don't call the function twice

* btn-xs
2018-11-29 19:23:40 -06:00
Tony Murray
3e35ee0e7d Refactored and update Location Geocoding (#9359)
- Fix location so it is a regular database relation (this allows multiple devices to be accurately linked to one location and saves api calls)
- Parse coordinates from the location more consistently
- Add settings to webui
- ~~Used [PHP Geocoder](http://geocoder-php.org/), which has lots of backends and is well tested. (also includes reverse and geoip)~~
- Google Maps, Bing, Mapquest, and OpenStreetMap supported initially.
- Default to OpenStreetMap, which doesn't require a key.  They will liberally hand out bans if you exceed 1 query per second though.
- All other Geocoding APIs require an API key. (Google requires a credit card on file, but seems to be the most accurate)
- Update all (I think) sql queries to handle the new structure
- Remove final vestiges of override_sysLocation as a device attribute
- Update existing device groups and rules in DB
- Tested all APIs with good/bad location, no/bad/good key, and no connection.
- Cannot fix advanced queries that use location

This blocks #8868

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-11-28 22:49:18 +00:00
Tony Murray
1f4be31120 Release script updates to perform tag on github (#9283)
Full GH release creation.  Not much error handling.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-10-03 21:44:03 +01:00
Tony Murray
14a168b2a9 Keeps the dashboard sessions from expiring. (#9263)
* Keeps the dashboard sessions from expiring.
Route dashboard ajax calls through Laravel.
Boots minimal cookies and sessions.
Does not fix other pages for now, real fix is to fully port.

* Check Laravel auth for the legacy calls.
Display Laravel errors in the dashboard.
legacy auth checks are mostly extraneous now.
2018-09-30 21:23:00 -05:00
Tony Murray
6242f941f6 Update Eventlog WebUI/backend to use ajax (#9252)
* WIP Eventlog table

* Initial Eventlog rework

* fromdevice is not a request parameter

* updates

* remove unneeded field

* Cleanups
2018-09-24 08:07:00 +01:00
Tony Murray
21ca8bf0b0 Update syslog display and backend (#9228)
* Update syslog display backend
select boxes now dynamically load instead of loading all on pageload
select and table ajax backends completely redone with Laravel (just syslog for now)
duplicate url creation to Url utility class for now (uses Device model instead of array)
build short hostname functionality into Device->displayName() helper

* Fix whitespace

* Some tidying up. Split out displayName() and shortDisplayName()

* Enable auto-sizing.
Fix small error in Url

* Eager load device
Use bootstrap theme for select2
2018-09-20 21:33:03 +01:00
Tony Murray
79333c45f6 Allow trusted proxy via APP_TRUSTED_PROXIES (#9196)
* Allow trusted proxy via APP_TRUSTED_PROXIES
Set to '*' by default to emulate legacy behavior.
Set up doc describing environment variables

* Create helper to parse environment variables into arrays properly.

* Update doc blocks
2018-09-13 07:26:42 -05:00
Tony Murray
e8cf6bb385 Fix API auth issues (#9185)
* Fix API auth issues
Api access page now creates tokens with the correct ID.
Correctly creates users for legacy user tokens.
Fix Ldap comparison
Laravel Util class to make code easier to access/read

* More api access page fixes

* fix style
2018-09-11 22:36:52 -05:00
Tony Murray
ade409c4ec Disable test devices to block normal poller (#9131)
Still could happen, but much less likely.
2018-09-03 07:46:02 +01:00
Neil Lathwood
27088d0b50 Update to add breaking-change label + fix users list formatting (#9121) 2018-08-31 20:42:19 -05:00
Tony Murray
f73f7ec5ce Fix invalid test data (#9099)
* Fix invalid test data
and add test for it

* fix bgp

* add missing comma back
2018-08-28 14:18:59 -05:00
Tony Murray
184f5a5f99 Fix invalid json in test data (#9015)
* Fix invalid json in test data
Most simply needed the syntax fixed
fail2ban needed updated data (added non-zero values in test data too)
awplus: ntp application cannot be enabled by tests since it doesn't use snmp extend... ntp app is enabled by ntp module... pass on that for now.

* missed awplus change somehow...
2018-08-16 11:11:03 -05:00
Tony Murray
4c6f917d9e Updates to snmptrap handling (#9010)
* Updates to snmptrap handling
fix a bug in findDeviceByIP.  Add more tests for that.
Move handle outside of the Trap class, it doesn't fit.
Add developer docs.

* fix tests copy paste issue.

* Fix findByIp when port may not exist.

* Logging: Output context (and extra) if they exist

* Generic trap event logging and new config setting.
2018-08-14 07:56:16 +01:00
TheGreatDoc
9940c884f5 Fixed snmptraps. (#8898)
* Fixed snmptraps.

* Fixed space

* Added bgp down/up and authentication failure

* Fixed typo

* Fixed some typos, arrays, astext and format_hostname

* Updated documentation

* Moved code to a function

* Some refactor

* Minor fixes

* Minor fixes 2

* More minor fixes

* Changes requested by Tony

* Minor fixes

* Moved include to snmptrap.php

* Refactor traps to use object oriented code.

Should trigger events too/instead, but we'll leave that.
Testing todo

* Add tests and fix things so they actually work
Not checking events yet.

* Fixed typo and severity level

* Update composer deps, I think the lock file wasn't right.
add json and mbstring extension deps while I'm at it.

* Fix several issues with phpunit fixtures
2018-08-11 16:37:44 -05:00
Tony Murray
d566544743 Find snmpsimd binary rather than infer snmpsimd.py (#8952)
instead of calling snmpsimd.py blindly

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-08-04 22:26:49 +01:00
Tony Murray
eeb3d58f5b Improved Logging and Debugging (#8870)
Use Log facility when Laravel is booted.
Update init.php so we can easily boot Laravel for CLI scripts. (and just Eloquent, but that may go away)
Move all debug setup into set_debug() function and use that across all scripts.
Log Laravel database queries.
Send debug output to librenms log file when enabling debug in the webui.
Allow for colorized Log CLI output. (currently will leave % tags in log file output)

** Needs testing and perhaps tweaking still.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-13 23:08:00 +01:00
Tony Murray
846cf11331 Support specifying submodules in poller and discovery for debug (#8896)
Mostly useful for debugging

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-12 21:30:39 +01:00
Neil Lathwood
faba6d9d02 Fix PRs that were created before the last tag but merged since it (#8858) 2018-07-03 07:00:58 -05:00
Neil Lathwood
c6fc763586 Added docs for generating changelog + fixed some code (#8845) 2018-06-26 17:29:52 -05:00
Neil Lathwood
0a983e998e Add Changelog generator (#8791)
* feature: Added initial changelog generator

* More updates

* Update and rename release.php to Release.php

* Update Kernel.php
2018-06-15 07:55:31 -05:00
vivia11
94693b0d9e Use CISCO-VRF-MIB for vrfs on non MPLS cisco devices (#8756)
* Add vrf for non mpls cisco devices, add mib and test data

* Fix custom_where in vrf module

* Better VRF module test data collection

* Update ios_6500.json

* don't include ifVrf in ports

* update all ports add dependencies to pre-commit.php

* new json files
2018-05-25 23:27:54 -05:00
PipoCanaja
f663f8dec4 device: Added FDB table support for VRP using HUAWEI-L2MAM-MIB (#8719)
* Adding VRP support for FDB table using HUAWEI-L2MAM-MIB

* Test data + review corrections

* Test data correction

* Remove double text

* PHP EOL at the right spot

* Rename vrp_5720.fdb.snmprec to vrp_5720.snmprec

* Update vrp_5720.snmprec

* Create vrp_5720.json

* Update vrp_5720.json

* Update vrp_5720.json

* Fix fdb query, test data, and mibs
2018-05-19 09:46:42 +01:00
Tony Murray
909aacd27a Add fdb-table module to tests (#8723)
* Add fdb-table module to tests
Also add the capability to select fields on joins, makes it easier to have consistent data without extra storage.

* add ticks for safety

* support multiple fields, but not aliasing

* more ticks

* Fix up selects

* Add Vrf Support too

* Fix application select
2018-05-17 09:42:00 -05:00
Tony Murray
8afafe7eb0 refactor: pre-commit.php updates (#8679)
Disable the log for unit tests to speed things up hopefully
2018-05-09 12:53:45 +01:00
Tony Murray
d1e8e2177f fix: Fixed Q-BRIDGE-MIB VLANs (#8623)
* Fix Q-BRIDGE-MIB VLANs
Properly handle dot1qVlanTimeMark index (by ignoring it instead of assuming it is 0)
Adds test data for at least one device that uses Q-BRIDGE-MIB.

Fixes: #8516

* fix whitespace

* remove unused variable
2018-04-27 14:22:49 +01:00
Neil Lathwood
fbbc2570c6 Refactor: Exclude modules from json test data when empty (#8533)
* refactor: Exclude modules from json test data when empty

* applied murrants patch
2018-04-12 19:37:13 -05:00
Tony Murray
cc377353f3 Snmpsim fail when it fails (#8392)
* Update snmpsim to fail better if it can't be started.
The main issue is the isRunning() check was done too soon after forking, the sh process was still running.

* increase default delay to 2s

* Pull this back, all we need is the sleep really.

* Remove removed exception
2018-04-05 22:49:58 +01:00
Tony Murray
3cd6d90239 Fix save-test-data (#8399)
But now it buffers all output...
2018-03-16 09:11:49 +00:00
Tony Murray
2bcc76625c Module tests update: per module output, only store modules data that are run (#8355)
* Test data: don't save data for modules that were not run

* Sometimes mibs are prefixed with +

* Save output for each module and only output the errored module.
Also makes it easier to identify modules that were ran.

* Clean up json files.
Message when no module data exists.

* verbose output when phpunit --debug is used
order by for ports
avoid graphite output in tests
only load module_tables.yaml once

* use explode and strpos instead of regex since it is failing...

* Fix some warnings

* Fix whitespace
2018-03-14 22:28:01 +00:00
Tony Murray
b2ce9b173b save-test-data.php: Print a message when we encounter a bad module name (#8274)
* Print a message when we encounter a bad module name (likely a typo)

* Raise exception for invalid module name and handle it.
2018-02-25 20:03:18 -06:00
Tony Murray
11147d3bbf Major Processors rewrite (#8066)
* Extract DiscoveryItem and move some things to better places.
Extract model class
Fix up model construction.  I have problem with construction...
Makeshift model working.  Switch constructor to factory.  discover() and create()
Support legacy discovery.
Remove uneeded custom pollers
Remove netonix custom detection as we try ucd on all os now.
Add a few yaml procs.  Fix a couple things.
More processor discovery conversions
Move Calix e7 to standard hrProcessorLoad, but it doesn't fully implement the HR-MIB, move things around to make it work.
Add a few yaml procs.  Fix a couple things. Correct some stupid mib stuff.
Move more, drop php 5.3
Add netscaler which uses string indexes.  Port fiberhome to yaml and use skip_values
More conversions.  BroadcomProcessorUsage Trait
Serveriron and Ironware share some mibs.  Create a common abstract os for them.
Add yaml support for mib specification in each data entry
Make legacy discover_processor() set 0 for hrDeviceIndex

Untangle Dell switch OS processors

Use use shared OS for groups if they don't have a specific group.
fix silly mib mistake

Make index optional

Move HR and UCD to Traits and out of Processor.

* forgot to update the fortiswitch index

* Make sgos and avaya-ers match the old index.

* fix comware test data

* fix merge errors

* fix dsm and remove pointless empty modules

* file not found exception is in the wrong place.

* Updated processor development docs
2018-02-05 07:39:13 -06:00
Tony Murray
f1a2e8d3b4 Exclude some modules when collecting all modules (#8124) 2018-01-25 21:07:10 +00:00
Tony Murray
f7941797b6 Fix testing state sensor index dumping order. (#8117)
Add netonix state sensor test data
2018-01-21 07:43:10 -06:00
Tony Murray
494c29aefa Split save-test-data.php into two scripts, allow mass update (#8115)
* Add mass update to save-test-data.
Fail when snmpsim doesn't start and try to give a hint.

* Split save-test-data.php into two scripts

One for updating snmprec files.
One for updating db dump files.
2018-01-20 20:22:47 -06:00
Tony Murray
88b4e31912 refactor: Refactored bgp-peers (#7972)
* bgp-peers update
Use IP parsing library more effectively.
Fix update issue
Include comware bgp-peers test data
deeper refactoring
Allow JunOS fallback to BGP4-MIB, some models only use that.
Add junos bgp4 data
Actually update all bpg data in the poller

* fix arista data. refactor some to prevent missing data from mucking things up

* refactor to use $oid_map for BGP4-MIB too

* A few comments to break up teh file a bit

* Junos bgp data an a quick revert for now.

* small cleanup

* fix ios bgp polling, add test data

* check the correct variable
2018-01-18 20:54:38 +00:00
Tony Murray
44c1ee5f43 Fix tests... (#8080)
Don't exit when we can't add a device, fail :)
2018-01-12 15:24:12 -06:00
Tony Murray
b2a54dae3b Testing updates: module testing and collect only (#8063)
* Testing updates module testing and collect only

* cache clear no-longer needed.
2018-01-09 11:40:26 -06:00
Neil Lathwood
42e5819130 refactor: Updated discovery/poller to use numeric sysObjectID (#7922)
* refactor: Updated discovery to use a core module for sysDescr/sysObjectID use

* final update hopefully

* revert changes

* more changes + docs

* migrated poller to use numerical sysObjectID

* more updates for sysObjectID

* update any alert rules which might have enterprises. in

* moved schema file

* small updates

* updated getHostOS()

* scrut fixes

* updated sysObjectId -> sysObjectID

* updated sysObjectId -> sysObjectID

* updated remainder of sysObjectId -> sysObjectID

* another sysObjectId -> sysObjectID

* fixed secureplatform test data

* Fix tests: $device is not pulled from the database before polling
Also, update the db in the core discovery module.
2018-01-06 23:00:47 -06:00
Tony Murray
6a122f21f4 Improve test data capture multi-line data matching (#8023) 2018-01-04 10:25:29 -06:00
Neil Lathwood
7d0ad5c4ef tests: Added testing for Arista EOS BGP (#7984)
* tests: Added testing fro Arista EOS BGP

* fixing tests

* small updates

* last one

* more updates

* Account for no space after = in snmp data
always load_os() when a new device is freshly loaded by device_by_id_cache()

* don't be fatal in load_os()

* Update json to correct data
2017-12-30 10:31:06 +00:00
Tony Murray
20865b0ca4 OS Testing update (#7971)
* OS Testing update
Option on pre-commit.php to run tests for just one OS. Reorganize tests a bit to support this.
Improved bgp-peers capture and os module capture (initial)
Better output when phpunit tests fail

* fix a couple issues.  Better output for another

* Use the correct setup function
2017-12-28 16:12:08 -06:00
Tony Murray
e24b55ba98 Fix module dependency selection. (#7933) 2017-12-20 21:32:21 -06:00
Tony Murray
fff66d3c00 Feature: Generic discovery and poller tests (#7873)
* Processor Tests!

* Capture data from live devices easily.

* fix up some stuff, remove powerconnect things as they seem to be just broken.

* formatting, fix missing assignment
add netonix processor data

* fix multi-line, always add sysDescr and sysObjectID
ios cpm test file

* revert composer change and fix whitespace issues

* add help text

* missed help text

* tighter debug output

* handle empty strings properly and mibs with numbers

* use keys for sorting as intended

* fix type with empty data

* oops :)

* whitespace fix

* try installing fping

* Fix TestCase collision + cleanup

* mark TestCase as abstract
don't run two instances of snmpsim

* better database dumps, improved capture

* style fixes

* fix quotes add a few more tables

* add --prefer-new, properly merge data

* Support separate discovery and poller data. But don't waste space if they aren't needed.

* refactor to use class
collects all code in one place for reusability

* reorganize

* Print out when not saving.

* Support for running multiple (or all) modules at once.

* tidy

* Change unit test to a generic name and test all modules we have data for.

* Add documentation and a few more tidies

* whitespace fixes

* Fix tests, add a couple more modules, more docs

* More docs updates

* Fix scrutinizer issues

* add bgp-peers
2017-12-20 08:36:49 -06:00
Tony Murray
2e73b75297 fix: Use memcached to lock daily processes on Distributed Pollers (#7735)
* fix: use memcached to lock daily processes on Distributed Pollers

* All the locks!
2017-11-24 09:37:52 +00:00
Tony Murray
c0b267b49c refactor: Discovery protocols re-write (#7380)
* refactor: discovery protocols re-write
extract more code to make it easier to read
use snmpwalk_group
walk lldpRemTable instead of lldpRemoteSystemsData
Optimize discover_device to reduce dns lookups of bad dns names
Make discovery_by_ip behavior consistent between cdp and lldp
move can_skip_discovery to discovery functions

Tested lldp, cdp, and ospf.

* always discover links even if the remote device isn't in system
Fix discover_link always updating

* fix style

* fix update, not fetching id

* increase platform length.  Most devices use sysDescr here, which is max length 255

* remove orphaned links

* update db_schema.yaml
2017-10-04 08:16:23 +01:00
Tony Murray
1d777693e1 feature: New IP parsing classes. Removes usage of Pear Net_IPv4 and Net_IPv6. (#7106)
* Refactor remove use of pear Net_IPv4 and Net_IPv6

Caution, I added tests to test the functionality of the new functions, but I'm not 100% certain the replaced usages match 100%.  Please test.

Left usage in generate-iplist.php because I'm not sure the use of that script, so I didn't want to implement additional functionality to support it.

Add support for ipv6 fo ipInNetwork() function

Add phpdocs to functions
Remove accidental inclusion

Use binary operations instead of string for networKFromIp()
Tidy up binary operations in ipInNetwork()

tidy parsing cidr network

Change to a class based implementation.

Update phpdocs.
Fully catch exceptions for hex parsing.

Fix mistake.

support older phpunit versions

Fix php shifting to negative with php 5

* Fix graph authorization. Uncaught exception and checking an ipv4 against an ipv6 network == fail.

* Also remove colons from hex strings.

* Fix typo in bgp polling
Two passes of removing invalid characters from hex strings. (That way we don't mangle valid IPs)
2017-08-08 14:14:58 -05:00