Commit Graph

316 Commits

Author SHA1 Message Date
Tony Murray
cb005210d2 Resubmit #9608 (#9941)
* Reorganize trap tests

* Testing db DRIVER to prevent .env from interfering

* New code to detect if Laravel is booted.  Hopefully more reliable.

* WIP external test process

* revert module test helper

* Use .env in Eloquent::boot()

* Fix test database settings loading

* fix undefined classes
(didn't find the one I needed)

* Fix incorrect Config usages
And RrdDefinition return type

* fix .env loading

* use the right DB

* slightly more accurate isConnected

* Move db_name to DBSetupTest specifically

* restore $_SERVER in AuthSSOTest

* missed item

* WIP

* tear down in the correct order.

* some testing cleanups

* remove check for duplicate event listener, it's not working right

* Don't need this change anymore

* Implement Log::event to replace legacy function log_event()

* fix port tests

* fix up tests

* remove pointless TrapTestCase class

* fix style

* Fix db config not being merged...

* skip env check for tests

* defer database operations until after Laravel is booted.

* don't include dbFaciale...

* redundant use
2019-03-12 23:59:03 -05:00
Tony Murray
eb648a1d6f Revert "Changes to improve testing (#9608)" (#9937)
This reverts commit d1a0ccda46.
2019-03-12 00:49:14 -05:00
Tony Murray
d1a0ccda46 Changes to improve testing (#9608)
* Reorganize trap tests

* Testing db DRIVER to prevent .env from interfering

* New code to detect if Laravel is booted.  Hopefully more reliable.

* WIP external test process

* revert module test helper

* Use .env in Eloquent::boot()

* Fix test database settings loading

* fix undefined classes
(didn't find the one I needed)

* Fix incorrect Config usages
And RrdDefinition return type

* fix .env loading

* use the right DB

* slightly more accurate isConnected

* Move db_name to DBSetupTest specifically

* restore $_SERVER in AuthSSOTest

* missed item

* WIP

* tear down in the correct order.

* some testing cleanups

* remove check for duplicate event listener, it's not working right

* Don't need this change anymore

* Implement Log::event to replace legacy function log_event()

* fix port tests

* fix up tests

* remove pointless TrapTestCase class

* fix style
2019-03-11 22:59:39 -05:00
Tony Murray
8d3a3c9a3e Handle unexpected os when loading yaml (#9790) 2019-02-06 14:24:39 -06:00
Tony Murray
c4ca7a4f1f Fix os loading issues (#9763)
* Fix os loading issues

* remove inaccurate debug message
2019-01-30 14:53:43 -06:00
Tony Murray
9b0a427ab0 Update common.php 2019-01-29 07:40:15 -06:00
Tony Murray
7d48c81273 fix snmp debug clipping of quote when obscuring data 2019-01-28 15:41:50 -06:00
Tony Murray
eb59287d34 Fix devices page search & OS loading (#9752)
* Fix devices page search

* add a definition_loaded flag to prevent false positives for already loaded OS

* Fix copy paste error

* and another
2019-01-28 10:37:41 -06:00
Tony Murray
ce6fae8dd1 Rewrite netcmd and ripe whois tools (#9724)
* Port RIPE whois to Laravel

* remove netcmd.php and port to laravel.
Escape CLI and set it to stream output live.

* fix browser buffer bust on a few browsers.
2019-01-25 15:30:58 -06:00
Tony Murray
8ce81f2423 Update database schema display (#9708)
* Update database schema display

* handle missing migrations table
2019-01-22 06:47:07 -06:00
TheGreatDoc
bfcfff6b5e Merge pull request #9639 from murrant/snmp-exec-timeout
Increase snmp execution time limit to 20 minutes
2019-01-16 13:57:38 +01:00
Tony Murray
d277dee14d Fix some issues with sensor limits (#9638)
* Fix Zynos temp warn limit
And others, but this seems to be the only one that uses
remove some odd limit restriction in the poller

* More consistent rounding for sensor limits

* round all numeric limits to 10 digits

* update test data

* Remove complex calculation as negative limit values are already swapped as needed.
2019-01-10 19:48:37 -06:00
Tony Murray
561e2fb6e2 Increase snmp execution time limit to 10 minutes
This is needed for discovery in some cases (mostly the arp-table module)
2019-01-10 11:18:34 -06:00
Tony Murray
def8b3e514 Add librenms command (#9619)
* Add librenms command

Hook up to artisan.  Hide dev commands and most other commands if app environment is production.
Register all commands for php artisan or when not in production.

* remove dead end line

* Console application name and version.
Had to shift some stuff from legacy code, but deferred others as it was too extensive of a change.

* switch check order

* always get local version now

* update array format

* whitepace

* fix style
2019-01-08 21:42:56 -06:00
Tony Murray
010ff08e7f Fix for snmp queries that take longer than 60s (#9574) 2018-12-20 19:51:06 -06:00
PipoCanaja
2d84606abf BUG - correcting size and non significant zeroes (#9575) 2018-12-20 19:50:36 -06:00
Tony Murray
74882e3950 Dashboard widget update (#9515)
Implemented in Laravel
Doesn't use legacy PHP session
Several widgets have new features and settings, for example:

- Multiple ports in one graph
- Maps settings are configurable and override system settings but default to system settings
- Graylog stream and/or device selection
- Much improved graph widget selection
- Many more

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-16 21:18:17 +00:00
Tony Murray
fad0bffc43 Escape SNMP & IPMI shell commands (#9537)
hopefully doesn't break anything
Mostly issues with snmp oids and options containing spaces.  Try to remove all of those.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-16 13:42:50 +00:00
Tony Murray
8bddfe7225 Fixed plugins using d_echo (#9498)
Move d_echo to helpers.php and include in autoload
Don't remove from common.php yet to be extra safe.
2018-12-08 13:16:49 +00:00
Tony Murray
c7f70b6e47 Added detection if this is a git based install or not. (#9379)
Not sure about the warning or text.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-07 22:29:37 +00:00
Tony Murray
93f44d67f8 Fixed customers page (#9521)
move customers table backend to Laravel

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-07 22:24:53 +00:00
Tony Murray
3e35ee0e7d Refactored and update Location Geocoding (#9359)
- Fix location so it is a regular database relation (this allows multiple devices to be accurately linked to one location and saves api calls)
- Parse coordinates from the location more consistently
- Add settings to webui
- ~~Used [PHP Geocoder](http://geocoder-php.org/), which has lots of backends and is well tested. (also includes reverse and geoip)~~
- Google Maps, Bing, Mapquest, and OpenStreetMap supported initially.
- Default to OpenStreetMap, which doesn't require a key.  They will liberally hand out bans if you exceed 1 query per second though.
- All other Geocoding APIs require an API key. (Google requires a credit card on file, but seems to be the most accurate)
- Update all (I think) sql queries to handle the new structure
- Remove final vestiges of override_sysLocation as a device attribute
- Update existing device groups and rules in DB
- Tested all APIs with good/bad location, no/bad/good key, and no connection.
- Cannot fix advanced queries that use location

This blocks #8868

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-11-28 22:49:18 +00:00
PipoCanaja
41cc344106 VRP: Discovery improvement for optical power + Mib upgrade (#9225)
* Huawei Mib file update

* VRP: Improve sensor discovery for optical equipment

* VRP: Improve sensor discovery for optical equipment

* VRP: Adding Test data after sensor dbm update
2018-09-19 10:00:11 -05:00
Tony Murray
32a7c50189 Use Laravel authentication (#8702)
* Use Laravel for authentication
Support legacy auth methods
Always create DB entry for users (segregate by auth method)

Port api auth to Laravel

restrict poller errors to devices the user has access to

Run checks on every page load.  But set a 5 minute (configurable) timer.
Only run some checks if the user is an admin

Move toastr down a few pixels so it isn't as annoying.

Fix menu not loaded on laravel pages when twofactor is enabled for the system, but disabled for the user.
Add two missing menu entries in the laravel menu

Rewrite 2FA code
Simplify some and verify code before applying

Get http-auth working
Handle legacy $_SESSION differently.  Allows Auth::once(), etc to work.

* Fix tests and mysqli extension check

* remove duplicate Toastr messages

* Fix new items

* Rename 266.sql to 267.sql
2018-09-11 07:51:35 -05:00
Tony Murray
a2e33efba8 Use PDO for MySQL queries (#8935)
Gut dbFacile
mres() now does nothing... could have some vulnerable queries, but most are ok

TODO
- [x] IRC client broke
- [x] Install broke

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-08-17 21:29:20 +01:00
Tony Murray
50bb72f530 Remove port association mode map from the database (#8941)
* Remove port association mode map from the database

* Renamed sql file

* Renamed sql file
2018-08-05 09:50:13 +01:00
Tony Murray
25dc44a574 fixed can_ping_device() logic (#8906)
DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-17 08:16:13 +01:00
Neil Lathwood
ad6ec7f72a Refactor alert templates to use Laravel Blade templating engine (#8803)
* Initial start on new templating support

* further updates

* more updates

* More working version

* Last fixes

* Small updates

* modified test to look for dbname.

* Schema update

* Added fix for not being able to disable Philips Hue transport

* Updated sql file

* Updated db_schema

* Set myclabs/deep-copy to be 1.7.x version in composer

* Fixes from murrant

* Forced nikic/php-parser to version 3.1.x in composer

* Updated composer to use custom fork of string-blade-compiler + fixed transport use

* Updated to always use correct template

* Merged legacy and blade templating engines

* Removed template type + fixed test-alert.php

* Added more template placeholders

* Added ability to reference data in templates as $alert->X

* Updated docs for templates

* Updated db_schema.yaml

* Added $alert->builder

* Clarify the use of $alert->builder

* Fixed the use of $alert->transport

* renamed schema file

* Added template validation

* Small update to fix travis issue

* Add Docs text to title bar

* Updated amqp to new of version

* Consistency in alert rule and template title bars
2018-07-14 22:15:43 +01:00
Tony Murray
eeb3d58f5b Improved Logging and Debugging (#8870)
Use Log facility when Laravel is booted.
Update init.php so we can easily boot Laravel for CLI scripts. (and just Eloquent, but that may go away)
Move all debug setup into set_debug() function and use that across all scripts.
Log Laravel database queries.
Send debug output to librenms log file when enabling debug in the webui.
Allow for colorized Log CLI output. (currently will leave % tags in log file output)

** Needs testing and perhaps tweaking still.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-13 23:08:00 +01:00
Richard Mayhew
528b91a056 corrected display of minigraph when using sysName as hostname (#8842)
* corrected display of minigraph when using sysName as hostname

* Check to see if its an IP or hostname. Make sure all 3 scenarios work.

* removed test rrd symlink

* removed test rrd symlink

* reverted old change

* Improve and add tests.
2018-07-02 08:23:09 -05:00
Richard Mayhew
ab5d17d715 Allow the hostname to resolve to the sysName, ie Dynamic DNS (#8810) 2018-06-23 21:59:09 +01:00
Tony Murray
9152becec7 refactor: Moved AD user and group filter functions into ActiveDirectoryAuthorizer (#8545) 2018-04-11 10:06:46 +01:00
Tony Murray
7250376104 refactor: Don't access $_SESSION directly for Auth (#8513)
* Don't access $_SESSION directly for Auth

* fix style

* add property annotations
2018-04-07 21:55:28 +01:00
sorribes22
c22b31506e api: Added lattitude and longitude on devices/device fetch (#8515)
* 'lat' and 'lng' columns added on list_devices()

* 'lat' and 'lng' columns added on device fetch

Helped by https://github.com/laf

* Commit user.name changed
2018-04-07 13:29:46 +01:00
Neil Lathwood
03076c4025 feature: Added new alert rule builder UI and rule mapping UI (#8293)
* feature: Added new alert rule builder UI

* Updated to export sql queries

* More updates

* more changes

* removed debug

* fix scrut

* Updated to include import options + various other fixes

* fix rule

* Populate name from collection rules.

* Fix default rule import
Allow new and old style rules in the collection.
Don't add new yet as I'm not sure GenSQL() is working.

* Fix GenSQL call

* Extract filter building to class so it is nicely contained in one place

* moved schema

* some fixes and tweaks

* travis fixes

* Some more features / updates

* Fix up my mistakes when adding default rules

* Use a modal for new alert (Incomplete)
Larger dialog!!
Remove page loading stuff.

Working:
Loading rules, resetting dialog, importing from collection.

Not working yet:
select width
device limited rule access? don't know what this is...

Lots of unused stuff to delete...

* reload "table" after save

* fixed editing rule

* Auto select2 width

* Reload window on save

* Restore per-device alert. Remove debug.

* Small cleanups. Rule Name first.

* Restore button to button type. Rename schema.

* Fixes: wrong command to reload window, remove extra attributes, rule is never passed

* Fixed old rule editing

* some small updates for old imports

* travis update to use trusty

* maybe travis fix

* Ability to set alert rule mappings on the rule edit screen

* pip installs one line, no quiet for deploy

* update schema def

* Fix style and some copyright headers

* fix docs missing file

* Allow new versions of snmpsim and libraries

* Parser WIP

* Fix default rules insert

* reorganize

* Legacy import first draft done

* Implement saving
Skip translation to sql for now

* Working on glues

* small rule collection fix

* Working on glues

* Working on glues

* Docs updates + small UI changes

* Parser WIP

* reorganize

* Legacy import first draft done

* Implement saving
Skip translation to sql for now

* Working on glues

* Working on glues

* Working on glues

* Add table mapping, should move to it's own class

* WIP

* Glue working!!

* Extract Schema class

* Some final touches.
revert alerts_rules.json for now.

* Finish up initial implementation
Needs more tests

* Fix a few places

* small doc updates

* Fix finding tables in grouped rules.

* remove unused code

* code format fixes

* Some quick tests for Schema
Simplified output for findRelationshipPath. Always includes start and target in the result.
This simplifies a lot of code in QueryBuilderParser.php
This also always loads the target table data now (which we want)

* Make bill_id the PRIMARY index for the bills table

* Load macros from a json file in misc instead of the database.

* Fix whitespace and wrong key for collection.

* Handle IN properly when generating SQL

* Fix glue (devices.device_id = ports.port_id) is incorrect :D
Show ALL tables we can resolve relationships for in the query builder filter.

* Remove all macros from the database
Remove insert statements, leave updates to update user's existing rules.
2018-03-14 20:25:19 +00:00
James Andrewartha
70714fe811 security: Hide more confidential data (SNMPv3, IPMI) from the debug output (#8303) 2018-02-27 15:41:44 +00:00
Tony Murray
b2762d9fa7 refactor: Prevent function collisions with Laravel (#8166)
Make function call compatible
drop insensitive starts_with and ends_with for now as they aren't needed.
2018-01-29 21:58:21 +00:00
crcro
7e03905e63 webui: Small graylog cleanup (#8057) 2018-01-09 20:47:33 +00:00
Zmegolaz
2c9468979f Load device attributes before load_os() in device_by_id_cache(). (#8044) 2018-01-08 13:37:34 -06:00
Neil Lathwood
7d0ad5c4ef tests: Added testing for Arista EOS BGP (#7984)
* tests: Added testing fro Arista EOS BGP

* fixing tests

* small updates

* last one

* more updates

* Account for no space after = in snmp data
always load_os() when a new device is freshly loaded by device_by_id_cache()

* don't be fatal in load_os()

* Update json to correct data
2017-12-30 10:31:06 +00:00
Neil Lathwood
27368a2d98 webui: Stop stripping html tags from snmp creds (#7951) 2017-12-25 09:51:31 -06:00
crcro
520fc4a268 webui: Graylog table refresh (#7832)
* initial release

* style fixes
2017-12-24 19:59:16 +00:00
crcro
71711326fb webui: Syslog table refresh (#7796)
* initial changes

* final changes

* added missing copyright, removed commented old code
2017-12-24 19:57:54 +00:00
Tony Murray
fff66d3c00 Feature: Generic discovery and poller tests (#7873)
* Processor Tests!

* Capture data from live devices easily.

* fix up some stuff, remove powerconnect things as they seem to be just broken.

* formatting, fix missing assignment
add netonix processor data

* fix multi-line, always add sysDescr and sysObjectID
ios cpm test file

* revert composer change and fix whitespace issues

* add help text

* missed help text

* tighter debug output

* handle empty strings properly and mibs with numbers

* use keys for sorting as intended

* fix type with empty data

* oops :)

* whitespace fix

* try installing fping

* Fix TestCase collision + cleanup

* mark TestCase as abstract
don't run two instances of snmpsim

* better database dumps, improved capture

* style fixes

* fix quotes add a few more tables

* add --prefer-new, properly merge data

* Support separate discovery and poller data. But don't waste space if they aren't needed.

* refactor to use class
collects all code in one place for reusability

* reorganize

* Print out when not saving.

* Support for running multiple (or all) modules at once.

* tidy

* Change unit test to a generic name and test all modules we have data for.

* Add documentation and a few more tidies

* whitespace fixes

* Fix tests, add a couple more modules, more docs

* More docs updates

* Fix scrutinizer issues

* add bgp-peers
2017-12-20 08:36:49 -06:00
bergroth
84fdfdf542 Junos dwdm interface sensor support (#7714)
* Junos dwdm interface sensor support  …
  four new sensors
    Chromatic Dispersion ps/nm
    Delay s (seconds)
    Q_factor (dB)
    preFEC_BER (ratio)

  reuse of
    SNR, dbm, temperature, current

* new dwdm sensors more generic, ber and quality_factor

* new sensors updated

* renamed html graphs new sensors

*  fixed ber polling

* Change to base SI Hz and s.

* update sensor divisors to bigint

* rename sql-schema/216.sql to 218.sql

* YAML junos dwmd sensor definitions, except BER

* bigint divisor fixed

* deafult 1 divisior

* small fixes including schema change for another PR

* schema fix
2017-12-12 20:27:10 -06:00
Tony Murray
c22c879983 fix: Remove faulty memcached code (not related to distributed polling) (#7881) 2017-12-10 20:40:45 +00:00
Tony Murray
b620692426 refactor: Update all applications to store metrics (#7853)
* Update all applications to store metrics

* fix db schema

* Add glue test
2017-12-06 22:13:10 +00:00
Tony Murray
5405498ed6 alerts: Adde generic alerts for state sensors (#7812)
* feature: Generic alerts for state sensors
A bit of a hack, but gets the job done. Perhaps someone can refactor the ResolveGlues() function sometime to make it better.

* Fix mistakes

* Add docs and collection rules, critical enabled by default

* moved schema file to 221.sql
2017-12-02 23:03:36 +00:00
Tony Murray
4a03e7838e feature: Save application metrics to db for alerting (#7828)
* feature: save application metrics to db for alerting
However, alerting will not work because ResolveGlues() is broken.
Can add workaround after state_translations alerting is merged

Does not update all applications yet, not sure if that should be done here or in another PR.

Introduces two handy functions dbDeleteOrphans() and array_by_column().  Will replace those in other locations after this is merged or separate them out if this is not merged.

* remove accidental inclusions

* Add db schema
2017-12-01 07:53:26 +00:00
Tony Murray
257a314fca feature: Validate session directory is writable (#7784) 2017-11-23 08:29:10 +00:00