Commit Graph

323 Commits

Author SHA1 Message Date
Stuart Henderson
161849219e Avoid unnecessary net-snmp long options for version info (#10405)
Long option support in net-snmp relies on non-portable usage of getopt(3) with "-:" in the argument string, this doesn't work on all OS. Replace with the alternative -V which works everywhere.
2019-07-02 16:52:55 +01:00
Tony Murray
f3ba8947f7 Use Config helper (#10339)
remove usage of global variable
2019-06-23 00:29:12 -05:00
Tony Murray
1a60c44eb0 Device groups rewrite (#10346)
* Device Groups rewrite
Updated web ui
Static or dynamic groups allowed
Alert rule query builder
Translation support
Permissions support

* cleanup, make relationship save, and validate it

* builder WIP

* rules builder and rules saving/loading

* Parse query builder to Laravel Fluent query

* Upgrade existing groups when editing.
Properly update only dynamic groups when polling.

* remove unused old code
Update API and other places to use Eloquent

* debug output in poller restored

* Fix up some things
creating static
improved validation
fix js error on creation
Fix static groups in polling

* hide pattern for static group

* Implement authorization
Use in the menu too

* update schema

* fix rollback

* Don't abort on invalid queries

* fixes to query builder

* add test data, looks like macros aren't handled (omitted them because groups don't use them generally)

* Add macro support for QueryBuilderFluentParser

* add test for macro that accepts value

* More space in forms
Retain rules when converted to static
no duplicate names allowed

* Better error feedback
Update related devices on save

* Add button icon

* format

* update docs

* fix tests

* Fix some QueryBuilderFluentParser issues with OR
updated/more test data

* Show device groups runtime
fix querybuilder.json format

* Store table joins in the rules to minimize polling time
Update group joins in daily.sh (and when they are saved)

* Update daily.php

* Add units to time
2019-06-19 16:01:53 -05:00
Tony Murray
c2b09b8f80 Revert "Rewritten device groups (including static) (#10295)" (#10340)
This reverts commit cfc51d51f5.
2019-06-16 21:52:57 -05:00
Tony Murray
cfc51d51f5 Rewritten device groups (including static) (#10295)
* Device Groups rewrite
Updated web ui
Static or dynamic groups allowed
Alert rule query builder
Translation support
Permissions support

* cleanup, make relationship save, and validate it

* builder WIP

* rules builder and rules saving/loading

* Parse query builder to Laravel Fluent query

* Upgrade existing groups when editing.
Properly update only dynamic groups when polling.

* remove unused old code
Update API and other places to use Eloquent

* debug output in poller restored

* Fix up some things
creating static
improved validation
fix js error on creation
Fix static groups in polling

* hide pattern for static group

* Implement authorization
Use in the menu too

* update schema

* fix rollback

* Don't abort on invalid queries

* fixes to query builder

* add test data, looks like macros aren't handled (omitted them because groups don't use them generally)

* Add macro support for QueryBuilderFluentParser

* add test for macro that accepts value

* More space in forms
Retain rules when converted to static
no duplicate names allowed

* Better error feedback
Update related devices on save

* Add button icon

* format

* update docs

* fix tests
2019-06-16 08:27:22 -05:00
Tony Murray
54e4d0910a Sanitize report name in pdf.php (#10270)
Authentication users could include arbitrary file.
2019-05-29 09:39:17 -05:00
Tony Murray
36431dd296 Security fix: unauthorized access (#10091)
* Security fix: unauthorized access
Affects nginx users:
Moved php files outside of public html directory (Apache was protected by .htaccess)

Affects all users:
Some files did not check for authentication and could disclose some info.
Better checks before including files from user input

* git mv html/includes/ includes/html
git mv html/pages/ includes/html/
2019-04-11 23:26:42 -05:00
Tony Murray
cb005210d2 Resubmit #9608 (#9941)
* Reorganize trap tests

* Testing db DRIVER to prevent .env from interfering

* New code to detect if Laravel is booted.  Hopefully more reliable.

* WIP external test process

* revert module test helper

* Use .env in Eloquent::boot()

* Fix test database settings loading

* fix undefined classes
(didn't find the one I needed)

* Fix incorrect Config usages
And RrdDefinition return type

* fix .env loading

* use the right DB

* slightly more accurate isConnected

* Move db_name to DBSetupTest specifically

* restore $_SERVER in AuthSSOTest

* missed item

* WIP

* tear down in the correct order.

* some testing cleanups

* remove check for duplicate event listener, it's not working right

* Don't need this change anymore

* Implement Log::event to replace legacy function log_event()

* fix port tests

* fix up tests

* remove pointless TrapTestCase class

* fix style

* Fix db config not being merged...

* skip env check for tests

* defer database operations until after Laravel is booted.

* don't include dbFaciale...

* redundant use
2019-03-12 23:59:03 -05:00
Tony Murray
eb648a1d6f Revert "Changes to improve testing (#9608)" (#9937)
This reverts commit d1a0ccda46.
2019-03-12 00:49:14 -05:00
Tony Murray
d1a0ccda46 Changes to improve testing (#9608)
* Reorganize trap tests

* Testing db DRIVER to prevent .env from interfering

* New code to detect if Laravel is booted.  Hopefully more reliable.

* WIP external test process

* revert module test helper

* Use .env in Eloquent::boot()

* Fix test database settings loading

* fix undefined classes
(didn't find the one I needed)

* Fix incorrect Config usages
And RrdDefinition return type

* fix .env loading

* use the right DB

* slightly more accurate isConnected

* Move db_name to DBSetupTest specifically

* restore $_SERVER in AuthSSOTest

* missed item

* WIP

* tear down in the correct order.

* some testing cleanups

* remove check for duplicate event listener, it's not working right

* Don't need this change anymore

* Implement Log::event to replace legacy function log_event()

* fix port tests

* fix up tests

* remove pointless TrapTestCase class

* fix style
2019-03-11 22:59:39 -05:00
Tony Murray
8d3a3c9a3e Handle unexpected os when loading yaml (#9790) 2019-02-06 14:24:39 -06:00
Tony Murray
c4ca7a4f1f Fix os loading issues (#9763)
* Fix os loading issues

* remove inaccurate debug message
2019-01-30 14:53:43 -06:00
Tony Murray
9b0a427ab0 Update common.php 2019-01-29 07:40:15 -06:00
Tony Murray
7d48c81273 fix snmp debug clipping of quote when obscuring data 2019-01-28 15:41:50 -06:00
Tony Murray
eb59287d34 Fix devices page search & OS loading (#9752)
* Fix devices page search

* add a definition_loaded flag to prevent false positives for already loaded OS

* Fix copy paste error

* and another
2019-01-28 10:37:41 -06:00
Tony Murray
ce6fae8dd1 Rewrite netcmd and ripe whois tools (#9724)
* Port RIPE whois to Laravel

* remove netcmd.php and port to laravel.
Escape CLI and set it to stream output live.

* fix browser buffer bust on a few browsers.
2019-01-25 15:30:58 -06:00
Tony Murray
8ce81f2423 Update database schema display (#9708)
* Update database schema display

* handle missing migrations table
2019-01-22 06:47:07 -06:00
TheGreatDoc
bfcfff6b5e Merge pull request #9639 from murrant/snmp-exec-timeout
Increase snmp execution time limit to 20 minutes
2019-01-16 13:57:38 +01:00
Tony Murray
d277dee14d Fix some issues with sensor limits (#9638)
* Fix Zynos temp warn limit
And others, but this seems to be the only one that uses
remove some odd limit restriction in the poller

* More consistent rounding for sensor limits

* round all numeric limits to 10 digits

* update test data

* Remove complex calculation as negative limit values are already swapped as needed.
2019-01-10 19:48:37 -06:00
Tony Murray
561e2fb6e2 Increase snmp execution time limit to 10 minutes
This is needed for discovery in some cases (mostly the arp-table module)
2019-01-10 11:18:34 -06:00
Tony Murray
def8b3e514 Add librenms command (#9619)
* Add librenms command

Hook up to artisan.  Hide dev commands and most other commands if app environment is production.
Register all commands for php artisan or when not in production.

* remove dead end line

* Console application name and version.
Had to shift some stuff from legacy code, but deferred others as it was too extensive of a change.

* switch check order

* always get local version now

* update array format

* whitepace

* fix style
2019-01-08 21:42:56 -06:00
Tony Murray
010ff08e7f Fix for snmp queries that take longer than 60s (#9574) 2018-12-20 19:51:06 -06:00
PipoCanaja
2d84606abf BUG - correcting size and non significant zeroes (#9575) 2018-12-20 19:50:36 -06:00
Tony Murray
74882e3950 Dashboard widget update (#9515)
Implemented in Laravel
Doesn't use legacy PHP session
Several widgets have new features and settings, for example:

- Multiple ports in one graph
- Maps settings are configurable and override system settings but default to system settings
- Graylog stream and/or device selection
- Much improved graph widget selection
- Many more

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-16 21:18:17 +00:00
Tony Murray
fad0bffc43 Escape SNMP & IPMI shell commands (#9537)
hopefully doesn't break anything
Mostly issues with snmp oids and options containing spaces.  Try to remove all of those.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-16 13:42:50 +00:00
Tony Murray
8bddfe7225 Fixed plugins using d_echo (#9498)
Move d_echo to helpers.php and include in autoload
Don't remove from common.php yet to be extra safe.
2018-12-08 13:16:49 +00:00
Tony Murray
c7f70b6e47 Added detection if this is a git based install or not. (#9379)
Not sure about the warning or text.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-07 22:29:37 +00:00
Tony Murray
93f44d67f8 Fixed customers page (#9521)
move customers table backend to Laravel

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-07 22:24:53 +00:00
Tony Murray
3e35ee0e7d Refactored and update Location Geocoding (#9359)
- Fix location so it is a regular database relation (this allows multiple devices to be accurately linked to one location and saves api calls)
- Parse coordinates from the location more consistently
- Add settings to webui
- ~~Used [PHP Geocoder](http://geocoder-php.org/), which has lots of backends and is well tested. (also includes reverse and geoip)~~
- Google Maps, Bing, Mapquest, and OpenStreetMap supported initially.
- Default to OpenStreetMap, which doesn't require a key.  They will liberally hand out bans if you exceed 1 query per second though.
- All other Geocoding APIs require an API key. (Google requires a credit card on file, but seems to be the most accurate)
- Update all (I think) sql queries to handle the new structure
- Remove final vestiges of override_sysLocation as a device attribute
- Update existing device groups and rules in DB
- Tested all APIs with good/bad location, no/bad/good key, and no connection.
- Cannot fix advanced queries that use location

This blocks #8868

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-11-28 22:49:18 +00:00
PipoCanaja
41cc344106 VRP: Discovery improvement for optical power + Mib upgrade (#9225)
* Huawei Mib file update

* VRP: Improve sensor discovery for optical equipment

* VRP: Improve sensor discovery for optical equipment

* VRP: Adding Test data after sensor dbm update
2018-09-19 10:00:11 -05:00
Tony Murray
32a7c50189 Use Laravel authentication (#8702)
* Use Laravel for authentication
Support legacy auth methods
Always create DB entry for users (segregate by auth method)

Port api auth to Laravel

restrict poller errors to devices the user has access to

Run checks on every page load.  But set a 5 minute (configurable) timer.
Only run some checks if the user is an admin

Move toastr down a few pixels so it isn't as annoying.

Fix menu not loaded on laravel pages when twofactor is enabled for the system, but disabled for the user.
Add two missing menu entries in the laravel menu

Rewrite 2FA code
Simplify some and verify code before applying

Get http-auth working
Handle legacy $_SESSION differently.  Allows Auth::once(), etc to work.

* Fix tests and mysqli extension check

* remove duplicate Toastr messages

* Fix new items

* Rename 266.sql to 267.sql
2018-09-11 07:51:35 -05:00
Tony Murray
a2e33efba8 Use PDO for MySQL queries (#8935)
Gut dbFacile
mres() now does nothing... could have some vulnerable queries, but most are ok

TODO
- [x] IRC client broke
- [x] Install broke

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-08-17 21:29:20 +01:00
Tony Murray
50bb72f530 Remove port association mode map from the database (#8941)
* Remove port association mode map from the database

* Renamed sql file

* Renamed sql file
2018-08-05 09:50:13 +01:00
Tony Murray
25dc44a574 fixed can_ping_device() logic (#8906)
DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-17 08:16:13 +01:00
Neil Lathwood
ad6ec7f72a Refactor alert templates to use Laravel Blade templating engine (#8803)
* Initial start on new templating support

* further updates

* more updates

* More working version

* Last fixes

* Small updates

* modified test to look for dbname.

* Schema update

* Added fix for not being able to disable Philips Hue transport

* Updated sql file

* Updated db_schema

* Set myclabs/deep-copy to be 1.7.x version in composer

* Fixes from murrant

* Forced nikic/php-parser to version 3.1.x in composer

* Updated composer to use custom fork of string-blade-compiler + fixed transport use

* Updated to always use correct template

* Merged legacy and blade templating engines

* Removed template type + fixed test-alert.php

* Added more template placeholders

* Added ability to reference data in templates as $alert->X

* Updated docs for templates

* Updated db_schema.yaml

* Added $alert->builder

* Clarify the use of $alert->builder

* Fixed the use of $alert->transport

* renamed schema file

* Added template validation

* Small update to fix travis issue

* Add Docs text to title bar

* Updated amqp to new of version

* Consistency in alert rule and template title bars
2018-07-14 22:15:43 +01:00
Tony Murray
eeb3d58f5b Improved Logging and Debugging (#8870)
Use Log facility when Laravel is booted.
Update init.php so we can easily boot Laravel for CLI scripts. (and just Eloquent, but that may go away)
Move all debug setup into set_debug() function and use that across all scripts.
Log Laravel database queries.
Send debug output to librenms log file when enabling debug in the webui.
Allow for colorized Log CLI output. (currently will leave % tags in log file output)

** Needs testing and perhaps tweaking still.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-13 23:08:00 +01:00
Richard Mayhew
528b91a056 corrected display of minigraph when using sysName as hostname (#8842)
* corrected display of minigraph when using sysName as hostname

* Check to see if its an IP or hostname. Make sure all 3 scenarios work.

* removed test rrd symlink

* removed test rrd symlink

* reverted old change

* Improve and add tests.
2018-07-02 08:23:09 -05:00
Richard Mayhew
ab5d17d715 Allow the hostname to resolve to the sysName, ie Dynamic DNS (#8810) 2018-06-23 21:59:09 +01:00
Tony Murray
9152becec7 refactor: Moved AD user and group filter functions into ActiveDirectoryAuthorizer (#8545) 2018-04-11 10:06:46 +01:00
Tony Murray
7250376104 refactor: Don't access $_SESSION directly for Auth (#8513)
* Don't access $_SESSION directly for Auth

* fix style

* add property annotations
2018-04-07 21:55:28 +01:00
sorribes22
c22b31506e api: Added lattitude and longitude on devices/device fetch (#8515)
* 'lat' and 'lng' columns added on list_devices()

* 'lat' and 'lng' columns added on device fetch

Helped by https://github.com/laf

* Commit user.name changed
2018-04-07 13:29:46 +01:00
Neil Lathwood
03076c4025 feature: Added new alert rule builder UI and rule mapping UI (#8293)
* feature: Added new alert rule builder UI

* Updated to export sql queries

* More updates

* more changes

* removed debug

* fix scrut

* Updated to include import options + various other fixes

* fix rule

* Populate name from collection rules.

* Fix default rule import
Allow new and old style rules in the collection.
Don't add new yet as I'm not sure GenSQL() is working.

* Fix GenSQL call

* Extract filter building to class so it is nicely contained in one place

* moved schema

* some fixes and tweaks

* travis fixes

* Some more features / updates

* Fix up my mistakes when adding default rules

* Use a modal for new alert (Incomplete)
Larger dialog!!
Remove page loading stuff.

Working:
Loading rules, resetting dialog, importing from collection.

Not working yet:
select width
device limited rule access? don't know what this is...

Lots of unused stuff to delete...

* reload "table" after save

* fixed editing rule

* Auto select2 width

* Reload window on save

* Restore per-device alert. Remove debug.

* Small cleanups. Rule Name first.

* Restore button to button type. Rename schema.

* Fixes: wrong command to reload window, remove extra attributes, rule is never passed

* Fixed old rule editing

* some small updates for old imports

* travis update to use trusty

* maybe travis fix

* Ability to set alert rule mappings on the rule edit screen

* pip installs one line, no quiet for deploy

* update schema def

* Fix style and some copyright headers

* fix docs missing file

* Allow new versions of snmpsim and libraries

* Parser WIP

* Fix default rules insert

* reorganize

* Legacy import first draft done

* Implement saving
Skip translation to sql for now

* Working on glues

* small rule collection fix

* Working on glues

* Working on glues

* Docs updates + small UI changes

* Parser WIP

* reorganize

* Legacy import first draft done

* Implement saving
Skip translation to sql for now

* Working on glues

* Working on glues

* Working on glues

* Add table mapping, should move to it's own class

* WIP

* Glue working!!

* Extract Schema class

* Some final touches.
revert alerts_rules.json for now.

* Finish up initial implementation
Needs more tests

* Fix a few places

* small doc updates

* Fix finding tables in grouped rules.

* remove unused code

* code format fixes

* Some quick tests for Schema
Simplified output for findRelationshipPath. Always includes start and target in the result.
This simplifies a lot of code in QueryBuilderParser.php
This also always loads the target table data now (which we want)

* Make bill_id the PRIMARY index for the bills table

* Load macros from a json file in misc instead of the database.

* Fix whitespace and wrong key for collection.

* Handle IN properly when generating SQL

* Fix glue (devices.device_id = ports.port_id) is incorrect :D
Show ALL tables we can resolve relationships for in the query builder filter.

* Remove all macros from the database
Remove insert statements, leave updates to update user's existing rules.
2018-03-14 20:25:19 +00:00
James Andrewartha
70714fe811 security: Hide more confidential data (SNMPv3, IPMI) from the debug output (#8303) 2018-02-27 15:41:44 +00:00
Tony Murray
b2762d9fa7 refactor: Prevent function collisions with Laravel (#8166)
Make function call compatible
drop insensitive starts_with and ends_with for now as they aren't needed.
2018-01-29 21:58:21 +00:00
crcro
7e03905e63 webui: Small graylog cleanup (#8057) 2018-01-09 20:47:33 +00:00
Zmegolaz
2c9468979f Load device attributes before load_os() in device_by_id_cache(). (#8044) 2018-01-08 13:37:34 -06:00
Neil Lathwood
7d0ad5c4ef tests: Added testing for Arista EOS BGP (#7984)
* tests: Added testing fro Arista EOS BGP

* fixing tests

* small updates

* last one

* more updates

* Account for no space after = in snmp data
always load_os() when a new device is freshly loaded by device_by_id_cache()

* don't be fatal in load_os()

* Update json to correct data
2017-12-30 10:31:06 +00:00
Neil Lathwood
27368a2d98 webui: Stop stripping html tags from snmp creds (#7951) 2017-12-25 09:51:31 -06:00
crcro
520fc4a268 webui: Graylog table refresh (#7832)
* initial release

* style fixes
2017-12-24 19:59:16 +00:00
crcro
71711326fb webui: Syslog table refresh (#7796)
* initial changes

* final changes

* added missing copyright, removed commented old code
2017-12-24 19:57:54 +00:00