Commit Graph

162 Commits

Author SHA1 Message Date
SourceDoctor
11246ca8e1 option to ignore device status (#15697)
* option to ignore device status

* .

* .

* schema update
2024-01-31 13:18:34 -06:00
eskyuu
30953c386f Add functionality for custom maps (weathermaps) (#15633)
* Initial commit with editor.

* Added custom map models and database migrations.
Modified the controller and view to support saving the custom map settings to the database

* Added menu items and sorted out access permissions for maps and nodes

* Cleaned up some of the conditions in the javascript section of the blade

* Started work on the map data save

* Save of map nodes and edges is complete

* Got the map to load data on page load and added the delete functionality

* Fixed a typo and made link colour black if intertface is down

* Various usability fix-ups

* Show the save button on node and edge delete

* Fixed up access for users without global read

* Increase typeahead search size and standardised the way modals are triggered.

* Update data fetch to copy values into array so I can add more fields

* Convert blank array check to use count()

* Formatting changes

* More formatting fixes

* Formatting again

* DB schema update

* Revert previous commit

* Pass device id to pages

* Remove bad characters from javascript

* Re-add the - character in search results

* Update to avoid background colour being set to the current colour for offline devices

* Fixed a bug in speed detection when no suffix is given

* Fixed up the speed colour calculation and added comments

* Update default edge font size to 12

* Reduce arrow size

* Formatting fix

* Update the custom map controller to handle null interface speeds

* Alter JSON columns to be longtext instead

* Only refresh map data on successful save

* Update labels on default settings to make it clear that they are not saved

* Added timestamps to all custom map tables
Use HasFactory instead of static definitions for custom map tables
convert JSON DB fields to longtext and updated PHP to do the appropriate JSON decoding as a result

* Added missing vis.js images for the editor

* Split the custom map blade into different pages

* formatting fixes

* Initial commit with editor.

* Added custom map models and database migrations.
Modified the controller and view to support saving the custom map settings to the database

* Added menu items and sorted out access permissions for maps and nodes

* Cleaned up some of the conditions in the javascript section of the blade

* Started work on the map data save

* Save of map nodes and edges is complete

* Got the map to load data on page load and added the delete functionality

* Various usability fix-ups

* Show the save button on node and edge delete

* Fixed up access for users without global read

* Increase typeahead search size and standardised the way modals are triggered.

* Convert blank array check to use count()

* Formatting changes

* More formatting fixes

* Formatting again

* DB schema update

* Revert previous commit

* Pass device id to pages

* Remove bad characters from javascript

* Re-add the - character in search results

* Update to avoid background colour being set to the current colour for offline devices

* Reduce arrow size

* Only refresh map data on successful save

* Update labels on default settings to make it clear that they are not saved

* Added timestamps to all custom map tables
Use HasFactory instead of static definitions for custom map tables
convert JSON DB fields to longtext and updated PHP to do the appropriate JSON decoding as a result

* Added missing vis.js images for the editor

* Split the custom map blade into different pages

* Updated the custom maps to use the select2 searches for ports and devices

* Fix port search clearing with select2

* Update DB schema to add timestamps

* Add the ability to set a node alignment value where nodes will align to a grid

* Add a checkbox to re-center edge lines

* Schema update for node alignment

* Removed unused route

* Fixups after rebase

* Remove DevicePortSearchController

* Rebase fixups

* Remove unneeded controller

* Formatting fixes

* Update all network map documentation

* Fixed typo in doc

* Change background imgae database migration

* Update migration for custom map background to fix schema error

* Place a try/catch around the BLOB->MEDIUMBLOB migration

* Formatting fix

* Moved custom map background image location and added some SVG images to test as image options

* Updated the editor to use a static set of device images

* Update the image logic in the editor and added to the viewer

* DB Schema update

* Formatting

* remove svg height/width attributes

* Added some more icon options for arrows

* Added database migration to allow nodes to link to another custom map
Fixed an error in the image migration

* Added the ability to link a node to another custom map

* Formatting fixes

* DB Schema update

* Remove images-custom directory

* Explicitly cast map ID to int

* Made the image selection list dynamic based on the contents of the custom map icons directory

* Formatting fix

* Double-clicking on a link will take you to the link

* Remove whitespace

* Add translations
fix an xss and hopefully not add any new ones
refactor node image to use translations with fallback

* split modals out into separate files
return width/height to avoid js scope issues

* Formatting fixes

* refactor edit select page into a "manage" page
Still left:
validation/custom request
Controller refactor
ui tweaks

* MapSettingsRequest

* Refactor more routes, policy, controller
I think this is the last refactor. Everything is now organized in a standard way.
Missing a method to check if a user has access to a map

* Fix booleans and style

* Add versioning to the background image to prevent browser caching

* Fixed the background image update by splitting it into a separate modal
Changed the delete button on the map editor screen to return to the map list

* Formatting fix

* Added double-click actions in editor to edit nodes and edges

---------

Co-authored-by: Tony Murray <murraytony@gmail.com>
2024-01-31 08:56:59 -06:00
PipoCanaja
a7a6e67e87 NAC - Improve search in WebUI - Keep Historical data (#15629)
* search improvement for PortNac

* rename func

* style

* DB migration for timestamps

* style

* rules

* fix manually db_schema

* remove vlan only search for now

* add a bool column to ident historical nac entries

* add columns to the table

* queryByOui renamed everywhere

* age value instead of delete

* style

* style

* use Illuminate\Support\Facades\DB;

* tests fix

* module_tables.yaml

* dump model instead of tests/module_tables.yaml

* tests

* testVrp

* daily

* config

* display historical NAC entries in global view

* same for NAC device page

* nac tab on device/port view

* and display the tab link if necessary

* filter by port as well

* historical data in port tab

* formatters
2024-01-24 09:15:40 +01:00
Neil Lathwood
7325995954 Updated bill_data table, alter indexes and add new column (#15751)
* Updated bill_data table, alter indexes and add new column

* Added missing <

* Removed first

* Moved column in bill_data

* Updated schmea changes

* Updated schmea

* Updated migration

* Use id alias

* index after and only index

* Update and rename 2024_01_09_223917_bill_data_new_column.php to 2024_01_09_223917_bill_data_new_primary.php

* Trying these schema changes

* Another go

* And another

* Debugging

* Revert change

* Revert changes + debugging

* Updated schemas again

* Trying to rename column name

* Updated db_schema.yaml

* renamed column back to id

* Added debugging

* Added debugging

* Added debugging

* Added debugging

* Added debugging

* Remove redundant primary id

* Remove var_dump

* Remove var_dump

* I swear

---------

Co-authored-by: Tony Murray <murraytony@gmail.com>
2024-01-22 06:35:36 -06:00
Stef
5733942aa2 WebUI - Alert Rule Editor: new notes field (#15631)
* add a new notes field to the rule editor. use a bigger text field for sql queries to avoid mistakes...
* add api doc
* allow notes field from the collection
* add a sample notes to the collection
* lint and db schema
* unmix some schema changes from a nother PR
* unmix schema update No°2
* unmix schema update No°3 - silly me
* add strip_tags, minor optimisation, db default value
* apply linting
* db_schema empty '' as default
* update db_schema.yaml
* default value changed as 'BLOB, TEXT, GEOMETRY or JSON column 'notes' can't have a default value'
* better keep the migration in 2023...
* Update 2023_12_12_171400_alert_rule_note.php
* Update create-alert-item.inc.php

---------

Co-authored-by: PipoCanaja <38363551+PipoCanaja@users.noreply.github.com>
2024-01-17 09:09:19 +01:00
PipoCanaja
b12ee04bda bug - timos MPLS - more poller fixes (#15624)
* fix 1, ErrorException: Undefined array key sdpFarEndInetAddress in /opt/librenms/LibreNMS/OS/Timos.php:584

* fix2: don't try mplsSdpBinds polling if prerequisites not met

* fix 3: ErrorException: Attempt to read property lsp_path_id on null in /opt/librenms/LibreNMS/OS/Timos.php:790

* style

* fix phpstan

* schema_update

* schema

* migration_style

* migration drops and recreate

* migration drops and recreate

* migration drops and recreate

* style
2023-12-17 20:15:46 -06:00
amyjohn000
a8480e5e0a Update the type of nummonbssid column in the access_points table (#15647)
* Update the type of nummonbssid field in the access_points table
* Update 2023_12_15_105529_access_points_nummonbssid_integer.php
Fix format errors
2023-12-15 09:16:46 +01:00
AdamB
a61c11db7e Make vminfo.vmwVmGuestOS wider (#15595)
With tools installed, the string provided by ESXi contains a lot more information. In 8.9, Red Hat have further extended the string to include a machine readable string, parsable by tools that are compatible with OVAL:

Linux 4.18.0-513.5.1.el8_9.x86_64 Red Hat Enterprise Linux 8.9 (Ootpa) 8.9 Red Hat Enterprise Linux 8.9 (Ootpa) cpe:/o:redhat:enterprise_linux:8::baseos
2023-11-22 22:53:19 -03:00
William Edwards
df46a77cd9 Increase max package name length to 128 characters (#14895)
* Increase max package name length to 80 characters

One of my packages has a name of 65 characters. LibreNMS limits package names to 64 characters (`varchar(64) NOT NULL`). As the name of my package is truncated, polling fails with the following error:

```
Error polling unix-agent module for http-jglt01.gmdo.ha.cyberfusion.cloud. PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '458-python3-cyberfusion-cluster-configurations-redirect-fast-...' for key 'packages_device_id_name_manager_arch_version_build_unique' in /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:117
```

There is no well-defined max package name length. An email on the `debian-devel` mailing list (https://lists.debian.org/debian-devel/2011/04/msg00981.html) states that the length of package names should not exceed 40 characters, and that the length of file names should not exceed 90 characters, but these limits are not enforced. Lintian does not check package name length, but does check that file name length does not exceed 80 characters (tag 'package-has-long-file-name').

Summarised: there are no well-defined and enforced limits, and those that Lintian advices differ from the aforementioned post on the `debian-devel` mailing list. Therefore, I have landed on the semi-random limit of 80 characters (which the original limit of 64 characters seems to be as well).

* Create 2023_11_04_125846_packages_increase_name_column_length.php

* Update db_schema.yaml

---------

Co-authored-by: Tony Murray <murraytony@gmail.com>
2023-11-04 14:37:53 -05:00
Tony Murray
9c2d2cd412 OSPF tables fix integer types (#15528)
All Counter32 and Gauge32 types should be unsigned integers to match the range that can be returned via snmp
2023-10-31 10:11:56 -05:00
Tony Murray
cdb0ff8ea8 Cisco ASA table add alert defaults (#15488)
Fixes issue when values weren't set
Can't set an upper limit since models are different
Also, no way to set this in the webui and the default alert rule doesn't use the values, so yeah, probably could have deleted the fields.
2023-10-20 08:43:16 -05:00
Tony Murray
1cbcbdabbb Never set bgp and ports delta fields to unsigned (#15468)
it can make the migration fail.
2023-10-16 21:14:57 +02:00
Tony Murray
6ccb17e4ea Restore _delta fields to signed (#15458) 2023-10-15 13:14:19 +02:00
Tony Murray
df88899873 Change port and BGP stats fields to unsigned (#15449)
Should match the value ranges we can receive from snmp now.
2023-10-12 22:42:36 -05:00
Tony Murray
0540c56d0f Add application metrics model and relationships (#15417)
* Add application metrics model and relationships

* update schema check file

* Add missed index

* Update testing definition

* sqlite can't create the id later

* update schema dumps

* update testing_persistent schema

* update baseline
2023-10-12 02:24:27 -05:00
Tony Murray
ac23c133d9 Fix an agent bug if a process ran more than 999 days (#15411)
* Fix an agent bug if a process ran more than 999 days
Increase it to 274 years before that happens ;)
Fix a few other small bugs

* Update schema check file
2023-10-07 20:14:00 -05:00
Tony Murray
087d588102 Vmware vminfo modernize (#15008)
* Vmware vminfo
Remove legacy file and migrate to OS discovery

* tighter

* ios_stp-vlans working correctly now

* Make vmwVmGuestOS nullable

* Discover os info too

* VM Info module

* Apply fixes from StyleCI

* Fix log severity

* Fix log severity (more)

* VM Info module

* Poll with ESXi too because it is lightweight
add test data

* poller data now too

---------

Co-authored-by: StyleCI Bot <bot@styleci.io>
2023-10-05 19:49:26 -05:00
Zane C. Bowers-Hadley
2618a99be5 Application Soft Delete (#15270)
* add the softdeletes migrations for applications

* add working migration file

* add deleted_at to db schema.yaml for applications

* update includes/html/forms/application-update.inc.php to work with softdeletes

* update includes/html/pages/device/edit/apps.inc.php for softdelete

* update includes/discovery/applications.inc.php to work with softdelete

* minor updates to application-update.inc.php for disabling

* style cleanup

* set discovered when running discovery

* update application tests to include deleted_at

* add deleted_at to a missed test

* a few more tweaks for opensips

* add a missing deleted_at for linux_suricata_extract-v1

* fix fillable for Application model

* massive cleanup of the application update widget thingy

* improve the code for discovery and using Laravel

* add a missing line to app/Models/Application

* add a missing include to app/Models/Application.php

* record includes for Application model

* remove apps from the applications table when a device is deleted

* revert to using upcert and where for discovery to fix CI

* make discovered fillable and set it when running discovery... convert back to firstOrNew

* clean up application discovery a bit and use observer

* style fix

* spelling fix... disablaed -> disabled

* rever removal to just use where

* cleanup app removal on delete

* add restored to ModuleModelObserver

* delete -> forcedelete fix

* apply the suggested changes

* use murrants other suggestion

* style fix
2023-09-06 16:34:39 -05:00
Zane C. Bowers-Hadley
6dbfbe99bc for the applications table set the defaults for app_instance and app_instance to '' (#15278)
* add migration file for new defaults for applications table

* update the db_schema yaml and tweak the migration so it does not change the varchar size for app_status
2023-09-01 15:01:12 -05:00
Tony Murray
976cb53ead Fix install failing this migration (#15262)
Unsure if this the correct fix or if it even fixes the issue :D
https://community.librenms.org/t/install-fails/22181
2023-08-29 08:00:17 -05:00
Tony Murray
744ff7543e Fix missing roles (#15256) 2023-08-28 09:39:03 -05:00
Tony Murray
2cd207028a Implement RBAC (only built in roles) (#15212)
* Install bouncer

* Seeder and level migration

* Display and edit roles

* remove unused deluser page

* Update Radius and SSO to assign roles

* update AlertUtil direct level check to use roles instead

* rewrite ircbot auth handling

* Remove legacy auth getUserlist and getUserlevel methods, add getRoles
Set roles in LegacyUserProvider

* Small cleanups

* centralize role sync code
show roles on user preferences page

* VueSelect component WIP and a little docs

* WIP

* SelectControllers id and text fields.

* LibrenmsSelect component extracted from SettingSelectDynamic

* Handle multiple selections

* allow type coercion

* full width settings

* final style adjustments

* Final compiled assets update

* Style fixes

* Fix SSO tests

* Lint cleanups

* small style fix

* don't use json yet

* Update baseline for usptream package issues

* Change schema, not 100% sure it is correct
not sure why xor doesn't work
2023-08-28 00:13:40 -05:00
Tony Murray
3170ec1399 Fix vendor_ouis duplicates in migration (#15202)
* Fix vendor_ouis duplicates in migration
Can't trim the duplicates efficiently due to lacking an index on oui.

* fix style

* restrict columns to make strict group by happy.
2023-08-04 20:57:48 -05:00
Tony Murray
034ca2d418 Handle empty dbSchema table (#15198)
For example during install.  Seeder is run after migrations.
2023-08-04 08:31:00 -05:00
Tony Murray
12f8bb2040 MAC Vendor OUI use scheduler (#15187)
* MAC Vendor OUI use scheduler
Add command to update `lnms maintenance:fetch-ouis`
Show vendor column in tables if mac_oui.enabled is set to true
Improve scheduler validation handle non-standard install directories and systems without systemd
Add index to table to improve speed and improve mac->vendor lookup speed
Scheduled weekly with random wait to prevent stampeding herd issues for upstream
drop oui update from daily

* MAC Vendor OUI use scheduler
Add command to update `lnms maintenance:fetch-ouis`
Show vendor column in tables if mac_oui.enabled is set to true

* Lint fixes and better prefix detection

* update schema file
2023-08-03 19:29:30 -05:00
Jellyfrog
5a56e9081e Use Laravel schema migration command directly (#14039)
* Add new schema migration command

* Apply fixes from StyleCI
2023-08-03 19:26:48 -05:00
Andy Norwood
64c4650801 Add FDB table vendor search drop down (#15072)
* Add vendor search and related functions

* Add OUIDB cache for vendor lookup

* Add vendor drop down

* appy style CI changes

* Apply style CI and lint changes

* more styleCI changes

* update type hinting

* Edit mac_oui cache lock name and function

* Update MAC OUI message during daily

* Use DB for vendor lookup

* New vendor_oui table migration

* New MAC OUI to database function

* Update readbleOUI to use DB rather than cache

* Make StyleCI changes

* styleCI tweak

* Remove lock release to allow refresh timer

* change migration name to match table

* add schema dump

* update schema

* styleCI tweak
2023-07-21 11:30:13 -05:00
Jellyfrog
2b3575a5e9 Laravel 10.x Shift (#14995)
* Apply code style

* Remove explicit call to register policies

* Shift core files

* Shift config files

* Default config files

In an effort to make upgrading the constantly changing config files
easier, Shift defaulted them and merged your true customizations -
where ENV variables may not be used.

* Bump Laravel dependencies

* Add type hints for Laravel 10

* Shift cleanup

* wip

* wip

* sync translation

* Sync back config

* Public Path Binding

* QueryException

* monolog

* db::raw

* monolog

* db::raw

* fix larastan collections

* fix phpstan bug looping forever

* larastan errors

* larastan: fix column type

* styleci

* initialize array

* fixes

* fixes

---------

Co-authored-by: Shift <shift@laravelshift.com>
2023-05-24 22:21:54 +02:00
Tony Murray
b19c783f04 Fix timetaken columns too small (#15042) 2023-05-16 07:04:09 -05:00
Jellyfrog
b0998b99a4 Adopt anonymous migrations (#15005)
Co-authored-by: Shift <shift@laravelshift.com>
2023-04-26 19:01:55 -05:00
Jellyfrog
0a351b49fd Laravel 9.x Shift (#14504)
* Move `resources/lang` folder

* Shift registered middleware

* Remove `fruitcake/laravel-cors` dependency

* Streamline `$commands` property

* Upgrade to Flysystem 3.0

* Shift core files

* Convert `optional()` to nullsafe operator

* Remove unnecessary `$model` property

* Convert route options to fluent methods

Laravel 8 adopts the tuple syntax for controller actions. Since the old options array is incompatible with this syntax, Shift converted them to use modern, fluent methods.

* Convert deprecated `$dates` property to `$casts`

* Shift config files

* Default config files

In an effort to make upgrading the constantly changing config files
easier, Shift defaulted them and merged your true customizations -
where ENV variables may not be used.

* Bump Laravel dependencies

* Use `<env>` tags for configuration

`<env>` tags have a lower precedence than system environment variables making it easier to overwrite PHPUnit configuration values in additional environments, such a CI.

Review this blog post for more details on configuration precedence when testing Laravel: https://jasonmccreary.me/articles/laravel-testing-configuration-precedence/

* Fix error provider

* Match new symfony syntax

* Match upstream syntax

* Fix route syntax

* generate composer.lock

* Sync back configs

* routes

* composer

* Fix more flare

* fix cors

* sync lang

* Apply fixes from StyleCI (#14517)

Co-authored-by: StyleCI Bot <bot@styleci.io>

* bump larastan

* update packages

* wip

* Temporarily lower phpstan level

* Update phpstan.neon

* wip

* wip

* wip

* Apply fixes from StyleCI (#14592)

Co-authored-by: StyleCI Bot <bot@styleci.io>

* test

* Update CiHelper.php

* Update test.yml

* Update CiHelper.php

* Update CiHelper.php

* Apply fixes from StyleCI (#14616)

Co-authored-by: StyleCI Bot <bot@styleci.io>

* test?

* fix phpstan problems

* dont run snmpsim on github ci

* Fix whitespace

* More whitespace

* More whitespace ???

* I think the space broke it

* fix the reset of the whitespace

* hard code auth guard

---------

Co-authored-by: Shift <shift@laravelshift.com>
Co-authored-by: StyleCI Bot <bot@styleci.io>
Co-authored-by: Tony Murray <murraytony@gmail.com>
2023-04-17 06:51:35 -05:00
Tony Murray
52f5f10231 Fix some lint issues (#14962) 2023-04-15 09:02:41 -05:00
Tony Murray
cafdec6285 Remove unused port fields (#14963)
* Remove unused port fields
(and set vlan to default to null)

* Update test data for deleted fields

* update vlan to null

* missed field removals

* More correct migration

* Update new data

* restore accidental deletions
2023-04-14 16:47:26 -05:00
Jellyfrog
ca8b780494 Store empty user_func as null (#14900) 2023-03-14 18:42:45 +01:00
PipoCanaja
8e3fe223ef Sensors - Allow changing RRD type in YAML and PHP sensor discovery (#14208)
* new row variant

* syntax

* syntax

* DB

* syntax

* schema

* test

* allAdvaTests

* Tests

* Tests

* Tests

* missingMigration

* missingMigration

* Tests

* Tests

* Tests

* Tests

* tests

* tests

* tests

* tests

* tests

* test

* fix_Aos7

* fix snmprec arista

* fix snmprec arista

* fix snmprec arista

* tests

* merge_fix

* aos7 backslash

* aos7

* more

* more

* defaultValue

* arista

* typehint

* tests

* tests

* voss

* aos7

* aos7

* aos7

* testWithoutarista-mos_metamux48-c-0-16.json

* tests

* ciscosb

* new tests with 2004 and only sensors module

* adding wireless module again

* adding wireless module again (cont)

* airos

* aos7

* arista

* arista

* bdcom

* fortigate

* fortigate

* fortigate

* ftos

* ftos

* tests on 1804

* tests once more

* tests once more, cont

* tests once more, cont

* tests once more, cont

* tests once more, cont

* tests once more, cont

* more

* bdcom

* comware

* edgeosolt

* enexus

* arista-mos

* removeAristaAgain

* ies500_manual_edit

* ies500_manual_edit

* mni

* volius

* zywal

* manually imported arista-mos

* manually imported arista-mos

* arista-mos test is triggering something ...

* restoreArista

* tests

* tests

* aristamos

* ies52xxM_4206.json

* revert linux_supermicro.json

* revert linux_supermicro.json

* benuos

* fortiadc

* rnx-updu
2022-10-25 08:31:02 -05:00
PipoCanaja
53bfb24ef9 Migrate xDSL code to module, and add support for VDSL2 MIB (#14207)
* use component to discover if xDSL polling is needed

use component to discover if xDSL polling is needed

* Components OK, Polling in correct files, no DB for VDSL

* GUI

GUI_suite

* per port as well

* rename

* interface listing

* draytek_snmpsim

* fix arraymerge

fix names and max value

* schema

schema

style

* remove one dbFetchRows

remove 2x dbFetchCell

style

style

remove Legacy dbFetchRow

tests

tests

eloquent

more eloquent

more eloquent

one more gone

* fix properties access

eloquent_insert_update

style

tests

tests

tests

tests

* tests

tests

tests

* adslLineCoding

* Models

* fix not nullable cols in DB from code

default values

typo

rename

typo

schema

fix

fix

vdsl fix now

typo

typo

fix size

fix size

* Power values for VDSL

Power values for VDSL

Power values for VDSL

DB

* cleanup

* Rrd::checkRrdExists

* always enable DSL discovery

style

* xdsl module

* cleanup and move to Module

cleanup and move to Module

cleanup and move to Module

cleanup and move to Module

* Fix display

* fix polling and tenth

* remove legacy poller

* Style and Cosmetics

Cosmetics

Cleanup

* Translations

Translations

* exists

exists

* add test support for xdsl

* remove last component call

unused

* translations

* remove non standard onclick event on xdsl line

* Update Discovery Support.md

Update Poller Support.md

toner_gone

* Notification for removal of lnms config:set enable_ports_adsl true

* enable on devices with potential DSL interfaces

* tests are working now

fix teldat tests

* os_schema

* teldat

* move to new module structure

* move to new module structure

* wrong dump function

* wrong dump function

* laravel_through_key hidden

* Update notifications.rss

* Update notifications.rss

Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-09-07 19:29:17 -05:00
Zane C. Bowers-Hadley
0bbcde1227 add the ability for storing app data to prevent spamming of the event log via via component usage (#14087)
* initial work on add the ability to save/fetch app data

* update to use get_app_data for ZFS

* update the poller for the new app_data stuff

* ZFS now logs changes to pools

* add schema update for app_data stuff

* small formatting fix

* add a missing \

* now adds a column

* sql-schema is no longer used, so remove the file that was added here

* misc cleanups

* rename the method in database/migrations/2022_07_03_1947_add_app_data.php

* hopefully fix the migration bit

* add the column to misc/db_schema.yaml

* more misc small DB fixes

* update the test as the json column uses collat of utf8mb4_bin

* revert the last change and try manually setting it to what is expected

* remove a extra ;

* update suricata as well

* correct the instance -> instances in one location to prevent the old instance list from being stomped

* remove a extra ;

* update fail2ban to use it as well

* remove two unused functions as suricata and fail2ban no longer use components

* style cleanup

* postgres poller updated to use it

* update html side of the postgres bits

* chronyd now uses app data bits now as well

* portactivity now uses it as well

* style fix

* sort the returned arrays from app_data

* correct log message for port activity

* collocation change

* try re-ordering it

* add in the new data column to the tests

* remove a extra ,

* hmm... ->collate('utf8mb4_unicode_ci') is not usable as apparently collate does not exist

* change the column type from json to longtext

* mv chronyd stuff while I sort out the rest of the tests... damn thing is always buggy

* hmm... fix a missing line then likely move stuff back

* style fix

* add fillable

* add the expexcted data for fail2ban json

* escape a " I missed

* add data for portactivity

* add suricata app data

* add app data to zfs legacy test

* put the moved tests back into place and update zfs-v1 test

* add app data for chronyd test

* add app data for fail2ban legacy test

* update zfs v1 app data

* add some notes on application dev work

* add Developing/Application-Notes.md to mkdocs.yml

* add data column to it

* added various suggestions from bennet-esyoil

* convert from isset to sizeof

* type fix

* fully remove the old save app data function and move it into a helper function... the other still needs cleaned up prior to removal

* update docs

* get_app_data is fully removed now as well

* a few style fixes

* add $casts

* update chronyd test

* attempt to fix the data

* more doc cleanup and try changing the cast

* style fix

* revert the changes to the chronyd test

* apply a few of murrant's suggestions

* document working with ->data as json and non-josn

* remove two no-longer used in this PR exceptions

* ->data now operates transparently

* style fix

* update data tests

* fix json

* test fix

* update the app notes to reflect how app data now works

* app test fix

* app data fix for linux_lsi

* json fix

* minor doc cleanup

* remove duplicate querty and use json_decode instead

* style fix

* modelize the app poller

* use a anon func instead of foreach

* test update

* style cleanup

* style cleanup

* another test cleanup

* more test cleanup

* reverse the test changes and add in some more glue code

* revert one of the test changes

* another small test fix

* Make things use models
Left some array access, but those will still work just fine.

* missed chronyd and portactivity

* rename poll to avoid make it any confusion

* Remove extra save and fix timestamp

* save any changes made to app->data

* nope, that was not it

* What are magic methods and how do they work?

* fix two typos

* update linux_lsi test

* change quote type

Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-07-22 16:01:55 -05:00
Tony Murray
9320e6cd06 Prevent duplicate plugin table entries (#14120)
* Prevent duplicate plugin table entries
Some sort of race condition.
Add a unique index, this will cause the create query to fail when it tries to add a new entry for an existing plugin.

* Add index
2022-07-20 15:25:45 +02:00
Jellyfrog
f3281b218f Fix some broken migrations (#14040) 2022-06-14 15:01:07 +02:00
Bennet Gallein
8fdd1be3a0 feat(apps/mysql): add error-state to non-responsive mysql-servers (#13993)
* feat(apps/mysql): add error-state to non-responsive mysql-servers

* feat(alerting): add alert-rule for offline mysql-servers

* fixup! feat(alerting): add alert-rule for offline mysql-servers

* chore(style): replaced ticks

* fixup! chore(style): replaced ticks

* feat: added migration to make app_status field bigger

* feat: moved error-detection to the application_update function

* fixup! feat: moved error-detection to the application_update function

* chore: updated schema

* chore: fix styling

* Cleaner way to handle the error states

* regex should not include OK

Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-06-10 16:22:41 -05:00
Tony Murray
32f1ce494e Dashboard code cleanup (#13996)
* Dashboard Cleanup
Remove static widgets table, list of available widgets should not be in the database.
Remove legacy ajax scripts
Cleanup and reorganize controllers

* reorganize code to put all dashboard things into it's controller
better url scheme while supporting the original

* lint clean ups

* properly formatted language file

* style fixes

* update schema
2022-05-31 08:08:40 -05:00
Charly Forot
e34e6e07a5 BGP unnumbered support for Cumulus (#13785)
* Add Cumulus-BGPUN-MIB
* Add bgpPeerIface column into bgpPeers table
* update mysql database for BGP unnumbered support
* Rename BGP unumbered MIB for Cumulus
* remove duplicate Cumulus BGPUM mib
* Use custom BGPUN minb for Cumulus while polling bgp peers
* Update BGP peers discovery for Cumulus with unumbered support
* Update BGP peers polling for Cumulus with unumbered support
* Correct typing error
* Add BGP unumbered Cumulus tests units
* Remove unused SQL file
* Update migrations databse by adding bgpPeerIface into bgpPeers table
* Correct spacing error for continous integration
* Update DB column bgpPeerIface to point to ifIndex
* fix: bgpPeerIface column exist
* fix: use Eloquent instead of dbFetch deprecated
* fix: delete unused namespace
* fix: type bgpPeerIface
* fix: add root cumulus mib lacking
* move root cumulus mib into mibs/
* fix: missing root cumulus MIB
* fix: bgpPeerIface not expected for other OS
* fix: add bgpPeerIface into json tests for other OS
* fix: add bgpPeerIface for poller tests OS
* add relathionship between bgpPeers and Ports tables
* Revert "fix: bgpPeerIface not expected for other OS"
This reverts commit f979482106.
* fix: HasOne not imported
* fix: CI php placement of use line
* fix: bgpPeerIface has not default value
* updates way to fill bgpPeerIface
* remove relationship between BgpPeer and Ipv4Address/Ipv6Address
* fix: CI php syntax
fix: CI php syntax
fix: CI php syntax
fix: CI php syntax
* updates cumulus tests
* updates bgpPeerIface already filled
* fix : bgpPeerIface wrong ifIndex
* fix: update cumulus test for polling bgpPeerIface
* fix: CI PHP syntax
* fix: update cumulus test for polling bgpPeerIface
* fix: update cumulus test for polling bgpPeerIface
* Fix Fortigate Test
* Tests
Co-authored-by: PipoCanaja <38363551+PipoCanaja@users.noreply.github.com>
2022-05-22 14:27:28 +02:00
William Irvine
5672d10a79 Add ISIS discovery and polling for iosxe devices (#13880)
* Add ISIS discovery and polling for iosxe devices

* remove ModuleModelObserver

* add php stan exclusions as they are already present for the base module

* change port_id to cached lookup

* Create model object instead of using arrays and set properties directly

* remove unneeded space...

* remove null from non nullable field

* revert to extending from ciscowlc rather than os

* remove OS module

* remove phpstan exclusions and fix errors

* add spacing...

* add spacing....

* add spacing

* again...

* Add tests

* Update Iosxe.php

* Update IsisAdjacency.php

* Create 2022_04_08_085504_isis_adjacencies_table_add_index.php

* Update db_schema.yaml

* Update iosxe_asr920.json

* Update Iosxe.php

* Update Iosxe.php

* Update junos_mx5t_isis.json

Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-04-08 16:33:41 -05:00
Jellyfrog
09929bd686 Implement OAuth and SAML2 support (#13764)
* Implement OAuth and SAML2 support via Socialite

* Add socialite docs

* fixes

* Additional information added

* wip

* 22.3.0 targeted version

* Allow mysql auth as long as there is a password saved

Co-authored-by: laf <gh+n@laf.io>
Co-authored-by: Tony Murray <murraytony@gmail.com>
2022-02-20 22:05:51 +01:00
Tony Murray
29bd6789cb Stp module rewrite (#13570)
* STP module rewrite WIP

* Finish rewrite

* Ignore disabled and log root/topology changes

* Remove interfaces for now

* fix style

* Lint fixes

* Document ResolvesPortIds and hide map functions

* whitespace fixes

* Revert to stpInstances in case someone writes mstp support

* missed one

* phpstan fixes

* Handle table and oids separately

* forgot to register observer

* Test data and correct non-table handling in SnmpResponse->table()

* update test

* test data

* revert aos7 silly things

* minimal polling

* Update test data

* order ports_ntp and rename new field to port_index

* forgot the db_schema

* revert ciena-sds port things

* MSTP support, maybe

* Adding test data

* Filter bad lines instead of discarding the entire snmp response
and capture fixes and test data

* fresh data

* add os data

* update data, ignore unfound ports, obviously bad device implementation.

* fixes

* Ignore context files in os detection test

* Remove empty table data

* add ciena-sds vlan

* designatedCost column is too small

* Update stp webui

* Refactor code to interfaces, to allow vendor mibs

* update schema

* fix issues added by abstraction

* STP fixes

* Default to no context for vlan 1

* never store vlan 1

* Update test data

* remove eltex brokenness

* fix style

* fix stan

* Fix Rewrite MAC to Hex padding with floats

* fix sqlite migration
2022-01-30 16:28:18 -06:00
Tony Murray
a95efd6d2a Global search: search device display (#13583)
* Global search: search device display
Add display field to search (also port_desc_descr, portName, and bgpPeerDescr)
Rewrite backend
update typeahead bundle
update devices and ports indexes
reduce some port field sizes so we can index them

* Style fixes

* remove nonsense
2021-12-06 16:12:24 +01:00
Tony Murray
790c4b3402 Configurable device display name (#13528)
* Configurable device display name
You can just set the display name in device settings.

It also accepts simple template format with  the variables: hostname, sysName, sysName_fallback, ip

Default controlled by device_display_default, (set from old force_hostname_to_sysname and force_ip_to_sysname settings

* remove second argument to format_hostname()

* Style fixes

* Update schema

* update phpstan baseline

* Improved settings strings (and add for translation)
2021-11-18 15:46:22 -06:00
pepperoni-pi
4b572b9b31 Added migration file to change cpwVcID to unsigned integer (#13510) 2021-11-12 14:15:50 -06:00
Mike Beattie
8d5d0b12c3 Increase length of devices_attribs.attrib_type column (#13395)
* Increase length of devices_attribs.attrib_type column

closes #13381

Signed-off-by: Mike Beattie <mike@ethernal.org>

* Update db_schema.yaml

Co-authored-by: Tony Murray <murraytony@gmail.com>
2021-10-20 19:50:15 -05:00
Tony Murray
fb934e2ede Disable ExamplePlugin (#13391) 2021-10-20 08:02:01 -05:00