Add functionality for custom maps (weathermaps) (#15633)

* Initial commit with editor.

* Added custom map models and database migrations.
Modified the controller and view to support saving the custom map settings to the database

* Added menu items and sorted out access permissions for maps and nodes

* Cleaned up some of the conditions in the javascript section of the blade

* Started work on the map data save

* Save of map nodes and edges is complete

* Got the map to load data on page load and added the delete functionality

* Fixed a typo and made link colour black if intertface is down

* Various usability fix-ups

* Show the save button on node and edge delete

* Fixed up access for users without global read

* Increase typeahead search size and standardised the way modals are triggered.

* Update data fetch to copy values into array so I can add more fields

* Convert blank array check to use count()

* Formatting changes

* More formatting fixes

* Formatting again

* DB schema update

* Revert previous commit

* Pass device id to pages

* Remove bad characters from javascript

* Re-add the - character in search results

* Update to avoid background colour being set to the current colour for offline devices

* Fixed a bug in speed detection when no suffix is given

* Fixed up the speed colour calculation and added comments

* Update default edge font size to 12

* Reduce arrow size

* Formatting fix

* Update the custom map controller to handle null interface speeds

* Alter JSON columns to be longtext instead

* Only refresh map data on successful save

* Update labels on default settings to make it clear that they are not saved

* Added timestamps to all custom map tables
Use HasFactory instead of static definitions for custom map tables
convert JSON DB fields to longtext and updated PHP to do the appropriate JSON decoding as a result

* Added missing vis.js images for the editor

* Split the custom map blade into different pages

* formatting fixes

* Initial commit with editor.

* Added custom map models and database migrations.
Modified the controller and view to support saving the custom map settings to the database

* Added menu items and sorted out access permissions for maps and nodes

* Cleaned up some of the conditions in the javascript section of the blade

* Started work on the map data save

* Save of map nodes and edges is complete

* Got the map to load data on page load and added the delete functionality

* Various usability fix-ups

* Show the save button on node and edge delete

* Fixed up access for users without global read

* Increase typeahead search size and standardised the way modals are triggered.

* Convert blank array check to use count()

* Formatting changes

* More formatting fixes

* Formatting again

* DB schema update

* Revert previous commit

* Pass device id to pages

* Remove bad characters from javascript

* Re-add the - character in search results

* Update to avoid background colour being set to the current colour for offline devices

* Reduce arrow size

* Only refresh map data on successful save

* Update labels on default settings to make it clear that they are not saved

* Added timestamps to all custom map tables
Use HasFactory instead of static definitions for custom map tables
convert JSON DB fields to longtext and updated PHP to do the appropriate JSON decoding as a result

* Added missing vis.js images for the editor

* Split the custom map blade into different pages

* Updated the custom maps to use the select2 searches for ports and devices

* Fix port search clearing with select2

* Update DB schema to add timestamps

* Add the ability to set a node alignment value where nodes will align to a grid

* Add a checkbox to re-center edge lines

* Schema update for node alignment

* Removed unused route

* Fixups after rebase

* Remove DevicePortSearchController

* Rebase fixups

* Remove unneeded controller

* Formatting fixes

* Update all network map documentation

* Fixed typo in doc

* Change background imgae database migration

* Update migration for custom map background to fix schema error

* Place a try/catch around the BLOB->MEDIUMBLOB migration

* Formatting fix

* Moved custom map background image location and added some SVG images to test as image options

* Updated the editor to use a static set of device images

* Update the image logic in the editor and added to the viewer

* DB Schema update

* Formatting

* remove svg height/width attributes

* Added some more icon options for arrows

* Added database migration to allow nodes to link to another custom map
Fixed an error in the image migration

* Added the ability to link a node to another custom map

* Formatting fixes

* DB Schema update

* Remove images-custom directory

* Explicitly cast map ID to int

* Made the image selection list dynamic based on the contents of the custom map icons directory

* Formatting fix

* Double-clicking on a link will take you to the link

* Remove whitespace

* Add translations
fix an xss and hopefully not add any new ones
refactor node image to use translations with fallback

* split modals out into separate files
return width/height to avoid js scope issues

* Formatting fixes

* refactor edit select page into a "manage" page
Still left:
validation/custom request
Controller refactor
ui tweaks

* MapSettingsRequest

* Refactor more routes, policy, controller
I think this is the last refactor. Everything is now organized in a standard way.
Missing a method to check if a user has access to a map

* Fix booleans and style

* Add versioning to the background image to prevent browser caching

* Fixed the background image update by splitting it into a separate modal
Changed the delete button on the map editor screen to return to the map list

* Formatting fix

* Added double-click actions in editor to edit nodes and edges

---------

Co-authored-by: Tony Murray <murraytony@gmail.com>
This commit is contained in:
eskyuu
2024-01-31 22:56:59 +08:00
committed by GitHub
parent 7228dbfa23
commit 30953c386f
58 changed files with 4051 additions and 99 deletions

View File

@@ -0,0 +1,27 @@
# Availability Map
LibreNMS has the following page to show an availability map:
- Overview -> Maps -> Availability
This map will show all devices on a single page, with each device
having either a box or a coloured square representing its status.
## Widget
There is an availability map widget that can be added to a dashboard
to give a quick overview of the status of all devices on the network.
## Settings
```bash
# Set the compact view mode for the availability map
lnms config:set webui.availability_map_compact false
# Size of the box for each device in the availability map (not compact)
lnms config:set webui.availability_map_box_size 165
# Sort by status instead of hostname
lnms config:set webui.availability_map_sort_status false
# Show the device group drop-down on the availabiltiy map page
lnms config:set webui.availability_map_use_device_groups true
```

View File

@@ -0,0 +1,139 @@
# Custom Map
LibreNMS has the ability to create custom maps to give a quick
overview of parts of the network including up/down status of devices
and link utilisation. These are also referred to as weather maps.
## Viewer
Once some maps have been created, they will be visible to any users who
have read access to all devices on a given map. Custom maps are available
through the Overview -> Maps -> Custom Maps menu.
Some key points about the viewer are:
- Nodes will change colour if they are down or disabled
- Links are only associated with a single network interface
- Link utilisation can only be shown if the link speed is known
- Link speed is decoded from SNMP if possible (Upload/Download) and defaults
to the physical speed if SNMP data is not available, or cannot be decoded
- Links will change colour as follows:
- Black if the link is down, or the max speed is unknown
- Green at 0% utilisation, with a gradual change to
- Yellow at 50% utilisation, with a gradual change to
- Orange at 75% utilisation, with a gradual change to
- Red at 100% utilisation, with a gradual change to
- Purple at 150% utilisation and above
## Editor
To access the custom map editor, a user must be an admin. The editor
is accessed through the Overview -> Maps -> Custom Map Editor menu.
Once you are in the editor, you will be given a drop-down list of all
the custom maps so you can choose one to edit, or select "Create New Map"
to create a new map.
### Map Settings
When you create a new map, you will be presented with a page to set
some global map settings. These are:
- *Name*: The name for the map
- *Width*: The width of the map in pixels
- *Height*: The height of the map in pixels
- *Node Alignment*: When devices are added to the map, this will align
the devices to an invisible grid this many pixels wide, which can help
to make the maps look better. This can be set to 0 to disable.
- *Background*: An image (PNG/JPG) up to 2MB can be uploaded as a background.
These settings can be changed at any stage by clicking on the "Edit Map Settings"
button in the top-left of the editor.
### Nodes
Once you have a map, you can start by adding "nodes" to the map. A node
represents a device, or an external point in the network (e.g. the internet)
To add a node, you click on the "Add Node" button in the control bar, then
click on the map area where you want to add the node. You will then be aked
for the following information:
- *Label*: The text to display on this point in the network
- *Device*: If this node represents a device, you can select the device from
the drop-down. This will overwrite the label, which you can then change if
you want to.
- *Style*: You can select the style of the node. If a device has been selected
you can choose the LibreNMS icon by choosing "Device Image". You can also
choose "Icon" to select an image for the device.
- *Icon*: If you choose "Icon" in the style box, you can select from a list of
images to represent this node
There are also options to choose the size and colour of the node and the font.
Once you have finished choosing the options for the node, you can press Save to
add it to the map. NOTE: This does not save anything to the database immediately.
You need to click on the "Save Map" button in the top-right to save your changes
to the database.
You can edit a node at any time by selecting it on the map and clicking on the
"Edit Node" button in the control bar.
You can also modify the default settings for all new nodes by clicking on the
"Edit Node Default" button at the top of the page.
### Edges
Once you have 2 or more nodes, you can add links between the nodes. These are
called edges in the editor. To add a link, click on the "Add Edge" button in
the control bar, then click on one of the nodes you want to link and drag the
cursor to the second node that you want to link. You will then be prompted for
the following information:
- *From*: The node that the link runs from (it will default to first node you selected)
- *To*: The node that the link runs to (it will default to the second node you selected)
- *Port*: If the From or To node is linked to a device, you can select an interface
from one of the devices and the custom map will show traffic utilisation for
the selected interface.
- *Reverse Port Direction*: If the selected port displays data in the wrong
direction for the link, you can reverse it by toggling this option.
- *Line Style*: You can try different line styles, especially if you are running
multiple links between the same 2 nodes
- *Show percent usage*: Choose whether to have text on the lines showing the link
utilisation as a percentage
- *Recenter Line*: If you tick this box, the centre point of the line will be moved
back to half way between the 2 nodes when you click on the save button.
Once you have finished choosing the options for the node, you can press Save to
add it to the map. NOTE: This does not save anything to the database immediately.
You need to click on the "Save Map" button in the top-right to save your changes
to the database.
Once you press save, you it will create 3 objects on the screen, 2 arrows and a
round node in the middle. Having the 3 objects allows you to move the mid point
of the line off centre, and also allows us to display bandwidth information for
both directions of the link.
You can edit an edge at any time by selecting it on the map and clicking on the
"Edit Edge" button in the control bar.
You can also modify the default settings for all new edges by clicking on the
"Edit Edge Default" button at the top of the page.
### Re-Render
When you drag items around the map, some of the lines will bend. This will cause a
"Re-Render Map" button to appear at the top-right of the page. This button can be
clicked on to cause all lines to be re-drawn the way they will be shown in the viewer.
### Save Map
Once you are happy with a set of changes that you have made, you can click on the
"Save Map" button in the top-right of the page to commit changes to the database.
This will cause anyone viewing the map to see the new version the next time their
page refreshes.
## Adding Images
You can add your own images to use on the custom map by copying files into the
html/images/custommap/icons/ directory. Any files with a .svg, .png or .jpg extension
will be shown in the image selection drop-down in the custom map editor.

View File

@@ -0,0 +1,11 @@
# Dependency Map
LibreNMS has the ability to show you a dynamic network map based on
device dependencies that have been configure. These maps are accessed
through the following menu options:
- Overview -> Maps -> Device Dependency
- Overview -> Maps -> Device Groups Dependencies
## Settings
The map display can be configured by altering the [VisJS-Config.md](Vis JS Options)

View File

@@ -1,89 +1,33 @@
# Network Map
LibreNMS has the ability to show you a network map based on:
LibreNMS has the ability to show you a dynamic network map based on
data collected from devices. These maps are accessed through the
following menu options:
- Overview -> Maps -> Network
- Overview -> Maps -> Device Group Maps
- The Neighbours -> Map tab when viewing a single device
(the Neighbours tab will only show if a device has xDP neighbours)
These network maps can be based on:
- xDP Discovery
- MAC addresses
- MAC addresses (ARP entries matching interface IP and MAC)
By default, both are are included but you can enable / disable either
one using the following config option:
```php
$config['network_map_items'] = array('mac','xdp');
```bash
lnms config:set 'network_map_items' "('mac','xdp')"
```
Either remove mac or xdp depending on which you want.
XDP is based on FDP, CDP and LLDP support based on the device type.
A global map will be drawn from the information in the database, it is
worth noting that this could lead to a large network map. Network maps
for individual devices are available showing the relationship with
other devices. Also you can Build Device Groups and those Device
Groups can be drawn with Network Map.
It is worth noting that the global map could lead to a large network
map that is slow to render and interact with. The network map on the
device neighbour page, or building device groups and using the device
group maps will be more usable on large networks.
## Network Map Configurator
[This link](https://visjs.github.io/vis-network/docs/network/) will
show you all the options and explain what they do.
You may also access the dynamic configuration interface [example
here](https://visjs.github.io/vis-network/examples/network/other/configuration.html)
from within LibreNMS by adding the following to config.php
```php
$config['network_map_vis_options'] = '{
"configure": { "enabled": true},
}';
```
### Note
You may want to disable the automatic page refresh while you're
tweaking your configuration, as the refresh will reset the dynamic
configuration UI to the values currently saved in config.php This can
be done by clicking on the Settings Icon then Refresh Pause.
### Configurator Output
Once you've achieved your desired map appearance, click the generate
options button at the bottom to be given the necessary parameters to
add to your config.php file. You will need to paste the generated
config into config.php the format will need to look something like
this. Note that the configurator will output the config with `var options`
you will need to strip them out and at the end of the config you need to
add an `}';` see the example below.
```php
$config['network_map_vis_options'] = '{
"nodes": {
"color": {
"background": "rgba(20,252,18,1)"
},
"font": {
"face": "tahoma"
},
"physics": false
},
"edges": {
"smooth": {
"forceDirection": "none"
}
},
"interaction": {
"hover": true,
"multiselect": true,
"navigationButtons": true
},
"manipulation": {
"enabled": true
},
"physics": {
"barnesHut": {
"avoidOverlap": 0.11
},
"minVelocity": 0.75
}
}';
```
![Example Network Map](/img/networkmap.png)
## Settings
The map display can be configured by altering the [VisJS-Config.md](Vis JS Options)

View File

@@ -0,0 +1,71 @@
# Vis JS Configuration
The [Network Maps](Network-Map.md) and [Dependency Maps](Dependency-Map.md) all use a common configuration for
the vis.js library, which affects the way the maps are rendered, as well
as the way that users can interact with the maps. This configuration can
be adjusted by following the instructions below.
[This link](https://visjs.github.io/vis-network/docs/network/) will
show you all the options and explain what they do.
You may also access the dynamic configuration interface [example
here](https://visjs.github.io/vis-network/examples/network/other/configuration.html)
from within LibreNMS by adding the following to config.php
```php
$config['network_map_vis_options'] = '{
"configure": { "enabled": true},
}';
```
### Note
You may want to disable the automatic page refresh while you're
tweaking your configuration, as the refresh will reset the dynamic
configuration UI to the values currently saved in config.php This can
be done by clicking on the Settings Icon then Refresh Pause.
### Configurator Output
Once you've achieved your desired map appearance, click the generate
options button at the bottom to be given the necessary parameters to
add to your config.php file. You will need to paste the generated
config into config.php the format will need to look something like
this. Note that the configurator will output the config with `var options`
you will need to strip them out and at the end of the config you need to
add an `}';` see the example below.
```php
$config['network_map_vis_options'] = '{
"nodes": {
"color": {
"background": "rgba(20,252,18,1)"
},
"font": {
"face": "tahoma"
},
"physics": false
},
"edges": {
"smooth": {
"forceDirection": "none"
}
},
"interaction": {
"hover": true,
"multiselect": true,
"navigationButtons": true
},
"manipulation": {
"enabled": true
},
"physics": {
"barnesHut": {
"avoidOverlap": 0.11
},
"minVelocity": 0.75
}
}';
```
![Example Network Map](/img/networkmap.png)

View File

@@ -45,31 +45,52 @@ We have two current mapping engines available:
zoom levels](https://wiki.openstreetmap.org/wiki/Zoom_levels).
- *Grouping radius*: Markers are grouped by area. This value define
the maximum size of grouping areas.
- *Show devices*: Show devices based on there status.
- *Show devices*: Show devices based on status.
Example Settings:
![Example World Map Settings](/img/world-map-widget-settings.png)
### Device Overview World Map Settings
If a device has a location with a valid latitude and logitude, the
device overview page will have a panel showing the device on a world
map. The following settings affect this map:
```bash
# Does the world map start opened, or does the user need to clivk to view
lnms config:set device_location_map_open false
# Do we show all other devices on the map as well
lnms config:set device_location_map_show_devices false
# Do we show a network map based on device dependencies
lnms config:set device_location_map_show_device_dependencies false
```
## Offline OpenStreet Map
If you can't access OpenStreet map directly you can run a local [tile
server](http://wiki.openstreetmap.org/wiki/Tile_servers). To specify a
different url you can set:
```php
$config['leaflet']['tile_url'] = 'localhost.com';
```bash
lnms config:set leaflet.tile_url 'localhost.com'
```
## Additional Leaflet config
```php
$config['map']['engine'] = "leaflet";
$config['leaflet']['default_lat'] = "51.981074";
$config['leaflet']['default_lng'] = "5.350342";
$config['leaflet']['default_zoom'] = 8;
// Device grouping radius in KM default 80KM
$config['leaflet']['group_radius'] = 1;
```bash
lnms config:set map.engine leaflet
lnms config:set leaflet.default_lat "51.981074"
lnms config:set leaflet.default_lng "5.350342"
lnms config:set leaflet.default_zoom 8
# Device grouping radius in KM default 80KM
lnms config:set leaflet.group_radius 1
# Enable network map on world map
lnms config:set network_map_show_on_worldmap true
# Use CDP/LLDP for network map, or device dependencies
lnms config:set network_map_worldmap_link_type xdp/depends
# Do not show devices that have notifications disabled
lnms config:set network_map_worldmap_show_disabled_alerts false
```
## Geocode engine config
@@ -98,13 +119,13 @@ Further custom options are available to load different maps of the
world, set default coordinates of where the map will zoom and the zoom
level by default. An example of this is:
```php
$config['map']['engine'] = "jquery-mapael";
$config['mapael']['default_map'] = 'mapael-maps/united_kingdom/united_kingdom.js';
$config['mapael']['map_width'] = 400;
$config['mapael']['default_lat'] = '50.898482';
$config['mapael']['default_lng'] = '-3.401402';
$config['mapael']['default_zoom'] = 20;
```bash
lnms config:set map.engine jquery-mapael
lnms config:set mapael.default_map 'mapael-maps/united_kingdom/united_kingdom.js'
lnms config:set mapael.map_width 400
lnms config:set mapael.default_lat '50.898482'
lnms config:set mapael.default_lng '-3.401402'
lnms config:set mapael.default_zoom 20
```
A list of maps can be found in ```html/js/maps/``` or ```html/js/mapael-maps/```.