mirror of
https://github.com/checktheroads/hyperglass
synced 2024-05-11 05:55:08 +00:00
complete restructure 2
This commit is contained in:
BIN
.flask_cache/1842a0ff0c5b4bdbdca506a3489cceed
Normal file
BIN
.flask_cache/1842a0ff0c5b4bdbdca506a3489cceed
Normal file
Binary file not shown.
BIN
.flask_cache/2029240f6d1128be89ddc32729463129
Normal file
BIN
.flask_cache/2029240f6d1128be89ddc32729463129
Normal file
Binary file not shown.
BIN
.flask_cache/2752cbed1c3342df07c3a4888c00dfdd
Normal file
BIN
.flask_cache/2752cbed1c3342df07c3a4888c00dfdd
Normal file
Binary file not shown.
BIN
.flask_cache/29247a84a0a0a129b1e0ca9dad5c0444
Normal file
BIN
.flask_cache/29247a84a0a0a129b1e0ca9dad5c0444
Normal file
Binary file not shown.
BIN
.flask_cache/3e24c65d0dab693559fbfe16164473e6
Normal file
BIN
.flask_cache/3e24c65d0dab693559fbfe16164473e6
Normal file
Binary file not shown.
BIN
.flask_cache/577248bd1fd6a03775f9c0a64b184799
Normal file
BIN
.flask_cache/577248bd1fd6a03775f9c0a64b184799
Normal file
Binary file not shown.
BIN
.flask_cache/74bc75c0764cbc805c70645cef1f026b
Normal file
BIN
.flask_cache/74bc75c0764cbc805c70645cef1f026b
Normal file
Binary file not shown.
BIN
.flask_cache/bbc9d74b6ce9914445ee9365223c30a1
Normal file
BIN
.flask_cache/bbc9d74b6ce9914445ee9365223c30a1
Normal file
Binary file not shown.
BIN
.flask_cache/f907d0eb682db4a28aa578fee970f056
Normal file
BIN
.flask_cache/f907d0eb682db4a28aa578fee970f056
Normal file
Binary file not shown.
@@ -1,18 +0,0 @@
|
||||
# Configuration
|
||||
|
||||
Hyperglass configuration files are stored `hyperglass/hyperglass/config`, in [TOML](https://github.com/toml-lang/toml) format.
|
||||
|
||||
Example configuration files are provided and end in `.example`. All example configuration files should be copied to their `.toml` name & extension. For example:
|
||||
|
||||
```console
|
||||
$ cd hyperglass/hyperglass/config
|
||||
$
|
||||
$ cp blacklist.toml.example blacklist.toml
|
||||
$ cp commands.toml.example commands.toml
|
||||
$ cp config.toml.example config.toml
|
||||
$ cp devices.toml.example devices.toml
|
||||
```
|
||||
|
||||
## `requires_ipv6_cidr.toml`
|
||||
|
||||
Some platforms (namely Cisco IOS) are unable to perform a BGP lookup by IPv6 host address (e.g. 2001:db8::1), but must perform the lookup by prefix (e.g. 2001:db8::/48). `requires_ipv6_cidr.toml` is a list (TOML array) of network operating systems that require this (in Netmiko format).
|
@@ -1,4 +1,4 @@
|
||||
Authentication parameters are stored in the `devices.toml` file, at `hyperglass/hyperglass/config/devices.toml`. The array of tables simply stores the username and password for a device. SSH Key authentication is not yet supported.
|
||||
Authentication parameters are stored in the `devices.toml` file, at `hyperglass/hyperglass/configuration/devices.toml`. The array of tables simply stores the username and password for a device. SSH Key authentication is not yet supported.
|
||||
|
||||
Example:
|
||||
|
||||
|
@@ -1,4 +1,4 @@
|
||||
Blacklisted querys are defined in `hyperglass/hyperglass/config/blacklist.toml`.
|
||||
Blacklisted querys are defined in `hyperglass/hyperglass/configuration/blacklist.toml`.
|
||||
|
||||
The blacklist is a simple TOML array (list) of host IPs or prefixes that you do not want end users to be able to query. For example, if you want to prevent users from looking up 198.18.0.0/15 or any contained host or prefix, you can add it to the blacklist:
|
||||
|
||||
|
@@ -10,7 +10,7 @@
|
||||
}
|
||||
</style>
|
||||
|
||||
From `hyperglass/hyperglass/config/config.toml`:
|
||||
From `hyperglass/hyperglass/configuration/config.toml`:
|
||||
|
||||
### site_title
|
||||
|
||||
@@ -62,7 +62,7 @@ See [primary_asn](#primary_asn) parameter.
|
||||
| ------- | ------------- |
|
||||
| Boolean | `True` |
|
||||
|
||||
Enables or disables entire footer element, which contains text defined in `hyperglass/hyperglass/templates/footer.md`.
|
||||
Enables or disables entire footer element, which contains text defined in `hyperglass/hyperglass/render/templates/footer.md`.
|
||||
|
||||
### enable_credit
|
||||
|
||||
|
@@ -1,4 +1,4 @@
|
||||
Commands are defined in `hyperglass/hyperglass/config/commands.toml`. Formatted as a nested array of tables, each table defines the commands that will be used to execute the queries on the routers.
|
||||
Commands are defined in `hyperglass/hyperglass/configuration/commands.toml`. Formatted as a nested array of tables, each table defines the commands that will be used to execute the queries on the routers.
|
||||
|
||||
Each table contains three nested tables:
|
||||
|
||||
|
@@ -1,4 +1,4 @@
|
||||
Devices/routers are defined in `hyperglass/hyperglass/config/devices.toml`. `devices.toml` is effectively an array of hash tables/dictionaries/key value pairs:
|
||||
Devices/routers are defined in `hyperglass/hyperglass/configuration/devices.toml`. `devices.toml` is effectively an array of hash tables/dictionaries/key value pairs:
|
||||
|
||||
```toml
|
||||
[[router]]
|
||||
|
@@ -1,4 +1,4 @@
|
||||
From `hyperglass/hyperglass/config/config.toml`:
|
||||
From `hyperglass/hyperglass/configuration/config.toml`:
|
||||
|
||||
### primary_asn
|
||||
|
||||
|
16
docs/configuration/index.md
Normal file
16
docs/configuration/index.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# Configuration
|
||||
|
||||
Hyperglass configuration files are stored in `hyperglass/hyperglass/configuration/`, in [TOML](https://github.com/toml-lang/toml) format.
|
||||
|
||||
```console
|
||||
hyperglass/configuration/
|
||||
├── blacklist.toml
|
||||
├── commands.toml
|
||||
├── configuration.toml
|
||||
├── devices.toml
|
||||
└── requires_ipv6_cidr.toml
|
||||
```
|
||||
|
||||
## `requires_ipv6_cidr.toml`
|
||||
|
||||
Some platforms (namely Cisco IOS) are unable to perform a BGP lookup by IPv6 host address (e.g. 2001:db8::1), but must perform the lookup by prefix (e.g. 2001:db8::/48). `requires_ipv6_cidr.toml` is a list (TOML array) of network operating systems that require this (in Netmiko format).
|
@@ -1,4 +1,4 @@
|
||||
Proxy servers are defined in `hyperglass/hyperglass/config/devices.toml`. Each proxy definition is a unique TOML table, for example:
|
||||
Proxy servers are defined in `hyperglass/hyperglass/configuration/devices.toml`. Each proxy definition is a unique TOML table, for example:
|
||||
|
||||
```toml
|
||||
[proxy.'jumpbox1']
|
||||
@@ -37,7 +37,7 @@ Plain text password for SSH authentication to the proxy server/jumpbox.
|
||||
|
||||
Device type/vendor name as recognized by [Netmiko](https://github.com/ktbyers/netmiko). See [supported device types](#supported-device-types) for a full list.
|
||||
|
||||
!!! info "Compatibility"
|
||||
!!! note "Compatibility"
|
||||
Hyperglass has only been tested with `linux_ssh` as of this writing.
|
||||
|
||||
#### ssh_command
|
||||
|
@@ -1,4 +1,4 @@
|
||||
More than likely, you'll want to "lock down" what commands can be executed with the credentials you've provided in `hyperglass/hyperglass/config/devices.toml`. It is **strongly** recommended to use a low privilege read only account and not your full administrator account. Even though Hyperglass is coded to only run certain commands to begin with, you're more than likely still exposing the server Hyperglass runs on to the internet, and on that server is a plain text file with your router's credentials in it. Take precautions.
|
||||
More than likely, you'll want to "lock down" what commands can be executed with the credentials you've provided in `hyperglass/hyperglass/configuration/devices.toml`. It is **strongly** recommended to use a low privilege read only account and not your full administrator account. Even though Hyperglass is coded to only run certain commands to begin with, you're more than likely still exposing the server Hyperglass runs on to the internet, and on that server is a plain text file with your router's credentials in it. Take precautions.
|
||||
|
||||
# Creating Restricted Accounts
|
||||
|
||||
@@ -18,7 +18,7 @@ parser view hyperglass
|
||||
username hyperglass privilege 15 view hyperglass secret <secret>
|
||||
```
|
||||
|
||||
!!! info "Terminal"
|
||||
!!! note "Terminal"
|
||||
The `terminal length` and `terminal width` commands are required by Netmiko for session handling. If you remove these, Hyperglass will not work.
|
||||
|
||||
## Cisco IOS-XR
|
||||
|
@@ -13,26 +13,57 @@ Under the main `hyperglass/hyperglass/` directory, the following basic structure
|
||||
```
|
||||
hyperglass/
|
||||
├── __init__.py
|
||||
├── app.py
|
||||
├── cmd_construct.py
|
||||
├── cmd_execute.py
|
||||
├── cmd_parser.py
|
||||
├── config/
|
||||
├── command/
|
||||
├── configuration/
|
||||
├── gunicorn_config.py
|
||||
├── hyperglass.py
|
||||
├── manage.py
|
||||
├── render/
|
||||
├── static/
|
||||
├── templates/
|
||||
├── templates.py
|
||||
└── vars.py
|
||||
└── wsgi.py
|
||||
```
|
||||
|
||||
### Scripts
|
||||
|
||||
#### `hyperglass.py`
|
||||
|
||||
Main Flask application. Passes input to the `command.execute` module.
|
||||
|
||||
#### `manage.py`
|
||||
|
||||
Management script for perfoming one-off actions. For now, the only action implemented is a manual clearing of the Flask-cache cache. To clear the cache, run `python3 manage.py --clearcache`.
|
||||
|
||||
### Directories
|
||||
|
||||
#### config
|
||||
|
||||
The `config/` directory contains all TOML config files used by Hyperglass:
|
||||
#### command/
|
||||
|
||||
```
|
||||
hyperglass/config/
|
||||
hyperglass/command/
|
||||
├── __init__.py
|
||||
├── construct.py
|
||||
├── execute.py
|
||||
└── parse.py
|
||||
```
|
||||
|
||||
##### `execute.py`
|
||||
|
||||
Matches router name to router IP, OS, and credentials. Passes data to `cmd_construct.py`, uses the results to execute the Netmiko action. Also performs error handling in the event of a [blacklist](/configuration/blacklist) match.
|
||||
|
||||
##### `construct.py`
|
||||
|
||||
Constructs full commands to run on routers from `hyperglass/hyperglass/config/commands.toml`. Also performs error handling in the event of input errors.
|
||||
|
||||
##### `parser.py`
|
||||
|
||||
Parses output before presentation to the user. For the time being, only BGP output from Cisco IOS is parsed. This is because for BGP Community and AS_PATH lookups, Cisco IOS returns results for *all* address families, including VPNv4. This script ensures that only IPv4 and IPv6 address family output is returned.
|
||||
|
||||
#### configuration/
|
||||
|
||||
The `configuration/` directory contains all TOML config files used by Hyperglass:
|
||||
|
||||
```
|
||||
hyperglass/configuration/
|
||||
├── __init__.py
|
||||
├── blacklist.toml
|
||||
├── commands.toml
|
||||
├── config.toml
|
||||
@@ -40,25 +71,27 @@ hyperglass/config/
|
||||
└── requires_ipv6_cidr.toml
|
||||
```
|
||||
|
||||
#### static
|
||||
As a module, `configuration` imports configuration from TOML configuration files, defines default values, and exports each as a variable that can be called in other scripts.
|
||||
|
||||
#### static/
|
||||
|
||||
The `static/` directory contains all static HTML/CSS/JS files used for serving the site:
|
||||
|
||||
```
|
||||
hyperglass/static/
|
||||
├── css
|
||||
├── css/
|
||||
│ ├── hyperglass.css
|
||||
│ └── icofont
|
||||
├── images
|
||||
├── images/
|
||||
│ ├── brand.svg
|
||||
│ ├── favicon
|
||||
│ ├── hyperglass-dark.png
|
||||
│ └── hyperglass-light.png
|
||||
├── js
|
||||
├── js/
|
||||
│ ├── hyperglass.js
|
||||
│ ├── jquery-3.4.0.min.js
|
||||
│ └── jquery-3.4.0.min.map
|
||||
└── sass
|
||||
└── sass/
|
||||
├── base
|
||||
├── components
|
||||
├── custom
|
||||
@@ -73,12 +106,14 @@ hyperglass/static/
|
||||
- `css/icofont/` Completely free alternative to FontAwesome - [Icofont](https://icofont.com/).
|
||||
- `js/hyerpglass.js` Basic Javascript helper to perform AJAX queries necessary to pull in dynamic information and render content.
|
||||
|
||||
#### templates
|
||||
#### render/
|
||||
|
||||
The `templates/` directory contains HTML and Sass Jinja2 templates:
|
||||
The `render/` directory contains the `render` module, which renders HTML and Sass templates, compiles Sass to CSS.
|
||||
|
||||
```
|
||||
templates/
|
||||
hyperglass/render/
|
||||
├── __init__.py
|
||||
└── templates/
|
||||
├── 415.html
|
||||
├── 429.html
|
||||
├── base.html
|
||||
@@ -88,6 +123,8 @@ templates/
|
||||
└── index.html
|
||||
```
|
||||
|
||||
`render/templates/` contains the Jinja2 templates themselves:
|
||||
|
||||
- `415.html` General error page template.
|
||||
- `429.html` Site load rate limit page.
|
||||
- `base.html` Base template inherited by all other templates. Contains HTML `head`, JavaScript, etc.
|
||||
@@ -95,33 +132,3 @@ templates/
|
||||
- `footer.md` Text that appears in the footer, if enabled. Markdown will be rendered as HTML.
|
||||
- `hyperglass.scss` Generates SCSS file for Bulma and local customizations.
|
||||
- `index.html` Main page template.
|
||||
|
||||
### Scripts
|
||||
|
||||
#### `app.py`
|
||||
|
||||
Main Flask application. Passes input to `cmd_execute.py`
|
||||
|
||||
#### `cmd_execute.py`
|
||||
|
||||
Matches router name to router IP, OS, and credentials. Passes data to `cmd_construct.py`, uses the results to execute the Netmiko action. Also performs error handling in the event of a [blacklist](/configuration/blacklist) match.
|
||||
|
||||
#### `cmd_construct.py`
|
||||
|
||||
Constructs full commands to run on routers from `hyperglass/hyperglass/config/commands.toml`. Also performs error handling in the event of input errors.
|
||||
|
||||
#### `cmd_parser.py`
|
||||
|
||||
Parses output before presentation to the user. For the time being, only BGP output from Cisco IOS is parsed. This is because for BGP Community and AS_PATH lookups, Cisco IOS returns results for *all* address families, including VPNv4. This script ensures that only IPv4 and IPv6 address family output is returned.
|
||||
|
||||
#### `manage.py`
|
||||
|
||||
Management script for perfoming one-off actions. For now, the only action implemented is a manual clearing of the Flask-cache cache.
|
||||
|
||||
#### `templates.py`
|
||||
|
||||
Renders HTML and Sass templates, compiles Sass to CSS.
|
||||
|
||||
#### `vars.py`
|
||||
|
||||
Imports configuration from TOML configuration files, defines default values, and exports each as a variable that can be called in other scripts.
|
||||
|
@@ -28,7 +28,7 @@ $ pip3 install -r requirements.txt
|
||||
## Clone Example Configuration Files
|
||||
|
||||
```
|
||||
$ cd /opt/hyperglass/hyperglass/config/
|
||||
$ cd /opt/hyperglass/hyperglass/configuration/
|
||||
$ for f in *.example; do cp $f `basename $f .example`; done;
|
||||
```
|
||||
|
||||
|
@@ -11,4 +11,4 @@ Gunicorn is a WSGI server written in Python.
|
||||
|
||||
## Configure
|
||||
|
||||
Locate your `gunicorn` executable with `which gunicorn`
|
||||
Locate your `gunicorn` executable with `which gunicorn`.
|
||||
|
3
hyperglass/command/__init__.py
Normal file
3
hyperglass/command/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
from hyperglass.command import execute
|
||||
from hyperglass.command import construct
|
||||
from hyperglass.command import parse
|
@@ -1,27 +1,30 @@
|
||||
import sys
|
||||
import logging
|
||||
import toml
|
||||
import re
|
||||
import sys
|
||||
import toml
|
||||
import logging
|
||||
from netaddr import *
|
||||
from loguru import logger
|
||||
|
||||
# Local imports
|
||||
import vars
|
||||
from hyperglass import configuration
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
# Load TOML config file
|
||||
devices = toml.load(open("./config/devices.toml"))
|
||||
devices = configuration.devices()
|
||||
|
||||
# Load TOML commands file
|
||||
commands = toml.load(open("./config/commands.toml"))
|
||||
commands = configuration.commands()
|
||||
|
||||
# Filter config to router list
|
||||
routers_list = devices["router"]
|
||||
|
||||
logger.add(sys.stderr)
|
||||
|
||||
# Receives JSON from Flask, constucts the command that will be passed to the router
|
||||
# Also handles input validation & error handling
|
||||
def cmd_construct(router, cmd, ipprefix):
|
||||
inputParams = router, cmd, ipprefix
|
||||
log.warning(*inputParams)
|
||||
def construct(router, cmd, ipprefix):
|
||||
input_params = (router, cmd, ipprefix)
|
||||
logger.info(*input_params)
|
||||
try:
|
||||
# Loop through routers config file, match input router with configured routers, set variables
|
||||
for r in routers_list:
|
||||
@@ -44,8 +47,8 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
if cmd == "Query Type":
|
||||
msg = "You must select a query type."
|
||||
code = 415
|
||||
log.error(msg, code, *inputParams)
|
||||
return (msg, code, *inputParams)
|
||||
logger.error(msg, code, *input_params)
|
||||
return (msg, code)
|
||||
# BGP Community Query
|
||||
elif cmd in ["bgp_community"]:
|
||||
# Extended Communities, new-format
|
||||
@@ -57,7 +60,7 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 200
|
||||
log.warning(
|
||||
logger.warning(
|
||||
msg, code, router, type, command
|
||||
)
|
||||
return (msg, code, router, type, command)
|
||||
@@ -70,7 +73,7 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 200
|
||||
log.warning(
|
||||
logger.warning(
|
||||
msg, code, router, type, command
|
||||
)
|
||||
return (msg, code, router, type, command)
|
||||
@@ -86,7 +89,7 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 200
|
||||
log.warning(
|
||||
logger.warning(
|
||||
msg, code, router, type, command
|
||||
)
|
||||
return (msg, code, router, type, command)
|
||||
@@ -95,8 +98,8 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 415
|
||||
log.error(msg, code, *inputParams)
|
||||
return (msg, code, *inputParams)
|
||||
logger.error(msg, code, *input_params)
|
||||
return (msg, code)
|
||||
# BGP AS_PATH Query
|
||||
elif cmd in ["bgp_aspath"]:
|
||||
if re.match(".*", ipprefix):
|
||||
@@ -107,7 +110,7 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 200
|
||||
log.warning(
|
||||
logger.warning(
|
||||
msg, code, router, type, command
|
||||
)
|
||||
return (msg, code, router, type, command)
|
||||
@@ -116,8 +119,8 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 415
|
||||
log.error(msg, code, *inputParams)
|
||||
return (msg, code, *inputParams)
|
||||
logger.error(msg, code, *input_params)
|
||||
return (msg, code)
|
||||
# BGP Route Query
|
||||
elif cmd in ["bgp_route"]:
|
||||
try:
|
||||
@@ -130,7 +133,7 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 200
|
||||
log.warning(
|
||||
logger.warning(
|
||||
msg, code, router, type, command
|
||||
)
|
||||
return (
|
||||
@@ -149,7 +152,7 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 200
|
||||
log.warning(
|
||||
logger.warning(
|
||||
msg, code, router, type, command
|
||||
)
|
||||
return (
|
||||
@@ -165,8 +168,8 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 415
|
||||
log.error(msg, code, *inputParams)
|
||||
return (msg, code, *inputParams)
|
||||
logger.error(msg, code, *input_params)
|
||||
return (msg, code)
|
||||
# Ping/Traceroute
|
||||
elif cmd in ["ping", "traceroute"]:
|
||||
try:
|
||||
@@ -181,7 +184,7 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 200
|
||||
log.warning(
|
||||
logger.warning(
|
||||
msg, code, router, type, command
|
||||
)
|
||||
return (
|
||||
@@ -202,7 +205,7 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 200
|
||||
log.warning(
|
||||
logger.warning(
|
||||
msg, code, router, type, command
|
||||
)
|
||||
return (
|
||||
@@ -217,15 +220,15 @@ def cmd_construct(router, cmd, ipprefix):
|
||||
i=ipprefix
|
||||
)
|
||||
code = 415
|
||||
log.error(msg, code, *inputParams)
|
||||
return (msg, code, *inputParams)
|
||||
logger.error(msg, code, *input_params)
|
||||
return (msg, code)
|
||||
else:
|
||||
msg = "Command {i} not found.".format(i=cmd)
|
||||
code = 415
|
||||
log.error(msg, code, *inputParams)
|
||||
return (msg, code, *inputParams)
|
||||
logger.error(msg, code, *input_params)
|
||||
return (msg, code)
|
||||
except:
|
||||
error_msg = log.error(
|
||||
error_msg = logger.error(
|
||||
"Input router IP {router} does not match the configured router IP of {ip}".format(
|
||||
router=router, ip=r["address"]
|
||||
)
|
@@ -1,17 +1,15 @@
|
||||
import sys
|
||||
import logging
|
||||
import toml
|
||||
import time
|
||||
from netmiko import ConnectHandler
|
||||
from netmiko import redispatch
|
||||
from netaddr import *
|
||||
from cmd_construct import cmd_construct
|
||||
import vars
|
||||
import cmd_parser as parser
|
||||
from loguru import logger
|
||||
from netmiko import redispatch
|
||||
from netmiko import ConnectHandler
|
||||
from hyperglass import configuration
|
||||
from hyperglass.command import construct
|
||||
from hyperglass.command import parse
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
# Load TOML devices file
|
||||
devices = toml.load(open("./config/devices.toml"))
|
||||
devices = configuration.devices()
|
||||
# Filter config to router list
|
||||
routers_list = devices["router"]
|
||||
# Filter config to credential list
|
||||
@@ -19,14 +17,16 @@ credentials_list = devices["credential"]
|
||||
# Filter config to proxy servers
|
||||
proxies_list = devices["proxy"]
|
||||
|
||||
blacklist_config = toml.load(open("./config/blacklist.toml"))
|
||||
blacklist_config = configuration.blacklist()
|
||||
blacklist = IPSet(blacklist_config["blacklist"])
|
||||
|
||||
general_error = "Error connecting to device."
|
||||
|
||||
logger.add(sys.stderr)
|
||||
|
||||
def cmd_execute(lg_data):
|
||||
log.warning(lg_data)
|
||||
|
||||
def execute(lg_data):
|
||||
logger.info(lg_data)
|
||||
# Check POST data from JS, if location matches a configured router's
|
||||
# location, use the router's configured IP address to connect
|
||||
for r in routers_list:
|
||||
@@ -39,17 +39,17 @@ def cmd_execute(lg_data):
|
||||
if IPNetwork(lg_data["ipprefix"]).ip in blacklist:
|
||||
msg = "{i} is not allowed.".format(i=lg_data["ipprefix"])
|
||||
code = 405
|
||||
log.error(msg, code, lg_data)
|
||||
logger.error(msg, code, lg_data)
|
||||
return (msg, code, lg_data)
|
||||
# If netaddr library throws an exception, return a user-facing error.
|
||||
except:
|
||||
msg = "{i} is not a valid IP Address.".format(i=lg_data["ipprefix"])
|
||||
code = 415
|
||||
log.error(msg, code, lg_data)
|
||||
logger.error(msg, code, lg_data)
|
||||
return (msg, code, lg_data)
|
||||
# Send "clean" request to cmd_construct to build the command that will be sent to the router
|
||||
# Send "clean" request to constructor to build the command that will be sent to the router
|
||||
print(lg_router_address)
|
||||
msg, status, router, type, command = cmd_construct(
|
||||
msg, status, router, type, command = construct.construct(
|
||||
lg_router_address, lg_data["cmd"], lg_data["ipprefix"]
|
||||
)
|
||||
# Loop through proxy config, match configured proxy name for each router with a configured proxy
|
||||
@@ -71,7 +71,7 @@ def cmd_execute(lg_data):
|
||||
else:
|
||||
msg = "Router does not have a proxy configured."
|
||||
code = 415
|
||||
log.error(msg, code, lg_data)
|
||||
logger.error(msg, code, lg_data)
|
||||
return (msg, code, lg_data)
|
||||
|
||||
# Matches router with configured credential
|
||||
@@ -90,7 +90,7 @@ def cmd_execute(lg_data):
|
||||
else:
|
||||
msg = "Credential {i} does not exist".format(i=configured_credential)
|
||||
code = 415
|
||||
log.error(msg, code, lg_data)
|
||||
logger.error(msg, code, lg_data)
|
||||
return (general_error, code, lg_data)
|
||||
|
||||
# Connect to the router via netmiko library, return the command output
|
||||
@@ -102,7 +102,7 @@ def cmd_execute(lg_data):
|
||||
except:
|
||||
msg = "Unable to reach target {l}".format(l=lg_data["router"])
|
||||
code = 415
|
||||
log.error(msg, code, lg_data)
|
||||
logger.error(msg, code, lg_data)
|
||||
return (general_error, code, lg_data)
|
||||
|
||||
# Connect to the proxy server via netmiko library, then log into the router
|
||||
@@ -141,7 +141,7 @@ def cmd_execute(lg_data):
|
||||
p=nm_proxy["host"], d=nm_host["host"]
|
||||
)
|
||||
code = 415
|
||||
log.error(msg, code, lg_data)
|
||||
logger.error(msg, code, lg_data)
|
||||
return (general_error, code, lg_data)
|
||||
|
||||
nm_host = {
|
||||
@@ -163,11 +163,11 @@ def cmd_execute(lg_data):
|
||||
try:
|
||||
if connection_proxied is True:
|
||||
output_proxied = getOutputProxy(configured_proxy)
|
||||
parsed_output = parser.parse(output_proxied, type, lg_data["cmd"])
|
||||
parsed_output = parse.parse(output_proxied, type, lg_data["cmd"])
|
||||
return parsed_output, status, router, type, command
|
||||
elif connection_proxied is False:
|
||||
output_direct = getOutputDirect()
|
||||
parsed_output = parser.parse(output_direct, type, lg_data["cmd"])
|
||||
parsed_output = parse.parse(output_direct, type, lg_data["cmd"])
|
||||
return parsed_output, status, router, type, command
|
||||
except:
|
||||
raise
|
@@ -1,19 +1,47 @@
|
||||
import toml
|
||||
import os
|
||||
import math
|
||||
import toml
|
||||
|
||||
dir = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
|
||||
def blacklist():
|
||||
f = os.path.join(dir, "blacklist.toml")
|
||||
t = toml.load(f)
|
||||
return t
|
||||
|
||||
|
||||
def commands():
|
||||
f = os.path.join(dir, "commands.toml")
|
||||
t = toml.load(f)
|
||||
return t
|
||||
|
||||
|
||||
def configuration():
|
||||
f = os.path.join(dir, "configuration.toml")
|
||||
t = toml.load(f)
|
||||
return t
|
||||
|
||||
|
||||
def devices():
|
||||
f = os.path.join(dir, "devices.toml")
|
||||
t = toml.load(f)
|
||||
return t
|
||||
|
||||
|
||||
def requires_ipv6_cidr():
|
||||
f = os.path.join(dir, "requires_ipv6_cidr.toml")
|
||||
t = toml.load(f)
|
||||
return t
|
||||
|
||||
# Load TOML config file
|
||||
config = toml.load(open("./config/config.toml"))
|
||||
|
||||
# Filter config to branding variables
|
||||
branding = config["branding"]
|
||||
branding = configuration()["branding"]
|
||||
|
||||
# Filter config to general variables
|
||||
general = config["general"]
|
||||
general = configuration()["general"]
|
||||
|
||||
# Load TOML devices file
|
||||
devices = toml.load(open("./config/devices.toml"))
|
||||
# Filter config to router list
|
||||
routers_list = devices["router"]
|
||||
routers_list = devices()["router"]
|
||||
|
||||
|
||||
class dev:
|
@@ -3,5 +3,6 @@ import multiprocessing
|
||||
command = "/usr/local/bin/gunicorn"
|
||||
pythonpath = "/opt/hyperglass/hyperglass"
|
||||
bind = "[::1]:8001"
|
||||
workers = multiprocessing.cpu_count() * 2
|
||||
workers = 1 # multiprocessing.cpu_count() * 2
|
||||
user = "www-data"
|
||||
timeout = 60
|
@@ -1,46 +1,45 @@
|
||||
# Module Imports
|
||||
import logging
|
||||
from flask import Flask, request, Response, jsonify, flash
|
||||
from flask_limiter import Limiter
|
||||
from flask_limiter.util import get_remote_address
|
||||
from flask_caching import Cache
|
||||
import sys
|
||||
import json
|
||||
import toml
|
||||
from loguru import logger
|
||||
from flask import Flask, request, Response, jsonify, flash
|
||||
from flask_caching import Cache
|
||||
from flask_limiter import Limiter
|
||||
from flask_limiter.util import get_remote_address
|
||||
|
||||
# Local Imports
|
||||
import vars
|
||||
from cmd_execute import cmd_execute
|
||||
import templates
|
||||
from hyperglass import render
|
||||
from hyperglass import configuration
|
||||
from hyperglass.command import execute
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
logger.add(sys.stderr)
|
||||
|
||||
# Load TOML config file
|
||||
devices = toml.load(open("./config/devices.toml"))
|
||||
devices = configuration.devices()
|
||||
# Filter config file to list of routers & subsequent configurations
|
||||
routers_list = devices["router"]
|
||||
# Filter config file to array of operating systems that require IPv6 BGP lookups in CIDR format
|
||||
ipv6_cidr_list = toml.load(open("./config/requires_ipv6_cidr.toml"))[
|
||||
"requires_ipv6_cidr"
|
||||
]
|
||||
ipv6_cidr_list = configuration.requires_ipv6_cidr()
|
||||
# Main Flask definition
|
||||
app = Flask(__name__, static_url_path="/static")
|
||||
|
||||
# Flask-Limiter Config
|
||||
rate_limit_query = vars.gen.rate_limit_query() + " per minute"
|
||||
rate_limit_site = vars.gen.rate_limit_site() + "per minute"
|
||||
rate_limit_query = configuration.gen.rate_limit_query() + " per minute"
|
||||
rate_limit_site = configuration.gen.rate_limit_site() + "per minute"
|
||||
limiter = Limiter(app, key_func=get_remote_address, default_limits=[rate_limit_site])
|
||||
|
||||
# Render Main Flask-Limiter Error Message
|
||||
@app.errorhandler(429)
|
||||
def error429(e):
|
||||
"""Renders full error page for too many site queries"""
|
||||
html = templates.html.renderTemplate("429")
|
||||
html = render.html.renderTemplate("429")
|
||||
return html, 429
|
||||
|
||||
|
||||
def error415():
|
||||
"""Renders full error page for generic errors"""
|
||||
html = templates.html.renderTemplate("415")
|
||||
html = render.html.renderTemplate("415")
|
||||
return html, 415
|
||||
|
||||
|
||||
@@ -59,8 +58,8 @@ cache = Cache(
|
||||
app,
|
||||
config={
|
||||
"CACHE_TYPE": "filesystem",
|
||||
"CACHE_DIR": vars.gen.cache_directory(),
|
||||
"CACHE_DEFAULT_TIMEOUT": vars.gen.cache_timeout(),
|
||||
"CACHE_DIR": configuration.gen.cache_directory(),
|
||||
"CACHE_DEFAULT_TIMEOUT": configuration.gen.cache_timeout(),
|
||||
},
|
||||
)
|
||||
|
||||
@@ -79,14 +78,14 @@ def clearCache():
|
||||
@limiter.limit(rate_limit_site)
|
||||
def site():
|
||||
"""Main front-end web application"""
|
||||
html = templates.html.renderTemplate("index")
|
||||
html = render.html.renderTemplate("index")
|
||||
return html
|
||||
|
||||
|
||||
# Test route for various tests
|
||||
@app.route("/test", methods=["GET"])
|
||||
def testRoute():
|
||||
html = templates.html.renderTemplate("test")
|
||||
html = render.html.renderTemplate("test")
|
||||
return html
|
||||
|
||||
|
||||
@@ -130,16 +129,18 @@ def lg():
|
||||
cache_key = str(lg_data)
|
||||
# Check if cached entry exists
|
||||
if cache.get(cache_key) is None:
|
||||
cache_value = cmd_execute(lg_data)
|
||||
log.debug(cache_value[1:])
|
||||
cache_value = execute.execute(lg_data)
|
||||
logger.info(cache_value[1:])
|
||||
value_output = cache_value[0]
|
||||
value_code = cache_value[1]
|
||||
value_params = cache_value[2:]
|
||||
log.debug("No cache match for: ", cache_key, "\nAdding cache entry...")
|
||||
logger.info("No cache match for: {cache_key}".format(cache_key=cache_key))
|
||||
# If it doesn't, create a cache entry
|
||||
try:
|
||||
cache.set(cache_key, value_output)
|
||||
log.debug("\nAdded cache entry: ", *value_params)
|
||||
logger.info(
|
||||
"Added cache entry: {value_params}".format(value_params=value_params)
|
||||
)
|
||||
except:
|
||||
raise RuntimeError("Unable to add output to cache.", 415, *value_params)
|
||||
# If 200, return output
|
||||
@@ -150,7 +151,11 @@ def lg():
|
||||
return Response(cache.get(cache_key), value_code)
|
||||
# If it does, return the cached entry
|
||||
else:
|
||||
log.debug("Cache match for: ", cache_key, "\nReturning cached entry...")
|
||||
logger.info(
|
||||
"Cache match for: {cache_key}, returning cached entry...".format(
|
||||
cache_key=cache_key
|
||||
)
|
||||
)
|
||||
try:
|
||||
return cache.get(cache_key)
|
||||
except:
|
||||
@@ -160,8 +165,3 @@ def lg():
|
||||
)
|
||||
# Upon exception, render generic error
|
||||
return Response(errorGeneral(id))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
templates.css.renderTemplate()
|
||||
app.run(host="0.0.0.0", debug=vars.gen.debug(), port=5000)
|
@@ -1,21 +1,17 @@
|
||||
import os
|
||||
import sys
|
||||
import app
|
||||
import logging
|
||||
from loguru import logger
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
# from hyperglass.hyperglass import app
|
||||
|
||||
|
||||
def clearcache():
|
||||
try:
|
||||
app.clearCache()
|
||||
except:
|
||||
raise
|
||||
logger.add(sys.stderr)
|
||||
|
||||
|
||||
for arg in sys.argv:
|
||||
if arg == "clearcache":
|
||||
try:
|
||||
clearcache()
|
||||
hyperglass.hyperglass.app.clearcache()
|
||||
logger.info("Successfully cleared cache.")
|
||||
except:
|
||||
raise
|
||||
logger.error("Failed to clear cache.")
|
||||
|
107
hyperglass/render/__init__.py
Normal file
107
hyperglass/render/__init__.py
Normal file
@@ -0,0 +1,107 @@
|
||||
import os
|
||||
import sass
|
||||
import codecs
|
||||
import jinja2
|
||||
import subprocess
|
||||
from markdown2 import Markdown
|
||||
from flask import render_template
|
||||
|
||||
from hyperglass import configuration
|
||||
|
||||
dir = os.path.dirname(os.path.abspath(__file__))
|
||||
file_loader = jinja2.FileSystemLoader(dir)
|
||||
env = jinja2.Environment(loader=file_loader)
|
||||
|
||||
# Converts templates/footer.md from Markdown to HTML
|
||||
md = Markdown()
|
||||
# footer_file = os.path.join(dir, "templates/footer.md")
|
||||
footer_template = env.get_template("templates/footer.md")
|
||||
footer_jinja = footer_template.render(title=configuration.brand.title())
|
||||
footer = footer_jinja
|
||||
|
||||
# Functions for rendering Jinja2 templates & importing variables
|
||||
|
||||
|
||||
class html:
|
||||
def renderTemplate(t):
|
||||
if t == "index":
|
||||
template = env.get_template("templates/index.html")
|
||||
elif t == "429":
|
||||
template = env.get_template("templates/429.html")
|
||||
elif t == "415":
|
||||
template = env.get_template("templates/415.html")
|
||||
elif t == "test":
|
||||
template = env.get_template("templates/429.html")
|
||||
return template.render(
|
||||
# General
|
||||
primary_asn=configuration.gen.primary_asn(),
|
||||
google_analytics=configuration.gen.google_analytics(),
|
||||
enable_recaptcha=configuration.gen.enable_recaptcha(),
|
||||
enable_bgp_route=configuration.gen.enable_bgp_route(),
|
||||
enable_bgp_community=configuration.gen.enable_bgp_community(),
|
||||
enable_bgp_aspath=configuration.gen.enable_bgp_aspath(),
|
||||
enable_ping=configuration.gen.enable_ping(),
|
||||
enable_traceroute=configuration.gen.enable_traceroute(),
|
||||
cache_timeout=configuration.gen.cache_timeout(),
|
||||
message_rate_limit_query=configuration.gen.message_rate_limit_query(),
|
||||
# Branding
|
||||
site_title=configuration.brand.site_title(),
|
||||
title=configuration.brand.title(),
|
||||
subtitle=configuration.brand.subtitle(),
|
||||
title_mode=configuration.brand.title_mode(),
|
||||
color_hero=configuration.brand.color_hero(),
|
||||
enable_credit=configuration.brand.enable_credit(),
|
||||
enable_footer=configuration.brand.enable_footer(),
|
||||
footer_content=md.convert(footer),
|
||||
logo_path=configuration.brand.logo_path(),
|
||||
logo_width=configuration.brand.logo_width(),
|
||||
placeholder_prefix=configuration.brand.placeholder_prefix(),
|
||||
show_peeringdb=configuration.brand.show_peeringdb(),
|
||||
text_results=configuration.brand.text_results(),
|
||||
text_location=configuration.brand.text_location(),
|
||||
text_cache=configuration.brand.text_cache(),
|
||||
text_415_title=configuration.brand.text_415_title(),
|
||||
text_415_subtitle=configuration.brand.text_415_subtitle(),
|
||||
text_415_button=configuration.brand.text_415_button(),
|
||||
text_help_bgp_route=configuration.brand.text_help_bgp_route(),
|
||||
text_help_bgp_community=configuration.brand.text_help_bgp_community(),
|
||||
text_help_bgp_aspath=configuration.brand.text_help_bgp_aspath(),
|
||||
text_help_ping=configuration.brand.text_help_ping(),
|
||||
text_help_traceroute=configuration.brand.text_help_traceroute(),
|
||||
text_limiter_title=configuration.brand.text_limiter_title(),
|
||||
text_limiter_subtitle=configuration.brand.text_limiter_subtitle(),
|
||||
# Devices
|
||||
device_networks=configuration.dev.networks(),
|
||||
# device_location=configuration.dev.location(),
|
||||
device_name=configuration.dev.name(),
|
||||
)
|
||||
|
||||
|
||||
class css:
|
||||
def renderTemplate():
|
||||
try:
|
||||
template = env.get_template("templates/hyperglass.scss")
|
||||
rendered_output = template.render(
|
||||
color_btn_submit=configuration.brand.color_btn_submit(),
|
||||
color_progressbar=configuration.brand.color_progressbar(),
|
||||
color_tag_loctitle=configuration.brand.color_tag_loctitle(),
|
||||
color_tag_cmdtitle=configuration.brand.color_tag_cmdtitle(),
|
||||
color_tag_cmd=configuration.brand.color_tag_cmd(),
|
||||
color_tag_loc=configuration.brand.color_tag_loc(),
|
||||
color_hero=configuration.brand.color_hero(),
|
||||
primary_font_url=configuration.brand.primary_font_url(),
|
||||
primary_font_name=configuration.brand.primary_font_name(),
|
||||
mono_font_url=configuration.brand.mono_font_url(),
|
||||
mono_font_name=configuration.brand.mono_font_name(),
|
||||
)
|
||||
with open("static/sass/hyperglass.scss", "w") as scss_output:
|
||||
scss_output.write(rendered_output)
|
||||
except:
|
||||
raise TypeError("Error rendering Jinja2 template.")
|
||||
try:
|
||||
generated_sass = sass.compile(filename="static/sass/hyperglass.scss")
|
||||
with open("static/css/hyperglass.css", "w") as css_output:
|
||||
css_output.write(generated_sass)
|
||||
print("\n", "* Sass templates rendered to CSS files.", "\n")
|
||||
except:
|
||||
raise TypeError("Error rendering Sass template.")
|
@@ -1,103 +0,0 @@
|
||||
from flask import render_template
|
||||
import vars
|
||||
import jinja2
|
||||
import subprocess
|
||||
import codecs
|
||||
from markdown2 import Markdown
|
||||
import sass
|
||||
|
||||
file_loader = jinja2.FileSystemLoader(".")
|
||||
env = jinja2.Environment(loader=file_loader)
|
||||
|
||||
# Converts templates/footer.md from Markdown to HTML
|
||||
md = Markdown()
|
||||
footer_file = env.get_template("templates/footer.md")
|
||||
footer_jinja = footer_file.render(title=vars.brand.title())
|
||||
footer = footer_jinja
|
||||
|
||||
# Functions for rendering Jinja2 templates & importing variables
|
||||
|
||||
|
||||
class html:
|
||||
def renderTemplate(t):
|
||||
if t == "index":
|
||||
template = env.get_template("templates/index.html")
|
||||
elif t == "429":
|
||||
template = env.get_template("templates/429.html")
|
||||
elif t == "415":
|
||||
template = env.get_template("templates/415.html")
|
||||
elif t == "test":
|
||||
template = env.get_template("templates/429.html")
|
||||
return template.render(
|
||||
# General
|
||||
primary_asn=vars.gen.primary_asn(),
|
||||
google_analytics=vars.gen.google_analytics(),
|
||||
enable_recaptcha=vars.gen.enable_recaptcha(),
|
||||
enable_bgp_route=vars.gen.enable_bgp_route(),
|
||||
enable_bgp_community=vars.gen.enable_bgp_community(),
|
||||
enable_bgp_aspath=vars.gen.enable_bgp_aspath(),
|
||||
enable_ping=vars.gen.enable_ping(),
|
||||
enable_traceroute=vars.gen.enable_traceroute(),
|
||||
cache_timeout=vars.gen.cache_timeout(),
|
||||
message_rate_limit_query=vars.gen.message_rate_limit_query(),
|
||||
# Branding
|
||||
site_title=vars.brand.site_title(),
|
||||
title=vars.brand.title(),
|
||||
subtitle=vars.brand.subtitle(),
|
||||
title_mode=vars.brand.title_mode(),
|
||||
color_hero=vars.brand.color_hero(),
|
||||
enable_credit=vars.brand.enable_credit(),
|
||||
enable_footer=vars.brand.enable_footer(),
|
||||
footer_content=md.convert(footer),
|
||||
logo_path=vars.brand.logo_path(),
|
||||
logo_width=vars.brand.logo_width(),
|
||||
placeholder_prefix=vars.brand.placeholder_prefix(),
|
||||
show_peeringdb=vars.brand.show_peeringdb(),
|
||||
text_results=vars.brand.text_results(),
|
||||
text_location=vars.brand.text_location(),
|
||||
text_cache=vars.brand.text_cache(),
|
||||
text_415_title=vars.brand.text_415_title(),
|
||||
text_415_subtitle=vars.brand.text_415_subtitle(),
|
||||
text_415_button=vars.brand.text_415_button(),
|
||||
text_help_bgp_route=vars.brand.text_help_bgp_route(),
|
||||
text_help_bgp_community=vars.brand.text_help_bgp_community(),
|
||||
text_help_bgp_aspath=vars.brand.text_help_bgp_aspath(),
|
||||
text_help_ping=vars.brand.text_help_ping(),
|
||||
text_help_traceroute=vars.brand.text_help_traceroute(),
|
||||
text_limiter_title=vars.brand.text_limiter_title(),
|
||||
text_limiter_subtitle=vars.brand.text_limiter_subtitle(),
|
||||
# Devices
|
||||
device_networks=vars.dev.networks(),
|
||||
# device_location=vars.dev.location(),
|
||||
device_name=vars.dev.name(),
|
||||
)
|
||||
|
||||
|
||||
class css:
|
||||
def renderTemplate():
|
||||
try:
|
||||
template = env.get_template("templates/hyperglass.scss")
|
||||
rendered_output = template.render(
|
||||
color_btn_submit=vars.brand.color_btn_submit(),
|
||||
color_progressbar=vars.brand.color_progressbar(),
|
||||
color_tag_loctitle=vars.brand.color_tag_loctitle(),
|
||||
color_tag_cmdtitle=vars.brand.color_tag_cmdtitle(),
|
||||
color_tag_cmd=vars.brand.color_tag_cmd(),
|
||||
color_tag_loc=vars.brand.color_tag_loc(),
|
||||
color_hero=vars.brand.color_hero(),
|
||||
primary_font_url=vars.brand.primary_font_url(),
|
||||
primary_font_name=vars.brand.primary_font_name(),
|
||||
mono_font_url=vars.brand.mono_font_url(),
|
||||
mono_font_name=vars.brand.mono_font_name(),
|
||||
)
|
||||
with open("static/sass/hyperglass.scss", "w") as scss_output:
|
||||
scss_output.write(rendered_output)
|
||||
except:
|
||||
raise TypeError("Error rendering Jinja2 template.")
|
||||
try:
|
||||
generated_sass = sass.compile(filename="static/sass/hyperglass.scss")
|
||||
with open("static/css/hyperglass.css", "w") as css_output:
|
||||
css_output.write(generated_sass)
|
||||
print("\n", "* Sass templates rendered to CSS files.", "\n")
|
||||
except:
|
||||
raise TypeError("Error rendering Sass template.")
|
@@ -1,4 +1,6 @@
|
||||
import app
|
||||
from hyperglass.hyperglass import app
|
||||
|
||||
application = app
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run()
|
||||
application.run()
|
||||
|
@@ -7,3 +7,4 @@ netmiko
|
||||
netaddr
|
||||
markdown2
|
||||
libsass
|
||||
loguru
|
||||
|
Reference in New Issue
Block a user