diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index dbfcf4527..0027599f4 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -1,28 +1,41 @@ -### Issue type: + Please note that issues which do not fall under any of the below categories + will be closed. +---> +### Issue type +[ ] Feature request +[ ] Bug report +[ ] Documentation -**Python version:** -**NetBox version:** +### Environment +* Python version: +* NetBox version: +### Description diff --git a/docs/api/examples.md b/docs/api/examples.md index 5082534bc..4ec2f0f33 100644 --- a/docs/api/examples.md +++ b/docs/api/examples.md @@ -4,7 +4,8 @@ Supported HTTP methods: * `GET`: Retrieve an object or list of objects * `POST`: Create a new object -* `PUT`: Update an existing object +* `PUT`: Update an existing object, all mandatory fields must be specified +* `PATCH`: Updates an existing object, only specifiying the field to be changed * `DELETE`: Delete an existing object To authenticate a request, attach your token in an `Authorization` header: @@ -104,12 +105,19 @@ $ curl -X POST -H "Authorization: Token d2f763479f703d80de0ec15254237bc651f9cdc0 ### Modify an existing site -Make an authenticated `PUT` request to the site detail endpoint. As with a create (POST) request, all mandatory fields must be included. +Make an authenticated `PUT` request to the site detail endpoint. As with a create (`POST`) request, all mandatory fields must be included. ``` $ curl -X PUT -H "Authorization: Token d2f763479f703d80de0ec15254237bc651f9cdc0" -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://localhost:8000/api/dcim/sites/16/ --data '{"name": "Renamed Site", "slug": "renamed-site"}' ``` +### Modify an object by changing a field + +Make an authenticated `PATCH` request to the device endpoint. With `PATCH`, unlike `POST` and `PUT`, we only specify the field that is being changed. In this example, we add a serial number to a device. +``` +$ curl -X PATCH -H "Authorization: Token d2f763479f703d80de0ec15254237bc651f9cdc0" -H "Content-Type: application/json" -H "Accept: application/json; indent=4" http://localhost:8000/api/dcim/devices/2549/ --data '{"serial": "FTX1123A090"}' +``` + ### Delete an existing site Send an authenticated `DELETE` request to the site detail endpoint. diff --git a/docs/api/overview.md b/docs/api/overview.md index a9ad115f8..39a4109f9 100644 --- a/docs/api/overview.md +++ b/docs/api/overview.md @@ -1,8 +1,42 @@ NetBox v2.0 and later includes a full-featured REST API that allows its data model to be read and manipulated externally. +# What is a REST API? + +REST stands for [representational state transfer](https://en.wikipedia.org/wiki/Representational_state_transfer). It's a particular type of API which employs HTTP to create, retrieve, update, and delete objects from a database. (This set of operations is commonly referred to as CRUD.) Each type of operation is associated with a particular HTTP verb: + +* `GET`: Retrieve an object or list of objects +* `POST`: Create an object +* `PUT` / `PATCH`: Modify an existing object. `PUT` requires all mandatory fields to be specified, while `PATCH` only expects the field that is being modified to be specified. +* `DELETE`: Delete an existing object + +The NetBox API represents all objects in [JavaScript Object Notation (JSON)](http://www.json.org/). This makes it very easy to interact with NetBox data on the command line with common tools. For example, we can request an IP address from NetBox and output the JSON using `curl` and `jq`. (Piping the output through `jq` isn't strictly required but makes it much easier to read.) + +``` +$ curl -s http://localhost:8000/api/ipam/ip-addresses/2954/ | jq '.' +{ + "custom_fields": {}, + "nat_outside": null, + "nat_inside": null, + "description": "An example IP address", + "id": 2954, + "family": 4, + "address": "5.101.108.132/26", + "vrf": null, + "tenant": null, + "status": { + "label": "Active", + "value": 1 + }, + "role": null, + "interface": null +} +``` + +Each attribute of the NetBox object is expressed as a field in the dictionary. Fields may include their own nested objects, as in the case of the `status` field above. Every object includes a primary key named `id` which uniquely identifies it in the database. + # URL Hierarchy -NetBox's entire REST API is housed under the API root, `/api/`. The API's URL structure is divided at the root level by application: circuits, DCIM, extras, IPAM, secrets, and tenancy. Within each application, each model has its own path. For example, the provider and circuit objects are located under the "circuits" application: +NetBox's entire API is housed under the API root at `https:///api/`. The URL structure is divided at the root level by application: circuits, DCIM, extras, IPAM, secrets, and tenancy. Within each application, each model has its own path. For example, the provider and circuit objects are located under the "circuits" application: * /api/circuits/providers/ * /api/circuits/circuits/ @@ -13,9 +47,9 @@ Likewise, the site, rack, and device objects are located under the "DCIM" applic * /api/dcim/racks/ * /api/dcim/devices/ -The full hierarchy of available endpoints can be viewed by navigating to the API root (e.g. /api/) in a web browser. +The full hierarchy of available endpoints can be viewed by navigating to the API root in a web browser. -Each model generally has two URLs associated with it: a list URL and a detail URL. The list URL is used to request a list of multiple objects or to create a new object. The detail URL is used to retrieve, update, or delete an existing object. All objects are referenced by their numeric primary key (ID). +Each model generally has two views associated with it: a list view and a detail view. The list view is used to request a list of multiple objects or to create a new object. The detail view is used to retrieve, update, or delete an existing object. All objects are referenced by their numeric primary key (`id`). * /api/dcim/devices/ - List devices or create a new device * /api/dcim/devices/123/ - Retrieve, update, or delete the device with ID 123 diff --git a/docs/configuration/optional-settings.md b/docs/configuration/optional-settings.md index 05e60dcac..9cc4e75fd 100644 --- a/docs/configuration/optional-settings.md +++ b/docs/configuration/optional-settings.md @@ -17,7 +17,7 @@ ADMINS = [ ## BANNER_BOTTOM -Setting these variables will display content in a banner at the top and/or bottom of the page, respectively. To replicate the content of the top banner in the bottom banner, set: +Setting these variables will display content in a banner at the top and/or bottom of the page, respectively. HTML is allowed. To replicate the content of the top banner in the bottom banner, set: ``` BANNER_TOP = 'Your banner text' @@ -26,6 +26,12 @@ BANNER_BOTTOM = BANNER_TOP --- +## BANNER_LOGIN + +The value of this variable will be displayed on the login page above the login form. HTML is allowed. + +--- + ## BASE_PATH Default: None @@ -135,11 +141,61 @@ An API consumer can request an arbitrary number of objects by appending the "lim --- -## NETBOX_USERNAME +## MEDIA_ROOT -## NETBOX_PASSWORD +Default: $BASE_DIR/netbox/media/ -If provided, NetBox will use these credentials to authenticate against devices when collecting data. +The file path to the location where media files (such as image attachments) are stored. By default, this is the `netbox/media` directory within the base NetBox installation path. + +--- + +## NAPALM_USERNAME + +## NAPALM_PASSWORD + +NetBox will use these credentials when authenticating to remote devices via the [NAPALM library](https://napalm-automation.net/), if installed. Both parameters are optional. + +Note: If SSH public key authentication has been set up for the system account under which NetBox runs, these parameters are not needed. + +--- + +## NAPALM_ARGS + +A dictionary of optional arguments to pass to NAPALM when instantiating a network driver. See the NAPALM documentation for a [complete list of optional arguments](http://napalm.readthedocs.io/en/latest/support/#optional-arguments). An example: + +``` +NAPALM_ARGS = { + 'api_key': '472071a93b60a1bd1fafb401d9f8ef41', + 'port': 2222, +} +``` + +Note: Some platforms (e.g. Cisco IOS) require an argument named `secret` to be passed in addition to the normal password. If desired, you can use the configured `NAPALM_PASSWORD` as the value for this argument: + +``` +NAPALM_USERNAME = 'username' +NAPALM_PASSWORD = 'MySecretPassword' +NAPALM_ARGS = { + 'secret': NAPALM_PASSWORD, + # Include any additional args here +} +``` + +--- + +## NAPALM_TIMEOUT + +Default: 30 seconds + +The amount of time (in seconds) to wait for NAPALM to connect to a device. + +--- + +## NETBOX_USERNAME (Deprecated) + +## NETBOX_PASSWORD (Deprecated) + +These settings have been deprecated and will be removed in NetBox v2.2. Please use `NAPALM_USERNAME` and `NAPALM_PASSWORD` instead. --- diff --git a/docs/installation/ldap.md b/docs/installation/ldap.md index 0d546863e..bb1618e32 100644 --- a/docs/installation/ldap.md +++ b/docs/installation/ldap.md @@ -72,11 +72,14 @@ AUTH_LDAP_USER_DN_TEMPLATE = "uid=%(user)s,ou=users,dc=example,dc=com" # You can map user attributes to Django attributes as so. AUTH_LDAP_USER_ATTR_MAP = { "first_name": "givenName", - "last_name": "sn" + "last_name": "sn", + "email": "mail" } ``` # User Groups for Permissions +!!! Info + When using Microsoft Active Directory, Support for nested Groups can be activated by using `GroupOfNamesType()` instead of `NestedGroupOfNamesType()` for AUTH_LDAP_GROUP_TYPE. ```python from django_auth_ldap.config import LDAPSearch, GroupOfNamesType @@ -108,12 +111,3 @@ AUTH_LDAP_GROUP_CACHE_TIMEOUT = 3600 * `is_active` - All users must be mapped to at least this group to enable authentication. Without this, users cannot log in. * `is_staff` - Users mapped to this group are enabled for access to the administration tools; this is the equivalent of checking the "staff status" box on a manually created user. This doesn't grant any specific permissions. * `is_superuser` - Users mapped to this group will be granted superuser status. Superusers are implicitly granted all permissions. - -It is also possible map user attributes to Django attributes: - -```python -AUTH_LDAP_USER_ATTR_MAP = { - "first_name": "givenName", - "last_name": "sn", -} -``` diff --git a/docs/installation/migrating-to-python3.md b/docs/installation/migrating-to-python3.md new file mode 100644 index 000000000..e99018252 --- /dev/null +++ b/docs/installation/migrating-to-python3.md @@ -0,0 +1,33 @@ +# Migration + +Remove Python 2 packages + +```no-highlight +# apt-get remove --purge -y python-dev python-pip +``` + +Install Python 3 packages + +```no-highlight +# apt-get install -y python3 python3-dev python3-pip +``` + +Install Python Packages + +```no-highlight +# cd /opt/netbox +# pip3 install -r requirements.txt +``` + +Gunicorn Update + +```no-highlight +# pip uninstall gunicorn +# pip3 install gunicorn +``` + +Re-install LDAP Module (optional if using LDAP for auth) + +```no-highlight +sudo pip3 install django-auth-ldap +``` diff --git a/docs/installation/netbox.md b/docs/installation/netbox.md index c7c2eb8ed..d1aa5d001 100644 --- a/docs/installation/netbox.md +++ b/docs/installation/netbox.md @@ -20,7 +20,7 @@ Python 3: ```no-highlight # yum install -y epel-release -# yum install -y gcc python34 python34-devel python34-setuptools libxml2-devel libxslt-devel libffi-devel graphviz openssl-devel +# yum install -y gcc python34 python34-devel python34-setuptools libxml2-devel libxslt-devel libffi-devel graphviz openssl-devel redhat-rpm-config # easy_install-3.4 pip # ln -s -f python3.4 /usr/bin/python ``` @@ -29,7 +29,7 @@ Python 2: ```no-highlight # yum install -y epel-release -# yum install -y gcc python2 python-devel python-pip libxml2-devel libxslt-devel libffi-devel graphviz openssl-devel +# yum install -y gcc python2 python-devel python-pip libxml2-devel libxslt-devel libffi-devel graphviz openssl-devel redhat-rpm-config ``` You may opt to install NetBox either from a numbered release or by cloning the master branch of its repository on GitHub. @@ -97,6 +97,14 @@ Python 2: # pip install -r requirements.txt ``` +### NAPALM Automation + +As of v2.1.0, NetBox supports integration with the [NAPALM automation](https://napalm-automation.net/) library. NAPALM allows NetBox to fetch live data from devices and return it to a requester via its REST API. Installation of NAPALM is optional. To enable it, install the `napalm` package using pip or pip3: + +```no-highlight +# pip3 install napalm +``` + # Configuration Move into the NetBox configuration directory and make a copy of `configuration.example.py` named `configuration.py`. diff --git a/docs/installation/web-server.md b/docs/installation/web-server.md index 9da487f13..f9a304ff5 100644 --- a/docs/installation/web-server.md +++ b/docs/installation/web-server.md @@ -1,13 +1,9 @@ -# Web Server Installation - We'll set up a simple WSGI front end using [gunicorn](http://gunicorn.org/) for the purposes of this guide. For web servers, we provide example configurations for both [nginx](https://www.nginx.com/resources/wiki/) and [Apache](http://httpd.apache.org/docs/2.4). (You are of course free to use whichever combination of HTTP and WSGI services you'd like.) We'll also use [supervisord](http://supervisord.org/) to enable service persistence. !!! info For the sake of brevity, only Ubuntu 16.04 instructions are provided here, but this sort of web server and WSGI configuration is not unique to NetBox. Please consult your distribution's documentation for assistance if needed. -```no-highlight -# apt-get install -y gunicorn supervisor -``` +# Web Server Installation ## Option A: nginx @@ -55,7 +51,7 @@ Restart the nginx service to use the new configuration. # service nginx restart ``` -To enable SSL, consider this guide on [securing nginx with Let's Encrypt](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-14-04). +To enable SSL, consider this guide on [securing nginx with Let's Encrypt](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-16-04). ## Option B: Apache @@ -100,10 +96,16 @@ Save the contents of the above example in `/etc/apache2/sites-available/netbox.c # service apache2 restart ``` -To enable SSL, consider this guide on [securing Apache with Let's Encrypt](https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-14-04). +To enable SSL, consider this guide on [securing Apache with Let's Encrypt](https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-16-04). # gunicorn Installation +Install gunicorn using `pip3` (Python 3) or `pip` (Python 2): + +```no-highlight +# pip3 install gunicorn +``` + Save the following configuration in the root netbox installation path as `gunicorn_config.py` (e.g. `/opt/netbox/gunicorn_config.py` per our example installation). Be sure to verify the location of the gunicorn executable on your server (e.g. `which gunicorn`) and to update the `pythonpath` variable if needed. If using CentOS/RHEL, change the username from `www-data` to `nginx` or `apache`. ```no-highlight @@ -116,6 +118,12 @@ user = 'www-data' # supervisord Installation +Install supervisor: + +```no-highlight +# apt-get install -y supervisor +``` + Save the following as `/etc/supervisor/conf.d/netbox.conf`. Update the `command` and `directory` paths as needed. If using CentOS/RHEL, change the username from `www-data` to `nginx` or `apache`. ```no-highlight diff --git a/docs/shell/intro.md b/docs/shell/intro.md new file mode 100644 index 000000000..df92cb7cd --- /dev/null +++ b/docs/shell/intro.md @@ -0,0 +1,194 @@ +NetBox includes a Python shell withing which objects can be directly queried, created, modified, and deleted. To enter the shell, run the following command: + +``` +./manage.py nbshell +``` + +This will launch a customized version of [the built-in Django shell](https://docs.djangoproject.com/en/dev/ref/django-admin/#shell) with all relevant NetBox models pre-loaded. (If desired, the stock Django shell is also available by executing `./manage.py shell`.) + +``` +$ ./manage.py nbshell +### NetBox interactive shell (jstretch-laptop) +### Python 2.7.6 | Django 1.11.3 | NetBox 2.1.0-dev +### lsmodels() will show available models. Use help() for more info. +``` + +The function `lsmodels()` will print a list of all available NetBox models: + +``` +>>> lsmodels() +DCIM: + ConsolePort + ConsolePortTemplate + ConsoleServerPort + ConsoleServerPortTemplate + Device + ... +``` + +## Querying Objects + +Objects are retrieved by forming a [Django queryset](https://docs.djangoproject.com/en/dev/topics/db/queries/#retrieving-objects). The base queryset for an object takes the form `.objects.all()`, which will return a (truncated) list of all objects of that type. + +``` +>>> Device.objects.all() +, , , , , '...(remaining elements truncated)...']> +``` + +Use a `for` loop to cycle through all objects in the list: + +``` +>>> for device in Device.objects.all(): +... print(device.name, device.device_type) +... +(u'TestDevice1', ) +(u'TestDevice2', ) +(u'TestDevice3', ) +(u'TestDevice4', ) +(u'TestDevice5', ) +... +``` + +To count all objects matching the query, replace `all()` with `count()`: + +``` +>>> Device.objects.count() +1274 +``` + +To retrieve a particular object (typically by its primary key or other unique field), use `get()`: + +``` +>>> Site.objects.get(pk=7) + +``` + +### Filtering Querysets + +In most cases, you want to retrieve only a specific subset of objects. To filter a queryset, replace `all()` with `filter()` and pass one or more keyword arguments. For example: + +``` +>>> Device.objects.filter(status=STATUS_ACTIVE) +, , , , , '...(remaining elements truncated)...']> +``` + +Querysets support slicing to return a specific range of objects. + +``` +>>> Device.objects.filter(status=STATUS_ACTIVE)[:3] +, , ]> +``` + +The `count()` method can be appended to the queryset to return a count of objects rather than the full list. + +``` +>>> Device.objects.filter(status=STATUS_ACTIVE).count() +982 +``` + +Relationships with other models can be traversed by concatenting field names with a double-underscore. For example, the following will return all devices assigned to the tenant named "Pied Piper." + +``` +>>> Device.objects.filter(tenant__name='Pied Piper') +``` + +This approach can span multiple levels of relations. For example, the following will return all IP addresses assigned to a device in North America: + +``` +>>> IPAddress.objects.filter(interface__device__site__region__slug='north-america') +``` + +!!! note + While the above query is functional, it is very inefficient. There are ways to optimize such requests, however they are out of the scope of this document. For more information, see the [Django queryset method reference](https://docs.djangoproject.com/en/dev/ref/models/querysets/) documentation. + +Reverse relationships can be traversed as well. For example, the following will find all devices with an interface named "em0": + +``` +>>> Device.objects.filter(interfaces__name='em0') +``` + +Character fields can be filtered against partial matches using the `contains` or `icontains` field lookup (the later of which is case-insensitive). + +``` +>>> Device.objects.filter(name__icontains='testdevice') +``` + +Similarly, numeric fields can be filtered by values less than, greater than, and/or equal to a given value. + +``` +>>> VLAN.objects.filter(vid__gt=2000) +``` + +Multiple filters can be combined to further refine a queryset. + +``` +>>> VLAN.objects.filter(vid__gt=2000, name__icontains='engineering') +``` + +To return the inverse of a filtered queryset, use `exclude()` instead of `filter()`. + +``` +>>> Device.objects.count() +4479 +>>> Device.objects.filter(status=STATUS_ACTIVE).count() +4133 +>>> Device.objects.exclude(status=STATUS_ACTIVE).count() +346 +``` + +!!! info + The examples above are intended only to provide a cursory introduction to queryset filtering. For an exhaustive list of the available filters, please consult the [Django queryset API docs](https://docs.djangoproject.com/en/dev/ref/models/querysets/). + +## Creating and Updating Objects + +New objects can be created by instantiating the desired model, defining values for all required attributes, and calling `save()` on the instance. + +``` +>>> lab1 = Site.objects.get(pk=7) +>>> myvlan = VLAN(vid=123, name='MyNewVLAN', site=lab1) +>>> myvlan.save() +``` + +Alternatively, the above can be performed as a single operation: + +``` +>>> VLAN(vid=123, name='MyNewVLAN', site=Site.objects.get(pk=7)).save() +``` + +To modify an object, retrieve it, update the desired field(s), and call `save()` again. + +``` +>>> vlan = VLAN.objects.get(pk=1280) +>>> vlan.name +u'MyNewVLAN' +>>> vlan.name = 'BetterName' +>>> vlan.save() +>>> VLAN.objects.get(pk=1280).name +u'BetterName' +``` + +!!! warning + The Django ORM provides methods to create/edit many objects at once, namely `bulk_create()` and `update()`. These are best avoided in most cases as they bypass a model's built-in validation and can easily lead to database corruption if not used carefully. + +## Deleting Objects + +To delete an object, simply call `delete()` on its instance. This will return a dictionary of all objects (including related objects) which have been deleted as a result of this operation. + +``` +>>> vlan + +>>> vlan.delete() +(1, {u'extras.CustomFieldValue': 0, u'ipam.VLAN': 1}) +``` + +To delete multiple objects at once, call `delete()` on a filtered queryset. It's a good idea to always sanity-check the count of selected objects _before_ deleting them. + +``` +>>> Device.objects.filter(name__icontains='test').count() +27 +>>> Device.objects.filter(name__icontains='test').delete() +(35, {u'extras.CustomFieldValue': 0, u'dcim.DeviceBay': 0, u'secrets.Secret': 0, u'dcim.InterfaceConnection': 4, u'extras.ImageAttachment': 0, u'dcim.Device': 27, u'dcim.Interface': 4, u'dcim.ConsolePort': 0, u'dcim.PowerPort': 0}) +``` + +!!! warning + Deletions are immediate and irreversible. Always think very carefully before calling `delete()` on an instance or queryset. diff --git a/mkdocs.yml b/mkdocs.yml index 8b77b289d..f204749d5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -8,6 +8,7 @@ pages: - 'Web Server': 'installation/web-server.md' - 'LDAP (Optional)': 'installation/ldap.md' - 'Upgrading': 'installation/upgrading.md' + - 'Migrating to Python3': 'installation/migrating-to-python3.md' - 'Configuration': - 'Mandatory Settings': 'configuration/mandatory-settings.md' - 'Optional Settings': 'configuration/optional-settings.md' @@ -23,6 +24,8 @@ pages: - 'Authentication': 'api/authentication.md' - 'Working with Secrets': 'api/working-with-secrets.md' - 'Examples': 'api/examples.md' + - 'Shell': + - 'Introduction': 'shell/intro.md' markdown_extensions: - admonition: diff --git a/netbox/circuits/api/serializers.py b/netbox/circuits/api/serializers.py index cdab3427a..d2432374f 100644 --- a/netbox/circuits/api/serializers.py +++ b/netbox/circuits/api/serializers.py @@ -6,7 +6,7 @@ from circuits.models import Provider, Circuit, CircuitTermination, CircuitType from dcim.api.serializers import NestedSiteSerializer, InterfaceSerializer from extras.api.customfields import CustomFieldModelSerializer from tenancy.api.serializers import NestedTenantSerializer -from utilities.api import ModelValidationMixin +from utilities.api import ValidatedModelSerializer # @@ -45,7 +45,7 @@ class WritableProviderSerializer(CustomFieldModelSerializer): # Circuit types # -class CircuitTypeSerializer(ModelValidationMixin, serializers.ModelSerializer): +class CircuitTypeSerializer(ValidatedModelSerializer): class Meta: model = CircuitType @@ -111,7 +111,7 @@ class CircuitTerminationSerializer(serializers.ModelSerializer): ] -class WritableCircuitTerminationSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableCircuitTerminationSerializer(ValidatedModelSerializer): class Meta: model = CircuitTermination diff --git a/netbox/circuits/forms.py b/netbox/circuits/forms.py index d9954e55b..2a2cd6843 100644 --- a/netbox/circuits/forms.py +++ b/netbox/circuits/forms.py @@ -170,6 +170,7 @@ class CircuitFilterForm(BootstrapMixin, CustomFieldFilterForm): queryset=Site.objects.annotate(filter_count=Count('circuit_terminations')), to_field_name='slug' ) + commit_rate = forms.IntegerField(required=False, min_value=0, label='Commit rate (Kbps)') # diff --git a/netbox/circuits/models.py b/netbox/circuits/models.py index 1acd3f4a0..17b38a5d8 100644 --- a/netbox/circuits/models.py +++ b/netbox/circuits/models.py @@ -13,22 +13,6 @@ from utilities.models import CreatedUpdatedModel from .constants import * -def humanize_speed(speed): - """ - Humanize speeds given in Kbps (e.g. 10000000 becomes '10 Gbps') - """ - if speed >= 1000000000 and speed % 1000000000 == 0: - return '{} Tbps'.format(speed / 1000000000) - elif speed >= 1000000 and speed % 1000000 == 0: - return '{} Gbps'.format(speed / 1000000) - elif speed >= 1000 and speed % 1000 == 0: - return '{} Mbps'.format(speed / 1000) - elif speed >= 1000: - return '{} Mbps'.format(float(speed) / 1000) - else: - return '{} Kbps'.format(speed) - - @python_2_unicode_compatible class Provider(CreatedUpdatedModel, CustomFieldModel): """ @@ -139,10 +123,6 @@ class Circuit(CreatedUpdatedModel, CustomFieldModel): def termination_z(self): return self._get_termination('Z') - def commit_rate_human(self): - return '' if not self.commit_rate else humanize_speed(self.commit_rate) - commit_rate_human.admin_order_field = 'commit_rate' - @python_2_unicode_compatible class CircuitTermination(models.Model): @@ -173,11 +153,3 @@ class CircuitTermination(models.Model): return CircuitTermination.objects.select_related('site').get(circuit=self.circuit, term_side=peer_side) except CircuitTermination.DoesNotExist: return None - - def port_speed_human(self): - return humanize_speed(self.port_speed) - port_speed_human.admin_order_field = 'port_speed' - - def upstream_speed_human(self): - return '' if not self.upstream_speed else humanize_speed(self.upstream_speed) - upstream_speed_human.admin_order_field = 'upstream_speed' diff --git a/netbox/dcim/api/serializers.py b/netbox/dcim/api/serializers.py index 09da3ced7..ebfb781e0 100644 --- a/netbox/dcim/api/serializers.py +++ b/netbox/dcim/api/serializers.py @@ -15,7 +15,7 @@ from dcim.models import ( ) from extras.api.customfields import CustomFieldModelSerializer from tenancy.api.serializers import NestedTenantSerializer -from utilities.api import ChoiceFieldSerializer, ModelValidationMixin +from utilities.api import ChoiceFieldSerializer, ValidatedModelSerializer # @@ -38,7 +38,7 @@ class RegionSerializer(serializers.ModelSerializer): fields = ['id', 'name', 'slug', 'parent'] -class WritableRegionSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableRegionSerializer(ValidatedModelSerializer): class Meta: model = Region @@ -100,7 +100,7 @@ class NestedRackGroupSerializer(serializers.ModelSerializer): fields = ['id', 'url', 'name', 'slug'] -class WritableRackGroupSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableRackGroupSerializer(ValidatedModelSerializer): class Meta: model = RackGroup @@ -111,7 +111,7 @@ class WritableRackGroupSerializer(ModelValidationMixin, serializers.ModelSeriali # Rack roles # -class RackRoleSerializer(ModelValidationMixin, serializers.ModelSerializer): +class RackRoleSerializer(ValidatedModelSerializer): class Meta: model = RackRole @@ -216,7 +216,7 @@ class RackReservationSerializer(serializers.ModelSerializer): fields = ['id', 'rack', 'units', 'created', 'user', 'description'] -class WritableRackReservationSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableRackReservationSerializer(ValidatedModelSerializer): class Meta: model = RackReservation @@ -227,7 +227,7 @@ class WritableRackReservationSerializer(ModelValidationMixin, serializers.ModelS # Manufacturers # -class ManufacturerSerializer(ModelValidationMixin, serializers.ModelSerializer): +class ManufacturerSerializer(ValidatedModelSerializer): class Meta: model = Manufacturer @@ -292,7 +292,7 @@ class ConsolePortTemplateSerializer(serializers.ModelSerializer): fields = ['id', 'device_type', 'name'] -class WritableConsolePortTemplateSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableConsolePortTemplateSerializer(ValidatedModelSerializer): class Meta: model = ConsolePortTemplate @@ -311,7 +311,7 @@ class ConsoleServerPortTemplateSerializer(serializers.ModelSerializer): fields = ['id', 'device_type', 'name'] -class WritableConsoleServerPortTemplateSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableConsoleServerPortTemplateSerializer(ValidatedModelSerializer): class Meta: model = ConsoleServerPortTemplate @@ -330,7 +330,7 @@ class PowerPortTemplateSerializer(serializers.ModelSerializer): fields = ['id', 'device_type', 'name'] -class WritablePowerPortTemplateSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritablePowerPortTemplateSerializer(ValidatedModelSerializer): class Meta: model = PowerPortTemplate @@ -349,7 +349,7 @@ class PowerOutletTemplateSerializer(serializers.ModelSerializer): fields = ['id', 'device_type', 'name'] -class WritablePowerOutletTemplateSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritablePowerOutletTemplateSerializer(ValidatedModelSerializer): class Meta: model = PowerOutletTemplate @@ -369,7 +369,7 @@ class InterfaceTemplateSerializer(serializers.ModelSerializer): fields = ['id', 'device_type', 'name', 'form_factor', 'mgmt_only'] -class WritableInterfaceTemplateSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableInterfaceTemplateSerializer(ValidatedModelSerializer): class Meta: model = InterfaceTemplate @@ -388,7 +388,7 @@ class DeviceBayTemplateSerializer(serializers.ModelSerializer): fields = ['id', 'device_type', 'name'] -class WritableDeviceBayTemplateSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableDeviceBayTemplateSerializer(ValidatedModelSerializer): class Meta: model = DeviceBayTemplate @@ -399,7 +399,7 @@ class WritableDeviceBayTemplateSerializer(ModelValidationMixin, serializers.Mode # Device roles # -class DeviceRoleSerializer(ModelValidationMixin, serializers.ModelSerializer): +class DeviceRoleSerializer(ValidatedModelSerializer): class Meta: model = DeviceRole @@ -418,7 +418,7 @@ class NestedDeviceRoleSerializer(serializers.ModelSerializer): # Platforms # -class PlatformSerializer(ModelValidationMixin, serializers.ModelSerializer): +class PlatformSerializer(ValidatedModelSerializer): class Meta: model = Platform @@ -473,14 +473,10 @@ class DeviceSerializer(CustomFieldModelSerializer): device_bay = obj.parent_bay except DeviceBay.DoesNotExist: return None - return { - 'id': device_bay.device.pk, - 'name': device_bay.device.name, - 'device_bay': { - 'id': device_bay.pk, - 'name': device_bay.name, - } - } + context = {'request': self.context['request']} + data = NestedDeviceSerializer(instance=device_bay.device, context=context).data + data['device_bay'] = NestedDeviceBaySerializer(instance=device_bay, context=context).data + return data class WritableDeviceSerializer(CustomFieldModelSerializer): @@ -520,7 +516,7 @@ class ConsoleServerPortSerializer(serializers.ModelSerializer): read_only_fields = ['connected_console'] -class WritableConsoleServerPortSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableConsoleServerPortSerializer(ValidatedModelSerializer): class Meta: model = ConsoleServerPort @@ -540,7 +536,7 @@ class ConsolePortSerializer(serializers.ModelSerializer): fields = ['id', 'device', 'name', 'cs_port', 'connection_status'] -class WritableConsolePortSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableConsolePortSerializer(ValidatedModelSerializer): class Meta: model = ConsolePort @@ -560,7 +556,7 @@ class PowerOutletSerializer(serializers.ModelSerializer): read_only_fields = ['connected_port'] -class WritablePowerOutletSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritablePowerOutletSerializer(ValidatedModelSerializer): class Meta: model = PowerOutlet @@ -580,7 +576,7 @@ class PowerPortSerializer(serializers.ModelSerializer): fields = ['id', 'device', 'name', 'power_outlet', 'connection_status'] -class WritablePowerPortSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritablePowerPortSerializer(ValidatedModelSerializer): class Meta: model = PowerPort @@ -668,7 +664,7 @@ class PeerInterfaceSerializer(serializers.ModelSerializer): ] -class WritableInterfaceSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableInterfaceSerializer(ValidatedModelSerializer): class Meta: model = Interface @@ -690,7 +686,15 @@ class DeviceBaySerializer(serializers.ModelSerializer): fields = ['id', 'device', 'name', 'installed_device'] -class WritableDeviceBaySerializer(ModelValidationMixin, serializers.ModelSerializer): +class NestedDeviceBaySerializer(serializers.ModelSerializer): + url = serializers.HyperlinkedIdentityField(view_name='dcim-api:devicebay-detail') + + class Meta: + model = DeviceBay + fields = ['id', 'url', 'name'] + + +class WritableDeviceBaySerializer(ValidatedModelSerializer): class Meta: model = DeviceBay @@ -713,7 +717,7 @@ class InventoryItemSerializer(serializers.ModelSerializer): ] -class WritableInventoryItemSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableInventoryItemSerializer(ValidatedModelSerializer): class Meta: model = InventoryItem @@ -745,7 +749,7 @@ class NestedInterfaceConnectionSerializer(serializers.ModelSerializer): fields = ['id', 'url', 'connection_status'] -class WritableInterfaceConnectionSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableInterfaceConnectionSerializer(ValidatedModelSerializer): class Meta: model = InterfaceConnection diff --git a/netbox/dcim/api/views.py b/netbox/dcim/api/views.py index d32c63bfa..56d4221da 100644 --- a/netbox/dcim/api/views.py +++ b/netbox/dcim/api/views.py @@ -3,7 +3,6 @@ from collections import OrderedDict from rest_framework.decorators import detail_route from rest_framework.mixins import ListModelMixin -from rest_framework.permissions import IsAuthenticated from rest_framework.response import Response from rest_framework.viewsets import GenericViewSet, ModelViewSet, ViewSet @@ -21,7 +20,7 @@ from dcim import filters from extras.api.serializers import RenderedGraphSerializer from extras.api.views import CustomFieldModelViewSet from extras.models import Graph, GRAPH_TYPE_INTERFACE, GRAPH_TYPE_SITE -from utilities.api import ServiceUnavailable, WritableSerializerMixin +from utilities.api import IsAuthenticatedOrLoginNotRequired, ServiceUnavailable, WritableSerializerMixin from .exceptions import MissingFilterException from . import serializers @@ -272,15 +271,17 @@ class DeviceViewSet(WritableSerializerMixin, CustomFieldModelViewSet): ip_address = str(device.primary_ip.address.ip) d = driver( hostname=ip_address, - username=settings.NETBOX_USERNAME, - password=settings.NETBOX_PASSWORD + username=settings.NAPALM_USERNAME, + password=settings.NAPALM_PASSWORD, + timeout=settings.NAPALM_TIMEOUT, + optional_args=settings.NAPALM_ARGS ) try: d.open() for method in napalm_methods: response[method] = getattr(d, method)() except Exception as e: - raise ServiceUnavailable("Error connecting to the device: {}".format(e)) + raise ServiceUnavailable("Error connecting to the device at {}: {}".format(ip_address, e)) d.close() return Response(response) @@ -385,7 +386,7 @@ class ConnectedDeviceViewSet(ViewSet): * `peer-device`: The name of the peer device * `peer-interface`: The name of the peer interface """ - permission_classes = [IsAuthenticated] + permission_classes = [IsAuthenticatedOrLoginNotRequired] def get_view_name(self): return "Connected Device Locator" diff --git a/netbox/dcim/constants.py b/netbox/dcim/constants.py index f2c047910..9c2993b60 100644 --- a/netbox/dcim/constants.py +++ b/netbox/dcim/constants.py @@ -58,6 +58,7 @@ IFACE_FF_1GE_FIXED = 1000 IFACE_FF_1GE_GBIC = 1050 IFACE_FF_1GE_SFP = 1100 IFACE_FF_10GE_FIXED = 1150 +IFACE_FF_10GE_CX4 = 1170 IFACE_FF_10GE_SFP_PLUS = 1200 IFACE_FF_10GE_XFP = 1300 IFACE_FF_10GE_XENPAK = 1310 @@ -106,6 +107,7 @@ IFACE_FF_CHOICES = [ [IFACE_FF_100ME_FIXED, '100BASE-TX (10/100ME)'], [IFACE_FF_1GE_FIXED, '1000BASE-T (1GE)'], [IFACE_FF_10GE_FIXED, '10GBASE-T (10GE)'], + [IFACE_FF_10GE_CX4, '10GBASE-CX4 (10GE)'], ] ], [ diff --git a/netbox/dcim/filters.py b/netbox/dcim/filters.py index e3579085a..a9f58d2a4 100644 --- a/netbox/dcim/filters.py +++ b/netbox/dcim/filters.py @@ -1,6 +1,7 @@ from __future__ import unicode_literals import django_filters +from netaddr import EUI from netaddr.core import AddrFormatError from django.contrib.auth.models import User @@ -8,7 +9,7 @@ from django.db.models import Q from extras.filters import CustomFieldFilterSet from tenancy.models import Tenant -from utilities.filters import NullableModelMultipleChoiceFilter, NumericInFilter +from utilities.filters import NullableCharFieldFilter, NullableModelMultipleChoiceFilter, NumericInFilter from .models import ( ConsolePort, ConsolePortTemplate, ConsoleServerPort, ConsoleServerPortTemplate, Device, DeviceBay, DeviceBayTemplate, DeviceRole, DeviceType, STATUS_CHOICES, IFACE_FF_LAG, Interface, InterfaceConnection, @@ -113,6 +114,7 @@ class RackFilter(CustomFieldFilterSet, django_filters.FilterSet): method='search', label='Search', ) + facility_id = NullableCharFieldFilter() site_id = django_filters.ModelMultipleChoiceFilter( queryset=Site.objects.all(), label='Site (ID)', @@ -156,7 +158,7 @@ class RackFilter(CustomFieldFilterSet, django_filters.FilterSet): class Meta: model = Rack - fields = ['facility_id', 'type', 'width', 'u_height', 'desc_units'] + fields = ['type', 'width', 'u_height', 'desc_units'] def search(self, queryset, name, value): if not value.strip(): @@ -271,6 +273,7 @@ class DeviceTypeFilter(CustomFieldFilterSet, django_filters.FilterSet): class DeviceTypeComponentFilterSet(django_filters.FilterSet): devicetype_id = django_filters.ModelMultipleChoiceFilter( queryset=DeviceType.objects.all(), + name='device_type_id', label='Device type (ID)', ) @@ -383,6 +386,8 @@ class DeviceFilter(CustomFieldFilterSet, django_filters.FilterSet): to_field_name='slug', label='Platform (slug)', ) + name = NullableCharFieldFilter() + asset_tag = NullableCharFieldFilter() site_id = django_filters.ModelMultipleChoiceFilter( queryset=Site.objects.all(), label='Site (ID)', @@ -439,7 +444,7 @@ class DeviceFilter(CustomFieldFilterSet, django_filters.FilterSet): class Meta: model = Device - fields = ['name', 'serial', 'asset_tag'] + fields = ['serial'] def search(self, queryset, name, value): if not value.strip(): @@ -457,7 +462,8 @@ class DeviceFilter(CustomFieldFilterSet, django_filters.FilterSet): if not value: return queryset try: - return queryset.filter(interfaces__mac_address=value).distinct() + mac = EUI(value.strip()) + return queryset.filter(interfaces__mac_address=mac).distinct() except AddrFormatError: return queryset.none() @@ -569,7 +575,8 @@ class InterfaceFilter(django_filters.FilterSet): if not value: return queryset try: - return queryset.filter(mac_address=value) + mac = EUI(value.strip()) + return queryset.filter(mac_address=mac) except AddrFormatError: return queryset.none() @@ -596,10 +603,11 @@ class InventoryItemFilter(DeviceComponentFilterSet): to_field_name='slug', label='Manufacturer (slug)', ) + asset_tag = NullableCharFieldFilter() class Meta: model = InventoryItem - fields = ['name', 'part_id', 'serial', 'asset_tag', 'discovered'] + fields = ['name', 'part_id', 'serial', 'discovered'] class ConsoleConnectionFilter(django_filters.FilterSet): diff --git a/netbox/dcim/forms.py b/netbox/dcim/forms.py index bf6516b55..9cf66395f 100644 --- a/netbox/dcim/forms.py +++ b/netbox/dcim/forms.py @@ -12,7 +12,7 @@ from ipam.models import IPAddress from tenancy.forms import TenancyForm from tenancy.models import Tenant from utilities.forms import ( - APISelect, add_blank_choice, ArrayFieldSelectMultiple, BootstrapMixin, BulkEditForm, BulkEditNullBooleanSelect, + APISelect, ArrayFieldSelectMultiple, add_blank_choice, BootstrapMixin, BulkEditForm, BulkEditNullBooleanSelect, ChainedFieldsMixin, ChainedModelChoiceField, CommentField, ConfirmationForm, CSVChoiceField, ExpandableNameField, FilterChoiceField, FlexibleModelChoiceField, Livesearch, SelectWithDisabled, SmallTextarea, SlugField, FilterTreeNodeMultipleChoiceField, @@ -28,12 +28,6 @@ from .models import ( ) -FORM_STATUS_CHOICES = [ - ['', '---------'], -] - -FORM_STATUS_CHOICES += STATUS_CHOICES - DEVICE_BY_PK_RE = '{\d+\}' @@ -728,13 +722,28 @@ class DeviceForm(BootstrapMixin, TenancyForm, CustomFieldForm): # Compile list of choices for primary IPv4 and IPv6 addresses for family in [4, 6]: - ip_choices = [] - interface_ips = IPAddress.objects.filter(family=family, interface__device=self.instance) - ip_choices += [(ip.id, '{} ({})'.format(ip.address, ip.interface)) for ip in interface_ips] - nat_ips = IPAddress.objects.filter(family=family, nat_inside__interface__device=self.instance)\ - .select_related('nat_inside__interface') - ip_choices += [(ip.id, '{} ({} NAT)'.format(ip.address, ip.nat_inside.interface)) for ip in nat_ips] - self.fields['primary_ip{}'.format(family)].choices = [(None, '---------')] + ip_choices + ip_choices = [(None, '---------')] + # Collect interface IPs + interface_ips = IPAddress.objects.select_related('interface').filter( + family=family, interface__device=self.instance + ) + if interface_ips: + ip_choices.append( + ('Interface IPs', [ + (ip.id, '{} ({})'.format(ip.address, ip.interface)) for ip in interface_ips + ]) + ) + # Collect NAT IPs + nat_ips = IPAddress.objects.select_related('nat_inside').filter( + family=family, nat_inside__interface__device=self.instance + ) + if nat_ips: + ip_choices.append( + ('NAT IPs', [ + (ip.id, '{} ({})'.format(ip.address, ip.nat_inside.address)) for ip in nat_ips + ]) + ) + self.fields['primary_ip{}'.format(family)].choices = ip_choices # If editing an existing device, exclude it from the list of occupied rack units. This ensures that a device # can be flipped from one face to another. @@ -934,7 +943,7 @@ class DeviceBulkEditForm(BootstrapMixin, CustomFieldBulkEditForm): device_role = forms.ModelChoiceField(queryset=DeviceRole.objects.all(), required=False, label='Role') tenant = forms.ModelChoiceField(queryset=Tenant.objects.all(), required=False) platform = forms.ModelChoiceField(queryset=Platform.objects.all(), required=False) - status = forms.ChoiceField(choices=FORM_STATUS_CHOICES, required=False, initial='', label='Status') + status = forms.ChoiceField(choices=add_blank_choice(STATUS_CHOICES), required=False, initial='') serial = forms.CharField(max_length=50, required=False, label='Serial Number') class Meta: diff --git a/netbox/dcim/migrations/0042_interface_ff_10ge_cx4.py b/netbox/dcim/migrations/0042_interface_ff_10ge_cx4.py new file mode 100644 index 000000000..77bea6bc6 --- /dev/null +++ b/netbox/dcim/migrations/0042_interface_ff_10ge_cx4.py @@ -0,0 +1,25 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.11.4 on 2017-08-29 21:00 +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('dcim', '0041_napalm_integration'), + ] + + operations = [ + migrations.AlterField( + model_name='interface', + name='form_factor', + field=models.PositiveSmallIntegerField(choices=[['Virtual interfaces', [[0, 'Virtual'], [200, 'Link Aggregation Group (LAG)']]], ['Ethernet (fixed)', [[800, '100BASE-TX (10/100ME)'], [1000, '1000BASE-T (1GE)'], [1150, '10GBASE-T (10GE)'], [1170, '10GBASE-CX4 (10GE)']]], ['Ethernet (modular)', [[1050, 'GBIC (1GE)'], [1100, 'SFP (1GE)'], [1200, 'SFP+ (10GE)'], [1300, 'XFP (10GE)'], [1310, 'XENPAK (10GE)'], [1320, 'X2 (10GE)'], [1350, 'SFP28 (25GE)'], [1400, 'QSFP+ (40GE)'], [1500, 'CFP (100GE)'], [1600, 'QSFP28 (100GE)']]], ['Wireless', [[2600, 'IEEE 802.11a'], [2610, 'IEEE 802.11b/g'], [2620, 'IEEE 802.11n'], [2630, 'IEEE 802.11ac'], [2640, 'IEEE 802.11ad']]], ['FibreChannel', [[3010, 'SFP (1GFC)'], [3020, 'SFP (2GFC)'], [3040, 'SFP (4GFC)'], [3080, 'SFP+ (8GFC)'], [3160, 'SFP+ (16GFC)']]], ['Serial', [[4000, 'T1 (1.544 Mbps)'], [4010, 'E1 (2.048 Mbps)'], [4040, 'T3 (45 Mbps)'], [4050, 'E3 (34 Mbps)']]], ['Stacking', [[5000, 'Cisco StackWise'], [5050, 'Cisco StackWise Plus'], [5100, 'Cisco FlexStack'], [5150, 'Cisco FlexStack Plus'], [5200, 'Juniper VCP']]], ['Other', [[32767, 'Other']]]], default=1200), + ), + migrations.AlterField( + model_name='interfacetemplate', + name='form_factor', + field=models.PositiveSmallIntegerField(choices=[['Virtual interfaces', [[0, 'Virtual'], [200, 'Link Aggregation Group (LAG)']]], ['Ethernet (fixed)', [[800, '100BASE-TX (10/100ME)'], [1000, '1000BASE-T (1GE)'], [1150, '10GBASE-T (10GE)'], [1170, '10GBASE-CX4 (10GE)']]], ['Ethernet (modular)', [[1050, 'GBIC (1GE)'], [1100, 'SFP (1GE)'], [1200, 'SFP+ (10GE)'], [1300, 'XFP (10GE)'], [1310, 'XENPAK (10GE)'], [1320, 'X2 (10GE)'], [1350, 'SFP28 (25GE)'], [1400, 'QSFP+ (40GE)'], [1500, 'CFP (100GE)'], [1600, 'QSFP28 (100GE)']]], ['Wireless', [[2600, 'IEEE 802.11a'], [2610, 'IEEE 802.11b/g'], [2620, 'IEEE 802.11n'], [2630, 'IEEE 802.11ac'], [2640, 'IEEE 802.11ad']]], ['FibreChannel', [[3010, 'SFP (1GFC)'], [3020, 'SFP (2GFC)'], [3040, 'SFP (4GFC)'], [3080, 'SFP+ (8GFC)'], [3160, 'SFP+ (16GFC)']]], ['Serial', [[4000, 'T1 (1.544 Mbps)'], [4010, 'E1 (2.048 Mbps)'], [4040, 'T3 (45 Mbps)'], [4050, 'E3 (34 Mbps)']]], ['Stacking', [[5000, 'Cisco StackWise'], [5050, 'Cisco StackWise Plus'], [5100, 'Cisco FlexStack'], [5150, 'Cisco FlexStack Plus'], [5200, 'Juniper VCP']]], ['Other', [[32767, 'Other']]]], default=1200), + ), + ] diff --git a/netbox/dcim/migrations/0043_device_component_name_lengths.py b/netbox/dcim/migrations/0043_device_component_name_lengths.py new file mode 100644 index 000000000..a52f50859 --- /dev/null +++ b/netbox/dcim/migrations/0043_device_component_name_lengths.py @@ -0,0 +1,70 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.11.4 on 2017-08-29 21:26 +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('dcim', '0042_interface_ff_10ge_cx4'), + ] + + operations = [ + migrations.AlterField( + model_name='consoleport', + name='name', + field=models.CharField(max_length=50), + ), + migrations.AlterField( + model_name='consoleporttemplate', + name='name', + field=models.CharField(max_length=50), + ), + migrations.AlterField( + model_name='consoleserverport', + name='name', + field=models.CharField(max_length=50), + ), + migrations.AlterField( + model_name='consoleserverporttemplate', + name='name', + field=models.CharField(max_length=50), + ), + migrations.AlterField( + model_name='devicebaytemplate', + name='name', + field=models.CharField(max_length=50), + ), + migrations.AlterField( + model_name='interface', + name='name', + field=models.CharField(max_length=64), + ), + migrations.AlterField( + model_name='interfacetemplate', + name='name', + field=models.CharField(max_length=64), + ), + migrations.AlterField( + model_name='poweroutlet', + name='name', + field=models.CharField(max_length=50), + ), + migrations.AlterField( + model_name='poweroutlettemplate', + name='name', + field=models.CharField(max_length=50), + ), + migrations.AlterField( + model_name='powerport', + name='name', + field=models.CharField(max_length=50), + ), + migrations.AlterField( + model_name='powerporttemplate', + name='name', + field=models.CharField(max_length=50), + ), + ] diff --git a/netbox/dcim/models.py b/netbox/dcim/models.py index e24a05e18..14d49e1b7 100644 --- a/netbox/dcim/models.py +++ b/netbox/dcim/models.py @@ -13,6 +13,7 @@ from django.core.exceptions import ValidationError from django.core.validators import MaxValueValidator, MinValueValidator from django.db import models from django.db.models import Count, Q, ObjectDoesNotExist +from django.db.models.expressions import RawSQL from django.urls import reverse from django.utils.encoding import python_2_unicode_compatible @@ -379,6 +380,16 @@ class Rack(CreatedUpdatedModel, CustomFieldModel): return list(reversed(available_units)) + def get_reserved_units(self): + """ + Return a dictionary mapping all reserved units within the rack to their reservation. + """ + reserved_units = {} + for r in self.reservations.all(): + for u in r.units: + reserved_units[u] = r + return reserved_units + def get_0u_devices(self): return self.devices.filter(position=0) @@ -616,7 +627,7 @@ class ConsolePortTemplate(models.Model): A template for a ConsolePort to be created for a new Device. """ device_type = models.ForeignKey('DeviceType', related_name='console_port_templates', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) class Meta: ordering = ['device_type', 'name'] @@ -632,7 +643,7 @@ class ConsoleServerPortTemplate(models.Model): A template for a ConsoleServerPort to be created for a new Device. """ device_type = models.ForeignKey('DeviceType', related_name='cs_port_templates', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) class Meta: ordering = ['device_type', 'name'] @@ -648,7 +659,7 @@ class PowerPortTemplate(models.Model): A template for a PowerPort to be created for a new Device. """ device_type = models.ForeignKey('DeviceType', related_name='power_port_templates', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) class Meta: ordering = ['device_type', 'name'] @@ -664,7 +675,7 @@ class PowerOutletTemplate(models.Model): A template for a PowerOutlet to be created for a new Device. """ device_type = models.ForeignKey('DeviceType', related_name='power_outlet_templates', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) class Meta: ordering = ['device_type', 'name'] @@ -684,15 +695,16 @@ class InterfaceQuerySet(models.QuerySet): To order interfaces naturally, the `name` field is split into six distinct components: leading text (type), slot, subslot, position, channel, and virtual circuit: - {type}{slot}/{subslot}/{position}:{channel}.{vc} + {type}{slot}/{subslot}/{position}/{subposition}:{channel}.{vc} - Components absent from the interface name are ignored. For example, an interface named GigabitEthernet0/1 would - be parsed as follows: + Components absent from the interface name are ignored. For example, an interface named GigabitEthernet1/2/3 + would be parsed as follows: name = 'GigabitEthernet' - slot = None - subslot = 0 - position = 1 + slot = 1 + subslot = 2 + position = 3 + subposition = 0 channel = None vc = 0 @@ -701,17 +713,35 @@ class InterfaceQuerySet(models.QuerySet): """ sql_col = '{}.name'.format(self.model._meta.db_table) ordering = { - IFACE_ORDERING_POSITION: ('_slot', '_subslot', '_position', '_channel', '_vc', '_type', 'name'), - IFACE_ORDERING_NAME: ('_type', '_slot', '_subslot', '_position', '_channel', '_vc', 'name'), + IFACE_ORDERING_POSITION: ( + '_slot', '_subslot', '_position', '_subposition', '_channel', '_vc', '_type', '_id', 'name', + ), + IFACE_ORDERING_NAME: ( + '_type', '_slot', '_subslot', '_position', '_subposition', '_channel', '_vc', '_id', 'name', + ), }[method] - return self.extra(select={ - '_type': "SUBSTRING({} FROM '^([^0-9]+)')".format(sql_col), - '_slot': "CAST(SUBSTRING({} FROM '([0-9]+)\/[0-9]+\/[0-9]+(:[0-9]+)?(\.[0-9]+)?$') AS integer)".format(sql_col), - '_subslot': "CAST(SUBSTRING({} FROM '([0-9]+)\/[0-9]+(:[0-9]+)?(\.[0-9]+)?$') AS integer)".format(sql_col), - '_position': "CAST(SUBSTRING({} FROM '([0-9]+)(:[0-9]+)?(\.[0-9]+)?$') AS integer)".format(sql_col), - '_channel': "COALESCE(CAST(SUBSTRING({} FROM ':([0-9]+)(\.[0-9]+)?$') AS integer), 0)".format(sql_col), - '_vc': "COALESCE(CAST(SUBSTRING({} FROM '\.([0-9]+)$') AS integer), 0)".format(sql_col), - }).order_by(*ordering) + + TYPE_RE = r"SUBSTRING({} FROM '^([^0-9]+)')" + ID_RE = r"CAST(SUBSTRING({} FROM '^(?:[^0-9]+)([0-9]+)$') AS integer)" + SLOT_RE = r"CAST(SUBSTRING({} FROM '^(?:[^0-9]+)([0-9]+)\/') AS integer)" + SUBSLOT_RE = r"CAST(SUBSTRING({} FROM '^(?:[^0-9]+)(?:[0-9]+\/)([0-9]+)') AS integer)" + POSITION_RE = r"CAST(SUBSTRING({} FROM '^(?:[^0-9]+)(?:[0-9]+\/){{2}}([0-9]+)') AS integer)" + SUBPOSITION_RE = r"CAST(SUBSTRING({} FROM '^(?:[^0-9]+)(?:[0-9]+\/){{3}}([0-9]+)') AS integer)" + CHANNEL_RE = r"COALESCE(CAST(SUBSTRING({} FROM ':([0-9]+)(\.[0-9]+)?$') AS integer), 0)" + VC_RE = r"COALESCE(CAST(SUBSTRING({} FROM '\.([0-9]+)$') AS integer), 0)" + + fields = { + '_type': RawSQL(TYPE_RE.format(sql_col), []), + '_id': RawSQL(ID_RE.format(sql_col), []), + '_slot': RawSQL(SLOT_RE.format(sql_col), []), + '_subslot': RawSQL(SUBSLOT_RE.format(sql_col), []), + '_position': RawSQL(POSITION_RE.format(sql_col), []), + '_subposition': RawSQL(SUBPOSITION_RE.format(sql_col), []), + '_channel': RawSQL(CHANNEL_RE.format(sql_col), []), + '_vc': RawSQL(VC_RE.format(sql_col), []), + } + + return self.annotate(**fields).order_by(*ordering) def connectable(self): """ @@ -727,7 +757,7 @@ class InterfaceTemplate(models.Model): A template for a physical data interface on a new Device. """ device_type = models.ForeignKey('DeviceType', related_name='interface_templates', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=64) form_factor = models.PositiveSmallIntegerField(choices=IFACE_FF_CHOICES, default=IFACE_FF_10GE_SFP_PLUS) mgmt_only = models.BooleanField(default=False, verbose_name='Management only') @@ -747,7 +777,7 @@ class DeviceBayTemplate(models.Model): A template for a DeviceBay to be created for a new parent Device. """ device_type = models.ForeignKey('DeviceType', related_name='device_bay_templates', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) class Meta: ordering = ['device_type', 'name'] @@ -932,6 +962,30 @@ class Device(CreatedUpdatedModel, CustomFieldModel): except DeviceType.DoesNotExist: pass + # Validate primary IPv4 address + if self.primary_ip4 and ( + self.primary_ip4.interface is None or + self.primary_ip4.interface.device != self + ) and ( + self.primary_ip4.nat_inside.interface is None or + self.primary_ip4.nat_inside.interface.device != self + ): + raise ValidationError({ + 'primary_ip4': "The specified IP address ({}) is not assigned to this device.".format(self.primary_ip4), + }) + + # Validate primary IPv6 address + if self.primary_ip6 and ( + self.primary_ip6.interface is None or + self.primary_ip6.interface.device != self + ) and ( + self.primary_ip6.nat_inside.interface is None or + self.primary_ip6.nat_inside.interface.device != self + ): + raise ValidationError({ + 'primary_ip6': "The specified IP address ({}) is not assigned to this device.".format(self.primary_ip6), + }) + def save(self, *args, **kwargs): is_new = not bool(self.pk) @@ -1042,7 +1096,7 @@ class ConsolePort(models.Model): A physical console port within a Device. ConsolePorts connect to ConsoleServerPorts. """ device = models.ForeignKey('Device', related_name='console_ports', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) cs_port = models.OneToOneField('ConsoleServerPort', related_name='connected_console', on_delete=models.SET_NULL, verbose_name='Console server port', blank=True, null=True) connection_status = models.NullBooleanField(choices=CONNECTION_STATUS_CHOICES, default=CONNECTION_STATUS_CONNECTED) @@ -1092,7 +1146,7 @@ class ConsoleServerPort(models.Model): A physical port within a Device (typically a designated console server) which provides access to ConsolePorts. """ device = models.ForeignKey('Device', related_name='cs_ports', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) objects = ConsoleServerPortManager() @@ -1113,7 +1167,7 @@ class PowerPort(models.Model): A physical power supply (intake) port within a Device. PowerPorts connect to PowerOutlets. """ device = models.ForeignKey('Device', related_name='power_ports', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) power_outlet = models.OneToOneField('PowerOutlet', related_name='connected_port', on_delete=models.SET_NULL, blank=True, null=True) connection_status = models.NullBooleanField(choices=CONNECTION_STATUS_CHOICES, default=CONNECTION_STATUS_CONNECTED) @@ -1157,7 +1211,7 @@ class PowerOutlet(models.Model): A physical power outlet (output) within a Device which provides power to a PowerPort. """ device = models.ForeignKey('Device', related_name='power_outlets', on_delete=models.CASCADE) - name = models.CharField(max_length=30) + name = models.CharField(max_length=50) objects = PowerOutletManager() @@ -1187,7 +1241,7 @@ class Interface(models.Model): blank=True, verbose_name='Parent LAG' ) - name = models.CharField(max_length=30) + name = models.CharField(max_length=64) form_factor = models.PositiveSmallIntegerField(choices=IFACE_FF_CHOICES, default=IFACE_FF_10GE_SFP_PLUS) enabled = models.BooleanField(default=True) mac_address = MACAddressField(null=True, blank=True, verbose_name='MAC Address') diff --git a/netbox/dcim/tests/test_models.py b/netbox/dcim/tests/test_models.py index 340c58092..b4101cfd5 100644 --- a/netbox/dcim/tests/test_models.py +++ b/netbox/dcim/tests/test_models.py @@ -98,3 +98,112 @@ class RackTestCase(TestCase): face=None, ) self.assertTrue(pdu) + + +class InterfaceTestCase(TestCase): + + def setUp(self): + + self.site = Site.objects.create( + name='TestSite1', + slug='my-test-site' + ) + self.rack = Rack.objects.create( + name='TestRack1', + facility_id='A101', + site=self.site, + u_height=42 + ) + self.manufacturer = Manufacturer.objects.create( + name='Acme', + slug='acme' + ) + + self.device_type = DeviceType.objects.create( + manufacturer=self.manufacturer, + model='FrameForwarder 2048', + slug='ff2048' + ) + self.role = DeviceRole.objects.create( + name='Switch', + slug='switch', + ) + + def test_interface_order_natural(self): + device1 = Device.objects.create( + name='TestSwitch1', + device_type=self.device_type, + device_role=self.role, + site=self.site, + rack=self.rack, + position=10, + face=RACK_FACE_REAR, + ) + interface1 = Interface.objects.create( + device=device1, + name='Ethernet1/3/1' + ) + interface2 = Interface.objects.create( + device=device1, + name='Ethernet1/5/1' + ) + interface3 = Interface.objects.create( + device=device1, + name='Ethernet1/4' + ) + interface4 = Interface.objects.create( + device=device1, + name='Ethernet1/3/2/4' + ) + interface5 = Interface.objects.create( + device=device1, + name='Ethernet1/3/2/1' + ) + interface6 = Interface.objects.create( + device=device1, + name='Loopback1' + ) + + self.assertEqual( + list(Interface.objects.all().order_naturally()), + [interface1, interface5, interface4, interface3, interface2, interface6] + ) + + def test_interface_order_natural_subinterfaces(self): + device1 = Device.objects.create( + name='TestSwitch1', + device_type=self.device_type, + device_role=self.role, + site=self.site, + rack=self.rack, + position=10, + face=RACK_FACE_REAR, + ) + interface1 = Interface.objects.create( + device=device1, + name='GigabitEthernet0/0/3' + ) + interface2 = Interface.objects.create( + device=device1, + name='GigabitEthernet0/0/2.2' + ) + interface3 = Interface.objects.create( + device=device1, + name='GigabitEthernet0/0/0.120' + ) + interface4 = Interface.objects.create( + device=device1, + name='GigabitEthernet0/0/0' + ) + interface5 = Interface.objects.create( + device=device1, + name='GigabitEthernet0/0/1.117' + ) + interface6 = Interface.objects.create( + device=device1, + name='GigabitEthernet0' + ) + self.assertEqual( + list(Interface.objects.all().order_naturally()), + [interface4, interface3, interface5, interface2, interface1, interface6] + ) diff --git a/netbox/dcim/views.py b/netbox/dcim/views.py index 0e3114b8c..a491731c4 100644 --- a/netbox/dcim/views.py +++ b/netbox/dcim/views.py @@ -431,15 +431,10 @@ class RackView(View): prev_rack = Rack.objects.filter(site=rack.site, name__lt=rack.name).order_by('-name').first() reservations = RackReservation.objects.filter(rack=rack) - reserved_units = {} - for r in reservations: - for u in r.units: - reserved_units[u] = r return render(request, 'dcim/rack.html', { 'rack': rack, 'reservations': reservations, - 'reserved_units': reserved_units, 'nonracked_devices': nonracked_devices, 'next_rack': next_rack, 'prev_rack': prev_rack, @@ -969,7 +964,7 @@ class DeviceLLDPNeighborsView(PermissionRequiredMixin, View): device = get_object_or_404(Device, pk=pk) interfaces = Interface.objects.order_naturally( device.device_type.interface_ordering - ).filter( + ).connectable().filter( device=device ).select_related( 'connected_as_a', 'connected_as_b' diff --git a/netbox/extras/api/customfields.py b/netbox/extras/api/customfields.py index 52f127a7d..5554df924 100644 --- a/netbox/extras/api/customfields.py +++ b/netbox/extras/api/customfields.py @@ -10,6 +10,7 @@ from django.db import transaction from extras.models import ( CF_TYPE_BOOLEAN, CF_TYPE_DATE, CF_TYPE_SELECT, CustomField, CustomFieldChoice, CustomFieldValue, ) +from utilities.api import ValidatedModelSerializer # @@ -28,34 +29,47 @@ class CustomFieldsSerializer(serializers.BaseSerializer): for field_name, value in data.items(): - cf = custom_fields[field_name] + try: + cf = custom_fields[field_name] + except KeyError: + raise ValidationError( + "Invalid custom field for {} objects: {}".format(content_type, field_name) + ) - # Validate custom field name - if field_name not in custom_fields: - raise ValidationError("Invalid custom field for {} objects: {}".format(content_type, field_name)) + # Data validation + if value not in [None, '']: - # Validate boolean - if cf.type == CF_TYPE_BOOLEAN and value not in [True, False, 1, 0]: - raise ValidationError("Invalid value for boolean field {}: {}".format(field_name, value)) + # Validate boolean + if cf.type == CF_TYPE_BOOLEAN and value not in [True, False, 1, 0]: + raise ValidationError( + "Invalid value for boolean field {}: {}".format(field_name, value) + ) - # Validate date - if cf.type == CF_TYPE_DATE: - try: - datetime.strptime(value, '%Y-%m-%d') - except ValueError: - raise ValidationError("Invalid date for field {}: {}. (Required format is YYYY-MM-DD.)".format( - field_name, value - )) + # Validate date + if cf.type == CF_TYPE_DATE: + try: + datetime.strptime(value, '%Y-%m-%d') + except ValueError: + raise ValidationError( + "Invalid date for field {}: {}. (Required format is YYYY-MM-DD.)".format(field_name, value) + ) - # Validate selected choice - if cf.type == CF_TYPE_SELECT: - try: - value = int(value) - except ValueError: - raise ValidationError("{}: Choice selections must be passed as integers.".format(field_name)) - valid_choices = [c.pk for c in cf.choices.all()] - if value not in valid_choices: - raise ValidationError("Invalid choice for field {}: {}".format(field_name, value)) + # Validate selected choice + if cf.type == CF_TYPE_SELECT: + try: + value = int(value) + except ValueError: + raise ValidationError( + "{}: Choice selections must be passed as integers.".format(field_name) + ) + valid_choices = [c.pk for c in cf.choices.all()] + if value not in valid_choices: + raise ValidationError( + "Invalid choice for field {}: {}".format(field_name, value) + ) + + elif cf.required: + raise ValidationError("Required field {} cannot be empty.".format(field_name)) # Check for missing required fields missing_fields = [] @@ -68,7 +82,7 @@ class CustomFieldsSerializer(serializers.BaseSerializer): return data -class CustomFieldModelSerializer(serializers.ModelSerializer): +class CustomFieldModelSerializer(ValidatedModelSerializer): """ Extends ModelSerializer to render any CustomFields and their values associated with an object. """ @@ -111,16 +125,6 @@ class CustomFieldModelSerializer(serializers.ModelSerializer): defaults={'serialized_value': custom_field.serialize_value(value)}, ) - def validate(self, data): - """ - Enforce model validation (see utilities.api.ModelValidationMixin) - """ - model_data = data.copy() - model_data.pop('custom_fields', None) - instance = self.Meta.model(**model_data) - instance.clean() - return data - def create(self, validated_data): custom_fields = validated_data.pop('custom_fields', None) diff --git a/netbox/extras/api/serializers.py b/netbox/extras/api/serializers.py index 39ce63524..0eeab49ec 100644 --- a/netbox/extras/api/serializers.py +++ b/netbox/extras/api/serializers.py @@ -10,7 +10,7 @@ from extras.models import ( ACTION_CHOICES, ExportTemplate, Graph, GRAPH_TYPE_CHOICES, ImageAttachment, TopologyMap, UserAction, ) from users.api.serializers import NestedUserSerializer -from utilities.api import ChoiceFieldSerializer, ContentTypeFieldSerializer, ModelValidationMixin +from utilities.api import ChoiceFieldSerializer, ContentTypeFieldSerializer, ValidatedModelSerializer # @@ -104,7 +104,7 @@ class ImageAttachmentSerializer(serializers.ModelSerializer): return serializer(obj.parent, context={'request': self.context['request']}).data -class WritableImageAttachmentSerializer(ModelValidationMixin, serializers.ModelSerializer): +class WritableImageAttachmentSerializer(ValidatedModelSerializer): content_type = ContentTypeFieldSerializer() class Meta: diff --git a/netbox/extras/management/commands/nbshell.py b/netbox/extras/management/commands/nbshell.py index 48448c16f..a50b1384d 100644 --- a/netbox/extras/management/commands/nbshell.py +++ b/netbox/extras/management/commands/nbshell.py @@ -37,19 +37,29 @@ class Command(BaseCommand): def get_namespace(self): namespace = {} - # Gather Django models from each app + # Gather Django models and constants from each app for app in APPS: self.django_models[app] = [] + + # Models app_models = sys.modules['{}.models'.format(app)] for name in dir(app_models): model = getattr(app_models, name) try: - if issubclass(model, Model): + if issubclass(model, Model) and model._meta.app_label == app: namespace[name] = model self.django_models[app].append(name) except TypeError: pass + # Constants + try: + app_constants = sys.modules['{}.constants'.format(app)] + for name in dir(app_constants): + namespace[name] = getattr(app_constants, name) + except KeyError: + pass + # Load convenience commands namespace.update({ 'lsmodels': self._lsmodels, diff --git a/netbox/extras/management/commands/run_inventory.py b/netbox/extras/management/commands/run_inventory.py index 1e52b5c8f..335cdb783 100644 --- a/netbox/extras/management/commands/run_inventory.py +++ b/netbox/extras/management/commands/run_inventory.py @@ -13,8 +13,8 @@ from dcim.models import Device, InventoryItem, Site, STATUS_ACTIVE class Command(BaseCommand): help = "Update inventory information for specified devices" - username = settings.NETBOX_USERNAME - password = settings.NETBOX_PASSWORD + username = settings.NAPALM_USERNAME + password = settings.NAPALM_PASSWORD def add_arguments(self, parser): parser.add_argument('-u', '--username', dest='username', help="Specify the username to use") diff --git a/netbox/extras/models.py b/netbox/extras/models.py index 9d0e636ff..4afc3afcf 100644 --- a/netbox/extras/models.py +++ b/netbox/extras/models.py @@ -285,7 +285,7 @@ class TopologyMap(models.Model): # Add each device to the graph devices = [] - for query in device_set.split(';'): # Split regexes on semicolons + for query in device_set.strip(';').split(';'): # Split regexes on semicolons devices += Device.objects.filter(name__regex=query).select_related('device_role') for d in devices: bg_color = '#{}'.format(d.device_role.color) diff --git a/netbox/ipam/api/serializers.py b/netbox/ipam/api/serializers.py index 1374d3552..3ef152ebe 100644 --- a/netbox/ipam/api/serializers.py +++ b/netbox/ipam/api/serializers.py @@ -11,7 +11,7 @@ from ipam.models import ( PREFIX_STATUS_CHOICES, RIR, Role, Service, VLAN, VLAN_STATUS_CHOICES, VLANGroup, VRF, ) from tenancy.api.serializers import NestedTenantSerializer -from utilities.api import ChoiceFieldSerializer, ModelValidationMixin +from utilities.api import ChoiceFieldSerializer, ValidatedModelSerializer # @@ -45,7 +45,7 @@ class WritableVRFSerializer(CustomFieldModelSerializer): # Roles # -class RoleSerializer(ModelValidationMixin, serializers.ModelSerializer): +class RoleSerializer(ValidatedModelSerializer): class Meta: model = Role @@ -64,7 +64,7 @@ class NestedRoleSerializer(serializers.ModelSerializer): # RIRs # -class RIRSerializer(ModelValidationMixin, serializers.ModelSerializer): +class RIRSerializer(ValidatedModelSerializer): class Meta: model = RIR @@ -303,7 +303,7 @@ class ServiceSerializer(serializers.ModelSerializer): fields = ['id', 'device', 'name', 'port', 'protocol', 'ipaddresses', 'description'] -# TODO: Figure out how to use ModelValidationMixin with ManyToManyFields. Calling clean() yields a ValueError. +# TODO: Figure out how to use model validation with ManyToManyFields. Calling clean() yields a ValueError. class WritableServiceSerializer(serializers.ModelSerializer): class Meta: diff --git a/netbox/ipam/api/views.py b/netbox/ipam/api/views.py index 87511d5c5..abff09c15 100644 --- a/netbox/ipam/api/views.py +++ b/netbox/ipam/api/views.py @@ -98,7 +98,7 @@ class PrefixViewSet(WritableSerializerMixin, CustomFieldModelViewSet): # Create the new IP address data = request.data.copy() data['address'] = '{}/{}'.format(ipaddress, prefix.prefix.prefixlen) - data['vrf'] = prefix.vrf + data['vrf'] = prefix.vrf.pk if prefix.vrf else None serializer = serializers.WritableIPAddressSerializer(data=data) if serializer.is_valid(): serializer.save() @@ -115,7 +115,11 @@ class PrefixViewSet(WritableSerializerMixin, CustomFieldModelViewSet): limit = min(limit, settings.MAX_PAGE_SIZE) # Calculate available IPs within the prefix - ip_list = list(prefix.get_available_ips())[:limit] + ip_list = [] + for index, ip in enumerate(prefix.get_available_ips(), start=1): + ip_list.append(ip) + if index == limit: + break serializer = serializers.AvailableIPSerializer(ip_list, many=True, context={ 'request': request, 'prefix': prefix.prefix, diff --git a/netbox/ipam/forms.py b/netbox/ipam/forms.py index e19376e8e..4152b44c1 100644 --- a/netbox/ipam/forms.py +++ b/netbox/ipam/forms.py @@ -25,11 +25,8 @@ IP_FAMILY_CHOICES = [ (6, 'IPv6'), ] -PREFIX_MASK_LENGTH_CHOICES = [ - ('', '---------'), -] + [(i, i) for i in range(1, 128)] - -IPADDRESS_MASK_LENGTH_CHOICES = PREFIX_MASK_LENGTH_CHOICES + [(128, 128)] +PREFIX_MASK_LENGTH_CHOICES = add_blank_choice([(i, i) for i in range(1, 128)]) +IPADDRESS_MASK_LENGTH_CHOICES = add_blank_choice([(i, i) for i in range(1, 129)]) # diff --git a/netbox/ipam/migrations/0018_remove_service_uniqueness_constraint.py b/netbox/ipam/migrations/0018_remove_service_uniqueness_constraint.py new file mode 100644 index 000000000..77e083ef3 --- /dev/null +++ b/netbox/ipam/migrations/0018_remove_service_uniqueness_constraint.py @@ -0,0 +1,19 @@ +# -*- coding: utf-8 -*- +# Generated by Django 1.11.3 on 2017-08-03 19:37 +from __future__ import unicode_literals + +from django.db import migrations + + +class Migration(migrations.Migration): + + dependencies = [ + ('ipam', '0017_ipaddress_roles'), + ] + + operations = [ + migrations.AlterUniqueTogether( + name='service', + unique_together=set([]), + ), + ] diff --git a/netbox/ipam/models.py b/netbox/ipam/models.py index 04853c5da..214293b7d 100644 --- a/netbox/ipam/models.py +++ b/netbox/ipam/models.py @@ -512,6 +512,16 @@ class VLANGroup(models.Model): def get_absolute_url(self): return "{}?group_id={}".format(reverse('ipam:vlan_list'), self.pk) + def get_next_available_vid(self): + """ + Return the first available VLAN ID (1-4094) in the group. + """ + vids = [vlan['vid'] for vlan in self.vlans.order_by('vid').values('vid')] + for i in range(1, 4095): + if i not in vids: + return i + return None + @python_2_unicode_compatible class VLAN(CreatedUpdatedModel, CustomFieldModel): @@ -600,7 +610,6 @@ class Service(CreatedUpdatedModel): class Meta: ordering = ['device', 'protocol', 'port'] - unique_together = ['device', 'protocol', 'port'] def __str__(self): return '{} ({}/{})'.format(self.name, self.port, self.get_protocol_display()) diff --git a/netbox/ipam/tables.py b/netbox/ipam/tables.py index 65ab5b2e4..24bb9b3c2 100644 --- a/netbox/ipam/tables.py +++ b/netbox/ipam/tables.py @@ -34,7 +34,7 @@ RIR_ACTIONS = """ UTILIZATION_GRAPH = """ {% load helpers %} -{% if record.pk %}{% utilization_graph value %}{% else %}—{% endif %} +{% if record.pk %}{% utilization_graph record.get_utilization %}{% else %}—{% endif %} """ ROLE_ACTIONS = """ @@ -45,9 +45,9 @@ ROLE_ACTIONS = """ PREFIX_LINK = """ {% if record.has_children %} - + {% else %} - + {% endif %} {{ record.prefix }} @@ -80,7 +80,6 @@ IPADDRESS_LINK = """ IPADDRESS_DEVICE = """ {% if record.interface %} {{ record.interface.device }} - ({{ record.interface.name }}) {% else %} — {% endif %} @@ -121,6 +120,13 @@ VLAN_ROLE_LINK = """ """ VLANGROUP_ACTIONS = """ +{% with next_vid=record.get_next_available_vid %} + {% if next_vid and perms.ipam.add_vlan %} + + + + {% endif %} +{% endwith %} {% if perms.ipam.change_vlangroup %} {% endif %} @@ -204,10 +210,10 @@ class AggregateTable(BaseTable): class AggregateDetailTable(AggregateTable): child_count = tables.Column(verbose_name='Prefixes') - get_utilization = tables.TemplateColumn(UTILIZATION_GRAPH, orderable=False, verbose_name='Utilization') + utilization = tables.TemplateColumn(UTILIZATION_GRAPH, orderable=False, verbose_name='Utilization') class Meta(AggregateTable.Meta): - fields = ('pk', 'prefix', 'rir', 'child_count', 'get_utilization', 'date_added', 'description') + fields = ('pk', 'prefix', 'rir', 'child_count', 'utilization', 'date_added', 'description') # @@ -250,10 +256,10 @@ class PrefixTable(BaseTable): class PrefixDetailTable(PrefixTable): - get_utilization = tables.TemplateColumn(UTILIZATION_GRAPH, orderable=False, verbose_name='Utilization') + utilization = tables.TemplateColumn(UTILIZATION_GRAPH, orderable=False) class Meta(PrefixTable.Meta): - fields = ('pk', 'prefix', 'status', 'vrf', 'get_utilization', 'tenant', 'site', 'vlan', 'role', 'description') + fields = ('pk', 'prefix', 'status', 'vrf', 'utilization', 'tenant', 'site', 'vlan', 'role', 'description') # diff --git a/netbox/ipam/views.py b/netbox/ipam/views.py index 05f16aa35..4152cae91 100644 --- a/netbox/ipam/views.py +++ b/netbox/ipam/views.py @@ -16,6 +16,7 @@ from utilities.views import ( BulkCreateView, BulkDeleteView, BulkEditView, BulkImportView, ObjectDeleteView, ObjectEditView, ObjectListView, ) from . import filters, forms, tables +from .constants import IPADDRESS_ROLE_ANYCAST from .models import ( Aggregate, IPAddress, PREFIX_STATUS_ACTIVE, PREFIX_STATUS_DEPRECATED, PREFIX_STATUS_RESERVED, Prefix, RIR, Role, Service, VLAN, VLANGroup, VRF, @@ -285,11 +286,12 @@ class AggregateListView(ObjectListView): ipv4_total = 0 ipv6_total = 0 - for a in self.queryset: - if a.prefix.version == 4: - ipv4_total += a.prefix.size - elif a.prefix.version == 6: - ipv6_total += a.prefix.size / 2 ** 64 + for aggregate in self.queryset: + if aggregate.prefix.version == 6: + # Report equivalent /64s for IPv6 to keep things sane + ipv6_total += int(aggregate.prefix.size / 2 ** 64) + else: + ipv4_total += aggregate.prefix.size return { 'ipv4_total': ipv4_total, @@ -313,7 +315,7 @@ class AggregateView(View): ) child_prefixes = add_available_prefixes(aggregate.prefix, child_prefixes) - prefix_table = tables.PrefixTable(child_prefixes) + prefix_table = tables.PrefixDetailTable(child_prefixes) if request.user.has_perm('ipam.change_prefix') or request.user.has_perm('ipam.delete_prefix'): prefix_table.base_columns['pk'].visible = True @@ -472,11 +474,11 @@ class PrefixView(View): child_prefixes = Prefix.objects.filter( vrf=prefix.vrf, prefix__net_contained=str(prefix.prefix) ).select_related( - 'site', 'role' + 'site', 'vlan', 'role', ).annotate_depth(limit=0) if child_prefixes: child_prefixes = add_available_prefixes(prefix.prefix, child_prefixes) - child_prefix_table = tables.PrefixTable(child_prefixes) + child_prefix_table = tables.PrefixDetailTable(child_prefixes) if request.user.has_perm('ipam.change_prefix') or request.user.has_perm('ipam.delete_prefix'): child_prefix_table.base_columns['pk'].visible = True @@ -624,6 +626,9 @@ class IPAddressView(View): ).select_related( 'interface__device', 'nat_inside' ) + # Exclude anycast IPs if this IP is anycast + if ipaddress.role == IPADDRESS_ROLE_ANYCAST: + duplicate_ips = duplicate_ips.exclude(role=IPADDRESS_ROLE_ANYCAST) duplicate_ips_table = tables.IPAddressTable(list(duplicate_ips), orderable=False) # Related IP table diff --git a/netbox/netbox/configuration.docker.py b/netbox/netbox/configuration.docker.py index c57aca6f4..56f9da366 100644 --- a/netbox/netbox/configuration.docker.py +++ b/netbox/netbox/configuration.docker.py @@ -60,8 +60,8 @@ BASE_PATH = os.environ.get('BASE_PATH', '') MAINTENANCE_MODE = os.environ.get('MAINTENANCE_MODE', False) # Credentials that NetBox will use to access live devices. -NETBOX_USERNAME = os.environ.get('NETBOX_USERNAME', '') -NETBOX_PASSWORD = os.environ.get('NETBOX_PASSWORD', '') +NAPALM_USERNAME = os.environ.get('NAPALM_USERNAME', '') +NAPALM_PASSWORD = os.environ.get('NAPALM_PASSWORD', '') # Determine how many objects to display per page within a list. (Default: 50) PAGINATE_COUNT = os.environ.get('PAGINATE_COUNT', 50) diff --git a/netbox/netbox/configuration.example.py b/netbox/netbox/configuration.example.py index 2e08090c7..ce7a62464 100644 --- a/netbox/netbox/configuration.example.py +++ b/netbox/netbox/configuration.example.py @@ -38,11 +38,14 @@ ADMINS = [ # ['John Doe', 'jdoe@example.com'], ] -# Optionally display a persistent banner at the top and/or bottom of every page. To display the same content in both -# banners, define BANNER_TOP and set BANNER_BOTTOM = BANNER_TOP. +# Optionally display a persistent banner at the top and/or bottom of every page. HTML is allowed. To display the same +# content in both banners, define BANNER_TOP and set BANNER_BOTTOM = BANNER_TOP. BANNER_TOP = '' BANNER_BOTTOM = '' +# Text to include on the login page above the login form. HTML is allowed. +BANNER_LOGIN = '' + # Base URL path if accessing NetBox within a directory. For example, if installed at http://example.com/netbox/, set: # BASE_PATH = 'netbox/' BASE_PATH = '' @@ -93,9 +96,20 @@ MAINTENANCE_MODE = False # all objects by specifying "?limit=0". MAX_PAGE_SIZE = 1000 -# Credentials that NetBox will use to access live devices (future use). -NETBOX_USERNAME = '' -NETBOX_PASSWORD = '' +# The file path where uploaded media such as image attachments are stored. A trailing slash is not needed. Note that +# the default value of this setting is derived from the installed location. +# MEDIA_ROOT = '/opt/netbox/netbox/media' + +# Credentials that NetBox will uses to authenticate to devices when connecting via NAPALM. +NAPALM_USERNAME = '' +NAPALM_PASSWORD = '' + +# NAPALM timeout (in seconds). (Default: 30) +NAPALM_TIMEOUT = 30 + +# NAPALM optional arguments (see http://napalm.readthedocs.io/en/latest/support/#optional-arguments). Arguments must +# be provided as a dictionary. +NAPALM_ARGS = {} # Determine how many objects to display per page within a list. (Default: 50) PAGINATE_COUNT = 50 diff --git a/netbox/netbox/forms.py b/netbox/netbox/forms.py index 85343ec77..72a3ab8de 100644 --- a/netbox/netbox/forms.py +++ b/netbox/netbox/forms.py @@ -35,7 +35,7 @@ OBJ_TYPE_CHOICES = ( class SearchForm(BootstrapMixin, forms.Form): q = forms.CharField( - label='Query', widget=forms.TextInput(attrs={'style': 'width: 350px'}) + label='Search', widget=forms.TextInput(attrs={'style': 'width: 350px'}) ) obj_type = forms.ChoiceField( choices=OBJ_TYPE_CHOICES, required=False, label='Type' diff --git a/netbox/netbox/settings.py b/netbox/netbox/settings.py index b3f0b5187..735372b61 100644 --- a/netbox/netbox/settings.py +++ b/netbox/netbox/settings.py @@ -13,7 +13,9 @@ except ImportError: ) -VERSION = '2.1.0-dev' +VERSION = '2.1.6-dev' + +BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # Import required configuration parameters ALLOWED_HOSTS = DATABASE = SECRET_KEY = None @@ -27,8 +29,9 @@ for setting in ['ALLOWED_HOSTS', 'DATABASE', 'SECRET_KEY']: # Import optional configuration parameters ADMINS = getattr(configuration, 'ADMINS', []) -BANNER_BOTTOM = getattr(configuration, 'BANNER_BOTTOM', False) -BANNER_TOP = getattr(configuration, 'BANNER_TOP', False) +BANNER_BOTTOM = getattr(configuration, 'BANNER_BOTTOM', '') +BANNER_LOGIN = getattr(configuration, 'BANNER_LOGIN', '') +BANNER_TOP = getattr(configuration, 'BANNER_TOP', '') BASE_PATH = getattr(configuration, 'BASE_PATH', '') if BASE_PATH: BASE_PATH = BASE_PATH.strip('/') + '/' # Enforce trailing slash only @@ -44,10 +47,15 @@ LOGGING = getattr(configuration, 'LOGGING', {}) LOGIN_REQUIRED = getattr(configuration, 'LOGIN_REQUIRED', False) MAINTENANCE_MODE = getattr(configuration, 'MAINTENANCE_MODE', False) MAX_PAGE_SIZE = getattr(configuration, 'MAX_PAGE_SIZE', 1000) +MEDIA_ROOT = getattr(configuration, 'MEDIA_ROOT', os.path.join(BASE_DIR, 'media')).rstrip('/') +NAPALM_USERNAME = getattr(configuration, 'NAPALM_USERNAME', '') +NAPALM_PASSWORD = getattr(configuration, 'NAPALM_PASSWORD', '') +NAPALM_TIMEOUT = getattr(configuration, 'NAPALM_TIMEOUT', 30) +NAPALM_ARGS = getattr(configuration, 'NAPALM_ARGS', {}) +NETBOX_USERNAME = getattr(configuration, 'NETBOX_USERNAME', '') # Deprecated +NETBOX_PASSWORD = getattr(configuration, 'NETBOX_PASSWORD', '') # Deprecated PAGINATE_COUNT = getattr(configuration, 'PAGINATE_COUNT', 50) PREFER_IPV4 = getattr(configuration, 'PREFER_IPV4', False) -NETBOX_USERNAME = getattr(configuration, 'NETBOX_USERNAME', '') -NETBOX_PASSWORD = getattr(configuration, 'NETBOX_PASSWORD', '') SHORT_DATE_FORMAT = getattr(configuration, 'SHORT_DATE_FORMAT', 'Y-m-d') SHORT_DATETIME_FORMAT = getattr(configuration, 'SHORT_DATETIME_FORMAT', 'Y-m-d H:i') SHORT_TIME_FORMAT = getattr(configuration, 'SHORT_TIME_FORMAT', 'H:i:s') @@ -56,6 +64,19 @@ TIME_ZONE = getattr(configuration, 'TIME_ZONE', 'UTC') CSRF_TRUSTED_ORIGINS = ALLOWED_HOSTS +# Check for deprecated configuration parameters +config_logger = logging.getLogger('configuration') +config_logger.addHandler(logging.StreamHandler()) +config_logger.setLevel(logging.WARNING) +if NETBOX_USERNAME: + config_logger.warning('NETBOX_USERNAME is deprecated and will be removed in v2.2. Please use NAPALM_USERNAME instead.') + if not NAPALM_USERNAME: + NAPALM_USERNAME = NETBOX_USERNAME +if NETBOX_PASSWORD: + config_logger.warning('NETBOX_PASSWORD is deprecated and will be removed in v2.2. Please use NAPALM_PASSWORD instead.') + if not NAPALM_PASSWORD: + NAPALM_PASSWORD = NETBOX_PASSWORD + # Attempt to import LDAP configuration if it has been defined LDAP_IGNORE_CERT_ERRORS = False try: @@ -78,17 +99,15 @@ if LDAP_CONFIGURED: if LDAP_IGNORE_CERT_ERRORS: ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER) # Enable logging for django_auth_ldap - logger = logging.getLogger('django_auth_ldap') - logger.addHandler(logging.StreamHandler()) - logger.setLevel(logging.DEBUG) + ldap_logger = logging.getLogger('django_auth_ldap') + ldap_logger.addHandler(logging.StreamHandler()) + ldap_logger.setLevel(logging.DEBUG) except ImportError: raise ImproperlyConfigured( "LDAP authentication has been configured, but django-auth-ldap is not installed. You can remove " "netbox/ldap_config.py to disable LDAP." ) -BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) - # Database configuration.DATABASE.update({'ENGINE': 'django.db.backends.postgresql'}) DATABASES = { @@ -184,7 +203,6 @@ STATICFILES_DIRS = ( ) # Media -MEDIA_ROOT = os.path.join(BASE_DIR, 'media') MEDIA_URL = '/{}media/'.format(BASE_PATH) # Disable default limit of 1000 fields per request. Needed for bulk deletion of objects. (Added in Django 1.10.) @@ -204,6 +222,7 @@ SECRETS_MIN_PUBKEY_SIZE = 2048 # Django REST framework (API) REST_FRAMEWORK_VERSION = VERSION[0:3] # Use major.minor as API version REST_FRAMEWORK = { + 'ALLOWED_VERSIONS': [REST_FRAMEWORK_VERSION], 'DEFAULT_AUTHENTICATION_CLASSES': ( 'rest_framework.authentication.SessionAuthentication', 'utilities.api.TokenAuthentication', @@ -215,10 +234,14 @@ REST_FRAMEWORK = { 'DEFAULT_PERMISSION_CLASSES': ( 'utilities.api.TokenPermissions', ), + 'DEFAULT_RENDERER_CLASSES': ( + 'rest_framework.renderers.JSONRenderer', + 'utilities.api.FormlessBrowsableAPIRenderer', + ), 'DEFAULT_VERSION': REST_FRAMEWORK_VERSION, - 'ALLOWED_VERSIONS': [REST_FRAMEWORK_VERSION], 'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.AcceptHeaderVersioning', 'PAGE_SIZE': PAGINATE_COUNT, + 'VIEW_NAME_FUNCTION': 'utilities.api.get_view_name', } # Django debug toolbar diff --git a/netbox/project-static/css/base.css b/netbox/project-static/css/base.css index 573c1c967..b013aab97 100644 --- a/netbox/project-static/css/base.css +++ b/netbox/project-static/css/base.css @@ -81,6 +81,11 @@ footer p { } } +/* Navigation menu */ +li.subnav > a { + padding-left: 30px; +} + /* Forms */ label { font-weight: normal; @@ -324,13 +329,14 @@ li.occupied + li.available { } /* Devices */ -table.component-list tr.ipaddress td { - background-color: #eeffff; - padding-bottom: 4px; - padding-top: 4px; +table.component-list td.subtable { + padding: 0; + padding-left: 16px; } -table.component-list tr.ipaddress:hover td { - background-color: #e6f7f7; +table.component-list td.subtable td { + border: none; + padding-bottom: 6px; + padding-top: 6px; } /* AJAX loader */ diff --git a/netbox/project-static/js/secrets.js b/netbox/project-static/js/secrets.js index 638638623..d1ec3d883 100644 --- a/netbox/project-static/js/secrets.js +++ b/netbox/project-static/js/secrets.js @@ -43,7 +43,7 @@ $(document).ready(function() { success: function (response, status) { if (response.plaintext) { console.log("Secret retrieved successfully"); - $('#secret_' + secret_id).html(response.plaintext); + $('#secret_' + secret_id).text(response.plaintext); $('button.unlock-secret[secret-id=' + secret_id + ']').hide(); $('button.lock-secret[secret-id=' + secret_id + ']').show(); } else { diff --git a/netbox/secrets/api/serializers.py b/netbox/secrets/api/serializers.py index ff2eb1dfa..b7c4bac9a 100644 --- a/netbox/secrets/api/serializers.py +++ b/netbox/secrets/api/serializers.py @@ -5,14 +5,14 @@ from rest_framework.validators import UniqueTogetherValidator from dcim.api.serializers import NestedDeviceSerializer from secrets.models import Secret, SecretRole -from utilities.api import ModelValidationMixin +from utilities.api import ValidatedModelSerializer # # SecretRoles # -class SecretRoleSerializer(ModelValidationMixin, serializers.ModelSerializer): +class SecretRoleSerializer(ValidatedModelSerializer): class Meta: model = SecretRole diff --git a/netbox/templates/_base.html b/netbox/templates/_base.html index d718f60c1..1c085876b 100644 --- a/netbox/templates/_base.html +++ b/netbox/templates/_base.html @@ -31,201 +31,199 @@
    • @@ -233,14 +231,14 @@
    • @@ -254,16 +252,16 @@ {{ request.user|truncatechars:"30" }} {% else %} -
  • Log in
    • +
  • Log in
    • {% endif %}