Merged release v2.4.5

CHANGELOG.md

@@ -19,6 +19,27 @@ v2.5.0 (FUTURE)
 
 ---
 
+v2.4.5 (2018-10-02)
+
+## Enhancements
+
+* [#2392](https://github.com/digitalocean/netbox/issues/2392) - Implemented local context data for devices and virtual machines
+* [#2402](https://github.com/digitalocean/netbox/issues/2402) - Order and format JSON data in form fields
+* [#2432](https://github.com/digitalocean/netbox/issues/2432) - Link remote interface connections to the Interface view
+* [#2438](https://github.com/digitalocean/netbox/issues/2438) - API optimizations for tagged objects
+
+## Bug Fixes
+
+* [#2406](https://github.com/digitalocean/netbox/issues/2406) - Remove hard-coded limit of 1000 objects from API-populated form fields
+* [#2414](https://github.com/digitalocean/netbox/issues/2414) - Tags field missing from device/VM component creation forms
+* [#2442](https://github.com/digitalocean/netbox/issues/2442) - Nullify "next" link in API when limit=0 is passed
+* [#2443](https://github.com/digitalocean/netbox/issues/2443) - Enforce JSON object format when creating config contexts
+* [#2444](https://github.com/digitalocean/netbox/issues/2444) - Improve validation of interface MAC addresses
+* [#2455](https://github.com/digitalocean/netbox/issues/2455) - Ignore unique address enforcement for IPs with a shared/virtual role
+* [#2470](https://github.com/digitalocean/netbox/issues/2470) - Log the creation of device/VM components as object changes
+
+---
+
 v2.4.4 (2018-08-22)
 
 ## Enhancements

README.md

@@ -40,3 +40,18 @@ and run `upgrade.sh`.
 * [Docker container](https://github.com/ninech/netbox-docker) (via [@cimnine](https://github.com/cimnine))
 * [Vagrant deployment](https://github.com/ryanmerolle/netbox-vagrant) (via [@ryanmerolle](https://github.com/ryanmerolle))
 * [Ansible deployment](https://github.com/lae/ansible-role-netbox) (via [@lae](https://github.com/lae))
+
+# Related projects
+
+## Supported SDK
+
+- [pynetbox](https://github.com/digitalocean/pynetbox) Python API client library for Netbox.
+
+## Community SDK
+
+- [netbox-client-ruby](https://github.com/ninech/netbox-client-ruby) A ruby client library for Netbox v2.
+
+## Ansible Inventory
+
+- [netbox-as-ansible-inventory](https://github.com/AAbouZaid/netbox-as-ansible-inventory) Ansible dynamic inventory script for Netbox.
+

docs/additional-features/context-data.md

@@ -1,3 +1,5 @@
 # Contextual Configuration Data
 
 Sometimes it is desirable to associate arbitrary data with a group of devices to aid in their configuration. For example, you might want to associate a set of syslog servers for all devices at a particular site. Context data enables the association of arbitrary data to devices and virtual machines grouped by region, site, role, platform, and/or tenant. Context data is arranged hierarchically, so that data with a higher weight can be entered to override more general lower-weight data. Multiple instances of data are automatically merged by NetBox to present a single dictionary for each object.
+
+Devices and Virtual Machines may also have a local config context defined. This local context will always overwrite the rendered config context objects for the Device/VM. This is useful in situations were the device requires a one-off value different from the rest of the environment.

docs/core-functionality/devices.md

@@ -99,7 +99,7 @@ Device bays represent the ability of a device to house child devices. For exampl
 
 # Platforms
 
-A platform defines the type of software running on a device or virtual machine. This can be helpful when it is necessary to distinguish between, for instance, different feature sets. Note that two devices of same type may be assigned different platforms: for example, one Juniper MX240 running Junos 14 and another running Junos 15.
+A platform defines the type of software running on a device or virtual machine. This can be helpful when it is necessary to distinguish between, for instance, different feature sets. Note that two devices of the same type may be assigned different platforms: for example, one Juniper MX240 running Junos 14 and another running Junos 15.
 
 The platform model is also used to indicate which [NAPALM](https://napalm-automation.net/) driver NetBox should use when connecting to a remote device. The name of the driver along with optional parameters are stored with the platform. See the [API documentation](api/napalm-integration.md) for more information on NAPALM integration.
 

mkdocs.yml

@@ -1,4 +1,5 @@
 site_name: NetBox
+theme: readthedocs
 repo_url: https://github.com/digitalocean/netbox
 
 pages:

netbox/circuits/api/views.py

@@ -27,7 +27,7 @@ class CircuitsFieldChoicesViewSet(FieldChoicesViewSet):
 #
 
 class ProviderViewSet(CustomFieldModelViewSet):
-    queryset = Provider.objects.all()
+    queryset = Provider.objects.prefetch_related('tags')
     serializer_class = serializers.ProviderSerializer
     filter_class = filters.ProviderFilter
 
@@ -57,7 +57,7 @@ class CircuitTypeViewSet(ModelViewSet):
 #
 
 class CircuitViewSet(CustomFieldModelViewSet):
-    queryset = Circuit.objects.select_related('type', 'tenant', 'provider')
+    queryset = Circuit.objects.select_related('type', 'tenant', 'provider').prefetch_related('tags')
     serializer_class = serializers.CircuitSerializer
     filter_class = filters.CircuitFilter
 

netbox/dcim/api/serializers.py

@@ -410,7 +410,7 @@ class DeviceSerializer(TaggitSerializer, CustomFieldModelSerializer):
             'id', 'name', 'display_name', 'device_type', 'device_role', 'tenant', 'platform', 'serial', 'asset_tag',
             'site', 'rack', 'position', 'face', 'parent_device', 'status', 'primary_ip', 'primary_ip4', 'primary_ip6',
             'cluster', 'virtual_chassis', 'vc_position', 'vc_priority', 'comments', 'tags', 'custom_fields', 'created',
-            'last_updated',
+            'last_updated', 'local_context_data',
         ]
         validators = []
 
@@ -446,7 +446,7 @@ class DeviceWithConfigContextSerializer(DeviceSerializer):
             'id', 'name', 'display_name', 'device_type', 'device_role', 'tenant', 'platform', 'serial', 'asset_tag',
             'site', 'rack', 'position', 'face', 'parent_device', 'status', 'primary_ip', 'primary_ip4', 'primary_ip6',
             'cluster', 'virtual_chassis', 'vc_position', 'vc_priority', 'comments', 'tags', 'custom_fields',
-            'config_context', 'created', 'last_updated',
+            'config_context', 'created', 'last_updated', 'local_context_data',
         ]
 
     def get_config_context(self, obj):

netbox/dcim/api/views.py

@@ -1,7 +1,7 @@
 from collections import OrderedDict
 
 from django.conf import settings
-from django.http import HttpResponseBadRequest, HttpResponseForbidden
+from django.http import HttpResponseForbidden
 from django.shortcuts import get_object_or_404
 from drf_yasg import openapi
 from drf_yasg.openapi import Parameter
@@ -58,7 +58,7 @@ class RegionViewSet(ModelViewSet):
 #
 
 class SiteViewSet(CustomFieldModelViewSet):
-    queryset = Site.objects.select_related('region', 'tenant')
+    queryset = Site.objects.select_related('region', 'tenant').prefetch_related('tags')
     serializer_class = serializers.SiteSerializer
     filter_class = filters.SiteFilter
 
@@ -98,7 +98,7 @@ class RackRoleViewSet(ModelViewSet):
 #
 
 class RackViewSet(CustomFieldModelViewSet):
-    queryset = Rack.objects.select_related('site', 'group__site', 'tenant')
+    queryset = Rack.objects.select_related('site', 'group__site', 'tenant').prefetch_related('tags')
     serializer_class = serializers.RackSerializer
     filter_class = filters.RackFilter
 
@@ -152,7 +152,7 @@ class ManufacturerViewSet(ModelViewSet):
 #
 
 class DeviceTypeViewSet(CustomFieldModelViewSet):
-    queryset = DeviceType.objects.select_related('manufacturer')
+    queryset = DeviceType.objects.select_related('manufacturer').prefetch_related('tags')
     serializer_class = serializers.DeviceTypeSerializer
     filter_class = filters.DeviceTypeFilter
 
@@ -226,7 +226,7 @@ class DeviceViewSet(CustomFieldModelViewSet):
         'device_type__manufacturer', 'device_role', 'tenant', 'platform', 'site', 'rack', 'parent_bay',
         'virtual_chassis__master',
     ).prefetch_related(
-        'primary_ip4__nat_outside', 'primary_ip6__nat_outside',
+        'primary_ip4__nat_outside', 'primary_ip6__nat_outside', 'tags',
     )
     filter_class = filters.DeviceFilter
 
@@ -313,31 +313,31 @@ class DeviceViewSet(CustomFieldModelViewSet):
 #
 
 class ConsolePortViewSet(ModelViewSet):
-    queryset = ConsolePort.objects.select_related('device', 'cs_port__device')
+    queryset = ConsolePort.objects.select_related('device', 'cs_port__device').prefetch_related('tags')
     serializer_class = serializers.ConsolePortSerializer
     filter_class = filters.ConsolePortFilter
 
 
 class ConsoleServerPortViewSet(ModelViewSet):
-    queryset = ConsoleServerPort.objects.select_related('device', 'connected_console__device')
+    queryset = ConsoleServerPort.objects.select_related('device', 'connected_console__device').prefetch_related('tags')
     serializer_class = serializers.ConsoleServerPortSerializer
     filter_class = filters.ConsoleServerPortFilter
 
 
 class PowerPortViewSet(ModelViewSet):
-    queryset = PowerPort.objects.select_related('device', 'power_outlet__device')
+    queryset = PowerPort.objects.select_related('device', 'power_outlet__device').prefetch_related('tags')
     serializer_class = serializers.PowerPortSerializer
     filter_class = filters.PowerPortFilter
 
 
 class PowerOutletViewSet(ModelViewSet):
-    queryset = PowerOutlet.objects.select_related('device', 'connected_port__device')
+    queryset = PowerOutlet.objects.select_related('device', 'connected_port__device').prefetch_related('tags')
     serializer_class = serializers.PowerOutletSerializer
     filter_class = filters.PowerOutletFilter
 
 
 class InterfaceViewSet(ModelViewSet):
-    queryset = Interface.objects.select_related('device')
+    queryset = Interface.objects.select_related('device').prefetch_related('tags')
     serializer_class = serializers.InterfaceSerializer
     filter_class = filters.InterfaceFilter
 
@@ -353,13 +353,13 @@ class InterfaceViewSet(ModelViewSet):
 
 
 class DeviceBayViewSet(ModelViewSet):
-    queryset = DeviceBay.objects.select_related('installed_device')
+    queryset = DeviceBay.objects.select_related('installed_device').prefetch_related('tags')
     serializer_class = serializers.DeviceBaySerializer
     filter_class = filters.DeviceBayFilter
 
 
 class InventoryItemViewSet(ModelViewSet):
-    queryset = InventoryItem.objects.select_related('device', 'manufacturer')
+    queryset = InventoryItem.objects.select_related('device', 'manufacturer').prefetch_related('tags')
     serializer_class = serializers.InventoryItemSerializer
     filter_class = filters.InventoryItemFilter
 
@@ -391,7 +391,7 @@ class InterfaceConnectionViewSet(ModelViewSet):
 #
 
 class VirtualChassisViewSet(ModelViewSet):
-    queryset = VirtualChassis.objects.all()
+    queryset = VirtualChassis.objects.prefetch_related('tags')
     serializer_class = serializers.VirtualChassisSerializer
 
 

netbox/dcim/fields.py

@@ -1,10 +1,7 @@
-from netaddr import EUI, mac_unix_expanded
-
 from django.core.exceptions import ValidationError
 from django.core.validators import MinValueValidator, MaxValueValidator
 from django.db import models
-
+from netaddr import AddrFormatError, EUI, mac_unix_expanded
-from .formfields import MACAddressFormField
 
 
 class ASNField(models.BigIntegerField):
@@ -33,7 +30,7 @@ class MACAddressField(models.Field):
             return value
         try:
             return EUI(value, version=48, dialect=mac_unix_expanded_uppercase)
-        except ValueError as e:
+        except AddrFormatError as e:
             raise ValidationError(e)
 
     def db_type(self, connection):
@@ -43,11 +40,3 @@ class MACAddressField(models.Field):
         if not value:
             return None
         return str(self.to_python(value))
-
-    def form_class(self):
-        return MACAddressFormField
-
-    def formfield(self, **kwargs):
-        defaults = {'form_class': self.form_class()}
-        defaults.update(kwargs)
-        return super(MACAddressField, self).formfield(**defaults)

netbox/dcim/formfields.py

@@ -1,25 +0,0 @@
-from django import forms
-from django.core.exceptions import ValidationError
-from netaddr import EUI, AddrFormatError
-
-
-#
-# Form fields
-#
-
-class MACAddressFormField(forms.Field):
-    default_error_messages = {
-        'invalid': "Enter a valid MAC address.",
-    }
-
-    def to_python(self, value):
-        if not value:
-            return None
-
-        if isinstance(value, EUI):
-            return value
-
-        try:
-            return EUI(value, version=48)
-        except AddrFormatError:
-            raise ValidationError("Please specify a valid MAC address.")

netbox/dcim/forms.py

@@ -16,7 +16,7 @@ from utilities.forms import (
     AnnotatedMultipleChoiceField, APISelect, add_blank_choice, ArrayFieldSelectMultiple, BootstrapMixin, BulkEditForm,
     BulkEditNullBooleanSelect, ChainedFieldsMixin, ChainedModelChoiceField, CommentField, ComponentForm,
     ConfirmationForm, CSVChoiceField, ExpandableNameField, FilterChoiceField, FilterTreeNodeMultipleChoiceField,
-    FlexibleModelChoiceField, Livesearch, SelectWithDisabled, SelectWithPK, SmallTextarea, SlugField,
+    FlexibleModelChoiceField, JSONField, Livesearch, SelectWithDisabled, SelectWithPK, SmallTextarea, SlugField,
 )
 from virtualization.models import Cluster
 from .constants import (
@@ -25,7 +25,6 @@ from .constants import (
     RACK_TYPE_CHOICES, RACK_WIDTH_CHOICES, RACK_WIDTH_19IN, RACK_WIDTH_23IN, SITE_STATUS_CHOICES, SUBDEVICE_ROLE_CHILD,
     SUBDEVICE_ROLE_PARENT, SUBDEVICE_ROLE_CHOICES,
 )
-from .formfields import MACAddressFormField
 from .models import (
     DeviceBay, DeviceBayTemplate, ConsolePort, ConsolePortTemplate, ConsoleServerPort, ConsoleServerPortTemplate,
     Device, DeviceRole, DeviceType, Interface, InterfaceConnection, InterfaceTemplate, Manufacturer, InventoryItem,
@@ -821,16 +820,19 @@ class DeviceForm(BootstrapMixin, TenancyForm, CustomFieldForm):
     )
     comments = CommentField()
     tags = TagField(required=False)
+    local_context_data = JSONField(required=False)
 
     class Meta:
         model = Device
         fields = [
             'name', 'device_role', 'device_type', 'serial', 'asset_tag', 'site', 'rack', 'position', 'face',
             'status', 'platform', 'primary_ip4', 'primary_ip6', 'tenant_group', 'tenant', 'comments', 'tags',
+            'local_context_data'
         ]
         help_texts = {
             'device_role': "The function this device serves",
             'serial': "Chassis serial number",
+            'local_context_data': "Local config context data overwrites all sources contexts in the final rendered config context"
         }
         widgets = {
             'face': forms.Select(attrs={'filter-for': 'position'}),
@@ -1190,6 +1192,7 @@ class ConsolePortForm(BootstrapMixin, forms.ModelForm):
 
 class ConsolePortCreateForm(ComponentForm):
     name_pattern = ExpandableNameField(label='Name')
+    tags = TagField(required=False)
 
 
 class ConsoleConnectionCSVForm(forms.ModelForm):
@@ -1360,6 +1363,7 @@ class ConsoleServerPortForm(BootstrapMixin, forms.ModelForm):
 
 class ConsoleServerPortCreateForm(ComponentForm):
     name_pattern = ExpandableNameField(label='Name')
+    tags = TagField(required=False)
 
 
 class ConsoleServerPortConnectionForm(BootstrapMixin, ChainedFieldsMixin, forms.Form):
@@ -1457,6 +1461,7 @@ class PowerPortForm(BootstrapMixin, forms.ModelForm):
 
 class PowerPortCreateForm(ComponentForm):
     name_pattern = ExpandableNameField(label='Name')
+    tags = TagField(required=False)
 
 
 class PowerConnectionCSVForm(forms.ModelForm):
@@ -1627,6 +1632,7 @@ class PowerOutletForm(BootstrapMixin, forms.ModelForm):
 
 class PowerOutletCreateForm(ComponentForm):
     name_pattern = ExpandableNameField(label='Name')
+    tags = TagField(required=False)
 
 
 class PowerOutletConnectionForm(BootstrapMixin, ChainedFieldsMixin, forms.Form):
@@ -1852,7 +1858,7 @@ class InterfaceCreateForm(ComponentForm, forms.Form):
     enabled = forms.BooleanField(required=False)
     lag = forms.ModelChoiceField(queryset=Interface.objects.all(), required=False, label='Parent LAG')
     mtu = forms.IntegerField(required=False, min_value=1, max_value=32767, label='MTU')
-    mac_address = MACAddressFormField(required=False, label='MAC Address')
+    mac_address = forms.CharField(required=False, label='MAC Address')
     mgmt_only = forms.BooleanField(
         required=False,
         label='OOB Management',
@@ -1860,6 +1866,7 @@ class InterfaceCreateForm(ComponentForm, forms.Form):
     )
     description = forms.CharField(max_length=100, required=False)
     mode = forms.ChoiceField(choices=add_blank_choice(IFACE_MODE_CHOICES), required=False)
+    tags = TagField(required=False)
 
     def __init__(self, *args, **kwargs):
 
@@ -2097,6 +2104,7 @@ class DeviceBayForm(BootstrapMixin, forms.ModelForm):
 
 class DeviceBayCreateForm(ComponentForm):
     name_pattern = ExpandableNameField(label='Name')
+    tags = TagField(required=False)
 
 
 class PopulateDeviceBayForm(BootstrapMixin, forms.Form):

netbox/dcim/migrations/0063_device_local_context_data.py

@@ -0,0 +1,19 @@
+# Generated by Django 2.0.8 on 2018-09-16 02:01
+
+import django.contrib.postgres.fields.jsonb
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('dcim', '0062_interface_mtu'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='device',
+            name='local_context_data',
+            field=django.contrib.postgres.fields.jsonb.JSONField(blank=True, null=True),
+        ),
+    ]

netbox/dcim/migrations/0064_remove_platform_rpc_client.py

@@ -6,7 +6,7 @@ from django.db import migrations
 class Migration(migrations.Migration):
 
     dependencies = [
-        ('dcim', '0062_interface_mtu'),
+        ('dcim', '0063_device_local_context_data'),
     ]
 
     operations = [

netbox/dcim/models.py

@@ -8,7 +8,7 @@ from django.contrib.postgres.fields import ArrayField, JSONField
 from django.core.exceptions import ObjectDoesNotExist, ValidationError
 from django.core.validators import MaxValueValidator, MinValueValidator
 from django.db import models
-from django.db.models import Count, Q, ObjectDoesNotExist
+from django.db.models import Count, Q
 from django.urls import reverse
 from mptt.models import MPTTModel, TreeForeignKey
 from taggit.managers import TaggableManager

netbox/dcim/tables.py

@@ -612,10 +612,12 @@ class PowerConnectionTable(BaseTable):
 class InterfaceConnectionTable(BaseTable):
     device_a = tables.LinkColumn('dcim:device', accessor=Accessor('interface_a.device'),
                                  args=[Accessor('interface_a.device.pk')], verbose_name='Device A')
-    interface_a = tables.Column(verbose_name='Interface A')
+    interface_a = tables.LinkColumn('dcim:interface', accessor=Accessor('interface_a'),
+                                    args=[Accessor('interface_a.pk')], verbose_name='Interface A')
     device_b = tables.LinkColumn('dcim:device', accessor=Accessor('interface_b.device'),
                                  args=[Accessor('interface_b.device.pk')], verbose_name='Device B')
-    interface_b = tables.Column(verbose_name='Interface B')
+    interface_b = tables.LinkColumn('dcim:interface', accessor=Accessor('interface_b'),
+                                    args=[Accessor('interface_b.pk')], verbose_name='Interface B')
 
     class Meta(BaseTable.Meta):
         model = InterfaceConnection

netbox/extras/models.py

@@ -688,9 +688,22 @@ class ConfigContext(models.Model):
     def get_absolute_url(self):
         return reverse('extras:configcontext', kwargs={'pk': self.pk})
 
+    def clean(self):
+
+        # Verify that JSON data is provided as an object
+        if type(self.data) is not dict:
+            raise ValidationError(
+                {'data': 'JSON data must be in object form. Example: {"foo": 123}'}
+            )
+
 
 class ConfigContextModel(models.Model):
 
+    local_context_data = JSONField(
+        blank=True,
+        null=True,
+    )
+
     class Meta:
         abstract = True
 
@@ -704,6 +717,10 @@ class ConfigContextModel(models.Model):
         for context in ConfigContext.objects.get_for_object(self):
             data.update(context.data)
 
+        # If the object has local config context data defined, that data overwrites all rendered data
+        if self.local_context_data is not None:
+            data.update(self.local_context_data)
+
         return data
 
 

netbox/extras/views.py

@@ -104,9 +104,11 @@ class ObjectConfigContextView(View):
 
         obj = get_object_or_404(self.object_class, pk=pk)
         source_contexts = ConfigContext.objects.get_for_object(obj)
+        model_name = self.object_class._meta.model_name
 
         return render(request, 'extras/object_configcontext.html', {
-            self.object_class._meta.model_name: obj,
+            model_name: obj,
+            'obj': obj,
             'rendered_context': obj.get_config_context(),
             'source_contexts': source_contexts,
             'base_template': self.base_template,

netbox/ipam/api/views.py

@@ -31,7 +31,7 @@ class IPAMFieldChoicesViewSet(FieldChoicesViewSet):
 #
 
 class VRFViewSet(CustomFieldModelViewSet):
-    queryset = VRF.objects.select_related('tenant')
+    queryset = VRF.objects.select_related('tenant').prefetch_related('tags')
     serializer_class = serializers.VRFSerializer
     filter_class = filters.VRFFilter
 
@@ -51,7 +51,7 @@ class RIRViewSet(ModelViewSet):
 #
 
 class AggregateViewSet(CustomFieldModelViewSet):
-    queryset = Aggregate.objects.select_related('rir')
+    queryset = Aggregate.objects.select_related('rir').prefetch_related('tags')
     serializer_class = serializers.AggregateSerializer
     filter_class = filters.AggregateFilter
 
@@ -71,7 +71,7 @@ class RoleViewSet(ModelViewSet):
 #
 
 class PrefixViewSet(CustomFieldModelViewSet):
-    queryset = Prefix.objects.select_related('site', 'vrf__tenant', 'tenant', 'vlan', 'role')
+    queryset = Prefix.objects.select_related('site', 'vrf__tenant', 'tenant', 'vlan', 'role').prefetch_related('tags')
     serializer_class = serializers.PrefixSerializer
     filter_class = filters.PrefixFilter
 
@@ -243,7 +243,7 @@ class IPAddressViewSet(CustomFieldModelViewSet):
     queryset = IPAddress.objects.select_related(
         'vrf__tenant', 'tenant', 'nat_inside', 'interface__device__device_type', 'interface__virtual_machine'
     ).prefetch_related(
-        'nat_outside'
+        'nat_outside', 'tags',
     )
     serializer_class = serializers.IPAddressSerializer
     filter_class = filters.IPAddressFilter
@@ -264,7 +264,7 @@ class VLANGroupViewSet(ModelViewSet):
 #
 
 class VLANViewSet(CustomFieldModelViewSet):
-    queryset = VLAN.objects.select_related('site', 'group', 'tenant', 'role')
+    queryset = VLAN.objects.select_related('site', 'group', 'tenant', 'role').prefetch_related('tags')
     serializer_class = serializers.VLANSerializer
     filter_class = filters.VLANFilter
 
@@ -274,6 +274,6 @@ class VLANViewSet(CustomFieldModelViewSet):
 #
 
 class ServiceViewSet(ModelViewSet):
-    queryset = Service.objects.select_related('device')
+    queryset = Service.objects.select_related('device').prefetch_related('tags')
     serializer_class = serializers.ServiceSerializer
     filter_class = filters.ServiceFilter

netbox/ipam/constants.py

@@ -49,6 +49,16 @@ IPADDRESS_ROLE_CHOICES = (
     (IPADDRESS_ROLE_CARP, 'CARP'),
 )
 
+IPADDRESS_ROLES_NONUNIQUE = (
+    # IPAddress roles which are exempt from unique address enforcement
+    IPADDRESS_ROLE_ANYCAST,
+    IPADDRESS_ROLE_VIP,
+    IPADDRESS_ROLE_VRRP,
+    IPADDRESS_ROLE_HSRP,
+    IPADDRESS_ROLE_GLBP,
+    IPADDRESS_ROLE_CARP,
+)
+
 # VLAN statuses
 VLAN_STATUS_ACTIVE = 1
 VLAN_STATUS_RESERVED = 2

netbox/ipam/models.py

@@ -587,7 +587,11 @@ class IPAddress(ChangeLoggedModel, CustomFieldModel):
         if self.address:
 
             # Enforce unique IP space (if applicable)
-            if (self.vrf is None and settings.ENFORCE_GLOBAL_UNIQUE) or (self.vrf and self.vrf.enforce_unique):
+            if self.role not in IPADDRESS_ROLES_NONUNIQUE and (
+                self.vrf is None and settings.ENFORCE_GLOBAL_UNIQUE
+            ) or (
+                self.vrf and self.vrf.enforce_unique
+            ):
                 duplicate_ips = self.get_duplicates()
                 if duplicate_ips:
                     raise ValidationError({

netbox/ipam/tests/test_models.py

@@ -2,6 +2,7 @@ import netaddr
 from django.core.exceptions import ValidationError
 from django.test import TestCase, override_settings
 
+from ipam.constants import IPADDRESS_ROLE_VIP
 from ipam.models import IPAddress, Prefix, VRF
 
 
@@ -57,3 +58,8 @@ class TestIPAddress(TestCase):
         IPAddress.objects.create(vrf=vrf, address=netaddr.IPNetwork('192.0.2.1/24'))
         duplicate_ip = IPAddress(vrf=vrf, address=netaddr.IPNetwork('192.0.2.1/24'))
         self.assertRaises(ValidationError, duplicate_ip.clean)
+
+    @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
+    def test_duplicate_nonunique_role(self):
+        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPADDRESS_ROLE_VIP)
+        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPADDRESS_ROLE_VIP)

netbox/netbox/api.py

@@ -1,3 +1,4 @@
+from django.conf import settings
 from rest_framework import authentication, exceptions
 from rest_framework.pagination import LimitOffsetPagination
 from rest_framework.permissions import DjangoModelPermissions, SAFE_METHODS
@@ -56,7 +57,6 @@ class TokenPermissions(DjangoModelPermissions):
     """
     def __init__(self):
         # LOGIN_REQUIRED determines whether read-only access is provided to anonymous users.
-        from django.conf import settings
         self.authenticated_users_only = settings.LOGIN_REQUIRED
         super(TokenPermissions, self).__init__()
 
@@ -102,8 +102,6 @@ class OptionalLimitOffsetPagination(LimitOffsetPagination):
 
     def get_limit(self, request):
 
-        from django.conf import settings
-
         if self.limit_query_param:
             try:
                 limit = int(request.query_params[self.limit_query_param])
@@ -121,6 +119,22 @@ class OptionalLimitOffsetPagination(LimitOffsetPagination):
 
         return self.default_limit
 
+    def get_next_link(self):
+
+        # Pagination has been disabled
+        if not self.limit:
+            return None
+
+        return super(OptionalLimitOffsetPagination, self).get_next_link()
+
+    def get_previous_link(self):
+
+        # Pagination has been disabled
+        if not self.limit:
+            return None
+
+        return super(OptionalLimitOffsetPagination, self).get_previous_link()
+
 
 #
 # Miscellaneous

netbox/project-static/js/forms.js

@@ -82,7 +82,7 @@ $(document).ready(function() {
             }
 
             if ($(parent).val() || $(parent).attr('nullable') == 'true') {
-                var api_url = child_field.attr('api-url') + '&limit=1000';
+                var api_url = child_field.attr('api-url');
                 var disabled_indicator = child_field.attr('disabled-indicator');
                 var initial_value = child_field.attr('initial');
                 var display_field = child_field.attr('display-field') || 'name';

netbox/secrets/api/views.py

@@ -46,7 +46,7 @@ class SecretViewSet(ModelViewSet):
     queryset = Secret.objects.select_related(
         'device__primary_ip4', 'device__primary_ip6', 'role',
     ).prefetch_related(
-        'role__users', 'role__groups',
+        'role__users', 'role__groups', 'tags',
     )
     serializer_class = serializers.SecretSerializer
     filter_class = filters.SecretFilter

netbox/templates/dcim/device_edit.html

@@ -77,6 +77,12 @@
             </div>
         </div>
     {% endif %}
+    <div class="panel panel-default">
+        <div class="panel-heading"><strong>Local Config Context Data</strong></div>
+        <div class="panel-body">
+            {% render_field form.local_context_data %}
+        </div>
+    </div>
     <div class="panel panel-default">
         <div class="panel-heading"><strong>Tags</strong></div>
         <div class="panel-body">

netbox/templates/dcim/inc/interface.html

@@ -44,7 +44,7 @@
                 <a href="{% url 'dcim:device' pk=connected_iface.device.pk %}">{{ connected_iface.device }}</a>
             </td>
             <td>
-                <span title="{{ connected_iface.get_form_factor_display }}">{{ connected_iface }}</span>
+                <a href="{% url 'dcim:interface' pk=connected_iface.pk %}"><span title="{{ connected_iface.get_form_factor_display }}">{{ connected_iface }}</span></a>
             </td>
         {% endwith %}
     {% elif iface.circuit_termination %}

netbox/templates/dcim/interface.html

@@ -134,7 +134,9 @@
                         </tr>
                         <tr>
                             <td>Name</td>
-                            <td>{{ connected_interface.name }}</td>
+                            <td>
+                                <a href="{{ connected_interface.get_absolute_url }}">{{ connected_interface.name }}</a>
+                            </td>
                         </tr>
                         <tr>
                             <td>Type</td>

netbox/templates/dcim/interface_edit.html

@@ -14,11 +14,6 @@
             {% render_field form.mgmt_only %}
             {% render_field form.description %}
             {% render_field form.mode %}
-        </div>
-    </div>
-    <div class="panel panel-default">
-        <div class="panel-heading"><strong>Tags</strong></div>
-        <div class="panel-body">
             {% render_field form.tags %}
         </div>
     </div>

netbox/templates/extras/object_configcontext.html

@@ -16,6 +16,24 @@
             </div>
         </div>
         <div class="col-md-6">
+            <div class="panel panel-default">
+                <div class="panel-heading">
+                    <strong>Local Context</strong>
+                </div>
+                <div class="panel-body">
+                    {% if obj.local_context_data %}
+                        <pre>{{ obj.local_context_data|render_json }}</pre>
+                    {% else %}
+                        <span class="text-muted">None</span>
+                    {% endif %}
+                </div>
+                <div class="panel-footer">
+                    <span class="help-block">
+                        <i class="fa fa-info-circle"></i>
+                        The local config context overwrites all source contexts.
+                    </span>
+                </div>
+            </div>
             <div class="panel panel-default">
                 <div class="panel-heading">
                     <strong>Source Contexts</strong>

netbox/templates/virtualization/interface_edit.html

@@ -11,6 +11,7 @@
             {% render_field form.mtu %}
             {% render_field form.description %}
             {% render_field form.mode %}
+            {% render_field form.tags %}
         </div>
     </div>
     {% if obj.mode %}

netbox/templates/virtualization/virtualmachine_edit.html

@@ -48,6 +48,12 @@
             </div>
         </div>
     {% endif %}
+    <div class="panel panel-default">
+        <div class="panel-heading"><strong>Local Config Context Data</strong></div>
+        <div class="panel-body">
+            {% render_field form.local_context_data %}
+        </div>
+    </div>
     <div class="panel panel-default">
         <div class="panel-heading"><strong>Tags</strong></div>
         <div class="panel-body">

netbox/tenancy/api/views.py

@@ -28,6 +28,6 @@ class TenantGroupViewSet(ModelViewSet):
 #
 
 class TenantViewSet(CustomFieldModelViewSet):
-    queryset = Tenant.objects.select_related('group')
+    queryset = Tenant.objects.select_related('group').prefetch_related('tags')
     serializer_class = serializers.TenantSerializer
     filter_class = filters.TenantFilter

netbox/utilities/forms.py

@@ -1,10 +1,11 @@
 import csv
 from io import StringIO
+import json
 import re
 
 from django import forms
 from django.conf import settings
-from django.contrib.postgres.forms import JSONField as _JSONField
+from django.contrib.postgres.forms.jsonb import JSONField as _JSONField, InvalidJSONInput
 from django.db.models import Count
 from django.urls import reverse_lazy
 from mptt.forms import TreeNodeMultipleChoiceField
@@ -554,9 +555,11 @@ class JSONField(_JSONField):
             self.widget.attrs['placeholder'] = ''
 
     def prepare_value(self, value):
+        if isinstance(value, InvalidJSONInput):
+            return value
         if value is None:
             return ''
-        return super(JSONField, self).prepare_value(value)
+        return json.dumps(value, sort_keys=True, indent=4)
 
 
 #

netbox/utilities/views.py

@@ -708,22 +708,17 @@ class ComponentCreateView(View):
         if form.is_valid():
 
             new_components = []
-            data = deepcopy(form.cleaned_data)
+            data = deepcopy(request.POST)
+            data[self.parent_field] = parent.pk
 
             for name in form.cleaned_data['name_pattern']:
-                component_data = {
+
-                    self.parent_field: parent.pk,
+                # Initialize the individual component form
-                    'name': name,
+                data['name'] = name
-                }
+                component_form = self.model_form(data)
-                # Replace objects with their primary key to keep component_form.clean() happy
+
-                for k, v in data.items():
-                    if hasattr(v, 'pk'):
-                        component_data[k] = v.pk
-                    else:
-                        component_data[k] = v
-                component_form = self.model_form(component_data)
                 if component_form.is_valid():
-                    new_components.append(component_form.save(commit=False))
+                    new_components.append(component_form)
                 else:
                     for field, errors in component_form.errors.as_data().items():
                         # Assign errors on the child form's name field to name_pattern on the parent form
@@ -733,26 +728,10 @@ class ComponentCreateView(View):
                             form.add_error(field, '{}: {}'.format(name, ', '.join(e)))
 
             if not form.errors:
-                self.model.objects.bulk_create(new_components)
 
-                # ManyToMany relations are bulk created via the through model
+                # Create the new components
-                m2m_fields = [field for field in component_form.fields if type(component_form.fields[field]) in M2M_FIELD_TYPES]
+                for component_form in new_components:
-                if m2m_fields:
+                    component_form.save()
-                    for field in m2m_fields:
-                        field_links = []
-                        for new_component in new_components:
-                            for related_obj in component_form.cleaned_data[field]:
-                                # The through model columns are the id's of our M2M relation objects
-                                through_kwargs = {}
-                                new_component_column = new_component.__class__.__name__ + '_id'
-                                related_obj_column = related_obj.__class__.__name__ + '_id'
-                                through_kwargs.update({
-                                    new_component_column.lower(): new_component.id,
-                                    related_obj_column.lower(): related_obj.id
-                                })
-                                field_link = getattr(self.model, field).through(**through_kwargs)
-                                field_links.append(field_link)
-                        getattr(self.model, field).through.objects.bulk_create(field_links)
 
                 messages.success(request, "Added {} {} to {}.".format(
                     len(new_components), self.model._meta.verbose_name_plural, parent

netbox/virtualization/api/serializers.py

@@ -105,6 +105,7 @@ class VirtualMachineSerializer(TaggitSerializer, CustomFieldModelSerializer):
         fields = [
             'id', 'name', 'status', 'site', 'cluster', 'role', 'tenant', 'platform', 'primary_ip', 'primary_ip4',
             'primary_ip6', 'vcpus', 'memory', 'disk', 'comments', 'tags', 'custom_fields', 'created', 'last_updated',
+            'local_context_data',
         ]
 
 
@@ -115,6 +116,7 @@ class VirtualMachineWithConfigContextSerializer(VirtualMachineSerializer):
         fields = [
             'id', 'name', 'status', 'cluster', 'role', 'tenant', 'platform', 'primary_ip', 'primary_ip4', 'primary_ip6',
             'vcpus', 'memory', 'disk', 'comments', 'tags', 'custom_fields', 'config_context', 'created', 'last_updated',
+            'local_context_data',
         ]
 
     def get_config_context(self, obj):

netbox/virtualization/api/views.py

@@ -33,7 +33,7 @@ class ClusterGroupViewSet(ModelViewSet):
 
 
 class ClusterViewSet(CustomFieldModelViewSet):
-    queryset = Cluster.objects.select_related('type', 'group')
+    queryset = Cluster.objects.select_related('type', 'group').prefetch_related('tags')
     serializer_class = serializers.ClusterSerializer
     filter_class = filters.ClusterFilter
 
@@ -45,7 +45,7 @@ class ClusterViewSet(CustomFieldModelViewSet):
 class VirtualMachineViewSet(CustomFieldModelViewSet):
     queryset = VirtualMachine.objects.select_related(
         'cluster__site', 'role', 'tenant', 'platform', 'primary_ip4', 'primary_ip6'
-    )
+    ).prefetch_related('tags')
     filter_class = filters.VirtualMachineFilter
 
     def get_serializer_class(self):
@@ -58,6 +58,8 @@ class VirtualMachineViewSet(CustomFieldModelViewSet):
 
 
 class InterfaceViewSet(ModelViewSet):
-    queryset = Interface.objects.filter(virtual_machine__isnull=False).select_related('virtual_machine')
+    queryset = Interface.objects.filter(
+        virtual_machine__isnull=False
+    ).select_related('virtual_machine').prefetch_related('tags')
     serializer_class = serializers.InterfaceSerializer
     filter_class = filters.InterfaceFilter

netbox/virtualization/forms.py

@@ -6,7 +6,6 @@ from taggit.forms import TagField
 
 from dcim.constants import IFACE_FF_VIRTUAL, IFACE_MODE_ACCESS, IFACE_MODE_TAGGED_ALL
 from dcim.forms import INTERFACE_MODE_HELP_TEXT
-from dcim.formfields import MACAddressFormField
 from dcim.models import Device, DeviceRole, Interface, Platform, Rack, Region, Site
 from extras.forms import AddRemoveTagsForm, CustomFieldBulkEditForm, CustomFieldForm, CustomFieldFilterForm
 from ipam.models import IPAddress
@@ -15,7 +14,8 @@ from tenancy.models import Tenant
 from utilities.forms import (
     AnnotatedMultipleChoiceField, APISelect, APISelectMultiple, BootstrapMixin, BulkEditForm, BulkEditNullBooleanSelect,
     ChainedFieldsMixin, ChainedModelChoiceField, ChainedModelMultipleChoiceField, CommentField, ComponentForm,
-    ConfirmationForm, CSVChoiceField, ExpandableNameField, FilterChoiceField, SlugField, SmallTextarea, add_blank_choice
+    ConfirmationForm, CSVChoiceField, ExpandableNameField, FilterChoiceField, JSONField, SlugField, SmallTextarea,
+    add_blank_choice
 )
 from .constants import VM_STATUS_CHOICES
 from .models import Cluster, ClusterGroup, ClusterType, VirtualMachine
@@ -245,6 +245,7 @@ class VirtualMachineForm(BootstrapMixin, TenancyForm, CustomFieldForm):
         )
     )
     tags = TagField(required=False)
+    local_context_data = JSONField(required=False)
 
     class Meta:
         model = VirtualMachine
@@ -252,6 +253,9 @@ class VirtualMachineForm(BootstrapMixin, TenancyForm, CustomFieldForm):
             'name', 'status', 'cluster_group', 'cluster', 'role', 'tenant', 'platform', 'primary_ip4', 'primary_ip6',
             'vcpus', 'memory', 'disk', 'comments', 'tags',
         ]
+        help_texts = {
+            'local_context_data': "Local config context data overwrites all sources contexts in the final rendered config context",
+        }
 
     def __init__(self, *args, **kwargs):
 
@@ -413,11 +417,12 @@ class VirtualMachineFilterForm(BootstrapMixin, CustomFieldFilterForm):
 #
 
 class InterfaceForm(BootstrapMixin, forms.ModelForm):
+    tags = TagField(required=False)
 
     class Meta:
         model = Interface
         fields = [
-            'virtual_machine', 'name', 'form_factor', 'enabled', 'mac_address', 'mtu', 'description', 'mode',
+            'virtual_machine', 'name', 'form_factor', 'enabled', 'mac_address', 'mtu', 'description', 'mode', 'tags',
             'untagged_vlan', 'tagged_vlans',
         ]
         widgets = {
@@ -454,8 +459,9 @@ class InterfaceCreateForm(ComponentForm):
     form_factor = forms.ChoiceField(choices=VIFACE_FF_CHOICES, initial=IFACE_FF_VIRTUAL, widget=forms.HiddenInput())
     enabled = forms.BooleanField(required=False)
     mtu = forms.IntegerField(required=False, min_value=1, max_value=32767, label='MTU')
-    mac_address = MACAddressFormField(required=False, label='MAC Address')
+    mac_address = forms.CharField(required=False, label='MAC Address')
     description = forms.CharField(max_length=100, required=False)
+    tags = TagField(required=False)
 
     def __init__(self, *args, **kwargs):
 

netbox/virtualization/migrations/0008_virtualmachine_local_context_data.py

@@ -0,0 +1,19 @@
+# Generated by Django 2.0.8 on 2018-09-16 02:01
+
+import django.contrib.postgres.fields.jsonb
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('virtualization', '0007_change_logging'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='virtualmachine',
+            name='local_context_data',
+            field=django.contrib.postgres.fields.jsonb.JSONField(blank=True, null=True),
+        ),
+    ]