Support 202102 (#950)

* install django-grainy

* nsp to grainy first iteration

* nsp to grainy second iteration

* grainy and django-grainy pinned to latest releases

* Fix typo

* Update djangorestframework, peeringdb, django-ratelimit

* Rewrite login view ratelimit decorator

* Relock pipfile

* add list() to make copy of dictionaries before iterating

* relock pipfile with python3.9
change docker to use python3.9

* add ordering to admin search queryset for deskproticket and email

* add org api key and begin to write tests

* additional key tests

* add drf-api-keys to pipfile

* Wire orgapikey to modelviewsetpermissions

* Update api key helper functions

* add put test

* Add Org API key tab to frontend

* Add user api key model

* Update user key handling and tests

* Update APIPermissionsApplicator to make it work w requests

* Add org api key perm panel

* add org key permissions

* Add user api key views

* Add templates for handling user api key (adding, not revoking)

* relock pipfile

* assorted fixes and tweaks

* Add general user group permissions and org user group perms

* refactor org api key perms

* Add tests for api keys

* Add docstrings to permissions helpers

* Add api key examples

* squash migrations

* remove custom api key header config

* Change api key test setup

* Update permissions for grainy change

* Bump up pipfile and pipfile.lock

* Add API Key to Verification Queue Item

* Delete travis

* Add workaround to Dockerfile

* update pipfile and sort out migrations

* Add comment to Dockerfile

* Re-add API Key migrations

* Add locale to .gitignore

* remove suggest functionality from ix

* Update test to recognize that IX api no longer has suggest function

* Add test to outlaw POSTing an IX w its org equal to the suggest entity org

* Add meta information geowarning

* Add alert to demonstrate UI

* Add error to fac update

* Add template warning for geovalidation

* Add geowarning meta js

* cover absent meta_response test case

* Update styles for geowarning

* refactor geotag warning implementation

* null lat and long on unsuccessful geo locate

* modify geovalidation frontend update

* Add deskproticket model email field

* Add missing span

* add email to org keys

* Add email to org key tests

* update serializer with rdap validation wrapper

* update admin for api keys

* Enable writing an email as part of org key creation

* Add email validation to org api key form

* fix css style on perm row

* Add suggested info to api response

* display suggested address on frontend

* add needs geocode to serializer

* save lat long on forward geonormalization

* add address suggestion submit button

* Add suggested address popin to ADD facility form

* Fix css

* add lat and long rounding to geocodenabled model clean method

* add migration and regression test for lat long decimal db constraint

* Add another regression test for model decimal places

* Get deskpro functions passing isort and flake

* Update ticket_queue_deletion_prevented

* update ticket_queue_deletion_prevented for use with org api key

* add template for org key dpt from asnauto skipvq

* Update deskproticket for rdap error

* add facility aka

* add aka to serializer and views

* black and isort test api keys

* fix typo in org key deskpro template

* skip or rewrite unapplicable org key tests, and add as_set tests

* adjust api key test comments

* Add vqi_notify to signals

* Add reversion comments for api keys and helper function

* update how org keys are added to verification queue items

* rename verification queue item fk from api_key to org_key

* fix group id error

* update key tests with correct http header info

* check both user and key, not just user

* templates fiex

* adapt deskpro integration to work with email only

* make org api keys editable for desc and email

* pipfile relock

* edit test setupdata settings for groups

* Change comment to signify we don't need to remove code

* address untranslated accept button

* Add docstrings to the serializer functions

* Add loading shim

* Add migration for all longname and aka

* Add aka and long name to views and serializers

* delete migration w decimals

* standardize serializer lat and long fields

* Add clean rounding for lat and long

* fix serializer error

* api key admin improvements

* fix linebreak in user api key form

* remove debug prints

* Add rounding util

* Add rounding to lat and long fields

* remove 'clean' from geocode method (logic now in admin form)

* remove erroneous tests

* revert serializer changes

* Fix migrations

* Add long name and aka to admin models

* Update API key docs

* Add documentation for api keys

* fix typo

* fix org api key revoke broken by editable api keys

* doc tweaks

* doc tweaks

* doc tweaks

* black format

* fix migration hierarchy

* docs

* docs

* api key permissions screenshot

* formatting

* formatting

* padding fixed

* remove one image

* fix get_user_from_request type checking
take out POST only valdiator for entity suggest

* didnt mean to commit the django-peeringdb mount

* fix suggest on PUT net
fix tests

* black formatting

* update org key permission template

* install rust for cryptography

* pipfile relock (django-peeringdb to 2.6)

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: Elliot Frank <elliot@20c.com>

peeringdb_server/deskpro.py

@@ -2,36 +2,50 @@
 DeskPro API Client
 """
 
-import uuid
-import re
-import requests
 import datetime
+import re
+import uuid
 
-from django.template import loader
-from django.conf import settings
 import django.urls
+import requests
+from django.conf import settings
+from django.template import loader
 
-from peeringdb_server.models import DeskProTicket
 from peeringdb_server.inet import RdapNotFoundError
+from peeringdb_server.models import is_suggested, DeskProTicket
+from peeringdb_server.permissions import get_user_from_request, get_org_key_from_request
+
+from django.utils.translation import override
 
 
 def ticket_queue(subject, body, user):
     """ queue a deskpro ticket for creation """
 
-    ticket = DeskProTicket.objects.create(
+    DeskProTicket.objects.create(
         subject=f"{settings.EMAIL_SUBJECT_PREFIX}{subject}",
         body=body,
         user=user,
     )
 
 
+def ticket_queue_email_only(subject, body, email):
+    """ queue a deskpro ticket for creation """
+
+    DeskProTicket.objects.create(
+        subject=f"{settings.EMAIL_SUBJECT_PREFIX}{subject}",
+        body=body,
+        email=email,
+        user=None,
+    )
+
+
 class APIError(IOError):
     def __init__(self, msg, data):
         super().__init__(msg)
         self.data = data
 
 
-def ticket_queue_asnauto_skipvq(user, org, net, rir_data):
+def ticket_queue_asnauto_skipvq(request, org, net, rir_data):
     """
     queue deskro ticket creation for asn automation action: skip vq
     """
@@ -46,13 +60,28 @@ def ticket_queue_asnauto_skipvq(user, org, net, rir_data):
     else:
         org_name = org.name
 
-    ticket_queue(
-        f"[ASNAUTO] Network '{net_name}' approved for existing Org '{org_name}'",
-        loader.get_template("email/notify-pdb-admin-asnauto-skipvq.txt").render(
-            {"user": user, "org": org, "net": net, "rir_data": rir_data}
-        ),
-        user,
-    )
+    user = get_user_from_request(request)
+    if user:
+        ticket_queue(
+            f"[ASNAUTO] Network '{net_name}' approved for existing Org '{org_name}'",
+            loader.get_template("email/notify-pdb-admin-asnauto-skipvq.txt").render(
+                {"user": user, "org": org, "net": net, "rir_data": rir_data}
+            ),
+            user,
+        )
+        return
+
+    org_key = get_org_key_from_request(request)
+    if org_key:
+        ticket_queue_email_only(
+            f"[ASNAUTO] Network '{net_name}' approved for existing Org '{org_name}'",
+            loader.get_template(
+                "email/notify-pdb-admin-asnauto-skipvq-org-key.txt"
+            ).render(
+                {"org_key": org_key, "org": org, "net": net, "rir_data": rir_data}
+            ),
+            org_key.email,
+        )
 
 
 def ticket_queue_asnauto_affil(user, org, net, rir_data):
@@ -107,7 +136,53 @@ def ticket_queue_asnauto_create(
     )
 
 
-def ticket_queue_rdap_error(user, asn, error):
+def ticket_queue_vqi_notify(instance, rdap):
+    item = instance.item
+    user = instance.user
+    org_key = instance.org_key
+
+    with override("en"):
+        entity_type_name = str(instance.content_type)
+
+    title = f"{entity_type_name} - {item}"
+
+    if is_suggested(item):
+        title = f"[SUGGEST] {title}"
+
+    if user:
+        ticket_queue(
+            title,
+            loader.get_template("email/notify-pdb-admin-vq.txt").render(
+                {
+                    "entity_type_name": entity_type_name,
+                    "suggested": is_suggested(item),
+                    "item": item,
+                    "user": user,
+                    "rdap": rdap,
+                    "edit_url": "%s%s" % (settings.BASE_URL, instance.item_admin_url),
+                }
+            ),
+            user,
+        )
+
+    elif org_key:
+        ticket_queue_email_only(
+            title,
+            loader.get_template("email/notify-pdb-admin-vq-org-key.txt").render(
+                {
+                    "entity_type_name": entity_type_name,
+                    "suggested": is_suggested(item),
+                    "item": item,
+                    "org_key": org_key,
+                    "rdap": rdap,
+                    "edit_url": "%s%s" % (settings.BASE_URL, instance.item_admin_url),
+                }
+            ),
+            org_key.email,
+        )
+
+
+def ticket_queue_rdap_error(request, asn, error):
     if isinstance(error, RdapNotFoundError):
         return
     error_message = f"{error}"
@@ -115,14 +190,29 @@ def ticket_queue_rdap_error(user, asn, error):
     if re.match("(.+) returned 400", error_message):
         return
 
-    subject = f"[RDAP_ERR] {user.username} - AS{asn}"
-    ticket_queue(
-        subject,
-        loader.get_template("email/notify-pdb-admin-rdap-error.txt").render(
-            {"user": user, "asn": asn, "error_details": error_message}
-        ),
-        user,
-    )
+    user = get_user_from_request(request)
+
+    if user:
+        subject = f"[RDAP_ERR] {user.username} - AS{asn}"
+        ticket_queue(
+            subject,
+            loader.get_template("email/notify-pdb-admin-rdap-error.txt").render(
+                {"user": user, "asn": asn, "error_details": error_message}
+            ),
+            user,
+        )
+        return
+
+    org_key = get_org_key_from_request(request)
+    if org_key:
+        subject = f"[RDAP_ERR] {org_key.email} - AS{asn}"
+        ticket_queue_email_only(
+            subject,
+            loader.get_template("email/notify-pdb-admin-rdap-error-org-key.txt").render(
+                {"org_key": org_key, "asn": asn, "error_details": error_message}
+            ),
+            org_key.email,
+        )
 
 
 class APIClient:
@@ -162,23 +252,60 @@ class APIClient:
         )
         return self.parse_response(response)
 
-    def require_person(self, user):
-        person = self.get("people", {"primary_email": user.email})
+    def require_person(self, email, user=None):
+
+        """
+        Gets or creates a deskpro person using the deskpro API
+
+        At the minimum this needs to be passed an email
+        address.
+
+        If a peeringdb user instance is also specified, it will
+        be used to fill in name information.
+
+        Arguments:
+
+        - email(`str`)
+        - user(`User`)
+        """
+
+        person = self.get("people", {"primary_email": email})
+
         if not person:
-            person = self.create(
-                "people",
-                {
-                    "primary_email": user.email,
-                    "first_name": user.first_name,
-                    "last_name": user.last_name,
-                    "name": user.full_name,
-                },
-            )
+
+            payload = {"primary_email": email}
+
+            if user:
+                payload.update(
+                    first_name=user.first_name,
+                    last_name=user.last_name,
+                    name=user.full_name,
+                )
+            else:
+                payload.update(name=email)
+
+            person = self.create("people", payload)
 
         return person
 
     def create_ticket(self, ticket):
-        person = self.require_person(ticket.user)
+
+        """
+        Creates a deskpro ticket using the deskpro API
+
+        Arguments:
+
+        - ticket (`DeskProTicket`)
+        """
+
+        if ticket.user:
+            person = self.require_person(ticket.user.email, user=ticket.user)
+        elif ticket.email:
+            person = self.require_person(ticket.email)
+        else:
+            raise ValueError(
+                "Either user or email need to be specified on the DeskProTicket instance"
+            )
 
         if not ticket.deskpro_id:
 
@@ -263,7 +390,7 @@ class FailingMockAPIClient(MockAPIClient):
         )
 
 
-def ticket_queue_deletion_prevented(user, instance):
+def ticket_queue_deletion_prevented(request, instance):
     """
     queue deskpro ticket to notify about the prevented
     deletion of an object #696
@@ -297,19 +424,43 @@ def ticket_queue_deletion_prevented(user, instance):
 
     model_name = instance.__class__.__name__.lower()
 
-    # create ticket
+    # Create ticket if a request was made by user or UserAPIKey
+    user = get_user_from_request(request)
+    if user:
+        ticket_queue(
+            subject,
+            loader.get_template("email/notify-pdb-admin-deletion-prevented.txt").render(
+                {
+                    "user": user,
+                    "instance": instance,
+                    "admin_url": settings.BASE_URL
+                    + django.urls.reverse(
+                        f"admin:peeringdb_server_{model_name}_change",
+                        args=(instance.id,),
+                    ),
+                }
+            ),
+            user,
+        )
+        return
 
-    ticket_queue(
-        subject,
-        loader.get_template("email/notify-pdb-admin-deletion-prevented.txt").render(
-            {
-                "user": user,
-                "instance": instance,
-                "admin_url": settings.BASE_URL
-                + django.urls.reverse(
-                    f"admin:peeringdb_server_{model_name}_change", args=(instance.id,)
-                ),
-            }
-        ),
-        user,
-    )
+    # Create ticket if request was made by OrgAPIKey
+    org_key = get_org_key_from_request(request)
+    if org_key:
+        ticket_queue_email_only(
+            subject,
+            loader.get_template(
+                "email/notify-pdb-admin-deletion-prevented-org-key.txt"
+            ).render(
+                {
+                    "org_key": org_key,
+                    "instance": instance,
+                    "admin_url": settings.BASE_URL
+                    + django.urls.reverse(
+                        f"admin:peeringdb_server_{model_name}_change",
+                        args=(instance.id,),
+                    ),
+                }
+            ),
+            org_key.email,
+        )