Support 202203 fixes (#1148)

* remove survey notifications

* fixing old reference of IXF_IMPORTER_DAYS_UNTIL_TICKET through EnvironmentSettings, this setting is no longer controlled through that and should come straight from settings

* fix session auth not setting x-auth-id header (#1120)
fix basic auth not setting x-auth-id header on success (#1120)
fix api key auth only setting prefix in x-auth-id header (#1120)
fix x-auth-id header not being cleared between requests (#1120)

* fix issue with rest throttling breaking api-cache generation (#1146)

* add caching for get_permission_holder_from_request - fixes perfomance issues in #1147

* fix intermediate issue with api_cache rest throttle tests

* sanitize cache key names for state normalization (#1079)
each state normalization lookup moved into its own transaction so errors dont cause us to lose already obtained data (#1079)
write cache regardess of --commit on or off (#1079)
add a sanity check for running non-committal mode without --limit (#1079)

* fix issue with ip block rate limiting if x-forwarded-for is set (#1126)

* better handling of melissa timeouts through retrying (#1079)
fix state normalization cache timeout to have no expiry (#1079)
normalization command will display validation errors at the end and exit with a return code if there are any (#1079)

* automatically apply address field normalization for `state` (#1079)

* additional tests

* only do a sanity check for --limit if no specific object is targeted

* linting

Co-authored-by: Stefan Pratter <stefan@20c.com>

peeringdb_server/geo.py

@@ -2,9 +2,12 @@
 Utilities for geocoding and geo normalization.
 """
 
+import re
+
 import googlemaps
 import requests
 import structlog
+import unidecode
 from django.core.cache import cache
 from django.utils.translation import gettext_lazy as _
 
@@ -268,15 +271,25 @@ class Melissa:
         This will use django-cache if it exists
         """
 
-        key = f"geo.normalize.state.{country_code}.{state}"
+        if not state:
+            return state
+
+        # clean up state value for cache key
+
+        state_clean = unidecode.unidecode(state.lower()).strip()
+        state_clean = re.sub(r"[^a-zA-Z]+", "", state_clean)
+
+        key = f"geo.normalize.state.{country_code}.{state_clean}"
 
         value = cache.get(key)
         if value is None:
+
             result = self.global_address(country=country_code, address1=state)
+
             try:
                 record = result["Records"][0]
                 value = record.get("AdministrativeArea") or state
             except (KeyError, IndexError):
                 value = state
-            cache.set(key, value)
+            cache.set(key, value, timeout=None)
         return value