0784265f80
* Sorting icon from google material icons #1419 Manual IX-F import request queue can get stuck #1182 IX Object Creation Per Policy #1364 Creating a new network not possible #1401 IX-F Importer: Cosmetic issue with "resolved" emails and double-slashes in URLs after the FQDN #1334 Add a "Delete Affiliation" button/option to the profile #1226 Redis and negative caching #1431 * linting * update gen_docs to use py3.11 * fix issue with api docs schema regen * regen apidoc schema and db schema graph * fix username validation for social media * Add test case for social media validation * linting * tests shouldnt use redis * also fix session cache setup (although not used atm) * linting * all caches to localmemcache during testing --------- Co-authored-by: 20C <code@20c.com> Co-authored-by: Matt Griswold <grizz@20c.com>
2.9 KiB
Developer Documentation on Negative Caching in PeeringDB
Overview
Negative caching provides a way to remember previously seen HTTP error codes (e.g., 401, 403, 429) and cache these responses. This minimizes the need to constantly re-evaluate requests that we expect will return the same error codes. This document provides developers with information about the negative caching system implemented in PeeringDB using Redis.
Development Considerations
When working with the negative caching system, developers should be aware of the following:
- Testing Repeated Responses: If you are writing tests or manually testing repeated requests that initially return HTTP error codes like 401, 403, or 429, remember that the response might be cached by the negative caching system. To get accurate and consistent test results:
- You may need to manually clear the negative cache before retrying the request.
- Do not assume that just because you received an error once, subsequent requests within the cache duration will re-evaluate permissions or rate limits.
- When in doubt, turn
NEGATIVE_CACHE_ENABLEDoff during initial implementation / testing
Negative Cache Settings
Below are the default settings for the negative caching system. Each setting controls how long a specific error code is cached:
-
401 - Unauthorized: Cached for 1 minute.
set_option("NEGATIVE_CACHE_EXPIRY_401", 60) -
403 - Forbidden: Cached for 10 seconds. This is typically due to permission check failures. It's essential to keep this cache duration short, espe ially since some permission checks on write requests (POST, PUT) might require checking the payload to determine the right permissions.
set_option("NEGATIVE_CACHE_EXPIRY_403", 10) -
429 - Too Many Requests: Cached for 10 seconds. This setting should remain low to avoid interference with the existing REST API rate limiting. If this cache is too long, it can obscure the accurate rate limit reset time that's already in place.
set_option("NEGATIVE_CACHE_EXPIRY_429", 10) -
Inactive Users and Keys: Cached for 1 hour. This helps in reducing checks for users or API keys that have been marked as inactive.
set_option("NEGATIVE_CACHE_EXPIRY_INACTIVE_AUTH", 3600) -
Global Negative Cache Switch: Controls whether negative caching is globally enabled or disabled.
set_option("NEGATIVE_CACHE_ENABLED", True)
Adjusting Settings
When making changes to the settings above:
- Ensure you understand the implications of the change.
- Test any changes in a development or staging environment before applying to production to avoid unintended behaviors.
- Monitor the application and Redis performance after changes, as caching can affect resource utilization.
Clearing the cache manually
from django.core.cache import cache
...
cache.clear()