1
0
mirror of https://github.com/peeringdb/peeringdb.git synced 2024-05-11 05:55:09 +00:00
Commit Graph

168 Commits

Author SHA1 Message Date
Matt Griswold
4bbe4f1d3d Beta fixes 202202 (#1112)
* remove survey notifications

* fixing old reference of IXF_IMPORTER_DAYS_UNTIL_TICKET through EnvironmentSettings, this setting is no longer controlled through that and should come straight from settings

* fix pdb_load_data ixlan.status error

Co-authored-by: Stefan Pratter <stefan@20c.com>
2022-02-10 06:17:26 -06:00
Matt Griswold
e2619a001f Support 202201 (#1111)
* remove survey notifications

* substantially rate limit unauthenticated /api/ queries to encourage authenticated queries #853

* move api throttle class configuration to settings (#853)

* #722 with a more generic validation approach

* Add organisations and registered users to "Global System Statistics" in footer #620

* poetry relock

* linting

* regen docs

* fix test data

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: David Poarch <dpoarch@20c.com>
2022-02-08 13:14:27 -06:00
Matt Griswold
677f94f550 Support 202111 (#1101)
* Block registering private ASN ranges

* Add a continental region field for facilities #1007

* Incorrect order of search results #232

* Allow users to upload a small logo to their record #346

* Sponsor badge CSS and text translation issues #453

* IXP and Facility summary #18

* Add sales email and phone contact to ix object #949

* Increase timeout timer for IX-F JSON importer to 30s #962

* IX-F manually triggered import bugs #1054

* remove survey notifications

* IX-F importer: manual add followed by IX-F prompted add can result in "The server rejected your data" #807

* performance fix for net view

* api documentation generate broken #956

* fix poc set missing from net view

* Auto focus cursor on search field on main website #695

* PeeringDB website has a poor choice of line-breaks for IPv6 addresses. #748

* Delete childless org objects #838

* linting

* poetry lock

* re-add mistakingly dropped translation to email template

* fix template variables

* regen docs

* regen api docs

* add u2f api

* fix merge cruft

* fido u2f webauthn first steps

* dont need this

* typo

* first ux steps for security key management, u2f views

* u2f auth

* linting

* Cascade delete when performed by superuser in django-admin #735

* security key passwordless login

* reset webauthn migrations

* security key 2fa

* linting

* ux cleanup

* cleanup

* Add IXP to AS record / dropdown limited #469

* ignore orgs that have pre-dated or finished sponsorships attached

* split out to django_security_keys

* default to 90 days

* make URL required for new objects #374

* move CONN_MAX_AGE to proper location and expose as env var (#1060)

* Error message for invalid password on email change

* Registering a new facility or exchange organization is broken #1062

* Creating a facility that matches the name of a soft-deleted facility will cause the entry to bypass the verification queue #901

* irr source validator doesn't allow for hyphens in source #921

* split out javascript

* split out js

* Clicking "Add" to add a user api-key without providing a name for the key raises Internal Error #1033

* remove security key admin views

* global stats don't show up at login screen #284

* Possible for "pending" exchange to have "deleted" ixlan (#1077)

* webauthn finalizations

* oidc support for oauth provider #1070

* Revert "Merge branch 'gh_1070' into 'support_202111'"

* remove unused dev mounts

* linting

* add django-security-key and relock poetry

* re-add imports removed through linting

* docs regen

* oauth toolkit pinned to 1.6.1
django-security-keys pinned to 1.0.1

* remove debug message

* linting

* linting

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: David Poarch <dpoarch@20c.com>
2022-01-11 08:56:47 -06:00
jlamanna
a5abe0e78f Nanog 83 Hackathon improvements to the PeeringDB Website (#1083)
* Nanog 83 Hackathon improvements to the PeeringDB Website

- If a user inputs only numbers, search for ASN only
- If a user inputs what looks like a IPv4 or IPv6 address, search Network IP addresses only
- Use direct SQL in the above instances rather than Haystack fuzzy matching, though search fields for ASN and ipaddr[46] were added to the Haystack search index

* Revert change to prepare_term

* remove print

* run black

* add ipv4 and 6 search tests

Co-authored-by: Jeff Kala <jeff.l.kala@gmail.com>
2022-01-10 07:40:26 -06:00
Matt Griswold
c9108741cf fix atomicity (#1087)
* remove survey notifications

* fixes #1086 atomicity issues

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-11-12 11:16:25 -06:00
Matt Griswold
8ef76dc5ab Support 202110 (#1085)
* Block registering private ASN ranges

* Add a continental region field for facilities #1007

* Incorrect order of search results #232

* Allow users to upload a small logo to their record #346

* Sponsor badge CSS and text translation issues #453

* IXP and Facility summary #18

* Add sales email and phone contact to ix object #949

* Increase timeout timer for IX-F JSON importer to 30s #962

* IX-F manually triggered import bugs #1054

* remove survey notifications

* IX-F importer: manual add followed by IX-F prompted add can result in "The server rejected your data" #807

* performance fix for net view

* api documentation generate broken #956

* fix poc set missing from net view

* Auto focus cursor on search field on main website #695

* PeeringDB website has a poor choice of line-breaks for IPv6 addresses. #748

* Delete childless org objects #838

* linting

* poetry lock

* re-add mistakingly dropped translation to email template

* fix template variables

* regen docs

* regen api docs

* fix merge cruft

* django-peeringdb to 2.11 and poetry relock

* linting

* comment out django-peeringdb volume mount

* add missing declaration

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: David Poarch <dpoarch@20c.com>
2021-11-09 09:47:38 -06:00
Matt Griswold
1eda45ffeb ignore new ixf field (#1082)
* ignore new ixf field

* pre-commit

* gh actions linting fix
2021-10-30 10:20:04 -05:00
Matt Griswold
423552a20a Fix undefined sponsor (#1074)
* fix undefined sponsor badges

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-10-21 12:38:35 -05:00
Matt Griswold
0348d9fc28 Support 202109 fixes (#1071)
* remove survey notifications

* humanize file size error when nginx triggers a 413 response (#346)

* linting

* add max file size note to logo help tooltip

* fix issue with advanced search no longer show entries for non-sponsor organizations (#453)

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-10-19 11:36:30 -05:00
Matt Griswold
c21130eed9 Docs 202109 (#1067)
* module docstrings

* db schema graph

* dev docs first pass

* dev docs pass 2

* add generated notification to top of generated docs files

* linting

* regen docs

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: Sunshine Buchholz <sunshine@20c.com>
2021-10-15 03:25:38 -05:00
Matt Griswold
8361e18ed4 346 fixes 2 (#1069)
* remove survey notifications

* remove old logo file when a new logo file is uploaded #346
loading-shim for upload

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-10-15 01:30:04 -05:00
Matt Griswold
f1e2e8e05d 346 fixes (#1066)
* org logo now uploads to separate directory than sponsorship logos and normalizes file name to org-{id} #346

* validation fixes for logo upload (#346)

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-10-14 02:03:10 -05:00
Matt Griswold
0c4ff0b2de Support 202109 (#1064)
* Block registering private ASN ranges

* Add a continental region field for facilities #1007

* Incorrect order of search results #232

* Allow users to upload a small logo to their record #346

* Sponsor badge CSS and text translation issues #453

* IXP and Facility summary #18

* poetry relock

* linting

* add libgcc

* lint

* bump version to 2.31.0-beta

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: David Poarch <dpoarch@20c.com>
2021-10-12 11:05:25 -05:00
Matt Griswold
dbaffdc2d8 remove survey notifications (#1063)
Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-10-11 12:09:09 -05:00
Matt Griswold
fadbcc03c9 move notification message for survey to banner (#1058)
Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-09-23 13:32:17 -05:00
S. R. Wadleigh
6fe5e6c46d Survey 20210923 (#1056)
* add 2021 october 8th survey text and link to the from page index below the register link

* trim leading space
2021-09-23 00:42:24 -05:00
Matt Griswold
636f37eb12 Support 202108 (#1051)
* add OPERATIONS_EMAIL setting

* fixes #1019: redundant saves to deleted netixlans during ix-f import

* private pocs are no longer valid (#944)

* poetry relock (handleref update)

* fixes #1032: __id api filter not working

* Additional self-selection fields for Facilities #800

* advanced search fields for available voltage, property and diverse serving substations (#1016)

* When network sets netixlan speed to 1200000 only 1T is shown instead of 1.2T ... sometimes #500

* add search-data

* comment out mount points for api-cache, search-data, django-peeringdb

* poetry relock (django-peeringdb 2.9.0)

* linting

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-09-14 08:59:09 -05:00
Matt Griswold
1addb7aae3 fix signals not being imported (#1037)
* fix signals not being imported

* move utility funciton out of signals
remove signals import from migrations

* note for signals import to be not removed

* better explanation

* fix user auto verify test

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-08-26 13:23:33 -05:00
Matt Griswold
77282c4b66 Prep 2.29 (#1024)
* django3, py39, lgtm, linting (#715)

* IX-F Importer: ticket status change when posting re-occuring conflict to existing resolved ticket (#920)

* fix recaptcha requirement for user creation in django-admin (#715)

* IX-F Importer: fix command output buffering #967

* Drop dot1q_support field #903

* fix test (#967)

* Add name, city, country to ixfac (GET operation) #166

* additional tests fir #166

* Allow IXP to trigger ix-f importer for their exchange #779

* add docker compose for dev

* add selinux labels for mountpoints

* fixes #1013: The process to permanently remove old soft-deleted network contacts pdb_delete_pocs raises a false ProtectedAction

* fix api test

* relock poetry

* remove django_namespace_perms from installed apps

* fix user permissios ui

* remove remaining references to django namespace perms

* linting

* copy tox.ini

* comment flake8 check until we figure out why it ignores configs from tox.ini

* black format

* poetry lock

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-08-18 08:21:22 -05:00
Matt Griswold
63aab93f04 fix issue with update_index not respecting the --age parameter (#1010)
* fix issue with update_index not respecting the --age parameter

* fix ghactions poetry cache

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-07-24 21:46:18 -05:00
Matt Griswold
3dd72435ca Fix 23 search result order (#1002)
* fix search results listed in wrong order (#23)

* no longer need to call highlight

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-07-13 08:21:43 -05:00
Matt Griswold
5147028bee clean up / format / poetry (#1000)
* stub in poetry for pipenv

* re-add tester image

* add pre-commit / formatting

* fix ghactions

* revert test data whitespace, exclude tests/data

* revert ws

* decruft, rm tox/pipenv

* install dev packages for base image

* add lgtm config to force to py3
2021-07-10 10:12:35 -05:00
Matt Griswold
c458173667 search results page (#999)
finalize indexing

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-07-08 21:00:23 -05:00
Matt Griswold
7c3d160dec Support 202106 (#994)
* fixes #965: intermittent bug during consolidation of notifications

* fixes #863: better visibility for input validation errors

* fixes #375: re-evaluate affiliation requests on email change

* fixes #741: remove data quality validation for superusers

* fixes #587: customizable pagination in django admin

* fixes #923: Prevent deletion of a last technical contact if there is an existing netixlan object

* better search #23 (first pass)

* black format

* haystack test config to run_tests.py
remove old django_init.py test settings

* black format

* haystack test config fixes

* #23 better search (pt.2)

* rate limit distance queries (#23)
rate limiting based on query filters

* settings controlled blocking of distance filter for unauthenticated / unverified users (#23)

* fix distance filter throttling with api key auth (#23)

* fix anon user access check on distance queries

* search index and performance tweaks

* fix org_id not set in search results

* org id to int

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-07-07 17:57:04 -05:00
Matt Griswold
de413ef1f6 advanced search fixes (#984)
* fix advanced search error when not logged in

* fix error with data/my_organizations if user is not logged in

* fix 500 error when viewing advanced search as a user that is not affiliated with any org

* tests for adv search init and anonymous data/my_organizations request

Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-05-26 00:33:55 -05:00
Matt Griswold
fbc72ea682 Support 202104 (#980)
* Add migration for service level and terms

* Add service level and terms to UI and serializer, as well as data/enum

* Wire up data/enum endpoint and loader

* remove proto_ from ix UI

* derive fields for proto_unicast and proto_ipv6

* update tests for readonly fields

* Fix query for protocols

* Fix api bug with protocol

* add readonly fields to django admin

* rename readonly fields

* Add translation to names

* Add pdb api test for suggested facility re-add

* Add printing debuggin test

* add printing debugging serializer

* Update _undelete with _reapprove to handle pending cases

* Update tests (one is still failing)

* adjust suggest test

* Add ix_count to fac (834)

* Add test for ix_count on fac (834)

* Add fac_count to IX (836)

* add ix_count and fac_count to Network

* Refactor ix net_count filtering

* Add filtering for 834, 835, 836

* Remove duplicates from the Network's ix_count

* Setup Network for ix_count and fac_count (835)

* initial obj_counts for Facilities and Exchanges

* Add signals for updates to all counts

* add migration

* Add print statements to test

* introduce reversion to tests

* rename network count to net count across codebase

* fix network_count typo

* add migration to set default vals

* fix filter tests for obj_counts

* speed up migration

* fix failing tests

* fix final test

* sort out migration tree and add fac offered fields

* update frontend for facility dropdown offered_resilience

* First pass at advanced api search for user story 1

* melissa geo lookup first steps

* fix migration hierarchy

* working melissa integration

* begin ending filters for api endpoints

* add more org_present endpoints

* add search for IXs that match multiple networks

* extend logic to facility

* Add service level and terms to advanced search

* use address2 field for lookup

* melissa tests

* cleanup and docs

* uncomment offered_power

* developed offered_power component

* fix geo normalize existing cmd
normalize state

* change migration to match django-peeringdb

* add offered_space field

* Fill out remaining api filter fields

* Add org_not_present endpoint filter

* fix unit input ux

* more ux fixes

* remove merge cruft

* google for geocoding
various melissa improvements (consider result quality)

* fix tests

* refactor org_preset and org_not_present queries

* ix capacity api filters

* ix capacity filters for #802
advanced search ux for #802

* finalize advanced search UX for #802

* css fixes

* remove cruft

* fix net_count fac_count queries

* add new fields to create facility (#800)
tests for #802 and #800

* fix tests

* remove #800 changes

* fix capacity search

* more #800 changes to remove

* django-peeringdb 2.7.0 and pipenv relock

* black format

* pin black version

Co-authored-by: Elliot Frank <elliot@20c.com>
Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-05-19 08:11:30 -05:00
Matt Griswold
2162329ea1 Support 202103 (#966)
* Add check for existing pending affiliations to org

* Fix message

* 883 consolidation for deskpro tickets

* add views tests

* ipaddress normalization during search (#913)

* remove unused imports

* Update lat and long admin fields to not required

* black formatting

* relock pipfile

* fix conditions for failing asn affil test

* Update affiliation logic and add tests

* make name search case insensitive

* add asn to org_id lookup

* black format

* skip sync test while test.peeringdb.com is down

Co-authored-by: Elliot Frank <elliot@20c.com>
Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-04-13 08:59:23 -05:00
Matt Griswold
8cc0f13ec1 Support 202102 (#950)
* install django-grainy

* nsp to grainy first iteration

* nsp to grainy second iteration

* grainy and django-grainy pinned to latest releases

* Fix typo

* Update djangorestframework, peeringdb, django-ratelimit

* Rewrite login view ratelimit decorator

* Relock pipfile

* add list() to make copy of dictionaries before iterating

* relock pipfile with python3.9
change docker to use python3.9

* add ordering to admin search queryset for deskproticket and email

* add org api key and begin to write tests

* additional key tests

* add drf-api-keys to pipfile

* Wire orgapikey to modelviewsetpermissions

* Update api key helper functions

* add put test

* Add Org API key tab to frontend

* Add user api key model

* Update user key handling and tests

* Update APIPermissionsApplicator to make it work w requests

* Add org api key perm panel

* add org key permissions

* Add user api key views

* Add templates for handling user api key (adding, not revoking)

* relock pipfile

* assorted fixes and tweaks

* Add general user group permissions and org user group perms

* refactor org api key perms

* Add tests for api keys

* Add docstrings to permissions helpers

* Add api key examples

* squash migrations

* remove custom api key header config

* Change api key test setup

* Update permissions for grainy change

* Bump up pipfile and pipfile.lock

* Add API Key to Verification Queue Item

* Delete travis

* Add workaround to Dockerfile

* update pipfile and sort out migrations

* Add comment to Dockerfile

* Re-add API Key migrations

* Add locale to .gitignore

* remove suggest functionality from ix

* Update test to recognize that IX api no longer has suggest function

* Add test to outlaw POSTing an IX w its org equal to the suggest entity org

* Add meta information geowarning

* Add alert to demonstrate UI

* Add error to fac update

* Add template warning for geovalidation

* Add geowarning meta js

* cover absent meta_response test case

* Update styles for geowarning

* refactor geotag warning implementation

* null lat and long on unsuccessful geo locate

* modify geovalidation frontend update

* Add deskproticket model email field

* Add missing span

* add email to org keys

* Add email to org key tests

* update serializer with rdap validation wrapper

* update admin for api keys

* Enable writing an email as part of org key creation

* Add email validation to org api key form

* fix css style on perm row

* Add suggested info to api response

* display suggested address on frontend

* add needs geocode to serializer

* save lat long on forward geonormalization

* add address suggestion submit button

* Add suggested address popin to ADD facility form

* Fix css

* add lat and long rounding to geocodenabled model clean method

* add migration and regression test for lat long decimal db constraint

* Add another regression test for model decimal places

* Get deskpro functions passing isort and flake

* Update ticket_queue_deletion_prevented

* update ticket_queue_deletion_prevented for use with org api key

* add template for org key dpt from asnauto skipvq

* Update deskproticket for rdap error

* add facility aka

* add aka to serializer and views

* black and isort test api keys

* fix typo in org key deskpro template

* skip or rewrite unapplicable org key tests, and add as_set tests

* adjust api key test comments

* Add vqi_notify to signals

* Add reversion comments for api keys and helper function

* update how org keys are added to verification queue items

* rename verification queue item fk from api_key to org_key

* fix group id error

* update key tests with correct http header info

* check both user and key, not just user

* templates fiex

* adapt deskpro integration to work with email only

* make org api keys editable for desc and email

* pipfile relock

* edit test setupdata settings for groups

* Change comment to signify we don't need to remove code

* address untranslated accept button

* Add docstrings to the serializer functions

* Add loading shim

* Add migration for all longname and aka

* Add aka and long name to views and serializers

* delete migration w decimals

* standardize serializer lat and long fields

* Add clean rounding for lat and long

* fix serializer error

* api key admin improvements

* fix linebreak in user api key form

* remove debug prints

* Add rounding util

* Add rounding to lat and long fields

* remove 'clean' from geocode method (logic now in admin form)

* remove erroneous tests

* revert serializer changes

* Fix migrations

* Add long name and aka to admin models

* Update API key docs

* Add documentation for api keys

* fix typo

* fix org api key revoke broken by editable api keys

* doc tweaks

* doc tweaks

* doc tweaks

* black format

* fix migration hierarchy

* docs

* docs

* api key permissions screenshot

* formatting

* formatting

* padding fixed

* remove one image

* fix get_user_from_request type checking
take out POST only valdiator for entity suggest

* didnt mean to commit the django-peeringdb mount

* fix suggest on PUT net
fix tests

* black formatting

* update org key permission template

* install rust for cryptography

* pipfile relock (django-peeringdb to 2.6)

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: Elliot Frank <elliot@20c.com>
2021-03-09 13:30:30 -06:00
Matt Griswold
bf304ec122 Fixes 20210205 (#935)
* Add model check so you cannot double approve an admin org affiliation

* remove django namespace perms form INSTALLED_APPS

* script to fix users affected by #930

* black formatted

* add nsp back to installed apps for now
remove remaining nsp references

* fix initial_data fixtures (nsp->grainy)

* fixes #934: privacy permissions scope for pocs and ixf urls

* test that poc pirvacy scope works correctly on GUI (#934)

* black format

Co-authored-by: Elliot Frank <elliot@20c.com>
Co-authored-by: Stefan Pratter <stefan@20c.com>
2021-02-05 07:41:37 -06:00
Matt Griswold
ab1f07f379 777 fix address1 issue (#919)
* Check for None before assigning address1 (commandline tool)

* Add checking for address1 into model

Co-authored-by: Elliot Frank <elliot@20c.com>
2021-01-14 06:44:12 -06:00
Matt Griswold
b07baf3092 Support 202011 (#917)
* install django-grainy

* nsp to grainy first iteration

* Fix validation error message overflow

* Add migration, update views.py and template to add help_text to UI

* nsp to grainy second iteration

* grainy and django-grainy pinned to latest releases

* deskpro ticket cc (#875)

* black formatting

* move ac link to bottom for ticket body

* Fix typo

* Update djangorestframework, peeringdb, django-ratelimit

* Rewrite login view ratelimit decorator

* Relock pipfile

* add list() to make copy of dictionaries before iterating

* respect ix-f url visibilty in ix-f conflict emails

* Add type coercion to settings taken from environment variables

* Add bool handling

* relock pipfile with python3.9
change docker to use python3.9

* Check bool via isinstance

* add ordering to admin search queryset for deskproticket and email

* update settings with envvar_type option

* Add tooltips to add ix and add exchange views (in org)

* Add tooltip to suggest fac view

* get phone information in view

* add missing migration

* add migration and make org a geo model

* Wire normalization to put/create requests for Facility

* Update admin with new address fields

* Refactor serializer using mixin

* Add floor and suite to address API

* Write command to geonormalize existing entries

* Remove unnecessary method from model

* Add floor and suite to views

* Add ignore geo status

* Force refresh for fac and org updates

* adjust frontend typo

* add checking if update needs geosync

* redo error handling for geosync

* remove save keyword from geonormalize command script

* change raw_id_fields

* alternate autocomplete lookup field depending on where inline is called

* remove unnecessary error handling

* Add  csv option

* Fix bug
 with None vs empty string

* add regex parsing for suite and floor conversion

* Add migration that removes geo error as a field

* add geostatus update to command

* Ignore suite floor and address2 changes for api normalization

* update geomodel by removing geo_error

* Black models.py

* Black serializers.py

* remove geocode error from admin

* Add function for reversing pretty speed

* add conversion to export method

* fix typo

* fix speed value feedback after submit

* remove conditional

* Add error handling to create endpoint

* Refine floor and suite parsing regex

* Add geocoding tests

* Add json for tests

* IX-F Importer: Bogus output of "Preview" tool #896

* remove cruft

* black formatting

* IX-F Importer: history of changes per ixlan & netixlan #893

* 6 add geocode to org view

* 4 update geocode without refresh

* Update error display

* Fix bug with formatting translated string

* Add DateTimeFields to model

* Add update signals

* add last updated fields to views and serializers

* Add last updated model migration

* Add the data migration for last updated fields

* add test that tests a normal org user with create org permissions

* grainy to 1.7
django grainy to 1.9.1

* Fix formatting issues

* Adjust var names

* Refactor signals

* Temporary: save override from network model

* Empty vlan lists no longer cause error

* typo in ixf.py

* typo in admin

* Typos in model verbose names

* Add serializer IXLAN validation for ixf_ixp_import_enabled

* Add model validation to IXLan

* relock pipfile

* relock pipfile

* begin signal test file

* Remove full clean from save in ixlan

* use post_reversion_commit signal instead

* remove redundant save override

* remove cruft / debug code

* Add signal tests

* exclude organizations with city missing from commandline geosync

* Skip geosync if the only address information we have is a country

* initial commit for vlan matcher in importer

* Add more tests and remove unused imports

* update tests

* Actually add vlan matching to importer

* Add type checking for speed list and state

* Change how we register connection.state

* add bootstrap options

* add rdap cache command

* remove outdated perm docs

* rdap from master and relock

* propagate rdap settings to peeringdb.settings

* add loaddata for initial fixtures

* user friendly error message on RdapNotFound errors (#497)

* update rdap errors

* django-peeringdb to 2.5.0 and relock

* rdap to 1.2.0 and relock

* fix migration hierarchy

* add ignore_recurse_errors option

* add missing fields to mock
remove cruft missed during merge

* rdap to 1.2.1

* dont geo validate during api tests

* fix tests

* Add test file

* fix merge

* RDAP_SELF_BOOTSTRAP to False while running tests

* black formatted

* run black

* add github actions

* add runs on

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: Elliot Frank <elliot@20c.com>
2021-01-13 14:35:07 -06:00
Matt Griswold
299016282e Gh 724 (#897)
* remove log file writing from migration

* run tests on mysql

* fix tests (pt.1)

* fix tests (pt.2)

* fix all user_id errors in tests

* Fix geocode typo

* More test changes for mysql id issues

* Add coverage config that defines coverage db should go inside test folder

* update docs

* fix mysql user

* fix tests cli

* add mysql collate settings

* docs

* fix sync

* fix sync

* docs

* remove debug output

* remove XXX

* interim commit to move to dev box

* mv db local, rm after run

* updates for 724

* note layer error message and work around

* fix travis

* chown tests

* more travis fixes

* travis: touch Ctl/dev/.env

* write coverage report to ./coverage

* clean up docs

* formatting

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: Elliot Frank <elliot@20c.com>
2020-12-03 13:10:02 -06:00
Matt Griswold
b2fe79cdcd Ixf fixes 202011 (#898)
* Fix local var error and add test

* Add error handling that mentions ixlan and update one test

* Add conditional

* test to reproduce #889 and fixes for #889

* Remove member type checking

* check requirement_of_id instead of requirement_of as that can raise an exception if the relationship isn't cached

* ix-f suggestion to modify netixlan to clear an ip field is now stated as "Clear IPvX address" instead of "Set IPvX address"

* handle multiple vlans for prefixes defined on ixlan

* rewording

* fix tests

* black formatted

Co-authored-by: Elliot Frank <elliot@20c.com>
Co-authored-by: Stefan Pratter <stefan@20c.com>
2020-12-03 09:35:33 -06:00
Chris Caputo
d7680a3fb9 Network IX-F Postmortem: "Preview"->"Postmortem" (#894) 2020-11-27 11:04:47 -06:00
Matt Griswold
a2d14a6658 fix tests (#869) 2020-11-04 01:39:15 -06:00
Matt Griswold
16285c13f3 Support 202010 (#868)
* Add migrations for 463_745_775

* Black format the migration

* Bump up django_peeringdb version

* api require django-peeringdb 2.3.0.1 and later

* Set timezone to utc throughout sponsorship notifications

* Add government net type to existing migrtion

* Remove ixlan from peers at this exchange point

* Add new migration

* add backward migrtion

* Search on user__username not user object

* Black those changes

* Fix local erro bug

* Add test and fix speed update for deleted netixlan

* Handle attribute error

* Change language in email to reflect we don't email after x days

* Comment out EnvironmentSetting in django admin

* Disable the followup ticket and modify tests to correct deskproticket assertions

* exclude ixf tickets from deskpro publishing

* Add failing mock deskpro api and write a test with it

* Add anchor tags to notify-ixf-add

* a tags for conflict insert

* a tags for consolidated

* a tags modify

* a tags protocol conflict

* a tags remote changes

* a tags for remove

* a tags for resolved

* a tags source error

* Add target blank

* Add comment

* fix unhandled deskpro api errors

* Add mail attachment

* Remove strip_tags

* readd some strip_tags

* add explicit order for fields in admin control panel #861

* revert pretty_speed change

* Coerce is_rs_peer and operational to bool

* black formatted

* pipfile relock

Co-authored-by: Elliot Frank <elliot@20c.com>
Co-authored-by: Stefan Pratter <stefan@20c.com>
2020-11-03 18:26:15 -06:00
Stefan Pratter
fc675daf30 update facebook icon 2020-10-05 17:20:50 +00:00
Matt Griswold
04f1928b3c Updates 2.23 (#843)
* pipenv lock and pyupgrade

* pyupgrade, black format, add docs

* update for py3.7
2020-09-29 20:13:38 -05:00
Matt Griswold
aed211418d Fix typos1 conflicts resolved (#842)
* Update 0006_network_allow_ixp_update.py

fixes 'Sepcifies' typo in 0006_network_allow_ixp_update.py

* Update 0029_auto_20200401_1006.py

fixes 'Sepcifies' typo in 0029_auto_20200401_1006.py

* Update models.py

fixes 'Sepcifies' typo in models.py

* Update org_admin_views.py

fixes 'afiliation' typos in org_admin_views.py

* Update serializers.py

fix use of "it's" when it should instead be "its" in 'serializers.py'

* Update 0004_geocode_fields.py

correct use of "it's" to be "its" in '0004_geocode_fields.py', also fix spelling of 'syncronized'

* Update 0029_auto_20200401_1006.py

correct use of "it's" to be "its" in '0029_auto_20200401_1006.py', also fix spelling of 'syncronized'

* Update models.py

correct use of "it's" to be "its" in 'models.py', also fix spelling of 'syncronized'

* Update email_confirm.html

correct 're-initate' in email_confirm.html

* Update notify-pdb-admin-user-affil.txt

correct 'organzation' in 'notify-pdb-admin-user-affil.txt'

* Update op_create.md

correct 'organzation' in 'ip_create.md'

* Update notify-pdb-admin-asnauto-skipvq.txt

correct 'organzation' in 'notify-pdb-admin-asnauto-skipvq.txt'

* Update models.py

correct 'organzation' in 'models.py'

* Update notify-pdb-admin-asnauto-skipvq.txt

fix 'succesfully' in 'notify-pdb-admin-asnauto-skipvq.txt'

* Update views.py

fix 'succesfully' in 'views.py'

* Update pdb_migrate_ixlans.py

* Update models.py

several more replacements of "it's" with "its"

* Update rest.py

one more replacement of "it's" with "its"

* Update op_retrieve.md

replace "dont" with "don't"

* Update notify-org-admin-merge.txt

fix "dont" with "don't"

* Update error.html

fix "dont" with "don't"

* Update op_list.md

fix "dont" with "don't"

* Update serializers.py

fix "dont" with "don't"

* Update deskpro.py

fix "dont" with "don't"

* Update mock.py

fix "dont" with "don't"

* Update ixf.py

fix "dont" with "don't"

* Update admin.py

fix "dont" with "don't"

* Update api_schema.py

fix "dont" with "don't"

* Update signals.py

fix "dont" with "don't"

* Update views.py

fix "dont" with "don't"

* Update test_admin.py

fix "dont" with "don't"

* Update pdb_api_test.py

fix "dont" with "don't"

* Update models.py

fix "dont" with "don't"

* revert to "IXP" in help text

Co-authored-by: Theo Baschak <tbaschak@users.noreply.github.com>
Co-authored-by: Stefan Pratter <stefan@20c.com>
2020-09-29 15:01:47 -05:00
Matt Griswold
1fa46e2acb Support 202009 (#841)
* Add EmailMultiAlternatives import

* Add strip_tags import

* Add settings imports and new email test

* Add email increment to ixf and tests

* IX-F Importer: suggested update when it should be add + remove #832

* Take email increment out of if-else

* Add max and min speed settings

* Change validation check for models

* new speed validation

* Add basic user command

* Add pdb cleanup users tool

* Add pretty printing for speed

* Add users as a subparser

* add translation override to signals

* Add parser as parent of subparser

* refactor and change test

* Move override to cover single variable

* Add tooltip option for individual checkboxes

* address 'fix me' issue with field_help helper func

* Add zipcode validator and black format

* Make website required input but zipcode dependent on country

* Add net POC requirement to Netixlan serializer

* Website is now blank=False ie required in all forms

* refine error message

* Require email is not blank and add test

* Change error message

* add website and zipcode test, edit zipcode error message

* change placement of tooltip

* add question mark

* Add comment

* Add runtime error logging for ixp member import

* add uncaught error test

* delete two unused methods

* Rename test file and add different tests

* Add missing email imports (reproduces changes in hot_fix_gh_831)

* add resend methods

* Add missing email imports (reproduces changes in hot_fix_gh_831)

* Add pytest-mock to pipfile

* Add resend email mechanism

* Add email resending

* remove failing assertion

* fix for ticket_aged_proposals

* Wrap resending emails in conditional for commit

* Add resend email tests

* fix mail_Debug bug

* Figure out production mailing and resending settings

* Add stale info field

* default IXF_RESEND_FAILED_EMAILS to False
fix issue with sent being set even if email was not sent
fix issue with output stating resending of emails even if it wasnt

* IX-F Preview - shows the consolidated delete operation when it shouldn't (#824)

* black format (v 19.10)

* black formatting

* black formatting

* pipfile relock

* make changes from #825 play nice with changes from #833

* black to pipenv dev packages

Co-authored-by: Elliot Frank <elliot@20c.com>
Co-authored-by: Stefan Pratter <stefan@20c.com>
2020-09-29 13:07:56 -05:00
Matt Griswold
4bcaf8211c Gh 829 (#830)
* fix issue with ixlan serializer stemming from ixf_ixp_member_list_url being added to api (#829)

* add test for #829

Co-authored-by: Stefan Pratter <stefan@20c.com>
2020-08-26 05:50:12 -05:00
Matt Griswold
820e0e71e4 black format 2020-08-18 20:18:58 +00:00
Matt Griswold
31a0f886a7 Dotf fixes 3 (#820)
* [beta] IX-F importer: [showstopper]: Seeing 'Update' hint instead of 'Add' hint #816

* [beta] IX-F importer: list of hints should be case-independently sorted alphabetically #817

* [beta] IX-F importer: IX "Last Updated" timestamp is based on IX-F import time - not sure that is intended #812

* Improve soft delete error handling re: protectedactions

* Fix template bug

* Modify template again

* add a test to confirm ipaddr suggestions end up in template

* rebase #817 into this and adapt to changes in #817

* [beta] IX-F importer: notice about dismissed hints remains after no longer relevant #809

* Server Error 500 when deleting organisations #798

* black formatting

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: egfrank <elliot@20c.com>
2020-08-18 09:30:40 -05:00
Matt Griswold
ac047fa0bc Gh 815 (#818)
* [beta] IX-F importer: [showstopper]: unnecessary add and update suggested #815

* black formatting

Co-authored-by: Stefan Pratter <stefan@20c.com>
2020-08-18 09:29:11 -05:00
Matt Griswold
38a49b4a10 Gh 761 (#819)
* Make in_dfz a readonly field

* add migrations for in_dfz noop,
add validation error when trying to set in_dfz to false over api
fix template layout

* remove in_dfz from ixlanprefixinline admin

* fix error message

* black formatted

Co-authored-by: egfrank <elliot@20c.com>
Co-authored-by: Stefan Pratter <stefan@20c.com>
2020-08-18 09:28:18 -05:00
Matt Griswold
0e2829d82f Dotf fixes 2 (#797)
* [beta] IX-F importer: Adding entry results in 'The server rejected your data' #789

* [beta] IX-F importer: tooltip remains visible after clicking "Auto-resolve", also misspelling #788

* fix protocol-conflict notifications not going out if there are no other conflicts (#771)

* fix messy white-space in ix-f notificaiton emails (#790)

* add link to python regex documentation (#768)

* IX-F import preview when authenticating via basic auth broken #791
Re-add preview tests

* [beta] IX-F importer: Lack of "routeserver" status in IX-F JSON should imply "undefined" rather than "false" and result in no action #792

* speed not being present in the ix-f data should be ignored (#792)

* [beta] IX-F importer: ignore speed and is_rs_peer differences for now, but retain code #793

* do a simple dedupe of ip addresses before processing the ix-f export
remove email debug spam

* normalize ip addresses
bug fixes for protocol conflict

* fix typo and remove debug output

* bail if unable to reasonably dedupe duplicate ips in ix-f export data

* sanitization fixes

* reset ix error notification timer on successful import

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: Elliot Frank <elliot@20c.com>
2020-08-03 14:51:11 -05:00
mcmanuss8
8e831f54df Add youtube link, update twitter icon (#783)
Add youtube link, update twitter icon, fixes #774

Co-authored-by: Steve McManus <smcmanus@fb.com>
2020-08-03 13:39:32 -05:00
Matt Griswold
abe3e78d50 Dotf fixes (#781)
* fix issue where ix-f import would raise suggestions ipaddresses not that ixlan (#764)

* IX-F Suggestions: Leaving the editor and returning to it via back button issues (#765)

* IX-F importer: Clicking "Preview" (IXP Update Tools) on /net/ page resulted in 170 ticket resolutions (#769)
More robust testing

* black formatting (was lost after pyupgrade)

* Add regex searching to deskpro ticket subjects

* Change operational error

* IX-F suggestions: consolidate delete+add (#770)

* Add reset functions to commandline tool

* Fix commandline tool bugs

* Fix reset commandline tool bugs

* add commandline tool

* Ixlan needs to be set for import commandline tool

* Add email model

* Add admin view to emails

* Allow network and ix to be null

* save emails as part of ixf import

* Add email model

* Add email delete

* add iregex search and better comments

* fix ixlan selection for import

* redefine migration dependencies for this branch

* only enable search w start and end char

* Add caption to regex search

* Remove delete all ixfmemberdata option

* [beta] IX-F importer: don't bother about missing IPv{4,6} address when network is not doing IPv{4,6} (#771)

* Add cmdline tests

* Resolve email conflicts

* Add cmd tool reset tests

* add autocomplete to commandline tool

* Fix email bugs

* Fix email migrations

* Fix typos

* [beta] IX-F importer: prevent Admin Committee overload by initially limiting importer to IXes enabled by AC (#772)

* Finalize regex search for emails and deskprotickets

* Fix keyword bug

* fix typo

* protocol-conflict will now be handled in the notification consolidation

771 changes where if the network indicates neither ipv4 nor ipv6 support, it is handled as supporting both (eg the network didnt configure these at all)

realised that the importer command re instantiates the `Importer` class for each ixlan it processes, so moved the sending of consolidated notifications (#772) out of the `update` function and into the command itself after its done processing all the ixlans. This means for tests you will need to call `importer.notify_proposals` after `importer.update` to test the consolidated notifications.

fixed several MultipleObjectsReturned errors when network switch protocol support in between imports

* should be checking for "ix" in the form data (#773)

* Fix cmd ixf tests

* fix issue in log_peer

* Add commit check for reset tool

* fix importer bugs

* remove dupe IXFImportEmail definition

* ixfimportemail support ix__name and net__name searching

* ticket resolution responses

* Add commit to command ixf import changes

* fix modify entry header

* remove whitespace in notification about remote data changes

* Begin updating tests

* ixf-import command line tool to queue

* refactor conflict inserts

* Update import protocol tests, including tests for 770

* More test edits

* Change cmd tests

* better ixfmemberdata error handling and fix some test data

* dont reset the same ixfmemberdata requirement

* fix many bugs add many tests

* remove debug message

* fix bug during import when consolidating delete+add

* fix perfomance issue in IXFMemberData listing

* dont show reset flags on prod env

* Add regex search tests

* Add 772 tests

* remove debug output

* fix `test_resolve_deskpro_ticket` test

* black formatting

* remove dupe import

* fix issue with unique constraint error handling

* add test for ixp / network ip protocol notification

* add missing test data

Co-authored-by: Stefan Pratter <stefan@20c.com>
Co-authored-by: Elliot Frank <elliot@20c.com>
2020-07-27 04:36:27 +00:00
Chris Caputo
f9dedb5274 typo: "ixp" -> "IXP" (#780) 2020-07-24 10:12:06 -05:00
Matt Griswold
0c36f1a5c4 fix edit net issue asn not set (#358) (#763)
Co-authored-by: Stefan Pratter <stefan@20c.com>
2020-07-15 08:35:16 +00:00