mirror/github-octodns
1
0
Fork 0
mirror of https://github.com/github/octodns.git synced 2024-05-11 05:55:00 +00:00
Code Issues Releases Activity

Merge pull request #1053 from octodns/examples

Browse Source 
First pass at examples directory, basic/getting started example
This commit is contained in:
Ross McFarland
2023-11-14 14:51:20 -08:00
committed by GitHub
parent dd32a2731c 29832ebc9c
commit ee3ec58ef5
17 changed files with 853 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -10,6 +10,7 @@
coverage.xml
dist/
env/
examples/migrating-to-octodns/config/
htmlcov/
nosetests.xml
octodns.egg-info/
examples/README.md
+35
View File
@@ -0,0 +1,35 @@
### This is all a WIP
This is a WIP and will definitely be rough around the edges, but in the spirit
of not letting perfect get in the way of good enough, or really existing at
all. It's being uploaded as a starting point. PRs/suggestions welcome as are
ideas for other subjects to cover.
### Examples
* Getting started with a [basic octoDNS configuration](basic/) - new to octoDNS
this is the place to start. It'll walk you through the main pieces of DNS IaC
with octoDNS including the process of planning and applying changes.
* [Migrating to octoDNS](migrating-to-octodns/) - have an existing DNS setup
you'd like to bring into octoDNS check this example out right after
[basic](basic/). It'll walk you through the steps of using `octodns-dump` to
pull the existing data out of your provider into matching YAML config files on
disk.
### Running PowerDNS
If you'd like to play around with running the examples in this directory
interactively you'll need a target for pushing data to.
[octodns-powerdns](https://github.com/octodns/octodns-powerdns) is the best
stand-alone option for this and the examples directory makes extensive use of
it. There is a [docker-compose.yml](docker-compose.yml) file that should get a
fully functional copy of PowerDNS backed my MySQL with the API enabled along
with other relivant functionality. For any of the examples that refer to the
local PowerDNS instance the following instructions below should get it up and
running.
1. If you haven't already [install docker compose](https://docs.docker.com/compose/install/)
1. If you don't already have a copy of octoDNS checked out run `git clone https://github.com/octodns/octodns.git`
1. In a seperate terminal window or tab
1. cd into the examples directory `cd octodns/examples`
1. Run docker-compose up `docker-compose up`, this will start up MySQL and PowerDNS running them in the foreground with their logs printing to the terminal
examples/basic/README.md
+254
View File
@@ -0,0 +1,254 @@
## Basic octoDNS Setup
This is the starting point octoDNS config, it's pretty similar to what you
might see for managing a set of personal domains or a small business.
Most of the actual documentation for this example is found in the comments in
the YAML configuration files.
* [config/octodns.yaml](config/octodns.yaml)
* [config/my-domain.com.yaml](config/my-domain.com.yaml)
* [config/unused-domain.io.yaml](config/unused-domain.io.yaml)
From here on this README focuses on the general process of running octoDNS.
## Checking out the code and setting up the environment
You would not normally need to check out octoDNS itself, you instead would have
a git repo with only your configuration files. Here we're cloning the repo only
to get a copy of the example files.
```console
$ git clone https://github.com/octodns/octodns.git
$ cd octodns/examples/basic/
$ python3 -mvenv env
$ source ../env.sh
$ source env/bin/activate
(env) $ pip install -r requirements.txt
```
Finally check out [Running PowerDNS](../README.md#running-powerdns) to get a local
instance of PowerDNS up and going before continuing.
## Running octoDNS the first time
Once you have your configuration files and octoDNS installed you're ready to
run the sync command to get it to plan an initial set of changes.
```console
(env) $ octodns-sync --config-file=config/octodns.yaml
2023-08-23T15:09:51 [4577488384] INFO Manager __init__: config_file=config/octodns.yaml, (octoDNS 1.0.1)
2023-08-23T15:09:51 [4577488384] INFO Manager _config_executor: max_workers=1
2023-08-23T15:09:51 [4577488384] INFO Manager _config_include_meta: include_meta=False
2023-08-23T15:09:51 [4577488384] INFO Manager _config_auto_arpa: auto_arpa=False
2023-08-23T15:09:51 [4577488384] INFO Manager __init__: global_processors=[]
2023-08-23T15:09:51 [4577488384] INFO Manager __init__: provider=config (octodns.provider.yaml 1.0.1)
2023-08-23T15:09:51 [4577488384] INFO SpfSource[no-mail] __init__: id=no-mail, a_records=[], mx_records=[], ip4_addresses=[], ip6_addresses=[], includes=[], exists=[], soft_fail=False, merging_enabled=False, ttl=3600
2023-08-23T15:09:51 [4577488384] INFO Manager __init__: provider=no-mail (octodns_spf 0.0.2)
2023-08-23T15:09:51 [4577488384] INFO Manager __init__: provider=yaml (octodns.provider.yaml 1.0.1)
2023-08-23T15:09:51 [4577488384] INFO Manager sync: eligible_zones=[], eligible_targets=[], dry_run=True, force=False, plan_output_fh=<stdout>
2023-08-23T15:09:51 [4577488384] INFO Manager sync: sources=['config', 'no-mail']
2023-08-23T15:09:51 [4577488384] INFO Manager sync: dynamic zone=*, sources=[YamlProvider, SpfSource]
2023-08-23T15:09:51 [4577488384] INFO Manager sync: adding dynamic zone=my-domain.com.
2023-08-23T15:09:51 [4577488384] INFO Manager sync: adding dynamic zone=unused-domain.io.
2023-08-23T15:09:51 [4577488384] INFO Manager sync: zone=my-domain.com.
2023-08-23T15:09:51 [4577488384] INFO Manager sync: sources=['config', 'no-mail']
2023-08-23T15:09:51 [4577488384] INFO Manager sync: targets=['yaml']
2023-08-23T15:09:51 [4577488384] INFO Manager sync: zone=unused-domain.io.
2023-08-23T15:09:51 [4577488384] INFO Manager sync: sources=['config', 'no-mail']
2023-08-23T15:09:51 [4577488384] INFO Manager sync: targets=['yaml']
2023-08-23T15:09:51 [4577488384] INFO YamlProvider[config] populate: found 9 records, exists=False
2023-08-23T15:09:51 [4577488384] INFO SpfSource[no-mail] populate: found 0 records, exists=False
2023-08-23T15:09:51 [4577488384] INFO PowerDnsProvider[powerdns] plan: desired=my-domain.com.
2023-08-23T15:09:51 [4577488384] WARNING PowerDnsProvider[powerdns] root NS record supported, but no record is configured for my-domain.com.
2023-08-23T15:09:51 [4577488384] INFO PowerDnsProvider[powerdns] plan: Creates=9, Updates=0, Deletes=0, Existing Records=0
2023-08-23T15:09:51 [4577488384] INFO YamlProvider[config] populate: found 0 records, exists=False
2023-08-23T15:09:51 [4577488384] INFO SpfSource[no-mail] populate: found 1 records, exists=False
2023-08-23T15:09:51 [4577488384] INFO PowerDnsProvider[powerdns] plan: desired=unused-domain.io.
2023-08-23T15:09:51 [4577488384] WARNING PowerDnsProvider[powerdns] root NS record supported, but no record is configured for unused-domain.io.
2023-08-23T15:09:51 [4577488384] INFO PowerDnsProvider[powerdns] plan: Creates=1, Updates=0, Deletes=0, Existing Records=0
2023-08-23T15:09:51 [4577488384] INFO Plan
********************************************************************************
* unused-domain.io.
********************************************************************************
* powerdns (PowerDnsProvider)
* Create Zone<unused-domain.io.>
* Create <TxtRecord TXT 3600, unused-domain.io., ['v=spf1 -all']> ()
* Summary: Creates=1, Updates=0, Deletes=0, Existing Records=0
********************************************************************************
* my-domain.com.
********************************************************************************
* powerdns (PowerDnsProvider)
* Create Zone<my-domain.com.>
* Create <ARecord A 3600, my-domain.com., ['203.0.113.42', '203.0.113.43']> (config)
* Create <AaaaRecord AAAA 3600, my-domain.com., ['2001:db8::44']> (config)
* Create <TxtRecord TXT 3600, my-domain.com., ['some-verification=3becb991-932f-4433-a280-9df6f39b6194', 'v=spf1 -all', 'z-other-thing=this proves i have control over this domain']> (config)
* Create <ARecord A 3600, *.my-domain.com., ['203.0.113.45']> (config)
* Create <AaaaRecord AAAA 3600, *.my-domain.com., ['2001:db8::46']> (config)
* Create <TxtRecord TXT 3600, nadcbiqkbgq._companyname.my-domain.com., ['a-different-proof-of-ownership']> (config)
* Create <CnameRecord CNAME 3600, pointer.my-domain.com., look.over-here.net.> (config)
* Create <ARecord A 3600, www.my-domain.com., ['203.0.113.42', '203.0.113.43']> (config)
* Create <AaaaRecord AAAA 3600, www.my-domain.com., ['2001:db8::44']> (config)
* Summary: Creates=9, Updates=0, Deletes=0, Existing Records=0
********************************************************************************
```
### The log output
It's always a good idea to scan over the logging output of an octoDNS run. Most
of it is informational, telling you what has been configured and providing a
big picture idea of what's happening while planning. You will sometimes see
WARNINGS, these are generally telling you that there's something you should
think about or look into.
In the run above there was a warning about root/APEX NS records being supported
by the provider, YamlProvider, and not present in the config. Not all providers
support managing the root NS records, some just hard code their own name
servers.
```console
2023-08-23T15:09:51 [4577488384] WARNING PowerDnsProvider[powerdns] root NS record supported, but no record is configured for my-domain.com.
```
### The plan output
```console
********************************************************************************
* unused-domain.io.
********************************************************************************
* powerdns (PowerDnsProvider)
* Create Zone<unused-domain.io.>
* Create <TxtRecord TXT 3600, unused-domain.io., ['v=spf1 -all']> ()
* Summary: Creates=1, Updates=0, Deletes=0, Existing Records=0
********************************************************************************
* my-domain.com.
********************************************************************************
* powerdns (PowerDnsProvider)
* Create Zone<my-domain.com.>
* Create <ARecord A 3600, my-domain.com., ['203.0.113.42', '203.0.113.43']> (config)
* Create <AaaaRecord AAAA 3600, my-domain.com., ['2001:db8::44']> (config)
* Create <TxtRecord TXT 3600, my-domain.com., ['some-verification=3becb991-932f-4433-a280-9df6f39b6194', 'v=spf1 -all', 'z-other-thing=this proves i have control over this domain']> (config)
* Create <ARecord A 3600, *.my-domain.com., ['203.0.113.45']> (config)
* Create <AaaaRecord AAAA 3600, *.my-domain.com., ['2001:db8::46']> (config)
* Create <TxtRecord TXT 3600, nadcbiqkbgq._companyname.my-domain.com., ['a-different-proof-of-ownership']> (config)
* Create <CnameRecord CNAME 3600, pointer.my-domain.com., look.over-here.net.> (config)
* Create <ARecord A 3600, www.my-domain.com., ['203.0.113.42', '203.0.113.43']> (config)
* Create <AaaaRecord AAAA 3600, www.my-domain.com., ['2001:db8::44']> (config)
* Summary: Creates=9, Updates=0, Deletes=0, Existing Records=0
********************************************************************************
```
### Applying the changes
```console
(env) $ octodns-sync --config-file=config/octodns.yaml --doit
...
********************************************************************************
* unused-domain.io.
********************************************************************************
* powerdns (PowerDnsProvider)
* Create Zone<unused-domain.io.>
* Create <TxtRecord TXT 3600, unused-domain.io., ['v=spf1 -all']> ()
* Summary: Creates=1, Updates=0, Deletes=0, Existing Records=0
********************************************************************************
* my-domain.com.
********************************************************************************
* powerdns (PowerDnsProvider)
* Create Zone<my-domain.com.>
* Create <ARecord A 3600, my-domain.com., ['203.0.113.42', '203.0.113.43']> (config)
* Create <AaaaRecord AAAA 3600, my-domain.com., ['2001:db8::44']> (config)
* Create <TxtRecord TXT 3600, my-domain.com., ['some-verification=3becb991-932f-4433-a280-9df6f39b6194', 'v=spf1 -all', 'z-other-thing=this proves i have control over this domain']> (config)
* Create <ARecord A 3600, *.my-domain.com., ['203.0.113.45']> (config)
* Create <AaaaRecord AAAA 3600, *.my-domain.com., ['2001:db8::46']> (config)
* Create <TxtRecord TXT 3600, nadcbiqkbgq._companyname.my-domain.com., ['a-different-proof-of-ownership']> (config)
* Create <CnameRecord CNAME 3600, pointer.my-domain.com., look.over-here.net.> (config)
* Create <ARecord A 3600, www.my-domain.com., ['203.0.113.42', '203.0.113.43']> (config)
* Create <AaaaRecord AAAA 3600, www.my-domain.com., ['2001:db8::44']> (config)
* Summary: Creates=9, Updates=0, Deletes=0, Existing Records=0
********************************************************************************
2023-08-23T15:17:00 [4671815168] INFO PowerDnsProvider[powerdns] apply: making 1 changes to unused-domain.io.
2023-08-23T15:17:00 [4671815168] INFO PowerDnsProvider[powerdns] apply: making 9 changes to my-domain.com.
2023-08-23T15:17:00 [4671815168] INFO Manager sync: 10 total changes
```
```console
(env) $ octodns-sync --config-file=config/octodns.yaml
********************************************************************************
No changes were planned
********************************************************************************
```
That's it. You're now managing your DNS with octoDNS.
## Making a change
At some point down the road you need to make a change, maybe one of your web
boxes was replaced and there's a new IP address. The first step is to open up
the zone's YAML file and modify the YAML, removing 203.0.113.42 and adding
203.0.113.44.
## Running octoDNS to see the plan
Just like during the first run section above we'll first run octoDNS to see what changes it would make.
We'd skim the log lines again looking for unexpected WARNINGS and then take a look at the changes.
```console
(env) $ octodns-sync --config-file=config/octodns.yaml
...
********************************************************************************
* my-domain.com.
********************************************************************************
* powerdns (PowerDnsProvider)
* Create Zone<my-domain.com.>
* Update
* <ARecord A 3600, my-domain.com., ['203.0.113.42', '203.0.113.43']> ->
* <ARecord A 3600, my-domain.com., ['203.0.113.43', '203.0.113.44']> (config)
* Update
* <ARecord A 3600, www.my-domain.com., ['203.0.113.42', '203.0.113.43']> ->
* <ARecord A 3600, www.my-domain.com., ['203.0.113.43', '203.0.113.44']> (config)
* Summary: Creates=0, Updates=2, Deletes=0, Existing Records=9
********************************************************************************
```
Since we have used YAML anchors to share the values across both the root and
www A's we see that octoDNS will be making changes to both those records.
If there were unexpected things here we'd need to investigate what's changed.
Maybe there were other alterations in the YAML that hadn't been applied yet or
someone modified the records through another means.
Here we only see the expected changes so we're good to move forward with
applying them.
```console
(env) $ octodns-sync --config-file=config/octodns.yaml --doit
...
********************************************************************************
* my-domain.com.
********************************************************************************
* powerdns (PowerDnsProvider)
* Create Zone<my-domain.com.>
* Update
* <ARecord A 3600, my-domain.com., ['203.0.113.42', '203.0.113.43']> ->
* <ARecord A 3600, my-domain.com., ['203.0.113.43', '203.0.113.44']> (config)
* Update
* <ARecord A 3600, www.my-domain.com., ['203.0.113.42', '203.0.113.43']> ->
* <ARecord A 3600, www.my-domain.com., ['203.0.113.43', '203.0.113.44']> (config)
* Summary: Creates=0, Updates=2, Deletes=0, Existing Records=9
********************************************************************************
```
If we want we can run another plan to make sure there are no further pending changes.
```console
(env) $ octodns-sync --config-file=config/octodns.yaml
********************************************************************************
No changes were planned
********************************************************************************
```
## What's Next
* Check out [migrating to octoDNS](../migrating-to-octodns) for an example of how to create zone configuration YAML files from your existing provider's configuration
* For a complete list check out the [Examples Directory](../)
examples/basic/config/my-domain.com.yaml
+58
View File
@@ -0,0 +1,58 @@
---
# These are root/APEX records, things that live at the top-level of the zone.
# There are multiple records so there's a YAML list, each element in that list
# defines its own record.
'':
# We first have an A record, mapping A my-domain.com to two IPv4 addresses, we
# also use a YAML anchor to effectively store the values for later use.
- type: A
values: &WEB_A_VALUES
- 203.0.113.42
- 203.0.113.43
# Similar to the A above, this is a AAAA for my-domain.com with a single IPv6
# addresse.
- type: AAAA
values: &WEB_AAAA_VALUES
- 2001:DB8::44
# Finally it's common to have to prove ownership of a domain with a TXT value
# in an APEX record. The following are a couple made up examples of how that
# would be done. As you'll see when you plan things The SpfSource will merge
# its SPF value in with the values defined here
- type: TXT
values:
- some-verification=3becb991-932f-4433-a280-9df6f39b6194
- z-other-thing=this proves i have control over this domain
# This is a wildcard record, any A or AAAA query not matching a more specific
# answer defined in this file will match this name and receive the associated
# value in response.
'*':
- type: A
# Note here we have a single value rather than values and use `value`.
value: 203.0.113.45
- type: AAAA
value: 2001:DB8::46
# Note that the records in this zone are sorted, by default YamlProvider
# enforces alphabetical sorting of records and even the keys within record
# data. This is an organizational best practice, but can be disabled with the
# enforce_order parameter to YamlProvider
NADCBiQKBgQ._companyname:
type: TXT
value: a-different-proof-of-ownership
# We want www and the APEX to return the same values for A/AAAA queries. We
# stored the values earlier in YAML anchors so we can use them now on.
# This is a CNAME record
pointer:
type: CNAME
# CNAMEs can only have a single value so they require `value` to be used. As a
# best practice the target name must end with a `.` If you have a specific
# case where you need to omit the `.`, or other enforced best practices, see
# the lenience example for more information.
value: look.over-here.net.
# This is an example of a record with a `.` in it's name, it would result for a
# TXT query of NADCBiQKBgQ._companyname.my-domain.com. This specific example is
# another commonly required proof of overship record.
www:
- type: A
values: *WEB_A_VALUES
- type: AAAA
values: *WEB_AAAA_VALUES
examples/basic/config/octodns.yaml
+58
View File
@@ -0,0 +1,58 @@
---
providers:
# The primary source of record data in most setup is YAML files on disk. By
# convention it's named config.
config:
class: octodns.provider.yaml.YamlProvider
# There are a number of configuration options available to customize the
# behavior of the provider, but the defaults are generally suitable and the
# only required value is the name of the directory where the files can be
# found. Convention is the same config directory in which this file lives.
directory: ./config
# Having a no mail SPF record is considered best practice for any domains
# that you do not plan to use for email. The octodns-spf module provides
# SpfSource which by default adds the SPF value `v=spf1 -all` to accomplish
# this. If you have one or more domains that involve mail check out the email
# example after you're done here.
no-mail:
class: octodns_spf.SpfSource
# PowerDNS serves as a good option for a locally runnable standalone provider
# to use in our examples. See the top-level example README for information on
# how to run it and the documentation for octodns-powerdns for documentation
# about the details of the config below.
powerdns:
class: octodns_powerdns.PowerDnsProvider
host: 127.0.0.1
port: 8081
api_key: env/POWERDNS_API_KEY
timeout: 10
# If you wish to push your record data to multiple providers you'd define the
# other here and include them in the targets list down below. See the
# multiple-provider example for more details.
zones:
# This configuration is using dynamic zone config. Rather than having to
# explicitly list out all of the zones here with their coresponding
# configuration there's a single wildcard entry. Here the `config` source
# will look for zone files defined in its configured directory, the files
# alongside this one. For more details see the dynamic-zone-config example.
'*':
# This is the place(s) we look for the records in each zone
sources:
# The conig provider will be firs in line to provide records for the
# zones, and since we're using dynamic zone config it will also be
# responsible for defining what zones exist.
- config
# After config has added its records the we'll add an no email SPF value.
# It'll create the APEX TXT record if necessary. Again see the email
# example for more details.
- no-mail
# This is the place(s) we push the record data to when applying changes
targets:
# Here we're pushing things to our local PowerDNS instance
- powerdns
examples/basic/config/unused-domain.io.yaml
+2
View File
@@ -0,0 +1,2 @@
---
# Nothing in here, just an empty domain b/c it's not actually used for anything
examples/basic/requirements.txt
+3
View File
@@ -0,0 +1,3 @@
octodns>=1.0.1
octodns_powerdns>=0.0.5
octodns_spf>=0.0.2
examples/basic/target/my-domain.com.yaml
+31
View File
@@ -0,0 +1,31 @@
---
? ''
: - type: A
values:
- 203.0.113.42
- 203.0.113.43
- type: AAAA
value: 2001:db8::44
- type: TXT
values:
- some-verification=3becb991-932f-4433-a280-9df6f39b6194
- v=spf1 -all
- z-other-thing=this proves i have control over this domain
'*':
- type: A
value: 203.0.113.45
- type: AAAA
value: 2001:db8::46
nadcbiqkbgq._companyname:
type: TXT
value: a-different-proof-of-ownership
pointer:
type: CNAME
value: look.over-here.net.
www:
- type: A
values:
- 203.0.113.42
- 203.0.113.43
- type: AAAA
value: 2001:db8::44
examples/basic/target/unused-domain.io.yaml
+4
View File
@@ -0,0 +1,4 @@
---
? ''
: type: TXT
value: v=spf1 -all
examples/docker-compose.yml
+29
View File
@@ -0,0 +1,29 @@
version: "3.9" # optional since v1.27.0
volumes:
powerdns-db-data:
name: powerdns-db-data
services:
db:
image: mariadb:10.1
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: l3tmein
volumes:
- powerdns-db-data:/var/lib/mysql
powerdns:
image: psitrax/powerdns
ports:
- "53:53/tcp"
- "53:53/udp"
- "8081:8081"
environment:
MYSQL_HOST: db
MYSQL_PORT: 3306
MYSQL_USER: root
MYSQL_PASS: l3tmein
depends_on:
- db
command: --api=yes --api-key=its@secret --loglevel=99 --webserver=yes --webserver-address=0.0.0.0 --webserver-allow-from=0.0.0.0/0 --webserver-password=its@secret --webserver-port=8081 --enable-lua-records=shared --edns-subnet-processing=yes
examples/env.sh
+1
View File
@@ -0,0 +1 @@
export POWERDNS_API_KEY="its@secret"
examples/migrating-to-octodns/README.md
+271
View File
@@ -0,0 +1,271 @@
## Migrating to octoDNS via octodns-dump
Importing an existing DNS setup into octoDNS management is a very
straightforward process and can generally be completed in minutes.
Some relevant documentation for this example is in comments in the YAML
configuration files.
* [config/octodns.yaml](config/octodns.yaml)
* [populate/octodns.yaml](populate/octodns.yaml)
* [populate/my-dumpable.com.yaml](populate/my-dumpable.com.yaml)
* [populate/unused-dumpable.com.yaml](populate/unused-dumpable.com.yaml)
From here on this README focuses on the process of using `octodns-dump` to
import your existing DNS data into octoDNS.
## Checking out the code and setting up the environment
You would not normally need to check out octoDNS itself, you instead would have
a git repo with only your configuration files. Here we're cloning the repo only
to get a copy of the example files.
```console
$ git clone https://github.com/octodns/octodns.git
$ cd octodns/examples/basic/
$ python3 -mvenv env
$ source ../env.sh
$ source env/bin/activate
(env) $ pip install -r requirements.txt
```
If you were instead creating a repo for your config from scratch it might look
something like:
```console
$ mkdir octodnsed
$ cd octodnsed
$ git init -b main
$ mkdir config
# Using your editor of choice create the main config file with your customized
# version of things, examples/basic/config/octodns.yaml is a good place to
# start.
$ vim config/octodns.yaml
# follow the process here and once complete
$ git add config
$ git commit -m "Importting existing records into octoDNS"
$ git push -u origin main
```
## Populating some data that we can later dump
We need zone & record data in our provider in order to have something to dump
out into a YAML config, thus our first step is to populate PowerDNS. This is
just to have something to work with the the actual process that begins in the
next step. It's not something you'd normally do when migrating to octoDNS.
Step 0 is to get a local PowerDNS instance running.
Check out out [Running PowerDNS](../README.md#running-powerdns) for info on
starting that up. Once you've done that run the following. You can ignore the
output and move on to the next step.
```console
(env) $ octodns-sync --config-file=populate/octodns.yaml --doit
```
## Running octodns-dump
Once you have your configuration files and octoDNS installed you're ready to
dump your zone configs. Here we've assumed that the provider being used supports
`list_zones`, not all do. If you get an error to that effect see
[dynamic-zone-config](../dynamic-zone-config) for details on how to explicitly
list your zones in the config file.
We first tell octodns-dump where to find our config file. We then tell it that
we want to use the output provider defined in that file named `config` rather
than having a default one auto-created. This would allow us to customize the
details of the provider, e.g. things like the `default_ttl`.
We'll then tell dump where we would like it to write our zone files, in this
case we want it to put them into `config` provider's directory `./config`.
The quoted * tells octodns-dump to dynamically source the list of zones and
dump config files for all of them. If you would only like to dump a specific
zone you can replace it with the zone name, e.g. `my-domain.com.`, making sure
to include the trailing `.`
The final argument passed to `octodns-dump` is the source of data you'd like to
dump. Here that's our existing provider `powerdns`.
Note that if you're previously run other examples you may see entries in the
log output for those zones in addition to the ones we populated for this
exercise. They shouldn't cause any problems and can be ignored.
```console
(env) $ octodns-dump --config-file=config/octodns.yaml --output-provider config --output-dir ./config '*' powerdns
2023-09-12T09:39:59 [4580544000] INFO Manager __init__: config_file=config/octodns.yaml, (octoDNS 1.0.0+bca53089)
2023-09-12T09:39:59 [4580544000] INFO Manager _config_executor: max_workers=1
2023-09-12T09:39:59 [4580544000] INFO Manager _config_include_meta: include_meta=False
2023-09-12T09:39:59 [4580544000] INFO Manager _config_auto_arpa: auto_arpa=False
2023-09-12T09:39:59 [4580544000] INFO Manager __init__: global_processors=[]
2023-09-12T09:39:59 [4580544000] INFO Manager __init__: provider=config (octodns.provider.yaml 1.0.0+bca53089)
2023-09-12T09:40:00 [4580544000] INFO Manager __init__: provider=powerdns (octodns_powerdns 0.0.4+3189e9c2)
2023-09-12T09:40:00 [4580544000] INFO Manager dump: zone=*, output_dir=./config, output_provider=config, lenient=False, split=False, sources=['powerdns']
2023-09-12T09:40:00 [4580544000] INFO Manager dump: using specified output_provider=config
2023-09-12T09:40:00 [4580544000] INFO Manager sync: dynamic zone=*, sources=[PowerDnsProvider]
2023-09-12T09:40:00 [4580544000] INFO Manager sync: adding dynamic zone=my-dumpable.com.
2023-09-12T09:40:00 [4580544000] INFO Manager sync: adding dynamic zone=unused-domain.io.
Traceback (most recent call last):
File "/Users/ross/octodns/octodns/examples/migrating-to-octodns/env/bin/octodns-dump", line 8, in <module>
sys.exit(main())
^^^^^^
File "/Users/ross/octodns/octodns/examples/migrating-to-octodns/env/lib/python3.11/site-packages/octodns/cmds/dump.py", line 51, in main
manager.dump(
File "/Users/ross/octodns/octodns/examples/migrating-to-octodns/env/lib/python3.11/site-packages/octodns/manager.py", line 868, in dump
source.populate(zone, lenient=lenient)
File "/Users/ross/octodns/octodns/examples/migrating-to-octodns/env/lib/python3.11/site-packages/octodns_powerdns/__init__.py", line 433, in populate
record = Record.new(
^^^^^^^^^^^
File "/Users/ross/octodns/octodns/examples/migrating-to-octodns/env/lib/python3.11/site-packages/octodns/record/base.py", line 76, in new
raise ValidationError(fqdn, reasons, context)
octodns.record.exception.ValidationError: Invalid record "sshfp.my-dumpable.com."
- unrecognized algorithm "42"
- unrecognized algorithm "43"
```
We've intentionally configured data in the provider without following
(octoDNS's) best practices as it's common to run across these sorts of things
when migrating. In this case there is a SSHFP record with invalid `algorithm`s.
Regardless of whether or not you hit errors it is important to carefully look
over the output of octodns-dump to ensure that it was able to make sense of and
import everything it found. It usually can, but you may sometimes run across
things that it can't make sense of or otherwise doesn't support.
This is especially true if you have any advanced records configured in your
provider. octoDNS generally cannot convert records with functionality like
weights and/or geo-coding enabled. It'll generally import a "simple" version of
that record and indicate it's doing so with a WARNING log message. If you hit
this situation see [Dynamic Records](/docs/dynamic_records.md).
## Running octodns-dump again, now with --lenient
Back to our example, the invalid `algorithm`s aren't a huge deal and octoDNS
can be asked to ignore them with the `--lenient` argument.
```console
(env) $ octodns-dump --config-file=config/octodns.yaml --output-provider config --output-dir ./config '*' powerdns --lenient
2023-09-12T09:47:48 [4566433280] INFO Manager __init__: config_file=config/octodns.yaml, (octoDNS 1.0.0+bca53089)
2023-09-12T09:47:48 [4566433280] INFO Manager _config_executor: max_workers=1
2023-09-12T09:47:48 [4566433280] INFO Manager _config_include_meta: include_meta=False
2023-09-12T09:47:48 [4566433280] INFO Manager _config_auto_arpa: auto_arpa=False
2023-09-12T09:47:48 [4566433280] INFO Manager __init__: global_processors=[]
2023-09-12T09:47:48 [4566433280] INFO Manager __init__: provider=config (octodns.provider.yaml 1.0.0+bca53089)
2023-09-12T09:47:48 [4566433280] INFO Manager __init__: provider=powerdns (octodns_powerdns 0.0.4+3189e9c2)
2023-09-12T09:47:48 [4566433280] INFO Manager dump: zone=*, output_dir=./config, output_provider=config, lenient=True, split=False, sources=['powerdns']
2023-09-12T09:47:48 [4566433280] INFO Manager dump: using specified output_provider=config
2023-09-12T09:47:48 [4566433280] INFO Manager sync: dynamic zone=*, sources=[PowerDnsProvider]
2023-09-12T09:47:48 [4566433280] INFO Manager sync: adding dynamic zone=my-dumpable.com.
2023-09-12T09:47:48 [4566433280] INFO Manager sync: adding dynamic zone=unused-dumpable.com.
2023-09-12T09:47:48 [4566433280] WARNING Record Invalid record "sshfp.my-dumpable.com."
- unrecognized algorithm "42"
- unrecognized algorithm "43"
2023-09-12T09:47:48 [4566433280] INFO PowerDnsProvider[powerdns] populate: found 8 records, exists=True
2023-09-12T09:47:48 [4566433280] INFO YamlProvider[config] plan: desired=my-dumpable.com.
2023-09-12T09:47:48 [4566433280] INFO YamlProvider[config] plan: Creates=8, Updates=0, Deletes=0, Existing Records=0
2023-09-12T09:47:48 [4566433280] INFO YamlProvider[config] apply: making 8 changes to my-dumpable.com.
2023-09-12T09:47:48 [4566433280] INFO PowerDnsProvider[powerdns] populate: found 1 records, exists=True
2023-09-12T09:47:49 [4566433280] INFO YamlProvider[config] plan: desired=unused-dumpable.com.
2023-09-12T09:47:49 [4566433280] WARNING YamlProvider[config] root NS record supported, but no record is configured for unused-dumpable.com.
2023-09-12T09:47:49 [4566433280] INFO YamlProvider[config] plan: Creates=1, Updates=0, Deletes=0, Existing Records=0
2023-09-12T09:47:49 [4566433280] INFO YamlProvider[config] apply: making 1 changes to unused-dumpable.com.
```
With `--lenient` we now see a warning about the validation problems, but
octoDNS does its best to continue anyway.
## Examining the results
We can now take a look in the config/ directory to see the created zone files
along side our main config.
```console
(env) ls -1 config/
my-dumpable.com.yaml
octodns.yaml
unused-dumpable.com.yaml
```
Again note if you're run through other examples you may see more files for
their zones in this directory. They can be ignored for the purpose of the
example, but they will be imported and managed nonetheless.
If you open up the my-dumpable.com.yaml file you'll note that the SSHFP
record's values have 42 and 43 for their algorithm field.
With those values if you tried to generate a plan with `octodns-sync` now it
would fail with validation errors.
When migrating it's recommended to make as few changes to your config as
possible initially, that is you want to bring the setup as-is under octoDNS
management. This is the safest approach, change as little as possible and
incrementally work towards having a fully managed and compliant/best practice
config.
So for now we'll enable `lenient` on the SSHFP record by editing
my-dumpable.com.yaml adding `octodns.lenient = true` as shown below. For more
details on see [lenience](/docs/records.md#lenience).
```yaml
...
sshpf:
# TODO: look into bringing this record into compliance
octodns:
lenient: true
type: SSHFP
values:
- algorithm: 42
fingerprint: abcdef1234567890
fingerprint_type: 1
- algorithm: 43
fingerprint: abcdef1234567890
fingerprint_type: 1
...
```
## Generating our first plan
We can now ask octoDNS to create a plan to see if anything would change. Since
we've looked closely at the log output zone configuration files we have a pretty
good idea the answer is no.
```console
(env) $ octodns-sync --config-file=config/octodns.yaml
...
2023-09-12T10:04:27 [4608876032] INFO PowerDnsProvider[powerdns] plan: desired=unused-dumpable.com.
2023-09-12T10:04:27 [4608876032] INFO PowerDnsProvider[powerdns] populate: found 1 records, exists=True
2023-09-12T10:04:27 [4608876032] WARNING PowerDnsProvider[powerdns] root NS record supported, but no record is configured for unused-dumpable.com.
2023-09-12T10:04:27 [4608876032] INFO PowerDnsProvider[powerdns] plan: No changes
2023-09-12T10:04:27 [4608876032] INFO YamlProvider[config] populate: found 5 records, exists=False
2023-09-12T10:04:27 [4608876032] INFO PowerDnsProvider[powerdns] plan: desired=exxampled.com.
2023-09-12T10:04:27 [4608876032] INFO PowerDnsProvider[powerdns] populate: found 5 records, exists=True
2023-09-12T10:04:27 [4608876032] INFO PowerDnsProvider[powerdns] plan: No changes
2023-09-12T10:04:27 [4608876032] INFO Plan
********************************************************************************
No changes were planned
********************************************************************************
```
Most of the output above is omitted, this is the final few lines showing that
no changes need to be made to get `powerdns` to match `config`. It does have a
warning about the lack of root NS records in `unused-dumpable.com.`, but we'll
ignore that in this example.
If you had unsupported record types or advanced features in use this may not be
the case and you may see changes. It's completely safe to generate a plan,
octoDNS won't make any of the changes listed.
If you see things in the plan output section it's time to triple check and make
sure they're, hopefully minimal, modifications your OK with making.
If the changes are due to advanced functionality you'll need to step back and
plan a careful migration over to [Dynamic Records](/docs/dynamic_records.md)
which is beyond the scope of this example.
## What's Next
So now you can commit your config and start managing you DNS with octoDNS rather
than clicking buttons in UIs or using whatever you previous had used.
* Check out [octoDNS basic example](../basic) for an example of how to create zone configuration YAML files from your existing provider's configuration
* For a complete list check out the [Examples Directory](../)
examples/migrating-to-octodns/config/octodns.yaml
+31
View File
@@ -0,0 +1,31 @@
---
providers:
# See the basic example for descriptions of the basics of this config file,
# here we'll focus on things related to octodns-dump.
# This is our config provider, it sources YAML from disk, but to start we
# don't have any files in the ./config directory. We need to dump what's in
# our provider, in this case a local PowerDNS setup, into this config
# directory as a starting point.
config:
class: octodns.provider.yaml.YamlProvider
directory: ./config
# This is our existing provider, the one we want to bring under octoDNS
# management. This example will walk through dumping the existing data in
# this provider into the config provider's directory above. At that point we
# should be ready to start managing things with octoDNS going forward.
powerdns:
class: octodns_powerdns.PowerDnsProvider
host: 127.0.0.1
port: 8081
api_key: env/POWERDNS_API_KEY
timeout: 10
zones:
'*':
sources:
- config
targets:
- powerdns
examples/migrating-to-octodns/populate/my-dumpable.com.yaml
+41
View File
@@ -0,0 +1,41 @@
---
# Just some records so we'll have something to dump, see the basic example for
# documentation/descriptions of what's in here.
'':
- type: A
values:
- 203.0.113.42
- 203.0.113.43
- type: AAAA
values: &WEB_AAAA_VALUES
- 2001:DB8::44
- type: NS
values:
- ns1.some-provider.com.
- ns2.some-provider.com.
- ns3.some-provider.com.
- ns4.some-provider.com.
- type: TXT
values:
- some-verification=3becb991-932f-4433-a280-9df6f39b6194
- z-other-thing=this proves i have control over this domain
'*':
- type: A
value: 203.0.113.45
- type: AAAA
value: 2001:DB8::46
sshfp:
type: SSHFP
values:
# These values won't pass validation. They'll be used to walk through the
# process of safely bringing an existing config up to octoDNS's
# recommended best practices.
- algorithm: 42
fingerprint: abcdef1234567890
fingerprint_type: 1
- algorithm: 43
fingerprint: abcdef1234567890
fingerprint_type: 1
www:
- type: CNAME
value: my-dumpable.com.
examples/migrating-to-octodns/populate/octodns.yaml
+26
View File
@@ -0,0 +1,26 @@
---
providers:
# This is just a quick and easy way to load some data into PowerDNS that we
# can dump while working through the example. See the basic example for
# documentation of what's in here
config:
class: octodns.provider.yaml.YamlProvider
directory: ./populate
powerdns:
class: octodns_powerdns.PowerDnsProvider
host: 127.0.0.1
port: 8081
api_key: env/POWERDNS_API_KEY
timeout: 10
zones:
'*':
# we're disabling strict validation here so we can load up data that won't
# meet octoDNS's best practices so that we can show
lenient: true
sources:
- config
targets:
- powerdns
examples/migrating-to-octodns/populate/unused-dumpable.com.yaml
+5
View File
@@ -0,0 +1,5 @@
---
'':
type: TXT
values:
- v=spf1 -all
examples/migrating-to-octodns/requirements.txt
+3
View File
@@ -0,0 +1,3 @@
octodns>=1.0.1
octodns_powerdns>=0.0.5
octodns_spf>=0.0.2