Bulk record generation in aws route53 hosted zone with terraform

Question

pretty new to the Terraform world, wanna understand what's the best way to import the dns zone data in bulk to aws_route53 via terraform.

I'm essentially looking to see how to replicate Import Zone File functionality in route53 via Terraform

for instance, assuming the zone data is a pretty big file generated by dig axfr ... like this :


; <<>> DiG 9.10.6 <<>> axfr zonetransfer.me @nsztm1.digi.ninja.
;; global options: +cmd
zonetransfer.me.    7200    IN  SOA nsztm1.digi.ninja. robin.digi.ninja. 2019100801 172800 900 1209600 3600
zonetransfer.me.    301 IN  TXT "google-site-verification=tyP28J7JAUHA9fw2sHXMgcCC0I6XBmmoVi04VlMewxA"
zonetransfer.me.    7200    IN  MX  0 ASPMX.L.GOOGLE.COM.
zonetransfer.me.    7200    IN  MX  10 ALT1.ASPMX.L.GOOGLE.COM.
zonetransfer.me.    7200    IN  MX  10 ALT2.ASPMX.L.GOOGLE.COM.
zonetransfer.me.    7200    IN  MX  20 ASPMX2.GOOGLEMAIL.COM.
zonetransfer.me.    7200    IN  MX  20 ASPMX3.GOOGLEMAIL.COM.
zonetransfer.me.    7200    IN  MX  20 ASPMX4.GOOGLEMAIL.COM.
zonetransfer.me.    7200    IN  MX  20 ASPMX5.GOOGLEMAIL.COM.
zonetransfer.me.    7200    IN  A   5.196.105.14
zonetransfer.me.    7200    IN  NS  nsztm1.digi.ninja.
zonetransfer.me.    7200    IN  NS  nsztm2.digi.ninja.
_acme-challenge.zonetransfer.me. 301 IN TXT "6Oa05hbUJ9xSsvYy7pApQvwCUSSGgxvrbdizjePEsZI"
... (and much more)

It looks pretty painful/error-prone to create these resources one-by-by in a config file like:

//main.tf
resource "aws_route53_zone" "example" {
  name = "zonetransfer.me."
}

resource "aws_route53_record" "A" {
    zone_id = aws_route53_zone.example.zone_id
    name = aws_route53_zone.example.name
    type = "A"
    ttl = "7200"
    records = ["5.196.105.14"]
}

resource "aws_route53_record" "MX" {
 //
}

// all records here

Ideally I'd like to go with something like :

//main.tf
resource "aws_route53_zone" "example" {
  name = "zonetransfer.me."
}

resource "aws_route53_record" "all_records" {
    file = "<path_to_zone_data_file.txt>"
}

Any suggestions/pointers on how to handle this would be appreciated

You can't do that in Terraform. You'll have to use your first option. You could look at tools like github.com/dtan4/terraforming or github.com/GoogleCloudPlatform/terraformer to mass import a lot of existing resources. — ydaetskcoR

mariux mariux · Accepted Answer · 2020-04-25T01:16:35

If you can export it to JSON you could transform it (e.g. using jq or other tools) to get a list of records that you could transform using tools like json2hcl and feed the result into a terraform module that takes records as a list - like mineiros-io/route53/aws.

module "repository" {
  source  = "mineiros-io/route53/aws"
  version = "0.1.0"

  name = "zonetransfer.me"

  records = [
    {
      name    = "www"
      type    = "A"
      records = ["127.0.0.1"]
    },{
      ....
    },
    ....
  ]
}

I did a migration of multiple domains with hundreds of records each from Digital Ocean to AWS that way; involving only minimal manual work. As this was a one-time task I do not have the exact conversion command lines in place anymore.

the tricky part was to group records by type and name (e.g. for MX records) using jq's group_by() to result in the following:

    {
      name = ""
      type = "MX"
      records = [
        "1 aspmx.l.google.com",
        "10 aspmx2.googlemail.com",
        "10 aspmx3.googlemail.com",
        "5 alt1.aspmx.l.google.com",
        "5 alt2.aspmx.l.google.com"
      ]
    },

Bulk record generation in aws route53 hosted zone with terraform

1 Answers