Missing DNS validation record when using terraform aws_acm_certificate_validation

Question

I have been stuck on a Terraform error for a whole day whilst trying to create an AWS Route53 resource and an AWS Certificate Manager resource. These 2 bits are part of a wider project (a web site hosted in s3 through its static serving feature).

Specifically the error pops up during the DNS validation of the certificate, when the CNAMEs record are inserted as DNS record in Route53.

I'll lay out the error and then I'll describe the setup.

Error

terraform plan -var-file=production.vars

Creating...
module.infrastructure.aws_route53_record.idarth-validation-record: Still creating... [10s elapsed]
module.infrastructure.aws_route53_record.idarth-validation-record: Still creating... [20s elapsed]
module.infrastructure.aws_route53_record.idarth-validation-record: Still creating... [30s elapsed]
module.infrastructure.aws_route53_record.idarth-validation-record: Still creating... [40s elapsed]
module.infrastructure.aws_route53_record.idarth-validation-record: Still creating... [50s elapsed]
module.infrastructure.aws_route53_record.idarth-validation-record: Still creating... [1m0s elapsed]
module.infrastructure.aws_route53_record.idarth-validation-record: Still creating... [1m10s elapsed]
module.infrastructure.aws_route53_record.idarth-validation-record: Creation complete after 1m12s [id=ZB4TSGZTTZ3CQ__7bc5230529c8192e8e697aeab0ec0eb9.idarth.com._CNAME]
module.infrastructure.aws_acm_certificate_validation.idarth-ssl-certificate: Creating...
2019/08/24 18:32:40 [ERROR] module.infrastructure: eval: *terraform.EvalSequence, err: 1 error occurred:
    * missing www.idarth.com DNS validation record: _18ff46dac48c6d852b696306dfa57093.www.idarth.com

2019/08/24 18:32:40 [TRACE] [walkApply] Exiting eval tree: module.infrastructure.aws_acm_certificate_validation.idarth-ssl-certificate

Error: 1 error occurred:
    * missing www.idarth.com DNS validation record: _18ff46dac48c6d852b696306dfa57093.www.idarth.com



  on ../modules/route53.tf line 14, in resource "aws_acm_certificate_validation" "idarth-ssl-certificate":
  14: resource "aws_acm_certificate_validation" "idarth-ssl-certificate" {

NOTE: I have not included the execution plan that created other bits of the infrastructure, but I only reported the problematic bit.

Here are my tf files:

route53.tf

resource "aws_route53_zone" "idarth-hosted-zone" {
  name = "${var.domain_name}"
}


resource "aws_route53_record" "idarth-validation-record" {
  name    = "${aws_acm_certificate.idarth-ssl-certificate.domain_validation_options.0.resource_record_name}"
  type    = "${aws_acm_certificate.idarth-ssl-certificate.domain_validation_options.0.resource_record_type}"
  zone_id = "${aws_route53_zone.idarth-hosted-zone.zone_id}"
  records = ["${aws_acm_certificate.idarth-ssl-certificate.domain_validation_options.0.resource_record_value}"]
  ttl     = "60"
}

resource "aws_acm_certificate_validation" "idarth-ssl-certificate" {
  provider        = "aws.us_east_1"
  certificate_arn = "${aws_acm_certificate.idarth-ssl-certificate.arn}"
  validation_record_fqdns = [
    "${aws_route53_record.idarth-validation-record.fqdn}"
  ]
}

resource "aws_route53_record" "idarth-record-domain" {
  zone_id = "${aws_route53_zone.idarth-hosted-zone.zone_id}"
  name = "${var.domain_name}"
  type = "A"

  alias {
    name = "${aws_cloudfront_distribution.idarth-cloudfront-distr.domain_name}"
    zone_id = "${aws_cloudfront_distribution.idarth-cloudfront-distr.hosted_zone_id}"
    evaluate_target_health = false
  }
}

resource "aws_route53_record" "idarth-record-domain-www" {
  zone_id = "${aws_route53_zone.idarth-hosted-zone.zone_id}"
  name = "${var.domain_name_www}"
  type = "A"

  alias {
    name = "${aws_cloudfront_distribution.idarth-cloudfront-distr.domain_name}"
    zone_id = "${aws_cloudfront_distribution.idarth-cloudfront-distr.hosted_zone_id}"
    evaluate_target_health = false
  }
}

ssl_certificate.tf

provider "aws" {
  alias           = "us_east_1"
  region          = "us-east-1"
}

resource "aws_acm_certificate" "idarth-ssl-certificate" {
  provider        = "aws.us_east_1"

  domain_name       = "${var.domain_name}"
  subject_alternative_names = ["${var.domain_name_www}"]
  validation_method = "DNS"

  lifecycle {
    create_before_destroy = true
  }

  tags = {
        Project = "${var.name}-${var.env}"
        Scope    = "personal-blog"
    }
}

distribution.tf

resource "aws_cloudfront_distribution" "idarth-cloudfront-distr" {
  depends_on = ["aws_acm_certificate_validation.idarth-ssl-certificate"]

  origin {
    domain_name = "${aws_s3_bucket.idarth-static-site-host.bucket_regional_domain_name}"
    origin_id   = "${var.domain_name}"

    /*s3_origin_config {
      origin_access_identity = "origin-access-identity/cloudfront/ABCDEFG1234567"
    }*/
  }

  enabled             = true
  is_ipv6_enabled     = true
  default_root_object = "index.html"

  /*logging_config {
    include_cookies = false
    bucket          = "mylogs.s3.amazonaws.com"
    prefix          = "myprefix"
  }*/

  aliases = ["${var.domain_name}", "${var.domain_name_www}"]

  default_cache_behavior {
    allowed_methods  = ["GET", "HEAD"]
    cached_methods   = ["GET", "HEAD"]
    target_origin_id = "${var.domain_name}"

    forwarded_values {
      query_string = false

      cookies {
        forward = "none"
      }
    }

    compress = true
    viewer_protocol_policy = "redirect-to-https"
    min_ttl                = 0
    default_ttl            = 3600
    max_ttl                = 86400
  }


  #price_class = "PriceClass_200"

  restrictions {
    geo_restriction {
      restriction_type = "none"
      locations        = []
    }
  }

  viewer_certificate {
    acm_certificate_arn  = "${aws_acm_certificate_validation.idarth-ssl-certificate.certificate_arn}"
    ssl_support_method  = "sni-only"
  }


    tags = {
        Project = "${var.name}-${var.env}"
        Scope    = "personal-blog"
    }
}

Terraform version: 0.12.7, aws provider version: v2.25.0_x4

Error log analysis

As I spent one day trying to debug the error above, here are my thoughts:

• The certificate is generating 2 CNAMEs for the 2 domains (variables: var.domain_name, var.domain_name_www): _7bc5230529c8192e8e697aeab0ec0eb9.idarth.com._CNAME, _18ff46dac48c6d852b696306dfa57093.www.idarth.com
• The creation of the first one, as you see in the logs of the execution plan, is created successfully, whilst the second is causing problems.
• Looking in the AWS console, I could see the first CNAME inserted in the DNS hosted zone, but not the second. Even if inserted the DNS hosted zone, the certificate for that record is still resulting in pending validation.

This is what I could find so far, but I have no idea on how to move ahead. Anyone has been here before and could help with the below?

Thanks!

Answer 1

Not sure if this is still relevant but I had the same issue today and found this question here, so perhaps I'll leave the answer for posterity.

Turns out that the validation resource needs to include all generated CNAMEs, like so:

resource "aws_acm_certificate" "some-cert" {
  provider = "aws.us-east-1"

  domain_name               = "some.domain"
  validation_method         = "DNS"
  subject_alternative_names = ["www.some.domain"]
}

resource "aws_route53_record" "cert-validations" {
  count = length(aws_acm_certificate.some-cert.domain_validation_options)

  zone_id = var.zone_id
  name    = element(aws_acm_certificate.some-cert.domain_validation_options.*.resource_record_name, count.index)
  type    = element(aws_acm_certificate.some-cert.domain_validation_options.*.resource_record_type, count.index)
  records = [element(aws_acm_certificate.some-cert.domain_validation_options.*.resource_record_value, count.index)]
  ttl     = 60
}

resource "aws_acm_certificate_validation" "cert-validation" {
  provider = "aws.us-east-1"

  certificate_arn         = aws_acm_certificate.some-cert.arn
  validation_record_fqdns = aws_route53_record.cert-validations.*.fqdn
}

Note in particular that we have a single aws_acm_certificate_validation resource, but it contains a list of multiple validation_record_fqdns, from all the generated validation CNAME DNS records.

Hope that helps!

Answer 2

Here's how I solved this in my configuration with a slight modification to the answer from Marcin Wyszynski. Because validation can result in duplicate DNS records, use the allow_overwrite = true in aws_route53_record to bypass already exists errors and ensure they are all created.

config.tf

# locals must be used so ${var.env} can be interpolated in the definition
locals {
  tags = {
    Name        = "${var.domain_name}"
    Environment = "${var.env}"
  }
  cert_sans = ["www.${var.domain_name}", "cdn.${var.domain_name}", "*.${var.domain_name}"]
}

variable "env" {
  default = "production"
}

variable "domain_name" {
  default = "your-domain.com"
}

acm.tf

resource "aws_acm_certificate" "site" {
  domain_name               = var.domain_name
  validation_method         = "DNS"
  tags                      = local.tags
  subject_alternative_names = local.cert_sans

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_route53_record" "cert_validations" {
  count = length(local.cert_sans) + 1

  zone_id         = aws_route53_zone.public.zone_id
  allow_overwrite = true # This is what allowed for conflict resolution in DNS
  name            = element(aws_acm_certificate.site.domain_validation_options.*.resource_record_name, count.index)
  type            = element(aws_acm_certificate.site.domain_validation_options.*.resource_record_type, count.index)
  records         = [element(aws_acm_certificate.site.domain_validation_options.*.resource_record_value, count.index)]
  ttl             = 60
}

resource "aws_acm_certificate_validation" "cert_validation" {
  certificate_arn         = aws_acm_certificate.site.arn
  validation_record_fqdns = aws_route53_record.cert_validations.*.fqdn

  timeouts {
    create = "120m"
  }
}

Answer 3

The zone shouldn't have a final . ?

resource "aws_route53_zone" "idarth-hosted-zone" {
  name = "${var.dns_zone}."
}

Also on a similar scenario, I used a CNAME for the DNS record pointing to the CloudFront distribution

resource aws_route53_record www {
    zone_id = data.aws_route53_zone.selected.zone_id
    name    = "openbanking.${var.dns_zone}"
    type    = "CNAME"
    records = [aws_cloudfront_distribution.open_banking_public_website.domain_name]
    ttl     = 60
}

HTH

Answer 4

I bumped into this issue generating a cert that had four Subject Alternative names in different Route 53 zones. I'm not sure I'd have fixed it if it weren't by stumbling across the fix in a weird lucid moment at 2:30 AM :)

Given a cert with up to four Subject Alternative names:

locals {
  subject_alternative_names = compact([
    "${var.include_apex_zone_names_in_aliases ? "${var.env}.${var.dns_zone_name}" : null }",
    "${var.include_apex_zone_names_in_aliases ? var.last_mile_apex_zone_name : null }",
    "${var.env}.${var.last_mile_apex_zone_name}"
  ])
}
resource "aws_acm_certificate" "cert" {
  domain_name       = var.include_apex_zone_names_in_aliases ? var.dns_zone_name : "${var.env}.${var.dns_zone_name}"
  subject_alternative_names = local.subject_alternative_names
  validation_method = "DNS"

  lifecycle {
    create_before_destroy = true
  }
}

I had to create ACM cert validation records in each DNS zone so I had four blocks like this:

resource "aws_route53_record" "acm_validation" {
  for_each = {
    for dvo in aws_acm_certificate.cert.domain_validation_options : dvo.domain_name => {
      name   = dvo.resource_record_name
      record = dvo.resource_record_value
      type   = dvo.resource_record_type
    } if dvo.domain_name == var.dns_zone_name
  }

  allow_overwrite = true
  name            = each.value.name
  records         = [each.value.record]
  ttl             = 60
  type            = each.value.type
  zone_id         = var.create_apex_zones ? aws_route53_zone.apex[0].zone_id : data.aws_route53_zone.apex.0.zone_id
}

The code originally had an aws_acm_cert_validation resource ripped straight out of examples in the documentation:

resource "aws_acm_certificate_validation" "cert" {
  certificate_arn         = aws_acm_certificate.cert.arn
  validation_record_fqdns = [for record in aws_route53_record.acm_validation : record.fqdn]
}

But I had to merge all of the aws_route53_record resources into a map for the validation_record_fqdns parameter:

resource "aws_acm_certificate_validation" "cert" {
  certificate_arn         = aws_acm_certificate.cert.arn
  validation_record_fqdns = [for record in merge(aws_route53_record.acm_validation, aws_route53_record.env_acm_validation, aws_route53_record.last_mile_apex_acm_validation, aws_route53_record.last_mile_environment_acm_validation) : record.fqdn]
}

Answer 5

You can use validate_certificate = false as a workaround now, but this has been solved in the newest version of this module, try to update your terraform.