Terraform target aws_volume_attachment with only its corresponding aws_instance resource from a list

Question

I am not able to target a single aws_volume_attachment with its corresponding aws_instance via -target. The problem is that the aws_instance is taken from a list by using count.index, which forces terraform to refresh all aws_instance resources from that list.

In my concrete case I am trying to manage a consul cluster with terraform. The goal is to be able to reinit a single aws_instance resource via the -target flag, so I can upgrade/change the whole cluster node by node without downtime.

I have the following tf code:

### IP suffixes
variable "subnet_cidr" { "10.10.0.0/16" }

// I want nodes with addresses 10.10.1.100, 10.10.1.101, 10.10.1.102
variable "consul_private_ips_suffix" {
  default = {
    "0" = "100"
    "1" = "101"
    "2" = "102"
  }
}

###########
# EBS
#
// Get existing data EBS via Name Tag
data "aws_ebs_volume" "consul-data" {
  count = "${length(keys(var.consul_private_ips_suffix))}"

  filter {
    name   = "volume-type"
    values = ["gp2"]
  }

  filter {
    name   = "tag:Name"
    values = ["${var.platform_type}.${var.platform_id}.consul.data.${count.index}"]
  }
}

#########
# EC2
#
resource "aws_instance" "consul" {
  count      = "${length(keys(var.consul_private_ips_suffix))}"

  ...

  private_ip = "${cidrhost(aws_subnet.private-b.cidr_block, lookup(var.consul_private_ips_suffix, count.index))}"
}

resource "aws_volume_attachment" "consul-data" {
  count       = "${length(keys(var.consul_private_ips_suffix))}"

  device_name = "/dev/sdh"
  volume_id   = "${element(data.aws_ebs_volume.consul-data.*.id, count.index)}"
  instance_id = "${element(aws_instance.consul.*.id, count.index)}"
}

This works perfectly fine for initializing the cluster. Now I make a change in my user_data init script of the consul nodes and want to rollout node by node. I run terraform plan -target=aws_volume_attachment.consul_data[0] to reinit node 0. This is when I run into the above mentioned problem, that terraform renders all aws_instance resources because of instance_id = "${element(aws_instance.consul.*.id, count.index)}".

Is there a way to "force" tf to target a single aws_volume_attachment with only its corresponding aws_instance resource?

Martin Atkins Martin Atkins · Accepted Answer · 2017-07-06T23:54:55

At the time of writing this sort of usage is not possible due to the fact that, as you've seen, an expression like aws_instance.consul.*.id creates a dependency on all the instances, before the element function is applied.

The -target option is not intended for routine use and is instead provided only for exceptional circumstances such as recovering carefully from an unintended change.

For this specific situation it may work better to use the ignore_changes lifecycle setting to prevent automatic replacement of the instances when user_data changes, like this:

resource "aws_instance" "consul" {
  count      = "${length(keys(var.consul_private_ips_suffix))}"

  ...

  private_ip = "${cidrhost(aws_subnet.private-b.cidr_block, lookup(var.consul_private_ips_suffix, count.index))}"

  lifecycle {
    ignore_changes = ["user_data"]
  }
}

With this set, Terraform will detect but ignore changes to the user_data attribute. You can then get the gradual replacement behavior you want by manually tainting the resources one at a time:

$ terraform taint aws_instance.consul[0]

On the next plan, Terraform will then see that this resource instance is tainted and produce a plan to replace it. This gives you direct control over when the resources are replaced, so you can therefore ensure that e.g. the consul leave step gets a chance to run first, or whatever other cleanup you need to do.

This workflow is recommended over -target because it makes the replacement step explicit. -target can be confusing in a collaborative environment because there is no evidence of its use, and thus no clear explanation of how the current state was reached. taint, on the other hand, explicitly marks your intention in the state where other team members can see it, and then replaces the resource via the normal plan/apply steps.

Terraform target aws_volume_attachment with only its corresponding aws_instance resource from a list

1 Answers