Terraform EKS tagging

Question

I am having this issue of Terraform EKS tagging and don't seem to find workable solution to tag all the VPC subnets when a new cluster is created.

To provide some context: We have one AWS VPC where we deployment several EKS cluster into the subnets. We do not create VPC or subnets are part of the EKS cluster creation. Therefore, the terraform code creating a cluster doesn't get to tag existing subnets and VPC. Although EKS will add the required tags, they are automatically removed next time we run terraform apply on the VPC.

My attempt to workaround is to provide a terraform.tfvars file within the VPC to as follows:

eks_tags = 
 [
 "kubernetes.io/cluster/${var.cluster-1}", "shared", 
 "kubernetes.io/cluster/${var.cluster-2}", "shared",
 "kubernetes.io/cluster/${var.cluster-2}", "shared",
]

Then within the VPC and subnets resources, we do something like

    resource "aws_vpc" "demo" {
      cidr_block = "10.0.0.0/16"

      tags = "${
        map(
         ${var.eks_tags}
        )
     }"
    }

However, the above does not seem to work. I have tried various Terraform 0.11 functions from https://www.terraform.io/docs/configuration-0-11/interpolation.html but not of them help.

Has anyone ben able to resolve this issue?

The idea that we always create new VPC and subnet for every EKS cluster is wrong. Obviously, the has to be a way to tag existing VPC and subnet resources using Terraform?

rsmets rsmets · Accepted Answer · 2020-11-21T01:05:43

You can now use the aws provider ignore_tags attribute so that the tags made with the aws_ec2_tag resource do not get removed next time the VPC module is applied.

For example the provider becomes:

provider "aws" {
  profile = "terraform"
  region  = "us-west-1"
  
  // This is necessary so that tags required for eks can be applied to the vpc without changes to the vpc wiping them out.
  // https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/resource-tagging
  ignore_tags {
    key_prefixes = ["kubernetes.io/"]
  }
}

Add you can then leverage the aws_ec2_tag resource like so in your EKS module without worrying about the tag getting removed next time the VPC module is applied.

/*
  Start of resource tagging logic to update the provided vpc and its subnets with the necessary tags for eks to work
  The toset() function is actually multiplexing the resource block, one for every item in the set. It is what allows 
  for setting a tag on each of the subnets in the vpc.
*/
resource "aws_ec2_tag" "vpc_tag" {
  resource_id = data.terraform_remote_state.vpc.outputs.vpc_id
  key         = "kubernetes.io/cluster/${var.cluster_name}"
  value       = "shared"
}

resource "aws_ec2_tag" "private_subnet_tag" {
  for_each    = toset(data.terraform_remote_state.vpc.outputs.private_subnets)
  resource_id = each.value
  key         = "kubernetes.io/role/elb"
  value       = "1"
}

resource "aws_ec2_tag" "private_subnet_cluster_tag" {
  for_each    = toset(data.terraform_remote_state.vpc.outputs.private_subnets)
  resource_id = each.value
  key         = "kubernetes.io/cluster/${var.cluster_name}"
  value       = "shared"
}

resource "aws_ec2_tag" "public_subnet_tag" {
  for_each    = toset(data.terraform_remote_state.vpc.outputs.public_subnets)
  resource_id = each.value
  key         = "kubernetes.io/role/elb"
  value       = "1"
}

resource "aws_ec2_tag" "public_subnet_cluster_tag" {
  for_each    = toset(data.terraform_remote_state.vpc.outputs.public_subnets)
  resource_id = each.value
  key         = "kubernetes.io/cluster/${var.cluster_name}"
  value       = "shared"
}

Terraform EKS tagging

3 Answers