EKS Node Group Terraform - Add label to specific node

1
votes

I'm provisioning EKS with managed nodes through Terraform. No issues there, it's all working fine.

My problem is that I want to add a label to one of my nodes to use as a nodeSelector in one of my deployments. I have an app that is backed by an EBS persistent volume which obviously is only available in a single AZ, so I want my pod to schedule there.

I can add a label pretty easily with:

kubectl label nodes <my node> <key>=<value>

And actually this is fine, that is until you do something like update the node group to the next version. The labels don't persist, which makes sense as they are not managed by Amazon.

Is there a way, either through terraform or something else to set these labels and make them persist. I notice that the EKS provider for Terraform has a labels option, but it seems like that will add the label to all nodes in the Node Group, and that's not what I want. I've looked around, but can't find anything.

3
kubernetesterraformamazon-eks

3 Answers

4
votes

You may not need to add a label to a specific node to solve your problem. Amazon as a cloud provider adds some Kubernetes labels to each node in a managed node group. Example:

labels:  
  failure-domain.beta.kubernetes.io/region: us-east-1
  failure-domain.beta.kubernetes.io/zone: us-east-1a
  kubernetes.io/hostname: ip-10-10-10-10.ec2.internal...
  kubernetes.io/os: linux
  topology.ebs.csi.aws.com/zone: us-east-1a
  topology.kubernetes.io/region: us-east-1
  topology.kubernetes.io/zone: us-east-1a

The exact labels available to you will depend on the version of Kubernetes you are running. Try running kubectl get nodes -o json | jq '.items[].metadata.labels' to see the labels set on each node in your cluster.

I recommend using topology.kubernetes.io/zone to match the availability zone containing your EBS volume. According to the Kubernetes documentation, both nodes and persistent volumes should have this label populated by the cloud provider.

Hope this helps. Let me know if you still have questions.

2
votes

You can easily achieve that with Terraform:

resource "aws_eks_node_group" "example" {
  ...
  labels = {
    label_key = "label_value"
  }
}
0
votes

Add a second node group (with the desired node info) and label that node group.