On EKS how do I verify I configured a Spot Instance through Terraform

Question

I followed this documentation: https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/docs/spot-instances.md

and successfully provisioned An EKS cluster.

I asked kubectl to describe the node and got:

➜  ~ kubectl describe node ip-10-0-1-205.us-east-2.compute.internal
Name:               ip-10-0-1-205.us-east-2.compute.internal
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
beta.kubernetes.io/instance-type=t2.medium
beta.kubernetes.io/os=linux
failure-domain.beta.kubernetes.io/region=us-east-2
failure-domain.beta.kubernetes.io/zone=us-east-2a
kubernetes.io/arch=amd64
kubernetes.io/hostname=ip-10-0-1-205.us-east-2.compute.internal
kubernetes.io/os=linux
node.kubernetes.io/instance-type=t2.medium
node.kubernetes.io/lifecycle=spot
prefer=bot
topology.kubernetes.io/region=us-east-2
topology.kubernetes.io/zone=us-east-2a
Annotations:        node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Sun, 06 Dec 2020 18:09:26 +0200
Taints:             <none>
Unschedulable:      false
Lease:
HolderIdentity:  ip-10-0-1-205.us-east-2.compute.internal
AcquireTime:     <unset>
RenewTime:       Sun, 06 Dec 2020 21:03:06 +0200
Conditions:
Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
MemoryPressure   False   Sun, 06 Dec 2020 20:59:07 +0200   Sun, 06 Dec 2020 18:09:25 +0200   KubeletHasSufficientMemory   kubelet has sufficient memory available
DiskPressure     False   Sun, 06 Dec 2020 20:59:07 +0200   Sun, 06 Dec 2020 18:09:25 +0200   KubeletHasNoDiskPressure     kubelet has no disk pressure
PIDPressure      False   Sun, 06 Dec 2020 20:59:07 +0200   Sun, 06 Dec 2020 18:09:25 +0200   KubeletHasSufficientPID      kubelet has sufficient PID available
Ready            True    Sun, 06 Dec 2020 20:59:07 +0200   Sun, 06 Dec 2020 18:09:56 +0200   KubeletReady                 kubelet is posting ready status
Addresses:
InternalIP:   10.0.1.205
Hostname:     ip-10-0-1-205.us-east-2.compute.internal
InternalDNS:  ip-10-0-1-205.us-east-2.compute.internal
Capacity:
attachable-volumes-aws-ebs:  39
cpu:                         2
ephemeral-storage:           104845292Ki
hugepages-2Mi:               0
memory:                      4037584Ki
pods:                        17
Allocatable:
attachable-volumes-aws-ebs:  39
cpu:                         1930m
ephemeral-storage:           95551679124
hugepages-2Mi:               0
memory:                      3482576Ki
pods:                        17
System Info:
Machine ID:                 4283642d849e48e7ac935e6a6574599a
System UUID:                EC22BB5A-0463-5D55-ECDD-49865E6294F9
Boot ID:                    b568afc1-96f2-4669-895e-b3586b7758df
Kernel Version:             4.14.203-156.332.amzn2.x86_64
OS Image:                   Amazon Linux 2
Operating System:           linux
Architecture:               amd64
Container Runtime Version:  docker://19.3.6
Kubelet Version:            v1.18.9-eks-d1db3c
Kube-Proxy Version:         v1.18.9-eks-d1db3c
ProviderID:                   aws:///us-east-2a/i-045333340f54ac375
Non-terminated Pods:          (5 in total)
Namespace                   Name                                                    CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                   ----                                                    ------------  ----------  ---------------  -------------  ---
kube-system                 aws-node-9bp6w                                          10m (0%)      0 (0%)      0 (0%)           0 (0%)         173m
kube-system                 kube-proxy-52l4m                                        100m (5%)     0 (0%)      0 (0%)           0 (0%)         173m
monitoring                  prometheus-kube-prometheus-operator-576f4bf45b-wgz5v    0 (0%)        0 (0%)      0 (0%)           0 (0%)         166m
monitoring                  prometheus-prometheus-node-exporter-tnh6h               0 (0%)        0 (0%)      0 (0%)           0 (0%)         166m
wielder-services            bot-b5f557cc-d7b74                                      1600m (82%)   0 (0%)      1600Mi (47%)     0 (0%)         155m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource                    Requests      Limits
  --------                    --------      ------
cpu                         1710m (88%)   0 (0%)
memory                      1600Mi (47%)  0 (0%)
ephemeral-storage           0 (0%)        0 (0%)
hugepages-2Mi               0 (0%)        0 (0%)
attachable-volumes-aws-ebs  0             0
Events:                       <none>

The Only indication I could detect of the node being a spot instance in this description is the tag I created node.kubernetes.io/lifecycle=spot

In the AWS console looking at the node information I find:

Termination protection Disabled

Lifecycle normal

How do I know for sure I have provisioned a spot instance?

If I haven't provisioned a spot instance how do I go about doing it?

Lifecycle Normal means on-demand. It would say Lifecycle Spot for spot. You need to set a bid price in your launch config for spot to be provisioned instead of on-demand — jordanm

Rubber Duck Rubber Duck · Accepted Answer · 2020-12-16T11:37:25

Following @jordanm advise I searched high and low for documentation and finally came across something similar at https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/spot_instance_request https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/docs/spot-instances.md

and found this syntax in a related terraform configuration

spot_price = "1.10"

here is how it looks inside the EKS module:

module "eks" {
  source          = "terraform-aws-modules/eks/aws"
  cluster_name    = local.cluster_name
  cluster_version = var.kube_version
  subnets         = module.vpc.private_subnets
  vpc_id          = module.vpc.vpc_id
  enable_irsa     = true

  tags = {
    Environment = "training"
    GithubRepo  = "terraform-aws-eks"
    GithubOrg   = "terraform-aws-modules"
  }

  worker_groups = [
    
    {
      name                          = "worker-group-bot"
      instance_type                 = var.bot_instance
      disk_size                     = var.bot_disk_size
      additional_userdata           = "echo foo bar"
      additional_security_group_ids = [aws_security_group.worker_group_mgmt_two.id]
      asg_desired_capacity          = var.bot_desired
      asg_max_size                  = var.bot_max

      //      availability_zones = [var.availability_zone]
      //      subnets         = [module.vpc.private_subnets[0]]

      kubelet_extra_args      = "--node-labels=node.kubernetes.io/lifecycle=spot,prefer=bot"

      suspended_processes     = ["AZRebalance"]
      spot_price              = "1.10"

      tags = [
        {
          "key"                 = "k8s.io/cluster-autoscaler/enabled"
          "propagate_at_launch" = "false"
          "value"               = "true"
        },
        {
          "key"                 = "k8s.io/cluster-autoscaler/${local.cluster_name}"
          "propagate_at_launch" = "false"
          "value"               = "true"
        }
      ]
    },
  ]

  workers_additional_policies = ["arn:aws:iam::aws:policy/AutoScalingFullAccess"]
}

If someone knows where the documentation resides please comment.

On EKS how do I verify I configured a Spot Instance through Terraform

1 Answers