AWS ECS capacity provider using terraform

Question

I'm trying to add to my existing infrastructure managed by terraform a capacity provider for ECS cluster. Terraform apply returns with no errors, the new resource is added in the state file, but surprise surprise it doesn't appear in AWS GUI (ECS cluster->Capacity provider -> No results). If I use aws cli to list this resource outputs fine, also rebuilding everything doesn't help. Has anyone succeeded in adding capacity provider for ECS using terraform?

(I'm using provider version: “2.45.0”) Thank you!

mon mon · Accepted Answer · 2020-03-07T08:11:53

Please beware of [ECS] Add the ability to delete an ASG capacity provider. #632. Once created, it cannot be deleted, only to be updated.

resource "aws_ecs_cluster" "this" {
  name = "${var.PROJECT}_${var.ENV}_${local.ecs_cluster_name}"

  # List of short names of one or more capacity providers
  capacity_providers = local.enable_ecs_cluster_auto_scaling == true ? aws_ecs_capacity_provider.asg[*].name : []
}

resource "aws_ecs_capacity_provider" "asg" {
  count = local.enable_ecs_cluster_auto_scaling ? 1 : 0

  name = "${var.PROJECT}-${var.ENV}-ecs-cluster-capacity-provider"

  auto_scaling_group_provider {
    auto_scaling_group_arn         = local.asg_ecs_cluster_arn

    #--------------------------------------------------------------------------------
    # When using managed termination protection, managed scaling must also be used otherwise managed termination protection will not work.
    # https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cluster-capacity-providers.html#capacity-providers-considerations
    # Otherwise Error:
    # error creating capacity provider: ClientException: The managed termination protection setting for the capacity provider is invalid.
    # To enable managed termination protection for a capacity provider, the Auto Scaling group must have instance protection from scale in enabled.
    #--------------------------------------------------------------------------------
    managed_termination_protection = "ENABLED"

    managed_scaling {
      #--------------------------------------------------------------------------------
      # Whether auto scaling is managed by ECS. Valid values are ENABLED and DISABLED.
      # When creating a capacity provider, you can optionally enable managed scaling.
      # When managed scaling is enabled, ECS manages the scale-in/out of the ASG.
      #--------------------------------------------------------------------------------
      status                    = "ENABLED"
      minimum_scaling_step_size = local.ecs_cluster_autoscaling_min_step_size
      maximum_scaling_step_size = local.ecs_cluster_autoscaling_max_step_size
      target_capacity           = local.ecs_cluster_autoscaling_target_capacity
    }
  }
}

This worked and confirmed auto scaling decreased EC2 instances due to low resource usage, and the service tasks (docker containers) got relocated to running EC2 instances.

AWS bug (or design)

However, after terrafom destroy, when trying to run terraform apply again:

ClientException: The specified capacity provider already exists.

Once fell in such a case, probably need to disable the capacity provider in Terraform scripts (would appear to delete the capacity provider resource, but actually it still exists due to the AWS bug).

Hence, probably the way to get around would be adding the immutable capacity provider to the cluster using CLI, providing the auto scaling group which the capacity provider points to still exists.

$ CAPACITY_PROVIDER=$(aws ecs describe-capacity-providers | jq -r '.capacityProviders[] | select(.status=="ACTIVE" and .name!="FARGATE" and .name!="FARGATE_SPOT") | .name')
$ aws ecs put-cluster-capacity-providers --cluster YOUR_ECS_CLUSTER --capacity-providers ${CAPACITY_PROVIDERS} --default-capacity-provider-strategy capacityProvider=${CAPACITY_PROVIDER},base=1,weight=1

{
    "cluster": {
        "clusterArn": "arn:aws:ecs:us-east-2:200506027189:cluster/YOUR_ECS_CLUSTER",
        "clusterName": "YOUR_ECS_CLUSTER",
        "status": "ACTIVE",
        "registeredContainerInstancesCount": 0,
        "runningTasksCount": 0,
        "pendingTasksCount": 0,
        "activeServicesCount": 0,
        "statistics": [],
        "tags": [],
        "settings": [
            {
                "name": "containerInsights",
                "value": "disabled"
            }
        ],
        "capacityProviders": [
            "YOUR_CAPACITY_PROVIDER"
        ],
        "defaultCapacityProviderStrategy": [
            {
                "capacityProvider": "YOUR_CAPACITY_PROVIDER",
                "weight": 1,
                "base": 1
            }
        ],
        "attachments": [
            {
                "id": "628ee192-4d0f-44be-85c0-049d796ed65c",
                "type": "asp",
                "status": "PRECREATED",
                "details": [
                    {
                        "name": "capacityProviderName",
                        "value": "YOUR_CAPACITY_PROVIDER"
                    },
                    {
                        "name": "scalingPlanName",
                        "value": "ECSManagedAutoScalingPlan-89682dcf-bb53-492f-8329-25d75458ea11"
                    }
                ]
            }
        ],
        "attachmentsStatus": "UPDATE_IN_PROGRESS"      <----- Takes time for the capacity provider to show up in ECS clsuter console
    }
}

AWS ECS capacity provider using terraform

2 Answers

AWS bug (or design)