CannotPullContainerError from public fargate service after creating ECR vpc endpoint with private dns names enabled

Question

In order to get Fargate services in private subnets to work, I created an ECR vpc endpoint, a S3 gateway endpoint and a logs vpc endpoint.

However, after creating the ECR endpoint my service in the public subnet could no longer pull containers: CannotPullContainerError: Error response from daemon

The service in the public subnet has Auto-assign public IP ENABLED.

If I turn private dns names enabled off for the ECR endpoint, the public service will run again, but now the services in the private subnet can't pull their container...

What am I missing?

Worth checking this blog, if you haven't already. There are quite a few endpoints you need to enable - aws.amazon.com/blogs/compute/… — GreenyMcDuff
I did read it. The launch type for the services is fargate, so as far as I can tell from docs.aws.amazon.com/AmazonECR/latest/userguide/… I should only have to enable com.amazonaws.region.ecr.dkr Amazon ECR VPC endpoint and the Amazon S3 gateway endpoint... — 2083

GreenyMcDuff GreenyMcDuff · Accepted Answer · 2019-11-27T13:42:01

I managed to solve this by implementing it as follows:

vpc.tf

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"

  create_vpc = var.create_vpc

  // other config ...

  enable_s3_endpoint = true

  enable_ecr_dkr_endpoint              = true
  ecr_dkr_endpoint_private_dns_enabled = true
  ecr_dkr_endpoint_security_group_ids  = aws_security_group.vpc_endpoints.*.id
}

resource "aws_security_group" "vpc_endpoints" {
  count = var.create_vpc ? 1 : 0

  name   = "PrivateLink endpoints security group"
  vpc_id = module.vpc.vpc_id
}

resource "aws_security_group_rule" "ingress_https" {
  count = var.create_vpc ? 1 : 0

  type        = "ingress"
  from_port   = 443
  to_port     = 443
  protocol    = "tcp"
  cidr_blocks = [module.vpc.private_subnets_cidr_blocks]

  description       = "HTTPS access to VPC Endpoints"
  security_group_id = aws_security_group.vpc_endpoints[0].id
}

ecs-security-groups.tf

variable "private_subnets_cidr_blocks" {}

resource "aws_security_group" "ecs" {
  count       = var.create ? 1 : 0
  name        = "${var.name}-ecs"
  vpc_id      = var.vpc_id
}

resource "aws_security_group_rule" "egress_https_vpc" {
  count = var.create ? 1 : 0

  type        = "egress"
  from_port   = 443
  to_port     = 443
  protocol    = "tcp"
  cidr_blocks = var.private_subnets_cidr_blocks

  description       = "HTTPS access to VPC Endpoints"
  security_group_id = aws_security_group.ecs[0].id
}

For reference, I got the CannotPullContainerError: Error response from daemon you mentioned above when my security groups on both the Endpoints and the ECS Service were not configured to allow HTTPS traffic between them

HTH

CannotPullContainerError from public fargate service after creating ECR vpc endpoint with private dns names enabled

1 Answers