Fargate task getting CannotPullContainerError even with NAT gateway defined

0
votes

I'm getting CannotPullContainerError trying to launch an ECS Fargate task through an AWS Step Function. The docs here say to add a NAT gateway to the subnet. I've done that and still get this error.

I'm using a private subnet, public IP disabled, and have a NAT gateway defined. I have a route table defined to map 0.0.0.0/0 to the NAT Gateway and this route table is associated with the subnet. Associated security group and network ACL allow all outbound traffic. The VPC has DNS resolution enabled.

I've reviewed these related questions:

AWS Fargate - CannotPullContainerError (500)?

Fargate error: cannot pull container hosted in ECR from a private subnet

Fargate Task with Nat Gateway fails to connect with RDS database

Executing Step Function "Tasks" using ECS Fargate

Is there something else I'm missing? I've seen lots of questions here but have already addressed the things mention (usually NAT gateway and route table).

Error:

CannotPullContainerError: Error response from daemon: 
Get https://123456789012.dkr.ecr.us-west-2.amazonaws.com/v2/:
net/http: request canceled while waiting for connection
(Client.Timeout exceeded while awaiting headers)"

Hopefully useful information:

subnetId: subnet-015a0400000000
networkInterfaceId: eni-04e50000000
privateIPv4Address: 10.51.17.8-2c
ClusterArn: arn:aws:ecs:us-west-2:951740000000:cluster/step-function-executor
ContainerArn: arn:aws:ecs:us-west-2:951740000000:container/08450000000",
Image: 951740000000.dkr.ecr.us-west-2.amazonaws.com/step-function-image:latest
NetworkBindings: []
NetworkInterfaces:
    AttachmentId: 4a3b0000000
    PrivateIpv4Address": 10.51.17.8
TaskArn: arn:aws:ecs:us-west-2:951740000000:task/690d0000000
TaskDefinitionArn: arn:aws:ecs:us-west-2:951740000000:task-definition/step-function-xyz
LaunchType: FARGATE
PullStartedAt: 1599440424569
PullStoppedAt: 1599440513569

Route table:

    Destination       Target
    -------------     ---------------
    10.41.0.0./16     local
    0.0.0.0/0         nat-046d0000000

NAT Gateway

    Gateway ID: nat-046d0000000
    Private IP: 10.51.x.x
    Elastic IP Address 52.13.x.x
1
amazon-web-servicesdockeraws-fargate
Does your ECS task execution role allow access to ECR? – Marcin
Also is the NAT gateway in public subnet with working internet connectivity? – Marcin
@Marcin The subnet has "auto-assign public IP address" turned on, so that is what is needed to make it public, right? – Samuel Neff
You also would need internet gateway (IGW) attached to your VPC and a route table in a public subnet to the IGW. – Marcin
@Marcin thanks, the role didn't have the ECR permissions. I added it now, but still same error. – Samuel Neff

1 Answers

1
votes

In the end the problem was with security groups. I added an existing security group to the AWS Step Function definition and that resolved the problem.