Container is not able to call S3 in Fargate

2
votes

I'm not able to synchronize a log-folder to s3 inside a container.

I'm trying to get the following setup:

  • Docker Container with installed awscli

    • there are logfiles and other files generated inside the container

  • There is a cronjob, which calls the "aws s3 sync" command through a shell-script.

The synchronisation is not working properly and I'm not sure why not.

I tried the following, which worked just fine:

  • provided access key/secret access key inside the docker container

    • this worked locally, with plain ECS and with fargate
    • but it's not recommended to use the access keys

  • plain ECS without any keys (just the IAM role)

    • this worked too

I played a little with the configuration and read through the documentation. The only hints I got are:

  • Has it something to do with the network mode "awsvpc"? (which fargate has to use)
  • Has it something to do with the "AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" path variable?
    • I found a few hits there on the web, but I'm not sure if it's set or not. I'm not able to look inside the container in fargate.
2
amazon-web-servicesaws-fargate
stackoverflow.com/questions/51451041/… – error404
Not a good idea to generate log files inside the container. Write the logs to the console of the container and send them using the awslogs – OK999
I know sending logs to cloudwatch is a better idea. But it's not always suitable for older projects. The logging to S3 should work at least... – quiqon
I think I'm facing a similar problem. Did you ever find a solution? – MGJ-123
Sorry I never found a solution for this problem. – quiqon

2 Answers

0
votes

ECS Task Definition has two parameters related to defining IAM Role.

executionRoleArn - Provides access to the task or container to start running by performing needed actions such as pulling images from ECR, writing logs to Cloudwatch.

taskRoleArn - Allows the Task to execute AWS API calls to interact with AWS resources such as S3, etc...

0
votes

In my case i had a shell script which i used to call using entrypoint in the task definition. I had correctly set the Task Role with access to S3 however it did not work. So using the information provided here https://forums.aws.amazon.com/thread.jspa?threadID=273767#898645 i added the first line in my shell script as

export AWS_CONTAINER_CREDENTIALS_RELATIVE_URI

Still it did not work. Then i upgraded the AWS cli on the docker container to version 2 and it worked. So for me the real problem was that the docker image had an old CLI version.