Amazon S3 sync to local machine failed

Question

I'm new to AWS and I'm trying to download a bunch of files from my S3 bucket to my local machine using aws s3 sync as described in http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html. I used the following command:

aws s3 sync s3://outputbucket/files/ .

I got the following error: A client error (AccessDenied) occurred when calling the ListObjects operation: Access Denied Completed 1 part(s) with ... file(s) remaining

Even though I have configured my access key ID & secret access key as described in http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html

Where might the problem be?

Answer 1

Assuming that you are an Administrator and/or you have set your credentials properly, it is possible that you are using an old AWS CLI.

I encountered this while using the packaged AWS CLI with Ubuntu 14.04.

The solution that worked for me is to remove the AWS CLI prepackaged with Ubuntu, and download it from python-pip instead:

sudo apt-get remove awscli
sudo apt-get install python-pip
sudo pip install awscli

Many thanks to this link: https://forums.aws.amazon.com/thread.jspa?threadID=173124

Answer 2

To perform a file sync, two sets of permissions are required:

• ListObjects to obtain a list of files to copy
• GetObjects to access the objects

If you are using your "root" user that comes with your AWS account, you will automatically have these permissions.

If you are using a user created within Identity and Access Management (IAM), you will need to assign these permissions to the User. The easiest way is to assign the AmazonS3FullAccess policy, which gives access to all S3 functions.

Answer 3

In my case the credentials stored in ~/.aws/config were being clobbered by a competing profile sourced in ~/.zshrc. Run env | grep AWS to check.