Google Cloud Storage transfer from Amazon S3 - Invalid access key

Question

I'm trying to create a transfer from my S3 bucket to Google Cloud - it's basically the same problem as in this question, but none of the answers work for me. Whenever I try to make a transfer, I get the following error:

Invalid access key. Make sure the access key for your S3 bucket is correct, or set the bucket permissions to Grant Everyone.

I've tried the following policies, to no success:

First policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*",
                "s3:GetBucketLocation"
            ],
            "Resource": "*"
        }
    ]
}

Second policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": "*"
    }
  ]
}

Third policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::my-bucket-name",
                "arn:aws:s3:::my-bucket-name/*"
            ]
        }
    ]
}

I've also made sure to grant the 'List' permission to 'Everyone'. Tried this on buckets in two different locations - Sao Paulo and Oregon. I'm starting to run out of ideas, hope you can help.

So, you've configured an AWS IAM user with the policies shown, provided that IAM user's access/secret keys to Google's Storage Transfer tool, and the transfer's failing? Have you verified the creds/policy by using them yourself with the awscli to list and download from the bucket? Have you enabled CloudTrail logging on your account to see what API actions and which credentials/resources the Transfer tool is actually coming in with? — jarmod
@jarmod yes, I have no problem using the awscli to download from the bucket. CloudTrail doesn't show any indication of Google Cloud trying to access the bucket. I've just enabled Amazon S3 server access logs, maybe that will show something. — user2634633
Yes, good idea. I'm not sure if actions like HTTP GET that causes an S3 server access log will also cause a corresponding CoudTrail log (S3 GetObject). If you're seeing no logs in either location, then that would seem to suggest the wrong keys or wrong bucket are being used in the Google Transfer request (unlikely presumably). Note that the logs do not appear instantaneously. — jarmod

Ker-Yu Ker-Yu · Accepted Answer · 2017-11-08T20:52:30

I know this question is over a year old but I just encountered the same error when trying to do the transfer via the console. I worked around this by executing IT via the gsutils command line tool instead.

After installing and configuring the tool, simply run:

gsutils cp s3://sourcebucket gs://targetbucket

Hope this is helpful!

Google Cloud Storage transfer from Amazon S3 - Invalid access key

1 Answers