Cannot copy files from S3 to EC2 Instance

Question

I have an EC2 instance which has all inbound traffic open. It is assigned the below role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::test-bucket"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:ListObject"
            ],
            "Resource": [
                "arn:aws:s3:::test-bucket/*"
            ]
        }
    ]
}

I am trying to copy a file from an S3 bucket to this instance.

aws s3 cp s3://test-bucket/test.csv /home/ubuntu/

I am getting this error:

fatal error: Connect timeout on endpoint URL: "https://test-bucket.s3.eu-central-1.amazonaws.com/test.csv"

What am I doing wrong? How can I get the public IP address used by my S3 bucket? I can try to specify that in my outbound rules for the EC2.

You'll need s3:GetObject on the bucket "arn:aws:s3:::test-bucket" in addition to the objects in the bucket "arn:aws:s3:::test-bucket/*" — MarkAWard
is the region correct? is your bucket in eu-central-1? can set the region at the cli with --region=eu-west-1 or in your ~/.aws/config — MarkAWard
@MarkAWard - I added the region as well. I still get the same error. Do you think I should change my outbound rule? What is the public ip of S3? — Seeker90

Taterhead Taterhead · Accepted Answer · 2019-05-03T09:40:04

How can I get the public IP address used by my S3 bucket? I can try to specify that in my outbound rules for the EC2

Go to your S3 console with a modern, non-mobile web browser. Click on an object inside of a bucket. To the right will popout a details pane for the object that show Overview, Properties and Permissions. In the Overview is the object URL. Typically it follows this URL pattern:

https://s3.amazonaws.com/<bucket-name>/<object key>

ping s3.amazonaws.com to get the ip address. Right now, I get: 52.216.185.149

To get the range of s3 ips use the aws powershell command:

Get-AWSPublicIpAddressRange -ServiceKey s3 -Region eu-central-1

IpPrefix            IpAddressFormat Region       Service
--------            --------------- ------       -------
52.219.72.0/22      Ipv4            eu-central-1 S3
52.219.44.0/22      Ipv4            eu-central-1 S3
52.92.68.0/22       Ipv4            eu-central-1 S3
54.231.192.0/20     Ipv4            eu-central-1 S3

A simpler approach to download s3 files to your ubuntu instance, I suggest you follow the steps in this guide that uses the s3 sync command. The guide uses a role to include your s3 bucket paticulars. Note the tips indicate that you cannot change IAM roles for your ec2 instance after it has been launched, but that is not true anymore. You can create a role and attach it to an already running instance. I would use the s3 cp command as opposed to s3 sync.

Cannot copy files from S3 to EC2 Instance

1 Answers