why aws s3 transfer acceleration is not working?

1
votes

I have to upload some files that are between 3 and 7Gb to s3. The default upload speed when using the AWS Console is about 1.3 Mbs. I read about transfer acceleration here:

https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration-examples.html#transfer-acceleration-examples-aws-cli

So I followed the steps:

  1. Turn on transfer acceleration on the bucket, in the console. Then...

    aws s3api put-bucket-accelerate-configuration --bucket [bucket name] --accelerate-configuration Status=Enabled

    aws configure set default.s3.use_accelerate_endpoint true

    aws s3 cp some_file.txt s3://[bucket]/some_file.txt --region us-east-1 --endpoint-ur http://[bucket].s3-accelerate.amazonaws.com

It uploads still at the same 1.3 Mbs. I am WFH so subject to Wifi, but still I wish it could be better. Is there anything else? Do I need to use Python boto3? I was hoping this would be quicker.

2
amazon-web-servicesamazon-s3data-transfer
It's more likely that you are limited by the upload speed of your home internet. - jellycsc
I did upload speed test and got 11 Mbs. - Chuck Pedro
Are you in the same geographical region as the S3 bucket? - jarmod
Speed test apps normally report bits per second, while aws s3 cp reports bytes per second. 11 Mbit/s roughly equals 1.3 Mbyte/sec. Your WiFi is probably faster than that. - Parsifal
Same region - US East. - Chuck Pedro

2 Answers

2
votes

From Amazon S3 Transfer Acceleration - Amazon Simple Storage Service:

Amazon S3 Transfer Acceleration enables fast, easy, and secure transfers of files over long distances between your client and an S3 bucket. Transfer Acceleration takes advantage of Amazon CloudFront’s globally distributed edge locations. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path.

Basically, instead of traversing the Internet to get to the AWS endpoint, traffic is directed to the closest Edge Location and then goes across the Amazon network to the desired region.

If your closest Edge Location is in the same location as an AWS Region, then you will gain no benefit from using Amazon S3 Transfer Acceleration. This is because the traffic will follow exactly the same path.

You can use the Amazon S3 Transfer Acceleration Speed Comparison tool to test whether it provides additional speed.

0
votes

You might check your maximum upstream bandwidth with a speed-tester like google's. That'll set an upper bound for the maximum upload speed you can expect.