AWS S3 upload fails: RequestTimeTooSkewed

21
votes

I'm using

aws s3 sync ~/folder/ s3:// --delete

to upload (and sync) a large number of files to an S3 bucket. Some - but not all - of the files fail, throwing this error message:

upload failed: to s3://bucketname/folder/ A client error (RequestTimeTooSkewed) occurred when calling the UploadPart operation: The difference between the request time and the current time is too large

I know that the cause of this error is usually a local time that's out of sync with Internet time, but I'm running NTP (on my Ubuntu PC) and the date/time seem absolutely accurate - and this error has only been reported for about 15 out of the forty or so files I've uploaded so far. Some of the files are relatively large - up to about 70MB each - and my upload speeds aren't fantastic: could S3 possibly be comparing the initial and completion times and reporting their difference as an error? Thanks,

6
amazon-web-servicesamazon-s3
It doesn't matter how long your upload takes as S3 uses only the Date (or x-amz-date) header to compare times and it does not rely on the upload completion time. - dcro

6 Answers

27
votes

The time verification happens at the start of your upload to S3, so it won't be to do with files taking too long to upload.

Try comparing your system time with what S3 is reporting and see if there is any unnecessary time drift, just to make sure:

# Time from Amazon
$ curl http://s3.amazonaws.com -v

# Time on your local machine
$ date -u

(Time is returned in UTC)

12
votes

I was running aws s3 cp inside docker container on a MacBook Pro and got this error. Restart the Docker for Mac fixed this issue.

4
votes

Amazon S3 uses NTP for its system clocks, to sync with your clock. Run

sudo apt-get install ntp

then open /etc/ntp.conf and add at the bottom

server 0.amazon.pool.ntp.org iburst
server 1.amazon.pool.ntp.org iburst
server 2.amazon.pool.ntp.org iburst
server 3.amazon.pool.ntp.org iburst

Then run

sudo service ntp restart

2
votes

It now seems that multipart uploads were failing on aws s3. Using s3cmd instead works perfectly.

1
votes

You have to sync you local time on your machine. The time is out of world time.

0
votes

I'm having the issue on MacOS. I fixed it by Preference -> Date & Time -> check the box "Set date and time automatically"