Upload a file to s3 given a https s3 url

1
votes

I'm trying to copy an object from one an s3 bucket to my s3 bucket, but I'm given a http url of the file like this;

https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348

If i throw this is the browser i get a csv no worries at all (note: i've obscured parts of the real url so it won't work for you).

I'm using copyObject API in node with the following parameters:

var params = { Bucket: "me-buket", CopySource: "https://not-my-bucket.s3.amazonaws.com/123456738160996-2019-03-23-eo_branch_cta_view-v2-cf5e81e668466de13406877ad681895f6fdd50f38ab7fdb48193d78210098b81-acZMLZ.csv.gz?Signature=12345678Hr3ght0cOPw7bLz%2FJWM%3D&AWSAccessKeyId=123456&Expires=1554017348" , Key: "data" };

I keep getting the error: "InvalidArgument: Unsupported copy source parameter."

What am I doing wrong??

2
node.jsamazon-s3aws-sdk

2 Answers

1
votes

S3 does not support using a pre-signed URL (or any other URL) as CopySource. The value you want is /${bucket}/${key} which in this case is /not-my-bucket/1234567381...acZMLZ.csv.gz (the leading https:// replaced by a leading slash, the .s3.amazonaws.com removed, and the ? and everything following it removed).

But... that still isn't likely to work, unless the user invoking copyObject() has permission to read the source object directly, which is unlikely... otherwise the third party wouldn't have given you the pre-signed URL.

"Fixing" your request as described above will probably change the error to AccessDenied.

Without permission to read the object directly, you can't use copyObject(). You'll need to download the file and then upload it to your bucket in two steps.

0
votes

The above error indicates that the CopySource paramenter which you are giving is incorrect. The correct way to give CopySource parameter will be as follows:

var params = { Bucket: "your_bucket", CopySource: "/source_bucketname_url/sourceobjectkey"};

Note : The name of the source bucket and key name of the source object must be separated by a slash (/).