How to copy object in aws s3 from private bucket to public bucket without downloading?

Question

Copying object across buckets in s3 in the same region is easier with this Request:

AmazonS3 pS3client = new AmazonS3Client(new BasicAWSCredentials(/*supressed*/));
String key = "key";
pS3client.copyObject("sourceBucket", key, "destinationBucket", key);

But when sourceBucket is private access buckets and needs pre-signed urls to access the objects in the bucket the above request fails.

Access denied for the file due to private access.

com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: <>), S3 Extended Request ID: <> at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1579) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1249) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)

How to make s3 request or give pre signed parameters in the copyObject request to copy from private bucket to public destinationBucket?

For workaround obvious solution would be to use GeneratePresignedUrlRequest and get the pre-signed url to access the sourceBucket's object, download in temp file and the use putObject to upload in the destination bucket. That is too much network - so was wondering what is the better alternative if any?

Matt Houser Matt Houser · Accepted Answer · 2017-12-20T14:04:15

The s3:CopyObject command cannot use pre-signed URLs.

In order to use the s3:CopyObject command, the AWS credentials being used simply requires read access to the source bucket, and write access to the target bucket.

If the two buckets are in the same AWS account, then this should be straight forward.

However, if the buckets are in different accounts, then you'll need to apply a bucket policy on the source bucket that grants read access to the target-bucket-owning AWS account, and use the target-bucket-owning AWS account to perform the copy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DelegateS3Access",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:root"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME",
                "arn:aws:s3:::BUCKET_NAME/*"
            ]
        }
    ]
}

Replace BUCKET_NAME with the name of your source S3 bucket, and 123456789012 with the AWS account ID of the target AWS account. After editing, apply this policy on your source S3 bucket.

How to copy object in aws s3 from private bucket to public bucket without downloading?

1 Answers