Copy from S3 bucket in one account to S3 bucket in another account using Boto3 in AWS Lambda

Question

I have created a S3 bucket and created a file under my aws account. My account has trust relationship established with another account and I am able to put objects into the bucket in another account using Boto3. How can I copy objects from bucket in my account to bucket in another account using Boto3?

I see "access denied" when I use the code below -

source_session = boto3.Session(region_name = 'us-east-1')
    source_conn = source_session.resource('s3')
    src_conn = source_session.client('s3')
    dest_session = __aws_session(role_arn=assumed_role_arn, session_name='dest_session')
    dest_conn = dest_session.client ( 's3' )

    copy_source = { 'Bucket': bucket_name , 'Key': key_value }
    dest_conn.copy ( copy_source, dest_bucket_name , dest_key,ExtraArgs={'ServerSideEncryption':'AES256'}, SourceClient = src_conn )

In my case , src_conn has access to source bucket and dest_conn has access to destination bucket.

I believe the only way to achieve this by downloading and uploading the files.

AWS Session

client = boto3.client('sts')
            response = client.assume_role(RoleArn=role_arn, RoleSessionName=session_name)
            session = boto3.Session(
                aws_access_key_id=response['Credentials']['AccessKeyId'],
                aws_secret_access_key=response['Credentials']['SecretAccessKey'],
                aws_session_token=response['Credentials']['SessionToken'])

Where did you get "__aws_session()"? You should be using public interfaces such as AssumeRole(). — John Hanley
__aws_session is a function that I defined. I am assuming role in that function. — Punter Vicky
Defined where? Do you have a documentation link? Or is this something you wrote? — John Hanley
I wrote it and I have copied the code within that function in my post now. — Punter Vicky
Does your assumed role have permissions to access the source bucket? Using SourceClient does not give you additional permissions for the bucket copy. — John Hanley

boltronics boltronics · Accepted Answer · 2018-12-20T04:38:12

Another approach is to attach a policy to the destination bucket permitting access from the account hosting the source bucket. eg. something like the following should work (although you may want to tighten up the permissions as appropriate):

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<source account ID>:root" }, "Action": "s3:*", "Resource": [ "arn:aws:s3:::dst_bucket", "arn:aws:s3:::dst_bucket/*" ] } ] }

Then your Lambda hosted in your source AWS account should have no problems writing to the bucket(s) in the destination AWS account.

Copy from S3 bucket in one account to S3 bucket in another account using Boto3 in AWS Lambda

1 Answers