Is it possible using AWS CLI to copy the contents of S3 buckets between AWS accounts? I know it's possible to copy/sync between buckets in the same account, but I need to get the contents of an old AWS account into a new one. I have AWS CLI configured with two profiles, but I don't see how I can use both profiles in a single copy/sync command.
13
votes
4 Answers
20
votes
Very Simple. Let's say:
Old AWS Account = [email protected]
New AWS Account = [email protected]
Loginto the AWS console as [email protected]
Go to the bucket of your choice and apply below bucket policy:
{
"Statement": [
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket_name",
"Principal": {
"AWS": [
"[email protected]"
]
}
},
{
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::bucket_name/*",
"Principal": {
"AWS": [
"[email protected]"
]
}
}
]
}
I would guess that bucket_name and [email protected] is evident to you in above policy
Now, Make sure you are running AWS-CLI with the credentials of [email protected]
Run below command and the copy will happen like a charm:
aws s3 cp s3://bucket_name/some_folder/some_file.txt s3://[email protected]_acount/fromold_account.txt
Ofcourse, do make sure that [email protected] has write privileges to his own bucket [email protected]_acount which is used in above command to save the stuff copied from [email protected] bucket.
Hope this helps.
6
votes
Ok, I have this working now! Thanks for your answers. In the end I used a combination between @slayedbylucifer and @Sony Kadavan. What worked for me was a new bucket policy and a new user policy.
I added the following bucket policy (Account A):
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::myfoldername",
"Principal": {
"AWS": [
"arn:aws:iam::111111111111:user/myusername"
]
}
},
{
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::myfoldername",
"Principal": {
"AWS": [
"arn:aws:iam::111111111111:user/myusername"
]
}
}
]
}
And the following user policy (Account B):
{
"Version": "2012-10-17",
"Statement":{
"Effect":"Allow",
"Action":"s3:*",
"Resource":"arn:aws:s3:::myfoldername/*"
}
}
And used the following aws cli command (the region option was required because the accounts were in different regions):
aws --region us-east-1 s3 sync s3://myfoldername s3://myfoldername-accountb
1
votes
Yes, you can. You need to first create an IAM user in the second account and delegate permissions to it - read/write/list on specific S3 bucket. Once you do this then provide this IAM users's credentials to your CLI and it will work.
How to delegate permissions: Delegating Cross-Account Permissions to IAM Users - AWS Identity and Access Management : http://docs.aws.amazon.com/IAM/latest/UserGuide/DelegatingAccess.html#example-delegate-xaccount-roles
Sample S3 policy for delegation:
{
"Version": "2012-10-17",
"Statement" : {
"Effect":"Allow",
"Sid":"AccountBAccess1",
"Principal" : {
"AWS":"111122223333"
},
"Action":"s3:*",
"Resource":"arn:aws:s3:::mybucket/*"
}
}
When you do this on production setups, be more restrictive in the permissions. If your need is to copy from a bucket to another. Then on one side, you need to give only List and Get (not Put)
1
votes
In my case below mentioned command will work, hope so this will work for you as well. I have two different AWS accounts in different regions, and I want to copy my old bucket content into new one bucket. I have AWS CLI configured with two profiles.
Used the following aws cli command:
aws s3 cp --profile <profile1> s3://source_bucket_path/ --profile <profile2> s3://destination_bucket_path/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --recursive
