I could upload a file to a private S3 bucket successfully using following command:
aws s3 cp "myfile.txt" "s3://myfolder/myfile.txt" --region=us-east-1 --output=json
I would like to issue a AWS CLI command to return me a temporary URL download for myfile.txt and does anyone know how to?
I googled and look like I have to do some signing to get temporary URL such as: http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html
aws cli now supports presign command. You can run
$ aws s3 presign s3://test-bucket/test-file.txt
https://test-bucket/test-file.txt?Expires=1499152189&Signature=some-sha
This will generate an url which you can share it with anyone to download that file in 3600 seconds.
You can change time period with --expires-in
$ aws s3 presign s3://test-bucket/test-file.txt --expires-in 600
The generated url will expire in 10 minutes.
You can read more about presign in aws cli docs.
You can use the following URL format:
https://<bucket-name>.s3.amazonaws.com/<object or key name>
or old style:
https://s3.amazonaws.com/<bucket-name>/<object or key name>
To have it accessible you need to allow public access to your object or attach an appropriate bucket policy.
For example the following bucket policy shows public access to bucket zzzyyy object 'yyyeee'
$ aws s3 get-object-acl --bucket zzzyyy --key yyyeee
{
"Owner": {
"DisplayName": "owner",
"ID": "Some hash of owner"
},
"Grants": [
{
"Grantee": {
"DisplayName": "owner",
"ID": "Some hash of owner"
},
"Permission": "READ"
},
{
"Grantee": {
"DisplayName": "owner",
"ID": "Some hash of owner"
},
"Permission": "WRITE"
},
{
"Grantee": {
"DisplayName": "owner",
"ID": "Some hash of owner"
},
"Permission": "READ_ACP"
},
{
"Grantee": {
"DisplayName": "owner",
"ID": "Some hash of owner"
},
"Permission": "WRITE_ACP"
},
{
"Grantee": {
"URI": "http://acs.amazonaws.com/groups/global/AllUsers"
},
"Permission": "READ"
},
{
"Grantee": {
"URI": "http://acs.amazonaws.com/groups/global/AllUsers"
},
"Permission": "READ_ACP"
}
]
}
You can see examples of bucket policies here:
http://docs.aws.amazon.com/AmazonS3/latest/dev/AccessPolicyLanguage_UseCases_s3_a.html
You can also use the S3 Console as seen here:
Alternatively you can take a look at https://github.com/minio/mc and give us feedback. We always wanted to have a CLI dropbox interface, of course not as GUI but scriptable and to be automated. So we ended up implementing a convenient command for mc to address two use cases.
curl form to upload files (Presigned POST Policy)Following examples address both use cases.
$ mc share
NAME:
mc share - Download and upload documents.
USAGE:
mc share command [arguments...]
COMMANDS:
download Generate URL to download documents.
upload Generate ‘curl’ command to upload files.
list List the shared URLs
These commands can be used in regular sense or also from with in a script. For scripting purposes mc provides --json output for all the commands.
mc share download$ mc share download https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3
https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJVA5BMMU2RHO6IOQ%2F20151109%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20151109T060251Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=3c69928bee38b7b465547f9fa97146ff606480bbead8042c0351840c578a2ceb
$ mc --json share download https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3 | jq .
{
"expiry": {
"days": 7
},
"downloadUrl": "https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJVA5BMMU2RHO6IOQ%2F20151109%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20151109T060618Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=48f7a521396ac8e36443cc86ce16c2619f9fa81371aad9a309375c933ea48d99",
"keyName": "https://s3.amazonaws.com/ferenginar/sur_2013-08-05T05_48_49-07_00.mp3"
}
mc share upload$ mc share upload --expire=2h --content-type=image/png https://s3.amazonaws.com/backup/2007-Mar-2/...
curl https://s3.amazonaws.com/backup -F x-amz-credential=AKIAJVA5BMMU2RHO6IOQ/20151109/us-east-1/s3/aws4_request -F x-amz-date=20151109T061202Z -F x-amz-signature=61c234dd17fc9e5888417ac4546b37d61755e551674a9617f33f5232ec53c8b6 -F Content-Type=image/png -F bucket=backup -F policy=eyJleHBpcmF0aW9uIjoiMjAxNS0xMS0wOVQwODoxMjowMi40MDZaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIkQ29udGVudC1UeXBlIiwiaW1hZ2UvcG5nIl0sWyJlcSIsIiRidWNrZXQiLCJiYWNrdXAiXSxbInN0YXJ0cy13aXRoIiwiJGtleSIsIjIwMDctTWFyLTIvIl0sWyJlcSIsIiR4LWFtei1kYXRlIiwiMjAxNTExMDlUMDYxMjAyWiJdLFsiZXEiLCIkeC1hbXotYWxnb3JpdGhtIiwiQVdTNC1ITUFDLVNIQTI1NiJdLFsiZXEiLCIkeC1hbXotY3JlZGVudGlhbCIsIkFLSUFKVkE1Qk1NVTJSSE82SU9RLzIwMTUxMTA5L3VzLWVhc3QtMS9zMy9hd3M0X3JlcXVlc3QiXV19 -F x-amz-algorithm=AWS4-HMAC-SHA256 -F key=2007-Mar-2/<FILE> -F file=@<FILE>
$ mc --json share upload --expire=2h --content-type=image/png https://s3.amazonaws.com/backup/2007-Mar-2/... | jq .
{
"expiry": {
"hours": 2
},
"uploadCommand": "curl https://s3.amazonaws.com/backup -F x-amz-signature=4b349a15e76ab403424ec6b1e289336f67c02c427ccc98522d6f654eee5b7de6 -F Content-Type=image/png -F bucket=backup -F policy=eyJleHBpcmF0aW9uIjoiMjAxNS0xMS0wOVQwODoxMzoyMS4zODFaIiwiY29uZGl0aW9ucyI6W1siZXEiLCIkQ29udGVudC1UeXBlIiwiaW1hZ2UvcG5nIl0sWyJlcSIsIiRidWNrZXQiLCJiYWNrdXAiXSxbInN0YXJ0cy13aXRoIiwiJGtleSIsIjIwMDctTWFyLTIvIl0sWyJlcSIsIiR4LWFtei1kYXRlIiwiMjAxNTExMDlUMDYxMzIxWiJdLFsiZXEiLCIkeC1hbXotYWxnb3JpdGhtIiwiQVdTNC1ITUFDLVNIQTI1NiJdLFsiZXEiLCIkeC1hbXotY3JlZGVudGlhbCIsIkFLSUFKVkE1Qk1NVTJSSE82SU9RLzIwMTUxMTA5L3VzLWVhc3QtMS9zMy9hd3M0X3JlcXVlc3QiXV19 -F x-amz-algorithm=AWS4-HMAC-SHA256 -F x-amz-credential=AKIAJVA5BMMU2RHO6IOQ/20151109/us-east-1/s3/aws4_request -F x-amz-date=20151109T061321Z -F key=2007-Mar-2/<FILE> -F file=@<FILE> ",
"keyName": "https://s3.amazonaws.com/backup/2007-Mar-2/..."
}
Looking forward on some feedback. Thanks
Know this is 8 months late, but I just found out you can use Eclipse with the AWS Toolkit for Eclipse (http://aws.amazon.com/eclipse/) to generate pre-signed URLs with a right-click on the target file, even lets you set the expy date....It's pretty slick.
I don't think there is any pre built cli command \sub command to get private bucket url, for pre-sign you can refer presign .
In case if you want a public url in private bucket, that is object read is public not bucket.
For public aws url, you can use something - I have written this in powershell, you can put this $profile
Function getAwsUrl{
[CmdletBinding(DefaultParameterSetName="S3Path")]
param(
[parameter (ParameterSetName="S3Path",mandatory=$true)]$s3path,
[parameter (ParameterSetName="BucketKey")]$bucket,
[parameter (ParameterSetName="BucketKey",mandatory=$true)]$key
)
if($s3path)
{
$splits = $s3path.Split('//')
$bucket =$splits[2]
$key = $splits[3]
}
else
{
#user input bucket and key
$bucket =$bucket
$key = $key
}
#using standard aws region "us-west-2"
Write-Output "https://$bucket.s3.us-west-2.amazonaws.com/$key"
}
Example usage:
Make sure you pass --acl as public-read
aws s3 cp C:\temp\dumy.txt s3://fpd-uploads/ --acl public-read
getAwsUrl -s3path s3://fpd-uploads/dumy.txt
get-help getAwsUrl
NAME
getAwsUrl
SYNTAX
getAwsUrl -s3path <Object> [<CommonParameters>]
getAwsUrl -key <Object> [-bucket <Object>] [<CommonParameters>]
ALIASES
None
REMARKS
None
