Download Azure Blob with Shared Access Signature using wget or curl

8
votes

I know I can use the Azure CLI to accomplish this, but I'd like to use common Linux/Unix commands.

I have an Azure Blob Container, which is private. Let's call it https://my-account.blob.core.windows.net/my-blob-container/my-file.my-extension.

If my container is set to "Public read access for blobs only", everything works great.

$ wget https://my-account.blob.core.windows.net/my-blob-container/my-file.my-extension
--2016-04-28 16:11:15-- https://my-account.blob.core.windows.net/my-blob-container/my-file.my-extension
Resolving my-account.blob.core.windows.net (my-account.blob.core.windows.net)... 4.11.2.14
Connecting to my-account.blob.core.windows.net (my-account.blob.core.windows.net)|4.11.2.14|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21682 (21K) [application/octet-stream]
Saving to: ‘my-file.my-extension’

If my container is set to "No public access", and I create an Access policy called my-policy, with Read/Write/Delete/List for one year, I get the following direct link: https://my-account.blob.core.windows.net/my-blob-container/my-file.my-extension?sv=2015-02-21&si=my-policy&sr=b&sig=y%2B2pvIuR9rGQdvm%2FQKp0xNCWE%2B1G%2F2IqjHkWMhRcPUA%3D

If I copy the link into a browser, the file downloads automatically. If I use wget, I get a 404 error:

$ wget https://my-account.blob.core.windows.net/my-blob-container/my-file.my-extension?sv=2015-02-21&si=my-policy&sr=b&sig=y%2B2pvIuR9rGQdvm%2FQKp0xNCWE%2B1G%2F2IqjHkWMhRcPUA%3D
[4] 14106
[5] 14107
[6] 14108
[2]   Done                    si=my-policy
[3]   Done                    sr=b
$ --2016-04-29 01:25:54--  https://my-account.blob.core.windows.net/my-blob-container/my-file.my-extension?sv=2015-02-21
Resolving my-account.blob.core.windows.net (my-account.blob.core.windows.net)... 4.11.2.14
Connecting to my-account.blob.core.windows.net (my-account.blob.core.windows.net)|4.11.2.14|:443... connected.
HTTP request sent, awaiting response... 404 The specified resource does not exist.
2016-04-29 01:25:54 ERROR 404: The specified resource does not exist..

wget returns Exit Stats 8 ("Server issued an error response").

1
curlwgetazure-storage-blobs
Hi, did my answer help? - Steph Locke
Thanks for your answer @step-locke. I have tried adding st, se, and sp as you suggest, but the error message is the same (HTTP/1.1 404 The specified resource does not exist.). It still puzzles me because a web browser has no difficulty downloading the file with exactly the same URL. - numeratus
I've added an update. It looks like the key diff between our cmd-lines causes the error - you're not quoting the url. When I don't quote the URL, it too generates a 404 error - Steph Locke
You are correct; everything works as long as the URL is quoted. Thank you for your help! - numeratus
sometimes you just need to put the address in quotes ("https:// ....") - Dror Hilman

1 Answers

16
votes

The query is difficult to debug without the specifics but...

You must make sure that the URL is enclosed in speech marks when making the wget call due to the nature of some of the URL encoding (it's the same if you're doing a curl)

I can create a SAS at the container level that allows me to read a file in storage

wget "https://azurerpkg.blob.core.windows.net/azurer/doggies.jpg?st=2016-06-06T07%3A56%3A00Z&se=2018-06-07T07%3A56%3A00Z&sp=rl&sv=2015-04-05&sr=c&sig=nKQz5BcWoUaASBzSW7Hv0TzfMbTFcf0f%2B5mMtmTwQ2A%3D" -k -O "doggies.jpg"

If I create a specific SAS at the blob level my wget also works

wget "https://azurerpkg.blob.core.windows.net/azurer/doggies.jpg?st=2016-06-06T07%3A56%3A00Z&se=2018-06-07T07%3A56%3A00Z&sp=rl&sv=2015-04-05&sr=b&sig=ntNFXS6fUlHVkiqoqj2rHg1Pw6gSNosnYqj3CQ6GIrg%3D" -k -O doggies.jpg

A valid shared access signature query string requires arguments for the following parameters:

  • st
  • se
  • sp
  • sr
  • sig

Yours appears to be missing the se component which is the expiry date.