Upload to Azure Blob using SAS and REST

Question

I'm having trouble writing to an Azure Block Blob from C++ using a SAS (Shared Access Signature). I'm using the Blob REST API and Poco. The HTTP request returns error 404 (resource does not exist), but I can't figure out what I'm doing wrong.

I generate the SAS on the server in C# like this (seems to work fine):

CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("my-blob");
container.CreateIfNotExists();
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(40);
sasConstraints.Permissions = SharedAccessBlobPermissions.Write | SharedAccessBlobPermissions.List;
string sasContainerToken = container.GetSharedAccessSignature(sasConstraints);
return Request.CreateResponse(HttpStatusCode.OK, container.Uri + sasContainerToken);

In the Azure portal I can indeed see the Blob container being created as expected. I receive this SAS in C++ using an HTTP request. What I get looks like this (some names and signature replaced for security reasons):

https://myname.blob.core.windows.net/my-blob?sv=2012-02-12&se=2016-06-07T11%3A13%3A19Z&sr=c&sp=wl&sig=%%%%%%%%%%%%%%%%%%%%%%%

Then I try to create the file using Poco and the Blob REST API. That looks like this:

std::string cloudUrl = sasURI + "&restype=container";
std::string fileName = "fname.ext";
Poco::URI* uri = new Poco::URI(cloudUrl.c_str());
std::string* path = new std::string(uri->getPathAndQuery());
Poco::Net::HTTPSClientSession* session = new Poco::Net::HTTPSClientSession(uri->getHost(), uri->getPort());
std::string method = Poco::Net::HTTPRequest::HTTP_PUT;
Poco::Net::HTTPRequest* request = new Poco::Net::HTTPRequest(method, *path, Poco::Net::HTTPMessage::HTTP_1_1);
request->add("x-ms-blob-content-disposition", "attachment; filename=\"" + fileName + "\"");
request->add("x-ms-blob-type", "BlockBlob");
request->add("x-ms-meta-m1", "v1");
request->add("x-ms-meta-m2", "v2");
Poco::Net::HTTPResponse* httpResponse = new Poco::Net::HTTPResponse();
int fileContent = 42;
request->setContentLength(sizeof(int));
request->setKeepAlive(true);
std::ostream& outputStream = session->sendRequest(*request);
outputStream << fileContent;
std::istream &is = session->receiveResponse(*httpResponse);
Poco::Net::HTTPResponse::HTTPStatus status = httpResponse->getStatus();
std::ostringstream outString;
Poco::StreamCopier::copyStream(is, outString);
if (status != Poco::Net::HTTPResponse::HTTP_OK)
{
    Logger::log("Connection failed\nstatus:", status, "\nreason:", httpResponse->getReason(), "\nreasonForStatus:", httpResponse->getReasonForStatus(status), "\nresponseContent:", outString.str());
}

I've looked up here how the REST API works. I found here that when using a SAS I don't need to do regular authentication.

What am I doing wrong here? Why am I getting error 404?

Answer 1

I believe most of your code is correct, all you need to do is insert the file name in your SAS URL.

Now that I have seen this question more carefully, this is what is happening:

You're creating a SAS on a blob container (my-blob) and using this SAS to upload a file (let's call it fname.ext). However you're not including the file name in the SAS URL so Azure Storage Service is assuming that you're trying to upload a file called my-blob in a $root container so on the service side when Azure Blob Service tries to validate the SAS, it validates it against $root container. Because you created the SAS for my-blob container and Azure Service is using $root container, the SAS does not match and that's why you're getting 403 error.

What you need to do is insert the file name in your SAS URL. So your SAS URL (or Request URL) would be something like (notice that I added fname.ext there):

https://myname.blob.core.windows.net/my-blob/fname.ext?sv=2012-02-12&se=2016-06-07T11%3A13%3A19Z&sr=c&sp=wl&sig=%%%%%%%%%%%%%%%%%%%%%%%

Also, you don't need the following two lines of code:

request->add("x-ms-version", "2015-02-21");
request->add("x-ms-date", "2016-06-07");

As these are not really needed when using SAS.

Answer 2

I've finally figured out what was going wrong here. :)

There were two problems in the above code. The first is that the filename needed to be inserted into the URL, as Gaurav Mantri explained. This does the trick:

int indexOfQuestionMark = cloudUrl.find('?');
cloudUrl = cloudUrl.substr(0, indexOfQuestionMark) + "/" + fileName + cloudUrl.substr(indexOfQuestionMark);

The other problem is that I wasn't uploading enough bytes. sizeof(int) is 4 bytes while pushing 42 into a stream turns it into characters, making it only 2 bytes. The server then keeps waiting for the remaining 2 bytes. That makes this the correct line in the example code above:

request->setContentLength(2);

Also, it works without these three lines so I suppose they're not needed:

request->add("x-ms-blob-content-disposition", "attachment; filename=\"" + fileName + "\"");
request->add("x-ms-meta-m1", "v1");
request->add("x-ms-meta-m2", "v2");

Similarly, adding this doesn't seem needed: "&restype=container".

Finally, for writing the SharedAccessBlobPermissions.List rights aren't needed so those can be left out in SAS generation on the server side.

Answer 3

One possible reason for your error could be the request date being too old. You're setting the request date as Midnight UTC tonight. Azure Storage allows about 15 minutes of clock skewness. Request date/time being "too old" is one of the major reasons for this 403 error (apart from incorrect account key and expired token in case of a SAS).

This is how you're setting x-ms-date request header.

request->add("x-ms-date", "2016-06-07");

This header's value should be formatted in the following format:

request->add("x-ms-date", "Sun, 11 Oct 2009 21:49:13 GMT");

Usually in C# world, we would do a DateTime.UtcNow.ToString("R") to get the date/time in correct format.

Please change your code accordingly and see if that solves the problem.