Downloading Azure Append Blob during upload results in error 412

0
votes

I have an upload server that appends a new block to an append blob every minute. It appends about 2MB of data for each block:

await blob.AppendBlockAsync(new MemoryStream(data)).ConfigureAwait(false);

While this is running, I need to be able to download the whole append blob at any given time. Let's assume after 20 hours, I need to download the whole append blob, while the upload server continues. 20 hours x 60 minutes x 2MB = 2,400MB. I download like this:

await blob.DownloadToStreamAsync(ms).ConfigureAwait(false);

The issue is that I cannot download 2,400MB in less than one minute. As a result, when the upload server appends another block, the DownloadToStreamAsync method throws an exception:

The remote server returned an error: (412) The condition specified using HTTP conditional header(s) is not met..

DownloadToStreamAsync() includes an overload that allows specifying a Microsoft.WindowsAzure.Storage.AccessCondition, however, it doesn't seem there are any options that will solve this. I'll mark your answer as correct if you can solve this in C# only (not using any tools like AZCopy, etc.). Thank you!

2
c#azureazure-storageazure-blob-storage
What about listing the blocks and download them one by one by yourself rather than simply call DownloadToStreamAsync? Looks like DownloadToStreamAsync automatically pass in ETag access condition to avoid inconsistency in downloaded data. – Zhaoxing Lu
Yes, that will work, but shouldn't there be an easier way? This seems like a common use case for append blobs. – BlueSky
Did the given answer solve your problem? I'm also struggling with this. – Gert Arnold

2 Answers

1
votes

You can snapshot the append blob, and read from the snapshot blob.

 snapshotBlob = await appendBlob.CreateSnapshotAsync();
 text = await snapshotBlob.DownloadTextAsync();
 await snapshotBlob.DeleteIfExistsAsync();
0
votes

According to the error message, it seems that blob content has been changed when try to download the blob content. The ETag of the blob will be changed automatically if the blob is changed. Please have a try to use the following code to check and figure it out. More detail info about the storage conditional operations, please refer to the document.

CloudAppendBlob appendBlob = container.GetAppendBlobReference("myAppendBlob");
appendBlob.FetchAttributes();
var etag = appendBlob.Properties.ETag;
try
{
    appendBlob.DownloadToStream(ms, AccessCondition.GenerateIfMatchCondition(etag));
}
catch (Exception)
{
    appendBlob.FetchAttributes();
    var updateEtag = appendBlob.Properties.ETag;
    Console.WriteLine($"Original:{etag},Updated:{updateEtag}");
    //To Do list
    //appendBlob.DownloadToStream(ms, AccessCondition.GenerateIfMatchCondition(updateEtag));
}

Here is a similar issue you could refer to it.