how to know when uploading multiple files to the azure storage blob is finished in ASP.NET

2
votes

We are using azure function to process the documents uploaded on azure storage blob. We need to send an email after successful uploading of all documents on blob(i.e. 50 file at one time) as uploading status. How we can get the status that all files are uploaded successfully on our blob?

3
asp.net-mvcazureazure-functionsazure-storage-blobsazure-triggers
You can also confirm it using any web debugging proxy tool(ex. Fiddler) to capture the response message from storage server. 201 Created status code will be returned if the blob has been uploaded successfully stackoverflow.com/questions/21175293/… – SumanthMarigowda-MSFT

3 Answers

0
votes

I don't believe there is a built-in feature in Azure that would provide you the status programmatically or raise an event. However, a possible solution would be as follows:

  1. Before uploading all the 50 files as a batch, create a JSON file which contains the names of all the files that would be uploaded. Let's call this JSON file as Batch List.
  2. Upload the Batch List file first and then upload all your files that you need to upload.
  3. Through a polling process, determine if all the files in the Batch List exist in the Blob storage. If not then ignore until the next time you are able to do so. Once you determine that all the files in the Batch List exist in the blob storage then send the email as per your requirement. Delete the Batch List file.

This is a basic concept. It obviously can be more sophisticated but I hope you get the point.

1
votes

Azure Functions has a neat extension that allows you to do exactly that, Azure Durable Functions. What you're looking for here is the monitor pattern, polling until certain conditions are met. Here's an example that checks the weather status and then send a SMS using Twilio https://docs.microsoft.com/en-gb/azure/azure-functions/durable/durable-functions-monitor.

The code is up on Github at https://github.com/Azure/azure-functions-durable-extension/tree/master/samples/csx - checkout the examples starting with E3.

0
votes

More details would be required before a proper solution/recommendation can be provided.

How are you processing these files?

  • Are you reacting to an event as outlined below?
  • Are you processing these files off a queue which is why you don't have visibility to all the files?

You can use one of the following approaches depending on your requirements:

The Blob storage trigger starts a function when a new or updated blob is detected. The blob contents are provided as input to the function.

The Event Grid trigger has built-in support for blob events and can also be used to start a function when a new or updated blob is detected. For an example, see the Image resize with Event Grid tutorial.

Use Event Grid instead of the Blob storage trigger for the following scenarios:

  • Blob storage accounts
  • High scale
  • Minimizing latency