Best way to download many images into Azure App Service .Net Core app for processing

0
votes

I have over 500 large image files that I need to process in my .NET Core app hosted in an Azure App Service. That said, I need to download all of the images and run them through a machine learning categorization function in my code. I currently use blob storage as my mechanism for storing the images, but downloading all those images via blob rest api is slow. Is there a better architecture in Azure that I should be making use of to greatly increase performance of processing these images? Perhaps a storage mechanism much faster than blob storage?

1
azureazure-blob-storageazure-web-app-service
The blob storage should be the best choice, as well for downloading blobs, you can take a look at Azure Storage Data Movement Library. - Ivan Yang

1 Answers

0
votes

Yes, I tried at my side. Even the Storage Account is at the same location as my web app, it will take about 3-6 second to download a 30MB file. (In VM, it will only take less than 1 second)

My suggestions:

  1. You can zip your pictures into one archive file, and download it. It would be faster than downloading them one by one.

  2. You can use DownloadToFileParallelAsync method to download a file. It would be a little faster.

  3. You can refer to the official tutorial to Download large amounts of random data from Azure storage