How do I queue up request received from blob trigger?

Question

I have a function app which triggers when blob is uploaded or updated. But in my requirement, my container might receive thousands of blobs at a time (customer will upload blobs to container in batch). In this case, as I know function app will run in parallel for each blobs in container. But I need to process them in queue. So, I am thinking to make use of Queue Storage here. Whenever a blob is uploaded let me add that blob name as a queue message to Queue storage via Blob Triggered Function App then I will have a Queue Triggered Function App which will trigger and process the blob in queue. Correct me if my approach is wrong? Also, I need to know is there any way to add queue message from function app.

Answer 1

You should use in your scenario Azure Event Grid push-pull event pattern, where the storage account will push the event directly to the storage queue subscriber handler.

This solution didn't require any coding.

Answer 2

Your approach is correct. I did the same in my application When multiple user uploads images on blob, I added blob name and url on storage Queue. You can create Azure function to process message from the queue but make sure your Azure function batchSize is 1

Azure function Storage Queue trigger defatult batch size is 16 and it will process 16 messages in parallel. host.json

Example API code to create a new entity and upload entity detail in Queue.

await this.adRepository.AddEntity(ad);           
        var adDto = this.mapper.Map<AzureSearchServiceDto>(ad);

        //Push data in Queue
        await this.storageService.PushMessageIntoQueue(AppConstraint.ImageVerificationQueue, adDto .ToJson());

Function To verify the image and Create thumbnail images

[FunctionName("ImageVerificationFunction")]
    public async Task Run([QueueTrigger("%ImageVerificationQueue%", Connection = "StorageConnection")]string message, ILogger log)
    {
        log.LogInformation($"ImageVerificationFunction processed: {message}");
        var azureSearchServiceDto = JsonConvert.DeserializeObject<AzureSearchServiceDto>(message);
        await this.imageVerificationService.Verify(azureSearchServiceDto);
    }

storageService is my custom service to send messages in Storage Queue