Azure Data Factory Missing Blob Triggers

1
votes

I have created an ADF pipeline that should trigger when a blob is added to a storage container (say container1) and copy the blob to another storage container (say container2). All my blob names are alphanumeric with '-' (basically a GUID). I see that the ADF is triggered only a few times compared to the number of blobs in container1 (i.e if I have n files in container1, the ADF is triggered only x times where x<n).

I also observed that whenever the blobs created per second in container1 is high there are more missed triggers. I am not using any event batching in the event grid. My storage account is v2 BlockBlobStorage.

Is there a way I can resolve this?

1
azure-storage-blobsazure-data-factoryazure-blob-trigger
Have you filtered certain file types, so the number of triggers is less? – Joseph Xu
My blob trigger is only for blob created not blob deleted. But no blobs are deleted though. All the triggers are for created only – Sankar Mantripragada
Hi @Sankar Mantripragada,I've created several tests but unfortunately I couldn't reproduce your issues. Can you tell us any progress now? – Joseph Xu
Sometimes my storage can have around 100,000 blobs per hour, the triggers are missing if the inflow is that high. – Sankar Mantripragada

1 Answers

0
votes

I think it is difficult for us to get the correct answer in the community. We'd better move this issue to here. After MS's stress test to find out the possibility of bugs.