BlobTrigger in Azure WebJobs does not get triggered on files added after the WebJob is started

Question

When running the BlobOperations sample from the azure-webjobs-sdk-samples (1.0.0-rc1) (https://github.com/Azure/azure-webjobs-sdk-samples/tree/master/BasicSamples/BlobOperations) I have the following problem.

The BlobTriggers run fine when the WebJob is started:

Job host started
Executing: 'Functions.BlobNameFromQueueMessage' because New queue message 
    detected on 'persons'.
Executing: 'Functions.BlobToBlob' because New blob detected: 
    input/BlobOperations.txt
Executing: 'Functions.BlobTrigger' because New blob detected: 
    output/BlobOperations.txt

But when I add new files to the "input" (or "output") containers nothing happens, even after waiting for more than 10 minutes.

When I restart the WebJob, the files I uploaded do get picked up by the BlobTrigger!

This is the (unchanged) BlobTrigger from the samples:

public static void BlobToBlob([BlobTrigger("input/{name}")] TextReader input, 
    [Blob("output/{name}")] out string output)
{
    output = input.ReadToEnd();
}

The samples are using the latest version of the Azure Webjobs SDK:

<packages>
  <package id="Microsoft.Azure.WebJobs" version="1.0.0-rc1" targetFramework="net45" />
  <package id="Microsoft.Azure.WebJobs.Core" version="1.0.0-rc1" targetFramework="net45" />
  <package id="Microsoft.Data.Edm" version="5.6.0" targetFramework="net45" />
  <package id="Microsoft.Data.OData" version="5.6.0" targetFramework="net45" />
  <package id="Microsoft.Data.Services.Client" version="5.6.0" targetFramework="net45" />
  <package id="Microsoft.WindowsAzure.ConfigurationManager" version="1.8.0.0" targetFramework="net45" />
  <package id="Newtonsoft.Json" version="6.0.4" targetFramework="net45" />
  <package id="System.Spatial" version="5.6.0" targetFramework="net45" />
  <package id="WindowsAzure.Storage" version="4.0.1" targetFramework="net45" />
</packages>

I have not changed the code of the samples, except for the location of the packages (mine are in a shared location - D:\Development\Nuget.Packages) and the AzureWebJobsDashboard and AzureWebJobsStorage connection strings.

I have the same problem in a 'real' project that is about to go into Production.

Is this a bug, or am I doing something wrong?

Answer 1

BlobTrigger are not instant like queues. The SDK scans the blob container to detect new blobs or if existing blobs were updated and then triggers the functions listening on these blobs. The scan can take anywhere from seconds to minutes depending upon the size of the container. If your app has a requirement for instant processing then you should use queues and then bind to blobs. You can put the blob information such as container name and blob name as a queue message and then use the SDK model binding feature to bind to Blob attribute.

Answer 2

I have been having the same problem but have discovered that the trigger works for files uploaded through Azure Storage Explorer but never works when uploaded from Visual Studio. There may be something strange going off under the hood, but the only difference I can see is that Visual Studio is assigning the correct mime type where Azure Storage Explorer is defaulting to application/octet-stream. We have only been testing with MP4 files, if this makes any difference.

Whilst doing further testing using both Azure Storage Explorer and Visual Studio, we have noticed that it does not seem to be 100% reliable and has also missed uploads from Azure Storage Explorer.

I don't see how the problem could possibly be related to the nuget package issue described in another answer.

Answer 3

I found the cause of my problem: when I don't use a shared NuGet package location ("D:\Development\Nuget.Packages" in my case) it all works fine! The default location is "packages" directory in the root of the solution directory.

I assume that the presence of older version of packages in the shared packages folder (like 'Microsoft.Azure.WebJobs.0.5.0-beta' and 'Microsoft.Azure.WebJobs.Core.0.5.0-beta') are the cause of my problem, even though I haven't actually tested that. I any one wants me to, please let me know.

Update: I wanted to answer the question by Pranav Rastogi in the comment below, but when I reverted my Projects back to using "D:\Development\Nuget.Packages" (deleted packages folder, edited hintpaths in Project files, restarted VS) I could not replicate the problem: the BlobTrigger picked up files I uploaded to the "input" container like it's supposed to...