Azure Functions host.json: maxPollingInterval

2
votes

If I read the documentation for maxPollingInterval:

The maximum interval between queue polls

From here: https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue#hostjson-settings

I have set it to 00:01:00 in my host.json. How I understand it is that I have told the functions runtime that it may not do a poll every 2. minute. It has to do it atleast every minute, but if the queue from which it is reading is "hot", that is, there is a consistent flow of messages, it will do it more often, than the minute i have specified. If the flow of messages drops, then the runtime will begin to check less often, until it hits the one minute.

Is this correctly understood?

I have tried to find the code for this pooling mechanism in the webjob sdk: https://github.com/Azure/azure-webjobs-sdk and Azure Functions runtime: https://github.com/Azure/Azure-Functions, but I don't know what to look after (and I haven't spend that much time on it yet).

2
azureazure-functionsazure-functions-runtime
why do you need to / want to change it in the first place? is the default behaviour not working correctly for your case? – silent
@silent i just try to understand the mechanism :) – mslot

2 Answers

4
votes

The queuetrigger implements a random exponential back-off algorithm. As a result of this if there are no messages on the queue, the SDK will back off and start polling less frequently.

MaxPollingInterval allows you to configure this behavior. It for when a queue remains empty, the longest period of time to wait before checking for a message to.

As an example, if you set it as 00:01:00 and then send a message to the queue. Function will be triggered and then wait for 2 seconds to check, if there was no message, wait for 4 seconds , no message, wait for 8 seconds and so on. If you send a message to queue and the function be triggered it will turn to wait for 2 seconds , 4 seconds, 8 seconds, ..., 1 minute, 1 minute, 1 minute ...until queuetrigger recive message.

This is the Offical doc, and below is a test i do. You can have a look of time interval of my test and have a try by yourself. 

2019-10-15T07:53:32.884 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=aead524a-cdba-4e43-a956-1a8558dea037)
2019-10-15T07:53:32.884 [Information] Trigger Details: MessageId: e8e9fa50-cda3-4653-bddb-c446807ec986, DequeueCount: 1, InsertionTime: 10/15/2019 7:53:32 AM +00:00
2019-10-15T07:53:32.886 [Information] C# Queue trigger function processed: 1
2019-10-15T07:53:32.886 [Information] Executed 'Function1' (Succeeded, Id=aead524a-cdba-4e43-a956-1a8558dea037)
2019-10-15T07:56:27.105 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=eb01dce6-23fd-4128-bccd-8c26c104361f)
2019-10-15T07:56:27.105 [Information] Trigger Details: MessageId: 9aa0a4ff-d823-4703-a8b4-dca0ff7b61cb, DequeueCount: 1, InsertionTime: 10/15/2019 7:55:54 AM +00:00
2019-10-15T07:56:27.106 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=d5b92a82-5159-4c28-91b0-c84668916f13)
2019-10-15T07:56:27.106 [Information] Trigger Details: MessageId: b9445cc1-8d0e-4f37-9de2-aafd74df4b82, DequeueCount: 1, InsertionTime: 10/15/2019 7:55:55 AM +00:00
2019-10-15T07:56:27.106 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=86d6abb4-9a41-4bac-9259-17d85afd031f)
2019-10-15T07:56:27.106 [Information] Trigger Details: MessageId: f019251e-6d79-448c-baba-13bfd8401494, DequeueCount: 1, InsertionTime: 10/15/2019 7:55:57 AM +00:00
2019-10-15T07:56:27.106 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=01d94b5c-230d-4ae0-adb6-70a949ec97b5)
2019-10-15T07:56:27.106 [Information] Trigger Details: MessageId: a00e1340-e0f7-4aee-a4a7-addb9a210831, DequeueCount: 1, InsertionTime: 10/15/2019 7:55:58 AM +00:00
2019-10-15T07:56:27.107 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=fd928cf3-c0a7-4fa1-88a1-e31eecf069bd)
2019-10-15T07:56:27.107 [Information] Trigger Details: MessageId: 6a8cc1b9-c45c-42e2-b7bf-34f7e96f5471, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:01 AM +00:00
2019-10-15T07:56:27.107 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=8e6de5fc-6696-4030-a2cc-748b5da44ec7)
2019-10-15T07:56:27.107 [Information] Trigger Details: MessageId: 0b100915-7de1-44b9-a706-2fd866169563, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:02 AM +00:00
2019-10-15T07:56:27.107 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=9f1a6e13-93e7-443c-a021-c0b9d5f4dcfa)
2019-10-15T07:56:27.108 [Information] Trigger Details: MessageId: 713159e1-99cd-4727-9793-7540297eb6d0, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:04 AM +00:00
2019-10-15T07:56:27.108 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=564ca50f-58a7-4475-865d-03706298c5fd)
2019-10-15T07:56:27.108 [Information] Trigger Details: MessageId: c9bc6646-15c4-42b8-ac48-44deeb056844, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:06 AM +00:00
2019-10-15T07:56:27.113 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=2d5083ee-e839-4bd1-a59d-3f13a7f04ae2)
2019-10-15T07:56:27.114 [Information] Trigger Details: MessageId: 4a92ad18-3e53-443d-a721-1e84ae77c112, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:08 AM +00:00
2019-10-15T07:56:27.114 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=b512c3bf-2d80-4e6c-ac6d-fef77f2c93ef)
2019-10-15T07:56:27.114 [Information] Trigger Details: MessageId: 5eace6a2-98e4-4aca-8747-ff2a2d9c5198, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:10 AM +00:00
2019-10-15T07:56:27.114 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=0febda35-e777-4449-b8dd-a0895daa403e)
2019-10-15T07:56:27.114 [Information] Trigger Details: MessageId: 5e7b9f35-afa7-4edd-bb8a-cb98c12c25f5, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:12 AM +00:00
2019-10-15T07:56:27.115 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=78e66fcb-45a5-4612-8dde-ff2c31be1585)
2019-10-15T07:56:27.115 [Information] Trigger Details: MessageId: 623a077d-4bfb-4321-b6c0-2ef475ad11f4, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:15 AM +00:00
2019-10-15T07:56:27.115 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=01afae82-451e-42a7-9ed8-346737efdd01)
2019-10-15T07:56:27.115 [Information] Trigger Details: MessageId: 243a1af2-238d-48b3-930f-0ba629aeb293, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:17 AM +00:00
2019-10-15T07:56:27.115 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=b6aa18b7-d52c-4b33-83ec-d380a5431de8)
2019-10-15T07:56:27.115 [Information] Trigger Details: MessageId: 71c5d964-215a-4a62-a12c-b0d2c2a15756, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:20 AM +00:00
2019-10-15T07:56:27.116 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=3d1c2ed3-d79e-4492-8ce2-3f6980bef396)
2019-10-15T07:56:27.116 [Information] Trigger Details: MessageId: 23121a01-ef82-4339-8df3-0d4fed9f0cab, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:23 AM +00:00
2019-10-15T07:56:27.116 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=72e3bd65-4c14-4b21-ad1a-9055ec0c7491)
2019-10-15T07:56:27.116 [Information] Trigger Details: MessageId: df15e792-fecb-45c9-91d3-73d493faea4f, DequeueCount: 1, InsertionTime: 10/15/2019 7:56:24 AM +00:00
2019-10-15T07:56:27.123 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.123 [Information] Executed 'Function1' (Succeeded, Id=eb01dce6-23fd-4128-bccd-8c26c104361f)
2019-10-15T07:56:27.123 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.124 [Information] Executed 'Function1' (Succeeded, Id=d5b92a82-5159-4c28-91b0-c84668916f13)
2019-10-15T07:56:27.124 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.124 [Information] Executed 'Function1' (Succeeded, Id=86d6abb4-9a41-4bac-9259-17d85afd031f)
2019-10-15T07:56:27.126 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.126 [Information] Executed 'Function1' (Succeeded, Id=01d94b5c-230d-4ae0-adb6-70a949ec97b5)
2019-10-15T07:56:27.132 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.132 [Information] Executed 'Function1' (Succeeded, Id=fd928cf3-c0a7-4fa1-88a1-e31eecf069bd)
2019-10-15T07:56:27.133 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.133 [Information] Executed 'Function1' (Succeeded, Id=8e6de5fc-6696-4030-a2cc-748b5da44ec7)
2019-10-15T07:56:27.134 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.134 [Information] Executed 'Function1' (Succeeded, Id=9f1a6e13-93e7-443c-a021-c0b9d5f4dcfa)
2019-10-15T07:56:27.135 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.135 [Information] Executed 'Function1' (Succeeded, Id=564ca50f-58a7-4475-865d-03706298c5fd)
2019-10-15T07:56:27.142 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.142 [Information] Executed 'Function1' (Succeeded, Id=2d5083ee-e839-4bd1-a59d-3f13a7f04ae2)
2019-10-15T07:56:27.143 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.143 [Information] Executed 'Function1' (Succeeded, Id=b512c3bf-2d80-4e6c-ac6d-fef77f2c93ef)
2019-10-15T07:56:27.144 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.144 [Information] Executed 'Function1' (Succeeded, Id=0febda35-e777-4449-b8dd-a0895daa403e)
2019-10-15T07:56:27.150 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.150 [Information] Executed 'Function1' (Succeeded, Id=78e66fcb-45a5-4612-8dde-ff2c31be1585)
2019-10-15T07:56:27.152 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.152 [Information] Executed 'Function1' (Succeeded, Id=01afae82-451e-42a7-9ed8-346737efdd01)
2019-10-15T07:56:27.153 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.153 [Information] Executed 'Function1' (Succeeded, Id=b6aa18b7-d52c-4b33-83ec-d380a5431de8)
2019-10-15T07:56:27.159 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.160 [Information] C# Queue trigger function processed: 1
2019-10-15T07:56:27.160 [Information] Executed 'Function1' (Succeeded, Id=72e3bd65-4c14-4b21-ad1a-9055ec0c7491)
2019-10-15T07:56:27.161 [Information] Executed 'Function1' (Succeeded, Id=3d1c2ed3-d79e-4492-8ce2-3f6980bef396)
2019-10-15T07:57:25.420 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=a0ba0110-b590-4770-856b-dcd0b9b2fe7e)
2019-10-15T07:57:25.420 [Information] Trigger Details: MessageId: 5203a3c7-30c6-41f3-95af-dafac74512c8, DequeueCount: 1, InsertionTime: 10/15/2019 7:57:21 AM +00:00
2019-10-15T07:57:25.420 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=cb9f04e9-931d-4e09-89b5-0f346043c637)
2019-10-15T07:57:25.421 [Information] Trigger Details: MessageId: 2210c791-0f56-4eae-809d-2060591a5a9b, DequeueCount: 1, InsertionTime: 10/15/2019 7:57:23 AM +00:00
2019-10-15T07:57:25.421 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=0be83ef6-6872-4ed1-bea9-a89ade1aa031)
2019-10-15T07:57:25.421 [Information] Trigger Details: MessageId: 8ec52c1d-584a-4c95-ae74-93beca0ea8dc, DequeueCount: 1, InsertionTime: 10/15/2019 7:57:25 AM +00:00
2019-10-15T07:57:25.422 [Information] C# Queue trigger function processed: 1
2019-10-15T07:57:25.422 [Information] Executed 'Function1' (Succeeded, Id=a0ba0110-b590-4770-856b-dcd0b9b2fe7e)
2019-10-15T07:57:25.423 [Information] C# Queue trigger function processed: 1
2019-10-15T07:57:25.423 [Information] Executed 'Function1' (Succeeded, Id=cb9f04e9-931d-4e09-89b5-0f346043c637)
2019-10-15T07:57:25.429 [Information] C# Queue trigger function processed: 1
2019-10-15T07:57:25.429 [Information] Executed 'Function1' (Succeeded, Id=0be83ef6-6872-4ed1-bea9-a89ade1aa031)
2019-10-15T07:57:27.424 [Information] Executing 'Function1' (Reason='New queue message detected on 'myqueue-items'.', Id=14583d91-ec35-42aa-83c8-0727388b663d)
2019-10-15T07:57:27.424 [Information] Trigger Details: MessageId: 0c55402f-b47b-476e-ae5c-f64d9756b65f, DequeueCount: 1, InsertionTime: 10/15/2019 7:57:26 AM +00:00
2019-10-15T07:57:27.425 [Information] C# Queue trigger function processed: 1
2019-10-15T07:57:27.425 [Information] Executed 'Function1' (Succeeded, Id=14583d91-ec35-42aa-83c8-0727388b663d)

Do you understand now?

1
votes

@BowmanZhu, just to complete/update your answer, it seems what in Microsoft.Azure.WebJobs version 3.0.16, the polling is triggered differently. Checking mechanism is starting from the QueuePollingIntervals.Minimum(not 2 seconds) which is 100ms and exponentially increment it until it reaches the maximum.

Note: for Development environment the MaxPollingInterval default value is not 1 minute, but 2 seconds.

MaxPollingInterval set to 15 seconds (15 000ms):


dbug: Microsoft.Extensions.Hosting.Internal.Host[2]
      Hosting started
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 100 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 199 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 336 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 548 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 916 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 1851 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 3199 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 6222 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 13030 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 15000 ms before polling queue 'stackoverflow'.
dbug: Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener[2]
      Function 'ProcessQueueMessage' will wait 15000 ms before polling queue 'stackoverflow'.
...