How do Azure Functions scale out?

28
votes

The scaling documentation for Azure Functions is a bit light on details for how Azure Functions decide when to add more instances of an app.

Say for example I have a function that is triggered by a Github webhook. 10,000 people simultaneously commit to the Github repo (with no merge conflicts ;) ), and Github calls my function 10,000 times in a very short period of time.

What can I expect to happen? Specifically,

  1. Will Azure Functions throttle the webhook calls? i.e., will Azure Functions reject certain function calls if my function app is under high load?
  2. Does Azure Functions queue the requests somehow? If so, where/how?
  3. How many instances of my function app will Azure Functions create in this scenario? One for each request (i.e., 10,000), and each will run in parallel?
  4. If my app function was scaled down to zero instances, because there was no load on it, could I expect to see some "warm-up time" before the first function is executed? Roughly how long?
2
azureazure-functionswebhooks

2 Answers

24
votes
  1. Azure Functions won't reject a webhook call, but in the case of sudden, extreme load, some requests may timeout. For web apis, please include retry on the client, as a best practice.
  2. They aren't queued in any persistent place. They are (implementation detail) managed by IIS.
  3. (Implementation detail) Number of instances isn't a hard set thing. We have certain, unpublished protections in place, but we're designed to scale quite far. Your requests will be handled by multiple instances.
  4. Yes. Right now, it's pretty hefty (seconds), but we'll be working to improve it. For perf sensitive situations, a canary or a timer trigger to keep it awake is recommended.

I'm from the Azure Functions team. The things I marked as implementation details aren't promises and will likely also change as we evolve our service; just an attempt at transparency.

6
votes
  1. tested today. it took more than seconds :(
ACTUAL PERFORMANCE
--------------
ClientConnected:  13:58:41.589

ClientBeginRequest:   13:58:41.592

GotRequestHeaders:    13:58:41.592

ClientDoneRequest:    13:58:41.592

Determine Gateway:    0ms

DNS Lookup:       65ms

TCP/IP Connect:   40ms

HTTPS Handshake:  114ms

ServerConnected:  13:58:41.703

FiddlerBeginRequest:  13:58:41.816

ServerGotRequest: 13:58:41.817

ServerBeginResponse:  14:00:36.790

GotResponseHeaders:   14:00:36.790

ServerDoneResponse:   14:00:36.790

ClientBeginResponse:  14:00:36.790

ClientDoneResponse:   14:00:36.790


Overall Elapsed:  **0:01:55.198**