Does Default Retry-Handler in Microsoft Graph SDK Checks Whether the Endpoint is Throttled Before Sending the Request?

Question

I understand that Graph SDK implements a default retry handler that can take care of retrying when 429 occurs. After going through RetryHandler.cs class on GitHub I can see that every response is being checked for 429 and if there is a 429 (Too Many Requests) response, it uses Retry-After header (if available) or exponential back-off to determine the time for which the Task will be delayed.

For my question please consider the following scenario:

1. I have an azure function that uses client credential flow
2. It can be triggered by another application (not the user)
3. I have a graph service client that is a static object and consumes graph
4. One of the requests ends up being throttled (429) and the Tasks is delayed
5. While that Task is waiting (being delayed), another requests comes to the application that consumes the same graph resource

Question: Will the Graph Service Client, considering that it is the same static object and the other task is being delayed, send the request again to the Graph without considering that the endpoint is being throttled?

Thanks

Answer 1

Every request is sent to graph endpoint regardless of current pending throttled requests or endpoint throttled state (it does not maintain any state about that). GraphClient essentially uses HttpClient and RetryHandler is just a http client delegating handler (concept). Also, regarding your point about static object, it does not block new requests while there is pending retry of a previous one (that's where async task scheduling comes handy). In fact HttpClient is intended to be instantiated once and re-used throughout the life of an application. Instantiating an HttpClient class for every request will exhaust the number of sockets available under heavy loads. This will result in SocketException errors.

If the service starts throttling because you have overwhelmingly high volume of requests (through GraphAPI is designed for high volume), you should consider revisiting your application first why you have such high number of Graph API calls from the application. If you have possibility to batch requests to graph api, consider leveraging that also. Check the throttling guidance of Graph API https://docs.microsoft.com/en-us/graph/throttling. Even after then, if you want to handle retry in custom way, you can use Polly which supports Circuit breaking and Bulkhead https://github.com/App-vNext/Polly

Hopefully that answers your question. If you have follow up questions, please let me know.