Azure search Query Throttling

Question

I am trying to understand the throttling behavior that occurs with azure search queries. We are currently running on 3 replicas and 1 partition which seems like it should be more than enough to deal with 14 requests/sec and yet we see that the ~1% queries are getting throttled. (I have attached an image below). Right now our users just see an error being returned for those cases.

Can I please get more info on why these queries might be getting throttled; Can we do something to prevent this kind of throttling; Is this to be expected and we need to implement error handling for throttled queries and implement some kind of retry mechanism. Are there any best practices to deal this issue. Thanks!

   [![Azure Query Metrics][1]][1]

Answer 1

Azure Search will start throttling requests when the its overloaded and the failed requests rate exceeded a certain threshold, meaning it’s running out of resources. You can learn more about Azure Search performance and optimization considerations here

Note that throttled requests have the throttle-reason response header that contains information about why the request was throttled. The best strategy is to use an exponential retry policy on responses with status code of 207 and 503.