Overcoming Azure Vision Read API TPS limits

1
votes

I am working on a system where we are calling Vision Read API for extracting the contents from raster PDF. Files are of different size, ranging from one page to several hundreds of pages.

Files are stored in Azure Blob and there will be function to push files to Read API once when all files are uploaded to blob. There could be hundreds of files.

Therefore when process starts large nuber of documents are expected to be sent for text extraction per second. But Vision API has limit of 10 transactions per second including read.

I am wondering what would be best approach? Some type of throttling or queue?

Is there any integration available (say with queue) from where the Read API will pull documents and is there any type of push notification available to notify about completion of read operation? How can I prevent timeouts due to exceeding 10 TPS limit?

1
azurecomputer-visionazure-functionsocr

1 Answers

2
votes

Per my understanding , there is 2 key points you want to know :

  1. How to overcome 10 TPS limit while you have lot of files to read.
  2. Looking for a best approach to get the Read operation status and result.

Your question is a bit broad,maybe I can provide you with some suggestions:

For Q1, Generally ,if you reach TPS limit , you will get a HTTP 429 response , you must wait for some time to call API again, or else the next call of API will be refused. Usually we retry the operation using something like an exponential back off retry policy to handle the 429 error:

2.1) You need check the HTTP response code in your code.

2.2) When HTTP response code is 429, then retry this operation after N seconds which you can define by yourself such as 10 seconds…

For example, the following is a response of 429. You can set your wait time as (26 + n) seconds. (PS: you can define n by yourself here, such as n = 5…)

{ "error":{ "statusCode": 429, "message": "Rate limit is exceeded. Try again in 26 seconds." } }

2.3) If step 2 succeed, continue the next operation.

2.4) If step 2 fail with 429 too, retry this operation after N*N seconds (you can define by yourself too) which is an exponential back off retry policy..

2.5) If step 4 fail with 429 too, retry this operation after NNN seconds…

2.6) You should always wait for current operation to succeed, and the Waiting time will be exponential growth.

For Q2,, As we know , we can use this API to get Read operation status/result. If you want to get the completion notification/result, you should build a roll polling request for each of your operation at intervals.I,e each 10s to send a check request.You can use Azure function or Azure automation runbook to create asynchronous tasks to check read operation status and once its done , handle the result based on your requirement.

Hope it helps. If you have any further concerns , pls feel free to let me know .