"Operation was cancelled" exception is throwing for long running Azure indexer

Question

We are getting "Operation was cancelled" exception while Azure Indexer is running for larger records (around 2M+). Here are the log details -

"The operation was canceled. Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request. The I/O operation has been aborted because of either a thread exit or an application request "

We are running the indexer under thread. It is working for smaller records but for larger records (1M+), it is throwing Socket Exception.

Does anyone saw this error while running Azure Indexer for larger records (running for long time)?

(we have already increase httpclient timeout to maximum value for serviceClient object.)

Answer 1

This could happen because of happen because of excess http connections. Try to make your **HttpClient** static and see if anything improves. **HttpClient** timeout to maximum value is required to execute with maximum records.

You may also want to consider working to reduce your sql query time for best indexer performance. Also please share you code if possible.

Hope it helps.

Answer 2

Try set SearchServiceClient.HttpClient.Timeout to Timeout.InfiniteTimeSpan. You have to set the timeout before you send any request to Azure Cognitive Search.

client.HttpClient.Timeout = Timeout.InfiniteTimeSpan;