Azure website timing out after long process

Question

Team,

I have a Azure website published on Azure. The application reads around 30000 employees from an API and after the read is successful, it updates the secondary redis cache with all the 30,000 employees.

The timeout occurs in the second step whereby when it updates the secondary redis cache with all the employees. From my local it works fine. But as soon as i deploy this to Azure, it gives me a

500 - The request timed out.
The web server failed to respond within the specified time

From the blogs i came to know that the default time out is set as 4 mins for azure website.

I have tried all the fixes provided on the blogs like setting the command SCM_COMMAND_IDLE_TIMEOUT in the application settings to 3600.

I even tried putting the Azure redis cache session state provider settings as this in the web.config with inflated timeout figures. <add type="Microsoft.Web.Redis.RedisSessionStateProvider" name="MySessionStateStore" host="[name].redis.cache.windows.net" port="6380" accessKey="QtFFY5pm9bhaMNd26eyfdyiB+StmFn8=" ssl="true" abortConnect="False" throwOnError="true" retryTimeoutInMilliseconds="500000" databaseId="0" applicationName="samname" connectionTimeoutInMilliseconds="500000" operationTimeoutInMilliseconds="100000" />

The offending code responsible for the timeout is this: `

public void Update(ReadOnlyCollection<ColleagueReferenceDataEntity> entities) 
{ 
//Trace.WriteLine("Updating the secondary cache with colleague data"); 
var secondaryCache = this.Provider.GetSecondaryCache(); 

foreach (var entity in entities) 
{ 
try 
{ 
secondaryCache.Put(entity.Id, entity); 
} 
catch (Exception ex) 
{ 
// if a record fails - log and continue. 
this.Logger.Error(ex, string.Format("Error updating a colleague in secondary cache: Id {0}, exception {1}", entity.Id)); 
} 
} 
}

`

Is there any thing i can make changes to this code ?

Please can anyone help me...i have run out of ideas !

Sunny Sharma Sunny Sharma · Accepted Answer · 2019-02-12T04:46:13

You're doing it wrong! Redis is not a problem. The main request thread itself is getting terminated before the process is completed. You shouldn't let a request wait for that long. There's a hard-coded restriction on in-flight requests of 230-seconds max which can't be changed.

Read here: Why does my request time out after 230 seconds?

Assumption #1: You're loading the data on very first request from client-side!

Solution: If the 30000 employees record is for the whole application, and not per specific user - you can trigger the data load on app start-up, not on user request.

Assumption #2: You have individual users and for each of them you have to store 30000 employees data, on the first request from client-side.

Solution: Add a background job (maybe WebJob/Azure Function) to process the task. Upon request from client - return a 202 (Accepted with the job-status location in the header. The client can then poll for the status of the task at a certain frequency update the user accordingly!

Edit 1: For Assumption #1 - You can try batching the objects while pushing the objects to Redis. Currently, you're updating one object at one time, which will be 30000 requests this way. It is definitely will exhaust the 230 seconds limit. As a quick solution, batch multiple objects in one request to Redis. I hope it should do the trick!

UPDATE: As you're using StackExchange.Redis - use the following pattern to batch the objects mentioned here already.

Batch set data from Dictionary into Redis

The number of objects per requests varies depending on the payload size and bandwidth available. As your site is hosted on Azure, I do not thing bandwidth will be much of a concern

Hope that helps!

Azure website timing out after long process

1 Answers