Azure Web Apps Timeout long process request

1
votes

So, I have a web app running on Azure and there is a API endpoint which will handle a long process (about hour I can said). It is fine when running local on visual studio, but when it go to azure, it fail after request start 55 seconds.

I have research arount this issues over every platform, I alter a chances to add on this line of code on the end point.

System.Web.HttpContext.Current.Server.ScriptTimeout = 10000;

So now it able to extend to timeout, but still fail to process after 3 minute 50 second. Upon this point, I have find out there is other thing call Load Balancer. They all saying that the Azure Load Balancer will auto kill all request after 4 minutes. So I stuck on this place.

So I have try all these solution.

1.System.Web.HttpContext.Current.Server.ScriptTimeout = 10000; 2.System.Net.ServicePointManager.SetTcpKeepAlive(true, 30000, 30000); 3.Use Start New Thread to handle long process and return http status code to client initial that the process is started. 4.Make long process as another async function

What I trying to achieve is quite simple, a long process function that can be trigger by scheduler task or manually(AJAX call).

Any suggestion ?

3
c#ajaxasp.net-mvcazureweb-applications
I think you need to break down your description on what happens on API-client end, and then what happens on server end (or what you want to happen) I am curious to see more detail about approach 3. where you're returning immediately - why would this fail? – G. Stoynev
@G.Stoynev The client end point will be a AJAX call, and first design is to send the request to server then server return after this long process is finish, while the server end is start process entire flow (Check, add data to database, but record is huge). For approach 3 is just my stupid idea, so lets said the client didn't want to get result wheather the process is done or not, it get response from server and display "Process is start" to user. So the thread will like run on background, it no need response any result to user. Any Suggestion ? – Kira Hao
How about if the client initiates the Ajax call and the server return http accepted status if request has started to process but not completed. On client side the onsuccess handler can requery the server until it receives http OK status which indicates completed. Ofcourse client must know of sine way to identify the operation uniquely. – Frank Q.
Your "3rd" approach is most suitable for a typical web application. The way networks are implemented to serve HTTP traffic (talking underlying network infrastructure devices), you are very likely to bump into severe and impossible to resolve issues once your connection starts to look idle for extended period of time. I will elaborate in an answer – G. Stoynev
@FrankQ.that's my option for approach 3. Maybe is very similiar like u upload a file on onedrive, and it will show u the status and process bar. So I start the process and maybe there is a need for another api end point to return me the status of this process. Any reference you may see before? – Kira Hao

3 Answers

1
votes

I think that you hit the default timeout of Azure Web Apps (i think it is 3 minutes, if i remember correctly, but may be wrong). Could you set SCM_COMMAND_IDLE_TIMEOUT from the portal - web app settings => app settings => add the setting with the needed value (to say 360 (in seconds)). Reference. There is the functionality that will kill external operations when they are out of time (information is on the link above).

1
votes

So that is how I settle down in this process.

if that is a long run process, it shall be handle on web job, I have rewrite entire process into webjob by using Azure WebJob SDK on visual studio 2015.

Also, in order to speed up entire process (just to save cost on azure credit), I direct use DatabaseContext and make every required table runnig on ToList() with anonymous type. Finanlly, I get a list of item that needed to insert, and use BulkInsert.

Everything is done in 3 minutes Azure run time with 70k entries.

// Update for latest implementation

I am still using WebJob for some background process, but I speed up my process by using EntityFramework BulkInsert, along with Parallel.ForEach() for my logical process. Further more, I also use Parallel.ForEach() with using(var context=new DatabaseContext()){...} when deal with bulk update. (There is bulk.update library out there provide good solution too)

Besides, I do try Azure.Functions that seem to be replacement for Auzre.WebJob. It is faster trigger than WebJob by using same queue trigger.

For all background process, Azure come with few solutions for you, just share incase anyone need this.

  • WebJob
  • Functions
  • Logic Apps
0
votes

While I am not answering most of your questions, I chose to express an opinion that your "3." option, to return success status to AJAX client immediately is your first step in solving the issue.

Network infrastructure that implements the TCP/IP/HTTP traffic frowns on connections that seem idle for times longer than your typical web request. These behaviours are more often than not outside of your application (layer) - at lower, infrastructure layers. You may or may not have access to these and if you do you can "fix" your issue.

Using option 3. from your question, combined with polling or some form of push notification is your better solution IMO.