11
votes

I have an Azure WebJob that fails after only 20 minutes of running with the following log output:

[02/27/2015 00:30:11 > 3e9b72: SYS INFO] Status changed to Initializing
[02/27/2015 00:30:21 > 3e9b72: SYS INFO] Run script 'Namespace.Class.DataProcessor.exe' with script host - 'WindowsScriptHost'
[02/27/2015 00:30:21 > 3e9b72: SYS INFO] Status changed to Running
[02/27/2015 00:30:23 > 3e9b72: INFO] Found the following functions:
[02/27/2015 00:30:23 > 3e9b72: INFO] Namespace.Class.DataProcessor.Functions.ManualTrigger
[02/27/2015 00:30:23 > 3e9b72: INFO] Executing: 'Functions.ManualTrigger' because This was function was programmatically called via the host APIs.
[02/27/2015 00:51:35 > 3e9b72: ERR ] Thread was being aborted.
[02/27/2015 00:51:35 > 3e9b72: SYS INFO] WebJob process was aborted
[02/27/2015 00:51:35 > 3e9b72: SYS INFO] Status changed to Stopped
[02/27/2015 00:51:35 > 3e9b72: SYS INFO] Status changed to Failed
[02/27/2015 00:51:35 > 3e9b72: SYS ERR ] WebJob run failed due to: System.Threading.ThreadAbortException: Thread was being aborted.
   at Kudu.Core.Jobs.BaseJobRunner.RunJobInstance(JobBase job, IJobLogger logger, String runId)
   at Kudu.Core.Jobs.TriggeredJobRunner.<>c__DisplayClass2.<StartJobRun>b__0(Object _)

It is on schedule to start once a day when the system is quiet and after 20 minutes it fails with that error. This error only happens when the job is triggered automatically by the scheduler. If triggered manually from the Azure WebJob panel under the site there is no error even it it runs for many hours.

The job application outputs a '.' every few steps to keep Azure from thinking that it is idle. I've also configured the following settings on the website where the job runs which prevents it from timing out if it takes a while to respond or output something to the log:

WEBJOBS_IDLE_TIMEOUT = 14400 SCM_COMMAND_IDLE_TIMEOUT = 14400

The job action is a POST request to trigger the job which I suspect is the same mechanism that starts the job manually (and doesn't time out).

How can I prevent this from failing after 20 minutes? What setting do I need to change/add to allow the job to run for longer than 20 minutes?

1
Did you ever resolve this? I have the same issue. - jrummell
This appears to be related - stackoverflow.com/questions/27939766/… - jrummell
@jrummell My solution was to split the job into more focused functions and run them more frequently since it's the volume of built up data that causes it to take so long. Now functions don't take more than 10 minutes so they don't get shut down. Will test the "Always On" solution as well and then perhaps the "stopping_wait_time" settings as well and see if that works. - BrutalDev

1 Answers

-1
votes

Make sure you configure your website to run with the "Always On" setting enabled. This requires a standard level website (not available in free sites). From the documentation:

Always On. By default, web apps are unloaded if they are idle for some period of time. This lets the system conserve resources. In Basic or Standard mode, you can enable Always On to keep the app loaded all the time. If your app runs continuous web jobs, you should enable Always On, or the web jobs may not run reliably.