When running locally in the emulator the web worker works fine. However whenever I update my web worker running on an Azure VM I get the following exceptions exceptions in the event viewer and the role won't start:
Application: WaWorkerHost.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AggregateException
Stack: at System.Threading.Tasks.Task.Wait(Int32, System.Threading.CancellationToken) at System.Threading.Tasks.Task.Wait()
at Foo.PushProcess.WorkerRole.Run()
at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.StartRoleInternal() at Microsoft.WindowsAzure.ServiceRuntime.Implementation.Loader.RoleRuntimeBridge.b__2() at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
at System.Threading.ThreadHelper.ThreadStart()Inner Exception: A task was canceled.
Faulting application name: WaWorkerHost.exe, version: 2.6.1198.712, time stamp: 0x54eba731
Faulting module name: KERNELBASE.dll, version: 6.3.9600.17415, time stamp: 0x54505737
Exception code: 0xe0434352
Fault offset: 0x0000000000008b9c
Faulting process id: 0xfb8
Faulting application start time: 0x01d11e3128981a5d
Faulting application path: E:\base\x64\WaWorkerHost.exe
Faulting module path: D:\Windows\system32\KERNELBASE.dll
Report Id: 30631c5c-8a25-11e5-80c6-000d3a22f3ec
Faulting package full name:
Faulting package-relative application ID:
Session "MA_ETWSESSION_WAD_415df88f8a0447178dbd4c18f1349f0e_Foo.PushProcess_Foo.PushProcess_IN_0" failed to start with the following error: 0xC0000035
This is the relevant code:
public override void Run()
{
Trace.TraceInformation("Foo.PushProcess is running");
try
{
RunAsync(_cancellationTokenSource.Token).Wait(); // This is where the exceptions point to
}
catch (Exception ex)
{
Trace.TraceError("[WORKER] Run error: " + ex);
}
finally
{
_runCompleteEvent.Set();
}
}
public override bool OnStart()
{
// Set the maximum number of concurrent connections
ServicePointManager.DefaultConnectionLimit = 12;
// For information on handling configuration changes
// see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
bool result = base.OnStart();
_storageAccount = CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue("StorageConnectionString"));
var queueClient = _storageAccount.CreateCloudQueueClient();
_pushQueue = queueClient.GetQueueReference("pushes");
_pushQueue.CreateIfNotExists();
CreatePushBroker();
Trace.TraceInformation("Foo.PushProcess has been started");
return result;
}
private async Task RunAsync(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
Trace.TraceInformation("Working");
CloudQueueMessage message = null;
try
{
message = _pushQueue.GetMessage();
if (message != null)
{
ProcessItem(message);
}
}
catch (Exception ex)
{
if (message != null && message.DequeueCount > 5)
_pushQueue.DeleteMessage(message);
Trace.TraceError("[WORKER] Retrieval Failure: " + ex);
}
await Task.Delay(1000, cancellationToken);
}
}
Note some code has been omitted, however that is all run after the initialisation and in theory isn't reached by this exception.
I am completely at a loss as to what could cause this issue. Any help would be appreciated - even if only to aid me getting a helpful exception.
UPDATE
I have now reduced my code to the below - it is as simple as a web worker can possibly be - and I am still getting the exceptions. I believe that either the old worker is being cached, or there is an issue in the deployment procedure.
public override void Run()
{
Trace.TraceInformation("Foo.PushProcess is running");
try
{
RunAsync(_cancellationTokenSource.Token).Wait(); // This is where the exceptions point to
}
catch (Exception ex)
{
Trace.TraceError("[WORKER] Run error: " + ex);
}
finally
{
_runCompleteEvent.Set();
}
}
public override bool OnStart()
{
// Set the maximum number of concurrent connections
ServicePointManager.DefaultConnectionLimit = 12;
// For information on handling configuration changes
// see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
bool result = base.OnStart();
return result;
}
private async Task RunAsync(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
Trace.TraceInformation("Working");
// code removed for testing - no work is being done.
await Task.Delay(1000, cancellationToken);
}
}
