Async Await Deadlock even when running on Different Context

Question

Just as quick pre-text I am aware of what causes async await deadlock issues but am still having the problem. Hopefully I have just overlooked something simple.

I have an interesting problem where I am extending the save functionality of Entity Frameworks IdentityDBContext. I am extending this and overriding the methods.

int SaveChanges();
Task<int> SaveChangesAsync();
Task<int> SaveChangesAsync(CancellationToken)

The problem is that it is possible for any one of those calls to call an interface method on an object that returns an awaitable Task. This gets back into the whole running an async method synchronously. I have took precautions to avoid the deadlock but lets see some code so you can see the call chain.

The below is called from a UI button click event. Task.Run() is used to avoid a deadlock issue. At this point we are on the UI context and that is what it will block on with the .Wait()

public override int SaveChanges()
        {
            if (!preSaveExecuting)
            {
                preSaveExecuting = true;
                Task.Run(() => ExecutePreSaveTasks()).Wait();
                preSaveExecuting = false;
            }

            return base.SaveChanges();
        }

Now inside of the ExecutePreSaveTasks() function there is the following (useless code omitted for clarity.

private async Task ExecutePreSaveTask(){
    ValidateFields(); //Synchronous method returns void
    await CheckForCallbacks();
}

private async Task CheckForCallbacks(){
    //loop here that gets changed entities
    var eInsert = changedEntity.Entity as IEntityInsertModifier;
    var eUpdate = changedEntity.Entity as IEntityUpdateModifier;
    var eDelete = changedEntity.Entity as IEntityDeleteModifier;

    if (eInsert != null && changedEntity.State == EntityState.Added) await eInsert.OnBeforeInsert(this);
    if (eUpdate != null && changedEntity.State == EntityState.Modified) await eUpdate.OnBeforeUpdate(this);
    if (eDelete != null && changedEntity.State == EntityState.Deleted) await eDelete.OnBeforeDelete(this);
}

Now this part is the kicker. In one of the above "OnBeforeInsert" calls there is a call back to the DataContext to call "SaveChangesAsync" which gets awaited.

public async Task OnBeforeInsert(RcmDataContext context)
{
    await context.SaveChangesAsync();
    //some more code
}

Then finally in SaveChangesAsync

public override async Task<int> SaveChangesAsync()
{
    //some code that doesn't even run when this is called

    return await base.SaveChangesAsync();
}

Full call stack...

ButtonClick()
SaveChanges()
Task.Run(() ExecutePreSaveTasks()).Wait()
-->ValidateFields()
-->await CheckForCallbacks()
---->await object.OnBeforeInsert(this)
------>await SaveChangesAsync()
-------->await base.SaveChangesAsync()

This await never returns! Now my understanding is that when I call

Task.Run(Action)

That I am providing a new SynchronizationContext on which the callbacks can run. This will ensure that I do not get a deadlock condition. In fact I have debugged and verified that before I do Task.Run I am on the DispatcherSynchronizationContext and when I await the true async call in SaveChangesAsync that I am on a ThreadPool context (current context is null). However the deadlock still occurs?

Is the internal SaveChangesAsync call performing some special logic that is causing this or is my understanding flawed? Thank you to those who took the time to read and try to help.

p.s. I have also tried ConfigureAwait(false) on all Tasks just to see if it would help and it did not.

Since ExecutePreSaveTask is being executed via Task.Run, it should be running on a background thread and any awaits in that chain should await back to that thread and not the UI thread so that shouldn't cause a deadlock with the Wait. (famous last words); ConfigureAwait is probably useful here to avoid switching threads; but not a solution to your problem. Now, it's common to capture a context before a callback is called and use that context when invoking the callback. If that context is the UI context, that would explain the deadlock because the UI is blocked on Wait... — Peter Ritchie
The real solution is to never wait on the UI thread. The UI thread is a single threaded apartment and you have an implied and expected contract to not wait on that thread (it's more complicated than that; but I find it easier to just think "never wait on the UI thread"). — Peter Ritchie
Depending on you circumstances, getting rid of the Wait and disabling the button before the await and re-enabled after the await (in SaveChanges or the click handler if that calls SaveChanges) is the recommended approach. — Peter Ritchie
I know there is always a way around a problem such as this but my main goal is to understand why it is not obeying the rules I believe async await should be following. I am looking for the deeper cause of this issue so as to gain a better understanding of the implementation. I have trouble understanding why when .Wait() is called on a different context than the target async call why it would deadlock. I have succesfully used this approach without deadlock in other areas so think it may be tied to how the DbContext is doing the async call. — jpino
Well, await is complex and works under specific scenarios w.r.t. UI. None of which are really guaranteed to work if you block the UI. Anything that isn't tied directly to the UI should be using ConfigureAwait(false) because that code can't know the context and will cause problems in some scenarios without it. — Peter Ritchie

jpino jpino · Accepted Answer · 2015-03-27T22:54:56

Well a colleague of mine found the solution to the problem. This turned out to be an issue with Entity Framework and Caliburn.Micro BindableCollection.

The Caliburn.Micro collection fires property changed events on the UI thread whenever the collection changes. When saving the data via the data context Entity Framework was mutating the collection causing it to invoke events on the UI thread. Since the UI was busy waiting for the Task to complete it couldn't invoke the method and...deadlock.

I suppose the moral of the story is understand your 3rd party libraries. After switching to an ObservableCollection the problem went away.

Async Await Deadlock even when running on Different Context

1 Answers