await async pattern and work stealing threads

Question

I am trying to understand the underlying mechanism of async/await pattern and I thought I got it after reading the outstanding following article Work-Stealing in .NET 4.0 by Jennifer Marsman. What I understand is

1 - There is the global queue for the thread pool, as well as local queues for each thread in the thread pool

2 - Request comes in, is in the global queue, and then thread1 (T1) from thread pool grabs the request.

3 - This request is an async\await method. Once the await keyword is hit, a bookmark (callback) is created wrapped in a task (given that the task is not complete), and this task is placed in the local queue of T1. T1 returns back to the pool

4 - When the task completes, if T1 is not busy, T1 will handle the request. But if T1 is busy, another thread (call it T2) might actually steal this task from T1's local queue

Here is where my question comes in. How is this prohibited? Everything I read suggests that async\await doesn't change threading context. See the link MSDN explanation of async\await This also makes sense since in an MVC app, a request is bound to a thread. Which means that if a request comes to an async action method, I expect both the initial and continuation task to be completed by the same thread pool thread. How is work stealing threads not interfering with this? Appreciate any insight.

Answer 1

There are three semi-independent systems at work here: the thread pool (with work-stealing queues), the ASP.NET request context, and async/await.

The thread pool works as you describe: each thread has its own queue but can steal from other threads' queues if necessary. But this actually has little to do with how async/await works on ASP.NET. For the most part, you can completely ignore how work stealing queues work because the logical abstraction is of a single thread pool with a single queue. The work stealing queues are just an optimization.

The ASP.NET request context manages things like HttpContext.Current, security, and culture. It is not tied to a specific thread, but only one thread is allowed within a context at a time. This pattern is true for old-style asynchronous requests as well as the new-style async requests. Note that a request is bound to a thread from beginning to end only for synchronous requests; this is not true for asynchronous requests (and never has been). The ASP.NET request context is implemented as a synchronization context - specifically, an instance of AspNetSynchronizationContext.

When your code awaits an incomplete Task, by default await will capture the current context (which is SynchronizationContext.Current unless it is null, in which case it is the current TaskScheduler). When the Task completes, then the async method is continued within that context. I describe this behavior in more detail on my blog. You can think of async/await as "thread agnostic"; that is, they don't necessarily resume on a different thread, nor do they necessarily resume on the same thread. They leave all threading decisions up to the captured context.

One other side note is that there are two different types of Tasks, Promise Tasks and Delegate Tasks (as I describe on my blog). Only Delegate Tasks actually have code to run and are queued to the thread pool at all. So, when the await decides to suspend its method, it has no code to run and nothing is queued at that time; rather, it sets up a callback (continuation) that will queue the rest of the method in the future.

When the awaited task completes, that callback/continuation is run, which queues the remainder of the async method to that captured context. In theory, this could queue it to the thread pool, but in reality there's a shortcut that is almost always taken: The thread completing the task is usually a thread pool thread itself, so it just enters the request context directly and then resumes executing the async method without actually having to queue it anywhere.

So in the vast majority of cases, work-stealing queues don't come into play at all. It really only happens when the thread pool is overloaded with work.

But do note that it is entirely possible (and common) to have an async handler start on one thread and continue on another thread. This is usually not a problem because the request context is preserved, but thread-affine constructs like thread-local variables will not work correctly.

Answer 2

The way that this is handled is that every thread has a global SyncronisationContext object associated with it. This object is responsible for handling how the task is resumed as different situations have different requirements. For example if your doing GUI work, for the suspension to be transparent the task has to be resumed on the same thread. However, if you are in thread that doesn't have a special context set up then default SyncronisationContext is used which resumes on the threadpool. So the entire threadpool shares one SyncronisationContext with any threads that haven't had one set up. But GUI threads and MVC action threads set up special SyncronisationContexts per thread to ensure that it is resumed on the same thread. It is also possible to set up a task to resume under a different SyncronisationContext Using ConfigureAwait.

TL;DR Work stealing can't cross between SyncronisationContext's but one SyncronisationContext can contain multiple threads to allow work stealing.