Running multiple async tasks and waiting for them all to complete

Question

I need to run multiple async tasks in a console application, and wait for them all to complete before further processing.

There's many articles out there, but I seem to get more confused the more I read. I've read and understand the basic principles of the Task library, but I'm clearly missing a link somewhere.

I understand that it's possible to chain tasks so that they start after another completes (which is pretty much the scenario for all the articles I've read), but I want all my Tasks running at the same time, and I want to know once they're all completed.

What's the simplest implementation for a scenario like this?

Answer 1

Both answers didn't mention the awaitable Task.WhenAll:

var task1 = DoWorkAsync();
var task2 = DoMoreWorkAsync();

await Task.WhenAll(task1, task2);

The main difference between Task.WaitAll and Task.WhenAll is that the former will block (similar to using Wait on a single task) while the latter will not and can be awaited, yielding control back to the caller until all tasks finish.

More so, exception handling differs:

Task.WaitAll:

At least one of the Task instances was canceled -or- an exception was thrown during the execution of at least one of the Task instances. If a task was canceled, the AggregateException contains an OperationCanceledException in its InnerExceptions collection.

Task.WhenAll:

If any of the supplied tasks completes in a faulted state, the returned task will also complete in a Faulted state, where its exceptions will contain the aggregation of the set of unwrapped exceptions from each of the supplied tasks.

If none of the supplied tasks faulted but at least one of them was canceled, the returned task will end in the Canceled state.

If none of the tasks faulted and none of the tasks were canceled, the resulting task will end in the RanToCompletion state. If the supplied array/enumerable contains no tasks, the returned task will immediately transition to a RanToCompletion state before it's returned to the caller.

Answer 2

You could create many tasks like:

List<Task> TaskList = new List<Task>();
foreach(...)
{
   var LastTask = new Task(SomeFunction);
   LastTask.Start();
   TaskList.Add(LastTask);
}

Task.WaitAll(TaskList.ToArray());

Answer 3

The best option I've seen is the following extension method:

public static Task ForEachAsync<T>(this IEnumerable<T> sequence, Func<T, Task> action) {
    return Task.WhenAll(sequence.Select(action));
}

Call it like this:

await sequence.ForEachAsync(item => item.SomethingAsync(blah));

Or with an async lambda:

await sequence.ForEachAsync(async item => {
    var more = await GetMoreAsync(item);
    await more.FrobbleAsync();
});

Answer 4

You can use WhenAll which will return an awaitable Task or WaitAll which has no return type and will block further code execution simular to Thread.Sleep until all tasks are completed, canceled or faulted.

	WhenAll	WaitAll
Any of the supplied tasks completes in a faulted state	A task with the faulted state will be returned. The exceptions will contain the aggregation of the set of unwrapped exceptions from each of the supplied tasks.	An AggregateException will be thrown.
None of the supplied tasks faulted but at least one of them was canceled	The returned task will end in the TaskStatus.Canceled state	An AggregateException will be thrown which contains an OperationCanceledException in its InnerExceptions collection
An empty list was given	An ArgumentException will be thrown	The returned task will immediately transition to a TaskStatus.RanToCompletion State before it's returned to the caller.
	Doesn't block the current thread	Blocks the current thread

Example

var tasks = new Task[] {
    TaskOperationOne(),
    TaskOperationTwo()
};

Task.WaitAll(tasks);
// or
await Task.WhenAll(tasks);

If you want to run the tasks in a praticular order you can get inspiration from this anwser.

Answer 5

Do you want to chain the Tasks, or can they be invoked in a parallel manner?

For chaining
Just do something like

Task.Run(...).ContinueWith(...).ContinueWith(...).ContinueWith(...);
Task.Factory.StartNew(...).ContinueWith(...).ContinueWith(...).ContinueWith(...);

and don't forget to check the previous Task instance in each ContinueWith as it might be faulted.

For the parallel manner
The most simple method I came across: Parallel.Invoke Otherwise there's Task.WaitAll or you can even use WaitHandles for doing a countdown to zero actions left (wait, there's a new class: CountdownEvent), or ...

Answer 6

Yet another answer...but I usually find myself in a case, when I need to load data simultaneously and put it into variables, like:

var cats = new List<Cat>();
var dog = new Dog();

var loadDataTasks = new Task[]
{
    Task.Run(async () => cats = await LoadCatsAsync()),
    Task.Run(async () => dog = await LoadDogAsync())
};

try
{
    await Task.WhenAll(loadDataTasks);
}
catch (Exception ex)
{
    // handle exception
}

Answer 7

This is how I do it with an array Func<>:

var tasks = new Func<Task>[]
{
   () => myAsyncWork1(),
   () => myAsyncWork2(),
   () => myAsyncWork3()
};

await Task.WhenAll(tasks.Select(task => task()).ToArray()); //Async    
Task.WaitAll(tasks.Select(task => task()).ToArray()); //Or use WaitAll for Sync

Answer 8

There should be a more succinct solution than the accepted answer. It shouldn't take three steps to run multiple tasks simultaneously and get their results.

1. Create tasks
2. await Task.WhenAll(tasks)
3. Get task results (e.g., task1.Result)

Here's a method that cuts this down to two steps:

    public async Task<Tuple<T1, T2>> WhenAllGeneric<T1, T2>(Task<T1> task1, Task<T2> task2)
    {
        await Task.WhenAll(task1, task2);
        return Tuple.Create(task1.Result, task2.Result);
    }

You can use it like this:

var taskResults = await Task.WhenAll(DoWorkAsync(), DoMoreWorkAsync());
var DoWorkResult = taskResults.Result.Item1;
var DoMoreWorkResult = taskResults.Result.Item2;

This removes the need for the temporary task variables. The problem with using this is that while it works for two tasks, you'd need to update it for three tasks, or any other number of tasks. Also it doesn't work well if one of the tasks doesn't return anything. Really, the .Net library should provide something that can do this

Answer 9

I prepared a piece of code to show you how to use the task for some of these scenarios.

    // method to run tasks in a parallel 
    public async Task RunMultipleTaskParallel(Task[] tasks) {

        await Task.WhenAll(tasks);
    }
    // methode to run task one by one 
    public async Task RunMultipleTaskOneByOne(Task[] tasks)
    {
        for (int i = 0; i < tasks.Length - 1; i++)
            await tasks[i];
    }
    // method to run i task in parallel 
    public async Task RunMultipleTaskParallel(Task[] tasks, int i)
    {
        var countTask = tasks.Length;
        var remainTasks = 0;
        do
        {
            int toTake = (countTask < i) ? countTask : i;
            var limitedTasks = tasks.Skip(remainTasks)
                                    .Take(toTake);
            remainTasks += toTake;
            await RunMultipleTaskParallel(limitedTasks.ToArray());
        } while (remainTasks < countTask);
    }