How to process tasks as they complete -but each task requires different method to process task's result

Question

I'm using async/await to call few external APIs. All of them returns me a string value but in different format and requires their own processing. And I want to process the returned value as a task completes. I don't want to wait until all are completed and hence I'm using Task.WhenAny(). How can I process tasks as they complete and still use the correct "Process" method for each task as they complete?

I make some changes after my first post and here is the latest i have:

public async Task<List<string>> Get()
{
    var task1 = Method1Async();
    var task2 = Method1Async();
    var tasks = new List<Task<string>> {task1, task2};
    var results = new List<string>();
    while (tasks.Count > 0)
    {
        var justCompletedTask = await Task.WhenAny(tasks);//will not throw
        tasks.Remove(justCompletedTask);
        try
        {
           var result = await justCompletedTask;
           results.Add(result);
        }
        catch(Exception)
        { 
          //deal with it
        }
    }

    return results;
}

private async Task<string> Method1Async()
{
    //this may throw - something like forbidden or any other exception
    var task = _httpClient.GetStringAsync("api1's url here");
    var result = await Method1ResultProcessorAsync(task);
    return result;
}

private async Task<string> Method1ResultProcessorAsync(Task<string> task)
{
    //process task's result -if it successuflly completed and return that
    return await task; //for now
}

private async Task<string> Method2Async()
{
    //this may throw - something like forbidden or any other exception
    var task = _httpClient.GetStringAsync("api2's url here");
    var result = await Method2ResultProcessorAsync(task);
    return await task;
}

private async Task<string> Method2ResultProcessorAsync(Task<string> task)
{
    //This processing logic is entirely different from Method1ResultProcessor
    //process task's result -if it successfully completed and return that 
    return await task; //for now
}

I have two questions here:

Is this the right way to approach the problem?
How do i better handle exception here? This is very important so the failure of one should not fail the whole thing. As long as any of the methods succeed, it will be okay. But if all fails, I want to the Get method to throw.

Can you please tag with the language you are using. I never seen that before :-) — Klaus
Its a C# 5.0 feature async/await. Tag added. Thanks for point that I3arnon & Klaus — user3818435
Take a look on Task.ContinueWith msdn.microsoft.com/en-us/library/dd270696(v=vs.110).aspx — bohdan_trotsenko

svick svick · Accepted Answer · 2014-07-16T15:33:36

Since your processor methods already accept Task, you can just call them and they will asynchronously wait for their corresponding results:

public Task<string[]> Get()
{
    var task1 = Method1ResultProcessorAsync(Method1Async());
    var task2 = Method2ResultProcessorAsync(Method2Async());

    return Task.WhenAll(task1, task2);
}

Handling exceptions the way you describe will make this more complicated, but you can use something like:

public async Task<List<string>> Get()
{
    var task1 = Method1ResultProcessorAsync(Method1Async());
    var task2 = Method2ResultProcessorAsync(Method2Async());

    var tasks = new[] { task1, task2 };

    try
    {
        await Task.WhenAll(tasks);
    }
    catch {}

    var results = tasks.Where(t => t.Status == TaskStatus.RanToCompletion)
                       .Select(t => t.Result)
                       .ToList();

    if (results.Any())
        return results;

    // or maybe another exception,
    // since await handles AggregateException in a weird way
    throw new AggregateException(tasks.Select(t => t.Exception));
}

How to process tasks as they complete -but each task requires different method to process task's result

2 Answers