Develop Reference

Task.WhenAll but process results one by one

Let's say I have a list of Tasks, and I want to run them in parallel. But I don't need all of them to finish to continue, I can move on with just one. The following code waits for all the tasks to finish to move on. Is there a way for me to individually continue with the task that has completed while waiting for the other ones to finish?
List<string>[] x = await Task.WhenAll(new Task<List<string>>[] { task1, task2 })
// When task1 finishes, I want to process the result immediately
// instead of waiting on task2.

You're probably looking for Task.WhenAny.
I've used it for setting off a pile of tasks and then processing each of them as they become ready, but I suppose you could also just wait for one to finish and continue without the loop if you don't care about dealing with the rest.
while(tasks.Count() > 0)
{
var task = await Task.WhenAny(tasks);
tasks.Remove(task);
var taskresult = await task;
// process result
}

If you are using C# 8 and .NET Core you can take advantage of IAsyncEnumerable to hide this complexity from the consuming side.
Just like this:
static async Task Main(string[] args)
{
await foreach (var data in GetData())
{
Console.WriteLine(data);
}
Console.ReadLine();
}
static async IAsyncEnumerable<string> GetData()
{
List<Task<string>> tasks = new List<Task<string>> {GetData1(), GetData3(), GetData2()};
while (tasks.Any())
{
var finishedTask = await Task.WhenAny(tasks);
tasks.Remove(finishedTask);
yield return await finishedTask;
}
}
static async Task<string> GetData1()
{
await Task.Delay(5000);
return "Data1";
}
static async Task<string> GetData2()
{
await Task.Delay(3000);
return "Data2";
}
static async Task<string> GetData3()
{
await Task.Delay(2000);
return "Data3";
}

You can use Task.WhenAny instead.
Example "stolen" from Stephen Cleary's Blog:
var client = new HttpClient();
string results = await await Task.WhenAny(
client.GetStringAsync("http://example.com"),
client.GetStringAsync("http://microsoft.com"));
// results contains the HTML for whichever website responded first.
Responding to comment
You absolutely can keep track of the other tasks:
// supposing you have a list of Tasks in `myTasks`:
while( myTasks.Count > 0 )
{
var finishedTask = await Task.WhenAny(myTasks);
myTasks.Remove(finishedTask);
handleFinishedTask(finishedTask); // assuming this is a method that
// does the work on finished tasks.
}
The only thing you'd have to watch out for is :
The returned task will always end in the RanToCompletion state with its Result set to the first task to complete. This is true even if the first task to complete ended in the Canceled or Faulted state.
Remarks in WhenAny Doks(Emphasis by me)

In case you want to process the results in order of the completion of the tasks, there is the OrderByCompletion extension method that does exactly that in Stephen Cleary's Nito.AsyncEx library, with the signature below:
// Creates a new collection of tasks that complete in order.
public static List<Task<T>> OrderByCompletion<T>(this IEnumerable<Task<T>> #this);
Usage example:
Task<string>[] tasks = new[] { task1, task2, task3, task4 };
foreach (var task in tasks.OrderByCompletion())
{
string result = await task;
// Do something with result
}
If you prefer not having external dependencies, the source code is here.

Based on the answer of Peter Csala, here a extension method for IAsyncEnumerable:
public static async IAsyncEnumerable<T> OrderedByCompletion<T>(this IEnumerable<Task<T>> tasks)
{
List<Task<T>> taskList = new List<Task<T>>(tasks);
while (taskList.Count > 0)
{
var finishedTask = await Task.WhenAny(taskList);
taskList.Remove(finishedTask);
yield return await finishedTask;
}
}

When does 'await' immediately leave a method and when does it not?

In programs utilizing async-await, my understanding is like this:
an async method that IS NOT awaited will run in the background (?) and rest of the code will continue to execute before this non-awaited method finishes
an async method that IS awaited will wait until the method finishes before moving on to the next lines of code
The application below was written by me to check if the above statements are correct.
using System;
using System.Threading.Tasks;
namespace ConsoleApp3
{
class Program
{
static async Task Main(string[] args)
{
Console.WriteLine("Hello World!");
DoJob();
var z = 3;
Console.ReadLine();
}
static async Task DoJob()
{
var work1 = new WorkClass();
var work2 = new WorkClass();
while (true)
{
await work1.DoWork(500);
await work2.DoWork(1500);
}
}
}
public class WorkClass
{
public async Task DoWork(int delayMs)
{
var x = 1;
await Task.Delay(delayMs);
var y = 2;
}
}
}
Here are some of my observations:
The DoJob(); call is not awaited. However, the debugger shows me that the code inside of DoJob is being executed, just as if it was a normal non-async method.
When code execution gets to await work1.DoWork(500);, I would think "OK, so maybe now the DoJob method will be left and var z = 3; will be executed? After all, 'await' should leave the method." In reality, it just goes into DoWork and doesn't leave DoJob - var z = 3; is still not executed.
Finally, when execution reaches await Task.Delay(delayMs);, DoJob is left, and the var z = 3; is reached. After that, code after the Delay is executed.
The things that I don't understand:
Why does await Task.Delay(delayMs); leave the DoJob method, but await work1.DoWork(500); does not?
I see that DoJob is executing normally. I thought it would be done in the background (maybe by one of the thread pool threads?). Looks like it could block the thread if it was some long-running method, am I right?

Why does await Task.Delay(delayMs); leave the DoJob method, but await work1.DoWork(500); does not?
Because this code:
await work1.DoWork(500);
is the same as this code:
var task = work1.DoWork(500);
await task;
So your code is calling the method first, and then awaiting the returned task. It's common to talk about await as "awaiting method calls", but that's not what actually happens - technically, the method call is done first (synchronously), and then the returned task is awaited.
I see that DoJob is executing normally. I thought it would be done in the background (maybe by one of the thread pool threads?).
No; with true asynchronous operations, there is no thread that is blocked on that operation.
Looks like it could block the thread if it was some long-running method, am I right?
Yes.
my understanding is like this
I recommend reading my async intro for a better mental framework. In summary:
async enables the await keyword. It also generates a state machine that handles creating the Task return value and stuff like that.
await operates on an "awaitable" (usually a task). First, it checks to see if it's already complete; if it is, the async method continues executing synchronously.
If the awaitable is not already complete, then await (by default) captures its context and schedules the continuation of the async method to run on that context when the awaitable completes.

The compiler splits the code in an async method in chunks. 1 before the first await and 1 between each await and 1 after the last await.
The execution will return to the caller at the first non completed awaiter or the end of the method.
This method will only return a completed Task after fully executed:
async Task M1() => await Task.CompletedTask;
This method will only return an incomplete Task that will complete when the Task returned by Task.Dealy(1000) is completed:
async Task M2() => await Task.Delay(1000);
Here's a small example:
static async Task Main(string[] args)
{
var t = TwoAwaits();
Console.WriteLine("Execution returned to main");
await t;
}
private static async Task TwoAwaits()
{
Console.WriteLine("Before awaits");
await Task.CompletedTask;
Console.WriteLine("Between awaits #1");
await Task.Delay(1000);
Console.WriteLine("Between awaits #2");
await Task.Delay(1000);
Console.WriteLine("After awaits");
}
/*
Before awaits
Between awaits #1
Execution returned to main
Between awaits #2
After awaits
*/

Let's look at the four possibilities:
(1)
void Main()
{
Console.WriteLine($"Main 0 - {Thread.CurrentThread.ManagedThreadId}");
DoJob();
Console.WriteLine($"Main 1 - {Thread.CurrentThread.ManagedThreadId}");
}
public static async Task DoJob()
{
Console.WriteLine($"DoJob 0 - {Thread.CurrentThread.ManagedThreadId}");
Thread.Sleep(2000);
Console.WriteLine($"DoJob 1 - {Thread.CurrentThread.ManagedThreadId}");
}
This outputs:
Main 0 - 14
DoJob 0 - 14
DoJob 1 - 14
Main 1 - 14
It has a 2 second pause after DoJob 0.
(2)
async Task Main()
{
Console.WriteLine($"Main 0 - {Thread.CurrentThread.ManagedThreadId}");
await DoJob();
Console.WriteLine($"Main 1 - {Thread.CurrentThread.ManagedThreadId}");
}
public static async Task DoJob()
{
Console.WriteLine($"DoJob 0 - {Thread.CurrentThread.ManagedThreadId}");
Thread.Sleep(2000);
Console.WriteLine($"DoJob 1 - {Thread.CurrentThread.ManagedThreadId}");
}
Again this outputs:
Main 0 - 14
DoJob 0 - 14
DoJob 1 - 14
Main 1 - 14
(3)
async Task Main()
{
Console.WriteLine($"Main 0 - {Thread.CurrentThread.ManagedThreadId}");
await DoJob();
Console.WriteLine($"Main 1 - {Thread.CurrentThread.ManagedThreadId}");
}
public static Task DoJob()
{
return Task.Run(() =>
{
Console.WriteLine($"DoJob 0 - {Thread.CurrentThread.ManagedThreadId}");
Thread.Sleep(2000);
Console.WriteLine($"DoJob 1 - {Thread.CurrentThread.ManagedThreadId}");
});
}
This has different output because it has changed thread:
Main 0 - 15
DoJob 0 - 13
DoJob 1 - 13
Main 1 - 13
And finally:
async Task Main()
{
Console.WriteLine($"Main 0 - {Thread.CurrentThread.ManagedThreadId}");
DoJob();
Console.WriteLine($"Main 1 - {Thread.CurrentThread.ManagedThreadId}");
}
public static Task DoJob()
{
return Task.Run(() =>
{
Console.WriteLine($"DoJob 0 - {Thread.CurrentThread.ManagedThreadId}");
Thread.Sleep(2000);
Console.WriteLine($"DoJob 1 - {Thread.CurrentThread.ManagedThreadId}");
});
}
This has a different output again:
Main 0 - 13
Main 1 - 13
DoJob 0 - 12
DoJob 1 - 12
In this last case it is not waiting for DoJob because DoJob is running on a different thread.
So if you follow the logic here the issue is that async/await doesn't create (or use) a different thread. The method called must do that.

Before Async & Await, there was two type of methods. Those who returned the result directly, and those who received a callback function as a parameter. On the latter, the method was invoked in the same thread syncronously and returned no value, and later, on the same or different thread your callback function would have been called with the result. Historically all I/O (disk, network, even memory) worked with callbacks (actually: interrupts) but medium to high level languages like C# would mask all that internally so end users don't need to learn/write low level code.
This worked pretty well up to a point, except this optimization wasted some physical resources. For example Node.js outperformed several other languages/server platforms by their limitation that forces the developers to use the callback model instead of the 'managed' mode.
This pushed C# and other languages to go back to the callback model, but the code readability really suffered (code callback spaguetti). So Async and Await was introduced.
Async and await let's you write in the 'callback model' with 'managed' syntax. All callbacks are handled by the compiler.
Each time you write 'await' in an async method, your method is actually split into two methods connected by a callback.
Now, you can write an async method that does regular sync code, without awaits, nor thread switch or I/O. That 'async' method will actually run synchronously. So, it is actually the same to await method1() or call without await. Why? because your async call is not awaiting anything, so your async code is still one piece of continous code.
If inside your method you await one, two or more different methods, then your method will be split into one, two or more pieces. And only the first piece will be guaranteed to be run synchronously. All the other pieces will run on other thread depending on the code that you are awaiting.
TL;DR;
Async/Await method is does not guarantees multi-threading or parallel processing. That will actually depend on the payload (the called async method). For example, http downloads will tipically be paralellized if you manage your awaits because those are functions that are mostly waiters of an external response. On the other side, heavy CPU processing, like compressing a file, will require other form of cpu/thread management not provided by async/await.
If you do not await an async method, your code will surely run synchronously up to the first await of the called method, given it has one. But later on, it may or not run sync.

why does await Task.Delay(delayMs); leave the DoJob method, but await work1.DoWork(500); does not?
Because, up and until there is an actual asynchronous call, it's still in the same context. If DoWork was just:
public async Task DoWork(int delayMs)
{
var x = 1;
var y = 2;
return Task.CompletedTask;
}
there would be no need for a continuation and hence, you would debug all the way without "jumping" back to the orignal await call.

Here is how your application could be remodeled if you were forced to avoid async/await for some reason. See how complex it gets to replicate the logic inside the loop. Async/await is really a gift from the heavens!
using System;
using System.Threading.Tasks;
namespace ConsoleApp3
{
class Program
{
static Task Main(string[] args)
{
Console.WriteLine("Hello World!");
DoJob();
var z = 3;
Console.ReadLine();
return Task.CompletedTask;
}
static Task DoJob()
{
var work1 = new WorkClass();
var work2 = new WorkClass();
var tcs = new TaskCompletionSource<bool>();
Loop();
return tcs.Task;
void Loop()
{
work1.DoWork(500).ContinueWith(t1 =>
{
if (t1.IsFaulted) { tcs.SetException(t1.Exception); return; }
work2.DoWork(1500).ContinueWith(t2 =>
{
if (t2.IsFaulted) { tcs.SetException(t2.Exception); return; }
if (true) { Loop(); } else { tcs.SetResult(true); }
// The 'if (true)' corresponds to the 'while (true)'
// of the original code.
});
});
}
}
}
public class WorkClass
{
public Task DoWork(int delayMs)
{
var x = 1;
int y;
return Task.Delay(delayMs).ContinueWith(t =>
{
if (t.IsFaulted) throw t.Exception;
y = 2;
});
}
}
}

Async method within Task.Factory.StartNew wait all tasks before continuing

I have the following code:
public async Task<bool> ExecuteAsync()
{
return await someresult;
}
public async Task DoSomethingAsync()
{
var tasks = new List<Task>();
foreach (var item in someList)
{
var task = Task.Factory.StartNew(async () =>
{
await ExecuteAsync();
});
tasks.Add(task);
}
await Task.WhenAll(tasks);
// The program execution should stop here and next code should be run when all tasks are completed
}
Basically I am in a foreach loop where for each item I start a new Task.
Once all tasks are started I want the code after await Task.WhenAll(tasks); to be executed once all tasks are completed. Of course all are async therefore I am wondering if this is even possible? Because all are async the execution doesnt stop at Task.WhenAll(tasks) but continue even so not all tasks are completed. How can I fix this?

First of all, why you're wasting thread pools for for each task in the loop, your code could be written as:
public Task<bool> ExecuteAsync()
{
return ItReturnsATask();
}
public async Task DoSomethingAsync()
{
var tasks = new List<Task>();
foreach (var item in someList)
{
//ExecuteAsync can be replace by ItReturnsATask if there's no extra processing
tasks.Add(ExecuteAsync());
}
await Task.WhenAll(tasks);
}
Your code misses the point of asynchrony, It creates couple of threads from thread pool, and use them, these threads wasted here. As for your actual question, anything after await Task.WhenAll(tasks) doesn't get executed before all tasks are finished (assuming we're capturing the current context).

Wait for only one task to complete

Let's say I have two tasks, with the following requirements:
Both are asynchronous.
Both run in parallel
The moment one of them completes I need to know which one did.
I came up with the following code, but it just hangs after both tasks get started (the WaitAny function never returns). I am also getting a squiggly line under Run function telling me to add await inside it, but VS complains when I try to add it in front of Task.WaitAny. Should I be wrapping WaitAny in another Task? What am I doing wrong?
async void Run()
{
Task task1 = Task1();
Task task2 = Task2();
int completedTaskIdx = Task.WaitAny(task1, task2);
Debug.WriteLine("completedTaskIdx = {0}", completedTaskIdx.ToString());
}
async Task Task1()
{
Debug.WriteLine("Task 1 Start");
await Task.Delay(5000);
Debug.WriteLine("Task 1 Stop");
}
async Task Task2()
{
Debug.WriteLine("Task 2 Start");
await Task.Delay(10000);
Debug.WriteLine("Task 2 Stop");
}

Don't block the UI thread when using asnyc/await, you will cause dedlocks. Your WaitAny() causes you to get a deadlock. Use WhenAny instead, you can use Array.IndexOf( to translate the returned task back in to the index.
async Task Run()
{
Task task1 = Task1();
Task task2 = Task2();
var tasks = new[] {task1, task2};
Task completedTask = await Task.WhenAny(tasks);
//It is a good idea to await the retuned task, this is the point a execption would
//be raised if the task finished with a exception.
await completedTask;
int completedTaskIdx = Array.IndexOf(tasks, completedTask);
//.ToString() will cause you to have a bug, you are calling the
//wrong overload of WriteLine. The correct overload will call .ToString() for you.
Debug.WriteLine("completedTaskIdx = {0}", completedTaskIdx);
}
I also fixed a bug in your Debug.WriteLine( call, you where calling this overload when you wanted this overload. I also replaced your async void with async Task, you should never do asnyc void unless you are using it to match a event handler signature.

C# tasks are executed before Task.WhenAll

Why the tasks are executed before Task.WhenAll??
If you see here, from the below code snippet, first Console.WriteLine("This should be written first.."); should be printed because I am awaiting the tasks beneath to it..
But if you see the output result, the Tasks method result is being printed before the above statement. Ideally, the tasks method should be executed when I await them, but it seems that- the tasks methods are executed the moment I add them in tasks list. Why is it so?
Would you please do let me know why is this happening??
Code:
public static async Task Test()
{
var tasks = new List<Task>();
tasks.Add(PrintNumber(1));
tasks.Add(PrintNumber(2));
tasks.Add(PrintNumber(3));
Console.WriteLine("This should be written first..");
// This should be printed last..
await Task.WhenAll(tasks);
}
public static async Task PrintNumber(int number)
{
await Task.FromResult(0);
Console.WriteLine(number);
}
Output

When you call an async method you get a "hot" task in return. That means that the task already started running (and maybe even completed) before you get to await them. That means that it's quite possible for the tasks to run and complete before the call to Task.WhenAll.
In your case however, while the PrintNumber is marked async it isn't asynchronous at all since you're using Task.FromResult. The synchronous part of an asynchronous method (which is the part until you await an asynchronous task) is always executed synchronously on the calling thread and is done before the call returns. When you use Task.FromResult you get a completed task so all your method is just the synchronous part and is completed before the call returns.

When you await a completed task (as is created by Task.FromResult, it completes synchronously. This means that in your example, nothing is actually happening asynchronously, which explains the order of execution.
If instead, you were to
await Task.Yield();
you'd see output more in line with your expectations.

Task.FromResult won't cause yield and the task will be executed on the same thread. To achieve what you want you can do this:
public static async Task Test()
{
var tasks = new List<Task>();
tasks.Add(PrintNumber(1));
tasks.Add(PrintNumber(2));
tasks.Add(PrintNumber(3));
Console.WriteLine("This should be written first..");
// This should be printed last..
await Task.WhenAll(tasks);
}
public static async Task PrintNumber(int number)
{
await Task.Yield();
Console.WriteLine(number);
}

If you want a Task or tasks to run after something else, its easiest to write your code accordingly.
public static async Task Test()
{
Console.WriteLine("This should be written first..");
// These should be printed last..
await Task.WhenAll(new[]
{
PrintNumber(1),
PrintNumber(2),
PrintNumber(3)
});
}
following on from your comment.
So we have some functions,
async Task<Customer> GetRawCustomer()
{
...
}
async Task<string> GetCity(Customer customer)
{
...
}
async Task<string> GetZipCode(Customer customer)
{
...
}
We could use them like this
var rawCustomer = await GetRawCustomer();
var populationWork = new List<Task>();
Task<string> getCity;
if (string.IsNullOrWhiteSpace(rawCustomer.City))
{
getCity = GetCity(rawCustomer);
populationWork.Add(getCity);
}
Task<string> getZipCode;
if (string.IsNullOrWhiteSpace(rawCustomer.City))
{
getZipCode = GetZipCode(rawCustomer);
populationWork.Add(getZipCode);
}
...
await Task.WhenAll(populationWork);
if (getCity != null)
rawCustomer.City = getCity.Result;
if (getZipCode != null)
rawCustomer.ZipCode = getZipCode.Result;