Develop Reference

Process a list of tasks without concurrency

Given this list:
var tasks = new List<Task>
{
MyMethod1,
MyMethod2,
MyMethod3
};
And these methods:
async Task MyMethod1()
{
await SomeOtherMethod1();
}
async Task MyMethod2()
{
await SomeOtherMethod2();
}
async Task MyMethod3()
{
await SomeOtherMethod3();
}
Is it possible to do this where each task completes in order (no concurrency):
foreach (var task in tasks)
{
await task;
}
I haven't been able to find any way or examples where this is done. foreach just fires them all off. There is await foreach, but Task doesn't contain a public instance or extension definition for GetAsyncEnumerator.

foreach doesn't start the tasks. By convention, tasks are returned "hot" - that is, already running. So, the very fact that you have a list of tasks implies they are already running concurrently.
If you want to create a list of asynchronous actions to execute in the future (i.e., asynchronous delegates), then you want to use a List<Func<Task>> instead of a List<Task>, and then you can foreach over each delegate, invoking it and awaiting the returned task.

Do u mean you want to wait for them to finish one by one by their order?
You can do something simple like a regular for loop (actually foreach should have the same results) and .Wait() or I didn't fully understand what you are trying to achieve...
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace TestStuff
{
class Program
{
static void Main(string[] args)
{
Task t1 = new Task(() => {
Thread.Sleep(2000);
Console.WriteLine("I'm T1! I'm done!");
});
Task t2 = new Task(() => {
Thread.Sleep(1000);
Console.WriteLine("I'm T2! I'm done!");
});
Task t3 = new Task(() => {
Thread.Sleep(500);
Console.WriteLine("I'm T3! I'm done!");
});
List<Task> tasks = new List<Task>() { t1, t2, t3 };
for (int i = 0; i < tasks.Count; i++)
{
tasks[i].Start();
tasks[i].Wait();
}
}
}
}

Thank you everyone for the comments and answers. The answers really got me to a solution. I upvoted both answers. I'm putting an answer here, because it updates the code in the question to a working version that I'm using.
var tasks = new List<Func<Task>>
{
async () => await MyMethod1,
async () => await MyMethod2,
async () => await MyMethod3
};
foreach (var task in tasks)
{
await Task.Run(task);
}

How do I "WaitAll" on multiple spawned Task ConfigureAwait tasks?

I have a C# console app that spawns multiple task threads. I need to wait for all of them to finish before exiting the app otherwise it closes all the threads. How can I do a ".WaitAll()" on the ConfiguredTaskAwaiter (I'm assuming that's what I want to watch)
foreach (var entityType in entityTypes)
{
// Create a task for each entity to thread the calls
var task = Task.Run(() => TestTaskAsync(entityType)).ConfigureAwait(false); // Long running process
var awaiter = task.GetAwaiter();
Action showComplete = () => OnComplete($"PROCESSING FOR ENTITY ({entityType.Id}) HAS COMPLETED.");
awaiter.OnCompleted(showComplete);
}
// Should not continue until all tasks have completed

Configure await does not change any property of the task itself, it just tells the compiler whether you care about your current context after you await the task. Therefore
var task = Task.Delay(1000);
// ...
await task.ConfigureAwait(false);
is identical to
var task = Task.Delay(1000).ConfigureAwait(false);
// ...
await task;
In your specific case, it makes no sense to configure await for tasks you're not actually planning to await. You should aggregate them and await them all together instead.
What you probably want to do is
var tasks = new List<Task>();
foreach (var type in types)
{
var task = Task.Run(() =>
{
TestTaskAsync(entityType);
OnComplete($"PROCESSING FOR ENTITY ({entityType.Id}) HAS COMPLETED.");
});
tasks.Add(task);
}
await Task.WhenAll(tasks).ConfigureAwait(false);
This tells the compiler that you do not care what context you will have after the await. Since you are not awaiting the individual tasks, it makes no sense to worry about context after awaiting those. You will only be awaiting a task that represents them all, that is the task returned by Task.WhenAll.
I also took the liberty of putting the continuation in the callback. If you do not want to do that for any reason, feel free to split it the way you have in your question, it will work the same way.

var taskList = new List<Task>();
foreach (var entityType in entityTypes)
{
....
taskList.Add(task);
}
Task.WaitAll(taskList.ToArray())
but you should then async version called WhenAll :
await Task.WhenAll(taskList)

How to bundle async tasks for Task.WhenAll?

I fire up some async tasks in parallel like the following example:
var BooksTask = _client.GetBooks(clientId);
var ExtrasTask = _client.GetBooksExtras(clientId);
var InvoicesTask = _client.GetBooksInvoice(clientId);
var ReceiptsTask = _client.GetBooksRecceipts(clientId);
await Task.WhenAll(
BooksTask,
ExtrasTask,
InvoicesTask,
ReceiptsTask
);
model.Books = BooksTask.Result;
model.Extras = ExtrasTask.Result;
model.Invoices = InvoicesTask.Result;
model.Receipts = ReceiptsTask.Result;
This results in a lot of typing. I searched the .Net Framework for a way to shorten this up. I imagine it to be lile this. I call the class Collector as I don't know how to name the concept.
var collector = new Collector();
collector.Bind(_client.GetBooks(clientId), out model.Books);
collector.Bind(_client.GetBooksExtras(clientId), out model.Extras);
collector.Bind(_client.GetBooksInvoice(clientId), out model.Invoices);
collector.Bind(_client.GetBooksRecceipts(clientId), out model.Receipts);
collector.Run();
Is this a valid approach? Is there something like that?

Personally, I prefer the code in the question (but using await instead of Result for code maintainability reasons). As noted in andyb952's answer, the Task.WhenAll is not required. I do prefer it for readability reasons; it makes the semantics explicit and IMO makes the code easier to read.
I searched the .Net Framework for a way to shorten this up.
There isn't anything built-in, nor (to my knowledge) any libraries for this. I've thought about writing one using tuples. For your code, it would look like this:
public static class TaskHelpers
{
public static async Task<(T1, T2, T3, T4)> WhenAll<T1, T2, T3, T4>(Task<T1> task1, Task<T2> task2, Task<T3> task3, Task<T4> task4)
{
await Task.WhenAll(task1, task2, task3, task4).ConfigureAwait(false);
return (await task1, await task2, await task3, await task4);
}
}
With this helper in place, your original code simplifies to:
(model.Books, model.Extras, model.Invoices, model.Receipts) = await TaskHelpers.WhenAll(
_client.GetBooks(clientId),
_client.GetBooksExtras(clientId),
_client.GetBooksInvoice(clientId),
_client.GetBooksRecceipts(clientId)
);
But is it really more readable? So far, I have not been convinced enough to make this into a library.

In this case I believe that the WhenAll is kind of irrelevant as you are using the results immediately after. Changing to this will have the same effect.
var BooksTask = _client.GetBooks(clientId);
var ExtrasTask = _client.GetBooksExtras(clientId);
var InvoicesTask = _client.GetBooksInvoice(clientId);
var ReceiptsTask = _client.GetBooksRecceipts(clientId);
model.Books = await BooksTask;
model.Extras = await ExtrasTask;
model.Invoices = await InvoicesTask;
model.Receipts = await ReceiptsTask;
The awaits will take care of ensuring you don't move past the 4 later assignments until the tasks are all completed

As pointed out in andyb952's answer, in this case it's not really needed to call Task.WhenAll since all the tasks are hot and running.
But, there are situations where you may still desire to have an AsyncCollector type.
TL;DR:
Async helper function usage example
async Task Async(Func<Task> asyncDelegate) =>
await asyncDelegate().ConfigureAwait(false);
AsyncCollector implementation, usage example
var collector = new AsyncCollector();
collector.Register(async () => model.Books = await _client.GetBooks(clientId));
collector.Register(async () => model.Extras = await _client.GetBooksExtras(clientId));
collector.Register(async () => model.Invoices = await _client.GetBooksInvoice(clientId));
collector.Register(async () => model.Receipts = await _client.GetBooksReceipts(clientId));
await collector.WhenAll();
If you're worried about closures, see the note at the end.
Let's see why someone would want that.
This is the solution that runs the tasks concurrently:
var task1 = _client.GetFooAsync();
var task2 = _client.GetBarAsync();
// Both tasks are running.
var v1 = await task1;
var v2 = await task2;
// It doesn't matter if task2 completed before task1:
// at this point both tasks completed and they ran concurrently.
The problem
What about when you don't know how many tasks you'll use?
In this scenario, you can't define the task variables at compile time.
Storing the tasks in a collection, alone, won't solve the problem, since the result of each task was meant to be assigned to a specific variable!
var tasks = new List<Task<string>>();
foreach (var translation in translations)
{
var translationTask = _client.TranslateAsync(translation.Eng);
tasks.Add(translationTask);
}
await Task.WhenAll(tasks);
// Now there are N completed tasks, each with a value that
// should be associated to the translation instance that
// was used to generate the async operation.
Solutions
A workaround would be to assign the values based on the index of the task, which of course only works if the tasks were created (and stored) in the same order of the items:
await Task.WhenAll(tasks);
for (int i = 0; i < tasks.Count; i++)
translations[i].Value = await tasks[i];
A more appropriate solution would be to use Linq and generate a Task that identifies two operations: the fetch of the data and the assignment to its receiver
List<Task> translationTasks = translations
.Select(async t => t.Value = await _client.TranslateAsync(t.Eng))
// Enumerating the result of the Select forces the tasks to be created.
.ToList();
await Task.WhenAll(translationTasks);
// Now all the translations have been fetched and assigned to the right property.
This looks ok, until you need to execute the same pattern on another list, or another single value, then you start to have many List<Task> and Task inside your function that you need to manage:
var translationTasks = translations
.Select(async t => t.Value = await _client.TranslateAsync(t.Eng))
.ToList();
var fooTasks = foos
.Select(async f => f.Value = await _client.GetFooAsync(f.Id))
.ToList();
var bar = ...;
var barTask = _client.GetBarAsync(bar.Id);
// Now all tasks are running concurrently, some are also assigning the value
// to the right property, but now the "await" part is a bit more cumbersome.
bar.Value = await barTask;
await Task.WhenAll(translationTasks);
await Task.WhenAll(fooTasks);
A cleaner solution (imho)
In this situations, I like to use a helper function that wraps an async operation (any kind of operation), very similar to how the tasks are created with Select above:
async Task Async(Func<Task> asyncDelegate) =>
await asyncDelegate().ConfigureAwait(false);
Using this function in the previous scenario results in this code:
var tasks = new List<Task>();
foreach (var t in translations)
{
// The fetch of the value and its assignment are wrapped by the Task.
var fetchAndAssignTask = Async(async t =>
{
t.Value = await _client.TranslateAsync(t.Eng);
});
tasks.Add(fetchAndAssignTask);
}
foreach (var f in foos)
// Short syntax
tasks.Add(Async(async f => f.Value = await _client.GetFooAsync(f.Id)));
// It works even without enumerables!
var bar = ...;
tasks.Add(Async(async () => bar.Value = await _client.GetBarAsync(bar.Id)));
await Task.WhenAll(tasks);
// Now all the values have been fetched and assigned to their receiver.
Here you can find a full example of using this helper function, which without the comments becomes:
var tasks = new List<Task>();
foreach (var t in translations)
tasks.Add(Async(async t => t.Value = await _client.TranslateAsync(t.Eng)));
foreach (var f in foos)
tasks.Add(Async(async f => f.Value = await _client.GetFooAsync(f.Id)));
tasks.Add(Async(async () => bar.Value = await _client.GetBarAsync(bar.Id)));
await Task.WhenAll(tasks);
The AsyncCollector type
This technique can be easily wrapped inside a "Collector" type:
class AsyncCollector
{
private readonly List<Task> _tasks = new List<Task>();
public void Register(Func<Task> asyncDelegate) => _tasks.Add(asyncDelegate());
public Task WhenAll() => Task.WhenAll(_tasks);
}
Here a full implementation and here an usage example.
Note: as pointed out in the comments, there are risks involved when using closures and enumerators, but from C# 5 onwards the use of foreach is safe because closures will close over a fresh copy of the variable each time.
It you still would like to use this type with a previous version of C# and need the safety during closure, the Register method can be changed in order to accept a subject that will be used inside the delegate, avoiding closures.
public void Register<TSubject>(TSubject subject, Func<TSubject, Task> asyncDelegate)
{
var task = asyncDelegate(subject);
_tasks.Add(task);
}
The code then becomes:
var collector = new AsyncCollector();
foreach (var translation in translations)
// Register translation as a subject, and use it inside the delegate as "t".
collector.Register(translation,
async t => t.Value = await _client.TranslateAsync(t.Eng));
foreach (var foo in foos)
collector.Register(foo, async f.Value = await _client.GetFooAsync(f.Id));
collector.Register(bar, async b => b.Value = await _client.GetBarAsync(bar.Id));
await collector.WhenAll();

Creating awaitable tasks using LINQ

I want to create a collection of awaitable tasks, so that I can start them together and asynchronously process the result from each one as they complete.
I have this code, and a compilation error:
> cannot assign void to an implicitly-typed variable
If I understand well, the tasks return by Select don't have a return type, even though the delegate passed returns ColetaIsisViewModel, I would think:
public MainViewModel()
{
Task.Run(LoadItems);
}
async Task LoadItems()
{
IEnumerable<Task> tasks = Directory.GetDirectories(somePath)
.Select(dir => new Task(() =>
new ItemViewModel(new ItemSerializer().Deserialize(dir))));
foreach (var task in tasks)
{
var result = await task; // <-- here I get the compilation error
DoSomething(result);
}
}

You shouldn't ever use the Task constructor.
Since you're calling synchronous code (Deserialize), you could use Task.Run:
async Task LoadItems()
{
var tasks = Directory.GetDirectories(somePath)
.Select(dir => Task.Run(() =>
new ItemViewModel(new ItemSerializer().Deserialize(dir))));
foreach (var task in tasks)
{
var result = await task;
DoSomething(result);
}
}
Alternatively, you could use Parallel or Parallel LINQ:
void LoadItems()
{
var vms = Directory.GetDirectories(somePath)
.AsParallel().Select(dir =>
new ItemViewModel(new ItemSerializer().Deserialize(dir)))
.ToList();
foreach (var vm in vms)
{
DoSomething(vm);
}
}
Or, if you make Deserialize a truly async method, then you can make it all asynchronous:
async Task LoadItems()
{
var tasks = Directory.GetDirectories(somePath)
.Select(async dir =>
new ItemViewModel(await new ItemSerializer().DeserializeAsync(dir))));
foreach (var task in tasks)
{
var result = await task;
DoSomething(result);
}
}
Also, I recommend that you do not use fire-and-forget in your constructor. There are better patterns for asynchronous constructors.

I know the question has been answered, but you can always do this too:
var serializer = new ItemSerializer();
var directories = Directory.GetDirectories(somePath);
foreach (string directory in directories)
{
await Task.Run(() => serializer.Deserialize(directory))
.ContinueWith(priorTask => DoSomething(priorTask.Result));
}
Notice I pulled out the serializer instantiation (assuming there are no side effects).

Concurrent execution of async methods

Using the async/await model, I have a method which makes 3 different calls to a web service and then returns the union of the results.
var result1 = await myService.GetData(source1);
var result2 = await myService.GetData(source2);
var result3 = await myService.GetData(source3);
allResults = Union(result1, result2, result3);
Using typical await, these 3 calls will execute synchronously wrt each other. How would I go about letting them execute concurrently and join the results as they complete?

How would I go about letting them execute in parallel and join the results as they complete?
The simplest approach is just to create all the tasks and then await them:
var task1 = myService.GetData(source1);
var task2 = myService.GetData(source2);
var task3 = myService.GetData(source3);
// Now everything's started, we can await them
var result1 = await task1;
var result1 = await task2;
var result1 = await task3;
You might also consider Task.WhenAll. You need to consider the possibility that more than one task will fail... with the above code you wouldn't observe the failure of task3 for example, if task2 fails - because your async method will propagate the exception from task2 before you await task3.
I'm not suggesting a particular strategy here, because it will depend on your exact scenario. You may only care about success/failure and logging one cause of failure, in which case the above code is fine. Otherwise, you could potentially attach continuations to the original tasks to log all exceptions, for example.

You could use the Parallel class:
Parallel.Invoke(
() => result1 = myService.GetData(source1),
() => result2 = myService.GetData(source2),
() => result3 = myService.GetData(source3)
);
For more information visit: http://msdn.microsoft.com/en-us/library/system.threading.tasks.parallel(v=vs.110).aspx

As a more generic solution you can use the api I wrote below, it also allows you to define a real time throttling mechanism of max number of concurrent async requests.
The inputEnumerable will be the enumerable of your source and asyncProcessor is your async delegate (myservice.GetData in your example).
If the asyncProcessor - myservice.GetData - returns void or just a Task without any type, then you can simply update the api to reflect that. (just replace all Task<> references to Task)
public static async Task<TOut[]> ForEachAsync<TIn, TOut>(
IEnumerable<TIn> inputEnumerable,
Func<TIn, Task<TOut>> asyncProcessor,
int? maxDegreeOfParallelism = null)
{
IEnumerable<Task<TOut>> tasks;
if (maxDegreeOfParallelism != null)
{
SemaphoreSlim throttler = new SemaphoreSlim(maxDegreeOfParallelism.Value, maxDegreeOfParallelism.Value);
tasks = inputEnumerable.Select(
async input =>
{
await throttler.WaitAsync();
try
{
return await asyncProcessor(input).ConfigureAwait(false);
}
finally
{
throttler.Release();
}
});
}
else
{
tasks = inputEnumerable.Select(asyncProcessor);
}
await Task.WhenAll(tasks);
}