Let's say I have a list of Tasks, and I want to run them in parallel. But I don't need all of them to finish to continue, I can move on with just one. The following code waits for all the tasks to finish to move on. Is there a way for me to individually continue with the task that has completed while waiting for the other ones to finish?
List<string>[] x = await Task.WhenAll(new Task<List<string>>[] { task1, task2 })
// When task1 finishes, I want to process the result immediately
// instead of waiting on task2.
You're probably looking for Task.WhenAny.
I've used it for setting off a pile of tasks and then processing each of them as they become ready, but I suppose you could also just wait for one to finish and continue without the loop if you don't care about dealing with the rest.
while(tasks.Count() > 0)
{
var task = await Task.WhenAny(tasks);
tasks.Remove(task);
var taskresult = await task;
// process result
}
If you are using C# 8 and .NET Core you can take advantage of IAsyncEnumerable to hide this complexity from the consuming side.
Just like this:
static async Task Main(string[] args)
{
await foreach (var data in GetData())
{
Console.WriteLine(data);
}
Console.ReadLine();
}
static async IAsyncEnumerable<string> GetData()
{
List<Task<string>> tasks = new List<Task<string>> {GetData1(), GetData3(), GetData2()};
while (tasks.Any())
{
var finishedTask = await Task.WhenAny(tasks);
tasks.Remove(finishedTask);
yield return await finishedTask;
}
}
static async Task<string> GetData1()
{
await Task.Delay(5000);
return "Data1";
}
static async Task<string> GetData2()
{
await Task.Delay(3000);
return "Data2";
}
static async Task<string> GetData3()
{
await Task.Delay(2000);
return "Data3";
}
You can use Task.WhenAny instead.
Example "stolen" from Stephen Cleary's Blog:
var client = new HttpClient();
string results = await await Task.WhenAny(
client.GetStringAsync("http://example.com"),
client.GetStringAsync("http://microsoft.com"));
// results contains the HTML for whichever website responded first.
Responding to comment
You absolutely can keep track of the other tasks:
// supposing you have a list of Tasks in `myTasks`:
while( myTasks.Count > 0 )
{
var finishedTask = await Task.WhenAny(myTasks);
myTasks.Remove(finishedTask);
handleFinishedTask(finishedTask); // assuming this is a method that
// does the work on finished tasks.
}
The only thing you'd have to watch out for is :
The returned task will always end in the RanToCompletion state with its Result set to the first task to complete. This is true even if the first task to complete ended in the Canceled or Faulted state.
Remarks in WhenAny Doks(Emphasis by me)
In case you want to process the results in order of the completion of the tasks, there is the OrderByCompletion extension method that does exactly that in Stephen Cleary's Nito.AsyncEx library, with the signature below:
// Creates a new collection of tasks that complete in order.
public static List<Task<T>> OrderByCompletion<T>(this IEnumerable<Task<T>> #this);
Usage example:
Task<string>[] tasks = new[] { task1, task2, task3, task4 };
foreach (var task in tasks.OrderByCompletion())
{
string result = await task;
// Do something with result
}
If you prefer not having external dependencies, the source code is here.
Based on the answer of Peter Csala, here a extension method for IAsyncEnumerable:
public static async IAsyncEnumerable<T> OrderedByCompletion<T>(this IEnumerable<Task<T>> tasks)
{
List<Task<T>> taskList = new List<Task<T>>(tasks);
while (taskList.Count > 0)
{
var finishedTask = await Task.WhenAny(taskList);
taskList.Remove(finishedTask);
yield return await finishedTask;
}
}
Related
Task.WhenAll(params System.Threading.Tasks.Task[] tasks) returns Task, but what is the proper way to asquire task results after calling this method?
After awaiting that task, results can be acquired from the original task by awaiting it once again which should be fine as tasks are completed already. Also it is possible to get result using Task.Result property which is often considered not good practice
Task<TResult1> task1= ...
Task<TResult2> task2= ...
Task<TResult3> task3= ...
await Task.WhenAll(task1, task2, task3)
var a = task1.Result; // returns TResult1
var b = await task1; // also returns TResult1
Which one should I choose here and why?
If you really have only an IEnumerable<Task<TResult>> and the task will be created on-the-fly (e.g. due to a .Select()) you would execute your tasks two times.
So, be sure that you either give an already materialized collection to Task.WhenAll() or get the result from the return value of that method:
var someTasks = Enumerable.Range(1, 10).Select(i => { Task.Delay(i * 100); return i; });
// Bad style, cause someTasks is an IEnumerable created on-the-fly
await Task.WhenAll(someTasks);
foreach(var task in someTasks)
{
var taskResult = await task;
Console.WriteLine(taskResult);
}
// Okay style, cause tasks are materialized before waiting, but easy to misuse wrong variable name.
var myTasks = someTasks.ToList();
await Task.WhenAll(myTasks);
foreach(var task in myTasks)
{
Console.WriteLine(task.Result);
}
// Best style
var results = await Task.WhenAll(someTasks);
foreach(var result in results)
{
Console.WriteLine(result);
}
Update
Just read this in your question:
However, I could not find any overload that would return anything but Task.
This happens, if the collection of tasks you give to the Task.WhenAll() method don't share a common Task<T> type. This could happen, if you e.g. want to run two tasks in parallel, but both return a different value. In that case you have to materialize the tasks and afterwards check the results individually:
public static class Program
{
public static async Task Main(string[] args)
{
var taskOne = ReturnTwo();
var taskTwo = ReturnPi();
await Task.WhenAll(taskOne, taskTwo);
Console.WriteLine(taskOne.Result);
Console.WriteLine(taskTwo.Result);
Console.ReadKey();
}
private static async Task<int> ReturnTwo()
{
await Task.Delay(500);
return 2;
}
private static async Task<double> ReturnPi()
{
await Task.Delay(500);
return Math.PI;
}
}
Here is the overload that returns Task<TResult[]> - MS Docs
Example:
static async Task Test()
{
List<Task<string>> tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
var currentTask = GetStringAsync();
tasks.Add(currentTask);
}
string[] result = await Task.WhenAll(tasks);
}
static async Task<string> GetStringAsync()
{
await Task.Delay(1000);
return "Result string";
}
I have this code:
var task1 = operation1();
var task2 = operation2();
var result1 = await task1;
var result2 = await task2;
I do also handle UnobservedTaskException (by logging it). The problem that I face is that after task1 fails and first await results in exception, task2 completes in an error and then I have a log entry that I do not want to see as I will already log the first exception and at that point all subsequent exceptions are of no interest to me.
So I am wondering if there is a way to do something so that all tasks are "ignored" in a way after I get an exception.
I know I can use await Task.WhenAll, but the downside is that I have to wait for all exceptions to happen, though I know for sure that after first task results in exception, I don't need to wait for the other task to complete as the whole operation already failed.
Another possible solution is to write try/catch and cancel all tasks, but that's a bit cumbersome.
P.S.
The example is simplified, I do have multiple tasks running like that. So I am looking for a generic solution that would work for any number of tasks
As an alternative to Task.WhenAll you can use a progressive approach with Task.WhenAny.
When any of the Tasks finishes then you can examine the result and you can decide what to do next. (Please bear in mind that Task.WhenAny does not throw exception even it is awaited) The great thing about this approach is that you can easily add throttling (control the degree of parallelism) to this.
The basic implementation of progressive async for each
static async Task ProgressiveAsyncForEach(int degreeOfParallelism, params Task[] tasks)
{
var toBeProcessedTasks = new HashSet<Task>();
var remainingTasksEnumerator = tasks.GetEnumerator();
void AddNextTask()
{
if (!remainingTasksEnumerator.MoveNext()) return;
var nextTaskToProcess = (Task)remainingTasksEnumerator.Current;
toBeProcessedTasks.Add(nextTaskToProcess);
}
//Initialize
while (toBeProcessedTasks.Count < degreeOfParallelism)
{
AddNextTask();
}
while (toBeProcessedTasks.Count > 0)
{
var processedTask = await Task.WhenAny(toBeProcessedTasks).ConfigureAwait(false);
if (!processedTask.IsCompletedSuccessfully)
{
Console.WriteLine("One of the task has failed");
//TODO: log first failed task
//CONSIDER: cancel all the remaining tasks
return;
}
toBeProcessedTasks.Remove(processedTask);
AddNextTask();
}
}
Sample usage
static async Task Main(string[] args)
{
await ProgressiveAsyncForEach(2, Faulty(), Fast(), Slow());
Console.WriteLine("Application finished");
}
static async Task Slow()
{
Console.WriteLine("Slow started");
await Task.Delay(1000);
Console.WriteLine("Slow finished");
}
static async Task Fast()
{
Console.WriteLine("Fast started");
await Task.Delay(500);
Console.WriteLine("Fast finished");
}
static async Task Faulty()
{
Console.WriteLine("Faulty started");
await Task.Delay(700);
throw new Exception();
}
I want to ask you: I have code:
var task1 = await _connectionService.ValidateUriAsync(uri1);
OutputResult("ss", task1);
var task2 = await _connectionService.ValidateUriAsync(uri2);
OutputResult("bb", task2);
var task3 = await _connectionService.ValidateUriAsync(uri3);
OutputResult("cc", task3);
Now I'm waiting until each task finishes and then I output the result. But I would like to run all tasks independently (I know how to do that). But what I don't know is, when some task is completed I need to output result for each task. If task fails the output will be - task1 failed or Task1 success.
I tried this solution, but I will have to check which task is completed and than its result (true/false). It is complex. If I had 100 tasks, I cannot have 100 conditions.
var tasks = new[] {task1, task2, task3};
var process = tasks.Select(async task =>
{
var result = await task;
if(task == task1)assign property
});
await Task.WhenAll(proces);
EDIT:
Here is ValidateUriAsync func:
public async Task<bool> ValidateUriAsync(Uri uri)
{
try
{
var request = WebRequest.CreateHttp(uri);
var result = await request.GetResponseAsync();
return true;
}
catch (Exception e)
{
return false;
}
}
when some task is completed I need to output result for each task.
Don't think about this in terms of "reacting to tasks as they complete". Instead, think of your ValidateUriAsync method as an "operation", and what you want is to create a new higher-level "operation" that is "validate and assign".
With that mindset, the solution is more clear - introduce a new async method for the new operation:
private async Task ValidateAndOutputResult(Uri uri, string name)
{
var result = await _connectionService.ValidateUriAsync(uri);
OutputResult(name, result);
}
Now you can call the higher-level method, and use Task.WhenAll:
var tasks = new[]
{
ValidateAndOutputResult(uri1, "ss"),
ValidateAndOutputResult(uri2, "bb"),
ValidateAndOutputResult(uri3, "cc"),
};
await Task.WhenAll(tasks);
Imagine the following scenario :
public async Task DoMultipleWork() {
var uploadTask = UploadAsync(file);
var processingTask = Task.Run( () => DoCpuWork() );
await Task.WhenAll(uploadTask, processingTask);
Console.WriteLine("upload is done");
Console.WirteLine("processing is done");
}
How can I change that code so that it doesn't matter which one ends first, it execute some particular (sync or async) code.
So I fire the both task and when taskA or taskB ends, I just run some code (sync or async) independently of the other.
I think maybe ContinueWith but I'm not sure because it needs an another async method which is not really needed.
EDIT
As suggested by comments on answer, I want to clear that I want to execute different code depending on the task that completes, and get both Console.WriteLine executed as soon as the original task completes.
You want to use Task.WhenAny which returns the first task that completes. You can then tell which task completed by comparing to the original tasks. Before returning you should wait for the other one to complete explicitly (or wait for both with Task.WhenAll):
public async Task DoMultipleWork()
{
var uploadTask = UploadAsync(file);
var processingTask = Task.Run( () => DoCpuWork() );
var completedTask = await Task.WhenAny(uploadTask, processingTask);
Console.WriteLine("upload or processing is done");
if (completedTask == uploadTask)
{
// Upload completed
}
else
{
// Processing completed
}
await Task.WhenAll(uploadTask, processingTask) // Make sure both complete
Console.WriteLine("upload and processing are done");
}
As I describe on my blog, ContinueWith is dangerous unless you explicitly pass a scheduler. You should use await instead of ContinueWith in ~99% of cases (more detail in another blog post).
In your case:
private async Task UploadAsync(string filepath)
{
var result = await fileManager.UploadAsync(filepath);
Console.WriteLine($"Result from uploading file {result}");
}
private async Task ProcessAsync(string filepath, IProgress<T> progress)
{
await Task.Run(() => wordProcessor.Process(filepath, progress));
Console.WriteLine("processing completed");
}
...
await Task.WhenAll(UploadAsync(filepath), ProcessAsync(filepath, processingProgress));
public async Task DoMultipleWork() {
var uploadTask = UploadAsync(file);
var processingTask = Task.Run( () => DoCpuWork() );
uploadTask.ContinueWith((Task t) => Console.WriteLine("YOUR_MESSAGE"), TaskContinuationOptions.OnlyOnRanToCompletion);
processingTask.ContinueWith((Task t) => Console.WriteLine("YOUR_MESSAGE"), TaskContinuationOptions.OnlyOnRanToCompletion);
await Task.WhenAll(new []{uploadTask, processingTask});
}
Why the tasks are executed before Task.WhenAll??
If you see here, from the below code snippet, first Console.WriteLine("This should be written first.."); should be printed because I am awaiting the tasks beneath to it..
But if you see the output result, the Tasks method result is being printed before the above statement. Ideally, the tasks method should be executed when I await them, but it seems that- the tasks methods are executed the moment I add them in tasks list. Why is it so?
Would you please do let me know why is this happening??
Code:
public static async Task Test()
{
var tasks = new List<Task>();
tasks.Add(PrintNumber(1));
tasks.Add(PrintNumber(2));
tasks.Add(PrintNumber(3));
Console.WriteLine("This should be written first..");
// This should be printed last..
await Task.WhenAll(tasks);
}
public static async Task PrintNumber(int number)
{
await Task.FromResult(0);
Console.WriteLine(number);
}
Output
When you call an async method you get a "hot" task in return. That means that the task already started running (and maybe even completed) before you get to await them. That means that it's quite possible for the tasks to run and complete before the call to Task.WhenAll.
In your case however, while the PrintNumber is marked async it isn't asynchronous at all since you're using Task.FromResult. The synchronous part of an asynchronous method (which is the part until you await an asynchronous task) is always executed synchronously on the calling thread and is done before the call returns. When you use Task.FromResult you get a completed task so all your method is just the synchronous part and is completed before the call returns.
When you await a completed task (as is created by Task.FromResult, it completes synchronously. This means that in your example, nothing is actually happening asynchronously, which explains the order of execution.
If instead, you were to
await Task.Yield();
you'd see output more in line with your expectations.
Task.FromResult won't cause yield and the task will be executed on the same thread. To achieve what you want you can do this:
public static async Task Test()
{
var tasks = new List<Task>();
tasks.Add(PrintNumber(1));
tasks.Add(PrintNumber(2));
tasks.Add(PrintNumber(3));
Console.WriteLine("This should be written first..");
// This should be printed last..
await Task.WhenAll(tasks);
}
public static async Task PrintNumber(int number)
{
await Task.Yield();
Console.WriteLine(number);
}
If you want a Task or tasks to run after something else, its easiest to write your code accordingly.
public static async Task Test()
{
Console.WriteLine("This should be written first..");
// These should be printed last..
await Task.WhenAll(new[]
{
PrintNumber(1),
PrintNumber(2),
PrintNumber(3)
});
}
following on from your comment.
So we have some functions,
async Task<Customer> GetRawCustomer()
{
...
}
async Task<string> GetCity(Customer customer)
{
...
}
async Task<string> GetZipCode(Customer customer)
{
...
}
We could use them like this
var rawCustomer = await GetRawCustomer();
var populationWork = new List<Task>();
Task<string> getCity;
if (string.IsNullOrWhiteSpace(rawCustomer.City))
{
getCity = GetCity(rawCustomer);
populationWork.Add(getCity);
}
Task<string> getZipCode;
if (string.IsNullOrWhiteSpace(rawCustomer.City))
{
getZipCode = GetZipCode(rawCustomer);
populationWork.Add(getZipCode);
}
...
await Task.WhenAll(populationWork);
if (getCity != null)
rawCustomer.City = getCity.Result;
if (getZipCode != null)
rawCustomer.ZipCode = getZipCode.Result;