Task.WhenAll(params System.Threading.Tasks.Task[] tasks) returns Task, but what is the proper way to asquire task results after calling this method?
After awaiting that task, results can be acquired from the original task by awaiting it once again which should be fine as tasks are completed already. Also it is possible to get result using Task.Result property which is often considered not good practice
Task<TResult1> task1= ...
Task<TResult2> task2= ...
Task<TResult3> task3= ...
await Task.WhenAll(task1, task2, task3)
var a = task1.Result; // returns TResult1
var b = await task1; // also returns TResult1
Which one should I choose here and why?
If you really have only an IEnumerable<Task<TResult>> and the task will be created on-the-fly (e.g. due to a .Select()) you would execute your tasks two times.
So, be sure that you either give an already materialized collection to Task.WhenAll() or get the result from the return value of that method:
var someTasks = Enumerable.Range(1, 10).Select(i => { Task.Delay(i * 100); return i; });
// Bad style, cause someTasks is an IEnumerable created on-the-fly
await Task.WhenAll(someTasks);
foreach(var task in someTasks)
{
var taskResult = await task;
Console.WriteLine(taskResult);
}
// Okay style, cause tasks are materialized before waiting, but easy to misuse wrong variable name.
var myTasks = someTasks.ToList();
await Task.WhenAll(myTasks);
foreach(var task in myTasks)
{
Console.WriteLine(task.Result);
}
// Best style
var results = await Task.WhenAll(someTasks);
foreach(var result in results)
{
Console.WriteLine(result);
}
Update
Just read this in your question:
However, I could not find any overload that would return anything but Task.
This happens, if the collection of tasks you give to the Task.WhenAll() method don't share a common Task<T> type. This could happen, if you e.g. want to run two tasks in parallel, but both return a different value. In that case you have to materialize the tasks and afterwards check the results individually:
public static class Program
{
public static async Task Main(string[] args)
{
var taskOne = ReturnTwo();
var taskTwo = ReturnPi();
await Task.WhenAll(taskOne, taskTwo);
Console.WriteLine(taskOne.Result);
Console.WriteLine(taskTwo.Result);
Console.ReadKey();
}
private static async Task<int> ReturnTwo()
{
await Task.Delay(500);
return 2;
}
private static async Task<double> ReturnPi()
{
await Task.Delay(500);
return Math.PI;
}
}
Here is the overload that returns Task<TResult[]> - MS Docs
Example:
static async Task Test()
{
List<Task<string>> tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
var currentTask = GetStringAsync();
tasks.Add(currentTask);
}
string[] result = await Task.WhenAll(tasks);
}
static async Task<string> GetStringAsync()
{
await Task.Delay(1000);
return "Result string";
}
Related
Let's imagine some abstract code
private void Main()
{
var workTask1 = DoWork1();
var workTask2 = DoWork2();
var workTask3 = DoWork3();
await Task.WhenAll(workTask1, workTask2, workTask3);
AnalyzeWork(workTask1.Result, workTask2.Result, workTask3.Result);
}
private async Task<object> DoWork1()
{
var someOperationTask1 = someOperation1();
var someOperationTask2 = someOperation2();
await Task.WhenAll(someOperationTask1, someOperationTask2);
return new object
{
SomeOperationResult1 = someOperationTask1.Result,
SomeOperationResult2 = someOperationTask2.Result,
};
}
private async Task<object> DoWork2()
{
var someOperationTask3 = someOperation3();
var someOperationTask4 = someOperation4();
await Task.WhenAll(someOperationTask3, someOperationTask4);
return new object
{
SomeOperationResult3 = someOperationTask3.Result,
SomeOperationResult4 = someOperationTask4.Result,
};
}
private async Task<object> DoWork3()
{
var someOperationTask5 = someOperation5();
var someOperationTask6 = someOperation6();
await Task.WhenAll(someOperationTask5, someOperationTask6);
return new object
{
SomeOperationResult5 = someOperationTask5.Result,
SomeOperationResult6 = someOperationTask6.Result,
};
}
Where 3 methods are being run parallelly and each of them consists of 2 parallel's operations. And result of 3 methods is passed to some method.
My question is there are any restrictions? Is it ok to have nested Task.WhenAll and what's difference between nested Task.WhenAll and one level Task.WhenAll operations?
The only restrictions are the available memory of your system. The Task.WhenAll method attaches a continuation to each incomplete task, and this continuation is detached when that task completes. A continuation is a lightweight object similar to a Task. It's quite similar to what you get when you invoke the Task.ContinueWith method. Each continuation weights more or less about 100 bytes. It is unlikely that it will have any noticeable effect to your program, unless you need to Task.WhenAll tens of millions of tasks (or more) at once.
If you want a visual demonstration of what this method looks like inside, below is a rough sketch of its implementation:
// For demonstration purposes only. This code is full of bugs.
static Task WhenAll(params Task[] tasks)
{
var tcs = new TaskCompletionSource();
int completedCount = 0;
foreach (var task in tasks)
{
task.ContinueWith(t =>
{
completedCount++;
if (completedCount == tasks.Length) tcs.SetResult();
});
}
return tcs.Task;
}
Let's say I have a list of Tasks, and I want to run them in parallel. But I don't need all of them to finish to continue, I can move on with just one. The following code waits for all the tasks to finish to move on. Is there a way for me to individually continue with the task that has completed while waiting for the other ones to finish?
List<string>[] x = await Task.WhenAll(new Task<List<string>>[] { task1, task2 })
// When task1 finishes, I want to process the result immediately
// instead of waiting on task2.
You're probably looking for Task.WhenAny.
I've used it for setting off a pile of tasks and then processing each of them as they become ready, but I suppose you could also just wait for one to finish and continue without the loop if you don't care about dealing with the rest.
while(tasks.Count() > 0)
{
var task = await Task.WhenAny(tasks);
tasks.Remove(task);
var taskresult = await task;
// process result
}
If you are using C# 8 and .NET Core you can take advantage of IAsyncEnumerable to hide this complexity from the consuming side.
Just like this:
static async Task Main(string[] args)
{
await foreach (var data in GetData())
{
Console.WriteLine(data);
}
Console.ReadLine();
}
static async IAsyncEnumerable<string> GetData()
{
List<Task<string>> tasks = new List<Task<string>> {GetData1(), GetData3(), GetData2()};
while (tasks.Any())
{
var finishedTask = await Task.WhenAny(tasks);
tasks.Remove(finishedTask);
yield return await finishedTask;
}
}
static async Task<string> GetData1()
{
await Task.Delay(5000);
return "Data1";
}
static async Task<string> GetData2()
{
await Task.Delay(3000);
return "Data2";
}
static async Task<string> GetData3()
{
await Task.Delay(2000);
return "Data3";
}
You can use Task.WhenAny instead.
Example "stolen" from Stephen Cleary's Blog:
var client = new HttpClient();
string results = await await Task.WhenAny(
client.GetStringAsync("http://example.com"),
client.GetStringAsync("http://microsoft.com"));
// results contains the HTML for whichever website responded first.
Responding to comment
You absolutely can keep track of the other tasks:
// supposing you have a list of Tasks in `myTasks`:
while( myTasks.Count > 0 )
{
var finishedTask = await Task.WhenAny(myTasks);
myTasks.Remove(finishedTask);
handleFinishedTask(finishedTask); // assuming this is a method that
// does the work on finished tasks.
}
The only thing you'd have to watch out for is :
The returned task will always end in the RanToCompletion state with its Result set to the first task to complete. This is true even if the first task to complete ended in the Canceled or Faulted state.
Remarks in WhenAny Doks(Emphasis by me)
In case you want to process the results in order of the completion of the tasks, there is the OrderByCompletion extension method that does exactly that in Stephen Cleary's Nito.AsyncEx library, with the signature below:
// Creates a new collection of tasks that complete in order.
public static List<Task<T>> OrderByCompletion<T>(this IEnumerable<Task<T>> #this);
Usage example:
Task<string>[] tasks = new[] { task1, task2, task3, task4 };
foreach (var task in tasks.OrderByCompletion())
{
string result = await task;
// Do something with result
}
If you prefer not having external dependencies, the source code is here.
Based on the answer of Peter Csala, here a extension method for IAsyncEnumerable:
public static async IAsyncEnumerable<T> OrderedByCompletion<T>(this IEnumerable<Task<T>> tasks)
{
List<Task<T>> taskList = new List<Task<T>>(tasks);
while (taskList.Count > 0)
{
var finishedTask = await Task.WhenAny(taskList);
taskList.Remove(finishedTask);
yield return await finishedTask;
}
}
I want to ask you: I have code:
var task1 = await _connectionService.ValidateUriAsync(uri1);
OutputResult("ss", task1);
var task2 = await _connectionService.ValidateUriAsync(uri2);
OutputResult("bb", task2);
var task3 = await _connectionService.ValidateUriAsync(uri3);
OutputResult("cc", task3);
Now I'm waiting until each task finishes and then I output the result. But I would like to run all tasks independently (I know how to do that). But what I don't know is, when some task is completed I need to output result for each task. If task fails the output will be - task1 failed or Task1 success.
I tried this solution, but I will have to check which task is completed and than its result (true/false). It is complex. If I had 100 tasks, I cannot have 100 conditions.
var tasks = new[] {task1, task2, task3};
var process = tasks.Select(async task =>
{
var result = await task;
if(task == task1)assign property
});
await Task.WhenAll(proces);
EDIT:
Here is ValidateUriAsync func:
public async Task<bool> ValidateUriAsync(Uri uri)
{
try
{
var request = WebRequest.CreateHttp(uri);
var result = await request.GetResponseAsync();
return true;
}
catch (Exception e)
{
return false;
}
}
when some task is completed I need to output result for each task.
Don't think about this in terms of "reacting to tasks as they complete". Instead, think of your ValidateUriAsync method as an "operation", and what you want is to create a new higher-level "operation" that is "validate and assign".
With that mindset, the solution is more clear - introduce a new async method for the new operation:
private async Task ValidateAndOutputResult(Uri uri, string name)
{
var result = await _connectionService.ValidateUriAsync(uri);
OutputResult(name, result);
}
Now you can call the higher-level method, and use Task.WhenAll:
var tasks = new[]
{
ValidateAndOutputResult(uri1, "ss"),
ValidateAndOutputResult(uri2, "bb"),
ValidateAndOutputResult(uri3, "cc"),
};
await Task.WhenAll(tasks);
There is a Task variable and lets say the task is running right now.. by executing the following line.
await _task;
I was wondering what happens when I write this code:
await _task;
await _task;
would it execute the task twice ? Or throw an exception because it has already run ?
would it execute the task twice ? Or throw an exception because it has
already run ?
No and no. The only thing await does is call Task.GetAwaiter, it does not cause anything to run. If the task already ran to completion, it will either extract the value if it is a Task<T>, or run synchronously to the next line if it is a Task, since there is an optimization for already completed tasks.
A simple demo:
async Task Main()
{
var foo = FooAsync();
await foo;
await foo;
var bar = BarAsync();
var firstResult = await bar;
var secondResult = await bar;
Console.WriteLine(firstResult);
Console.WriteLine(secondResult);
}
public async Task FooAsync()
{
await Task.Delay(1);
}
public async Task<int> BarAsync()
{
await Task.Delay(1);
return 1;
}
If you drill down to the state machine itself, you'll see this:
this.<foo>5__1 = this.<>4__this.FooAsync();
taskAwaiter = this.<foo>5__1.GetAwaiter();
if (!taskAwaiter.IsCompleted)
{
this.<>1__state = 0;
this.<>u__1 = taskAwaiter;
M.<FooBar>d__0 <FooBar>d__ = this;
this.<>t__builder.AwaitUnsafeOnCompleted<TaskAwaiter, M.<FooBar>d__0>
(ref taskAwaiter, ref <FooBar>d__);
return;
}
This optimization first checks the completion of the task. If the task isn't complete, it will call UnsafeOnCompleted which will register the continuation. If it is complete, it breaks the switch and goes to:
this.<>1__state = -2;
this.<>t__builder.SetResult();
Which sets the result for the underlying Task, and that way you actually get the value synchronously.
I threw together this quick test:
var i = 0;
var task = Task.Run(() => { i++; Console.WriteLine(i); return i; });
var x = await task;
var y = await task;
Console.WriteLine(x);
Console.WriteLine(y);
It writes:
1
1
1
So, clearly, the task only runs once.
I'm using async/await to call few external APIs. All of them returns me a string value but in different format and requires their own processing.
And I want to process the returned value as a task completes. I don't want to wait until all are completed and hence I'm using Task.WhenAny(). How can I process tasks as they complete and still use the correct "Process" method for each task as they complete?
I make some changes after my first post and here is the latest i have:
public async Task<List<string>> Get()
{
var task1 = Method1Async();
var task2 = Method1Async();
var tasks = new List<Task<string>> {task1, task2};
var results = new List<string>();
while (tasks.Count > 0)
{
var justCompletedTask = await Task.WhenAny(tasks);//will not throw
tasks.Remove(justCompletedTask);
try
{
var result = await justCompletedTask;
results.Add(result);
}
catch(Exception)
{
//deal with it
}
}
return results;
}
private async Task<string> Method1Async()
{
//this may throw - something like forbidden or any other exception
var task = _httpClient.GetStringAsync("api1's url here");
var result = await Method1ResultProcessorAsync(task);
return result;
}
private async Task<string> Method1ResultProcessorAsync(Task<string> task)
{
//process task's result -if it successuflly completed and return that
return await task; //for now
}
private async Task<string> Method2Async()
{
//this may throw - something like forbidden or any other exception
var task = _httpClient.GetStringAsync("api2's url here");
var result = await Method2ResultProcessorAsync(task);
return await task;
}
private async Task<string> Method2ResultProcessorAsync(Task<string> task)
{
//This processing logic is entirely different from Method1ResultProcessor
//process task's result -if it successfully completed and return that
return await task; //for now
}
I have two questions here:
Is this the right way to approach the problem?
How do i better handle exception here? This is very important so the failure of one should not fail the whole thing. As long as any of the methods succeed, it will be okay. But if all fails, I want to the Get method to throw.
Since your processor methods already accept Task, you can just call them and they will asynchronously wait for their corresponding results:
public Task<string[]> Get()
{
var task1 = Method1ResultProcessorAsync(Method1Async());
var task2 = Method2ResultProcessorAsync(Method2Async());
return Task.WhenAll(task1, task2);
}
Handling exceptions the way you describe will make this more complicated, but you can use something like:
public async Task<List<string>> Get()
{
var task1 = Method1ResultProcessorAsync(Method1Async());
var task2 = Method2ResultProcessorAsync(Method2Async());
var tasks = new[] { task1, task2 };
try
{
await Task.WhenAll(tasks);
}
catch {}
var results = tasks.Where(t => t.Status == TaskStatus.RanToCompletion)
.Select(t => t.Result)
.ToList();
if (results.Any())
return results;
// or maybe another exception,
// since await handles AggregateException in a weird way
throw new AggregateException(tasks.Select(t => t.Exception));
}
Here is an alternative way of describing Method1Async() and Method2Async(). This is the demonstration of ContinueWith. Answering the question you ask in title – you do can use a different method for each task, that method will be called after the task completes.
var task1 = _httpClient.GetStringAsync("api1's url here").ContinueWith(t => Method1ResultProcessorAsync(t));
var task2 = _httpClient.GetStringAsync("api2's url here").ContinueWith(t => Method2ResultProcessorAsync(t));
You handle the exceptions correctly. Answering "But if all fails, I want to the Get method to throw.": just check whether results.Count == 0 before the return, and throw if it 0.