Develop Reference

Is there a way to wait for all tasks until a specific result is true, and then cancel the rest?

In my C# console application, I'm trying to run multiple tasks that do various data checks simultaneously. If one of the tasks returns true I should stop the other tasks since I have my actionable result. It's also very possible none of the functions return true
I have the code to run the tasks together (I think), I'm just having trouble getting to the finish line:
Task task1 = Task.Run(() => Task1(stoppingToken));
Task task2 = Task.Run(() => Task2(stoppingToken));
Task task3 = Task.Run(() => Task3(stoppingToken));
Task task4 = Task.Run(() => Task4(stoppingToken));
Task task5 = Task.Run(() => Task5(stoppingToken));
Task task6 = Task.Run(() => Task6(stoppingToken));
Task.WaitAll(task1, task2, task3, task4, task5, task6);
This is a little different than the answer in the linked question where the desired result is known (timeout value). I'm waiting for any of these tasks to possibly return true and then cancel the remaining tasks if they are still running
Task.WhenAny with cancellation of the non completed tasks and timeout

Here's a solution based on continuation tasks. The idea is to append continuation tasks to each of the original (provided) tasks, and check the result there. If it's a match, the completion source will be set with a result (if there's no match, the result won't be set at all).
Then, the code will wait for whatever happens first: either all the continuation tasks complete, or the task completion result will be set. Either way, we'll be ready to check the result of the task associated with task completion source (that's why we wait for the continuation tasks to complete, not the original tasks) and if it's set, it's pretty much an indication that we have a match (the additional check at the end is a little paranoid, but better safe than sorry I guess... :D)
public static async Task<bool> WhenAnyHasResult<T>(Predicate<T> isExpectedResult, params Task<T>[] tasks)
{
const TaskContinuationOptions continuationTaskFlags = TaskContinuationOptions.ExecuteSynchronously | TaskContinuationOptions.OnlyOnRanToCompletion | TaskContinuationOptions.AttachedToParent;
// Prepare TaskCompletionSource to be set only when one of the provided tasks
// completes with expected result
var tcs = new TaskCompletionSource<T>();
// For every provided task, attach a continuation task that fires
// once the original task was completed
var taskContinuations = tasks.Select(task =>
{
return task.ContinueWith(x =>
{
var taskResult = x.Result;
if (isExpectedResult(taskResult))
{
tcs.SetResult(taskResult);
}
},
continuationTaskFlags);
});
// We either wait for all the continuation tasks to be completed
// (it's most likely an indication that none of the provided tasks completed with the expected result)
// or for the TCS task to complete (which means a failure)
await Task.WhenAny(Task.WhenAll(taskContinuations), tcs.Task);
// If the task from TCS has run to completion, it means the result has been set from
// the continuation task attached to one of the tasks provided in the arguments
var completionTask = tcs.Task;
if (completionTask.IsCompleted)
{
// We will check once more to make sure the result is set as expected
// and return this as our outcome
var tcsResult = completionTask.Result;
return isExpectedResult(tcsResult);
}
// TCS result was never set, which means we did not find a task matching the expected result.
tcs.SetCanceled();
return false;
}
Now, the usage will be as follows:
static async Task ExampleWithBooleans()
{
Console.WriteLine("Example with booleans");
var task1 = SampleTask(3000, true);
var task2 = SampleTask(5000, false);
var finalResult = await TaskUtils.WhenAnyHasResult(result => result == true, task1, task2);
// go ahead and cancel your cancellation token here
Console.WriteLine("Final result: " + finalResult);
Debug.Assert(finalResult == true);
Console.WriteLine();
}
What's nice about putting it into a generic method, is that it works with any type, not only booleans, as a result of the original task.

Assuming your tasks return bool you can do something like this:
CancellationTokenSource source = new CancellationTokenSource();
CancellationToken stoppingToken = source.Token;
Task<bool> task1 = Task.Run(() => Task1(stoppingToken));
....
var tasks = new List<Task<bool>>
{
task1, task2, task3, ...
};
bool taskResult = false;
do
{
var finished = await Task.WhenAny(tasks);
taskResult = finished.Result;
tasks.Remove(finished);
} while (tasks.Any() && !taskResult);
source.Cancel();

You could use an asynchronous method that wraps a Task<bool> to another Task<bool>, and cancels a CancellationTokenSource if the result of the input task is true. In the example below this method is the IfTrueCancel, and it is implemented as local function. This way it captures the CancellationTokenSource, and so you don't have to pass it as argument on every call:
var cts = new CancellationTokenSource();
var stoppingToken = cts.Token;
var task1 = IfTrueCancel(Task.Run(() => Task1(stoppingToken)));
var task2 = IfTrueCancel(Task.Run(() => Task2(stoppingToken)));
var task3 = IfTrueCancel(Task.Run(() => Task3(stoppingToken)));
var task4 = IfTrueCancel(Task.Run(() => Task4(stoppingToken)));
var task5 = IfTrueCancel(Task.Run(() => Task5(stoppingToken)));
var task6 = IfTrueCancel(Task.Run(() => Task6(stoppingToken)));
Task.WaitAll(task1, task2, task3, task4, task5, task6);
async Task<bool> IfTrueCancel(Task<bool> task)
{
bool result = await task.ConfigureAwait(false);
if (result) cts.Cancel();
return result;
}
Another, quite different, solution to this problem could be to use PLINQ instead of explicitly created Tasks. PLINQ requires an IEnumerable of something in order to do parallel work on it, and in your case this something is the Task1, Task2 etc functions that you want to invoke. You could put them in an array of Func<CancellationToken, bool>, and solve the problem this way:
var functions = new Func<CancellationToken, bool>[]
{
Task1, Task2, Task3, Task4, Task5, Task6
};
bool success = functions
.AsParallel()
.WithDegreeOfParallelism(4)
.Select(function =>
{
try
{
bool result = function(stoppingToken);
if (result) cts.Cancel();
return result;
}
catch (OperationCanceledException)
{
return false;
}
})
.Any(result => result);
The advantage of this approach is that you can configure the degree of parallelism, and you don't have to rely on the ThreadPool availability for limiting the concurrency of the whole operation. The disadvantage is that all functions should have the same signature. You could overcome this disadvantage by declaring the functions as lambda expressions like this:
var functions = new Func<CancellationToken, bool>[]
{
ct => Task1(arg1, ct),
ct => Task2(arg1, arg2, ct),
ct => Task3(ct),
ct => Task4(arg1, arg2, arg3, ct),
ct => Task5(arg1, ct),
ct => Task6(ct)
};

Correct way to link Tasks together when return values are needed at different times

I hope this makes sense - Suppose I have the following code:
Task.Run(() =>
{
return Task.WhenAll
(
Task1,
Task2,
...
Taskn
)
.ContinueWith(tsks=>
{
TaskA (uses output from Tasks Task1 & Task2, say)
}
, ct)
.ContinueWith(res =>
{
TaskB (uses output from TaskA and Task3, say)
}
, ct);
});
So I want all my first N tasks to run concurrently (since we have no interdependencies), then only once they're all finished, to continue with a task that relies on their outputs (I get that for this, I can use the tsks.Result).
BUT THEN I want to continue with a task that relies on one of the first tasks and the result of TaskA.
I'm a bit lost how to structure my code correctly so I can access the results of my first set of tasks outside of the immediately proceeding ContinueWith.
My one thought was to assign return value to them within my method - Something like:
... declare variables outside of Tasks ...
Task.Run(() =>
{
return Task.WhenAll
(
Task.Run(() => { var1 = Task1.Result; }, ct),
...
Task.Run(() => { varn = Taskn.Result; }, ct),
)
.ContinueWith(tsks=>
{
TaskA (uses output from Tasks var1 & varn, say)
}
, ct)
.ContinueWith(res =>
{
TaskB (uses output from TaskA and var3, say)
}
, ct);
});
But, even though this works for me, I have no doubt that that is doing it wrong.
What is the correct way? Should I have a state object that contains all the necessary variables and pass that throughout all my tasks? Is there a better way in total?
Please forgive my ignorance here - I'm just VERY new to concurrency programming.

Since Task1, Task2, ... , TaskN are in scope for the call of WhenAll, and because by the time ContinueWith passes control to your next task all the earlier tasks are guaranteed to finish, it is safe to use TaskX.Result inside the code implementing continuations:
.ContinueWith(tsks=>
{
var resTask1 = Task1.Result;
...
}
, ct)
You are guaranteed to get the result without blocking, because the task Task1 has finished running.

Here is a way to do it with ConcurrentDictionary, which sounds like it might be applicable in your use case. Also, since you're new to concurrency, it shows you the Interlocked class as well:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Executing...");
var numOfTasks = 50;
var tasks = new List<Task>();
for (int i = 0; i < numOfTasks; i++)
{
var iTask = Task.Run(() =>
{
var counter = Interlocked.Increment(ref _Counter);
Console.WriteLine(counter);
if (counter == numOfTasks - 1)
{
Console.WriteLine("Waiting {0} ms", 5000);
Task.Delay(5000).Wait(); // to simulate a longish running task
}
_State.AddOrUpdate(counter, "Updated Yo!", (k, v) =>
{
throw new InvalidOperationException("This shouldn't occure more than once.");
});
});
tasks.Add(iTask);
}
Task.WhenAll(tasks)
.ContinueWith(t =>
{
var longishState = _State[numOfTasks - 1];
Console.WriteLine(longishState);
Console.WriteLine("Complete. longishState: " + longishState);
});
Console.ReadKey();
}
static int _Counter = -1;
static ConcurrentDictionary<int, string> _State = new ConcurrentDictionary<int, string>();
}
You get output similar to this (though it the Waiting line won't always be last before the continuation):

An elegant way to solve this is to use Barrier class.
Like this:
var nrOfTasks = ... ;
ConcurrentDictionary<int, ResultType> Results = new ConcurrentDictionary<int, ResultType>();
var barrier = new Barrier(nrOfTasks, (b) =>
{
// here goes the work of TaskA
// and immediatley
// here goes the work of TaskB, having the results of TaskA and any other task you might need
});
Task.Run(() => { Results[1] = Task1.Result; barrier.SignalAndWait(); }, ct),
...
Task.Run(() => { Results[nrOfTasks] = Taskn.Result; barrier.SignalAndWait(); }, ct

How to process tasks as they complete -but each task requires different method to process task's result

I'm using async/await to call few external APIs. All of them returns me a string value but in different format and requires their own processing.
And I want to process the returned value as a task completes. I don't want to wait until all are completed and hence I'm using Task.WhenAny(). How can I process tasks as they complete and still use the correct "Process" method for each task as they complete?
I make some changes after my first post and here is the latest i have:
public async Task<List<string>> Get()
{
var task1 = Method1Async();
var task2 = Method1Async();
var tasks = new List<Task<string>> {task1, task2};
var results = new List<string>();
while (tasks.Count > 0)
{
var justCompletedTask = await Task.WhenAny(tasks);//will not throw
tasks.Remove(justCompletedTask);
try
{
var result = await justCompletedTask;
results.Add(result);
}
catch(Exception)
{
//deal with it
}
}
return results;
}
private async Task<string> Method1Async()
{
//this may throw - something like forbidden or any other exception
var task = _httpClient.GetStringAsync("api1's url here");
var result = await Method1ResultProcessorAsync(task);
return result;
}
private async Task<string> Method1ResultProcessorAsync(Task<string> task)
{
//process task's result -if it successuflly completed and return that
return await task; //for now
}
private async Task<string> Method2Async()
{
//this may throw - something like forbidden or any other exception
var task = _httpClient.GetStringAsync("api2's url here");
var result = await Method2ResultProcessorAsync(task);
return await task;
}
private async Task<string> Method2ResultProcessorAsync(Task<string> task)
{
//This processing logic is entirely different from Method1ResultProcessor
//process task's result -if it successfully completed and return that
return await task; //for now
}
I have two questions here:
Is this the right way to approach the problem?
How do i better handle exception here? This is very important so the failure of one should not fail the whole thing. As long as any of the methods succeed, it will be okay. But if all fails, I want to the Get method to throw.

Since your processor methods already accept Task, you can just call them and they will asynchronously wait for their corresponding results:
public Task<string[]> Get()
{
var task1 = Method1ResultProcessorAsync(Method1Async());
var task2 = Method2ResultProcessorAsync(Method2Async());
return Task.WhenAll(task1, task2);
}
Handling exceptions the way you describe will make this more complicated, but you can use something like:
public async Task<List<string>> Get()
{
var task1 = Method1ResultProcessorAsync(Method1Async());
var task2 = Method2ResultProcessorAsync(Method2Async());
var tasks = new[] { task1, task2 };
try
{
await Task.WhenAll(tasks);
}
catch {}
var results = tasks.Where(t => t.Status == TaskStatus.RanToCompletion)
.Select(t => t.Result)
.ToList();
if (results.Any())
return results;
// or maybe another exception,
// since await handles AggregateException in a weird way
throw new AggregateException(tasks.Select(t => t.Exception));
}

Here is an alternative way of describing Method1Async() and Method2Async(). This is the demonstration of ContinueWith. Answering the question you ask in title – you do can use a different method for each task, that method will be called after the task completes.
var task1 = _httpClient.GetStringAsync("api1's url here").ContinueWith(t => Method1ResultProcessorAsync(t));
var task2 = _httpClient.GetStringAsync("api2's url here").ContinueWith(t => Method2ResultProcessorAsync(t));
You handle the exceptions correctly. Answering "But if all fails, I want to the Get method to throw.": just check whether results.Count == 0 before the return, and throw if it 0.

How to make Task.WaitAll() to break if any exception happened?

I want to make Task.WaitAll() to break out if any of the running tasks throws an exception, so that I don't have to wait for 60 seconds to finish. How do I achieve such behavior? If WaitAll() cannot achieve that, is there any other c# feature or workaround?
Task task1 = Task.Run(() => throw new InvalidOperationException());
Task task2 = ...
...
try
{
Task.WaitAll(new Task[]{task1, task2, ...}, TimeSpan.FromSeconds(60));
}
catch (AggregateException)
{
// If any exception thrown on any of the tasks, break out immediately instead of wait all the way to 60 seconds.
}

The following should do it without altering the code of the original tasks (untested):
static bool WaitAll(Task[] tasks, int timeout, CancellationToken token)
{
var cts = CancellationTokenSource.CreateLinkedTokenSource(token);
var proxyTasks = tasks.Select(task =>
task.ContinueWith(t => {
if (t.IsFaulted) cts.Cancel();
return t;
},
cts.Token,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Current).Unwrap());
return Task.WaitAll(proxyTasks.ToArray(), timeout, cts.Token);
}
Note it only tracks faulted tasks (those which threw). If you need to track cancelled tasks as well, make this change:
if (t.IsFaulted || t.IsCancelled) cts.Cancel();
Updated, waiting on the task proxies is redundant here, as pointed out by #svick in the comments. He proposes an improved version: https://gist.github.com/svick/9992598.

One way of doing that is to use CancellationTokenSource. You create cancellationtokensource, and pass it as an argument to Task.WaitAll. The idea is to wrap your task in try/catch block, and in case of exception, call cancel on cancellationtokensource.
Here's sample code
CancellationTokenSource mainCancellationTokenSource = new CancellationTokenSource();
Task task1 = new Task(() =>
{
try
{
throw new Exception("Exception message");
}
catch (Exception ex)
{
mainCancellationTokenSource.Cancel();
}
}, mainCancellationTokenSource.Token);
Task task2 = new Task(() =>
{
Thread.Sleep(TimeSpan.FromSeconds(3));
Console.WriteLine("Task is running");
}, mainCancellationTokenSource.Token);
task1.Start();
task2.Start();
Task.WaitAll(new[] { task1, task2},
6000, // 6 seconds
mainCancellationTokenSource.Token
);
}
catch (Exception ex)
{
// If any exception thrown on any of the tasks, break out immediately instead of wait all the way to 60 seconds.
}

Parallel class can do the job for you. You can use Parallel.For, ForEach or Invoke.
using System;
using System.Threading;
using System.Threading.Tasks;
namespace Sample_04_04_2014_01
{
class Program
{
public static void Main(string[] args)
{
try
{
Parallel.For(0,20, i => {
Console.WriteLine(i);
if(i == 5)
throw new InvalidOperationException();
Thread.Sleep(100);
});
}
catch(AggregateException){}
Console.Write("Press any key to continue . . . ");
Console.ReadKey(true);
}
}
}
If one of these tasks throws an exception then no other task will be executed excepting those whose execution was already started. For, ForEach and Invoke are waiting for all tasks to complete before to resume control to the calling code. You can have even a finer grain control if you use ParallelLoopState.IsExceptional. Parallel.Invoke is more suited for your case.

I wanted to suggest a slight modification to Noseratio's excellent answer above. In my case I needed to preserve the original exception thrown, and in a surrounding try/catch distinguish between cancelled and exception states.
public static void WaitUnlessFault( Task[] tasks, CancellationToken token )
{
var cts = CancellationTokenSource.CreateLinkedTokenSource(token);
foreach ( var task in tasks ) {
task.ContinueWith(t =>
{
if ( t.IsFaulted ) cts.Cancel();
},
cts.Token,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Current);
}
try {
Task.WaitAll(tasks, cts.Token);
}
catch ( OperationCanceledException ex ) {
var faultedTaskEx = tasks.Where(t => t.IsFaulted)
.Select(t => t.Exception)
.FirstOrDefault();
if ( faultedTaskEx != null )
throw faultedTaskEx;
else
throw;
}
}

Concurrent execution of async methods

Using the async/await model, I have a method which makes 3 different calls to a web service and then returns the union of the results.
var result1 = await myService.GetData(source1);
var result2 = await myService.GetData(source2);
var result3 = await myService.GetData(source3);
allResults = Union(result1, result2, result3);
Using typical await, these 3 calls will execute synchronously wrt each other. How would I go about letting them execute concurrently and join the results as they complete?

How would I go about letting them execute in parallel and join the results as they complete?
The simplest approach is just to create all the tasks and then await them:
var task1 = myService.GetData(source1);
var task2 = myService.GetData(source2);
var task3 = myService.GetData(source3);
// Now everything's started, we can await them
var result1 = await task1;
var result1 = await task2;
var result1 = await task3;
You might also consider Task.WhenAll. You need to consider the possibility that more than one task will fail... with the above code you wouldn't observe the failure of task3 for example, if task2 fails - because your async method will propagate the exception from task2 before you await task3.
I'm not suggesting a particular strategy here, because it will depend on your exact scenario. You may only care about success/failure and logging one cause of failure, in which case the above code is fine. Otherwise, you could potentially attach continuations to the original tasks to log all exceptions, for example.

You could use the Parallel class:
Parallel.Invoke(
() => result1 = myService.GetData(source1),
() => result2 = myService.GetData(source2),
() => result3 = myService.GetData(source3)
);
For more information visit: http://msdn.microsoft.com/en-us/library/system.threading.tasks.parallel(v=vs.110).aspx

As a more generic solution you can use the api I wrote below, it also allows you to define a real time throttling mechanism of max number of concurrent async requests.
The inputEnumerable will be the enumerable of your source and asyncProcessor is your async delegate (myservice.GetData in your example).
If the asyncProcessor - myservice.GetData - returns void or just a Task without any type, then you can simply update the api to reflect that. (just replace all Task<> references to Task)
public static async Task<TOut[]> ForEachAsync<TIn, TOut>(
IEnumerable<TIn> inputEnumerable,
Func<TIn, Task<TOut>> asyncProcessor,
int? maxDegreeOfParallelism = null)
{
IEnumerable<Task<TOut>> tasks;
if (maxDegreeOfParallelism != null)
{
SemaphoreSlim throttler = new SemaphoreSlim(maxDegreeOfParallelism.Value, maxDegreeOfParallelism.Value);
tasks = inputEnumerable.Select(
async input =>
{
await throttler.WaitAsync();
try
{
return await asyncProcessor(input).ConfigureAwait(false);
}
finally
{
throttler.Release();
}
});
}
else
{
tasks = inputEnumerable.Select(asyncProcessor);
}
await Task.WhenAll(tasks);
}