I have a series of Tasks in an array. If a Task is "Good" it returns a string. If it's "Bad": it return a null.
I want to be able to run all the Tasks in parallel, and once the first one comes back that is "Good", then cancel the others and get the "Good" result.
I am doing this now, but the problem is that all the tasks need to run, then I loop through them looking for the first good result.
List<Task<string>> tasks = new List<Task<string>>();
Task.WaitAll(tasks.ToArray());
I want to be able to run all the Tasks in parallel, and once the first one comes back that is "Good", then cancel the others and get the "Good" result.
This is misunderstanding, since Cancellation in TPL is co-operative, so once the Task is started, there's no way to Cancel it. CancellationToken can work before Task is started or later to throw an exception, if Cancellation is requested, which is meant to initiate and take necessary action, like throw custom exception from the logic
Check the following query, it has many interesting answers listed, but none of them Cancel. Following is also a possible option:
public static class TaskExtension<T>
{
public static async Task<T> FirstSuccess(IEnumerable<Task<T>> tasks, T goodResult)
{
// Create a List<Task<T>>
var taskList = new List<Task<T>>(tasks);
// Placeholder for the First Completed Task
Task<T> firstCompleted = default(Task<T>);
// Looping till the Tasks are available in the List
while (taskList.Count > 0)
{
// Fetch first completed Task
var currentCompleted = await Task.WhenAny(taskList);
// Compare Condition
if (currentCompleted.Status == TaskStatus.RanToCompletion
&& currentCompleted.Result.Equals(goodResult))
{
// Assign Task and Clear List
firstCompleted = currentCompleted;
break;
}
else
// Remove the Current Task
taskList.Remove(currentCompleted);
}
return (firstCompleted != default(Task<T>)) ? firstCompleted.Result : default(T);
}
}
Usage:
var t1 = new Task<string>(()=>"bad");
var t2 = new Task<string>(()=>"bad");
var t3 = new Task<string>(()=>"good");
var t4 = new Task<string>(()=>"good");
var taskArray = new []{t1,t2,t3,t4};
foreach(var tt in taskArray)
tt.Start();
var finalTask = TaskExtension<string>.FirstSuccess(taskArray,"good");
Console.WriteLine(finalTask.Result);
You may even return Task<Task<T>>, instead of Task<T> for necessary logical processing
You can achieve your desired results using following example.
List<Task<string>> tasks = new List<Task<string>>();
// ***Use ToList to execute the query and start the tasks.
List<Task<string>> goodBadTasks = tasks.ToList();
// ***Add a loop to process the tasks one at a time until none remain.
while (goodBadTasks.Count > 0)
{
// Identify the first task that completes.
Task<string> firstFinishedTask = await Task.WhenAny(goodBadTasks);
// ***Remove the selected task from the list so that you don't
// process it more than once.
goodBadTasks.Remove(firstFinishedTask);
// Await the completed task.
string firstFinishedTaskResult = await firstFinishedTask;
if(firstFinishedTaskResult.Equals("good")
// do something
}
EDIT : If you want to terminate all the tasks you can use CancellationToken.
For more detail read the docs.
I was looking into Task.WhenAny() which will trigger on the first "completed" task. Unfortunately, a completed task in this sense is basically anything... even an exception is considered "completed". As far as I can tell there is no other way to check for what you call a "good" value.
While I don't believe there is a satisfactory answer for your question I think there may be an alternative solution to your problem. Consider using Parallel.ForEach.
Parallel.ForEach(tasks, (task, state) =>
{
if (task.Result != null)
state.Stop();
});
The state.Stop() will cease the execution of the Parallel loop when it finds a non-null result.
Besides having the ability to cease execution when it finds a "good" value, it will perform better under many (but not all) scenarios.
Use Task.WhenAny It returns the finished Task. Check if it's null. If it is, remove it from the List and call Task.WhenAny Again.
If it's good, Cancel all Tasks in the List (they should all have a CancellationTokenSource.Token.
Edit:
All Tasks should use the same CancellationTokenSource.Token. Then you only need to cancel once.
Here is some code to clarify:
private async void button1_Click(object sender, EventArgs e)
{
CancellationTokenSource cancellationTokenSource = new CancellationTokenSource();
List<Task<string>> tasks = new List<Task<string>>();
tasks.Add(Task.Run<string>(() => // run your tasks
{
while (true)
{
if (cancellationTokenSource.Token.IsCancellationRequested)
{
return null;
}
return "Result"; //string or null
}
}));
while (tasks.Count > 0)
{
Task<string> resultTask = await Task.WhenAny(tasks);
string result = await resultTask;
if (result == null)
{
tasks.Remove(resultTask);
}
else
{
// success
cancellationTokenSource.Cancel(); // will cancel all tasks
}
}
}
Related
I have a bunch of requests to process, some of which may complete synchronously.
I'd like to gather all results that are immediately available and return them early, while waiting for the rest.
Roughly like this:
List<Task<Result>> tasks = new ();
List<Result> results = new ();
foreach (var request in myRequests) {
var task = request.ProcessAsync();
if (task.IsCompleted)
results.Add(task.Result); // or Add(await task) ?
else
tasks.Add(task);
}
// send results that are available "immediately" while waiting for the rest
if (results.Count > 0) SendResults(results);
results = await Task.WhenAll(tasks);
SendResults(results);
I'm not sure whether relying on IsCompleted might be a bad idea; could there be situations where its result cannot be trusted, or where it may change back to false again, etc.?
Similarly, could it be dangerous to use task.Result even after checking IsCompleted, should one always prefer await task? What if were using ValueTask instead of Task?
I'm not sure whether relying on IsCompleted might be a bad idea; could there be situations where its result cannot be trusted...
If you're in a multithreaded context, it's possible that IsCompleted could return false at the moment when you check on it, but it completes immediately thereafter. In cases like the code you're using, the cost of this happening would be very low, so I wouldn't worry about it.
or where it may change back to false again, etc.?
No, once a Task completes, it cannot uncomplete.
could it be dangerous to use task.Result even after checking IsCompleted.
Nope, that should always be safe.
should one always prefer await task?
await is a great default when you don't have a specific reason to do something else, but there are a variety of use cases where other patterns might be useful. The use case you've highlighted is a good example, where you want to return the results of finished tasks without awaiting all of them.
As Stephen Cleary mentioned in a comment below, it may still be worthwhile to use await to maintain expected exception behavior. You might consider doing something more like this:
var requestsByIsCompleted = myRequests.ToLookup(r => r.IsCompleted);
// send results that are available "immediately" while waiting for the rest
SendResults(await Task.WhenAll(requestsByIsCompleted[true]));
SendResults(await Task.WhenAll(requestsByIsCompleted[false]));
What if were using ValueTask instead of Task?
The answers above apply equally to both types.
You could use code like this to continually send the results of completed tasks while waiting on others to complete.
foreach (var request in myRequests)
{
tasks.Add(request.ProcessAsync());
}
// wait for at least one task to be complete, then send all available results
while (tasks.Count > 0)
{
// wait for at least one task to complete
Task.WaitAny(tasks.ToArray());
// send results for each completed task
var completedTasks = tasks.Where(t => t.IsCompleted);
var results = completedTasks.Where(t => t.IsCompletedSuccessfully).Select(t => t.Result).ToList();
SendResults(results);
// TODO: handle completed but failed tasks here
// remove completed tasks from the tasks list and keep waiting
tasks.RemoveAll(t => completedTasks.Contains(t));
}
Using only await you can achieve the desired behavior:
async Task ProcessAsync(MyRequest request, Sender sender)
{
var result = await request.ProcessAsync();
await sender.SendAsync(result);
}
...
async Task ProcessAll()
{
var tasks = new List<Task>();
foreach(var request in requests)
{
var task = ProcessAsync(request, sender);
// Dont await until all requests are queued up
tasks.Add(task);
}
// Await on all outstanding requests
await Task.WhenAll(tasks);
}
There are already good answers, but in addition of them here is my suggestion too, on how to handle multiple tasks and process each task differently, maybe it will suit your needs. My example is with events, but you can replace them with some kind of state management that fits your needs.
public interface IRequestHandler
{
event Func<object, Task> Ready;
Task ProcessAsync();
}
public class RequestHandler : IRequestHandler
{
// Hier where you wraps your request:
// private object request;
private readonly int value;
public RequestHandler(int value)
=> this.value = value;
public event Func<object, Task> Ready;
public async Task ProcessAsync()
{
await Task.Delay(1000 * this.value);
// Hier where you calls:
// var result = await request.ProcessAsync();
//... then do something over the result or wrap the call in try catch for example
var result = $"RequestHandler {this.value} - [{DateTime.Now.ToLongTimeString()}]";
if (this.Ready is not null)
{
// If result passes send the result to all subscribers
await this.Ready.Invoke($"RequestHandler {this.value} - [{DateTime.Now.ToLongTimeString()}]");
}
}
}
static void Main()
{
var a = new RequestHandler(1);
a.Ready += PrintAsync;
var b = new RequestHandler(2);
b.Ready += PrintAsync;
var c = new RequestHandler(3);
c.Ready += PrintAsync;
var d= new RequestHandler(4);
d.Ready += PrintAsync;
var e = new RequestHandler(5);
e.Ready += PrintAsync;
var f = new RequestHandler(6);
f.Ready += PrintAsync;
var requests = new List<IRequestHandler>()
{
a, b, c, d, e, f
};
var tasks = requests
.Select(x => Task.Run(x.ProcessAsync));
// Hier you must await all of the tasks
Task
.Run(async () => await Task.WhenAll(tasks))
.Wait();
}
static Task PrintAsync(object output)
{
Console.WriteLine(output);
return Task.CompletedTask;
}
I have an array of tasks and I am awaiting them with Task.WhenAll. My tasks are failing frequently, in which case I inform the user with a message box so that she can try again. My problem is that reporting the error is delayed until all tasks are completed. Instead I would like to inform the user as soon as the first task has thrown an exception. In other words I want a version of Task.WhenAll that fails fast. Since no such build-in method exists I tried to make my own, but my implementation does not behave the way I want. Here is what I came up with:
public static async Task<TResult[]> WhenAllFailFast<TResult>(
params Task<TResult>[] tasks)
{
foreach (var task in tasks)
{
await task.ConfigureAwait(false);
}
return await Task.WhenAll(tasks).ConfigureAwait(false);
}
This generally throws faster than the native Task.WhenAll, but usually not fast enough. A faulted task #2 will not be observed before the completion of task #1. How can I improve it so that it fails as fast as possible?
Update: Regarding cancellation, it is not in my requirements right now, but lets say that for consistency the first cancelled task should stop the awaiting immediately. In this case the combining task returned from WhenAllFailFast should have Status == TaskStatus.Canceled.
Clarification: Τhe cancellation scenario is about the user clicking a Cancel button to stop the tasks from completing. It is not about cancelling automatically the incomplete tasks in case of an exception.
Your best bet is to build your WhenAllFailFast method using TaskCompletionSource. You can .ContinueWith() every input task with a synchronous continuation that errors the TCS when the tasks end in the Faulted state (using the same exception object).
Perhaps something like (not fully tested):
using System;
using System.Threading;
using System.Threading.Tasks;
namespace stackoverflow
{
class Program
{
static async Task Main(string[] args)
{
var cts = new CancellationTokenSource();
cts.Cancel();
var arr = await WhenAllFastFail(
Task.FromResult(42),
Task.Delay(2000).ContinueWith<int>(t => throw new Exception("ouch")),
Task.FromCanceled<int>(cts.Token));
Console.WriteLine("Hello World!");
}
public static Task<TResult[]> WhenAllFastFail<TResult>(params Task<TResult>[] tasks)
{
if (tasks is null || tasks.Length == 0) return Task.FromResult(Array.Empty<TResult>());
// defensive copy.
var defensive = tasks.Clone() as Task<TResult>[];
var tcs = new TaskCompletionSource<TResult[]>();
var remaining = defensive.Length;
Action<Task> check = t =>
{
switch (t.Status)
{
case TaskStatus.Faulted:
// we 'try' as some other task may beat us to the punch.
tcs.TrySetException(t.Exception.InnerException);
break;
case TaskStatus.Canceled:
// we 'try' as some other task may beat us to the punch.
tcs.TrySetCanceled();
break;
default:
// we can safely set here as no other task remains to run.
if (Interlocked.Decrement(ref remaining) == 0)
{
// get the results into an array.
var results = new TResult[defensive.Length];
for (var i = 0; i < tasks.Length; ++i) results[i] = defensive[i].Result;
tcs.SetResult(results);
}
break;
}
};
foreach (var task in defensive)
{
task.ContinueWith(check, default, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);
}
return tcs.Task;
}
}
}
Edit: Unwraps AggregateException, Cancellation support, return array of results. Defend against array mutation, null and empty. Explicit TaskScheduler.
I recently needed once again the WhenAllFailFast method, and I revised #ZaldronGG's excellent solution to make it a bit more performant (and more in line with Stephen Cleary's recommendations). The implementation below handles around 3,500,000 tasks per second in my PC.
public static Task<TResult[]> WhenAllFailFast<TResult>(params Task<TResult>[] tasks)
{
if (tasks is null) throw new ArgumentNullException(nameof(tasks));
if (tasks.Length == 0) return Task.FromResult(new TResult[0]);
var results = new TResult[tasks.Length];
var remaining = tasks.Length;
var tcs = new TaskCompletionSource<TResult[]>(
TaskCreationOptions.RunContinuationsAsynchronously);
for (int i = 0; i < tasks.Length; i++)
{
var task = tasks[i];
if (task == null) throw new ArgumentException(
$"The {nameof(tasks)} argument included a null value.", nameof(tasks));
HandleCompletion(task, i);
}
return tcs.Task;
async void HandleCompletion(Task<TResult> task, int index)
{
try
{
var result = await task.ConfigureAwait(false);
results[index] = result;
if (Interlocked.Decrement(ref remaining) == 0)
{
tcs.TrySetResult(results);
}
}
catch (OperationCanceledException)
{
tcs.TrySetCanceled();
}
catch (Exception ex)
{
tcs.TrySetException(ex);
}
}
}
Your loop waits for each of the tasks in pseudo-serial, so that's why it waits for task1 to complete before checking if task2 failed.
You might find this article helpful on a pattern for aborting after the first failure: http://gigi.nullneuron.net/gigilabs/patterns-for-asynchronous-composite-tasks-in-c/
public static async Task<TResult[]> WhenAllFailFast<TResult>(
params Task<TResult>[] tasks)
{
var taskList = tasks.ToList();
while (taskList.Count > 0)
{
var task = await Task.WhenAny(taskList).ConfigureAwait(false);
if(task.Exception != null)
{
// Left as an exercise for the reader:
// properly unwrap the AggregateException;
// handle the exception(s);
// cancel the other running tasks.
throw task.Exception.InnerException;
}
taskList.Remove(task);
}
return await Task.WhenAll(tasks).ConfigureAwait(false);
}
I'm adding one more answer to this problem, not because I've found a faster solution, but because I am now a bit skeptical about starting multiple async void operations on an unknown SynchronizationContext. The solution I am proposing here is significantly slower. It's about 3 times slower than #ZaldronGG's excellent solution, and about 10 times slower than my previous async void-based implementation. It has though the advantage that after the completion of the returned Task<TResult[]>, it doesn't leak fire-and-forget continuations attached on the observed tasks. When this task is completed, all the continuations created internally by the WhenAllFailFast method have been cleaned up. Which is a desirable behavior for APIs is general, but in many scenarios it might not be important.
public static Task<TResult[]> WhenAllFailFast<TResult>(params Task<TResult>[] tasks)
{
ArgumentNullException.ThrowIfNull(tasks);
CancellationTokenSource cts = new();
Task<TResult> failedTask = null;
TaskContinuationOptions flags = TaskContinuationOptions.DenyChildAttach |
TaskContinuationOptions.ExecuteSynchronously;
Action<Task<TResult>> continuationAction = new(task =>
{
if (!task.IsCompletedSuccessfully)
if (Interlocked.CompareExchange(ref failedTask, task, null) is null)
cts.Cancel();
});
IEnumerable<Task> continuations = tasks.Select(task => task
.ContinueWith(continuationAction, cts.Token, flags, TaskScheduler.Default));
return Task.WhenAll(continuations).ContinueWith(allContinuations =>
{
cts.Dispose();
var localFailedTask = Volatile.Read(ref failedTask);
if (localFailedTask is not null)
return Task.WhenAll(localFailedTask);
// At this point all the tasks are completed successfully
Debug.Assert(tasks.All(t => t.IsCompletedSuccessfully));
Debug.Assert(allContinuations.IsCompletedSuccessfully);
return Task.WhenAll(tasks);
}, default, flags, TaskScheduler.Default).Unwrap();
}
This implementation is similar to ZaldronGG's in that it attaches one continuation on each task, with the difference being that these continuations are cancelable, and they are canceled en masse when the first non-successful task is observed. It also uses the Unwrap technique that I've discovered recently, which eliminates the need for the manual completion of a TaskCompletionSource<TResult[]> instance, and usually makes for a concise implementation.
If I have a list of tasks which I want to execute together but at the same time I want to execute a certain number of them together, so I await for one of them until one of them finishes, which then should mean awaiting should stop and a new task should be allowed to start, but when one of them finishes, I don't know how to stop awaiting for the task which is currently being awaited, I don't want to cancel the task, just stop awaiting and let it continue running in the background.
I have the following code
foreach (var link in SharedVars.DownloadQueue)
{
if (currentRunningCount != batch)
{
var task = DownloadFile(extraPathPerLink, link, totalLen);
_ = task.ContinueWith(_ =>
{
downloadQueueLinksTasks.Remove(task);
currentRunningCount--;
// TODO SHOULD CHANGE WHAT IS AWAITED
});
currentRunningCount++;
downloadQueueLinksTasks.Add(task);
}
if (currentRunningCount == batch)
{
// TODO SHOULD NOT AWAIT 0
await downloadQueueLinksTasks[0];
}
}
I found about Task.WhenAny but from this comment here I understood that the other tasks will be ignored so it's not the solution I want to achieve.
I'm sorry if the question is stupid or wrong but I can't seem to find any information related on how to solve it, or what is the name of the operation I want to achieve so I can even search correctly.
Solution Edit
All the answers provided are correct, I accepted the one I decided to use but still all of them are correct.
Thank you everyone, I learned a lot from all of you from these different answers and different ways to approach the problem and how to think about it.
What I learned about this specific problem was that I still needed to await for the other tasks left, so the solution was to have the Task.WhenAny inside the loop (which returns the finished task (this is also important)) AND Task.WhenAll outside the loop to await the other left tasks.
Task.WhenAny returns the Task which completed.
foreach (var link in SharedVars.DownloadQueue)
{
var task = DownloadFile(extraPathPerLink, link, totalLen);
downloadQueueLinksTasks.Add(task);
if (downloadQueueLinksTasks.Count == batch)
{
// Wait for any Task to complete, then remove it from
// the list of pending tasks.
var completedTask = await Task.WhenAny(downloadQueueLinksTasks);
downloadQueueLinksTasks.Remove(completedTask);
}
}
// Wait for all of the remaining Tasks to complete
await Task.WhenAll(downloadQueueLinksTasks);
You can use Task.WaitAny()
Here is the demonstration of the behavior:
public static async Task Main(string[] args)
{
IList<Task> tasks = new List<Task>();
tasks.Add(TestAsync(0));
tasks.Add(TestAsync(1));
tasks.Add(TestAsync(2));
tasks.Add(TestAsync(3));
tasks.Add(TestAsync(4));
tasks.Add(TestAsync(5));
var result = Task.WaitAny(tasks.ToArray());
Console.WriteLine("returned task id is {0}", result);
///do other operations where
//before exiting wait for other tasks so that your tasks won't get cancellation signal
await Task.WhenAll(tasks.ToArray());
}
public static async Task TestAsync(int i)
{
Console.WriteLine("Staring to wait" + i);
await Task.Delay(new Random().Next(1000, 10000));
Console.WriteLine("Task finished" + i);
}
Output:
Staring to wait0
Staring to wait1
Staring to wait2
Staring to wait3
Staring to wait4
Staring to wait5
Task finished0
returned task id is 0
Task finished4
Task finished2
Task finished1
Task finished5
Task finished3
What you're asking about is throttling, which for asynchronous code is best expressed via SemaphoreSlim:
var semaphore = new SemaphoreSlim(batch);
var tasks = SharedVars.DownloadQueue.Select(link =>
{
await semaphore.WaitAsync();
try { return DownloadFile(extraPathPerLink, link, totalLen); }
finally { semaphore.Release(); }
});
var results = await Task.WhenAll(tasks);
You should use Microsoft's Reactive Framework (aka Rx) - NuGet System.Reactive and add using System.Reactive.Linq; - then you can do this:
IDisposable subscription =
SharedVars.DownloadQueue
.ToObservable()
.Select(link =>
Observable.FromAsync(() => DownloadFile(extraPathPerLink, link, totalLen)))
.Merge(batch) //max concurrent downloads
.Subscribe(file =>
{
/* process downloaded file here
(no longer a task) */
});
If you need to stop the downloads before they would naturally finish just call subscription.Dispose().
This question already has answers here:
Using async/await for multiple tasks
(8 answers)
Closed 4 years ago.
I have a method in which I'm retrieving a list of deployments. For each deployment I want to retrieve an associated release. Because all calls are made to an external API, I now have a foreach-loop in which those calls are made.
public static async Task<List<Deployment>> GetDeployments()
{
try
{
var depjson = await GetJson($"{BASEURL}release/deployments?deploymentStatus=succeeded&definitionId=2&definitionEnvironmentId=5&minStartedTime={MinDateTime}");
var deployments = (JsonConvert.DeserializeObject<DeploymentWrapper>(depjson))?.Value?.OrderByDescending(x => x.DeployedOn)?.ToList();
foreach (var deployment in deployments)
{
var reljson = await GetJson($"{BASEURL}release/releases/{deployment.ReleaseId}");
deployment.Release = JsonConvert.DeserializeObject<Release>(reljson);
}
return deployments;
}
catch (Exception)
{
throw;
}
}
This all works perfectly fine. However, I do not like the await in the foreach-loop at all. I also believe this is not considered good practice. I just don't see how to refactor this so the calls are made parallel, because the result of each call is used to set a property of the deployment.
I would appreciate any suggestions on how to make this method faster and, whenever possible, avoid the await-ing in the foreach-loop.
There is nothing wrong with what you are doing now. But there is a way to call all tasks at once instead of waiting for a single task, then processing it and then waiting for another one.
This is how you can turn this:
wait for one -> process -> wait for one -> process ...
into
wait for all -> process -> done
Convert this:
foreach (var deployment in deployments)
{
var reljson = await GetJson($"{BASEURL}release/releases/{deployment.ReleaseId}");
deployment.Release = JsonConvert.DeserializeObject<Release>(reljson);
}
To:
var deplTasks = deployments.Select(d => GetJson($"{BASEURL}release/releases/{d.ReleaseId}"));
var reljsons = await Task.WhenAll(deplTasks);
for(var index = 0; index < deployments.Count; index++)
{
deployments[index].Release = JsonConvert.DeserializeObject<Release>(reljsons[index]);
}
First you take a list of unfinished tasks. Then you await it and you get a collection of results (reljson's). Then you have to deserialize them and assign to Release.
By using await Task.WhenAll() you wait for all the tasks at the same time, so you should see a performance boost from that.
Let me know if there are typos, I didn't compile this code.
Fcin suggested to start all Tasks, await for them all to finish and then start deserializing the fetched data.
However, if the first Task is already finished, but the second task not, and internally the second task is awaiting, the first task could already start deserializing. This would shorten the time that your process is idly waiting.
So instead of:
var deplTasks = deployments.Select(d => GetJson($"{BASEURL}release/releases/{d.ReleaseId}"));
var reljsons = await Task.WhenAll(deplTasks);
for(var index = 0; index < deployments.Count; index++)
{
deployments[index].Release = JsonConvert.DeserializeObject<Release>(reljsons[index]);
}
I'd suggest the following slight change:
// async fetch the Release data of Deployment:
private async Task<Release> FetchReleaseDataAsync(Deployment deployment)
{
var reljson = await GetJson($"{BASEURL}release/releases/{deployment.ReleaseId}");
return JsonConvert.DeserializeObject<Release>(reljson);
}
// async fill the Release data of Deployment:
private async Task FillReleaseDataAsync(Deployment deployment)
{
deployment.Release = await FetchReleaseDataAsync(deployment);
}
Then your procedure is similar to the solution that Fcin suggested:
IEnumerable<Task> tasksFillDeploymentWithReleaseData = deployments.
.Select(deployment => FillReleaseDataAsync(deployment)
.ToList();
await Task.WhenAll(tasksFillDeploymentWithReleaseData);
Now if the first task has to wait while fetching the release data, the 2nd task begins and the third etc. If the first task already finished fetching the release data, but the other tasks are awaiting for their release data, the first task starts already deserializing it and assigns the result to deployment.Release, after which the first task is complete.
If for instance the 7th task got its data, but the 2nd task is still waiting, the 7th task can deserialize and assign the data to deployment.Release. Task 7 is completed.
This continues until all tasks are completed. Using this method there is less waiting time because as soon as one task has its data it is scheduled to start deserializing
If i understand you right and you want to make the var reljson = await GetJson parralel:
Try this:
Parallel.ForEach(deployments, (deployment) =>
{
var reljson = await GetJson($"{BASEURL}release/releases/{deployment.ReleaseId}");
deployment.Release = JsonConvert.DeserializeObject<Release>(reljson);
});
you might limit the number of parallel executions such as:
Parallel.ForEach(
deployments,
new ParallelOptions { MaxDegreeOfParallelism = 4 },
(deployment) =>
{
var reljson = await GetJson($"{BASEURL}release/releases/{deployment.ReleaseId}");
deployment.Release = JsonConvert.DeserializeObject<Release>(reljson);
});
you might also want to be able to break the loop:
Parallel.ForEach(deployments, (deployment, state) =>
{
var reljson = await GetJson($"{BASEURL}release/releases/{deployment.ReleaseId}");
deployment.Release = JsonConvert.DeserializeObject<Release>(reljson);
if (noFurtherProcessingRequired) state.Break();
});
Say I have 10N items(I need to fetch them via http protocol), in the code N Tasks are started to get data, each task takes 10 items in sequence. I put the items in a ConcurrentQueue<Item>. After that, the items are processed in a thread-unsafe method one by one.
async Task<Item> GetItemAsync()
{
//fetch one item from the internet
}
async Task DoWork()
{
var tasks = new List<Task>();
var items = new ConcurrentQueue<Item>();
var handles = new List<ManualResetEvent>();
for i 1 -> N
{
var handle = new ManualResetEvent(false);
handles.Add(handle);
tasks.Add(Task.Factory.StartNew(async delegate
{
for j 1 -> 10
{
var item = await GetItemAsync();
items.Enqueue(item);
}
handle.Set();
});
}
//begin to process the items when any handle is set
WaitHandle.WaitAny(handles);
while(true)
{
if (all handles are set && items collection is empty) //***
break;
//in another word: all tasks are really completed
while(items.TryDequeue(out item))
{
AThreadUnsafeMethod(item); //process items one by one
}
}
}
I don't know what if condition can be placed in the statement marked ***. I can't use Task.IsCompleted property here, because I use await in the task, so the task is completed very soon. And a bool[] that indicates whether the task is executed to the end looks really ugly, because I think ManualResetEvent can do the same work. Can anyone give me a suggestion?
Well, you could build this yourself, but I think it's tons easier with TPL Dataflow.
Something like:
static async Task DoWork()
{
// By default, ActionBlock uses MaxDegreeOfParallelism == 1,
// so AThreadUnsafeMethod is not called in parallel.
var block = new ActionBlock<Item>(AThreadUnsafeMethod);
// Start off N tasks, each asynchronously acquiring 10 items.
// Each item is sent to the block as it is received.
var tasks = Enumerable.Range(0, N).Select(Task.Run(
async () =>
{
for (int i = 0; i != 10; ++i)
block.Post(await GetItemAsync());
})).ToArray();
// Complete the block when all tasks have completed.
Task.WhenAll(tasks).ContinueWith(_ => { block.Complete(); });
// Wait for the block to complete.
await block.Completion;
}
You can do a WaitOne with a timeout of zero to check the state. Something like this should work:
if (handles.All(handle => handle.WaitOne(TimeSpan.Zero)) && !items.Any())
break;
http://msdn.microsoft.com/en-us/library/cc190477.aspx
Thanks all. At last I found CountDownEvent is very suitable for this scenario. The general implementation looks like this:(for others' information)
for i 1 -> N
{
//start N tasks
//invoke CountDownEvent.Signal() at the end of each task
}
//see if CountDownEvent.IsSet here