I am trying to use Task.WhenAll(tasks).Wait(timeout) to wait for tasks to complete and after that process task results.
Consider this example:
var tasks = new List<Task<Foo>>();
tasks.Add(Task.Run(() => GetData1()));
tasks.Add(Task.Run(() => GetData2()));
Task.WhenAll(tasks).Wait(TimeSpan.FromSeconds(5));
var completedTasks = tasks
.Where(t => t.Status == TaskStatus.RanToCompletion)
.Select(t => t.Result)
.ToList();
// Process completed tasks
// ...
private Foo GetData1()
{
Thread.Sleep(TimeSpan.FromSeconds(4));
return new Foo();
}
private Foo GetData2()
{
Thread.Sleep(TimeSpan.FromSeconds(10));
// How can I get the result of this task once it completes?
return new Foo();
}
It is possible that one of these tasks will not complete their execution within 5 second timeout.
Is it possible to somehow process results of the tasks that have completed after specified timeout? Maybe I am not using right approach in this situation?
EDIT:
I am trying to get all task results that managed to complete within specified timeout. There could be the following outcomes after Task.WhenAll(tasks).Wait(TimeSpan.FromSeconds(5)):
First task completes within 5 seconds.
Second task completes within 5 seconds.
Both tasks complete within 5 seconds.
None of the tasks complete within 5 seconds. Is it possible to get task results that haven't completed within 5 seconds, but have completed later, lets say, after 10 seconds?
In the end with help of the user who removed his answer, I ended up with this solution:
private const int TimeoutInSeconds = 5;
private static void Main(string[] args)
{
var tasks = new List<Task>()
{
Task.Run( async() => await Task.Delay(30)),
Task.Run( async() => await Task.Delay(300)),
Task.Run( async() => await Task.Delay(6000)),
Task.Run( async() => await Task.Delay(8000))
};
Task.WhenAll(tasks).Wait(TimeSpan.FromSeconds(TimeoutInSeconds));
var completedTasks = tasks
.Where(t => t.Status == TaskStatus.RanToCompletion).ToList();
var incompleteTasks = tasks
.Where(t => t.Status != TaskStatus.RanToCompletion).ToList();
Task.WhenAll(incompleteTasks)
.ContinueWith(t => { ProcessDelayedTasks(incompleteTasks); });
ProcessCompletedTasks(completedTasks);
Console.ReadKey();
}
private static void ProcessCompletedTasks(IEnumerable<Task> delayedTasks)
{
Console.WriteLine("Processing completed tasks...");
}
private static void ProcessDelayedTasks(IEnumerable<Task> delayedTasks)
{
Console.WriteLine("Processing delayed tasks...");
}
Instead of Waitall, you probably just want to do some sort of Spin/sleep of 5 seconds and then query the list as you are above.
You should then be able to enumerate again after a few more seconds to see what else has finished.
If performance is a concern, you may want to have additional 'wrapping' to see if All tasks have completed before 5 seconds.
I think there's a possible loss of task items between
var completedTasks = tasks.Where(t => t.Status == TaskStatus.RanToCompletion).ToList();
and
var incompleteTasks = tasks.Where(t => t.Status != TaskStatus.RanToCompletion).ToList();
because some tasks may ran to completition during this time.
As a workaround (not correct though) you coud swap these lines. In this case some tasks may present in each (completedTasks and incompleteTasks) list. But maybe it's better than to be lost completely.
A unit test to compare number of started tasks and number of tasks in completedTasks and incompleteTasks lists may also be useful.
Related
I was trying to execute that Reactive + ReactiveUI Code
Observable.FromAsync(async () => //async code that must run on UI Thread, RxApp.MainThreadScheduler);
But that doesn't get executed on MainUIThread.
Can anyone tell me what's the point of having IScheduler as parameter since it doesn't seem to get executed on UI?
Try running this code:
void Main()
{
Console.WriteLine($"M {System.Threading.Thread.CurrentThread.ManagedThreadId}");
using (var els = new EventLoopScheduler())
{
els.Schedule(() => Console.WriteLine($"ELS {System.Threading.Thread.CurrentThread.ManagedThreadId}"));
Observable.FromAsync(() => GetNumberAsync(1)).Subscribe(x => Console.WriteLine($"S1 {System.Threading.Thread.CurrentThread.ManagedThreadId}"));
Observable.FromAsync(() => GetNumberAsync(2), els).Subscribe(x => Console.WriteLine($"S2 {System.Threading.Thread.CurrentThread.ManagedThreadId}"));
Console.ReadLine();
}
}
Task<int> GetNumberAsync(int run)
{
Console.WriteLine($"{run} GNA {System.Threading.Thread.CurrentThread.ManagedThreadId}");
return Task.Run(() =>
{
Console.WriteLine($"{run} TR {System.Threading.Thread.CurrentThread.ManagedThreadId}");
return 42;
});
}
I get an output like this:
M 1
ELS 20
1 GNA 1
2 GNA 1
1 TR 12
2 TR 13
S1 11
S2 20
You'll note that the EventLoopScheduler creates a thread with ID 20 and the continuation out of the FromAsync when using the els is run on that thread.
The scheduler isn't the one the task is run on, it's the one that the next operator in the pipeline executes using.
So I came with that solution:
Observable
...
.ObserveOnDispatcher()
.Select(_ => InvokingReturningTaskMethod().ToObservable())
.Concat()
.Subscribe()
That seems to be running on Dispatcher Thread
I wanted to find a better solution tho. Something that doesn't require to
InvokingReturningTaskMethod().ToObservable()
I have three methods that I call to do some number crunching that are as follows
results.LeftFront.CalcAi();
results.RightFront.CalcAi();
results.RearSuspension.CalcAi(geom, vehDef.Geometry.LTa.TaStiffness, vehDef.Geometry.RTa.TaStiffness);
Each of the functions is independent of each other and can be computed in parallel with no dead locks.
What is the easiest way to compute these in parallel without the containing method finishing until all three are done?
See the TPL documentation. They list this sample:
Parallel.Invoke(() => DoSomeWork(), () => DoSomeOtherWork());
So in your case this should just work:
Parallel.Invoke(
() => results.LeftFront.CalcAi(),
() => results.RightFront.CalcAi(),
() => results.RearSuspension.CalcAi(geom,
vehDef.Geometry.LTa.TaStiffness,
vehDef.Geometry.RTa.TaStiffness));
EDIT: The call returns after all actions have finished executing. Invoke() is does not guarantee that they will indeed run in parallel, nor does it guarantee the order in which the actions execute.
You can do this with tasks too (nicer if you later need Cancellation or something like results)
var task1 = Task.Factory.StartNew(() => results.LeftFront.CalcAi());
var task2 = Task.Factory.StartNew(() => results.RightFront.CalcAi());
var task3 = Task.Factory.StartNew(() =>results.RearSuspension.CalcAi(geom,
vehDef.Geometry.LTa.TaStiffness,
vehDef.Geometry.RTa.TaStiffness));
Task.WaitAll(task1, task2, task3);
In .NET 4, Microsoft introduced the Task Parallel Library which was designed to handle this kind of problem, see Parallel Programming in the .NET Framework.
To run parallel methods which are independent of each other ThreadPool.QueueUserWorkItem can also be used. Here is the sample method-
public static void ExecuteParallel(params Action[] tasks)
{
// Initialize the reset events to keep track of completed threads
ManualResetEvent[] resetEvents = new ManualResetEvent[tasks.Length];
// Launch each method in it's own thread
for (int i = 0; i < tasks.Length; i++)
{
resetEvents[i] = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(new WaitCallback((object index) =>
{
int taskIndex = (int)index;
// Execute the method
tasks[taskIndex]();
// Tell the calling thread that we're done
resetEvents[taskIndex].Set();
}), i);
}
// Wait for all threads to execute
WaitHandle.WaitAll(resetEvents);
}
More detail about this function can be found here:
http://newapputil.blogspot.in/2016/03/running-parallel-tasks-using.html
var task1 = SomeLongRunningTask();
var task2 = SomeOtherLongRunningTask();
await Task.WhenAll(task1, task2);
The benefit of this over Task.WaitAll is that this will release the thread and await the completion of the two tasks.
Let's say I have a list of objects myObjs of type List<Foo>.
I have a polly policy :
var policy = Policy.Handle<Exception>().RetryForever();
which i want to run methods on in parralel, but keep retrying each as they fail.
for (int i = 0; i < myObjs.Count; i++)
{
var obj = myObjs[i];
policy.Execute(() => Task.Factory.StartNew(() => obj.Do(), TaskCreationOptions.LongRunning));
}
Will this be called in parallel and retry each obj? So if myObjs[5].do() fails, will only that get retried while other objects just get executed once?
Also, am I supposed to use the ExecuteAsync() method which accepts Func<Task> instead of the Execute(Action) one as shown in the example? Do() is just a synchronous method, being launched in a separate thread.
Actual code looks like this where each() is just a foreach wrapper()
_consumers.ForEach(c => policy.Execute(() => Task.Factory.StartNew(() => c.Consume(startFromBeg), TaskCreationOptions.LongRunning)));
EDIT:
I tried the code:
class Foo
{
private int _i;
public Foo(int i)
{
_i = i;
}
public void Do()
{
//var rnd = new Random();
if (_i==2)
{
Console.WriteLine("err"+_i);
throw new Exception();
}
Console.WriteLine(_i);
}
}
var policy = Policy.Handle<Exception>().Retry(3);
var foos=Enumerable.Range(0, 5).Select(x => new Foo(x)).ToList();
foos.ForEach(c => policy.Execute(() => Task.Factory.StartNew(() => c.Do(), TaskCreationOptions.LongRunning)));
but am getting result:
0 1 err2 3 4 5
I thought it would retry 2 a few more times, but doesn't. Any idea why?
Whatever owns the tasks must wait for them somehow. Otherwise, exceptions will be ignored and the code will end before the tasks actually complete. So yes, you should probably be using policy.ExecuteAsync() instead. It would look something like this:
var tasks = myObjs
.Select(obj => Task.Factory.StartNew(() => obj.Do(), TaskCreationOptions.LongRunning))
.ToList();
// sometime later
await Task.WhenAll(tasks);
This question already has answers here:
TPL wait for task to complete with a specific return value
(2 answers)
Closed 7 years ago.
There are three tasks that I wish to run in parallel. I wish to examine the result of the first task that finished and do a check to decide if the result is good. If yes, I cancel all other tasks and return this result, if not, I will wait for next completed task and check if that is good and do the same if it is. (Think of being good as some simple check on a member of OutputDataType).
I continue this until I obtain a completed task with a good result, or all tasks return with results that are not good, in which case I return null.
Thank you in advance.
using System;
using System.Threading;
using System.Threading.Tasks;
namespace myNamespace
{
class Program
{
static void Main()
{
InputDataType data = getMyData();
OutputDataType x = Foo(data);
}
static OutputDataType Foo(InputDataType data)
{
Task<OutputDataType> task1 = null, task2 = null, taks3 = null;
Task<OutPutDataType>[] TaskArray = { task1, task2, task3 };
task1 = Task<SolutionInterpreters>.Factory.StartNew(() => RunFunc1(data));
task2 = Task<SolutionInterpreters>.Factory.StartNew(() => RunFunc2(data));
task3 = Task<SolutionInterpreters>.Factory.StartNew(() => RunFunc3(data));
/*
.
.
.
*/
}
}
}
In TPL, task cancellation is cooperative, so you need to define some means of indicating to your other tasks that they should stop executing. The most common way to do this is through CancellationTokenSource, whose token can be checked for cancellation within your RunFuncX methods.
static void Main()
{
InputDataType data = getMyData();
OutputDataType x = Foo(data).Result;
}
static async Task<OutputDataType> Foo(InputDataType data)
{
// Spawn your tasks, passing the cancellation token.
var cts = new CancellationTokenSource();
var task1 = Task.Factory.StartNew(() => RunFunc1(data, cts.Token));
var task2 = Task.Factory.StartNew(() => RunFunc2(data, cts.Token));
var task3 = Task.Factory.StartNew(() => RunFunc3(data, cts.Token));
var tasks = new [] { task1, task2, task3 };
// Loop while tasks are running.
while (tasks.Any())
{
// Wait for any task to complete.
var completedTask = await Task.WhenAny(tasks);
// If its result is good, indicate cancellation to the other tasks,
// and return the result.
if (IsResultGood(completedTask.Result))
{
cts.Cancel();
return completedTask.Result;
}
// Otherwise, remove the completed task from the array,
// and proceed to the next iteration.
tasks = tasks.Where(t => t != completedTask).ToArray();
}
// No good results.
return null;
}
Per Spo1ler's comment, I've updated your Foo method to an asynchronous implementation. This assumes you're using C# 5.0. When consuming the method, use await to get its result. If you're really running as a console application, your topmost call would need to block, so you can just get the task's Result.
I have this situation:
var tasks = new List<ITask> ...
Parallel.ForEach(tasks, currentTask => currentTask.Execute() );
Is it possible to instruct PLinq to wait for 500ms before the next thread is spawned?
System.Threading.Thread.Sleep(5000);
You are using Parallel.Foreach totally wrong, You should make a special Enumerator that rate limits itself to getting data once every 500 ms.
I made some assumptions on how your DTO works due to you not providing any details.
private IEnumerator<SomeResource> GetRateLimitedResource()
{
SomeResource someResource = null;
do
{
someResource = _remoteProvider.GetData();
if(someResource != null)
{
yield return someResource;
Thread.Sleep(500);
}
} while (someResource != null);
}
here is how your paralell should look then
Parallel.ForEach(GetRateLimitedResource(), SomeFunctionToProcessSomeResource);
There are already some good suggestions. I would agree with others that you are using PLINQ in a manner it wasn't meant to be used.
My suggestion would be to use System.Threading.Timer. This is probably better than writing a method that returns an IEnumerable<> that forces a half second delay, because you may not need to wait the full half second, depending on how much time has passed since your last API call.
With the timer, it will invoke a delegate that you've provided it at the interval you specify, so even if the first task isn't done, a half second later it will invoke your delegate on another thread, so there won't be any extra waiting.
From your example code, it sounds like you have a list of tasks, in this case, I would use System.Collections.Concurrent.ConcurrentQueue to keep track of the tasks. Once the queue is empty, turn off the timer.
You could use Enumerable.Aggregate instead.
var task = tasks.Aggregate((t1, t2) =>
t1.ContinueWith(async _ =>
{ Thread.Sleep(500); return t2.Result; }));
If you don't want the tasks chained then there is also the overload to Select assuming the tasks are in order of delay.
var tasks = Enumerable
.Range(1, 10)
.Select(x => Task.Run(() => x * 2))
.Select((x, i) => Task.Delay(TimeSpan.FromMilliseconds(i * 500))
.ContinueWith(_ => x.Result));
foreach(var result in tasks.Select(x => x.Result))
{
Console.WriteLine(result);
}
From the comments a better options would be to guard the resource instead of using the time delay.
static object Locker = new object();
static int GetResultFromResource(int arg)
{
lock(Locker)
{
Thread.Sleep(500);
return arg * 2;
}
}
var tasks = Enumerable
.Range(1, 10)
.Select(x => Task.Run(() => GetResultFromResource(x)));
foreach(var result in tasks.Select(x => x.Result))
{
Console.WriteLine(result);
}
In this case how about a Producer-Consumer pattern with a BlockingCollection<T>?
var tasks = new BlockingCollection<ITask>();
// add tasks, if this is an expensive process, put it out onto a Task
// tasks.Add(x);
// we're done producin' (allows GetConsumingEnumerable to finish)
tasks.CompleteAdding();
RunTasks(tasks);
With a single consumer thread:
static void RunTasks(BlockingCollection<ITask> tasks)
{
foreach (var task in tasks.GetConsumingEnumerable())
{
task.Execute();
// this may not be as accurate as you would like
Thread.Sleep(500);
}
}
If you have access to .Net 4.5 you can use Task.Delay:
static void RunTasks(BlockingCollection<ITask> tasks)
{
foreach (var task in tasks.GetConsumingEnumerable())
{
Task.Delay(500)
.ContinueWith(() => task.Execute())
.Wait();
}
}