I have following code:
static async Task Main()
{
ConcurrentExclusiveSchedulerPair concurrentExclusiveSchedulerPair = new ConcurrentExclusiveSchedulerPair(TaskScheduler.Default, 4);
var factory = new TaskFactory(concurrentExclusiveSchedulerPair.ConcurrentScheduler);
for (int i = 0; i < 10; i++)
{
factory.StartNew(ThreadTask);
}
concurrentExclusiveSchedulerPair.Complete();
await concurrentExclusiveSchedulerPair.Completion;
Console.WriteLine("Completed");
}
private static async Task ThreadTask()
{
var random = new Random();
await Task.Delay(random.Next(100, 200));
Console.WriteLine($"Finished {Thread.CurrentThread.ManagedThreadId}");
}
and program finishes executing before tasks are completed. I understand why does it happens as ThreadTask returns completed task and from ConcurrentExclusiveSchedulerPair point of view it does finish executing. I also know few workarounds but is there any correct way to run this pattern with async?
The concept of the TaskScheduler was devised before the advent of the async/await, and it ended up not being compatible with it. You can see an experiment that demonstrates this incompatibility here: How to run a Task on a custom TaskScheduler using await?
The abstraction that is available for controlling the behavior of async/await is the SynchronizationContext. It is quite similar to a TaskScheduler. So much actually that some people have been wondering why we need both: What is the conceptual difference between SynchronizationContext and TaskScheduler.
If you are interested for something like a SingleThreadSynchronizationContext, you can find an implementation here: Await, SynchronizationContext, and Console Apps
TaskScheduler is compatible with async/await, but some of the behavior can be surprising.
When await captures its context, it captures the current SynchronizationContext unless it is null, in which case it captures the current TaskScheduler. So, ThreadTask will begin executing with the concurrent scheduler, and after its await it will resume on that same concurrent scheduler.
However, the semantics can be surprising, because the way async works with a task scheduler is that the async method is split into multiple tasks. Each await is a "split" point where the method is broken up. And only those smaller tasks are what is actually scheduled by the TaskScheduler.
So in your case, your code is starting 10 ThreadTask invocations, each running on the concurrent scheduler, and each of them hits an await point. Then the code calls Complete on the scheduler, which tells it not to accept any more tasks. Then when the awaits complete, they schedule the async method continuation (as a task) to that task scheduler, which as already been completed.
So, while technically await was designed to work with TaskScheduler, in practice few people use it that way. In particular, the "concurrent" and "exclusive" task schedulers can have surprising semantics, since a method that is suspended due to an await does not count as "running" to a task scheduler.
Related
I have two long running tasks that I want to execute, to be followed with a continuation. The problem I am having is that the continuation is running before the two long running tasks have signalled. Those two tasks are not erroring, they've barely started when the continuation commences.
The intent is that the first two tasks can run in parallel, but the continuation should only run once they've both finished.
I have probably overlooked something really simple, but near as I can tell I am following the patterns suggested in the question Running multiple async tasks and waiting for them all to complete. I realise I could probably dispense with the .ContinueWith() as the code following the Task.WhenAll() will implicitly become a continuation, but I prefer to keep the explicit continuation (unless it is the cause of my issue).
private async Task MyMethod()
{
var tasks = new List<Task>
{
new Task(async () => await DoLongRunningTask1()),
new Task(async () => await DoLongRunningTask2())
};
Parallel.ForEach(tasks, async task => task.Start());
await Task.WhenAll(tasks)
.ContinueWith(async t =>{
//another long running bit of code with some parallel
//execution based on the results of the first two tasks
}
, TaskContinuationOptions.OnlyOnRanToCompletion);
}
private async Task DoLongRunningTask1()
{
... long running operation ...
}
private async Task DoLongRunningTask2()
{
... another long running operation ...
}
I appreciate any help or suggestions!
Your code has multiple issues:
Creating tasks with the non-generic Task constructor, with async delegate. This constructor accepts an Action, not a Func<Task>, so your async delegate is actually an async void, which is something to avoid. You can fix this by creating nested Task<Task>s, where the outer task represents the launching of the inner task.
Using the Parallel.ForEach with async delegate. This again results in async void delegate, for the same reason. The correct API to use when you want to parallelize asynchronous delegates is the Parallel.ForEachAsync, available from .NET 6 and later.
Using parallelism to start tasks. Starting a task with Start is not CPU-bound. The Start doesn't execute the task, it just schedules it for execution, normally on the ThreadPool. So you can replace the Parallel.ForEach with a simple foreach loop or the List.ForEach.
Awaiting an unwrapped ContinueWith continuation that has an async delegate. This continuation returns a nested Task<Task>. You should Unwrap this nested task, otherwise you are awaiting the launching of the asynchronous operation, not its completion.
Here is your code "fixed":
private async Task MyMethod()
{
List<Task<Task>> tasksOfTasks = new()
{
new Task<Task>(async () => await DoLongRunningTask1()),
new Task<Task>(async () => await DoLongRunningTask2())
};
tasksOfTasks.ForEach(taskTask => taskTask.Start());
await Task.WhenAll(tasksOfTasks.Select(taskTask => taskTask.Unwrap()))
.ContinueWith(async t =>
{
//another long running bit of code with some parallel
//execution based on the results of the first two tasks
}, TaskContinuationOptions.OnlyOnRanToCompletion).Unwrap();
}
In case of failure in the DoLongRunningTask1 or the DoLongRunningTask2 methods, this code will most likely result in a TaskCanceledException, which is what happens when you pass the TaskContinuationOptions.OnlyOnRanToCompletion flag and the antecedent task does not run to completion successfully. This behavior is unlikely to be what you want.
The above code also violates the guideline CA2008: Do not create tasks without passing a TaskScheduler. This guideline says that being depended on the ambient TaskScheduler.Current is not a good idea, and specifying explicitly the TaskScheduler.Default is recommended. You'll have to specify it in both the Start and ContinueWith methods.
If you think that all this is overly complicated, you are right. That's not the kind of code that you want to write and maintain, unless you have spent months studying the Task Parallel Library, you know it in and out, and you have no other higher-level alternative. It's the kind of code that you might find in specialized libraries, not in application code.
You are overusing async. Below is more cleaner rewrite of what you have done above.
private async Task MyMethod()
{
var tasks = new Task[] {DoLongRunningTask1(), DoLongRunningTask2()];
await Task.WhenAll(tasks);
//another long running bit of code with some parallel
//execution based on the results of the first two tasks
}
private async Task DoLongRunningTask1()
{
... long running operation ...
}
private async Task DoLongRunningTask2()
{
... another long running operation ...
}
I assumed DoLongRunningTask's are async in nature, if they are syncronious then start them using Task.Run within a ThreadPool thread and use resulting Task.
The work I'm trying to schedule is using the await keyword, hence the work itself is of type async Task.
I can't use Task.Factory.StartNew, even when I pass my scheduler, because it will consider the task "done" when it hits the first await of the work, making it leave the scheduler and continuing on the default scheduler.
I can't use var a = new Task(DoWork....); a.Start(myScheduler), because I'd be using an async void method, and we know the pitfalls to that.
I can't use Task.Run(), because it doesn't allow you to change the scheduler on which the task will be scheduled to launch.
If I can't use any of that, how will I achieve my goal, which is to schedule my async Work, which has to complete execution on the scheduler it was initially started on?
First, be aware that the TaskScheduler only applies to executing tasks. So when using an async delegate, the TaskScheduler will only be used in-between await points; when the async code is (asynchronously) waiting in an await, then there is no code "in" that scheduler. For some schedulers this isn't a problem, but it is a problem with schedulers like ConcurrentExclusiveSchedulerPair because the code is not "in" the scheduler during an await.
If your TaskScheduler will work as expected with async code, then you can create a task running on it by calling Task.Factory.StartNew and passing your TaskScheduler. This will return a Task<Task> or Task<Task<T>> since StartNew doesn't understand async code; you can call Unwrap() on that value to get a "normal" asynchronous Task/Task<T>.
Personally, I prefer creating your own TaskFactory instance with your own TaskScheduler (and other options), and then call that StartNew with Unwrap. E.g:
var factory = new TaskFactory(CancellationToken.None, TaskCreationOptions.DenyChildAttach,
TaskContinuationOptions.DenyChildAttach, myTaskScheduler);
var task = factory.StartNew(() => MyCodeAsync()).Unwrap();
If desired, I have written some Run overloads for TaskFactory that make the TaskFactory usage look more like Task.Run:
// Equivalent to the above code
var factory = new TaskFactory(CancellationToken.None, TaskCreationOptions.DenyChildAttach,
TaskContinuationOptions.DenyChildAttach, myTaskScheduler);
var task = factory.Run(() => MyCodeAsync());
I have Method :
private static void Method()
{
Console.WriteLine("Method() started");
for (var i = 0; i < 20; i++)
{
Console.WriteLine("Method() Counter = " + i);
Thread.Sleep(500);
}
Console.WriteLine("Method() finished");
}
And I want to start this method in a new Task.
I can start new task like this
var task = Task.Factory.StartNew(new Action(Method));
or this
var task = Task.Run(new Action(Method));
But is there any difference between Task.Run() and Task.Factory.StartNew(). Both of them are using ThreadPool and start Method() immediately after creating instance of the Task. When we should use first variant and when second?
The second method, Task.Run, has been introduced in a later version of the .NET framework (in .NET 4.5).
However, the first method, Task.Factory.StartNew, gives you the opportunity to define a lot of useful things about the thread you want to create, while Task.Run doesn't provide this.
For instance, lets say that you want to create a long running task thread. If a thread of the thread pool is going to be used for this task, then this could be considered an abuse of the thread pool.
One thing you could do in order to avoid this would be to run the task in a separate thread. A newly created thread that would be dedicated to this task and would be destroyed once your task would have been completed. You cannot achieve this with the Task.Run, while you can do so with the Task.Factory.StartNew, like below:
Task.Factory.StartNew(..., TaskCreationOptions.LongRunning);
As it is stated here:
So, in the .NET Framework 4.5 Developer Preview, we’ve introduced the
new Task.Run method. This in no way obsoletes Task.Factory.StartNew,
but rather should simply be thought of as a quick way to use
Task.Factory.StartNew without needing to specify a bunch of
parameters. It’s a shortcut. In fact, Task.Run is actually
implemented in terms of the same logic used for Task.Factory.StartNew,
just passing in some default parameters. When you pass an Action to
Task.Run:
Task.Run(someAction);
that’s exactly equivalent to:
Task.Factory.StartNew(someAction,
CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskScheduler.Default);
People already mentioned that
Task.Run(A);
Is equivalent to
Task.Factory.StartNew(A, CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskScheduler.Default);
But no one mentioned that
Task.Factory.StartNew(A);
Is equivalent to:
Task.Factory.StartNew(A, CancellationToken.None, TaskCreationOptions.None, TaskScheduler.Current);
As you can see two parameters are different for Task.Run and Task.Factory.StartNew:
TaskCreationOptions - Task.Run uses TaskCreationOptions.DenyChildAttach which means that children tasks can not be attached to the parent, consider this:
var parentTask = Task.Run(() =>
{
var childTask = new Task(() =>
{
Thread.Sleep(10000);
Console.WriteLine("Child task finished.");
}, TaskCreationOptions.AttachedToParent);
childTask.Start();
Console.WriteLine("Parent task finished.");
});
parentTask.Wait();
Console.WriteLine("Main thread finished.");
When we invoke parentTask.Wait(), childTask will not be awaited, even though we specified TaskCreationOptions.AttachedToParent for it, this is because TaskCreationOptions.DenyChildAttach forbids children to attach to it. If you run the same code with Task.Factory.StartNew instead of Task.Run, parentTask.Wait() will wait for childTask because Task.Factory.StartNew uses TaskCreationOptions.None
TaskScheduler - Task.Run uses TaskScheduler.Default which means that the default task scheduler (the one that runs tasks on Thread Pool) will always be used to run tasks. Task.Factory.StartNew on the other hand uses TaskScheduler.Current which means scheduler of the current thread, it might be TaskScheduler.Default but not always. In fact when developing Winforms or WPF applications it is required to update UI from the current thread, to do this people use TaskScheduler.FromCurrentSynchronizationContext() task scheduler, if you unintentionally create another long running task inside task that used TaskScheduler.FromCurrentSynchronizationContext() scheduler the UI will be frozen. A more detailed explanation of this can be found here
So generally if you are not using nested children task and always want your tasks to be executed on Thread Pool it is better to use Task.Run, unless you have some more complex scenarios.
See this blog article that describes the difference. Basically doing:
Task.Run(A)
Is the same as doing:
Task.Factory.StartNew(A, CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskScheduler.Default);
The Task.Run got introduced in newer .NET framework version and it is recommended.
Starting with the .NET Framework 4.5, the Task.Run method is the
recommended way to launch a compute-bound task. Use the StartNew
method only when you require fine-grained control for a long-running,
compute-bound task.
The Task.Factory.StartNew has more options, the Task.Run is a shorthand:
The Run method provides a set of overloads that make it easy to start
a task by using default values. It is a lightweight alternative to the
StartNew overloads.
And by shorthand I mean a technical shortcut:
public static Task Run(Action action)
{
return Task.InternalStartNew(null, action, null, default(CancellationToken), TaskScheduler.Default,
TaskCreationOptions.DenyChildAttach, InternalTaskOptions.None, ref stackMark);
}
According to this post by Stephen Cleary, Task.Factory.StartNew() is dangerous:
I see a lot of code on blogs and in SO questions that use Task.Factory.StartNew to spin up work on a background thread. Stephen Toub has an excellent blog article that explains why Task.Run is better than Task.Factory.StartNew, but I think a lot of people just haven’t read it (or don’t understand it). So, I’ve taken the same arguments, added some more forceful language, and we’ll see how this goes. :)
StartNew does offer many more options than Task.Run, but it is quite dangerous, as we’ll see. You should prefer Task.Run over Task.Factory.StartNew in async code.
Here are the actual reasons:
Does not understand async delegates. This is actually the same as
point 1 in the reasons why you would want to use StartNew. The problem
is that when you pass an async delegate to StartNew, it’s natural to
assume that the returned task represents that delegate. However, since
StartNew does not understand async delegates, what that task actually
represents is just the beginning of that delegate. This is one of the
first pitfalls that coders encounter when using StartNew in async
code.
Confusing default scheduler. OK, trick question time: in the
code below, what thread does the method “A” run on?
Task.Factory.StartNew(A);
private static void A() { }
Well, you know it’s a trick question, eh? If you answered “a thread
pool thread”, I’m sorry, but that’s not correct. “A” will run on
whatever TaskScheduler is currently executing!
So that means it could potentially run on the UI thread if an operation completes and it marshals back to the UI thread due to a continuation as Stephen Cleary explains more fully in his post.
In my case, I was trying to run tasks in the background when loading a datagrid for a view while also displaying a busy animation. The busy animation didn't display when using Task.Factory.StartNew() but the animation displayed properly when I switched to Task.Run().
For details, please see https://blog.stephencleary.com/2013/08/startnew-is-dangerous.html
Apart from the similarities i.e. Task.Run() being a shorthand for Task.Factory.StartNew(), there is a minute difference between their behaviour in case of sync and async delegates.
Suppose there are following two methods:
public async Task<int> GetIntAsync()
{
return Task.FromResult(1);
}
public int GetInt()
{
return 1;
}
Now consider the following code.
var sync1 = Task.Run(() => GetInt());
var sync2 = Task.Factory.StartNew(() => GetInt());
Here both sync1 and sync2 are of type Task<int>
However, difference comes in case of async methods.
var async1 = Task.Run(() => GetIntAsync());
var async2 = Task.Factory.StartNew(() => GetIntAsync());
In this scenario, async1 is of type Task<int>, however async2 is of type Task<Task<int>>
I just came across some code like:
var task = Task.Run(async () => { await Foo.StartAsync(); });
task.Wait();
(No, I don't know the inner-workings of Foo.StartAsync()). My initial reaction would be get rid of async/await and rewrite as:
var task = Foo.StartAsync();
task.Wait();
Would that be correct, or not (again, knowing nothing at all about Foo.StartAsync()). This answer to What difference does it make - running an 'async' action delegate with a Task.Run ... seems to indicate there may be cases when it might make sense, but it also says "To tell the truth, I haven't seen that many scenarios ..."
Normally, the intended usage for Task.Run is to execute CPU-bound code on a non-UI thread. As such, it would be quite rare for it to be used with an async delegate, but it is possible (e.g., for code that has both asynchronous and CPU-bound portions).
However, that's the intended usage. I think in your example:
var task = Task.Run(async () => { await Foo.StartAsync(); });
task.Wait();
It's far more likely that the original author is attempting to synchronously block on asynchronous code, and is (ab)using Task.Run to avoid deadlocks common in that situation (as I describe on my blog).
In essence, it looks like the "thread pool hack" that I describe in my article on brownfield asynchronous code.
The best solution is to not use Task.Run or Wait:
await Foo.StartAsync();
This will cause async to grow through your code base, which is the best approach, but may cause an unacceptable amount of work for your developers right now. This is presumably why your predecessor used Task.Run(..).Wait().
Mostly yes.
Using Task.Run like this is mostly used by people who don't understand how to execute an async method.
However, there is a difference. Using Task.Run means starting the async method on a ThreadPool thread.
This can be useful when the async method's synchronous part (the part before the first await) is substantial and the caller wants to make sure that method isn't blocking.
This can also be used to "get out of" the current context, for example where there isn't a SynchronizationContext.
It's worth noting that your method has to be marked async to be able to use the await keyword.
The code as written seems to be a workaround for running asynchronous code in a synchronous context. While I wouldn't say you should never ever do this, using async methods is preferable in almost every scenario.
// use this only when running Tasks in a synchronous method
// use async instead whenever possible
var task = Task.Run(async () => await Foo.StartAsync());
task.Wait();
Async methods, like your example of Foo.StartAsync(), should always return a Task object. This means that using Task.Run() to create another task is usually redundant in an async method. The task returned by the async method can simply be awaited by using the await keyword. The only reason you should use Task.Run() is when you are performing CPU-bound work that needs to be performed on a separate thread. The task returned by the async method can simply be awaited by using the await keyword. You can read more in depth details in Microsoft's guide to async programming.
In an async method, your code can be as simple as:
await Foo.StartAsync();
If you want to perform some other work while the task is running, you can assign the function to a variable and await the result (task completion) later.
For example:
var task = Foo.StartAsync();
// do some other work before waiting for task to finish
Bar();
Baz();
// now wait for the task to finish executing
await task;
With CPU-bound work that needs to be run on a separate thread, you can use Task.Run(), but you await the result instead of using the thread blocking Task.Wait():
var task = Task.Run(async () => await Foo.StartAsync());
// do some other work before waiting for task to finish
Bar();
Baz();
// now wait for the task to finish executing
await task;
I'm trying to understand async/await and have read a number of articles but am still confused about the synchronous/asynchronous nature.
I have the following test console app:
static void Main(string[] args)
{
var test = FooAsync();
Console.WriteLine("After FooAsync");
for (int i = 0; i < 100; i++)
Console.WriteLine("After that");
Console.ReadKey();
}
private static async Task FooAsync()
{
Console.WriteLine("Before delay");
await Task.Delay(1);
Console.WriteLine("After delay");
}
The code gives output along the lines of:
Before delay
After FooAsync
After that
After that
After that
After that
After delay
After that
.
.
I understand that async/await will not create a separate thread for processing and that at the point FooAsync reaches the await Task.Delay(1) line it will return back to Main as the task will not yet have completed, however, as we are only running on a single thread can someone explain what triggers the FooAsync method to resume at some arbitrary point within Main before Main can then continue?
Update
I take it back and i3arnon and dariogriffo are correct. The code does use multiple threads (as I'd have seen before had looked in the debugger or done the obvious as kha suggested). I'd been confused by the Threads section on the following page https://msdn.microsoft.com/en-us/library/hh191443.aspx#BKMK_Threads not realising that a "continuation" actually refers to a continuation task schedule to run as soon as the task being "awaited" finishes.
This isn't single threaded.
When the delay task completes the rest of the method is posted to the ThreadPool and runs concurrently with your main thread. The "trigger" here is the callback of the internal System.Threading.Timer being used inside Task.Delay.
This behaviour depends on the SynchronizationContext. In a UI environment this would have been posted to the same UI thread and would have to wait until that thread is free.
If you would have been waiting for the task returned from FooAsync then you would only have a single thread running each time.
Async/await may create new threads OR NOT, it depends of the nature of the operation.
If the operation is an IO (for example disks/network operations) probably is coded in a way it will not spin a new thread. You can read from here:
The async and await keywords don't cause additional threads to be created?
If you create your own Async operation and you create a thread, that's a different story, that's why you shouldn't do async over sync
http://blogs.msdn.com/b/pfxteam/archive/2012/04/13/10293638.aspx
You can check this also but using Thread.CurrentThread to get the Id of the process. (Add that to a Console.WriteLine)
It's a pretty common misconception that the async or await keywords create new threads. They don't.
The threads are created by running a Task. In this case, the thread is created by the Task.Delay call.