Develop Reference

Calling UI Thread from Parallel Loop using Dispacher.BeginInvoke [duplicate]

This question already has answers here:
Parallel foreach with asynchronous lambda
(10 answers)
Closed 1 year ago.
I have a simple asynchronous code below : this is a WPF with one button and one textBox.
I used some list with five integers to mimic 5 different tasks.
My intention was to achieve that when I run all five tasks in parallel and asynchronously ,
I can observe that the numbers are one by one added to the textbox.
And I achieved it. Method "DoSomething" run all five tasks in parallel and each of the task has different execution time (simulated by Task.Delay) so all results in the numbers appearing in the textbox one by one.
The only problem that I cannot figure out is: why in the textbox I have the string text "This is end text" displayed at first ?! If I await method DoSomething then it should be accomplished first and then the rest of the code should be executed.Even though in my case is a repainting of the GUI.
I guess that this might be caused by the use of Dispacher.BeginInvoke which may "cause some disturbance " to async/await mechanism. But I would appreciate small clue and how to avoid this behawior.
I know that I could use the Progress event to achieve similar effect but is there any other way that I can use Parallel loop and update results progressively in WPF avoiding such a unexpected behaviour which I described?
private async void Button_Click(object sender, RoutedEventArgs e)
{
await DoSomething();
tbResults.Text += "This is end text";
}
private async Task DoSomething()
{
List<int> numbers = new List<int>(Enumerable.Range(1, 5));
await Task.Run(()=> Parallel.ForEach(numbers,async i =>
{
await Task.Delay(i * 300);
await Dispatcher.BeginInvoke(() => tbResults.Text += i.ToString() + Environment.NewLine);
}));
}
// output is:
//This is end text 1 2 3 4 5 (all in separatę lines).
My questions:
Why the text is displayed prior the method DoSomething .
How to solve it/ avoid it , any alternative way to solve it ( except using Progress event ).
Any info will be highly appreciate.

The threads of Parallel.Foreach are "real" background threads. They are created and the application continues execution. The point is that Parallel.Foreach is not awaitable, therefore the execution continues while the threads of the Parallel.Foreach are suspended using await.
private async Task DoSomething()
{
List<int> numbers = new List<int>(Enumerable.Range(1, 5));
// Create the threads of Parallel.Foreach
await Task.Run(() =>
{
// Create the threads of Parallel.Foreach and continue
Parallel.ForEach(numbers,async i =>
{
// await suspends the thread and forces to return.
// Because Parallel.ForEach is not awaitable,
// execution leaves the scope of the Parallel.Foreach to continue.
await Task.Delay(i * 300);
await Dispatcher.BeginInvoke(() => tbResults.Text += i.ToString() + Environment.NewLine);
});
// After the threads are created the internal await of the Parallel.Foreach suspends background threads and
// forces to the execution to return from the Parallel.Foreach.
// The Task.Run thread continues.
Dispatcher.InvokeAsync(() => tbResults.Text += "Text while Parallel.Foreach threads are suspended");
// Since the background threads of the Parallel.Foreach are not attached
// to the parent Task.Run, the Task.Run completes now and returns
// i.e. Task.run does not wait for child background threads to complete.
// ==> Leave Task.Run as there is no work.
});
// Leave DoSomething() and continue to execute the remaining code in the Button_Click().
// Parallel.Foreach threads is still suspended until the await chain, in this case Button_Click(), is completed.
}
The solution is to implement the pattern suggested by Clemens' comment or an async implementation of the Producer Consumer pattern using e.g., BlockingCollection or Channel to gain more control over the fixed number of threads while distributing the "unlimited" number of jobs.
private async Task DoSomething(int number)
{
await Task.Delay(number * 300);
Dispatcher.Invoke(() => tbResults.Text += number + Environment.NewLine);
}
private async void ButtonBase_OnClick(object sender, RoutedEventArgs e)
{
List<int> numbers = new List<int>(Enumerable.Range(1, 5));
List<Task> tasks = new List<Task>();
// Alternatively use LINQ Select
foreach (int number in numbers)
{
Task task = DoSomething(number);
tasks.Add(task);
}
await Task.WhenAll(tasks);
tbResults.Text += "This is end text" + Environment.NewLine;
}
Discussing the comments
"my intention was to run tasks in parallel and "report" once they are
completed i.e. the taks which takes the shortest would "report" first
and so on."
This is exactly what is happening in the above solution. The Task with the shortest delay appends text to the TextBox first.
"But implementing your suggested await Task.WhenAll(tasks) causes
that we need to wait for all tasks to complete and then report all at
once."
To process Task objects in their order of completion, you would replace Task.WhenAll with Task.WhenAny. In case you are not only interested in the first completed Task, you would have to use Task.WhenAny in an iterative manner until all Task instances have been completed:
Process all Task objects in their order of completion
private async Task DoSomething(int number)
{
await Task.Delay(number * 300);
Dispatcher.Invoke(() => tbResults.Text += number + Environment.NewLine);
}
private async void ButtonBase_OnClick(object sender, RoutedEventArgs e)
{
List<int> numbers = new List<int>(Enumerable.Range(1, 5));
List<Task> tasks = new List<Task>();
// Alternatively use LINQ Select
foreach (int number in numbers)
{
Task task = DoSomething(number);
tasks.Add(task);
}
// Until all Tasks have completed
while (tasks.Any())
{
Task<int> nextCompletedTask = await Task.WhenAny(tasks);
// Remove the completed Task so that
// we can await the next uncompleted Task that completes first
tasks.Remove(nextCompletedTask);
// Get the result of the completed Task
int taskId = await nextCompletedTask;
tbResults.Text += $"Task {taskId} has completed." + Environment.NewLine;
}
tbResults.Text += "This is end text" + Environment.NewLine;
}
"Parallel.ForEach is not awaitable so I thought that wrapping it up in
Task.Run allows me to await it but this is because as you said "Since
the background threads of the Parallel.Foreach are not attached to the
parent Task.Run""
No that's not exactly what I have said. The key point is the third sentence of my answer: "The point is that Parallel.Foreach is not awaitable, therefore the execution continues while the threads of the Parallel.Foreach are suspended using await.".
This means: normally Parallel.Foreach executes synchronously: the calling context continues execution when all threads of Parallel.Foreach have completed. But since you called await inside those threads, you suspend them in an async/await manner.
Since Parallel.Foreach is not awaitable, it can't handle the await calls and acts like the suspended threads have completed naturally. Parallel.Foreach does not understand that the threads are just suspended by await and will continue later. In other words, the await chain is broken as Parallel.Foreach is not able to return the Task to the parent awaited Task.Run context to signal its suspension.
That's what I meant when saying that the threads of Parallel.Foreach are not attached to the Task.Run. They run in total isolation from the async/await infrastructure.
"async lambdas should be "only use with events""
No, that's not correct. When you pass an async lambda to a void delegate like Action<T> you are correct: the async lambda can't be awaited in this case. But when passing an async lambda to a Func<T> delegate where T is of type Task, your lamda can be awaited:
private void NoAsyncDelegateSupportedMethod(Action lambda)
{
// Since Action does not return a Task (return type is always void),
// the async lambda can't be awaited
lambda.Invoke();
}
private async Task AsyncDelegateSupportedMethod(Func<Task> asyncLambda)
{
// Since Func returns a Task, the async lambda can be awaited
await asyncLambda.Invoke();
}
public voi DoSoemthing()
{
// Not a good idea as NoAsyncDelegateSupportedMethod can't handle async lamdas: it defines a void delegate
NoAsyncDelegateSupportedMethod(async () => await Task.Delay(1));
// A good idea as AsyncDelegateSupportedMethod can handle async lamdas: it defines a Func<Task> delegate
AsyncDelegateSupportedMethod(async () => await Task.Delay(1));
}
As you can see your statement is not correct. You must always check the signature of the called method and its overloads. If it accepts a Func<Task> type delegate you are good to go.
That's how async support is added to Parallel.ForeachAsync: the API supports a Func<ValueTask> type delegate. For example Task.Run accepts a Func<Task> and therefore the following call is perfectly fine:
Task.Run(async () => await Task.Delay(1));
" I guess that you admit that .Net 6.0 brought the best solution :
which is Parallel.ForEachASYNC! [...] We can spawn a couple of threads
which deal with our tasks in parallel and we can await the whole loop
and we do not need to wait for all tasks to complte- they "report" as
they finish "
That's wrong. Parallel.ForeachAsync supports threads thatr use async/await, that's true. Indeed, your original example would no longer break the intended flow: because Parallel.ForeachAsync supports await in its threads, it can handle suspended threads and propagate the Task object properly from its threads to the caller context e.g., to the wrapping await Task.Run.
It now knows how to wait for and resume suspended threads.
Important: Parallel.ForeachAsync still completes AFTER ALL its threads have completed. You assumption "they "report" as they finish" is wrong. That's the most intuitive concurrent implementation of a foreach. foreach also completes after all items are enumerated.
The solution to process Task objects as they complete is still using the Task.WhenAny pattern from above.
In general, if you you don't need the extra features like partitioning etc. of the the Parallel.Foreach and Parallel.ForeachAsync, you can always use Task.WhenAll instead. Task.WhenAll and especially Parallel.ForeachAsync are equivalent, except for Parallel.ForeachAsync provides greater customization by default: it suppors techniques like throttling and partitioning without the extra code.

Good links from Clemens see in comment. Answering your questions:
In Parallel.ForEach you start/fire for each entry of numbera an async task, which you don't await. So you do only await, that Parallel.ForEach does finish and it does finish before the async tasks of it.
What you could do e.g. remove async inside of Parallel.ForEach and use Dispatcher.Invoke instead of Dispatcher.BeginInvoke, Thread.Sleep is an anti-pattern ;) , so depending on your task may be take another solution(edited: BionicCode has a nice one):
private async Task DoSomething()
{
var numbers = new List<int>(Enumerable.Range(1, 5));
await Task.Run(()=> Parallel.ForEach(numbers, i =>
{
Thread.Sleep(i * 300);
Dispatcher.Invoke(() => tbResults.Text += i.ToString() + Environment.NewLine);
}));
}

Nested async methods in a Parallel.ForEach

I have a method that runs multiple async methods within it. I have to iterate over a list of devices, and pass the device to this method. I am noticing that this is taking a long time to complete so I am thinking of using Parallel.ForEach so it can run this process against multiple devices at the same time.
Let's say this is my method.
public async Task ProcessDevice(Device device) {
var dev = await _deviceService.LookupDeviceIndbAsNoTracking(device);
var result = await DoSomething(dev);
await DoSomething2(dev);
}
Then DoSomething2 also calls an async method.
public async Task DoSomething2(Device dev) {
foreach(var obj in dev.Objects) {
await DoSomething3(obj);
}
}
The list of devices continuously gets larger over time, so the more this list grows, the longer it takes the program to finish running ProcessDevice() against each device. I would like to process more than one device at a time. So I have been looking into using Parallel.ForEach.
Parallel.ForEach(devices, async device => {
try {
await ProcessDevice(device);
} catch (Exception ex) {
throw ex;
}
})
It appears that the program is finishing before the device is fully processed. I have also tried creating a list of tasks, and then foreach device, add a new task running ProcessDevice to that list and then awaiting Task.WhenAll(listOfTasks);
var listOfTasks = new List<Task>();
foreach(var device in devices) {
var task = Task.Run(async () => await ProcessDevice(device));
listOfTasks.Add(task);
}
await Task.WhenAll(listOfTasks);
But it appears that the task is marked as completed before ProcessDevice() is actually finished running.
Please excuse my ignorance on this issue as I am new to parallel processing and not sure what is going on. What is happening to cause this behavior and is there any documentation that you could provide that could help me better understand what to do?

You can't mix async with Parallel.ForEach. Since your underlying operation is asynchronous, you'd want to use asynchronous concurrency, not parallelism. Asynchronous concurrency is most easily expressed with WhenAll:
var listOfTasks = devices.Select(ProcessDevice).ToList();
await Task.WhenAll(listOfTasks);

In your last example there's a few problems:
var listOfTasks = new List<Task>();
foreach (var device in devices)
{
await Task.Run(async () => await ProcessDevice(device));
}
await Task.WhenAll(listOfTasks);
Doing await Task.Run(async () => await ProcessDevice(device)); means you are not moving to the next iteration of the foreach loop until the previous one is done. Essentially, you're still doing them one at a time.
Additionally, you aren't adding any tasks to listOfTasks so it remains empty and therefore Task.WhenAll(listOfTasks) completes instantly because there's no tasks to await.
Try this:
var listOfTasks = new List<Task>();
foreach (var device in devices)
{
var task = Task.Run(async () => await ProcessDevice(device))
listOfTasks.Add(task);
}
await Task.WhenAll(listOfTasks);

I can explain the problem with Parallel.ForEach. An important thing to understand is that when the await keyword acts on an incomplete Task, it returns. It will return its own incomplete Task if the method signature allows (if it's not void). Then it is up to the caller to use that Task object to wait for the job to finish.
But the second parameter in Parallel.ForEach is an Action<T>, which is a void method, which means no Task can be returned, which means the caller (Parallel.ForEach in this case) has no way to wait until the job has finished.
So in your case, as soon as it hits await ProcessDevice(device), it returns and nothing waits for it to finish so it starts the next iteration. By the time Parallel.ForEach is finished, all it has done is started all the tasks, but not waited for them.
So don't use Parallel.ForEach with asynchronous code.
Stephen's answer is more appropriate. You can also use WSC's answer, but that can be dangerous with larger lists. Creating hundreds or thousands of new threads all at once will not help your performance.

not very sure it this if what you are asking for, but I can give example of how we start async process
private readonly Func<Worker> _worker;
private void StartWorkers(IEnumerable<Props> props){
Parallel.ForEach(props, timestamp => { _worker.Invoke().Consume(timestamp); });
}
Would recommend reading about Parallel.ForEach as it will do some part for you.

Async method blocking on unawaited task

In my current project, I have a piece of code that, after simplifying it down to where I'm having issues, looks something like this:
private async Task RunAsync(CancellationToken cancel)
{
bool finished = false;
while (!cancel.IsCancellationRequested && !finished)
finished = await FakeTask();
}
private Task<bool> FakeTask()
{
return Task.FromResult(false);
}
If I use this code without awaiting, I end up blocking anyway:
// example 1
var task = RunAsync(cancel); // Code blocks here...
... // Other code that could run while RunAsync is doing its thing, but is forced to wait
await task;
// example 2
var task = RunAsync(cancelSource.Token); // Code blocks here...
cancelSource.Cancel(); // Never called
In the actual project, I'm not actually using FakeTask, and there usually will be some Task.Delay I'm awaiting in there, so the code most of the time doesn't actually block, or only for a limited amount of iterations.
In unit testing, however, I'm using a mock object that does pretty much do what FakeTask does, so when I want to see if RunAsync responds to its CancellationToken getting cancelled the way I expect it to, I'm stuck.
I have found I can fix this issue by adding for example await Task.Delay(1) at the top of RunAsync, to force it to truly run asynchronous, but this feels a bit hacky. Are there better alternatives?

You have an incorrect mental picture of what await does. The meaning of await is:
Check to see if the awaitable object is complete. If it is, fetch its result and continue executing the coroutine.
If it is not complete, sign up the remainder of the current method as the continuation of the awaitable and suspend the coroutine by returning control to the caller. (Note that this makes it a semicoroutine.)
In your program, the "fake" awaitable is always complete, so there is never a suspension of the coroutine.
Are there better alternatives?
If your control flow logic requires you to suspend the coroutine then use Task.Yield.

Task.FromResult actually runs synchronously, as would await Task.Delay(0). If you want to actually simulate asynchronous code, call Task.Yield(). That creates an awaitable task that asynchronously yields back to the current context when awaited.

As #SLaks said, your code will run synchronously. One thing is running async code, and another thing is running parallel code.
If you need to run your code in parallel you can use Task.Run.
class Program
{
static async Task Main(string[] args)
{
var tcs = new CancellationTokenSource();
var task = Task.Run(() => RunAsync("1", tcs.Token));
var task2 = Task.Run(() => RunAsync("2", tcs.Token));
await Task.Delay(1000);
tcs.Cancel();
Console.ReadLine();
}
private static async Task RunAsync(string source, CancellationToken cancel)
{
bool finished = false;
while (!cancel.IsCancellationRequested && !finished)
finished = await FakeTask(source);
}
private static Task<bool> FakeTask(string source)
{
Console.WriteLine(source);
return Task.FromResult(false);
}
}

C#'s async methods execute synchronously up to the point where they have to wait for a result.
In your example there is no such point where the method has to wait for a result, so the loop keeps running forever and thereby blocking the caller.
Inserting an await Task.Yield() to simulate some real async work should help.

Does a method has to be async when invoked inside an async method?

Lets say I have a method defined as follows:
public async Task CreateUser()
{
await GetUserDetails();
GetUserOrder();
}
private void GetUserDetails() {
private void GetUserOrder() {
Does the method GetUserDetails(); and GetUserOrder() have to be async as well to avoid UI blocking ?
I cannot await the GetUserDetails() method since it is not async. How can I achieve this in c# ? I want to ensure all these methods are invoked step by step.

The relevant question is in a comment:
How can I ensure all my methods are invoked completely sequentially?
The fact that you're asking the question indicates that you don't understand what "await" is. Await is the sequencing operation on a asynchronous workflows. An await means this workflow will not proceed until the awaited task is complete. It's an asynchronous wait, hence the name await.
Consider this question: in a synchronous workflow, what is the sequencing operation?
No, really, give it some thought.
.
.
.
It is ;. When you say
fResult = foo();
bResult = bar();
qResult = qux();
that means that foo has to finish completely before bar can begin. That is not true for asynchronous workflows. If we have
fTask = fooAsync();
bTask = barAsync();
qTask = quxAsync();
Then the asynchronous operations can complete in any order. If we say
await fooAsync();
await barAsync();
await quxAsync();
Then barAsync will not start until fooAsync's task completes. await sequences the asynchronous workflow. The difference is that the thread can continue to do other unrelated work while asynchronously waiting for foo to complete, which is not true in a synchronous workflow; in a synchronous workflow the thread is already busy computing the foo result, so it can't do other work.

yes if you want to wait than you have to write await for that methods also. because after first await your code agian will be synchronous ..and if UI thread than it will run on it.
1.you code will be , so by this code you code become asynchronous for GetUserORder also. you just have to wrap method in Task construct and return
public async Task CreateUser()
{
await GetUserDetails();
await Task.Factory.SartNew(()=> GetUserOrder());
}
2.or you can do this also
public async Task CreateUser()
{
await Task.Factory.SartNew(()=>{
GetUserDetails();
GetUserOrder(); });
}
3.or you can do like this also, in below code will not wait for getuserorder method and excute await one method
public async Task CreateUser()
{
Task.Factory.SartNew(()=> GetUserOrder()).ContinueWith((t)=> Console.WriteLine("Completed");
await GetUserDetails();
}
4.or last one variation, here you start GetUserOrder first and dont wait for it than you call GetUserDetails in async fashion , and if you want to work on GetUserOrder method want to wait just use Wait method.
public async Task CreateUser()
{
var task = Task.Factory.SartNew(()=> GetUserOrder());
await GetUserDetails();
if(!task.IsCompleted)
task.Wait();
}
in your case you can go for 3 and if you want to wait go for 4th one.
As you asked me in comment what is difference between Task.Run and statnew method -: for that you can check this SO question : Regarding usage of Task.Start() , Task.Run() and Task.Factory.StartNew()

You should put await only in front of async methods. To run a synchronous one that you don't want to wait, you can use a new thread from the tread pool:
new Thread(() => DoSomething()).Start();
or
Task.Factory.SartNew(()=> DoSomething());
Here is the help page: https://msdn.microsoft.com/en-us/library/dd321439(v=vs.110).aspx
Otherwise, your call to GetUserDetails will have to finish before you execute the next line.

How to wait on Async tasks

static void Main(string[] args)
{
Action myAction = async () =>
{
await Task.Delay(5);
Console.WriteLine(Interlocked.Add(ref ExecutionCounter, 1));
};
var actions = new[] { myAction, myAction, myAction };
Task.WaitAll(actions.Select(a => Execute(a)).ToArray()); //This blocks, right?
Console.WriteLine("Done waiting on tasks.");
Console.ReadLine();
}
static int ExecutionCounter = 0;
private static Task Execute(Action a)
{
return Task.Factory.StartNew(async () =>
{
await Task.Delay(5);
a();
});
}
This seems simple enough, but naturally the output always looks like this (the order of the numbers change, of course):
Done waiting on tasks.
2
1
3
What am I missing here? Why doesn't Task.WaitAll block like I'm expecting it to?

So there are several separate bugs here.
First, for Execute, you're using StartNew with an async lambda. Since StartNew doesn't have a Task<Task> returning overload, like Task.Run does, you've got a method that returns a Task indicating when the asynchronous operation has finished starting, not when the asynchronous operation has finished, which means that the Task returned by Execute will be completed basically right away, rather than after Delay finishes or the action you call finishes. Additionally, there's simply no reason to use StartNew or Run at all when running asynchronous methods, you can just execute them normally and await them without pushing them to a thread pool thread.
Next, Execute accepts an Action, which implies that it's a synchronous method that doesn't compute any value. What you're providing is an asynchronous method, but as the delegate doesn't return a Task, Execute can't await it. If you want Execute to handle asynchronous methods, it needs to accept a delegate that returns a Task.
So given all of that Execute should look like this.
private static async Task Execute(Func<Task> action)
{
await Task.Delay(TimeSpan.FromMilliseconds(5));
await action();
}
Next onto the Main method. As mentioned before Execute is accepting an Action when you're trying to provide an async method. This means that when the action is run the code will continued executing before your actions have finished. You need to adjust it to using a Task returning method.
After all of that, your code still has a race condition in it, at a conceptual level, that will prevent you from theoretically getting the results in the right order. You're performing 3 different operations in parallel, and as a result of that, they can finish in any order. While you are atomically incrementing the counter, it's possible for one thread to increment the counter, then another to run, increment the counter, print its value, then have the other thread run again and print out the value, given you a possible output of what you have, even after fixing all of the bugs mentioned above. To ensure that the values are printed in order, you need to ensure that the increment and the console write are performed atomically.
Now you can write out your Main method like so:
int ExecutionCounter = 0;
object key = new object();
Func<Task> myAction = async () =>
{
await Task.Delay(TimeSpan.FromMilliseconds(5));
lock (key)
{
Console.WriteLine(++ExecutionCounter);
}
};
var actions = new[] { myAction, myAction, myAction };
Task.WaitAll(actions.Select(a => Execute(a)).ToArray()); //This blocks, right?
And yes, as your comment mentions, calling WaitAll will block, rather than being asynchronous.