Develop Reference

Unexpected values for AsyncLocal.Value when mixing ExecutionContext.SuppressFlow and tasks

In an application I am experiencing odd behavior due to wrong/unexpected values of AsyncLocal: Despite I suppressed the flow of the execution context, I the AsyncLocal.Value-property is sometimes not reset within the execution scope of a newly spawned Task.
Below I created a minimal reproducible sample which demonstrates the problem:
private static readonly AsyncLocal<object> AsyncLocal = new AsyncLocal<object>();
[TestMethod]
public void Test()
{
Trace.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
AsyncLocal.Value = "1";
Task anotherTask;
using (ExecutionContext.SuppressFlow())
{
anotherTask = Task.Run(() =>
{
Trace.WriteLine(AsyncLocal.Value); // "1" <- ???
Assert.IsNull(AsyncLocal.Value); // BOOM - FAILS
AsyncLocal.Value = "2";
});
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
In nine out of ten runs (on my pc) the outcome of the Test-method is:
.NET 6.0.2
"1"
-> The test fails
As you can see the test fails because within the action which is executed within Task.Run the the previous value is still present within AsyncLocal.Value (Message: 1).
My concrete questions are:
Why does this happen?
I suspect this happens because Task.Run may use the current thread to execute the work load. In that case, I assume lack of async/await-operators does not force the creation of a new/separate ExecutionContext for the action. Like Stephen Cleary said "from the logical call context’s perspective, all synchronous invocations are “collapsed” - they’re actually part of the context of the closest async method further up the call stack". If that’s the case I do understand why the same context is used within the action.
Is this the correct explanation for this behavior? In addition, why does it work flawlessly sometimes (about 1 run out of 10 on my machine)?
How can I fix this?
Assuming that my theory above is true it should be enough to forcefully introduce a new async "layer", like below:
private static readonly AsyncLocal<object> AsyncLocal = new AsyncLocal<object>();
[TestMethod]
public void Test()
{
Trace.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
AsyncLocal.Value = "1";
Task anotherTask;
using (ExecutionContext.SuppressFlow())
{
var wrapper = () =>
{
Trace.WriteLine(AsyncLocal.Value);
Assert.IsNull(AsyncLocal.Value);
AsyncLocal.Value = "2";
return Task.CompletedTask;
};
anotherTask = Task.Run(async () => await wrapper());
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
This seems to fix the problem (it consistently works on my machine), but I want to be sure that this is a correct fix for this problem.
Many thanks in advance

Why does this happen? I suspect this happens because Task.Run may use the current thread to execute the work load.
I suspect that it happens because Task.WaitAll will use the current thread to execute the task inline.
Specifically, Task.WaitAll calls Task.WaitAllCore, which will attempt to run it inline by calling Task.WrappedTryRunInline. I'm going to assume the default task scheduler is used throughout. In that case, this will invoke TaskScheduler.TryRunInline, which will return false if the delegate is already invoked. So, if the task has already started running on a thread pool thread, this will return back to WaitAllCore, which will just do a normal wait, and your code will work as expected (1 out of 10).
If a thread pool thread hasn't picked it up yet (9 out of 10), then TaskScheduler.TryRunInline will call TaskScheduler.TryExecuteTaskInline, the default implementation of which will call Task.ExecuteEntryUnsafe, which calls Task.ExecuteWithThreadLocal. Task.ExecuteWithThreadLocal has logic for applying an ExecutionContext if one was captured. Assuming none was captured, the task's delegate is just invoked directly.
So, it seems like each step is behaving logically. Technically, what ExecutionContext.SuppressFlow means is "don't capture the ExecutionContext", and that is what is happening. It doesn't mean "clear the ExecutionContext". Sometimes the task is run on a thread pool thread (without the captured ExecutionContext), and WaitAll will just wait for it to complete. Other times the task will be executed inline by WaitAll instead of a thread pool thread, and in that case the ExecutionContext is not cleared (and technically isn't captured, either).
You can test this theory by capturing the current thread id within your wrapper and comparing it to the thread id doing the Task.WaitAll. I expect that they will be the same thread for the runs where the async local value is (unexpectedly) inherited, and they will be different threads for the runs where the async local value works as expected.

If you can, I'd first consider whether it's possible to replace the thread-specific caches with a single shared cache. The app likely predates useful types such as ConcurrentDictionary.
If it isn't possible to use a singleton cache, then you can use a stack of async local values. Stacking async local values is a common pattern. I prefer wrapping the stack logic into a separate type (AsyncLocalValue in the code below):
public sealed class AsyncLocalValue
{
private static readonly AsyncLocal<ImmutableStack<object>> _asyncLocal = new();
public object Value => _asyncLocal.Value?.Peek();
public IDisposable PushValue(object value)
{
var originalValue = _asyncLocal.Value;
var newValue = (originalValue ?? ImmutableStack<object>.Empty).Push(value);
_asyncLocal.Value = newValue;
return Disposable.Create(() => _asyncLocal.Value = originalValue);
}
}
private static AsyncLocalValue AsyncLocal = new();
[TestMethod]
public void Test()
{
Console.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
Task anotherTask;
using (AsyncLocal.PushValue("1"))
{
using (AsyncLocal.PushValue(null))
{
anotherTask = Task.Run(() =>
{
Console.WriteLine("Observed: " + AsyncLocal.Value);
using (AsyncLocal.PushValue("2"))
{
}
});
}
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
This code sample uses Disposable.Create from my Nito.Disposables library.

Await new Task<T>( ... ) : Task does not run?

A continuation of a question asked here :
In the aforementioned question I have the following function which returns an object of type Task (for incremental testing purposes) :
private static Task<object> GetInstance( ) {
return new Task<object>( (Func<Task<object>>)(async ( ) => {
await SimpleMessage.ShowAsync( "TEST" );
return new object( );
} ) );
}
When I call await GetInstance( );, the function is called (and I assume the task is returned since no exception is thrown) but then the task just sits there.
I can only guess I am doing this wrong then.
I do not want this function to return a task that is already running ( that is IMPERATIVE ).
How do I asynchronously run the task returned by this function?

To create a Task already started
Try creating the task like this:
Task.Factory.StartNew<object>((Func<Task<object>>) ...);
To create a Task without starting it
If you don't want the task started, just use new Task<object>(...) as you were using, but then you need to call Start() method on that task before awaiting it!
[Reference]
My recommendation
Just make a method that returns the anonymous function, like this:
private static Func<object> GetFunction( ) {
return (Func<object>)(( ) => {
SimpleMessage.Show( "TEST" );
return new object( );
} );
}
Then get it and run it in a new Task whenever you need it (Also notice that I removed the async/await from the lambda expression, since you are putting it into a Task already):
Task.Factory.StartNew<object>(GetFunction());
One advantage to this is that you can also call it without putting it into a Task:
GetFunction()();

You should never use the Task constructor (or Task.Start).
I do not want this function to return a task that is already running ( that is IMPERATIVE ).
So, you want to return some code that won't execute until you call it? That's a delegate, not a Task.
private static Func<Task<object>> GetInstance()
{
return async () =>
{
await SimpleMessage.ShowAsync("TEST");
return new object();
};
}
When you're ready to execute it, call it just like any other delegate. Note that it returns a Task, so you can await the result:
var func = GetInstance();
// Delegate has not started executing yet
var task = func();
// Delegate has started executing
var result = await task;
// Delegate is done

You are stuck with a bad design there. I'll try to make something work for you under these constraints.
The only way to delay start tasks is to use the Task.Start method. (You also can use RunSynchronously but that doesn't really make use of any of the tasks features. At this point the task becomes an unthreaded lazy object.
So use the Task.Start method.
await does not start tasks. It waits for tasks that already run. Therefore await new Task(() => { }) always freezes forever.
Another problem here:
return new Task<object>( (Func<Task<object>>)(async ( ) => {
await SimpleMessage.ShowAsync( "TEST" );
return new object( );
} ) );
When you start that task it will complete nearly instantly and Task.Result will hold another task - the one returned by the async lambda. I doubt this is what you want. Hard to make this work and I don't know what you need.
This smells like the XY problem. Can you elaborate on what you want to accomplish? It feels like you have given yourself unnecessary constraints.

OK, I think I've figured this out. Please examine this code:
class Program
{
public static void Main(string[] args)
{
// Only RUN the task as needed. FooGet
// still allows you to generalize your task.
Task.Run(() =>
{
dynamic value = FooGet();
value.RunSynchronously();
Console.WriteLine(value.Result.Result.ToString());
});
while (true) Thread.Sleep(100);
}
private static Task<object> FooGet()
{
var task = new Task<object>(() => {
return asyncBar();
});
return task;
}
private async static Task<object> asyncBar()
{
// do work!
return "Hello, world!";
}
}

Did I actually gain anything here using tasks and async

I am trying to get to grips with Tasks and the Async Await keywords. I have a little sample method which essentially invokes n number of methods. Two key points to note are
I don't care about the order the methods run in
All methods should be called on a background thread
With this here is the code.
public async void Handle<T>(T entry) {
await Task.Run(() => {
Parallel.ForEach(_handlers, pair => {
pair.Value.Invoke(_reference, new object[] {
entry
});
});
});
My question is did I actually gain any async or parallelism out of the code above?

I'm assuming that you're running in a UI application, since parallel code on a server is quite rare.
In this case, wrapping parallel code in a Task.Run is perfectly normal, and a common pattern when you wish to execute parallel code asynchronously.
I would make a few changes:
Avoid async void. Return a Task, so you can handle errors more cleanly.
Follow the Task-based Asynchronous Pattern naming conventions (i.e.,
end your method with Async).
Return the Task directly instead of async/await (if your entire async method is just to await a task, you can avoid some overhead just by returning the task directly).
Like this:
public Task HandleAsync<T>(T entry) {
return Task.Run(() => {
Parallel.ForEach(_handlers, pair => {
pair.Value.Invoke(_reference, new object[] {
entry
});
});
});
I'd also consider two other possibilities:
Consider using Parallel.Invoke instead of Parallel.ForEach. It seems to me that Parallel.Invoke is a closer match to what you're actually trying to do. (See svick's comment below).
Consider leaving the parallel code in a synchronous method (e.g., public void Handle<T>(T entry)) and calling it with Task.Run (e.g., await Task.Run(() => Handle(entry));). If your Handle[Async] method is intended to be part of a general-purpose library, then you want to avoid the use of Task.Run within your asynchronous methods.

No, you've gained nothing here. Even if you made it return Task (so that the caller could check when everything had finished) you'd be better off without async:
public Task Handle<T>(T entry)
{
return Task.Run(() =>
{
Parallel.ForEach(_handlers, pair =>
{
pair.Value.Invoke(_reference, new object[] { entry });
});
});
}

How do I create a Task that uses await inside the body that behaves the same as the synchronous version when Wait is called?

I have some code that creates a task that does some slow work like this:
public static Task wait1()
{
return new Task(() =>
{
Console.WriteLine("Waiting...");
Thread.Sleep(10000);
Console.WriteLine("Done!");
});
}
In the real implementation, the Thread.Sleep will actually be a web service call. I would like to change the body of the method can use await (so it does not consume a thread during the network access/sleep). My first attempt (based on shotgun-debugging the compile errors) was this:
public static Task wait2()
{
return new Task(async () =>
{
Console.WriteLine("Waiting...");
await Task.Delay(10000);
Console.WriteLine("Done!");
});
}
However; this task doesn't seem to behave the same as the first one, because when I call .Wait() on it; it returns immediately.
Below is a full sample (console app) showing the differences (the app will end immediately when the second task starts).
What do I need to do so that I can call Start and Wait on a Task which happens to have code using await inside it? The tasks are queued and executed later by an agent, so it's vital that the task is not auto-started.
class Program
{
static void Main(string[] args)
{
var w1 = wait1();
w1.Start();
w1.Wait(); // This waits 110 seconds
var w2 = wait2();
w2.Start();
w2.Wait(); // This returns immediately
}
public static Task wait1()
{
return new Task(() =>
{
Console.WriteLine("Waiting...");
Thread.Sleep(10000);
Console.WriteLine("Done!");
});
}
public static Task wait2()
{
return new Task(async () =>
{
Console.WriteLine("Waiting...");
await Task.Delay(10000);
Console.WriteLine("Done!");
});
}
}

It seems like this isn't possible! See alexm's answer here:
Tasks returned by async methods are always hot i.e. they are created in Running state.
:-(
I've worked around this by making my agent queue Func<Task>s instead, and the overload that receives a task simply queues () => task. Then; when de-queing a task, I check if it's not running, and if so, start it:
var currentTask = currentTaskFunction();
if (currentTask.Status == TaskStatus.Created)
currentTask.Start();
It seems a little clunky to have to do this (if this simple workaround works; why the original restriction on async methods always being created hot?), but it seems to work for me :-)

You could write this as:
public static async Task Wait2()
{
Console.WriteLine("Waiting...");
await Task.Delay(10000);
Console.WriteLine("Done!");
}
In general, it's rarely a good idea to ever use new Task or new Task<T>. If you must launch a task using the ThreadPool instead of using the async/await language support to compose one, you should use Task.Run to start the task. This will schedule the task to run (which is important, tasks should always be "hot" by conventions).
Note that doing this will make it so you don't have to call Task.Start, as well.

To help you understand this realize that async / await essentially does not create a new thread but rather it schedules that portion of code to be ran at an available point in time.
When you create the new Task(async () => ...) you have a task that run an async method. When that inner async method hits an await the 'new Task' is considered complete because the rest of it has been scheduled. To help you understand better place some code (a lot if wanted) in the 'new Task' before the await command. It will all execute before the application terminates and once await is reached that task will believe it has completed. It then returns and exits the application.
The best way to avoid this is to not place any task or async methods inside of your task.
Remove the async keyword and the await keyword from the method and it will work as expected.
This is the same as creating a callback if you're familiar with that.
void MethodAsync(Action callback)
{
//...some code
callback?.Invoke();
}
//using this looks like this.
MethodAsync(() => { /*code to run when complete */});
//This is the same as
Task MethodAsync()
{
//... some code here
}
//using it
await MethodAsync();
/*code to run when complete */
The thing to understand is that you're creating a new task within a task basically. So the inner 'callback' is being created at the await keyword.
You're code looks like this..
void MethodAsync(Action callback)
{
//some code to run
callback?.Invoke(); // <- this is the await keyword
//more code to run.. which happens after we run whoever is
//waiting on callback
}
There's code missing obviously. If this doesn't make sense please feel free to contact me and I'll assist. async / await (meant to make things simpler) is a beast to wrap your head around at first. Afterward you get it then it'll probably be your favorite thing in c# since linq. :P

Try this:
public async static Task wait2()
{
Console.WriteLine("Waiting...");
await Task.Delay(2000);
Console.WriteLine("Done!");
}
But we aware that the task is already started so you don't have to call start:
var w2 = wait2();
//w2.Start();
w2.Wait();
I think the problem with your wait2 function is that is creating 2 task, the one in new Task(...) and another in Task.Delay(). You are waiting for the first one, but you are not waiting for the inner one.

c# simple background thread

I am looking for a simple way to do a task in background, then update something (on the main thread) when it completes. It's in a low level 'model' class so I can't call InvokeOnMainThread as I don't have an NSObject around. I have this method:
public void GetItemsAsync(Action<Item[]> handleResult)
{
Item[] items=null;
Task task=Task.Factory.StartNew(() =>
{
items=this.CreateItems(); // May take a second or two
});
task.ContinueWith(delegate
{
handleResult(items);
}, TaskScheduler.FromCurrentSynchronizationContext());
}
This seems to work OK, but
1) Is this the best (simplest) way?
2) I'm worried about the local variable:
Item{} items=null
What stops that disappearing when the method returns before the background thread completes?
Thanks.

I think your method slightly violates a Single Responsibility Principle, because it doing too much.
First of all I suggest to change CreateItems to return Task instead of wrapping it in the GetItemsAsync:
public Task<Item[]> CreateItems(CancellationToken token)
{
return Task.Factory.StartNew(() =>
// obtaining the data...
{});
}
CancellationToken is optional but can help you if you able to cancel this long running operation.
With this method you can remove GetItemsAsync entirely because its so simple to handle results by your client without passing this delegate:
// Somewhere in the client of your class
var task = yourClass.CreateItems(token);
task.ContinueWith(t =>
// Code of the delegate that previously
// passed to GetItemsAsync method
{}, TaskScheduler.FromCurrentSynchronizationContext());
Using this approach you'll get more clear code with only one responsibility. Task class itself is a perfect tool for representing asynchronous operation as a first class object. Using proposed technique you can easily mock you current implementation with a fake behavior for unit testing without changing your clients code.

Something like this:
public void GetItemsAsync(Action<Item[]> handleResult)
{
int Id = 11;
Task<Item[]> task = Task.Factory.StartNew(() => CreateItems(Id)); // May take a second or two
task.ContinueWith(t => handleResult(t.Result), TaskScheduler.FromCurrentSynchronizationContext());
}

Your code looks fine.
It's a perfect example of when to use async / await if you can use C# 5, but if not, you have to write it as you have done with continuations.
The items variable is captured in your lambdas, so that's fine too.
When you write a lambda that uses an outside variable, the C# compiler creates a class that contains the variable. This is called a closure and means you can access the variable inside your lambda.

Here is a nicer soultion:
public void GetItemsAsync(Action<Item[]> handleResult)
{
var task = Task.Factory.StartNew<Item[]>(() =>
{
return this.CreateItems(); // May take a second or two
});
task.ContinueWith(delegate
{
handleResult(task.Result);
}, TaskScheduler.FromCurrentSynchronizationContext());
}