Develop Reference

Unexpected values for AsyncLocal.Value when mixing ExecutionContext.SuppressFlow and tasks

In an application I am experiencing odd behavior due to wrong/unexpected values of AsyncLocal: Despite I suppressed the flow of the execution context, I the AsyncLocal.Value-property is sometimes not reset within the execution scope of a newly spawned Task.
Below I created a minimal reproducible sample which demonstrates the problem:
private static readonly AsyncLocal<object> AsyncLocal = new AsyncLocal<object>();
[TestMethod]
public void Test()
{
Trace.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
AsyncLocal.Value = "1";
Task anotherTask;
using (ExecutionContext.SuppressFlow())
{
anotherTask = Task.Run(() =>
{
Trace.WriteLine(AsyncLocal.Value); // "1" <- ???
Assert.IsNull(AsyncLocal.Value); // BOOM - FAILS
AsyncLocal.Value = "2";
});
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
In nine out of ten runs (on my pc) the outcome of the Test-method is:
.NET 6.0.2
"1"
-> The test fails
As you can see the test fails because within the action which is executed within Task.Run the the previous value is still present within AsyncLocal.Value (Message: 1).
My concrete questions are:
Why does this happen?
I suspect this happens because Task.Run may use the current thread to execute the work load. In that case, I assume lack of async/await-operators does not force the creation of a new/separate ExecutionContext for the action. Like Stephen Cleary said "from the logical call context’s perspective, all synchronous invocations are “collapsed” - they’re actually part of the context of the closest async method further up the call stack". If that’s the case I do understand why the same context is used within the action.
Is this the correct explanation for this behavior? In addition, why does it work flawlessly sometimes (about 1 run out of 10 on my machine)?
How can I fix this?
Assuming that my theory above is true it should be enough to forcefully introduce a new async "layer", like below:
private static readonly AsyncLocal<object> AsyncLocal = new AsyncLocal<object>();
[TestMethod]
public void Test()
{
Trace.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
AsyncLocal.Value = "1";
Task anotherTask;
using (ExecutionContext.SuppressFlow())
{
var wrapper = () =>
{
Trace.WriteLine(AsyncLocal.Value);
Assert.IsNull(AsyncLocal.Value);
AsyncLocal.Value = "2";
return Task.CompletedTask;
};
anotherTask = Task.Run(async () => await wrapper());
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
This seems to fix the problem (it consistently works on my machine), but I want to be sure that this is a correct fix for this problem.
Many thanks in advance

Why does this happen? I suspect this happens because Task.Run may use the current thread to execute the work load.
I suspect that it happens because Task.WaitAll will use the current thread to execute the task inline.
Specifically, Task.WaitAll calls Task.WaitAllCore, which will attempt to run it inline by calling Task.WrappedTryRunInline. I'm going to assume the default task scheduler is used throughout. In that case, this will invoke TaskScheduler.TryRunInline, which will return false if the delegate is already invoked. So, if the task has already started running on a thread pool thread, this will return back to WaitAllCore, which will just do a normal wait, and your code will work as expected (1 out of 10).
If a thread pool thread hasn't picked it up yet (9 out of 10), then TaskScheduler.TryRunInline will call TaskScheduler.TryExecuteTaskInline, the default implementation of which will call Task.ExecuteEntryUnsafe, which calls Task.ExecuteWithThreadLocal. Task.ExecuteWithThreadLocal has logic for applying an ExecutionContext if one was captured. Assuming none was captured, the task's delegate is just invoked directly.
So, it seems like each step is behaving logically. Technically, what ExecutionContext.SuppressFlow means is "don't capture the ExecutionContext", and that is what is happening. It doesn't mean "clear the ExecutionContext". Sometimes the task is run on a thread pool thread (without the captured ExecutionContext), and WaitAll will just wait for it to complete. Other times the task will be executed inline by WaitAll instead of a thread pool thread, and in that case the ExecutionContext is not cleared (and technically isn't captured, either).
You can test this theory by capturing the current thread id within your wrapper and comparing it to the thread id doing the Task.WaitAll. I expect that they will be the same thread for the runs where the async local value is (unexpectedly) inherited, and they will be different threads for the runs where the async local value works as expected.

If you can, I'd first consider whether it's possible to replace the thread-specific caches with a single shared cache. The app likely predates useful types such as ConcurrentDictionary.
If it isn't possible to use a singleton cache, then you can use a stack of async local values. Stacking async local values is a common pattern. I prefer wrapping the stack logic into a separate type (AsyncLocalValue in the code below):
public sealed class AsyncLocalValue
{
private static readonly AsyncLocal<ImmutableStack<object>> _asyncLocal = new();
public object Value => _asyncLocal.Value?.Peek();
public IDisposable PushValue(object value)
{
var originalValue = _asyncLocal.Value;
var newValue = (originalValue ?? ImmutableStack<object>.Empty).Push(value);
_asyncLocal.Value = newValue;
return Disposable.Create(() => _asyncLocal.Value = originalValue);
}
}
private static AsyncLocalValue AsyncLocal = new();
[TestMethod]
public void Test()
{
Console.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
Task anotherTask;
using (AsyncLocal.PushValue("1"))
{
using (AsyncLocal.PushValue(null))
{
anotherTask = Task.Run(() =>
{
Console.WriteLine("Observed: " + AsyncLocal.Value);
using (AsyncLocal.PushValue("2"))
{
}
});
}
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
This code sample uses Disposable.Create from my Nito.Disposables library.

How do I prevent by Rx test from hanging?

I am reproducing my Rx issue with a simplified test case below. The test below hangs. I am sure it is a small, but fundamental, thing that I am missing, but can't put my finger on it.
public class Service
{
private ISubject<double> _subject = new Subject<double>();
public void Reset()
{
_subject.OnNext(0.0);
}
public IObservable<double> GetProgress()
{
return _subject;
}
}
public class ObTest
{
[Fact]
private async Task SimpleTest()
{
var service = new Service();
var result = service.GetProgress().Take(1);
var task = Task.Run(async () =>
{
service.Reset();
});
await result;
}
}
UPDATE
My attempt above was to simplify the problem a little and understand it. In my case GetProgress() is a merge of various Observables that publish the download progress, one of these Observables is a Subject<double> that publishes 0 everytime somebody calls a method to delete the download.
The race condition identified by Enigmativity and Theodor Zoulias may(??) happen in real life. I display a view which attempts to get the progress, however, quick fingers delete it just in time.
What I need to understand a bit more is if the download is started again (subscription has taken place by now, by virtue of displaying a view, which has already made the subscription) and somebody again deletes it.
public class Service
{
private ISubject<double> _deleteSubject = new Subject<double>();
public void Reset()
{
_deleteSubject.OnNext(0.0);
}
public IObservable<double> GetProgress()
{
return _deleteSubject.Merge(downloadProgress);
}
}

Your code isn't hanging. It's awaiting an observable that sometimes never gets a value.
You have a race condition.
The Task.Run is sometimes executing to completion before the await result creates the subscription to the observable - so it never sees the value.
Try this code instead:
private async Task SimpleTest()
{
var service = new Service();
var result = service.GetProgress().Take(1);
var awaiter = result.GetAwaiter();
var task = Task.Run(() =>
{
service.Reset();
});
await awaiter;
}

The line await result creates a subscription to the observable. The problem is that the notification _subject.OnNext(0.0) may occur before this subscription, in which case the value will pass unobserved, and the await result will continue waiting for a notification for ever. In this particular example the notification is always missed, at least in my PC, because the subscription is delayed for around 30 msec (measured with a Stopwatch), which is longer than the time needed for the task that resets the service to complete, probably because the JITer must load and compile some RX-related assembly. The situation changes when I do a warm-up by calling new Subject<int>().FirstAsync().Subscribe() before running the example. In that case the notification is observed almost always, and the hanging is avoided.
I can think of two robust solutions to this problem.
The solution suggested by Enigmativity, to create an awaitable subscription before starting the task that resets the service. This can be done with either GetAwaiter or ToTask.
To use a ReplaySubject<T> instead of a plain vanilla Subject<T>.
Represents an object that is both an observable sequence as well as an observer. Each notification is broadcasted to all subscribed and future observers, subject to buffer trimming policies.
The ReplaySubject will cache the value so that it can be observed by the future subscription, eliminating the race condition. You could initialize it with a bufferSize of 1 to minimize the memory footprint of the buffer.

Unit testing break in foreach loop

I'm trying to unit test the cancel execution scenario in a class conceptually similar to the following:
public class ContextExecutor
{
public ContextExecutor(IContextRunner runner, IExecutionCanceler canceler)
{
this.runner = runner;
this.canceler = canceler;
}
public void Execute(IEnumerable<IContext> contexts)
{
foreach (var ctx in contexts)
{
if (canceler.IsCanceled)
{
break;
}
runner.Run(ctx);
}
}
readonly IContextRunner runner;
readonly IExecutionCanceler canceler;
}
public interface IContextRunner
{
void Run(IContext context);
}
public interface IExecutionCanceler
{
bool IsCanceled { get; }
}
The test case I was after should go through the following steps:
start ContextExecutor.Execute() asynchronously somehow;
put that method execution on hold until something unlocks it from unit test code;
unlock execution and let it perform 1 (..or 2, or..) loop runs, anyway less than full enumerable length;
invoke canceling by setting canceler.IsCanceled = true;
unlock loop execution free;
wait synchronously for method completion;
assert that loop has been invoked the expected nr of times.
I got tangled up with controlling loop execution locking/unlocking from unit test code. Apart from starting Execute() in a new thread, I avoided using threading synchronization primitives (e.g. semaphores and locks). I also had to discard a Task-based approach, as I could not change signatures to apply async/await constructs. I tried to play with the following, but with no luck:
inject a yield-powered function as IEnumerable<IContext> input parameter to hold loop on foreach() line, to release loop everytime another yield is hit, and try to control that from unit test code.
inject a IContextRunner runner powered by a Reactive Extension Subject to hold loop on runner.Run line, to release loop everytime another Subject.OnNext is hit, and try to control that from unit test code.
For that matters, unit testing framework is NUnit, while NSubstitute is the mocking framework and FluentAssertion is the assertion library of choice. I know how to arrange/act/assert with those.
What is so evident that I missing? Thanks
EDIT
To provide an example of what has been tried, this is a Task-based approach made after posting question and reading #Peter Duniho helpful comment:
// in unit test class
ContextExecutor executor;
IContextRunner runner;
IExecutionCanceler canceler;
IRunnableContext[] runnableContexts;
int totalNrOfContexts;
int nrOfContextToRun; // this will be < totalNrOfContexts
int actualNrOfContextRan;
[SetUp]
public virtual void before_each()
{
// create instance under test, mock dependencies, dummy input data
Initialize();
RunScenarioAsync().Wait();
}
async Task RunScenarioAsync()
{
// prepare mock IContextRunner so that for each input context:
// * there's a related TaskCompletionSource<Object>
// * Run() increments actualNrOfContextRan
// * Run() performs taskSource.Task.Wait();
List<TaskCompletionSource<Object>> runTaskSources = PrepareMockContextRunner();
canceler.IsCanceled.Returns(false); // let execution go initially
// queue up method under test to be processed asynchronously
var executeTask = Task.Run(() =>
{
executor.Execute(runnableContexts);
};
// "unlock" some IContextRunner.Run() invocations,
// for when they will be invoked
for (int i = 0; i < nrOfContextToRun; i++)
{
runTaskSources[i].SetResult(null);
await Task.Delay(0); // tried also with Delay(1) and without this line at all
}
// flag to cancel execution
canceler.IsCanceled.Returns(true);
// unlock all remaining IContextRunner.Run() invocations,
// again for when/if they will be invoked
for (int i = nrOfContextToRun; i < totalNrOfContexts; i++)
{
runTaskSources[i].SetResult(null);
await Task.Delay(0);
}
// wait until method under test completes
await executeTask;
}
[Test]
public void it_should_only_run_until_cancel()
{
int expected = nrOfContextToRun;
int actual = actualNrOfContextRan;
actual.Should().Be(expected);
}
The problem I have here (and similar to other approaches tried) is about giving and regain control to/from the method under test in a predictable way (that is, synchronizing).
Here, if there's no await Task.Delay() or if delay is 0ms, only 1 context is actually ran: the method under test has no chance to run the 2nd and 3rd one, it finds the canceling flag too soon. If delay is 1ms, method executes more context than expected before actually detecting the flag. Also tried with ticks instead of ms, but in my experience playing with delays usually means you're doing something wrong.

Async and Await - How is order of execution maintained?

I am actually reading some topics about the Task Parallel Library and the asynchronous programming with async and await. The book "C# 5.0 in a Nutshell" states that when awaiting an expression using the await keyword, the compiler transforms the code into something like this:
var awaiter = expression.GetAwaiter();
awaiter.OnCompleted (() =>
{
var result = awaiter.GetResult();
Let's assume, we have this asynchronous function (also from the referred book):
async Task DisplayPrimeCounts()
{
for (int i = 0; i < 10; i++)
Console.WriteLine (await GetPrimesCountAsync (i*1000000 + 2, 1000000) +
" primes between " + (i*1000000) + " and " + ((i+1)*1000000-1));
Console.WriteLine ("Done!");
}
The call of the 'GetPrimesCountAsync' method will be enqueued and executed on a pooled thread. In general invoking multiple threads from within a for loop has the potential for introducing race conditions.
So how does the CLR ensure that the requests will be processed in the order they were made? I doubt that the compiler simply transforms the code into the above manner, since this would decouple the 'GetPrimesCountAsync' method from the for loop.

Just for the sake of simplicity, I'm going to replace your example with one that's slightly simpler, but has all of the same meaningful properties:
async Task DisplayPrimeCounts()
{
for (int i = 0; i < 10; i++)
{
var value = await SomeExpensiveComputation(i);
Console.WriteLine(value);
}
Console.WriteLine("Done!");
}
The ordering is all maintained because of the definition of your code. Let's imagine stepping through it.
This method is first called
The first line of code is the for loop, so i is initialized.
The loop check passes, so we go to the body of the loop.
SomeExpensiveComputation is called. It should return a Task<T> very quickly, but the work that it'd doing will keep going on in the background.
The rest of the method is added as a continuation to the returned task; it will continue executing when that task finishes.
After the task returned from SomeExpensiveComputation finishes, we store the result in value.
value is printed to the console.
GOTO 3; note that the existing expensive operation has already finished before we get to step 4 for the second time and start the next one.
As far as how the C# compiler actually accomplishes step 5, it does so by creating a state machine. Basically every time there is an await there's a label indicating where it left off, and at the start of the method (or after it's resumed after any continuation fires) it checks the current state, and does a goto to the spot where it left off. It also needs to hoist all local variables into fields of a new class so that the state of those local variables is maintained.
Now this transformation isn't actually done in C# code, it's done in IL, but this is sort of the morale equivalent of the code I showed above in a state machine. Note that this isn't valid C# (you cannot goto into a a for loop like this, but that restriction doesn't apply to the IL code that is actually used. There are also going to be differences between this and what C# actually does, but is should give you a basic idea of what's going on here:
internal class Foo
{
public int i;
public long value;
private int state = 0;
private Task<int> task;
int result0;
public Task Bar()
{
var tcs = new TaskCompletionSource<object>();
Action continuation = null;
continuation = () =>
{
try
{
if (state == 1)
{
goto state1;
}
for (i = 0; i < 10; i++)
{
Task<int> task = SomeExpensiveComputation(i);
var awaiter = task.GetAwaiter();
if (!awaiter.IsCompleted)
{
awaiter.OnCompleted(() =>
{
result0 = awaiter.GetResult();
continuation();
});
state = 1;
return;
}
else
{
result0 = awaiter.GetResult();
}
state1:
Console.WriteLine(value);
}
Console.WriteLine("Done!");
tcs.SetResult(true);
}
catch (Exception e)
{
tcs.SetException(e);
}
};
continuation();
}
}
Note that I've ignored task cancellation for the sake of this example, I've ignored the whole concept of capturing the current synchronization context, there's a bit more going on with error handling, etc. Don't consider this a complete implementation.

The call of the 'GetPrimesCountAsync' method will be enqueued and executed on a pooled thread.
No. await does not initiate any kind of background processing. It waits for existing processing to complete. It is up to GetPrimesCountAsync to do that (e.g. using Task.Run). It's more clear this way:
var myRunningTask = GetPrimesCountAsync();
await myRunningTask;
The loop only continues when the awaited task has completed. There is never more than one task outstanding.
So how does the CLR ensure that the requests will be processed in the order they were made?
The CLR is not involved.
I doubt that the compiler simply transforms the code into the above manner, since this would decouple the 'GetPrimesCountAsync' method from the for loop.
The transform that you shows is basically right but notice that the next loop iteration is not started right away but in the callback. That's what serializes execution.

Unit testing methods with a local Thread

I have a method that looks like this:
protected void OnBarcodeScan(BarcodeScannerEventArgs e)
{
// We need to do this on a seperate thread so we don't block the main thread.
ThreadStart starter = () => SendScanMessage(e, _scanDelegates);
Thread scanThread = new Thread(starter);
scanThread.Start();
}
Then the thread goes off and does some logic (and ends up calling a delegate in my test).
My problem is that my unit test finishes before the thread does. So my test fails.
I can just add in System.Threading.Thread.Sleep(1000); and hope that the logic never takes more than a second (it should not). But that seems like a hack.
The problem is that I don't want to expose that thread to the outside world or even to the rest of the class.
Is there some cool way to find that thread again and wait for it in my unit test?
Something like this:
[TestMethod]
[HostType("Moles")]
public void AddDelegateToScanner_ScanHappens_ScanDelegateIsCalled()
{
// Arrange
bool scanCalled = false;
MCoreDLL.GetTopWindow = () => (new IntPtr(FauxHandle));
// Act
_scanner.AddDelegateToScanner(_formIdentity, ((evnt) => { scanCalled = true; }));
_scanner.SendScan(new BarcodeScannerEventArgs("12345678910"));
// This line is fake!
System.Threading.Thread.CoolMethodToFindMyThread().Join();
// Assert
Assert.IsTrue(scanCalled);
}
I obviously made up the CoolMethodToFindMyThread method. But is there some why to do that?

So if I understand how this works, then the delegates you register are the ones being called on the second thread, right? In that case, you can use thread synchronization in your test and the delegate that gets called. I do this kind of thing in my unit tests all the time.
Something like this:
[TestMethod]
[HostType("Moles")]
public void AddDelegateToScanner_ScanHappens_ScanDelegateIsCalled()
{
// Arrange
var scanCalledEvent = new ManualResetEvent(false);
MCoreDLL.GetTopWindow = () => (new IntPtr(FauxHandle));
// Act
_scanner.AddDelegateToScanner(_formIdentity, ((evnt) => { scanCalledEvent.Set(); }));
_scanner.SendScan(new BarcodeScannerEventArgs("12345678910"));
// Wait for event to fire
bool scanCalledInTime = scanCalledEvent.WaitOne(SOME_TIMEOUT_IN_MILLISECONDS);
// Assert
Assert.IsTrue(scanCalledInTime);
}
It's important to have some sort of timeout in there, otherwise if something goes wrong your test just locks up and that's kind of hard to debug. WaitOne will block until the event gets set or the timeout expires, the return value tells you which happened.
(WARNING: I may have the return value backwards - I don't remember off the top of my head if true means the event got set or if true means the timeout expired. Check the docs.)
There are several sync primitives you can use here, which one depends on what you want to do. ManualResetEvent usually works pretty well for me.

There's another way of doing things:
First, have an AutoResetEvent (or ManualResetEvent, if you feel like) in your test class.
In your test method:
//set up stuff
testEvent.WaitOne();
//ensure everything works
In your callback
testEvent.Set();
Your test method will then stop until the callback gets called.
Presumably you'll want some sort of timeout on that wait call as well.