Develop Reference

Unexpected values for AsyncLocal.Value when mixing ExecutionContext.SuppressFlow and tasks

In an application I am experiencing odd behavior due to wrong/unexpected values of AsyncLocal: Despite I suppressed the flow of the execution context, I the AsyncLocal.Value-property is sometimes not reset within the execution scope of a newly spawned Task.
Below I created a minimal reproducible sample which demonstrates the problem:
private static readonly AsyncLocal<object> AsyncLocal = new AsyncLocal<object>();
[TestMethod]
public void Test()
{
Trace.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
AsyncLocal.Value = "1";
Task anotherTask;
using (ExecutionContext.SuppressFlow())
{
anotherTask = Task.Run(() =>
{
Trace.WriteLine(AsyncLocal.Value); // "1" <- ???
Assert.IsNull(AsyncLocal.Value); // BOOM - FAILS
AsyncLocal.Value = "2";
});
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
In nine out of ten runs (on my pc) the outcome of the Test-method is:
.NET 6.0.2
"1"
-> The test fails
As you can see the test fails because within the action which is executed within Task.Run the the previous value is still present within AsyncLocal.Value (Message: 1).
My concrete questions are:
Why does this happen?
I suspect this happens because Task.Run may use the current thread to execute the work load. In that case, I assume lack of async/await-operators does not force the creation of a new/separate ExecutionContext for the action. Like Stephen Cleary said "from the logical call context’s perspective, all synchronous invocations are “collapsed” - they’re actually part of the context of the closest async method further up the call stack". If that’s the case I do understand why the same context is used within the action.
Is this the correct explanation for this behavior? In addition, why does it work flawlessly sometimes (about 1 run out of 10 on my machine)?
How can I fix this?
Assuming that my theory above is true it should be enough to forcefully introduce a new async "layer", like below:
private static readonly AsyncLocal<object> AsyncLocal = new AsyncLocal<object>();
[TestMethod]
public void Test()
{
Trace.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
AsyncLocal.Value = "1";
Task anotherTask;
using (ExecutionContext.SuppressFlow())
{
var wrapper = () =>
{
Trace.WriteLine(AsyncLocal.Value);
Assert.IsNull(AsyncLocal.Value);
AsyncLocal.Value = "2";
return Task.CompletedTask;
};
anotherTask = Task.Run(async () => await wrapper());
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
This seems to fix the problem (it consistently works on my machine), but I want to be sure that this is a correct fix for this problem.
Many thanks in advance

Why does this happen? I suspect this happens because Task.Run may use the current thread to execute the work load.
I suspect that it happens because Task.WaitAll will use the current thread to execute the task inline.
Specifically, Task.WaitAll calls Task.WaitAllCore, which will attempt to run it inline by calling Task.WrappedTryRunInline. I'm going to assume the default task scheduler is used throughout. In that case, this will invoke TaskScheduler.TryRunInline, which will return false if the delegate is already invoked. So, if the task has already started running on a thread pool thread, this will return back to WaitAllCore, which will just do a normal wait, and your code will work as expected (1 out of 10).
If a thread pool thread hasn't picked it up yet (9 out of 10), then TaskScheduler.TryRunInline will call TaskScheduler.TryExecuteTaskInline, the default implementation of which will call Task.ExecuteEntryUnsafe, which calls Task.ExecuteWithThreadLocal. Task.ExecuteWithThreadLocal has logic for applying an ExecutionContext if one was captured. Assuming none was captured, the task's delegate is just invoked directly.
So, it seems like each step is behaving logically. Technically, what ExecutionContext.SuppressFlow means is "don't capture the ExecutionContext", and that is what is happening. It doesn't mean "clear the ExecutionContext". Sometimes the task is run on a thread pool thread (without the captured ExecutionContext), and WaitAll will just wait for it to complete. Other times the task will be executed inline by WaitAll instead of a thread pool thread, and in that case the ExecutionContext is not cleared (and technically isn't captured, either).
You can test this theory by capturing the current thread id within your wrapper and comparing it to the thread id doing the Task.WaitAll. I expect that they will be the same thread for the runs where the async local value is (unexpectedly) inherited, and they will be different threads for the runs where the async local value works as expected.

If you can, I'd first consider whether it's possible to replace the thread-specific caches with a single shared cache. The app likely predates useful types such as ConcurrentDictionary.
If it isn't possible to use a singleton cache, then you can use a stack of async local values. Stacking async local values is a common pattern. I prefer wrapping the stack logic into a separate type (AsyncLocalValue in the code below):
public sealed class AsyncLocalValue
{
private static readonly AsyncLocal<ImmutableStack<object>> _asyncLocal = new();
public object Value => _asyncLocal.Value?.Peek();
public IDisposable PushValue(object value)
{
var originalValue = _asyncLocal.Value;
var newValue = (originalValue ?? ImmutableStack<object>.Empty).Push(value);
_asyncLocal.Value = newValue;
return Disposable.Create(() => _asyncLocal.Value = originalValue);
}
}
private static AsyncLocalValue AsyncLocal = new();
[TestMethod]
public void Test()
{
Console.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
Task anotherTask;
using (AsyncLocal.PushValue("1"))
{
using (AsyncLocal.PushValue(null))
{
anotherTask = Task.Run(() =>
{
Console.WriteLine("Observed: " + AsyncLocal.Value);
using (AsyncLocal.PushValue("2"))
{
}
});
}
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
This code sample uses Disposable.Create from my Nito.Disposables library.

How do I prevent by Rx test from hanging?

I am reproducing my Rx issue with a simplified test case below. The test below hangs. I am sure it is a small, but fundamental, thing that I am missing, but can't put my finger on it.
public class Service
{
private ISubject<double> _subject = new Subject<double>();
public void Reset()
{
_subject.OnNext(0.0);
}
public IObservable<double> GetProgress()
{
return _subject;
}
}
public class ObTest
{
[Fact]
private async Task SimpleTest()
{
var service = new Service();
var result = service.GetProgress().Take(1);
var task = Task.Run(async () =>
{
service.Reset();
});
await result;
}
}
UPDATE
My attempt above was to simplify the problem a little and understand it. In my case GetProgress() is a merge of various Observables that publish the download progress, one of these Observables is a Subject<double> that publishes 0 everytime somebody calls a method to delete the download.
The race condition identified by Enigmativity and Theodor Zoulias may(??) happen in real life. I display a view which attempts to get the progress, however, quick fingers delete it just in time.
What I need to understand a bit more is if the download is started again (subscription has taken place by now, by virtue of displaying a view, which has already made the subscription) and somebody again deletes it.
public class Service
{
private ISubject<double> _deleteSubject = new Subject<double>();
public void Reset()
{
_deleteSubject.OnNext(0.0);
}
public IObservable<double> GetProgress()
{
return _deleteSubject.Merge(downloadProgress);
}
}

Your code isn't hanging. It's awaiting an observable that sometimes never gets a value.
You have a race condition.
The Task.Run is sometimes executing to completion before the await result creates the subscription to the observable - so it never sees the value.
Try this code instead:
private async Task SimpleTest()
{
var service = new Service();
var result = service.GetProgress().Take(1);
var awaiter = result.GetAwaiter();
var task = Task.Run(() =>
{
service.Reset();
});
await awaiter;
}

The line await result creates a subscription to the observable. The problem is that the notification _subject.OnNext(0.0) may occur before this subscription, in which case the value will pass unobserved, and the await result will continue waiting for a notification for ever. In this particular example the notification is always missed, at least in my PC, because the subscription is delayed for around 30 msec (measured with a Stopwatch), which is longer than the time needed for the task that resets the service to complete, probably because the JITer must load and compile some RX-related assembly. The situation changes when I do a warm-up by calling new Subject<int>().FirstAsync().Subscribe() before running the example. In that case the notification is observed almost always, and the hanging is avoided.
I can think of two robust solutions to this problem.
The solution suggested by Enigmativity, to create an awaitable subscription before starting the task that resets the service. This can be done with either GetAwaiter or ToTask.
To use a ReplaySubject<T> instead of a plain vanilla Subject<T>.
Represents an object that is both an observable sequence as well as an observer. Each notification is broadcasted to all subscribed and future observers, subject to buffer trimming policies.
The ReplaySubject will cache the value so that it can be observed by the future subscription, eliminating the race condition. You could initialize it with a bufferSize of 1 to minimize the memory footprint of the buffer.

Why won't this async/await run asynchronously?

I've searched for the answer to this but according to many guides and SO questions this code still appears correct to me, yet it runs synchronously.
private void CheckConditions()
{
foreach (var obj in myObjects)
{
if (obj.ConditionMet)
{
HandleConditionAsync(obj);
}
}
DoOtherWork();
}
private async void HandleConditionAsync(MyObject obj)
{
// shouldn't control transfer back to CheckConditions() here while we wait for user input?
string userInput = await obj.MessagePromptAsync("hello user");
DoSomeBookkeeping(obj);
}
// (MyObject.cs)
private MessagePrompt messagePrompt; // inherits from UserControl
public async Task<string> MessagePromptAsync(string prompt)
{
return await Task.FromResult<string>(messagePrompt.Prompt(prompt));
}
// (MessagePrompt.cs)
public string Prompt(string prompt)
{
this.UIThread(() => this.SetMessagePrompt(prompt));
userInputAutoResetEvent.WaitOne();
return myResult; // set in a button handler that also sets the AutoResetEvent
}
I'm intending for CheckConditions() to continue along merrily but instead it is stuck on MessagePrompt's AutoResetEvent despite my async/awaits. The only thing I can figure might be wrong is that perhaps MessagePrompt's methods aren't able to run asynchronously due to some restriction from UserControl, its use of a UI thread reference, or maybe non-async methods at the top of the stack.

There's nothing in your code that's asynchronous. The only task you have, you created from a result value, meaning the Prompt() method has to complete and return its result before you'll even get the Task object back to wait on. That object will already be completed, so any await on it will complete immediately, once it has the Task to wait on.
Maybe you meant this instead:
public async Task<string> MessagePromptAsync(string prompt)
{
return await Task.Run(() => messagePrompt.Prompt(prompt));
}
Or alternatively (if you really do have nothing else in the MessagePromptAsync() method):
public Task<string> MessagePromptAsync(string prompt)
{
return Task.Run(() => messagePrompt.Prompt(prompt));
}
Note that this may lead to a different problem, depending on what DoOtherWork() and UIThread() actually do. If your UI thread gets tied up in DoOtherWork() and the UIThread() method is wrapping Dispatcher.Invoke() or similar, then you'll have a deadlock.
If that does not address your problem, please provide a good Minimal, Complete, and Verifiable code example that reliably reproduces the problem.

You need to make CheckConditions() async as well, and then await the call to HandleConditionAsync(MyObject obj). CheckConditions() runs synchronously in your sample.
private async Task CheckConditionsAsync()
{
foreach (var obj in myObjects)
{
if (obj.ConditionMet)
{
await HandleConditionAsync(obj);
}
}
DoOtherWork();
}
Also, and this is just a best practices thing, an async method should always return a Task when possible. The only time I've ever had to use async void is for compatibility with an event handler. You can see I've changed CheckConditions() this way, and HandleConditionAsync(MyObject obj) should be modified similarly. I also changed the method name to represent it's asynchronous behaviour.
If you need to run a method that returns a Task synchronously (and you shouldn't do this, this is an indication of something incorrect about your design), you can run it with Task.FromResult(MyMethodAsync()). Again, avoid doing this wherever you can, it defeats the purpose of making a method asynchronous to in the first place.

Unit testing break in foreach loop

I'm trying to unit test the cancel execution scenario in a class conceptually similar to the following:
public class ContextExecutor
{
public ContextExecutor(IContextRunner runner, IExecutionCanceler canceler)
{
this.runner = runner;
this.canceler = canceler;
}
public void Execute(IEnumerable<IContext> contexts)
{
foreach (var ctx in contexts)
{
if (canceler.IsCanceled)
{
break;
}
runner.Run(ctx);
}
}
readonly IContextRunner runner;
readonly IExecutionCanceler canceler;
}
public interface IContextRunner
{
void Run(IContext context);
}
public interface IExecutionCanceler
{
bool IsCanceled { get; }
}
The test case I was after should go through the following steps:
start ContextExecutor.Execute() asynchronously somehow;
put that method execution on hold until something unlocks it from unit test code;
unlock execution and let it perform 1 (..or 2, or..) loop runs, anyway less than full enumerable length;
invoke canceling by setting canceler.IsCanceled = true;
unlock loop execution free;
wait synchronously for method completion;
assert that loop has been invoked the expected nr of times.
I got tangled up with controlling loop execution locking/unlocking from unit test code. Apart from starting Execute() in a new thread, I avoided using threading synchronization primitives (e.g. semaphores and locks). I also had to discard a Task-based approach, as I could not change signatures to apply async/await constructs. I tried to play with the following, but with no luck:
inject a yield-powered function as IEnumerable<IContext> input parameter to hold loop on foreach() line, to release loop everytime another yield is hit, and try to control that from unit test code.
inject a IContextRunner runner powered by a Reactive Extension Subject to hold loop on runner.Run line, to release loop everytime another Subject.OnNext is hit, and try to control that from unit test code.
For that matters, unit testing framework is NUnit, while NSubstitute is the mocking framework and FluentAssertion is the assertion library of choice. I know how to arrange/act/assert with those.
What is so evident that I missing? Thanks
EDIT
To provide an example of what has been tried, this is a Task-based approach made after posting question and reading #Peter Duniho helpful comment:
// in unit test class
ContextExecutor executor;
IContextRunner runner;
IExecutionCanceler canceler;
IRunnableContext[] runnableContexts;
int totalNrOfContexts;
int nrOfContextToRun; // this will be < totalNrOfContexts
int actualNrOfContextRan;
[SetUp]
public virtual void before_each()
{
// create instance under test, mock dependencies, dummy input data
Initialize();
RunScenarioAsync().Wait();
}
async Task RunScenarioAsync()
{
// prepare mock IContextRunner so that for each input context:
// * there's a related TaskCompletionSource<Object>
// * Run() increments actualNrOfContextRan
// * Run() performs taskSource.Task.Wait();
List<TaskCompletionSource<Object>> runTaskSources = PrepareMockContextRunner();
canceler.IsCanceled.Returns(false); // let execution go initially
// queue up method under test to be processed asynchronously
var executeTask = Task.Run(() =>
{
executor.Execute(runnableContexts);
};
// "unlock" some IContextRunner.Run() invocations,
// for when they will be invoked
for (int i = 0; i < nrOfContextToRun; i++)
{
runTaskSources[i].SetResult(null);
await Task.Delay(0); // tried also with Delay(1) and without this line at all
}
// flag to cancel execution
canceler.IsCanceled.Returns(true);
// unlock all remaining IContextRunner.Run() invocations,
// again for when/if they will be invoked
for (int i = nrOfContextToRun; i < totalNrOfContexts; i++)
{
runTaskSources[i].SetResult(null);
await Task.Delay(0);
}
// wait until method under test completes
await executeTask;
}
[Test]
public void it_should_only_run_until_cancel()
{
int expected = nrOfContextToRun;
int actual = actualNrOfContextRan;
actual.Should().Be(expected);
}
The problem I have here (and similar to other approaches tried) is about giving and regain control to/from the method under test in a predictable way (that is, synchronizing).
Here, if there's no await Task.Delay() or if delay is 0ms, only 1 context is actually ran: the method under test has no chance to run the 2nd and 3rd one, it finds the canceling flag too soon. If delay is 1ms, method executes more context than expected before actually detecting the flag. Also tried with ticks instead of ms, but in my experience playing with delays usually means you're doing something wrong.

Unexpected behavior when passing async Actions around

I'm pretty familiar with the async/await pattern, but I'm bumping into some behavior that strikes me as odd. I'm sure there's a perfectly valid reason why it's happening, and I'd love to understand the behavior.
The background here is that I'm developing a Windows Store app, and since I'm a cautious, conscientious developer, I'm unit testing everything. I discovered pretty quickly that the ExpectedExceptionAttribute doesn't exist for WSAs. Weird, right? Well, no problem! I can more-or-less replicate the behavior with an extension method! So I wrote this:
public static class TestHelpers
{
// There's no ExpectedExceptionAttribute for Windows Store apps! Why must Microsoft make my life so hard?!
public static void AssertThrowsExpectedException<T>(this Action a) where T : Exception
{
try
{
a();
}
catch (T)
{
return;
}
Assert.Fail("The expected exception was not thrown");
}
}
And lo, it works beautifully.
So I continued happily writing my unit tests, until I hit an async method that I wanted to confirm throws an exception under certain circumstances. "No problem," I thought to myself, "I can just pass in an async lambda!"
So I wrote this test method:
[TestMethod]
public async Task Network_Interface_Being_Unavailable_Throws_Exception()
{
var webManager = new FakeWebManager
{
IsNetworkAvailable = false
};
var am = new AuthenticationManager(webManager);
Action authenticate = async () => await am.Authenticate("foo", "bar");
authenticate.AssertThrowsExpectedException<LoginFailedException>();
}
This, surprisingly, throws a runtime error. It actually crashes the test-runner!
I made an overload of my AssertThrowsExpectedException method:
public static async Task AssertThrowsExpectedException<TException>(this Func<Task> a) where TException : Exception
{
try
{
await a();
}
catch (TException)
{
return;
}
Assert.Fail("The expected exception was not thrown");
}
and I tweaked my test:
[TestMethod]
public async Task Network_Interface_Being_Unavailable_Throws_Exception()
{
var webManager = new FakeWebManager
{
IsNetworkAvailable = false
};
var am = new AuthenticationManager(webManager);
Func<Task> authenticate = async () => await am.Authenticate("foo", "bar");
await authenticate.AssertThrowsExpectedException<LoginFailedException>();
}
I'm fine with my solution, I'm just wondering exactly why everything goes pear-shaped when I try to invoke the async Action. I'm guessing because, as far as the runtime is concerned, it's not an Action, I'm just cramming the lambda into it. I know the lambda will happily be assigned to either Action or Func<Task>.

It is not surprising that it may crash the tester, in your second code fragment scenario:
Action authenticate = async () => await am.Authenticate("foo", "bar");
authenticate.AssertThrowsExpectedException<LoginFailedException>();
It's actually a fire-and-forget invocation of an async void method, when you call the action:
try
{
a();
}
The a() returns instantly, and so does the AssertThrowsExpectedException method. At the same time, some activity started inside am.Authenticate may continue executing in the background, possibly on a pool thread. What's exactly going on there depends on the implementation of am.Authenticate, but it may crash your tester later, when such async operation is completed and it throws LoginFailedException. I'm not sure what is the synchronization context of the unit test execution environment, but if it uses the default SynchronizationContext, the exception may indeed be thrown unobserved on a different thread in this case.
VS2012 automatically supports asynchronous unit tests, as long as the test method signatures are async Task. So, I think you've answered your own question by using await and Func<T> for your test.