Develop Reference

Unexpected values for AsyncLocal.Value when mixing ExecutionContext.SuppressFlow and tasks

In an application I am experiencing odd behavior due to wrong/unexpected values of AsyncLocal: Despite I suppressed the flow of the execution context, I the AsyncLocal.Value-property is sometimes not reset within the execution scope of a newly spawned Task.
Below I created a minimal reproducible sample which demonstrates the problem:
private static readonly AsyncLocal<object> AsyncLocal = new AsyncLocal<object>();
[TestMethod]
public void Test()
{
Trace.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
AsyncLocal.Value = "1";
Task anotherTask;
using (ExecutionContext.SuppressFlow())
{
anotherTask = Task.Run(() =>
{
Trace.WriteLine(AsyncLocal.Value); // "1" <- ???
Assert.IsNull(AsyncLocal.Value); // BOOM - FAILS
AsyncLocal.Value = "2";
});
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
In nine out of ten runs (on my pc) the outcome of the Test-method is:
.NET 6.0.2
"1"
-> The test fails
As you can see the test fails because within the action which is executed within Task.Run the the previous value is still present within AsyncLocal.Value (Message: 1).
My concrete questions are:
Why does this happen?
I suspect this happens because Task.Run may use the current thread to execute the work load. In that case, I assume lack of async/await-operators does not force the creation of a new/separate ExecutionContext for the action. Like Stephen Cleary said "from the logical call context’s perspective, all synchronous invocations are “collapsed” - they’re actually part of the context of the closest async method further up the call stack". If that’s the case I do understand why the same context is used within the action.
Is this the correct explanation for this behavior? In addition, why does it work flawlessly sometimes (about 1 run out of 10 on my machine)?
How can I fix this?
Assuming that my theory above is true it should be enough to forcefully introduce a new async "layer", like below:
private static readonly AsyncLocal<object> AsyncLocal = new AsyncLocal<object>();
[TestMethod]
public void Test()
{
Trace.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
AsyncLocal.Value = "1";
Task anotherTask;
using (ExecutionContext.SuppressFlow())
{
var wrapper = () =>
{
Trace.WriteLine(AsyncLocal.Value);
Assert.IsNull(AsyncLocal.Value);
AsyncLocal.Value = "2";
return Task.CompletedTask;
};
anotherTask = Task.Run(async () => await wrapper());
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
This seems to fix the problem (it consistently works on my machine), but I want to be sure that this is a correct fix for this problem.
Many thanks in advance

Why does this happen? I suspect this happens because Task.Run may use the current thread to execute the work load.
I suspect that it happens because Task.WaitAll will use the current thread to execute the task inline.
Specifically, Task.WaitAll calls Task.WaitAllCore, which will attempt to run it inline by calling Task.WrappedTryRunInline. I'm going to assume the default task scheduler is used throughout. In that case, this will invoke TaskScheduler.TryRunInline, which will return false if the delegate is already invoked. So, if the task has already started running on a thread pool thread, this will return back to WaitAllCore, which will just do a normal wait, and your code will work as expected (1 out of 10).
If a thread pool thread hasn't picked it up yet (9 out of 10), then TaskScheduler.TryRunInline will call TaskScheduler.TryExecuteTaskInline, the default implementation of which will call Task.ExecuteEntryUnsafe, which calls Task.ExecuteWithThreadLocal. Task.ExecuteWithThreadLocal has logic for applying an ExecutionContext if one was captured. Assuming none was captured, the task's delegate is just invoked directly.
So, it seems like each step is behaving logically. Technically, what ExecutionContext.SuppressFlow means is "don't capture the ExecutionContext", and that is what is happening. It doesn't mean "clear the ExecutionContext". Sometimes the task is run on a thread pool thread (without the captured ExecutionContext), and WaitAll will just wait for it to complete. Other times the task will be executed inline by WaitAll instead of a thread pool thread, and in that case the ExecutionContext is not cleared (and technically isn't captured, either).
You can test this theory by capturing the current thread id within your wrapper and comparing it to the thread id doing the Task.WaitAll. I expect that they will be the same thread for the runs where the async local value is (unexpectedly) inherited, and they will be different threads for the runs where the async local value works as expected.

If you can, I'd first consider whether it's possible to replace the thread-specific caches with a single shared cache. The app likely predates useful types such as ConcurrentDictionary.
If it isn't possible to use a singleton cache, then you can use a stack of async local values. Stacking async local values is a common pattern. I prefer wrapping the stack logic into a separate type (AsyncLocalValue in the code below):
public sealed class AsyncLocalValue
{
private static readonly AsyncLocal<ImmutableStack<object>> _asyncLocal = new();
public object Value => _asyncLocal.Value?.Peek();
public IDisposable PushValue(object value)
{
var originalValue = _asyncLocal.Value;
var newValue = (originalValue ?? ImmutableStack<object>.Empty).Push(value);
_asyncLocal.Value = newValue;
return Disposable.Create(() => _asyncLocal.Value = originalValue);
}
}
private static AsyncLocalValue AsyncLocal = new();
[TestMethod]
public void Test()
{
Console.WriteLine(System.Runtime.InteropServices.RuntimeInformation.FrameworkDescription);
var mainTask = Task.Factory.StartNew(() =>
{
Task anotherTask;
using (AsyncLocal.PushValue("1"))
{
using (AsyncLocal.PushValue(null))
{
anotherTask = Task.Run(() =>
{
Console.WriteLine("Observed: " + AsyncLocal.Value);
using (AsyncLocal.PushValue("2"))
{
}
});
}
}
Task.WaitAll(anotherTask);
});
mainTask.Wait(500000, CancellationToken.None);
}
This code sample uses Disposable.Create from my Nito.Disposables library.

How do I prevent by Rx test from hanging?

I am reproducing my Rx issue with a simplified test case below. The test below hangs. I am sure it is a small, but fundamental, thing that I am missing, but can't put my finger on it.
public class Service
{
private ISubject<double> _subject = new Subject<double>();
public void Reset()
{
_subject.OnNext(0.0);
}
public IObservable<double> GetProgress()
{
return _subject;
}
}
public class ObTest
{
[Fact]
private async Task SimpleTest()
{
var service = new Service();
var result = service.GetProgress().Take(1);
var task = Task.Run(async () =>
{
service.Reset();
});
await result;
}
}
UPDATE
My attempt above was to simplify the problem a little and understand it. In my case GetProgress() is a merge of various Observables that publish the download progress, one of these Observables is a Subject<double> that publishes 0 everytime somebody calls a method to delete the download.
The race condition identified by Enigmativity and Theodor Zoulias may(??) happen in real life. I display a view which attempts to get the progress, however, quick fingers delete it just in time.
What I need to understand a bit more is if the download is started again (subscription has taken place by now, by virtue of displaying a view, which has already made the subscription) and somebody again deletes it.
public class Service
{
private ISubject<double> _deleteSubject = new Subject<double>();
public void Reset()
{
_deleteSubject.OnNext(0.0);
}
public IObservable<double> GetProgress()
{
return _deleteSubject.Merge(downloadProgress);
}
}

Your code isn't hanging. It's awaiting an observable that sometimes never gets a value.
You have a race condition.
The Task.Run is sometimes executing to completion before the await result creates the subscription to the observable - so it never sees the value.
Try this code instead:
private async Task SimpleTest()
{
var service = new Service();
var result = service.GetProgress().Take(1);
var awaiter = result.GetAwaiter();
var task = Task.Run(() =>
{
service.Reset();
});
await awaiter;
}

The line await result creates a subscription to the observable. The problem is that the notification _subject.OnNext(0.0) may occur before this subscription, in which case the value will pass unobserved, and the await result will continue waiting for a notification for ever. In this particular example the notification is always missed, at least in my PC, because the subscription is delayed for around 30 msec (measured with a Stopwatch), which is longer than the time needed for the task that resets the service to complete, probably because the JITer must load and compile some RX-related assembly. The situation changes when I do a warm-up by calling new Subject<int>().FirstAsync().Subscribe() before running the example. In that case the notification is observed almost always, and the hanging is avoided.
I can think of two robust solutions to this problem.
The solution suggested by Enigmativity, to create an awaitable subscription before starting the task that resets the service. This can be done with either GetAwaiter or ToTask.
To use a ReplaySubject<T> instead of a plain vanilla Subject<T>.
Represents an object that is both an observable sequence as well as an observer. Each notification is broadcasted to all subscribed and future observers, subject to buffer trimming policies.
The ReplaySubject will cache the value so that it can be observed by the future subscription, eliminating the race condition. You could initialize it with a bufferSize of 1 to minimize the memory footprint of the buffer.

Windows 8 Thread improvements - alternative to Task.Run()?

I have inherited a C#/XAML/Win 8 application. There is some code which is set to run every n seconds.
The code that sets that up is:
if(!_syncThreadStarted)
{
await Task.Run(() => SyncToDatabase());
_syncThreadStarted = true;
}
The above code is ran once.
And then inside SyncToDatabase() we have:
while (true)
{
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate();
await Task.Delay(10); // after elapsed time re-run above code
}
The method DeserializeAndUpdate queries a in-memory collection of objects and pushes those objects to a web service.
Sometimes the send request to the web service takes longer than expected meaning duplicate items are sent.
Question: Is there a way to have a thread or some type of thread pool/background worker which I can stop/abort/destroy inside the method SyncToDatabase() , and then initialize/start it once we are done? This will ensure no subsequent requests are fired while a previous request is still pending.
Edit: I am not very knowledgeable when it comes to Threads, but the logic I want is:
Create thread which runs some method every x seconds, and when it starts that thread stop the "running every x seconds" part, after thread has complete start the "run every x seconds" part again.
E.g. if the thread kicks off at 10:01:30AM and does not complete until 10:01:39AM (9 seconds) the next thread should start at 10:01:44AM (5 seconds after work completed) - does that make sense? I do not want 2 or more threads running at the same time.
Here is my code for the above:
var period = TimeSpan.FromSeconds(5);
var completed = true;
ThreadPoolTimer syncTimer = ThreadPoolTimer.CreatePeriodicTimer(async (source) =>
{
// stop further threads from starting (in case this work takes longer than var period)
syncTimer.Cancel();
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate(); // makes webservices calls
Dispatcher.RunAsync(CoreDispatcerPriority.High, async () =>
{
// Update UI
}
completed = true;
}, period,
(source) =>
{
if(!completed)
{
syncTimer.Cancel(); // not sure if this is correct...
}
}
Thanks,
Andrew)

This is not specific to Windows 8. Usually Task.Run is used for CPU-bound work, to offload it to a pool thread and keep the UI (or the core service loop) responsive. In your case, as far as I can tell, the main payload is dbSyncer.DeserializeAndUpdate, which is already asynchronous and most likely network-IO bound, rather than CPU-bound.
Besides, the author of the original code does _syncThreadStarted = true after await Task.Run(() => SyncToDatabase()). That doesn't make sense, because the work on the pool thread would have been already done by the time _syncThreadStarted = true is executed, thanks to the await.
To cancel the loop inside SyncToDatabase you could use Task Cancellation Pattern. Is SyncToDatabase itself an async method? I presume so, because there's an await in the while loop. Given that, the code which calls it could look something like this:
if(_syncTask != null && !_syncTask.IsCompleted)
{
_ct.Cancel();
// here you may want to make sure that the pending task has been fully shut down,
// keeping possible re-entrancy in mind
// See: https://stackoverflow.com/questions/18999827/a-pattern-for-self-cancelling-and-restarting-task
_syncTask = null;
}
_ct = new CancellationTokenSource();
// _syncTask = SyncToDatabase(ct.Token); // do not await
// edited to run on another thread, as requested by the OP
var _syncTask = Task.Run(async () => await SyncToDatabase(ct.Token), ct.Token);
_syncThreadStarted = true;
And SyncToDatabase could look like:
async Task SyncToDatabase(CancellationToken token)
{
while (true)
{
token.ThrowIfCancellationRequested();
DatabaseSyncer dbSyncer = new DatabaseSyncer();
await dbSyncer.DeserializeAndUpdate();
await Task.Delay(10, token); // after elapsed time re-run above code
}
}
Check this answer for more details on how to cancel and restart a task.

I may have misunderstood the question, but the execution of SynchToDatabase() will wait on the completion of await dbSyncer.DeserializeAndUpdaet() (due to the await keyword, go figure ;)) before executing the continuation, which will then delay for 10 ms (do you want 10ms or did you mean 10 seconds? Parameter for Task.Delay is in milliseconds), then loop back to re-execute the DbSyncer method, so I don't see the problem.

Proper way to implement a never ending task. (Timers vs Task)

So, my app needs to perform an action almost continuously (with a pause of 10 seconds or so between each run) for as long as the app is running or a cancellation is requested. The work it needs to do has the possibility of taking up to 30 seconds.
Is it better to use a System.Timers.Timer and use AutoReset to make sure it doesn't perform the action before the previous "tick" has completed.
Or should I use a general Task in LongRunning mode with a cancellation token, and have a regular infinite while loop inside it calling the action doing the work with a 10 second Thread.Sleep between calls? As for the async/await model, I'm not sure it would be appropriate here as I don't have any return values from the work.
CancellationTokenSource wtoken;
Task task;
void StopWork()
{
wtoken.Cancel();
try
{
task.Wait();
} catch(AggregateException) { }
}
void StartWork()
{
wtoken = new CancellationTokenSource();
task = Task.Factory.StartNew(() =>
{
while (true)
{
wtoken.Token.ThrowIfCancellationRequested();
DoWork();
Thread.Sleep(10000);
}
}, wtoken, TaskCreationOptions.LongRunning);
}
void DoWork()
{
// Some work that takes up to 30 seconds but isn't returning anything.
}
or just use a simple timer while using its AutoReset property, and call .Stop() to cancel it?

I'd use TPL Dataflow for this (since you're using .NET 4.5 and it uses Task internally). You can easily create an ActionBlock<TInput> which posts items to itself after it's processed it's action and waited an appropriate amount of time.
First, create a factory that will create your never-ending task:
ITargetBlock<DateTimeOffset> CreateNeverEndingTask(
Action<DateTimeOffset> action, CancellationToken cancellationToken)
{
// Validate parameters.
if (action == null) throw new ArgumentNullException("action");
// Declare the block variable, it needs to be captured.
ActionBlock<DateTimeOffset> block = null;
// Create the block, it will call itself, so
// you need to separate the declaration and
// the assignment.
// Async so you can wait easily when the
// delay comes.
block = new ActionBlock<DateTimeOffset>(async now => {
// Perform the action.
action(now);
// Wait.
await Task.Delay(TimeSpan.FromSeconds(10), cancellationToken).
// Doing this here because synchronization context more than
// likely *doesn't* need to be captured for the continuation
// here. As a matter of fact, that would be downright
// dangerous.
ConfigureAwait(false);
// Post the action back to the block.
block.Post(DateTimeOffset.Now);
}, new ExecutionDataflowBlockOptions {
CancellationToken = cancellationToken
});
// Return the block.
return block;
}
I've chosen the ActionBlock<TInput> to take a DateTimeOffset structure; you have to pass a type parameter, and it might as well pass some useful state (you can change the nature of the state, if you want).
Also, note that the ActionBlock<TInput> by default processes only one item at a time, so you're guaranteed that only one action will be processed (meaning, you won't have to deal with reentrancy when it calls the Post extension method back on itself).
I've also passed the CancellationToken structure to both the constructor of the ActionBlock<TInput> and to the Task.Delay method call; if the process is cancelled, the cancellation will take place at the first possible opportunity.
From there, it's an easy refactoring of your code to store the ITargetBlock<DateTimeoffset> interface implemented by ActionBlock<TInput> (this is the higher-level abstraction representing blocks that are consumers, and you want to be able to trigger the consumption through a call to the Post extension method):
CancellationTokenSource wtoken;
ActionBlock<DateTimeOffset> task;
Your StartWork method:
void StartWork()
{
// Create the token source.
wtoken = new CancellationTokenSource();
// Set the task.
task = CreateNeverEndingTask(now => DoWork(), wtoken.Token);
// Start the task. Post the time.
task.Post(DateTimeOffset.Now);
}
And then your StopWork method:
void StopWork()
{
// CancellationTokenSource implements IDisposable.
using (wtoken)
{
// Cancel. This will cancel the task.
wtoken.Cancel();
}
// Set everything to null, since the references
// are on the class level and keeping them around
// is holding onto invalid state.
wtoken = null;
task = null;
}
Why would you want to use TPL Dataflow here? A few reasons:
Separation of concerns
The CreateNeverEndingTask method is now a factory that creates your "service" so to speak. You control when it starts and stops, and it's completely self-contained. You don't have to interweave state control of the timer with other aspects of your code. You simply create the block, start it, and stop it when you're done.
More efficient use of threads/tasks/resources
The default scheduler for the blocks in TPL data flow is the same for a Task, which is the thread pool. By using the ActionBlock<TInput> to process your action, as well as a call to Task.Delay, you're yielding control of the thread that you were using when you're not actually doing anything. Granted, this actually leads to some overhead when you spawn up the new Task that will process the continuation, but that should be small, considering you aren't processing this in a tight loop (you're waiting ten seconds between invocations).
If the DoWork function actually can be made awaitable (namely, in that it returns a Task), then you can (possibly) optimize this even more by tweaking the factory method above to take a Func<DateTimeOffset, CancellationToken, Task> instead of an Action<DateTimeOffset>, like so:
ITargetBlock<DateTimeOffset> CreateNeverEndingTask(
Func<DateTimeOffset, CancellationToken, Task> action,
CancellationToken cancellationToken)
{
// Validate parameters.
if (action == null) throw new ArgumentNullException("action");
// Declare the block variable, it needs to be captured.
ActionBlock<DateTimeOffset> block = null;
// Create the block, it will call itself, so
// you need to separate the declaration and
// the assignment.
// Async so you can wait easily when the
// delay comes.
block = new ActionBlock<DateTimeOffset>(async now => {
// Perform the action. Wait on the result.
await action(now, cancellationToken).
// Doing this here because synchronization context more than
// likely *doesn't* need to be captured for the continuation
// here. As a matter of fact, that would be downright
// dangerous.
ConfigureAwait(false);
// Wait.
await Task.Delay(TimeSpan.FromSeconds(10), cancellationToken).
// Same as above.
ConfigureAwait(false);
// Post the action back to the block.
block.Post(DateTimeOffset.Now);
}, new ExecutionDataflowBlockOptions {
CancellationToken = cancellationToken
});
// Return the block.
return block;
}
Of course, it would be good practice to weave the CancellationToken through to your method (if it accepts one), which is done here.
That means you would then have a DoWorkAsync method with the following signature:
Task DoWorkAsync(CancellationToken cancellationToken);
You'd have to change (only slightly, and you're not bleeding out separation of concerns here) the StartWork method to account for the new signature passed to the CreateNeverEndingTask method, like so:
void StartWork()
{
// Create the token source.
wtoken = new CancellationTokenSource();
// Set the task.
task = CreateNeverEndingTask((now, ct) => DoWorkAsync(ct), wtoken.Token);
// Start the task. Post the time.
task.Post(DateTimeOffset.Now, wtoken.Token);
}

I find the new Task-based interface to be very simple for doing things like this - even easier than using the Timer class.
There are some small adjustments you can make to your example. Instead of:
task = Task.Factory.StartNew(() =>
{
while (true)
{
wtoken.Token.ThrowIfCancellationRequested();
DoWork();
Thread.Sleep(10000);
}
}, wtoken, TaskCreationOptions.LongRunning);
You can do this:
task = Task.Run(async () => // <- marked async
{
while (true)
{
DoWork();
await Task.Delay(10000, wtoken.Token); // <- await with cancellation
}
}, wtoken.Token);
This way the cancellation will happen instantaneously if inside the Task.Delay, rather than having to wait for the Thread.Sleep to finish.
Also, using Task.Delay over Thread.Sleep means you aren't tying up a thread doing nothing for the duration of the sleep.
If you're able, you can also make DoWork() accept a cancellation token, and the cancellation will be much more responsive.

Here is what I came up with:
Inherit from NeverEndingTask and override the ExecutionCore method with the work you want to do.
Changing ExecutionLoopDelayMs allows you to adjust the time between loops e.g. if you wanted to use a backoff algorithm.
Start/Stop provide a synchronous interface to start/stop task.
LongRunning means you will get one dedicated thread per NeverEndingTask.
This class does not allocate memory in a loop unlike the ActionBlock based solution above.
The code below is sketch, not necessarily production code :)
:
public abstract class NeverEndingTask
{
// Using a CTS allows NeverEndingTask to "cancel itself"
private readonly CancellationTokenSource _cts = new CancellationTokenSource();
protected NeverEndingTask()
{
TheNeverEndingTask = new Task(
() =>
{
// Wait to see if we get cancelled...
while (!_cts.Token.WaitHandle.WaitOne(ExecutionLoopDelayMs))
{
// Otherwise execute our code...
ExecutionCore(_cts.Token);
}
// If we were cancelled, use the idiomatic way to terminate task
_cts.Token.ThrowIfCancellationRequested();
},
_cts.Token,
TaskCreationOptions.DenyChildAttach | TaskCreationOptions.LongRunning);
// Do not forget to observe faulted tasks - for NeverEndingTask faults are probably never desirable
TheNeverEndingTask.ContinueWith(x =>
{
Trace.TraceError(x.Exception.InnerException.Message);
// Log/Fire Events etc.
}, TaskContinuationOptions.OnlyOnFaulted);
}
protected readonly int ExecutionLoopDelayMs = 0;
protected Task TheNeverEndingTask;
public void Start()
{
// Should throw if you try to start twice...
TheNeverEndingTask.Start();
}
protected abstract void ExecutionCore(CancellationToken cancellationToken);
public void Stop()
{
// This code should be reentrant...
_cts.Cancel();
TheNeverEndingTask.Wait();
}
}

Queue a thread in .net

I have 2 functions that needs to be executed one after the other. In this function, async calls are made. How do I go about executing the second function after the async call is completed?
For eg.
public void main()
{
executeFn("1");
executeFn("2"); //I want this to be executed after 1 has finished.
}
private bool executeFn(string someval)
{
runSomeAsyncCode(); //This is some async uploading function that is yet to be defined.
}

You can use Thread.Join.
But then I do not see the point of async execution of those 2 functions as they become sequential.

Let runSomeAsyncCode() return an IAsyncResult and implement the BeginX EndX methods similar to the CLR Asynchronous Programming Model. Use the EndX method to wait for the code to finish executing.

Your async method you're calling must have something to notify the caller when it's completed am I correct? (otherwise it would be just execute and forget, which is unlikely) If so, you simply have to wait for the notification to come up and execute the second method.

try this:
public void main()
{
executeFn("1");
executeFn("2");
}
List<string> QueuedCalls = new List<string>(); // contains the queued items
bool isRunning = false; // indicates if there is an async operation running
private bool executeFn(string someval)
{
if(isRunning) { QueuedCalls.Add(someval); return; } // if there is an operation running, queue the call
else { isRunning = true; } // if there is not an operation running, then update the isRunning property and run the code
runSomeAsyncCode(); //undefined async operation here<-
isRunning = false; //get here when the async is completed, (updates the app telling it this operation is done)
if(QueuedCalls.Count != 0)//check if there is anything in the queue
{
//there is something in the queue, so remove it from the queue and execute it.
string val = QueuedCalls[0];
QueuedCalls.RemoveAt(0);
executeFn(val);
}
}
this way will not block any threads, and will simply execute the queued call when the first finnishs,which is what i believe you want! happy coding! now id recommend running the last section, at where it sets the isRunning to false, inside your async operation, or trigger it with an event or something, the only catch is that peice of code has to be executed when your async operation is completed, so however you want to do that is up to you

You can consider using Generic delegates execute the first method async then in the call back execute the other method async. If you are really worried executing them sync with respect to each other.

One simple way is to use a custom threadpool
http://www.codeplex.com/smartthreadpool
You can instantiate a separate threadpool, Set the threadpool size to 1, and queue the workers