For .NET Core, AsyncLocal is the replacement for CallContext. However, it is unclear how "safe" it is to use in ASP.NET Core.
In ASP.NET 4 (MVC 5) and earlier, the thread-agility model of ASP.NET made CallContext unstable. Thus in ASP.NET the only safe way to achieve the behavior of a per-request logical context, was to use HttpContext.Current.Items. Under the covers, HttpContext.Current.Items is implemented with CallContext, but it is done in a way that is safe for ASP.NET.
In contrast, in the context of OWIN/Katana Web API, the thread-agility model was not an issue. I was able to use CallContext safely, after careful considerations of how correctly to dispose it.
But now I'm dealing with ASP.NET Core. I would like to use the following middleware:
public class MultiTenancyMiddleware
{
private readonly RequestDelegate next;
static int random;
private static AsyncLocal<string> tenant = new AsyncLocal<string>();
//This is the new form of "CallContext".
public static AsyncLocal<string> Tenant
{
get { return tenant; }
private set { tenant = value; }
}
//This is the new verion of [ThreadStatic].
public static ThreadLocal<string> LocalTenant;
public MultiTenancyMiddleware(RequestDelegate next)
{
this.next = next;
}
public async Task Invoke(HttpContext context)
{
//Just some garbage test value...
Tenant.Value = context.Request.Path + random++;
await next.Invoke(context);
//using (LocalTenant = new AsyncLocal<string>()) {
// Tenant.Value = context.Request.Path + random++;
// await next.Invoke(context);
//}
}
}
So far, the above code seems to be working just fine. But there is at least one red flag. In the past, it was critical to ensure that CallContext was treated like a resource that must be freed after each invocation.
Now I see there is no self-evident way to "clean up" AsyncLocal.
I included code, commented out, showing how ThreadLocal<T> works. It is IDisposable, and so it has an obvious clean-up mechanism. In contrast, the AsyncLocal is not IDisposable. This is unnerving.
Is this because AsyncLocal is not yet in release-candidate condition? Or is this because it is truly no longer necessary to perform cleanup?
And even if AsyncLocal is being used properly in my above example, are there any kinds of old-school "thread agility" issues in ASP.NET Core that are going to make this middleware unworkable?
Special Note
For those unfamiliar with the issues CallContext has within ASP.NET apps, in this SO post, Jon Skeet references an in-depth discussion about the problem (which in turn references commentary from Scott Hanselman). This "problem" is not a bug - it is just a circumstance that must be carefully accounted for.
Furthermore, I can personally attest to this unfortunate behavior. When I build ASP.NET applications, I normally include load-tests as part of my automation test infrastructure. It is during load tests that I can witness CallContext become unstable (where perhaps 2% to 4% of requests show CallContext being corrupted. I have also seen cases where a Web API GET has stable CallContext behavior, but the POST operations are all unstable. The only way to achieve total stability is to rely on HttpContext.Current.Items.
However, in the case of ASP.NET Core, I cannot rely on HttpContext.Items...there is no such static access point. I'm also not yet able to create load tests for the .NET Core apps I'm tinkering with, which is partly why I've not answered this question for myself. :)
Again: Please understand that the "instability" and "problem" I'm discussing is not a bug at all. CallContext is not somehow flawed. The issue is simply a consequence of the thread dispatch model employed by ASP.NET. The solution is simply to know the issue exists, and to code accordingly (e.g. use HttpContext.Current.Items instead of CallContext, when inside an ASP.NET app).
My goal with this question is to understand how this dynamic applies (or does not) in ASP.NET Core, so that I don't accidentally build unstable code when using the new AsyncLocal construct.
I'm just looking into the source code of the ExecutionContext class for CoreClr:
https://github.com/dotnet/coreclr/blob/775003a4c72f0acc37eab84628fcef541533ba4e/src/mscorlib/src/System/Threading/ExecutionContext.cs
Base on my understanding of the code, the async local values are fields/variables of each ExecutionContext instance. They are not based on ThreadLocal or any thread specific persisted data store.
To verify this, in my testing with thread pool threads, an instance left in async local value is not accessible when the same thread pool thread is reused, and the "left" instance's destructor for cleaning up itself got called on next GC cycle, meaning the instance is GCed as expected.
Adding my two cents if someone lands on this page (like I did) after googling if AsyncLocal is "safe" in ASP.NET classic (non Core) application (some commenters have been asking this, and also I see a deleted answer asking about the same).
I wrote a small test that simulates asp.net's ThreadPool behavior
AsyncLocal is always cleared between requests even if thread pool re-uses an existing thread. So it is "safe" in that regard, no data will be leaked to another thread.
However, AsyncLocal can be cleared even within the same context (for example between code that runs in global.asax and the code that runs in controller). Because MVC-methods sometimes runs on a separate thread from non-MVC code, see this question for example: asp.net mvc 4, thread changed by model binding?
Using ThreadLocal is not safe b/c it preserves the value after the thread from Thread Pool is re-used. Never use ThreadLocal in web-applications. I know the question is not about ThreadLocal I'm just adding this warning to whoever considering using it, sorry.
Tested under ASP.NET MVC 5 .NET 4.7.2.
Overall, AsyncLocal seems like a perfect alternative to short-time caching stuff in HttpContext.Current in cases where you can't access the latter directly. You might end up re-calculating the cached value a bit more often though, but that's not a big problem.
Related
I've imported Microsoft.VisualStudio.Threading into .Net Core Web App. I did this specifically to make use of AsyncLazy<T>.
I wanted to make sure I did this right, so I imported the appropriate Analyzers.
The warnings and the documentation clearly state that a JoinableTaskFactory should be injected into my implementation.
My question is, how should I instantiate that JoinableTaskFactory in the configuration of my .Net Core Web App?
Is it as simple as
public void ConfigureServices(IServiceCollection services)
{
// ...
services.AddSingleton(new JoinableTaskFactory());
// ...
}
or, is that all wrong?
Yes, you can use the Microsoft.VisualStudio.Threading library in ASP.NET Core applications. But while JoinableTaskFactory would "work" in such an app, it's a pretty good sign you're doing something wrong.
The analyzers are always appropriate of course, and the warnings you saw were perhaps pointing out that you shouldn't call Task.Result, Task.Wait(), etc. These synchronously block threads and can severely reduce the scalability of your web app (or any app for that matter). You should use await instead wherever possible.
JoinableTaskFactory is there to step in when you can't use await but you still need to call async code. Using JTF.Run will still block the calling thread, but it will do so in a way that avoids deadlocks when you have a single threaded SynchronizationContext in the application. I don't think ASP.NET Core has such a thing, so that's not a concern. JTF.Run is still more efficient than Task.Wait since it can reuse the original thread for continuations instead of a second thread.
If you do choose to use JTF in your web app, if ASP.NET Core doesn't use a single-threaded SynchronizationContext then you can create and share a single JTF instance with the entire app. But if it does have a single-threaded SynchronizationContext, it will be one per web request, which means you need to create a new JoinableTaskContext for each request since these are affinitized to one SynchronizationContext. You always get your JTF instance from a JoinableTaskContext instance.
This page of the vs-threading docs says
The ThreadHelper.JoinableTaskFactory property only works for code running in the VS process. If your code is running out of proc (in the vstest.executionengine.exe runner for instance) it won't work.
So, as the name of the package, Microsoft.VisualStudio.Threading, suggests, it's intended to be used for Visual Studio extensions. The code that you linked for its implementation of AsyncLazy uses JoinableTaskFactory, so it's probably not suitable outside of Visual Studio. I certainly would never use it outside of a VS extension that needs to switch to the UI thread.
Steven Cleary's AsyncEx library has an AsyncLazy, and the wiki page links to this Steven Toub blog post titled AsyncLazy. The blog post points out that the lazy semantics doesn't really add much over what Task<T> provides, although since the value factory might do a lot of work before reaching an await his sample runs it on the thread pool and unwraps the Task<Task<T>>.
edit: As pointed out in the comments, my quote from the docs is somewhat out of context. However, the vs-threading library is about avoiding deadlocks when using async with a synchronisation context (primarily a GUI). ASP.NET Core, what the author of the question is using, does not have a synchronisation context, hence doesn't need to worry about deadlocking the main thread specifically. While using the vs-threading library probably won't cause problems, as my quote claimed, I still don't think it's a good fit for anything without a synchronisation context and there are much better alternatives, like using Task<T> directly and not need any AsyncLazy implementation.
I know that a Task may continue the execution on another thread, proven by this code.
public async Task Test()
{
var id1 = System.Environment.CurrentManagedThreadId;
await Task.Delay(1000);
var id2 = System.Environment.CurrentManagedThreadId;
Console.Write($"First {id1} then {id2}");
}
I expect the framework to handle memory barrieres, so that id1 is visible when accessed in the last statement.
But what if you are using some kind of framework e.g. NHibernate where the ISession is not thread safe. Frameworks may even check that the thread ID is still the same. How does this mix?
When building website, I'll to use a IOC container with a nested container per request, but when then thread could change within the same request, wouldn't this lead to all kind of problems? ThreadStatic would not work as expected
Not being thread safe usually means don't use it from multiple threads at the same time, rather than don't use it from one thread and then from another thread later.
I don't know about NHibernate specifically, but if it's a problem, consider using EF Core.
In general with async, don't use variables attached to specific threads, like thread local or thread static.
However, local variables, class members, logical call contexts, HttpContext if you're on asp.net etc continue to work. If you do have something that will be lost after an await, you can usually just save it to a local variable first.
The default ConfigureAwait(true) also restores some context on the continuation, but it can be hard to know what is restored and what isn't. Later versions of .net do a better job of this, like restoring the culture so that resources keep working after an await.
There's a good article by Stephen Toub here:
https://blogs.msdn.microsoft.com/pfxteam/2012/06/15/executioncontext-vs-synchronizationcontext/
Using async/await as shown here http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html, one of the good practices is using ConfigureAwait(false) so the method return does not have to come back to the request context. What are the potential consequences of using this? Asked another way, when would this not be recommended?
when would this not be recommended?
Your method must return to the same context if there's code further down in the method that depends on that context. If the remainder of that method does not require a particular context, then it's a good practice to use ConfigureAwait(false).
There are two primary examples: UI code and ASP.NET code.
UI code must run on the UI thread; this includes most UI widget access, and I extend the definition of "UI code" to include my ViewModels as well (there are some situations in WPF where you can get away with updating the UI from a background thread, but that's not true for all MVVM platforms). So, if your method ends with a textBox1.Text = "" or a myViewModel.MyObservableCollection.Add(4), then it has to return to the UI thread before it can execute that code.
ASP.NET code must run in an ASP.NET request context; this includes any code that depends on HttpContext.Current (and many System.Web APIs implicitly assume an ASP.NET request context). So, if your method ends with HttpContext.Current.Items..., then it has to return to the ASP.NET request context before it can execute that code. (Side note: in ASP.NET vNext on .NET 4.6 and higher, the ASP.NET request context is actually going away).
In practice, what this means is that most library code should use ConfigureAwait(false), since well-written library code does not have a dependency on a particular UI framework or System.Web. Similarly, most application code should not use ConfigureAwait(false), since it has to update the UI / send an HTTP response.
This question has been triggered by EF Data Context - Async/Await & Multithreading. I've answered that one, but haven't provided any ultimate solution.
The original problem is that there are a lot of useful .NET APIs out there (like Microsoft Entity Framework's DbContext), which provide asynchronous methods designed to be used with await, yet they are documented as not thread-safe. That makes them great for use in desktop UI apps, but not for server-side apps. [EDITED] This might not actually apply to DbContext, here is Microsoft's statement on EF6 thread safety, judge for yourself. [/EDITED]
There are also some established code patterns falling into the same category, like calling a WCF service proxy with OperationContextScope (asked here and here), e.g.:
using (var docClient = CreateDocumentServiceClient())
using (new OperationContextScope(docClient.InnerChannel))
{
return await docClient.GetDocumentAsync(docId);
}
This may fail because OperationContextScope uses thread local storage in its implementation.
The source of the problem is AspNetSynchronizationContext which is used in asynchronous ASP.NET pages to fulfill more HTTP requests with less threads from ASP.NET thread pool. With AspNetSynchronizationContext, an await continuation can be queued on a different thread from the one which initiated the async operation, while the original thread is released to the pool and can be used to serve another HTTP request. This substantially improves the server-side code scalability. The mechanism is described in great details in It's All About the SynchronizationContext, a must-read. So, while there is no concurrent API access involved, a potential thread switch still prevents us from using the aforementioned APIs.
I've been thinking about how to solve this without sacrificing the scalability. Apparently, the only way to have those APIs back is to maintain thread affinity for the scope of the async calls potentially affected by a thread switch.
Let's say we have such thread affinity. Most of those calls are IO-bound anyway (There Is No Thread). While an async task is pending, the thread it's been originated on can be used to serve a continuation of another similar task, which result is already available. Thus, it shouldn't hurt scalability too much. This approach is nothing new, in fact, a similar single-threaded model is successfully used by Node.js. IMO, this is one of those things that make Node.js so popular.
I don't see why this approach could not be used in ASP.NET context. A custom task scheduler (let's call it ThreadAffinityTaskScheduler) might maintain a separate pool of "affinity apartment" threads, to improve scalability even further. Once the task has been queued to one of those "apartment" threads, all await continuations inside the task will be taking place on the very same thread.
Here's how a non-thread-safe API from the linked question might be used with such ThreadAffinityTaskScheduler:
// create a global instance of ThreadAffinityTaskScheduler - per web app
public static class GlobalState
{
public static ThreadAffinityTaskScheduler TaScheduler { get; private set; }
public static GlobalState
{
GlobalState.TaScheduler = new ThreadAffinityTaskScheduler(
numberOfThreads: 10);
}
}
// ...
// run a task which uses non-thread-safe APIs
var result = await GlobalState.TaScheduler.Run(() =>
{
using (var dataContext = new DataContext())
{
var something = await dataContext.someEntities.FirstOrDefaultAsync(e => e.Id == 1);
var morething = await dataContext.someEntities.FirstOrDefaultAsync(e => e.Id == 2);
// ...
// transform "something" and "morething" into thread-safe objects and return the result
return data;
}
}, CancellationToken.None);
I went ahead and implemented ThreadAffinityTaskScheduler as a proof of concept, based on the Stephen Toub's excellent StaTaskScheduler. The pool threads maintained by ThreadAffinityTaskScheduler are not STA thread in the classic COM sense, but they do implement thread affinity for await continuations (SingleThreadSynchronizationContext is responsible for that).
So far, I've tested this code as console app and it appears to work as designed. I haven't tested it inside an ASP.NET page yet. I don't have a lot of production ASP.NET development experience, so my questions are:
Does it make sense to use this approach over simple synchronous invocation of non-thread-safe APIs in ASP.NET (the main goal is to avoid sacrificing scalability)?
Is there alternative approaches, besides using synchronous API invocations or avoiding those APis at all?
Has anyone used something similar in ASP.NET MVC or Web API projects and is ready to share his/her experience?
Any advice on how to stress-test and profile this approach with ASP.NET would be
appreciated.
Entity Framework will (should) handle thread jumps across await points just fine; if it doesn't, then that's a bug in EF. OTOH, OperationContextScope is based on TLS and is not await-safe.
1. Synchronous APIs maintain your ASP.NET context; this includes things such as user identity and culture that are often important during processing. Also, a number of ASP.NET APIs assume they are running on an actual ASP.NET context (I don't mean just using HttpContext.Current; I mean actually assuming that SynchronizationContext.Current is an instance of AspNetSynchronizationContext).
2-3. I have used my own single-threaded context nested directly within the ASP.NET context, in attempts to get async MVC child actions working without having to duplicate code. However, not only do you lose the scalability benefits (for that part of the request, at least), you also run into the ASP.NET APIs assuming that they're running on an ASP.NET context.
So, I have never used this approach in production. I just end up using the synchronous APIs when necessary.
You should not intertwine multithreading with asynchrony. The problem with an object not being thread-safe is when a single instance (or static) is accessed by multiple threads at the same time. With async calls the context is possibly accessed from a different thread in the continuation, but never at the same time (when not shared across multiple requests, but that isn't good in the first place).
So I'm left wondering how exactly asp.net is able to scope a static property, when (to my knowledge) asp.net is multi-threaded.
One theory goes that the ASP.NET guys maintain a different appdomain for every request ... but that doesn't seem feasible.
Another theory goes that the .Current method looks at the current Thread, and then uses that to look up the http context in some hashtable (or other static storage mechanism).
Either way, it's a technique that seems really useful ... I'd like to utilize it, but definitely don't want to be debugging shared state bugs :-/
It isn't an AppDomain per-request. If you want to use thread-specific state, try:
[ThreadStatic]
private static int foo;
public static int Foo {get {return foo;} set {foo = value;}}
Each thread now gets its own value of Foo (or rather: 'foo').
This is not to be used lightly - it does have costs, but is a valid way of sharing state on a per-thread basis. I've used this once, maybe twice - and I've written a lot of C#. Don't over-use it...
In particular, watch out for initialization issues (i.e. forgetting to do it), and remember to clean up after yourself etc. And be very careful if you use any async code, as any callbacks/workers/etc will have different state.
What Marc says is the easiest most likely for what you are after, however ASP.NET is actually somewhat more complicated than what say ThreadStatic does, because single requests actually can be processed by multiple threads.. what I believe happens with ASP.NET is that the executing thread explicitely is told to switch context, of course the hosting environment is scheduling the threads and it has context of which httpcontext needs executing, so it finds a thread, tells the thread which context it should run in.. then sends it off on its way.
So the solution really isn't all that pretty sadly, where as threadstatic is much simpler and probably suits needs 95% of the time.