After reading Stephen Cleary's article on async and asp.net, it is quite clear that using Async in web application is a big win on scalability (primarily due to non-blocking threads that are free to serve up more requests).
Although I am a little confused as to how does the Task object returned by the async operation remain in scope in an Aysnc WebServer implementation (lets say iis or self-hosted WebApi)
and not get collected.
for example,
If we have the following method in a low level Webserver implementation,
// some method that handles HttpListeners BeginContextAsync callback,
public void SomeHttpListenerCallback(HttpListenerContext context)
{
// Immediately set up the next begin context
// then handle the request
// forgive me if this is incorrect but this is my understanding of how an async request would be handled
// please feel free to point out the flaws
var task = messageHanlder1.SendAsync(context.Request, null);
task.ContinueWith(t => SendResponse(t));
}
public class MessageHandler1 : DelegatingHandler
{
protected async override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
Debug.WriteLine("Process request");
// Call the inner handler.
var response = await base.SendAsync(request, cancellationToken);
Debug.WriteLine("Process response");
return response;
}
}
SendResponse being some method that sends the response to clients socket.
As you might have noticed, task object drops out of scope in SomeHttpListenerCallback, since no other thread (theoretically) has any reference to the task object, wouldnt it be marked for collection in next GC cycle?
I have read Jeffrey Ritcher's explanation (Clr via C#) of how compiler converts an aysnc method into a method with a state machine, however I fail to understand what happens to the task object if no thread waits on it.
And if a thread waits on it, wouldnt we have 1 blocked thread per request?
Any help to point me in the right direction would be greatly appreciated.
First of all, if you have a Task and nothing references it then it will be collected by the GC.
However, that's almost never the case. There are 2 types of tasks, Promise tasks and Delegate tasks.
Promise tasks (asynchronous) are mostly created by TaskCompletionSource or the compiler for an async method. In both cases someone holds a reference to the task so it will be able to complete it.
Delegate tasks (synchronous) however are referenced by the thread executing the delegate in them.
In your case task is referenced by the state machine behind SendAsync until it completes while task is referencing the continuation Task. When it completes the continuation is scheduled and so is referenced by the TaskScheduler and the thread executing SendResponse. When the operation completes the task will no longer be referenced and could be eventually GCed.
You can see an example in the implementation of Task.Delay where the Task (actually DelayPromise which inherits from Task) is being referenced by the System.Threading.Timer used to complete the Task.
Because some arguments don't actually need a timer you can see a distinction in memory usage between this:
static void Main()
{
while (true)
{
Task.Delay(int.MaxValue);
}
}
And this:
static void Main()
{
while (true)
{
Task.Delay(-1); // No Timer as the delay is infinite.
}
}
For every async operation that you do there is some "native" operation behind it - a socket read or a file IO and such. These are native CLR operations or PInvoke calls. When starting such an operation the framework registers a callback to be called on completion. This callback is being used to complete tasks. That callback is being kept alive by "the system". For example when you say myFileStream.BeginRead(myCallback) then the callback will be called even if it is otherwise unreferenced.
Speaking in a simplified way the native operation itself keeps everything alive that must be worked on when completion happens.
This is not a hard rule but an IO framework would be pretty useless if it called your callback only sometimes.
There is nothing that prevents any Task from being collected. For example var dummy = new TaskCompletionSource<object>().Task; is eligible for collection immediately.
Related
My company has a Nuget Package they wrote that can do various common tasks easily for you. One of which is making HTTP requests. Normally I always make my HTTP requests asynchronous, however in this Nuget package is the following code:
protected T GetRequest<T>(string requestUri)
{
// Call the async method within a task to run it synchronously
return Task.Run(() => GetRequestAsync<T>(requestUri)).Result;
}
Which calls this function:
protected async Task<T> GetRequestAsync<T>(string requestUri)
{
// Set up the uri and the client
string uri = ParseUri(requestUri);
var client = ConfigureClient();
// Call the web api
var response = await client.GetAsync(uri);
// Process the response
return await ProcessResponse<T>(response);
}
My question is, is this code really running synchronously by just wrapping the GetRequestAsync(requestUri) inside a Task.Run and calling .Result on the returned task? This seems like a deadlock waiting to happen, and we are seeing issues in areas of our app that utilize this function when running at higher loads.
Accessing Task.Result will block the current thread until the Task is complete, so it is not asynchronous.
As for deadlocks, this shouldn't happen, as Task.Run uses another thread for GetRequestAsync, which is not being blocked by the call to Result.
The reason that will not cause a deadlock is because Task.Run will push the delegate to be executed in a threadpool thread. Threadpool threads has no SynchronizationContext therefore no deadlock happens as there is no sync context to lock on between the async method GetRequestAsync and the caller. Same as you could have called .Result directly on the actual async method as well within Task.Run() block and that would not have caused a deadlock either.
Very inefficient though as you freeze 1 thread ie. 1 Core in the CPU do nothing but wait for the async method and I/O calls within it to complete. That s probably why you see a freeze in high load scenarios.
If you have a sync/async deadlock issue due to capturing sync context and blocking on async call, the deadlock will happen irrespective of load on a single call..
This won't cause a deadlock. But it's surely a resources wasting as one of the threads may be blocked.
The deadlock though may be possible if GetRequest looked like this:
protected T GetRequest<T>(string requestUri)
{
var task = GetRequestAsync<T>(requestUri);
return task.Result;
// or
// return GetRequestAsync<T>(requestUri).Result;
}
In example above you can see that I call GetRequestAsync within the current thread. Let's give the thread a number 0. Consider this line from the GetRequestAsync - await client.GetAsync(uri). .GetAsync is executed by a thread 1. After .GetAsync is done, default task scheduler is trying to return execution flow to the thread that executed the line - to the thread 0. But the thread that executed the line (0) is blocked right now as after we executed GetRequestAsync(), we are blocking it (thread 0) with task.Result. Hence our thread 0 remains blocked as it cannot proceed with execution of GetRequestAsync after await client.GetAsync(uri) is done nor it can give us the Result.
It's a pretty common mistake and I suppose you meant this one when asked about the deadlock. Your code is not causing it because you are executing GetRequestAsync from within another thread.
I was reading a chapter about the await and async keywords in my C# book.
It was mainly explaining these kinds of method calls, where the caller uses the await keyword to wait for the called method to finish.
In this simple example I see no advantage but more importantly no difference in these 3 calls. Can anyone explain what difference it makes to the flow of the application? Is it only useful when the calling thread is the main GUI thread?
static async Task Main(string[] args)
{
WriteText();
await WriteTextAsync();
WriteTextAsync().Wait();
}
static void WriteText()
{
Thread.Sleep(3_000);
Console.WriteLine("Hello");
}
static async Task WriteTextAsync()
{
await Task.Run(() =>
{
Thread.Sleep(3_000);
Console.WriteLine("Hello");
});
}
Ps: If the calling thread of the method is waiting for the method to finish anyway it might as well be a normal call?
As my understanding about your question is
If the program waits for the response in await WriteTextAsync() line then what will be the benifit?
For client applications, such as Windows Store, Windows desktop and Windows Phone apps, the primary benefit of async is responsiveness. These types of apps use async chiefly to keep the UI responsive. For server applications, the primary benefit of async is scalability.
I will try to explain from the web-app point of view.
Suppose you have a web application depends on external resources like database call, when a client initiates a request ASP.NET takes one of its thread pool threads and assigns it to that request. Because it’s written synchronously, the request handler will call that external resource synchronously. This blocks the request thread until the call to the external resource returns. Figure 1 illustrates a thread pool with two threads, one of which is blocked waiting for an external resource.
Figure 1 Waiting Synchronously for an External Resource
Now if third client requests at the same time then there is no thread in thread pool available to assign the third request.
In asynchronous call, thread will not be stuck rather will be released and comes back to the thread pool which will facilitates to serve the third call.
When request server activities ends linke database call ends then SynchronizationContext resumes that call and returns repose to client.
Bellow image in simple analogy of aync call
There is lot of things happens under the hood. I wrote this maily from Async Programming : Introduction to Async/Await on ASP.NET and my understanding. It is highly recommended to have clear understanding before using async-wait.
I'll be referring to:
//Call 1
WriteText();
//Call 2
await WriteTextAsync();
//Call 3
WriteTextAsync().Wait();
The first call doesn't have any problem, if what you want to do is a synchronous wait. In a Console application, this is quite normal.
The problem arises in programs with a UI or those that require the best use of CPU resources, the most common case being web applications.
Call 2, using await performs an asynchronous wait for the result of WriteTextAsync. In its own, that's fine and what's considered normal. However, WriteTextAsync is a very good example of something you should never do:
static async Task WriteTextAsync()
{
// Let's create a Thread
await Task.Run(() =>
{
// just to block it completely, having it do nothing useful
Thread.Sleep(3_000);
Console.WriteLine("Hello");
});
}
Rather, the author should have used:
static async Task WriteTextAsync()
{
// Let's *not* create a new thread
await Task.Delay(3_000);
Console.WriteLine("Hello");
}
Maybe they were to point this out further down the line, but you didn't give us the book name to know this.
Call number 3 is what you would have to do when the calling method cannot be an async one and you have to call an async method, so:
// Think that for some reason you cannot change the signature,
// like in the case of an interface, and an async void would make your code
// never complete correctly
static void Main(string[] args)
{
//Call 3
WriteTextAsync().Wait();
}
Overall, I would suggest you to find a better book. Examples are easier to understand when asynchronous code is actually required.
When you say WriteText(), WriteText() will block your current thread until it completes, as it is synchronous.
When you say await WriteTextAsync(), you will spawn a new thread and you will not block computations that do not depend on the result of WriteTextAsync().
EDIT: According to Microsoft Docs When you say await WriteTextAsync(), the compiler schedules the rest of Main() for execution after WriteTextAsync() completes. Then the control is supposed to be returned to the caller of the async method. But who is the caller of Main()? As it turns out, there is no async Main() - it is actually the same synchronous Main() - and it's just syntactical sugar for not writing .Wait() inside it. So in this case this call is equivalent to WriteTextAsync().Wait()!
Finally, when you say WriteTextAsync().Wait(), you block your current thread again and wait for the result of WriteTextAsync().
The C# compiler already warns us if we have a method that has the async modifier but does not use the await operator.
According to this answer there's no point adding an await to the end of an async method (and in that case, just remove the async modifier).
But what if the method has an expensive synchronous operation it needs to perform before calling a subsequent true async method?
For example, if I'm using HttpClient:
private readonly HttpClient client = ...
public Task<HttpResponseMessage> CallMyWebServiceMethod() {
HttpRequestMessage request = this.SomeExpensiveButSynchronousMethod();
return this.client.SendAsync( request );
}
This code would block the caller (due to SomeExpensiveButSynchronousMethod).
However, if I change the code to this:
public async Task<HttpResponseMessage> CallMyWebServiceMethod() {
HttpRequestMessage request = this.SomeExpensiveButSynchronousMethod();
return await this.client.SendAsync( request );
}
and I call it like so:
HttpResponse response = await myWrapper.CallMyWebServiceMethod();
...I understand the TPL would spin up a background thread immediately and then run CallMyWebServiceMethod in the background thread, resuming whatever the parent code wants, making the entire call non-blocking in the process, before resuming after the Task completes and returns the HttpResponse.
...if that's the case then it seems contradictory.
If I'm wrong, and the call is blocking until it gets to SendAsync then how can I execute SomeExpensiveButSynchronousMethod on the same background thread as the HttpClient uses for its request?
According to this answer there's no point adding an await to the end of an async method (and in that case, just remove the async modifier).
That's an oversimplification. See my blog post on eliding async and await.
But what if the method has an expensive synchronous operation it needs to perform before calling a subsequent true async method?
This is a rare case, but the appropriate solution IMO is to execute the synchronous operation synchronously (i.e., not wrapped in a Task.Run) and be sure to document its behavior.
I understand the TPL would spin up a background thread immediately and then run CallMyWebServiceMethod in the background thread, resuming whatever the parent code wants, making the entire call non-blocking in the process, before resuming after the Task completes and returns the HttpResponse.
That is not at all what happens. You may find my async intro helpful. Quote:
The beginning of an async method is executed just like any other method. That is, it runs synchronously until it hits an “await” (or throws an exception).
In actuality, both of your examples synchronously block the caller while executing SomeExpensiveButSynchronousMethod.
If I'm wrong, and the call is blocking until it gets to SendAsync then how can I execute SomeExpensiveButSynchronousMethod on the same background thread as the HttpClient uses for its request?
HttpClient doesn't use a background thread for its request, so this part of the question doesn't make sense. For more information on how asynchronous I/O works, see my blog post There Is No Thread.
To answer the actual question:
Should I add the async modifier if I return a Task in an expensive method?
Yes. But the reason you should do so is not to "make it asynchronous"; it's so that any exceptions from SomeExpensiveButSynchronousMethod are captured and placed on the returned Task, which is the expected semantic for methods that follow the Task-based Asynchronous Pattern (TAP).
I understand the TPL would spin up a background thread immediately and then run CallMyWebServiceMethod
No, that's not what happens. The first synchronous part of an async method executes synchronously. If you want make sure it does not block the current thread, you should use Task.Run().
Your expensive method is going to block the caller either way. async doesn't magically create threads for you, that's what Task does (sometimes). So, if you add async and await the SendAsync, you're just adding state-machine overhead needlessly.
see also http://blog.stephencleary.com/2016/12/eliding-async-await.html
I am really curious how async/await enables your program not to be halted.
I really like the way how Stephen Cleary explains async/await: "I like to think of "await" as an "asynchronous wait". That is to say, the async method pauses until the awaitable is complete(so it waits), but the actual thread is not blocked (so it's asynchornous)."
I've read that async method works synchronously till compilator meets await keywords. Well. If compilator cannot figure out awaitable, then compilator queues the awaitable and yield control to the method that called method AccessTheWebAsync. OK.
Inside the caller (the event handler in this example), the processing pattern continues. The caller might do other work that doesn't depend on the result from AccessTheWebAsync before awaiting that result, or the caller might await immediately. The event handler is waiting for AccessTheWebAsync, and AccessTheWebAsync is waiting for GetStringAsync. Let's see an msdn example:
async Task<int> AccessTheWebAsync()
{
// You need to add a reference to System.Net.Http to declare client.
HttpClient client = new HttpClient();
// GetStringAsync returns a Task<string>. That means that when you await the
// task you'll get a string (urlContents).
Task<string> getStringTask = client.GetStringAsync("http://msdn.microsoft.com");
// You can do work here that doesn't rely on the string from GetStringAsync.
DoIndependentWork();
// The await operator suspends AccessTheWebAsync.
// - AccessTheWebAsync can't continue until getStringTask is complete.
// - Meanwhile, control returns to the caller of AccessTheWebAsync.
// - Control resumes here when getStringTask is complete.
// - The await operator then retrieves the string result from getStringTask.
string urlContents = await getStringTask;
// The return statement specifies an integer result.
// Any methods that are awaiting AccessTheWebAsync retrieve the length value.
return urlContents.Length;
}
Another article from msdn blog says that async/await does not create new thread or use other threads from thread pool. OK.
My questions:
Where does async/await execute awaitable code(in our example downloading a web site) cause control yields to the next row of code of our program and program just asks result of Task<string> getStringTask? We know that no new threads, no thread pool are not used.
Am I right in my silly assumption that CLR just switches the current executable code and awaitable part of the method between each other in scope of one thread? But changing the order of addends does not change the sum and UI might be blocked for some unnoticeable time.
Where does async/await execute awaitable code(in our example downloading a web site) cause control yields to the next row of code of our program and program just asks result of Task getStringTask? We know that no new threads, no thread pool are not used.
If the operation is truly asynchronous, then there's no code to "execute". You can think of it as all being handled via callbacks; the HTTP request is sent (synchronously) and then the HttpClient registers a callback that will complete the Task<string>. When the download completes, the callback is invoked, completing the task. It's a bit more complex than this, but that's the general idea.
I have a blog post that goes into more detail on how asynchronous operations can be threadless.
Am I right in my silly assumption that CLR just switches the current executable code and awaitable part of the method between each other in scope of one thread?
That's a partially true mental model, but it's incomplete. For one thing, when an async method resumes, its (former) call stack is not resumed along with it. So async/await are very different than fibers or co-routines, even though they can be used to accomplish similar things.
Instead of thinking of await as "switch to other code", think of it as "return an incomplete task". If the calling method also calls await, then it also returns an incomplete task, etc. Eventually, you'll either return an incomplete task to a framework (e.g., ASP.NET MVC/WebAPI/SignalR, or a unit test runner); or you'll have an async void method (e.g., UI event handler).
While the operation is in progress, you end up with a "stack" of task objects. Not a real stack, just a dependency tree. Each async method is represented by a task instance, and they're all waiting for that asynchronous operation to complete.
Where is continuation of awaitable part of method performed?
When awaiting a task, await will - by default - resume its async method on a captured context. This context is SynchronizationContext.Current unless it is null, in which case it is TaskScheduler.Current. In practice, this means that an async method running on a UI thread will resume on that UI thread; an async method handling an ASP.NET request will resume handling that same ASP.NET request (possibly on a different thread); and in most other cases the async method will resume on a thread pool thread.
In the example code for your question, GetStringAsync will return an incomplete task. When the download completes, that task will complete. So, when AccessTheWebAsync calls await on that download task, (assuming the download hasn't already finished) it will capture its current context and then return an incomplete task from AccessTheWebAsync.
When the download task completes, the continuation of AccessTheWebAsync will be scheduled to that context (UI thread, ASP.NET request, thread pool, ...), and it will extract the Length of the result while executing in that context. When the AccessTheWebAsync method returns, it sets the result of the task previously returned from AccessTheWebAsync. This in turn will resume the next method, etc.
In general the continuation (the part of your method after await) can run anywhere. In practice it tends to run on the UI thread (e.g. in a Windows application) or on the thread pool (e.g. in an ASP .NET server). It can also run synchronously on the caller thread in some cases ... really it depends on what kind of API you're calling and what synchronization context is being used.
The blog article you linked does not say that continuations are not run on thread pool threads, it merely says that marking a method as async does not magically cause invocations of the method to run on a separate thread or on the thread pool.
That is, they're just trying to tell you that if you have a method void Foo() { Console.WriteLine(); }, changing that to async Task Foo() { Console.WriteLine(); } doesn't suddenly cause an invocation of Foo(); to behave any differently at all – it'll still be executed synchronously.
If by "awaitable code" you mean the actual asynchronous operation, then you need to realize that it "executes" outside of the CPU so there's no thread needed and no code to run.
For example when you download a web page, most of the operation happens when your server sends and receives data from the web server. There's no code to execute while this happens. That's the reason you can "take over" the thread and do other stuff (other CPU operations) before awaiting the Task to get the actual result.
So to your questions:
It "executes" outside of the CPU (so it's not really executed). That could mean the network driver, a remote server, etc. (mostly I/O).
No. Truly asynchronous operations don't need to be executed by the CLR. They are only started and completed in the future.
A simple example is Task.Delay which creates a task that completes after an interval:
var delay = Task.Delay(TimeSpan.FromSeconds(30));
// do stuff
await delay;
Task.Delay internally creates and sets a System.Threading.Timer that will execute a callback after the interval and complete the task. System.Threading.Timer doesn't need a thread, it uses the system clock. So you have "awaitable code" that "executes" for 30 seconds but nothing actually happens in that time. The operation started and will complete 30 seconds in the future.
I'm evaluating the Async CTP.
How can I begin execution of an async function on another thread pool's thread?
static async Task Test()
{
// Do something, await something
}
static void Main( string[] args )
{
// Is there more elegant way to write the line below?
var t = TaskEx.Run( () => Test().Wait() );
// Doing much more in this same thread
t.Wait(); // Waiting for much more then just this single task, this is just an example
}
I'm new (my virginal post) to Stack Overflow, but I'm jazzed that you're asking about the Async CTP since I'm on the team working on it at Microsoft :)
I think I understand what you're aiming for, and there's a couple of things you're doing correctly, to get you there.
What I think you want:
static async Task Test()
{
// Do something, await something
}
static void Main(string[] args)
{
// In the CTP, use Task.RunEx(...) to run an Async Method or Async Lambda
// on the .NET thread pool
var t = TaskEx.RunEx(Test);
// the above was just shorthand for
var t = TaskEx.RunEx(new Func<Task>(Test));
// because the C# auto-wraps methods into delegates for you.
// Doing much more in this same thread
t.Wait(); // Waiting for much more then just this single task, this is just an example
}
Task.Run vs. Task.RunEx
Because this CTP installs on top of .NET 4.0, we didn't want to patch the actual System.Threading.Tasks.Task type in mscorlib. Instead, the playground APIs are named FooEx when they conflicted.
Why did we name some of them Run(...) and some of the RunEx(...)? The reason is because of redesigns in method overloading that we hadn't completed yet by the time we released the CTP. In our current working codebase, we've actually had to tweak the C# method overloading rules slightly so that the right thing happens for Async Lambdas - which can return void, Task, or Task<T>.
The issue is that when async method or lambdas return Task or Task<T>, they actually don't have the outer task type in the return expression, because the task is generated for you automatically as part of the method or lambda's invocation. This strongly seems to us like the right experience for code clarity, though that does make things quite different before, since typically the expression of return statements is directly convertible to the return type of the method or lambda.
So thus, both async void lambdas and async Task lambdas support return; without arguments. Hence the need for a clarification in method overload resolution to decide which one to pick. Thus the only reason why you have both Run(...) and RunEx(...) was so that we would make sure to have higher quality support for the other parts of the Async CTP, by the time PDC 2010 hit.
How to think about async methods/lambdas
I'm not sure if this is a point of confusion, but I thought I'd mention it - when you are writing an async method or async lambda, it can take on certain characteristics of whoever is invoking it. This is predicated on two things:
The type on which you are awaiting
And possibly the synchronization context (depending on above)
The CTP design for await and our current internal design are both very pattern-based so that API providers can help flesh out a vibrant set of things that you can 'await' on. This can vary based on the type on which you're awaiting, and the common type for that is Task.
Task's await implementation is very reasonable, and defers to the current thread's SynchronizationContext to decide how to defer work. In the case that you're already in a WinForms or WPF message loop, then your deferred execution will come back on the same message loop (as if you used BeginInvoke() the "rest of your method"). If you await a Task and you're already on the .NET threadpool, then the "rest of your method" will resume on one of the threadpool threads (but not necessarily the same one exactly), since they were pooled to begin with and most likely you're happy to go with the first available pool thread.
Caution about using Wait() methods
In your sample you used:
var t = TaskEx.Run( () => Test().Wait() );
What that does is:
In the surrounding thread synchronously call TaskEx.Run(...) to execute a lambda on the thread pool.
A thread pool thread is designated for the lambda, and it invokes your async method.
The async method Test() is invoked from the lambda. Because the lambda was executing on the thread pool, any continuations inside Test() can run on any thread in the thread pool.
The lambda doesn't actually vacate that thread's stack because it had no awaits in it. The TPL's behavior in this case depends on if Test() actually finished before the Wait() call. However, in this case, there's a real possibility that you will be blocking a thread pool thread while it waits for Test() to finish executing on a different thread.
That's the primary benefit of the 'await' operator is that it allows you to add code that executes later - but without blocking the original thread. In the thread pool case, you can achieve better thread utilization.
Let me know if you have other questions about the Async CTP for VB or C#, I'd love to hear them :)
It's usually up to the method returning the Task to determine where it runs, if it's starting genuinely new work instead of just piggy-backing on something else.
In this case it doesn't look like you really want the Test() method to be async - at least, you're not using the fact that it's asynchronous. You're just starting stuff in a different thread... the Test() method could be entirely synchronous, and you could just use:
Task task = TaskEx.Run(Test);
// Do stuff
t.Wait();
That doesn't require any of the async CTP goodness.
There would be, if this wasn't a console application. For example, if you do this in a Windows Forms application, you could do:
// Added to a button click event, for example
public async void button1_Click(object sender, EventArgs e)
{
// Do some stuff
await Test();
// Do some more stuff
}
However, there is no default SynchronizationContext in a console, so that won't work the way you'd expect. In a console application, you need to explicitly grab the task and then wait at the end.
If you're doing this in a UI thread in Windows Forms, WPF, or even a WCF service, there will be a valid SynchronizationContext that will be used to marshal back the results properly. In a console application, however, when control is "returned" at the await call, the program continues, and just exits immediately. This tends to mess up everything, and produce unexpected behavior.