Imagine we have an aspx web page that calls a stored procedure and 15 minutes later renders a table of data on a GridView. In my hypothetical, I'm not running, say, 4 asych which could happen in parallel-- just one long database proc.
At at least 3 places on the call stack, Microsoft lets me attempt to do things the asynch way, with async pages and web methods, async ADO.NET calls and things like the async key word and asynchronous delegates.
eg:
[WebMethod]
public IAsyncResult BeginLengthyProcedure( AsyncCallback cb, object s) {...}
[WebMethod]
public string EndLengthyProcedure(IAsyncResult call) {...}
(ref http://msdn.microsoft.com/en-us/library/aa480516.aspx )
My mental model was that IIS can only have so many "things (threads?)" at once handling requests and that if you use the async techniques the page will not exhaust the pool of threads available to take requests. I thought that while the async method is running, it consume OS threads and maybe could still crush the server with activity, but IIS will assume that it doesn't need to be counted against the maximum requests that it will deal with & thus the application remains responsive.
In the comments on this question here, I got confused about if and what resources are saved with async. I'm getting the impression from that discussion no technique with the keyword "asynch" in it will save any resource with the keywords "thread" or "resource" in it. Could it be that there is some resource that is freed up for other requests and maybe I just don't have the name of it?
Part B.
What is that limit, is it so high that only intense, 100s of requests per millisecond would hit it, or is it low enough that a few dozen users each running synchronous 15 minute pages could hit that limit?
Lucian Wischik, one of the spec leads involved with .NET Async, described asynchronous programming using the analogy of waiters (at a restaurant).
"A waiter’s job is to wait on a table until the patrons have finished their meal. If you want to serve two tables concurrently, you must hire two waiters."
That’s not true, is it? Why? Because you don’t need two waiters! You can just use the same waiter, and share him between tables.
Talk: Async Part 1 - the message-loop, and the Task type
So, rather than spinning up new threads (which is expensive) to simply sit around and wait, you enable your primary thread to put a sort of bookmark on the request which is taking a long time. This is analogous to allowing your primary waiter to check on other tables while the first table they served is busy choosing what to order or eating.
Now, if the long-running process is something your code is doing, like processing a list of items, async isn't going to save you anything. (I suppose the analogy there would be mowing lawns, in which case you'd need two mowers to mow two lawns concurrently.) Async is only useful when your code is waiting on a resource to become available or a request to be completed, such as an internet connection to be established or a query to return results. It saves you the expense and complexity associated with multi-threading.
Lucian provides an excellent talk on the subject for people who know little or nothing about asynchronous programming at the link above. While his talk focuses on async as applied by the .NET language, the theory extends to asynchronous programming in general.
Related
We are working on a old comparator.
When an user make a research, we are calling 10-30 different webservice (REST, SOAP) at the same time. Pretty classic so. Each webservice is reprensented by a Client in our application.
So the code is like:
//Get the request list of client to call
clientRqListToCall = BuildRequest(userContext);
List<Task> taskList = new List<Task>();
//Call the different client
Foreach (ClientRequest clientRq in clientRqListToCall) {
Task task = Task.Run(() => CallClient(clientRq));
taskList.Add(task);
}
//wait client until timeOut
Task mainWaiterTask = Task.WhenAll(taskList);
mainTask.ConfigureAwait(false);
mainTask.Wait(timeout);
Simple. (Not sure the configureAwait is needed). The response of each client is store in a field of ClientRequest, so we don't use mainTask.Result (if a client Timeout, we need to be able to continue with another's ones, and they timeout a lot! Client call behaviours are pretty similar to a fireandforget).
The application is a little old, our search engine is synchronous. The call of the different webservice are in the different CallClient callTree, according the to research context, 5 to 15 different function are call before the webservice call. Each webservice call is pretty long (1 to 15s each) ! This point seems to be important ! These are not pings simple pings requests.
Actions / Changes ?
So this is an I/O bound problem, we know Task.Run work pretty well for CPU-bound problem and not for I/O, the question is how to make this code better?
We read a lot of different article on the subject, thanks to Stephen Cleary (http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html)
But we arenot sure of our choice / road map, that s why i post this ticket.
We could make the code asynchronous, but we would have to rework the whole CallClient call tree (hundreds of functions). It is the only solution ? Of course we could migrate webservice one by one using bool argument hack (https://msdn.microsoft.com/en-us/magazine/mt238404.aspx).
=> Must we start with the most costly (in term of IO) webservice, or only the number of webservice call is important, and so we should start the easiest?
In others words, if i got 1 bigs client, with a 10s response average and a lot of data, must we start to async then first? Or should be start with littles ones (1-2s) with the same amount of data. I could be wrong but a thread is lock in synchronous way until task.run() finish so obvisouly the 10s Task lock a thread for the whole time, but in term of I/O free a thread asap could be better. Does the amount of data download is important? or should we only thinck in term of webservice timer?
Task.Run use the application threadPool, we have to choice between .Run(...) or Task.Factory.StartNew(..., TaskCreationOptions.LongRunning) in order to (lots of the time),
create new thread and so maybe got a better.
=> i made some test on subjet, using a console application, .Run() seems to be 25% to 33% faster than Task.Factory.StartNew in all scenario.
Of course this is an expected result, but on a webapp with like 200 users,
i am not sure the result would be the same, i fear the pool to be full and the Task jump to each others without be ended.
Note: If startNew is used, WaitAll(timeout) remplace WhenAll.
Today we got in average 20 to 50 customers can make a research at the same time. The application work without big issues, we dont have deadlock, but sometimes we can see some delay in the task execution in our side. Our Cpu usage is pretty low (<10%), Ram is green too (<25%)
I know there is plenty of tickets about Tasks, but it s hard to merge them together to match our problem. And we also read contradictory advices.
I have used Parallel.ForEach to handle multiple I/O operations before, I did not see it mentioned above. I am not sure it will handle quite what you need seeing the function that is passed into the loop is that same for each. Maybe coupled with a strategy pattern / delegates you can achieve what you need.
I have this code (the unimportant details are that it runs on EC2 instances in AWS, processing messages on an SQS queue).
The first statement in the method gets some data over http, the second statement saves state to a local dynamo data store.
public bool HandleMessage(OrderAcceptedMessage message)
{
var order = _orderHttpClient.GetById(message.OrderId);
_localDynamoRepo.SaveAcceptedOrder(message, order);
return true;
}
The performance characteristics are that the http round trip takes 100-200 milliseconds, and the dynamo write takes around 10 milliseconds.
Both of these operations have async versions. We could write it as follows:
public async Task<bool> HandleMessage(OrderAcceptedMessage message)
{
var order = await _orderHttpClient.GetByIdAsync(message.OrderId);
await _localDynamoRepo.SaveAcceptedOrderAsync(message, order);
return true;
}
So the guidance is that since the first operation "could take longer than 50 milliseconds to execute" it should make use of async and await. (1)
But what about the second, fast operation? Which of these two arguments is correct:
Do not make it async: It does not meet the 50ms criterion and it's not worth the overhead.
Do make it async: The overhead has already been paid by the previous operation. There is already task-based asynchrony happening and it's worth using it.
1) http://blog.stephencleary.com/2013/04/ui-guidelines-for-async.html
the unimportant details are that it runs on EC2 instances in AWS, processing messages on an SQS queue
Actually, I think that's an important detail. Because this is not a UI application; it's a server application.
the guidance is that since the first operation "could take longer than 50 milliseconds to execute"
This guidance only applies to UI applications. Thus, the 50ms guideline is meaningless here.
Which of these two arguments is correct
Asynchrony is not about speed. It's about freeing up threads. The 50ms guideline for UI apps is all about freeing up the UI thread. On the server side, async is about freeing up thread pool threads.
The question is how much scalability do you want/need? If your backend is scalable, then I generally recommend async, because that frees up thread pool threads. This makes your web app more scalable and more able to react to changes in load more quickly. But this only gives you a benefit if your backend can scale along with your web app.
First notice that in web apps the biggest cost of async is reduction of productivity. This is what we are weighing the benefits against. You need to think about how much code will be infected if you make this one method async.
The benefit is saving a thread for the duration of the call. A 200ms HTTP call is a pretty good case for async (although it's impossible to say for sure because it also depends on how often you perform the call).
The 50ms criterion is not hard number. In fact that recommendation is for realtime UI apps.
A more useful number is latency times frequency. That tells you how many threads are consumed in the long term average. Infrequent calls do not need to be optimized.
100 dynamo calls per second at 10ms come out at one thread blocked. This is nothing. So this probably is not a good candidate for async.
Of course if you make the first call async you can make the second one async as well at almost no incremental productivity cost because everything is infected already.
You can run the numbers yourself and decide based on that.
This might end up in an opinionated discussion...but let's try.
tl;dr: yes, keep it async.
You are in a library and you don't care about the synchronisation context, so you should not capture it and change your code into:
var order = await _orderHttpClient.GetByIdAsync(message.OrderId).ConfigureAwait(false);
await _localDynamoRepo.SaveAcceptedOrderAsync(message, order).ConfigureAwait(false);
Besides: after the first awaited call, you'll likely end up on a thread of the thread pool. So even if you use the non-async version SaveAcceptedOrder() it will not block. However, this is nothing you should rely on and you don't necessarily know the type of the async method (CPU bound or IO bound = "async by design"). If it is IO bound, there's no need to run it on a thread.
If you're making any remote call, make it async. Yes, DynamoDB calls are fast (except where one has a sub-par hash-key, or many gigabytes of data in a single table), but you're still making them over the internet (even if you're inside AWS EC2 etc), and so you should not ignore any of the Eight Fallacies of Distributed Computing - and especially not 1) The network is reliable or 2) Latency is zero.
At what point is it advisable to use async controllers in ASP.NET MVC.
Is there any coding or performance costs involved?
MSDN recommends using it for long running processes, but I was just curious if it would it be beneficial if we used it as a complete replacement to normal controllers?
We are planning to use WCF services with our controller methods.
First, async is not synonymous with "performance". In fact, using async can actually decrease performance as there's a non-trivial amount of overhead involved in async.
What async does do is release threads back to the pool when they're in a wait-state. This means that your web server is given a higher threshold before it exhausts it's "max requests" or, in other words, runs out of free threads to handle new requests.
In a synchronous request, the thread is tied up for the entire request. If there's some period of waiting involved (network latency from an API call, etc.) it's holding on to that thread even though no work is actually being done. If you got hit with 1000 simultaneous requests (the typical out-of-the-box max requests for a web server), then each further request would be queued until one for the first 1000 threads was returned to the pool.
In an async request, as soon as the thread is waiting on something to happen (i.e. not doing work), it is given back to the pool, even though the original request it was serving has not yet completed. This allows a new request to be served. When the original task that forfeited the thread completes, a new thread is requested from the pool to continue servicing that request. This effectively gives your server a little breathing room when under load. Other than that, async does nothing, at least in the context of a request being served by a web server.
In general, using async is recommended, because even that little bit of breathing room it provides may mean the difference between your server handling load or falling down. However, you should gauge your usage of async to ensure that you're actually buying something worthwhile of the overhead it adds. For example, MVC 6 lets you do things like render partials asynchronously. If your server is equipped with an enterprise class 15,000 RPM hard drive or an SSD, though, the period of waiting the thread would experience would likely be so minuscule that the passing of the thread back and forth would actually take more time than the operation itself, run synchronously.
I would say that this topic is nicely covered on this post:
When should I use Async Controllers in ASP.NET MVC?
My opinion is that it's good to use async actions when you call async methods in it (like I/O operations), it's not especially bad when you make an async action without any async calls inside, but:
You will have a needless thread switch, not a big performance penalty, but not nice either
VS will warn you, that there is no await in your async action, which can lead to unnecessary Task.Run calls
I've written a server which interacts with an MSSQL database. It's currently written in .NET 4.0 and uses NHibernate as an ORM to retrieve information from the database. When reading about .NET 4.5 and the introduction of the async/await keywords I learned that, unfortunately, NHibernate does not have support for async/await .
I don't understand why issuing an async call to a database would be beneficial. Don't all the requests queue at the database level anyway? Wouldn't async just increase points of failure without improving anything?
In general, the benefit is that you are not blocking the currently executing thread while a possibly expensive (asynchronous) operation is run. In the context of a WPF / Windows Form application, this mean you are not blocking the UI Thread (if the request is originating from that thread) and your application remains responsive.
In the context of a web application (say IIS), this mean you are releasing a thread in the pool while you are awaiting for the result. Since you are not locking the thread, it can be reused to accept another request and results in better performance in terms of accepted connections (not necessarily time / request).
Don't all the requests queue at the database level anyway?
No. Read Understanding how SQL Server executes a query. Any database server worth the name will be able to run hundreds of requests concurrently. Serialization is necessary only if the requests are correlated (eg. you need the output of query 1 to pass as a parameter to query 2) or when operating under transaction constraints (only one statement can be active at any time within a transaction).
There are at least two major advantages of async calls:
resource usage. W/o considering anything else, just changing the programming model to an event driven async model will result in order of magnitude increase of throughput you app can drive. This, of course, applies to back end apps (eg. a web server), not to a client user driven app that will not be able to send anything more than what the one user initiates. Read the articles linked from High Performance Windows programs. This is also important to read, even though a bit dated: Asynchronous Pages in ASP.NET 2.0
overlapping requests. The synchronous model doe snot allow to issue a query to the back end until the current one completes. A lot of times the application has the info necessary (the params) to make two or more uncorrelated requests, but it simply can. Doing async calls allow the controlling thread to issue all the request is parallel, and resume after they all complete.
Neither .Net 4.5 Tasks not NHibernate have good support for async DB programming. Good old BeginExecuteXXX is much more powerful actually, although a bit arcane to program against.
NHibernate can support true async calls. I already implemented it on my own branch
https://github.com/ReverseBlade/nhibernate-core/tree/nh_4.5.1
You can check it out and compile. It is compiled against .net 4.5.1. It is compatible with standart nhibernate and passes all tests.
Then you can use things like .ToListAsync(); or GetAsync(), it will make true async calls.
If you need help you can write a comment. Good luck
Good news. NHibernate supports async/await out of the box since v 5.0
You may be confusing language features with design pattens; async is syntactic sugar to help you manage background tasks, while asynchronous tasks just mean that you're running two or more threads.
Just because NHibernate doesn't support async doesn't mean that you can't run asynchronously. This is very beneficial to the user because you don't want to freeze the UI while you're performing a (relatively) long-running query to a DB/service, especially if the server is bogged down.
I suppose you could count this as a point of failure, but really just a few areas:
Exceptions - You'd have this problem on one thread anyway, but you should gracefully handle any database errors that you'd encounter.
UI Management - You don't want to let the user interact with the UI in such a way as to trigger multiple queries, so you might disable a button, etc.
Result Handling - When the query is complete, you need to ensure that you marshal the data back to the UI thread. In C# this can be done via Invoke/BeginInvoke, though whether you're in WinForms or WPF determines the details.
EDIT:
Some sample skeleton code assuming WPF and at least .NET 4.0
Task.Factory.StartNew(() =>
{
using (var client = new dbClient())
{
// Perform query here
this.Dispatcher.BeginInvoke(new Action(() =>
{
// Set data source, etc, i.e.
this.Items = result;
}));
}
}).ContinueWith(ex => Logger.LogException(ex), TaskContinuationOptions.OnlyOnFaulted);
You say:
Don't all the requests queue at the database level anyway?
If by "queue" you mean "single-servicing queue" than the answer is no. SQL Server is a highly asynchronous and multi-threaded service that can service many, many queries simultaneously.
Even at a physical level, queueing (i.e. physical device servicing) is simultaneously split across the number of CPU cores, and the number of physical disks the make up the disk array.
So the reason to make asynchronous calls to SQL Server is to be able to leverage some of that multi-threading/multi-servicing capacity into your own service.
Async has become a buzzword in .net and MS have introduced it in Web API 2 so that more requests can be handled whilst others are waiting on IO to finish.
Whilst I can see the benefit of this, is it really a concern? A x64 architecture has 30000+ threads in the Thread Pool so unless you have that many concurrent users on your website is async really required? Even if you have that many concurrent users without caching I'm pretty sure SQL Server will fall over with that many requests?
Apart from it being shiny when is there a real need to have async routing on a web framework?
Many of the other answers here are coming from a UI (desktop/mobile app) perspective, not a web server perspective.
Async has become a buzzword in .net and MS have introduced it in Web API 2 so that more requests can be handled whilst others are waiting on IO to finish.
async and await were introduced in .NET 4.5 / VS 2012. However, ASP.NET has had asynchronous request capability since .NET 2.0 - a very long time ago. And there have been people using it.
What async and await bring to the table is asynchronous code that is easy to maintain.
Whilst I can see the benefit of this, is it really a concern?
The key benefit of async on the server is scalability. Simply put, async tasks scale far better than threads.
#Joshua's comment is key regarding the memory; a thread takes a significant amount of memory (and don't forget the kernel-mode stack which cannot be paged out), while an async request literally only takes a few hundred bytes.
There's also bursting to consider. The .NET threadpool has a limited injection rate, so unless you set your minWorkerThread count to a value much higher than you normally need, then when you get a burst of traffic some requests will 503 before .NET can spin up enough threads to handle them. async keeps your threads free (as much as possible) so it handles bursting traffic better.
A x64 architecture has 30000+ threads in the Thread Pool so unless you have that many concurrent users on your website is async really required?
#Joshua is again correct when he points out that you're probably thinking of a request queue limit (which defaults to 1000 for the IIS queue and 5000 for the ASP.NET request limit). It's important to note that once this queue is filled (during bursty traffic), new requests are rejected with 503.
Even if you have that many concurrent users without caching I'm pretty sure SQL Server will fall over with that many requests?
Ah, now that's another question entirely.
I'm giving a talk at ThatConference 2013 specifically on async servers. One part of that talk is situations where async doesn't help (my Twitter update).
There's an excellent blog post here that takes the position that asynchronous db calls are just not worth the effort. It's important to note the assumptions in this post:
At the time that post was written, asynchronous web servers were difficult. These days we have async and more and more libraries are offering asynchronous APIs (e.g., Entity Framework).
The architecture assumes a single web server with a single SQL Server backend. This was a very common setup traditionally, but is quickly changing today.
Where async servers really shine is when your backend can also scale. E.g., a web service, Azure SQL, NoSQL cluster, etc. Example: I'm writing an MVC/WebAPI server that uses Azure SQL and Storage for its backend (for all practical purposes, I can act like they have infinite scalability); in that case, I'm going to make my server async. In situations like this, you can scale your server 10x or more by using async.
But if you just have a single SQL Server backend (and have no plans to change to Azure SQL), then there's no point in making your web server async because you're limited by your backend anyway.
When long operations can be efficiently executed in parallel. For instance, you have to execute two SQLs and load three pictures - do all five operations as async and await them all. In this case the overall time will be the longest duration of five operations, but not the sum of the durations.
Pre-fetch. If you can predict (with good probability) what user will do (e.g. almost certainly, (s)he will want to see the details...) you may start preparing the next page (frame, window) while user's reading the previous.
where did you get 30000 from. i dont remember exactly but I think Asp.net uses 12 x number of cores threads.
I have to use async, when operation take too long time (upload, export, processing) and user have to know about progress.
You need async in following scenarios
1) When you are performing a very long operation and you don't want to freeze your UI.
2) When you designed some task that needs to be completed in background.
For example, You are rendering images from database. But you don't want your page to be freeze at that time async is really helpful.