Develop Reference

Task in synchronous application - IO bound

We are working on a old comparator.
When an user make a research, we are calling 10-30 different webservice (REST, SOAP) at the same time. Pretty classic so. Each webservice is reprensented by a Client in our application.
So the code is like:
//Get the request list of client to call
clientRqListToCall = BuildRequest(userContext);
List<Task> taskList = new List<Task>();
//Call the different client
Foreach (ClientRequest clientRq in clientRqListToCall) {
Task task = Task.Run(() => CallClient(clientRq));
taskList.Add(task);
}
//wait client until timeOut
Task mainWaiterTask = Task.WhenAll(taskList);
mainTask.ConfigureAwait(false);
mainTask.Wait(timeout);
Simple. (Not sure the configureAwait is needed). The response of each client is store in a field of ClientRequest, so we don't use mainTask.Result (if a client Timeout, we need to be able to continue with another's ones, and they timeout a lot! Client call behaviours are pretty similar to a fireandforget).
The application is a little old, our search engine is synchronous. The call of the different webservice are in the different CallClient callTree, according the to research context, 5 to 15 different function are call before the webservice call. Each webservice call is pretty long (1 to 15s each) ! This point seems to be important ! These are not pings simple pings requests.
Actions / Changes ?
So this is an I/O bound problem, we know Task.Run work pretty well for CPU-bound problem and not for I/O, the question is how to make this code better?
We read a lot of different article on the subject, thanks to Stephen Cleary (http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html)
But we arenot sure of our choice / road map, that s why i post this ticket.
We could make the code asynchronous, but we would have to rework the whole CallClient call tree (hundreds of functions). It is the only solution ? Of course we could migrate webservice one by one using bool argument hack (https://msdn.microsoft.com/en-us/magazine/mt238404.aspx).
=> Must we start with the most costly (in term of IO) webservice, or only the number of webservice call is important, and so we should start the easiest?
In others words, if i got 1 bigs client, with a 10s response average and a lot of data, must we start to async then first? Or should be start with littles ones (1-2s) with the same amount of data. I could be wrong but a thread is lock in synchronous way until task.run() finish so obvisouly the 10s Task lock a thread for the whole time, but in term of I/O free a thread asap could be better. Does the amount of data download is important? or should we only thinck in term of webservice timer?
Task.Run use the application threadPool, we have to choice between .Run(...) or Task.Factory.StartNew(..., TaskCreationOptions.LongRunning) in order to (lots of the time),
create new thread and so maybe got a better.
=> i made some test on subjet, using a console application, .Run() seems to be 25% to 33% faster than Task.Factory.StartNew in all scenario.
Of course this is an expected result, but on a webapp with like 200 users,
i am not sure the result would be the same, i fear the pool to be full and the Task jump to each others without be ended.
Note: If startNew is used, WaitAll(timeout) remplace WhenAll.
Today we got in average 20 to 50 customers can make a research at the same time. The application work without big issues, we dont have deadlock, but sometimes we can see some delay in the task execution in our side. Our Cpu usage is pretty low (<10%), Ram is green too (<25%)
I know there is plenty of tickets about Tasks, but it s hard to merge them together to match our problem. And we also read contradictory advices.

I have used Parallel.ForEach to handle multiple I/O operations before, I did not see it mentioned above. I am not sure it will handle quite what you need seeing the function that is passed into the loop is that same for each. Maybe coupled with a strategy pattern / delegates you can achieve what you need.

Should I make a fast operation async if the method is already async

I have this code (the unimportant details are that it runs on EC2 instances in AWS, processing messages on an SQS queue).
The first statement in the method gets some data over http, the second statement saves state to a local dynamo data store.
public bool HandleMessage(OrderAcceptedMessage message)
{
var order = _orderHttpClient.GetById(message.OrderId);
_localDynamoRepo.SaveAcceptedOrder(message, order);
return true;
}
The performance characteristics are that the http round trip takes 100-200 milliseconds, and the dynamo write takes around 10 milliseconds.
Both of these operations have async versions. We could write it as follows:
public async Task<bool> HandleMessage(OrderAcceptedMessage message)
{
var order = await _orderHttpClient.GetByIdAsync(message.OrderId);
await _localDynamoRepo.SaveAcceptedOrderAsync(message, order);
return true;
}
So the guidance is that since the first operation "could take longer than 50 milliseconds to execute" it should make use of async and await. (1)
But what about the second, fast operation? Which of these two arguments is correct:
Do not make it async: It does not meet the 50ms criterion and it's not worth the overhead.
Do make it async: The overhead has already been paid by the previous operation. There is already task-based asynchrony happening and it's worth using it.
1) http://blog.stephencleary.com/2013/04/ui-guidelines-for-async.html

the unimportant details are that it runs on EC2 instances in AWS, processing messages on an SQS queue
Actually, I think that's an important detail. Because this is not a UI application; it's a server application.
the guidance is that since the first operation "could take longer than 50 milliseconds to execute"
This guidance only applies to UI applications. Thus, the 50ms guideline is meaningless here.
Which of these two arguments is correct
Asynchrony is not about speed. It's about freeing up threads. The 50ms guideline for UI apps is all about freeing up the UI thread. On the server side, async is about freeing up thread pool threads.
The question is how much scalability do you want/need? If your backend is scalable, then I generally recommend async, because that frees up thread pool threads. This makes your web app more scalable and more able to react to changes in load more quickly. But this only gives you a benefit if your backend can scale along with your web app.

First notice that in web apps the biggest cost of async is reduction of productivity. This is what we are weighing the benefits against. You need to think about how much code will be infected if you make this one method async.
The benefit is saving a thread for the duration of the call. A 200ms HTTP call is a pretty good case for async (although it's impossible to say for sure because it also depends on how often you perform the call).
The 50ms criterion is not hard number. In fact that recommendation is for realtime UI apps.
A more useful number is latency times frequency. That tells you how many threads are consumed in the long term average. Infrequent calls do not need to be optimized.
100 dynamo calls per second at 10ms come out at one thread blocked. This is nothing. So this probably is not a good candidate for async.
Of course if you make the first call async you can make the second one async as well at almost no incremental productivity cost because everything is infected already.
You can run the numbers yourself and decide based on that.

This might end up in an opinionated discussion...but let's try.
tl;dr: yes, keep it async.
You are in a library and you don't care about the synchronisation context, so you should not capture it and change your code into:
var order = await _orderHttpClient.GetByIdAsync(message.OrderId).ConfigureAwait(false);
await _localDynamoRepo.SaveAcceptedOrderAsync(message, order).ConfigureAwait(false);
Besides: after the first awaited call, you'll likely end up on a thread of the thread pool. So even if you use the non-async version SaveAcceptedOrder() it will not block. However, this is nothing you should rely on and you don't necessarily know the type of the async method (CPU bound or IO bound = "async by design"). If it is IO bound, there's no need to run it on a thread.

If you're making any remote call, make it async. Yes, DynamoDB calls are fast (except where one has a sub-par hash-key, or many gigabytes of data in a single table), but you're still making them over the internet (even if you're inside AWS EC2 etc), and so you should not ignore any of the Eight Fallacies of Distributed Computing - and especially not 1) The network is reliable or 2) Latency is zero.

How we can achieve performance by implementing async controller in asp.net MVC 6?

At what point is it advisable to use async controllers in ASP.NET MVC.
Is there any coding or performance costs involved?
MSDN recommends using it for long running processes, but I was just curious if it would it be beneficial if we used it as a complete replacement to normal controllers?
We are planning to use WCF services with our controller methods.

First, async is not synonymous with "performance". In fact, using async can actually decrease performance as there's a non-trivial amount of overhead involved in async.
What async does do is release threads back to the pool when they're in a wait-state. This means that your web server is given a higher threshold before it exhausts it's "max requests" or, in other words, runs out of free threads to handle new requests.
In a synchronous request, the thread is tied up for the entire request. If there's some period of waiting involved (network latency from an API call, etc.) it's holding on to that thread even though no work is actually being done. If you got hit with 1000 simultaneous requests (the typical out-of-the-box max requests for a web server), then each further request would be queued until one for the first 1000 threads was returned to the pool.
In an async request, as soon as the thread is waiting on something to happen (i.e. not doing work), it is given back to the pool, even though the original request it was serving has not yet completed. This allows a new request to be served. When the original task that forfeited the thread completes, a new thread is requested from the pool to continue servicing that request. This effectively gives your server a little breathing room when under load. Other than that, async does nothing, at least in the context of a request being served by a web server.
In general, using async is recommended, because even that little bit of breathing room it provides may mean the difference between your server handling load or falling down. However, you should gauge your usage of async to ensure that you're actually buying something worthwhile of the overhead it adds. For example, MVC 6 lets you do things like render partials asynchronously. If your server is equipped with an enterprise class 15,000 RPM hard drive or an SSD, though, the period of waiting the thread would experience would likely be so minuscule that the passing of the thread back and forth would actually take more time than the operation itself, run synchronously.

I would say that this topic is nicely covered on this post:
When should I use Async Controllers in ASP.NET MVC?
My opinion is that it's good to use async actions when you call async methods in it (like I/O operations), it's not especially bad when you make an async action without any async calls inside, but:
You will have a needless thread switch, not a big performance penalty, but not nice either
VS will warn you, that there is no await in your async action, which can lead to unnecessary Task.Run calls

Why does an async single task run faster than a normal single task?

I have a method which has just one task to do and has to wait for that task to complete:
public async Task<JsonResult> GetAllAsync()
{
var result = await this.GetAllDBAsync();
return Json(result, JsonRequestBehavior.AllowGet);
}
public async Task<List<TblSubjectSubset>> GetAllDBAsync()
{
return await model.TblSubjectSubsets.ToListAsync();
}
It is significantly faster than when I run it without async-await.
We know
The async and await keywords don't cause additional threads to be
created. Async methods don't require multithreading because an async
method doesn't run on its own thread. The method runs on the current
synchronization context and uses time on the thread only when the
method is active
According to this link: https://msdn.microsoft.com/en-us/library/hh191443.aspx#BKMK_Threads. What is the reason for being faster when we don't have another thread to handle the job?

"Asynchronous" does not mean "faster."
"Asynchronous" means "performs its operation in a way that it does not require a thread for the duration of the operation, thus allowing that thread to be used for other work."
In this case, you're testing a single request. The asynchronous request will "yield" its thread to the ASP.NET thread pool... which has no other use for it, since there are no other requests.
I fully expect asynchronous handlers to run slower than synchronous handlers. This is for a variety of reasons: there's the overhead of the async/await state machine, and extra work when the task completes to have its thread enter the request context. Besides this, the Win32 API layer is still heavily optimized for synchronous calls (expect this to change gradually over the next decade or so).
So, why use asynchronous handlers then?
For scalability reasons.
Consider an ASP.NET server that is serving more than one request - hundreds or thousands of requests instead of a single one. In that case, ASP.NET will be very grateful for the thread returned to it during its request processing. It can immediately use that thread to handle other requests. Asynchronous requests allow ASP.NET to handle more requests with fewer threads.
This is assuming your backend can scale, of course. If every request has to hit a single SQL Server, then your scalability bottleneck will probably be your database, not your web server.
But if your situation calls for it, asynchronous code can be a great boost to your web server scalability.
For more information, see my article on async ASP.NET.

I agree with Orbittman when he mentions the overhead involved in the application architecture. It doesn't make for a very good benchmark premise since you can't be sure if the degradation can indeed be solely attributed to the async vs non-async calls.
I've created a really simple benchmark to get a rough comparison between an async and a synchronous call and async loses every time in the overall timing actually, though the data gathering section always seems to end up the same. Have a look: https://gist.github.com/mattGuima/25cb7893616d6baaf970
Having said that, the same thought regarding the architecture applies. Frameworks handle async calls differently: Async and await - difference between console, Windows Forms and ASP.NET
The main thing to remember is to never confuse async with performance gain, because it is completely unrelated and most often it will result on no gain at all, specially with CPU-bound code. Look at the Parallel library for that instead.

Async await is not the silver bullet that some people think it is and in your example is not required. If you were processing the result of the awaitable operation after you received it then you would be able to return a task and continue on the calling thread. You wouldn't have to then wait for the rest of the operation to complete. You would be correct to remove the async/await in the above code.
It's not really possible to answer the question without seeing the calling code either as it depends on what the context is trying to trying to do with the response. What you are getting back is not just a Task but a task in the context of the method that will continue when complete. See http://codeblog.jonskeet.uk/category/eduasync/ for much better information regarding the inner workings of async/await.
Lastly I would question your timings as with an Ajax request to a database and back there other areas with potentially greater latency, such as the HTTP request and response and the DB connection itself. I assume that you're using an ORM and that alone can cause an overhead. I wonder whether it's the async/await that is the problem.

When do you really need async on a web framework?

Async has become a buzzword in .net and MS have introduced it in Web API 2 so that more requests can be handled whilst others are waiting on IO to finish.
Whilst I can see the benefit of this, is it really a concern? A x64 architecture has 30000+ threads in the Thread Pool so unless you have that many concurrent users on your website is async really required? Even if you have that many concurrent users without caching I'm pretty sure SQL Server will fall over with that many requests?
Apart from it being shiny when is there a real need to have async routing on a web framework?

Many of the other answers here are coming from a UI (desktop/mobile app) perspective, not a web server perspective.
Async has become a buzzword in .net and MS have introduced it in Web API 2 so that more requests can be handled whilst others are waiting on IO to finish.
async and await were introduced in .NET 4.5 / VS 2012. However, ASP.NET has had asynchronous request capability since .NET 2.0 - a very long time ago. And there have been people using it.
What async and await bring to the table is asynchronous code that is easy to maintain.
Whilst I can see the benefit of this, is it really a concern?
The key benefit of async on the server is scalability. Simply put, async tasks scale far better than threads.
#Joshua's comment is key regarding the memory; a thread takes a significant amount of memory (and don't forget the kernel-mode stack which cannot be paged out), while an async request literally only takes a few hundred bytes.
There's also bursting to consider. The .NET threadpool has a limited injection rate, so unless you set your minWorkerThread count to a value much higher than you normally need, then when you get a burst of traffic some requests will 503 before .NET can spin up enough threads to handle them. async keeps your threads free (as much as possible) so it handles bursting traffic better.
A x64 architecture has 30000+ threads in the Thread Pool so unless you have that many concurrent users on your website is async really required?
#Joshua is again correct when he points out that you're probably thinking of a request queue limit (which defaults to 1000 for the IIS queue and 5000 for the ASP.NET request limit). It's important to note that once this queue is filled (during bursty traffic), new requests are rejected with 503.
Even if you have that many concurrent users without caching I'm pretty sure SQL Server will fall over with that many requests?
Ah, now that's another question entirely.
I'm giving a talk at ThatConference 2013 specifically on async servers. One part of that talk is situations where async doesn't help (my Twitter update).
There's an excellent blog post here that takes the position that asynchronous db calls are just not worth the effort. It's important to note the assumptions in this post:
At the time that post was written, asynchronous web servers were difficult. These days we have async and more and more libraries are offering asynchronous APIs (e.g., Entity Framework).
The architecture assumes a single web server with a single SQL Server backend. This was a very common setup traditionally, but is quickly changing today.
Where async servers really shine is when your backend can also scale. E.g., a web service, Azure SQL, NoSQL cluster, etc. Example: I'm writing an MVC/WebAPI server that uses Azure SQL and Storage for its backend (for all practical purposes, I can act like they have infinite scalability); in that case, I'm going to make my server async. In situations like this, you can scale your server 10x or more by using async.
But if you just have a single SQL Server backend (and have no plans to change to Azure SQL), then there's no point in making your web server async because you're limited by your backend anyway.

When long operations can be efficiently executed in parallel. For instance, you have to execute two SQLs and load three pictures - do all five operations as async and await them all. In this case the overall time will be the longest duration of five operations, but not the sum of the durations.
Pre-fetch. If you can predict (with good probability) what user will do (e.g. almost certainly, (s)he will want to see the details...) you may start preparing the next page (frame, window) while user's reading the previous.

where did you get 30000 from. i dont remember exactly but I think Asp.net uses 12 x number of cores threads.

I have to use async, when operation take too long time (upload, export, processing) and user have to know about progress.

You need async in following scenarios
1) When you are performing a very long operation and you don't want to freeze your UI.
2) When you designed some task that needs to be completed in background.
For example, You are rendering images from database. But you don't want your page to be freeze at that time async is really helpful.