I'm using async/await in WebApi Controllers according to this article:
https://msdn.microsoft.com/en-us/magazine/dn802603.aspx
Hava look at this simplified code in my controller:
DataBaseData = await Task.Run( () => GetDataFunction() );
GetDataFunction is a function that will open a database connection, open a reader and read the data from the database.
In many examples I see it handled differently. The GetDataFunction itself is awaited. And within the function every single step is awaited.
For example:
connection.OpenAsync
reader.ReadAsycnc
reader.IsDBNullAsync
Why is this good practice? Why not just start a thread for the whole database access (with Task.Run)?
Update:
Thanks for the help. Now I got it. I did not get, that the asynchronous Api's do not start threads themselves. This really helped: blog.stephencleary.com/2013/11/there-is-no-thread.html
The article you linked to states:
You can kick off some background work by awaiting Task.Run, but there’s no point in doing so. In fact, that will actually hurt your scalability by interfering with the ASP.NET thread pool heuristics... As a general rule, don’t queue work to the thread pool on ASP.NET.
In other words, avoid Task.Run on ASP.NET.
Why is this good practice? Why not just start a thread for the whole database access (with Task.Run)?
It's because the asynchronous APIs do not use other threads. The entire point of async/await is to free up the current thread; not use another thread. I have a blog post describing how async works without needing threads.
So, the Task.Run (or custom thread) approach will use up (block) a thread while getting data from the database. Proper asynchronous methods (e.g., EF6 or the ADO.NET asynchronous APIs) do not; they allow the request thread to be used for other requests while that request is waiting for the database response.
I assume you are wondering why using
await Task.Run(() => GetDataFunction());
instead of
await GetDataFunction();
As you can see in Task.Run(Func) under the section Remarks there is written:
The Run(Func) method is used by language compilers to support the async and await keywords. It is not intended to be called directly from user code.
That means, you should use await GetDataFunction.
It's all about resource management and sharing.
When using connection.Open() method the thread that calls it has to wait for the connection to actually open. While waiting it does nothing but consume the CPU slice allocated to it by the operating system. However, when you await connection.OpenAsync() the thread frees up its resources and the OS can redistribute them to other threads.
Now, there's a catch: to actually gain from asynchronous calls they should take longer than the time it takes for the OS to switch contexts otherwise it will incur a drop in application performance.
The practice of awaiting on connection.OpenAsync(), reader.ReadAsync() etc. is due to the fact that these are potentially long running operations. In a system where de database resides on another machine and it is under a heavy load of requests, connecting to the database and getting the results of a query will take some time. Instead of blocking the CPU while waiting on the results to arrive why not allocate that time slice to another worker thread that waits in the scheduler queue to render the response for another client?
Now, about starting another thread for data access: don't do that!. Each new thread gets allocated about 1MB of memory space so beside wasting CPU time while waiting for database operations to finish you'll be wasting memory also. Furthermore, having such a heavy memory footprint would require a lot more runs for Garbage Collector which will freeze all of your threads when running.
Using Task.Run() will schedule the data access operations on a thread from the ThreadPool but you will lose the benefit of sharing the CPU time with another thread while waiting for database server to respond.
Related
I'm of the belief that you should never have to use Task.Run for any operation in .net core web context. If you have a long running task or CPU intensive task, you can offload it to a message queue for async processing. If you have a sync operation, that has no equivalent async method, then offloading to background thread does nothing for you, it actually makes in slightly worse.
What am I missing? Is there a genuine reason to use Task.Run in a high throughput server application?
Some quick examples:
A logging system where each worker thread can write to a queue and a worker thread is responsible for dequeuing items and writing them to a log file.
To access an apartment-model COM server with expensive initialization, where it may be better to keep a single instance on its own thread.
For logic that runs on a timer, e.g. a transaction that runs every 10 minutes to update an application variable with some sort of status.
CPU-bound operations where individual response time is more important than server throughput.
Logic that must continue to run after the HTTP response has been completed, e.g. if the total processing time would otherwise exceed an HTTP response timeout.
Worker threads for system operations, e.g. a long running thread that checks for expired cache entries.
Just to backup your belief:
Do not: Call Task.Run and immediately await it. ASP.NET Core already runs app code on normal Thread Pool threads, so calling
Task.Run only results in extra unnecessary Thread Pool scheduling.
Even if the scheduled code would block a thread, Task.Run does not
prevent that.
This is the official recommendation/best practice from Microsoft. Although it doesn't point out something you might have missed, it does tell you that it is a bad idea and why.
I am aware of how async await works. I know that when execution reaches to await, it release the thread and after IO completes, it fetches thread from threadpool and run the remaining code. This way threads are efficiently utilized. But I am confused in some use cases:
Should we use async methods for the very fast IO method, like cache read/write method? Would not they result into unnecessarily context switch. If we use sync method, execution will complete on same thread and context switch may not happen.
Does Async-await saves only memory consumption(by creating lesser threads). Or it also saves cpu as well? As far as I know, in case of sync IO, while IO takes place, thread goes into sleep mode. That means it does not consume cpu. Is this understanding correct?
I am aware of how async await works.
You are not.
I know that when execution reaches to await, it release the thread
It does not. When execution reaches an await, the awaitable operand is evaluated, and then it is checked to see if the operation is complete. If it is not, then the remainder of the method is signed up as the continuation of the awaitable, and a task representing the work of the current method is returned to the caller.
None of that is "releasing the thread". Rather, control returns to the caller, and the caller keeps executing on the current thread. Of course, if the current caller was the only thing on this thread, then the thread is done. But there is no requirement that an async method be the only call on a thread!
after IO completes
An awaitable need not be an IO operation, but let's suppose that it is.
it fetches thread from threadpool and run the remaining code.
No. It schedules the remaining code to run on the correct context. That context might be a threadpool thread. It might be the UI thread. It might be the current thread. It might be any number of things.
Should we use async methods for the very fast IO method, like cache read/write method?
The awaitable is evaluated. If the awaitable knows that it can complete the operation in a reasonable amount of time then it is perfectly within its rights to do the operation and return a completed task. In which case there is no penalty; you're just checking a boolean to see if the task is completed.
Would not they result into unnecessarily context switch.
Not necessarily.
If we use sync method, execution will complete on same thread and context switch may not happen.
I am confused as to why you think a context switch happens on an IO operation. IO operations run on hardware, below the level of OS threads. There's no thread sitting there servicing IO tasks.
Does Async-await saves only memory consumption(by creating lesser threads)
The purpose of await is to (1) make more efficient use of expensive worker threads by allowing workflows to become more asynchronous, and thereby freeing up threads to do work while waiting for high-latency results, and (2) to make the source code for asynchronous workflows resemble the source code for synchronous workflows.
As far as I know, in case of sync IO, while IO takes place, thread goes into sleep mode. That means it does not consume cpu. Is this understanding correct?
Sure but you have this completely backwards. YOU WANT TO CONSUME CPU. You want to be consuming as much CPU as possible all the time! The CPU is doing work on behalf of the user and if it is idle then its not getting its work done as fast as it could. Don't hire a worker and then pay them to sleep! Hire a worker, and as soon as they are blocked on a high-latency task, put them to work doing something else so the CPU stays as hot as possible all the time. The owner of that machine paid good money for that CPU; it should be running at 100% all the time that there is work to be done!
So let's come back to your fundamental question:
Does async await increases Context switching
I know a great way to find out. Write a program using await, write another one without, run them both, and measure the number of context switches per second. Then you'll know.
But I don't see why context switches per second is a relevant metric. Let's consider two banks with lots of customers and lots of employees. At bank #1 the employees work on one task until it is complete; they never switch context. If an employee is blocked on waiting for a result from another, they go to sleep. At bank #2, employees switch from one task to another when they are blocked, and are constantly servicing customer requests. Which bank do you think has faster customer service?
Should we use async methods for the very fast IO method, like cache read/write method?
Such an IO would not block in the classical sense. "Blocking" is a loosely defined term. Normally it means that the CPU must wait for the hardware.
This type of IO is purely CPU work and there are no context switches. This would typically happen if the app reads a file or socket slower than data can be provided. Here, async IO does not help performance at all. I'm not even sure it would be suitable to unblock the UI thread since all tasks might complete synchronously.
Or it also saves cpu as well?
It generally increases CPU usage in real-world loads. This is because the async machinery adds processing, allocations and synchronization. Also, we need to transition to kernel mode two times instead of once (first to initiate the IO, then to dequeue the IO completion notification).
Typical workloads run with <<100% CPU. A production server with >60% CPU would worry me since there is no margin for error. In such cases the thread pool work queues are almost always empty. Therefore, there are no context switching savings caused by processing multiple IO completions on one context switch.
That's why CPU usage generally increases (slightly), except if the machine is very high on CPU load and the work queues are often capable of delivering a new item immediately.
On the server async IO is mainly useful for saving threads. If you have ample threads available you will realize zero or negative gains. In particular any single IO will not become one bit faster.
That means it does not consume cpu.
It would be a waste to leave the CPU unavailable while an IO is in progress. To the kernel an IO is just a data structure. While it's in progress there is no CPU work to be done.
An anonymous person said:
For IO-bound tasks there may not be a major performance advantage to using separate threads just to wait for a result.
Pushing the same work to a different thread certainly does not help with throughput. This is added work, not reduced work. It's a shell game. (And async IO does not use a thread while it's running so all of this is based on a false assumption.)
A simple way to convince yourself that async IO generally costs more CPU than sync IO is to run a simple TCP ping/pong benchmark sync and async. Sync is faster. This is kind of an artificial load so it's just a hint at what's going on and not a comprehensive measurement.
See this question and answer;
Why use async controllers, when IIS already handles the request concurrency?
Ok, a thread consumes more resources then the async/await construction, but why? What is the core difference? You still need to remember all state etc, don't you?
Why would a thread pool be limited, but can you have tons of more idle async/await constructions?
Is it because async/await knows more about your application?
Well, let's imagine a web-server. Most of his time, all he does is wait. it doesn't really CPU-bound usually, but more of I/O bound. It waits for network I/O, disk I/O etc. After every time he waits, he has something (usually very short to do) and then all he does is waiting again. Now, the interesting part is what happend while he waits. In the most "trivial" case (that of course is absolutely not production), you would create a thread to deal with every socket you have.
Now, each of those threads has it's own cost. Some handles, 1MB of stack space... And of course, not all those threads can run in the same time - so the OS scheduler need to deal with that and choose the right thread to run each time (which means A LOT of context switching). It will work for 1 clients. It'll work for 10 clients. But, let's imagine 10,000 clients at the same time. 10,000 threads means 10GB of memory. That's more than the average web server in the world.
All of these resources, is because you dedicated a thread for a user. BUT, most of this threads does nothing! they just wait for something to happen. and the OS has API for async IO that allows you to just queue an operation that will be done once the IO operation completed, without having dedicated thread waiting for it.
If you use async/await, you can write application that will easily use less threads, and each of the thread will be utilized much more - less "doing nothing" time.
async/await is not the only way of doing that. You could have done this before async/await was introduced. BUT, async/await allows you to write code that's very readable and very easy to write that does that, and look almost as it runs just on a single thread (not a lot of callbacks and delegates moving around like before).
By combining the easy syntax of async/await and some features of the OS like async I/O (by using IO completion port), you can write much more scalable code, without losing readability.
Another famous sample is WPF/WinForms. You have the UI thread, that all he does is to process events, and usually has nothing special to do. But, you can't block it or the GUI will hang and the user won't like it. By using async/await and splitting each "hard" work to short operations, you can achieve responsible UI and readable code. If you have to access the DB to execute a query, you'll start the async operation from the UI thread, and then you'll "await" it until it ends and you have results that you can process in the UI thread (because you need to show them to the user, for example). You could have done it before, but using async/await makes it much more readable.
Hope it helps.
Creating a new thread allocates a separate memory area exclusive for this thread holding its resources, mainly its call stack which in Windows takes up 1MB of memory.
So if you have a 1000 idle threads you are using up at least 1GB of memory doing nothing.
The state for async operations takes memory as well but it's just the actual size needed for that operation and the state machine generated by the compiler and it's kept on the heap.
Moreover, using many threads and blocking them has another cost (which IMO is bigger). When a thread is blocked it is taken out of the CPU and switched with another (i.e. context-switch). That means that your threads aren't using their time-slices optimally when they get blocked. Higher rate of context switching means your machine does more overhead of context-switching and less actual work by the individual threads.
Using async-await appropriately enables using all the given time-slice since the thread, instead of blocking, goes back to the thread pool and takes another task to execute while the asynchronous operation continues concurrently.
So, in conclusion, the resources async await frees up are CPU and memory, which allows your server to handle more requests concurrently with the same amount of resources or the same amount of requests with less resources.
The important thing to realize here is that a blocked thread is not usable to do any other work until it becomes unblocked. A thread that encounters an await is free to return to the threadpool and pick up other work until the value being awaited becomes available.
When you call a synchronous I/O method, the thread executing your code is blocked waiting for the I/O to complete. To handle 1000 concurrent requests, you will need 1000 threads.
When you call an asynchronous I/O method, the thread is not blocked. It initializes the I/O operation and can work on something else. It can be the rest of your method (if you don't await), or it can be some other request if you await the I/O method. The thread pool doesn't need to create new threads for new requests, as all the threads can be used optimally and keep the CPUs busy.
Async I/O operations are actually implemented asynchronously at the OS level.
I am getting really confused here about multithreading :(
I am reading about the C# Async/Await keywords. I often read, that by using this async feature, the code gets executed "non-blocking". People put code examples in two categories "IO-Bound" and "CPU-bound" - and that I should not use a thread when I execute io-bound things, because that thread will just wait ..
I dont get it... If I do not want a user have to wait for an operation, I have to execute that operation on another thread, right ?
If I use the Threadpool, an instance of "Thread"-class, delegate.BeginInvoke or the TPL -- every asynchronous execution is done on another thread. (with or without a callback)
What you are missing is that not every asynchronous operation is done on another thread. Waiting on an IO operation or a web service call does not require the creation of a thread. On Windows this is done by using the OS I/O Completion Ports.
What happens when you call something like Stream.ReadAsync is that the OS will issue a read command to the disk and then return to the caller. Once the disk completes the read the notifies the OS kernel which will then trigger a call back to your processes. So there is no need to create a new threadpool thread that will just sit and block.
What is meant is this:
Suppose you query some data from a database (on another server) - you will send a request and just wait for the answer. Instead of having a thread block and wait for the return it's better to register an callback that get's called when the data comes back - this is (more or less) what async/await does.
It will free the thread to do other things (give it back to the pool) but once your data come back asynchronously it will get another thread and continue your code at the point you left (it's really some kind of state-machine that handles that).
If your calculation is really CPU intensive (let's say you are calculating prime-numbers) things are different - you are not waiting for some external IO, you are doing heavy work on the CPU - here it's a better idea to use a thread so that your UI will not block.
I dont get it... If I do not want a user have to wait for an operation, I have to execute that operation on another thread, right ?
Not exactly. An operation will take however long it is going to take. When you have a single-user application, running long-running things on a separate thread lets the user interface remain responsive. At the very least this allows the UI to have something like a "Cancel" button that can take user input and cancel processing on the other thread. For some single-user applications, it makes sense to allow the user to keep doing other things while a long-running task completes (for example let them work on one file while another file is uploading or downloading).
For web applications, you do not want to block a thread from the thread pool during lengthy(ish) IO, for example while reading from a database or calling another web service. This is because there are only a limited number of threads available in the thread pool, and if they are all in use, the web server will not be able to accept additional HTTP requests.
Okay , let me try to put it in sentences ...
Lets consider an example,
where I create an async method and call it with await keyword,
As far as my knowledge tells me,
The main thread will be released
In a separate thread, async method will start executing
Once it is executed, The pointer will resume from last position It left in main thread.
Question 1 : Will it come back to main thread or it will be a new thread ?
Question 2: Does it make any difference if the async method is CPU bound or network bound ? If yes, what ?
The important question
Question 3 : Assuming that is was a CPU bound method, What did I achieve? I mean - main thread was released, but at the same time, another thread was used from thread pool. what's the point ?
async does not start a new thread. Neither does await. I recommend you read my async intro post and follow up with the resources at the bottom.
async is not about parallel programming; it's about asynchronous programming. If you need parallel programming, then use the Task Parallel Library (e.g., PLINQ, Parallel, or - in very complex cases - raw Tasks).
For example, you could have an async method that does I/O-bound operations. There's no need for another thread in this scenario, and none will be created.
If you do have a CPU-bound method, then you can use Task.Run to create an awaitable Task that executes that method on a thread pool thread. For example, you could do something like await Task.Run(() => Parallel...); to treat some parallel processing as an asynchronous operation.
Execution of the caller and async method will be entirely on the current thread. async methods don't create a new thread and using async/await does not actually create additional threads. Instead, thread completions/callbacks are used with a synchronization context and suspending/giving control (think Node.js style programming). However, when control is issued to or returns to the await statement, it may end up being on a different completion thread (this depends on your application and some other factors).
Yes, it will run slower if it is CPU or Network bound. Thus the await will take longer.
The benefit is not in terms of threads believe it or not... Asynchronous programming does not necessarily mean multiple threads. The benefit is that you can continue doing other work that doesn't require the async result, before waiting for the async result... An example is a web server HTTP listener thread pool. If you have a pool of size 20 then your limit is 20 concurrent requests... If all of these requests spend 90% of their time waiting on database work, you could async/await the database work and the time during which you await the database result callback will be freed... The thread will return to the HTTP listener thread pool and another user can access your site while the original one waits for the DB work to be done, upping your total limit.
It's really about freeing up threads that wait on externally-bound and slow operations to do other things while those operations execute... Taking advantage of built-in thread pools.
Don't forget that the async part could be some long-running job, e.g. running a giant database query over the network, downloading a file from the internet, etc.