I am using a WebApi service controller, hosted by IIS,
and i'm trying to understand how this architecture really works:
When a WebPage client is sending an Async requests simultaneously, are all this requests executed in parallel at the WebApi controller ?
At the IIS app pool, i've noticed the queue size is set to 1,000 default value - Does it mean that 1,000 max threads can work in parallel at the same time at the WebApi server?
Or this value is only related to ths IIS queue?
I've read that the IIS maintains some kind of threads queue, is this queue sends its work asynchronously? or all the client requests sent by the IIS to the WebApi service are being sent synchronously?
The queue size you're looking at specifies the maximum number of requests that will be queued for each application pool (which typically maps to one w3wp worker process). Once the queue length is exceeded, 503 "Server Too Busy" errors will be returned.
Within each worker process, a number of threads can/will run. Each request runs on a thread within the worker process (defaulting to a maximum of 250 threads per process, I believe).
So, essentially, each request is processed on its own thread (concurrently - at least, as concurrently as threads get) but all threads for a particular app pool are (typically) managed by a single process. This means that requests are, indeed, executed asynchronously as far as the requests themselves are concerned.
In response to your comment; if you have sessions enabled (which you probably do), then ASP.NET will queue the requests in order maintain a lock on the session for each request. Try hitting your sleeping action in Chrome and then your quick-responding action in Firefox and see what happens. You should see that the two different sessions allow your requests to be executed concurrently.
Yes, all the requests will be executed in parallel using the threads from the CLR thread pool subject to limits. About the queue size set against the app pool, this limit is for IIS to start rejecting requests with a 503 - Service unavailable status code. Even before this happens, your requests will be queued by IIS/ASP.NET. That is because threads cannot be created at will. There is a limit to number of concurrent requests that can run which is set by MaxConcurrentRequestsPerCPU and a few other parameters. For 1000 threads to execute in parallel in a true sense, you will need 1000 CPU cores. Otherwise, threads will need to be time sliced and that adds overhead to the system. Hence, there are limits to number of threads. I believe it is very difficult to comprehensively answer your questions through a single answer here. You will probably need to read up a little bit and a good place to start will be http://blogs.msdn.com/b/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx.
Related
I have a design question on how to best approach a process within an existing DotNet2 web service I have inherited.
At a high level the process at the moment is as follows
Client
User starts new request via web service call from client/server app
Request and tasks created and saved to database.
New thread created that begins processing the request
Request ID returned to client (client polls using ID).
Thread
Loads up request detail and the multiple tasks
For each task it requests XML via another service (approx 2 sec wait)
Passes XML to another service for processing (approx 3 sec wait)
Repeat until all tasks complete
Marks request completed (client will know its finished)
Overall this takes approximately 30 seconds for a request of 6 tasks. With each task being performed sequentially it is clearly inefficient.
Would it be better to break out each task again on a separate thread and then when they are all complete mark the request as completed?
My reservation is that I am immediately duplicating the number of threads by up to a factor of 6-10 (number of tasks) and concerned on how this would impact on IIS. I estimate that I could cut a normal 30 second call down to around 5 seconds if I had each task processing concurrently but under load would this design suffer?
The current design is operating well and users have no problem with the time taken to process but I would prefer it work faster if possible.
Is this just a completely bad design and if so is there a better approach? I am limited by the current DotNet version at present.
Thanks
If you are worried about IIS performance you probably want to keep the jobs outside of IIS, so IMO I would consider queueing the tasks and creating a separate service to do the work. This approach would be more scaleable in that you could add or remove front end IIS servers or task processors to address a varying load. A large-scale system would most certainly perform the processing off of the front end server.
I am trying to determine what will happen when my Web API methods are being called simultaneously by two clients. In order to do so I am generating two async requests in Python:
rs = (grequests.post(url, data = json), grequests.post(url, data = json))
grequests.map(rs)
The requests call a non-async method in my WebApi which is changing my resource, which is a static class instance. From what I've understood, API requests should not run concurrently, which is what I want since I don't want the two requests to interfere with eachother. However, if I set a break point in my API method, the two requests seem to run concurrently (in parallell). This leads to some unwanted results since the two requests are changing the same resource in parallell.
Is there any way to make sure that the Web API handles the requests non-concurrently?
Requests to your Web API methods are handled by the IIS application pool (pool of threads) which initializes a thread for every synchronous request it receives. There is no way to tell IIS to run these threads non-concurrently.
I believe you have a misunderstanding of what a "non-async" Web API method is. When a Web API method is async that means it will share its application pool thread while it's in a wait state. This has the advantage of other requests not having to initialize a new thread (which is somewhat expensive). It also helps minimize the number of concurrent threads which in turn minimizes the number of threads that the application pool has to queue up.
For non-async methods, the IIS application pool will initialize a new thread for every request even if an inactive thread is available. Also, it will not share that thread with any other requests.
So the short answer to your question is no. There is no way to make sure that the Web API requests are handled non-concurrently. Brian Driscoll's comment is correct. You will have to lock your shared resources during a request.
I have an aspx page which loads 10 charts asynchronously through Jquery Ajax requests. The requests are being made to Generic Handlers which implement IReadOnlySessionState since access to a session variable is required but it is a read and this way I am not affected by the read write session lock that asp.net implements.
Through the debugger I am able to see that calls are happening asynchronously but it seems that there is a limit as some of the calls are entering the code only after the first few have completed. I am not sure if this is by design on IIS or a property inside the web.config.
Is there a limit of threads that one user/session can have at one time?
The way IIS and asp.net handles threads depends on the version of IIS you are using. There is a limit to the number of workerthreads and there are caps on the number of threads that must be left available. This means that only a certain number of threads can execute at once.
See: http://blogs.msdn.com/b/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx
If your ASP.NET application is using web services (WFC or ASMX) or
System.Net to communicate with a backend over HTTP you may need to
increase connectionManagement/maxconnection. For ASP.NET
applications, this is limited to 12 * #CPUs by the autoConfig feature.
Also from the same article: http://support.microsoft.com/kb/821268
If you have long running HTTP requests from AJAX your best bet is to do asynchronous http request handlers. Then the requests can be waiting on an IO thread, since asp.net has a lot more IO threads than workerthreads.
See: http://www.asp.net/web-forms/tutorials/aspnet-45/using-asynchronous-methods-in-aspnet-45
I am trying to simulate X number of concurrent requests for a WCF Service and measure the response time for each request. I want to have all the requests hit the Service at more or less the same time.
As the first step I spawned X number of Threads, using the Thread class, and have invoked the Start method. To synchronize all the requests, on the Thread callback I open the connection and have a Monitor.Wait to hold the request from being fired, till all the Threads are created and started. Once all the Threads are started, I call Monitor.PulseAll to trigger the method invocation on the WCF Client Proxy.
When I execute the requests this way, I see a huge delay in the response. A request that should just a few milliseconds, is taking about a second.
I also noticed huge lag between the time the request is dispatched and the time it was received at the service method. I measured this by send sending client time stamp as a parameter value to the service method for each request.
I have the following settings. Assume "X" to the Concurrent number of requests I want to fire. Also note with the following settings I don't get any Denial of Service issues.
The Call chain is as follows,Client->Service1->Service2->Service3
All Services are PerCall with Concurrency set to Multiple.
Throttling set to X Concurrent calls, X Concurrent Instances.
MaxConnections, ListenBacklog on the Service to X.
Min/Max Threads of ThreadPool set to X on both Client and Server (I have applied the patch provided by Microsoft).
Am not sure if the response time I'm measuring is accurate. Am I missing something very trivial?
Any inputs on this would be of great help.
Thanks.
-Krishnan
I did find the answer by myself the hard way. All this while, the way I was measuring the response time was wrong. One should spawn X number of threads, where X is the number of concurrent users one wants to simulate. In each thread, open the connection only once and have while loop to only execute the WCF Method that you want to test for a given duration. Measure the response time against each return, accumulate it and average it out against the number of calls that were executed within the given duration.
If all your outgoing calls are coming from a single process, it is likely that the runtime is either consolidating multiple requests onto a single open channel or capping the number of concurrent requests to a single target service. You may have better results if you move each simulated client into its own process, use a named EventWaitHandle to synchronize them, and call #Set() to unleash them all at once.
There is also a limit to the number of simultaneous pending (TCP SYN or SYN/ACK state) outgoing TCP connections allowed by desktop builds of Windows; if you encounter this limit you will get event 4226 logged and the additional simultaneous connections will be deferred.
My knowledge of how processes are handled by the ASP.Net worker process is woefully inadequate. I'm hoping some of the experts out there can fill me in.
If I crash the worker process with a System.OutOfMemoryException, what would the user experience be for other users who were being served by the same process? Would they get a blank screen? 503 error?
I'm going to attempt to test this scenario with some other folks in our lab, but I thought I would float this out there. I will update with our results.
UPDATE: Our results varied. If we artificially induced a OOM exception (for example by loading larger and larger PDFs into memory), other threads being served by that worker process would "hang" temporarily and then complete, while others seemingly would never return. Thank you for your responses.
W3WP.exe is the process
IIS runs all web apps in a generic worker process - w3wp.exe. Whether you write in ASP.NET, or ISAPI, or some other framework, the process that serves the web request is w3wp.exe. In the ASP.NET case, w3wp.exe loads the ASP.NET JIT-compiled DLLs and services the requests through them. In other cases, it works differently. But the key point is, w3wp.exe is the process. This model started in IIS6.0 and continues in IIS7.0.
Unexpected Failures
If the W3WP.exe fails unexpectedly, for any reason, all transactions it was handling will likely get 500 errors (Server error). IIS will start a new worker process in its place (MS calls this "Health Monitoring"), which means the web app will continue to run. Users that did not have a request being served by the failing process at the time of failure, will be unaware of any of this.
The HTTP 500 error that a client receives in this case will be indistinguishable from a 500 error that the client receives in the case of an application error, let's say an uncaught exception in your ASPNET application code.
For those requests that were in the failing process, there's no way to recover them. They will result in 500 errors at the browser. A 503 Server Busy results from IIS actively refusing the connection due to a threshold on the number of connections. A 503 does not result from an application failure, so you shouldn't expect to see 503 for in-flight transactions in the out-of-memory-and-crash scenario. On a heavily loaded system, you may see 503's as the process-crash-and-restart happens, as a secondary effect. If this is really what you're seeing, you need a larger margin of safety to handle the load in the single-error condition.
The Request Queue
IIS has a hand-off approach for requests. As they arrive on the network layer (Http.sys), they are placed in a queue, to be picked up by a worker process. Any requests waiting in the IIS queue to be handled by a WP will continue unaffected, though they might see a slight temporary increase in latency (service time) due to resource contention, since one fewer process is running on the server. Wait time in this queue is generally very very short, on a system that is configured properly.
It is when this queue is full that you will see 503 errors.
Auto restart of W3WP.exe
IIS has an auto-restart (or "nanny") facility, through which it restarts worker processes after they have exceeded configured thresholds, such as memory size, number of requests, or time-of-running. In all those cases, IIS will quiesce and restart worker processes when the configured threshold is reached. These pro-active restarts normally do not result in any disruption of requests. When IIS decides that a restart of a worker process is necessary, it prevents any new requests from arriving at that to-be-quiesced WP. Existing requests are drained: any in-flight transactions in that WP are allowed to complete normally. When all requests in the WP complete, then the WP dies and IIS starts a new one in its place. This new process then immediately begins picking up new requests from the dispatch queue. This is all transparent to users or browsers.
I say normally because it's possible that the worker process has become truly sick at the same time as the threshold has been reached. In that case the w3wp.exe may not respond to IIS within the configured "quiesce" timeout, and thus IIS has to eventually kill the process even though it hasn't reported that all of its in-flight requests have completed. This should be exceedingly rare, because it's two distinct exceptional conditions, but it happens. In this case, the in-flight requests will once again, get 500 errors.
Web gardens
Also - IIS allows multiple worker processes on a single server. MS calls this a "web garden", a play on words from "web farm". If you have a web garden set up, then transactions being served by w3wp.exe instances other than the failing one, will continue unaffected. "Unaffected" presumes though, that the out-of-memory error is localized, and not a system-wide problem.
Bottom Line
The bottom line is that there is no substitute for your own testing. The configuration options are pretty broad - from restart thresholds to web gardens and so on. Also the failure modes tend to be pretty complex and varied, whether it's memory, timeout, too busy, and so on. You'll want to understand what to expect.
ps: this Q&A really belongs on serverfault.com !!
references:
http://blogs.iis.net/thomad/archive/2008/05/07/the-iis-process-model-features.aspx
A new worker thread will be started and the user would not know anything happened. Unless it shuts down completely via rapid fail (http://technet.microsoft.com/en-us/library/cc779127(WS.10).aspx)
If it's an out of memory situation, iis usually just recycles the app pool.
As the other answers say, in most cases everything just restarts, and most users who did not have a pending request at the time will not notice much more than a delay.
However, if your application uses session variables with In-Proc session state, all session variables for all users will be lost when the app pool restarts. This may or may not have a negative effect on the users, depending on what you're doing with the session variables. You can avoid this by switching to StateServer or SQL Server session storage.