I am trying to determine what will happen when my Web API methods are being called simultaneously by two clients. In order to do so I am generating two async requests in Python:
rs = (grequests.post(url, data = json), grequests.post(url, data = json))
grequests.map(rs)
The requests call a non-async method in my WebApi which is changing my resource, which is a static class instance. From what I've understood, API requests should not run concurrently, which is what I want since I don't want the two requests to interfere with eachother. However, if I set a break point in my API method, the two requests seem to run concurrently (in parallell). This leads to some unwanted results since the two requests are changing the same resource in parallell.
Is there any way to make sure that the Web API handles the requests non-concurrently?
Requests to your Web API methods are handled by the IIS application pool (pool of threads) which initializes a thread for every synchronous request it receives. There is no way to tell IIS to run these threads non-concurrently.
I believe you have a misunderstanding of what a "non-async" Web API method is. When a Web API method is async that means it will share its application pool thread while it's in a wait state. This has the advantage of other requests not having to initialize a new thread (which is somewhat expensive). It also helps minimize the number of concurrent threads which in turn minimizes the number of threads that the application pool has to queue up.
For non-async methods, the IIS application pool will initialize a new thread for every request even if an inactive thread is available. Also, it will not share that thread with any other requests.
So the short answer to your question is no. There is no way to make sure that the Web API requests are handled non-concurrently. Brian Driscoll's comment is correct. You will have to lock your shared resources during a request.
Related
I'm working at an automation firm so we create processes for industrial automation. Previously this automation was done on the machine side of things, but we're slowly transitioning to controlling the machines with c#.
On my current project the production for one day takes about 2 hours. The operators of the factory have a web interface that we created in c# using asp.net core MVC in which they can start/pause/stop this production process.
When starting the process we await a function in our controller that is basically a while loop that controls this 2h long production process.
The problem is now that when I send out the REST request to start the production this request takes 2h to complete, I would prefer this request immediately completes and the production process starts on the background of my asp.net core application.
First I thought I could just leave out the await and simply do this in my controller (simplified code):
_ = _productionController.StartLongProcess(); // This contains the while loop
return Ok();
But since _productionController is scoped and all its dependencies are as well, these immediately get disposed of when the method returns and I can't access my database anymore for example.
The process should be able to continuously talk to our database to save information about the production process in case something fails, that way we can always pick off where we left off.
My question to you is now, are we tackling this the wrong way? I imagine it's bad practice to start these long running processes in the asp.net controller.
How do I make sure I always have access to my DatabaseContext in this long running process even though the REST request has already ended. Create a separate scope only for this method?
Starting ASP.NET Core 2.1, the right way to do this (within asp.net) is to extend BackgroundService (or implement IHostedService).
Incoming requests can tell the background service to start the long-running operation and return immediately. You'll of course need to handle cases where duplicate requests are sent in, or new requests are sent in before the existing request is completed.
The documentation page has an example where the BackgroundService reads commands of a queue and processes them one at a time.
How do I make sure I always have access to my DatabaseContext in this long running process even though the REST request has already ended. Create a separate scope only for this method?
Yes, create a separate scope.
My question to you is now, are we tackling this the wrong way? I imagine it's bad practice to start these long running processes in the asp.net controller.
We've done something similar in the past. As long as fault-tolerance (particularly w.r.t. app restarts) and idempotence are built into the long-running-operation's logic, you should be good to go.
REST requests are expected to be short, a few seconds at maximum.
So best practice here would be to offload a long running task to a background service and return a token where you can poll the service if the operation has already finished.
The background service could be a BackGroundWorker in Net Core. This is easy but not really fault tolerant, so some sort of db and retry logic could be good.
If you are in an intranet, you could also move to an inherently asynchronous protocol like RabbitMQ, where you send a StartOperation Message and then receive a Started Message when the process has completed.
Another option would be to use Hangfire. It will allow you to Enqueue the work that you want to execute to a persistent store e.g. SQL Server, MSMQ, Redis depending on what you have in your infrastructure. The job will then be picked up by a worker which can also run in the ASP.NET process or a windows service. It's distributed too so you can have a number of instances of the workers running. Also supports retrying failed jobs and has a dashboard to view the jobs. Best of all, it's free!
var jobId = BackgroundJob.Enqueue(() => ExecuteLongRunningProcess(parameter1));
https://www.hangfire.io/
Following is my understanding of the issue that you have posted:
You want to initiate a long running call, via Rest api call
You want to use the Async call, but not sure how to maintain the DB context for a long running call which is used for db communication on regular basis during the operation
Couple of important points:
Mostly you are not clear regarding working of the Async calls
When you make an Async call, then it stores the current thread synchronization context for the continuity using state machine, it doesn't block any thread pool thread, it utilize the hardware based concurrency
Can use ConfigureAwait(false) on backend to avoid explicit reentry in the current synchronization context, which is better for performance
Only challenge with Async calls to be genuine async the complete chain need to be Async enabled from the entry point, else the benefits can't be reaped, if you use Task.Wait or Task.Result anywhere, infact may even cause a deadlock in the ASP.Net
Regarding the long running operation, following are the options
A Simple async call as suggested above, though it can help you can make large number of async calls (thus scalability) but context will be lost if the client goes away and no way to reap the status of operation back
You can make a fire and forget call, and use a mechanism like ASP.Net SignalR, which is like IObservable over the network and can help in notifying the client when the processing finish
Best option would be using a messaging queue like Rabbit MQ, which doesn't run the risk of client going down, it acts a producer consumer and can notify when the client comes up, in this case MQ can be notified when the process finish and thus client can be informed. MQ can be used for both incoming and response message in an async manner
In case, where client wants to periodically come up and check the status of the request, then DB persistence is important, which can be updated at regular intervals and it can be checked what's the status of the long running process.
My question to you is now, are we tackling this the wrong way? I imagine it's bad practice to start these long running processes in the asp.net controller.
Generally, yes. Ideally an ASP.NET service does not have any long-running processes inside it - or at the very least, no long-running processes that can't be easily and quickly shut down.
Doing work outside of an HTTP request (i.e., request-extrinsic code) is most reliably achieved by adding a durable queue with a separate background processor. The "background processor" can be a Win32 service, possibly even on the same machine. With this architecture, the HTTP request places a request on the queue, and the processor picks up requests from the queue and executes them.
I am using a WebApi service controller, hosted by IIS,
and i'm trying to understand how this architecture really works:
When a WebPage client is sending an Async requests simultaneously, are all this requests executed in parallel at the WebApi controller ?
At the IIS app pool, i've noticed the queue size is set to 1,000 default value - Does it mean that 1,000 max threads can work in parallel at the same time at the WebApi server?
Or this value is only related to ths IIS queue?
I've read that the IIS maintains some kind of threads queue, is this queue sends its work asynchronously? or all the client requests sent by the IIS to the WebApi service are being sent synchronously?
The queue size you're looking at specifies the maximum number of requests that will be queued for each application pool (which typically maps to one w3wp worker process). Once the queue length is exceeded, 503 "Server Too Busy" errors will be returned.
Within each worker process, a number of threads can/will run. Each request runs on a thread within the worker process (defaulting to a maximum of 250 threads per process, I believe).
So, essentially, each request is processed on its own thread (concurrently - at least, as concurrently as threads get) but all threads for a particular app pool are (typically) managed by a single process. This means that requests are, indeed, executed asynchronously as far as the requests themselves are concerned.
In response to your comment; if you have sessions enabled (which you probably do), then ASP.NET will queue the requests in order maintain a lock on the session for each request. Try hitting your sleeping action in Chrome and then your quick-responding action in Firefox and see what happens. You should see that the two different sessions allow your requests to be executed concurrently.
Yes, all the requests will be executed in parallel using the threads from the CLR thread pool subject to limits. About the queue size set against the app pool, this limit is for IIS to start rejecting requests with a 503 - Service unavailable status code. Even before this happens, your requests will be queued by IIS/ASP.NET. That is because threads cannot be created at will. There is a limit to number of concurrent requests that can run which is set by MaxConcurrentRequestsPerCPU and a few other parameters. For 1000 threads to execute in parallel in a true sense, you will need 1000 CPU cores. Otherwise, threads will need to be time sliced and that adds overhead to the system. Hence, there are limits to number of threads. I believe it is very difficult to comprehensively answer your questions through a single answer here. You will probably need to read up a little bit and a good place to start will be http://blogs.msdn.com/b/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx.
I have an aspx page which loads 10 charts asynchronously through Jquery Ajax requests. The requests are being made to Generic Handlers which implement IReadOnlySessionState since access to a session variable is required but it is a read and this way I am not affected by the read write session lock that asp.net implements.
Through the debugger I am able to see that calls are happening asynchronously but it seems that there is a limit as some of the calls are entering the code only after the first few have completed. I am not sure if this is by design on IIS or a property inside the web.config.
Is there a limit of threads that one user/session can have at one time?
The way IIS and asp.net handles threads depends on the version of IIS you are using. There is a limit to the number of workerthreads and there are caps on the number of threads that must be left available. This means that only a certain number of threads can execute at once.
See: http://blogs.msdn.com/b/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx
If your ASP.NET application is using web services (WFC or ASMX) or
System.Net to communicate with a backend over HTTP you may need to
increase connectionManagement/maxconnection. For ASP.NET
applications, this is limited to 12 * #CPUs by the autoConfig feature.
Also from the same article: http://support.microsoft.com/kb/821268
If you have long running HTTP requests from AJAX your best bet is to do asynchronous http request handlers. Then the requests can be waiting on an IO thread, since asp.net has a lot more IO threads than workerthreads.
See: http://www.asp.net/web-forms/tutorials/aspnet-45/using-asynchronous-methods-in-aspnet-45
In WebForms ASP.NET site (IIS, single app pool), I have call to lengthy web service method referenced in Visual Studio as Service Reference (.NET 4.0). Unfortunately I must wait for information from web service before I can serve page to user. Currently web service is called synchronously so server can’t reuse current thread to process other requests which has performance impact.
Of course I can generate asynchronous operations for service reference in Visual Studio and call BeginGetFoo instead of GetFoo, but still I must wait somehow for result from web service.
Here comes question. If I use AsyncWaitHandle.WaitOne (as below) will it be any better in whole application performance terms from synchronous call I use today?
IAsyncResult result = fooSoapClient.BeginGetFoo();
result.AsyncWaitHandle.WaitOne();
var foo = fooSoapClient.EndGetFoo(result);
And of course, if waiting can be done better I am open for suggestions.
You want to use an Asynchronous Page. See "Wicked Code: Scalable Apps with Asynchronous Programming in ASP.NET", also Asynchronous Pages in ASP.NET 2.0, which talks about web services and Asynchronous Tasks with RegisterAsyncTask.
You'd still be hogging the thread. A "safe" option would be to use ASP.NET MVC's async controllers: http://www.aaronstannard.com/post/2011/01/06/asynchonrous-controllers-ASPNET-mvc.aspx
Ideally though, you shouldn't do long running things on a web request. Have a windows service or something process the long running task (that could get kicked off by a web request dropping something on a message queue or putting a task in a database) and poll from the client using ajax or something and then update the user when it's done.
If refactoring your code is not acceptable so you cannot follow #John Saunders's answer then the only thing you can do is increase the number of threads for the application. This will allow you to scale better but at some point it will have diminishing returns and you will start hurting the performance. What is more if you do not have users waiting on the request queue (i.e. more than 25 simultaneous users per core on your server) you don't need to do anything. Async programming in the web server helps only with scalability but not actual performance for a single user.
I have a quick simply question about requests in WCF. Does WCF automatically queue requests to a service(the service being a singleton) when multiple users request the same process, ie lets say I have a function that takes a while to complete, and two users make a call to this function, does WCF automatically queue the requests so that when the first request is finished it then starts processing the next?
~Just Wondering
The service behavior attribute on the contract defines how sessions, instances and concurrency are handled. See http://msdn.microsoft.com/en-us/library/ms731193.aspx for more details.
Basically you can configure it (1) handle one request at a time or (2) multiple requests at the same time.