In asp.net Web API how can I call long running blocking actions asynchronously?
I have a web api controller action that needs to make a relatively (10 seconds, as an example) long running DB call. This seems like a candidate for an asynchronous method. I can offload the long running task onto a new thread, and unblock my asp.net request thread to handle some other requests, so my simplified controller action would look like this:
public async Task<IHttpActionResult> Get()
{
IEnumerable<Thing> things = await Task.Run(() => DoLongDbCall());
return Ok(things);
}
I have encountered a couple of blogs (this one, for example) that suggest this may not be an optimal way to achieve this in asp.net. The author suggests using Task.FromResult() and doing the DB call synchronously, but i cant see how this helps; My request thread is still going to be blocked waiting for the DB call to return.
First, consider what happens with a synchronous call:
public IHttpActionResult Get()
{
IEnumerable<Thing> things = DoLongDbCall();
return Ok(things);
}
A request comes in, and ASP.NET grabs a thread pool thread to handle the request. This thread invokes the Get method, which does the work. One thread pool thread is used during the entire request.
Now, let's walk through what happens in the current code (using Task.Run):
public async Task<IHttpActionResult> Get()
{
IEnumerable<Thing> things = await Task.Run(() => DoLongDbCall());
return Ok(things);
}
A request comes in, and ASP.NET grabs a thread pool thread to handle the request. This thread invokes the Get method, which then grabs another thread pool thread to do the work and returns the original thread pool thread back to the thread pool. One thread pool thread is used during the entire request (and two thread pool threads are used for a very brief period of time).
So, the Task.Run code forces an extra thread transition without providing any benefit (the entire point of using async on the server side is to free up threads). This is why I recommend against using Task.Run (or any other way to run work on the thread pool) on ASP.NET.
A proper asynchronous solution would be to use asynchronous DB calls:
public async Task<IHttpActionResult> Get()
{
IEnumerable<Thing> things = await DoLongDbCallAsync();
return Ok(things);
}
A request comes in, and ASP.NET grabs a thread pool thread to handle the request. This thread invokes the Get method, which then starts the asynchronous operation and returns the thread pool thread back to the thread pool. Later, when the db call completes, ASP.NET grabs a thread pool thread to finish the request. For most of the request, no thread pool threads are used (one thread pool thread is used for a brief period of time at the beginning and end of the request).
my request thread is still going to be blocked waiting for the DB call to return.
That's not completely true:
Your request will still wait for the long operation to end, but your thread will be free to process other operations.
You must keep in mind that IIS has a reduced number of available threads (especially if running under non server systems) and freeing one that is not needed is always a good thing.
Moreover, if you have shortage of threads, and if you use Task.Run, the asynchronous operation will wait for an available thread, and if you don't release your current one, you'll end up with an horrible deadlock.
Related
I am trying to implement a task in fire and forget manner.
Lets look at the below piece of code.
public IHttpActionResult Update(int id)
{
var updatedResult = _updater.update(id);
// fire and forget a task
sendEmailToUser();
return ok();
}
private async Task sendEmailToUser()
{
var httpclient = new HttpClient();
// assume the client is initiated with required url and other headers
await httpclient.postasync("some url");
}
Given the above code, can i safely assume that whenever Update endpoint is called, sendEmailToUser task is triggered and will be run to completion ?
No. You should almost never start any background threads in web application. HTTP is suppose to be stateless and the web server was designed with that in mind.
The server might be put into sleep state when there is no incoming request for a set period of time. During that time all the background execution will be halt including the one you had. It might and might not get resume when the next request comes in.
Or when IIS decides to recycle your App domain on a scheduled basis your thread will get killed too.
If you really need background tasks then do that using windows service or run it as a separate console application.
Under normal conditions, it's reasonable to expect that the task will run to completion. It will go on independently.
Your biggest concerns, in this case, should be about the web API not being terminated, and the task not throwing an exception.
But if OP needs to be 100% sure, there are other safer ways to code that.
I'm trying to implement a functionality where there is a stored procedure on SQL Server that has to be called from ASP MVC application and processed on the background (it might take long since it calls another stored procedure remotely to process an excel file stored on a server). But the response of the last HTTP request should be returned back to client so the UI will not be hanging waiting for processing.
I have tried so many different ways but UI is still not responding right away.
I tried BackgroundWorker but it's not allowing the main thread to response back to client until its done processing,
also I tried:
Thread.QueueUserWorkItem(delegate { //method which performs stored procedure calls//
});
It still not returning response and HttpContext.Current not available in background thread.
Maybe there is a way to start background processing, pause it for letting main thread to return response to browser and then resume background thread to make all processing with stored procedure calls?
Am I missing something?
Could someone please give an idea how I can solve this problem? Would be much appreciated.
What I ended up with and it works fine in my case. I'm not sure about efficiency, but it perfectly works. So, the code which makes calls to a stored procedure I put on a separate thread, so that the main thread is finished while the processing off background calls is happening on a separate thread and finishes successfully after some period of time. At the same time UI is available so user can make another request which will also be processed the same way. I tested three requests. One after another overlapping, meaning that while first request was being processed on the background I submitted another one and yet another one. UI was responding immediately and all the work was done.
// ...main thread is working here
//put a call to stored procedure on a separate thread
Thread t = new Thread(()=> {
//call stored procedure which will run longer time since it calls another remote stored procedure and
//waits until it's done processing
});
t.Start();
// ...main thread continue to work here and finishes the request so it looks for user as the response is coming right away, all other stuff is being processed on that new thread so user even doesn't suspect
I shall quote Stephan Clearys great article:
When you use async on the server side (e.g., with ApiController), then you can treat each web request as an asynchronous operation. But when you yield, you only yield to the web server thread pool, not to the client. HTTP only allows a single response, so the response can only be sent when the request is fully complete.
Basically, this doesn't adhere to the HTTP protocol, where each request has only one response.
This can be achieved using multiple calls to the ASP.NET service, where a request returns a unique ID immediately, which the client can query multiple times for progress. You may look into SignalR for help with such an implementation:
What is SignalR and "real-time web" functionality? It's the ability to have your server-side code push content to the connected clients as it happens, in real-time.
There is part1 and part2 article by Dino Esposito outlines a way to achieve your polling using a client side timer and controller actions. You would basically serialize access to a progress worker controller method that returns task status and completion data. However, it may be a little chatty if you are only going to be performing one or two long running processes.
If the response to the client does not depend on the result of the background process (i.e. run the process in the background and don't have the UI waiting for it), then you could use Revalee (an open-source tool) to perform the background task.
The UI will request this route...
public class ForegroundController : Controller
{
public ActionResult InitiateBackgroundTask()
{
// The absolute URL that will be requested on the callback.
var callbackUri = new Uri("http://localhost/BackgroundTask/Callback");
// The information that will be needed to initiate the background task.
object state = "Any object";
this.CallbackNowAsync(callbackUri, state)
// ... your controller will now return a response to the browser
return View();
}
}
Background work will be executed in this controller...
public class BackgroundTaskController : Controller
{
[AllowAnonymous]
[HttpPost]
[CallbackAction]
public ActionResult Callback(Guid callbackId, object state)
{
// Perform the background work
// ... insert your background task code here ...
// Return a status code back to the Revalee Service.
return new HttpStatusCodeResult(HttpStatusCode.OK);
}
}
Revalee Project Site
EDIT I realised my question was not stated clearly enough and have edited it heavily.
This is a bit of an open ended question so apologies in advance.
In a nutshell, I want to implement IIS-style asynchronous request processing in an Azure worker role.
It may be very simple or it may be insanely hard - I am looking for pointers to where to research.
While my implementation will use Azure Workers and Service Bus Queues, the general principle is applicable to any scenario where a worker process is listening for incoming requests and then servicing them.
What IIS does
In IIS there is a fixed-size threadpool. If you deal with all request synchronously then the maximum number of requests you can deal with in parallel == maxthreads. However, if you have to do slow external I/O to serve requests then this is highly inefficient because you can end up with the server being idle, yet have all threads tied up waiting for external I/O to complete.
From MSDN:
On the Web server, the .NET Framework maintains a pool of threads that are used to service ASP.NET requests. When a request arrives, a thread from the pool is dispatched to process that request. If the request is processed synchronously, the thread that processes the request is blocked while the request is being processed, and that thread cannot service another request.
This might not be a problem, because the thread pool can be made large enough to accommodate many blocked threads. However, the number of threads in the thread pool is limited. In large applications that process multiple simultaneous long-running requests, all available threads might be blocked. This condition is known as thread starvation. When this condition is reached, the Web server queues requests. If the request queue becomes full, the Web server rejects requests with an HTTP 503 status (Server Too Busy).
In order to overcome this issue, IIS has some clever logic that allows you to deal with requests asynchronously:
When an asynchronous action is invoked, the following steps occur:
The Web server gets a thread from the thread pool (the worker thread) and schedules it to handle an incoming request. This worker thread initiates an asynchronous operation.
The worker thread is returned to the thread pool to service another Web request.
When the asynchronous operation is complete, it notifies ASP.NET.
The Web server gets a worker thread from the thread pool (which might be a different thread from the thread that started the asynchronous operation) to process the remainder of the request, including rendering the response.
The important point here is when the asynchronous request returns, the return action is scheduled to run on one of the same pool of threads that serves the initial incoming requests. This means that the system is limiting how much work it is doing concurrently and this is what I would like to replicate.
What I want to do
I want to create a Worker role which will listen for incoming work requests on Azure Service Bus Queues and also potentially on TCP sockets. Like IIS I want to have a maxium threadpool size and I want to limit how much actual work the worker is doing in parallel; If the worker is busy serving existing requests - whether new incoming ones or the callbacks from previous async calls - I don't want to pick up any new incoming requests until some threads have been freed up.
It is not a problem to limit how many jobs I start concurrently - that is easy to control; It is limiting how many I am actually working on concurrently.
Let's assume a threadpool of 100 threads.
I get 100 requests to send an email come in and each email takes 5 seconds to send to the SMTP server. If I limit my server to only process 100 requests at the same time then my server will be unable to do anything else for 5 seconds, while the CPU is completely idle. So, I don't really mind starting to send 1,000 or 10,000 emails at the same time, because 99% of the "request process time" will be spent waiting for external I/O and my server will still be very quiet.
So, that particular scenario I could deal with by just keeping on accepting incoming requests with no limit (or only limit the start of the request until I fire off the async call; as soon as the BeginSend is called, I'll return and start serving another request).
Now, imagine instead that I have a type of request that goes to the database to read some data, does some heavy calculation on it and then writes that back to the database. There are two database requests there that should be made asynchronous but 90% of the request processing time will be spent on my worker. So, if I follow the same logic as above and keep start async calls and just letting the return do whatever it needs to get a thread to continue on then I will end up with a server that is very overloaded.
Somehow, what IIS does is make sure that when an async call returns it uses the same fixed-size thread pool. This means that if I fire off a lot of async calls and they then return and start using my threads, IIS will not accept new requests until those returns have finished. And that is perfect because it ensures a sensible load on the server, especially when I have multiple load-balanced servers and a queue system that the servers pick work from.
I have this sneaky suspicion that this might be very simple to do, there is just something basic I am missing. Or maybe it is insanely hard.
Creating a threadpool should be considered as independent of Windows Azure. Since a Worker Role instance is effectively Windows 2008 Server R2 (or SP2), there's nothing really different. You'd just need to set things up from your OnStart() or Run().
One thing you wanted to do was use queue length as a determining factor when scaling to more/less worker instances. Note that Service Bus Queues don't advertise queue length, where Windows Azure Queues (based on Storage, vs. Service Bus) do. With Windows Azure Queues, you'll need to poll synchronously for messages (whereas Service Bus Queues have long-polling operations). Probably a good idea to review the differences between Service Bus Queues and Windows Azure Queues, here.
Have you considered having a dedicated WCF instance (not WAS or IIS hosted) to buffer the long running requests? It will have its own dedicated app pool, with a separate Max value setting from IIS that won't contend with your ASP.NET HTTP requests. (HTTP requests are served by
Then use IIS Async methods to call WCF with the constrained app pool.
I've used the SmartThreadPool project in the past as a per-instance pool and, if I'm reading you correctly, it should have all the callback and worker-limiting functionality you need. My company actually has it running currently on Azure for the exact purpose you describe of reading message bus requests asynchronously.
I have been digging around in this and found that it is indeed relatively easy.
http://www.albahari.com/threading/ has got some good information and I actually ended up buying the book which that website is essentially promoting.
What I found out is that;
Your application has a ThreadPool available to it by default
You can limit the number of threads available in the ThreadPool
When you use QueueUserWorkItem or Task.Factory.StartNew the job you start run on a Thread in the ThreadPool
When you use one of the asynchronous IO calls in the framework (Begin... methods or WebcClient.DownloadStringAsync etc) the the callbacks will also run on a Thread from the ThreadPool (what happens with the IO request itself is outside the scope of this discussion).
So far, so good. The problem is that I can keep calling Task.Factory.StartNew as much as I like and the ThreadPool will simply queue up the work until there are free threads to service them. So, in the case of an Azure Worker, I could easily empty the Queue even though my worker is busy servicing existing requests (and callbacks from existing requests). That is the core of my problem. What I want is to not take anything out of the queue until I actually have some free threads to service the request.
This is a very simple example of how this could be achieved. In essence, I am using an AutoResetEvent to make sure that I don't start another task from the queue until the previous task has actually started. Granted, I do actually take stuff out of the queue before there is a free thread, but on balance this should avoid crazy overloads of the worker and allow me to spin up more workers to share the load.
ThreadPool.SetMaxThreads(5, 1000); // Limit to 5 concurrent threads
ThreadPool.SetMinThreads(5, 10); // Ensure we spin up all threads
var jobStart = new AutoResetEvent(true);
// The "listen" loop
while (true)
{
var job = this.jobQueue.Dequeue();
jobStart.WaitOne(); // Wait until the previous job has actually been started
Task.Factory.StartNew(
() =>
{
jobStart.Set(); // Will happen when the threadpool allocates this job to a thread
this.Download(job);
});
}
This can - and probably should - be made a lot more sophisticated, including having timeouts, putting the work item back in the queue if a thread can't be allocated within a reasonable time and so on.
An alternative would be to use ThreadPool.GetAvailableThreads to check if there are free threads before starting to listen to the queue but that feels rather more error prone.
Somehow, what IIS does is make sure that when an async call returns
it uses the same fixed-size thread pool.
This is not true: When your code runs in response to an HTTP-Request you decide on what threads the continuation function executes. Usually, this is the thread pool. And the thread pool is an appdomain-wide resource that is shared among all requests.
I think IIS does less "magic" than you think it does. All it does is to limit the number of parallel HTTP-requests and the backlog size. You decide what happens once you have been given control by ASP.NET.
If your code is not protected against overloading the server, you will overload the server even on IIS.
From what I understand you want to constrain the number of threads used for processing a certain type of message at the same time.
One approach would be to simply wrap the message processor, invoked on a new thread with something like
try
{
Interlocked.Increment(ref count)
Process(message);
}
finally
{
Interlocked.Decrement(ref count)
}
Before invoking the wrapper, simply check if the ‘count’ is less than your threshold count; and stop polling/handling more messages till the count is sufficiently lower.
EDIT Added more information based on comment
Frans, not sure why you see the infrastructure and business code being coupled. Once you place your business process to be serviced as a task on a new thread to run asynchronously, you need not worry about performing additional IO bound calls asynchronously. This is a simpler model to program in.
Here is what I am thinking.
// semi - pseudo-code
// Infrastructure – reads messages from the queue
// (independent thread, could be a triggered by a timer)
while(count < maxCount && (message = Queue.GetMessage()) != null)
{
Interlocked.Increment(ref count);
// process message asynchronously on a new thread
Task.Factory.StartNew(() => ProcessWrapper(message));
}
// glue / semi-infrastructure - deals with message deletion and exceptions
void ProcessWrapper(Message message)
{
try
{
Process(message);
Queue.DeleteMessage(message);
}
catch(Exception ex)
{
// Handle exception here.
// Log, write to poison message queue etc ...
}
finally
{
Interlocked.Decrement(ref count)
}
}
// business process
void Process(Message message)
{
// actual work done here
;
}
We are dealing with processing and uploading of large files in windows service (.net 4.0). Process and upoload steps can take minutes to complete. Admin has the ability to mark job as cancelled, directly in database but to clear in memory queue it requires service restart. Goal is to abandon that job and pick next job in queue without service restart. Here's what I want to do:
In main entry point, start two tasks:
Task processTask = Task.Factory.StartNew(ProcessJob);
Task monitorTask = Task.Factory.StartNew(MonitorDB);
ProcessJob would call multiple long running steps like ProcessFile, UploadFile. We are checking for job status between the steps but job may be stuck in one of these long running steps.
If monitorTask detects job status change in DB, it should communicate that to main thread (through exception or message), so that main thread can quit, removing itself from the processing queue and allow next job in queue to start. Without Wait cannot get to exception but cannot wait because need to run these tasks in parallel.
At this time we are not concerned with the fact that some synch step in ProcessJob may still be going on and it may eventually complete. We would handle that in code.
So far in all of my applications, I have used Task.ContinueWith for success and failure, but never had to communicate back to main thread.
You could communicate between the monitor thread and the main thread through a BlockingCollection<T>.
Here's a simple example I wrote.
BlockingCollection<string> queue =
new BlockingCollection<string>();
// monitor thread.
Task.Factory.StartNew(() =>
{
while (true)
{
Thread.Sleep(1000);
queue.Add("event occured.");
}
});
// main thread.
while (true)
{
// blocks when no messages are in queue.
string message = queue.Take();
// kill process thread here.
}
You can use a CancellationToken for this. If you look at the Task class, there is an overload where you can pass in a CancellationToken. Store a reference to this token in your service and simply have your upload/process routine periodically check if the token has been cancelled like so:
if (yourToken.IsCancellationRequested)
break;
I know AsyncPageTask creates a thread which we can use for long running processes, i was wondering where this thread is created as i am sure it does not use IIS request processing thread, then where this thread exist ?
please can someone explain ? thanks in advance
The thread used for your long running process comes from I/O thread pool (request thread is then returned to the request thread pool). When all your PageAsyncTasks finish, ASP.Net gets back a request thread from the request thread pool.
So the workflow is basically this: request thread -> any number of I/O threads -> request thread.