I have a WCF service set to PerCall
I would like to know how I can send a Start call from the client to start a long running process, and send a Cancel command to cancel it
My WCF service looks something like this
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
public class Service1 : IService1
{
CancellationTokenSource cancelToken = new CancellationTokenSource();
public void Start()
{
var compute = Task.Factory.StartNew(StartLongRunningTask, cancelToken.Token);
}
public void Stop()
{
cancelToken.Cancel();
}
private void StartLongRunningTask()
{
//process here
}
}
I guess the problem here is that, each time a call comes to the server, it's treated as a new request.
So how should starting and cancelling a long running task in WCF be done?
EDIT: I'm hosting it as a windows service
I have a WCF service set to PerCall
... the problem here is that, each time a call comes to the server, it's treated as a new request.
Yup, that's exactly what you're telling it to do. If you can, just change to InstanceContextMode.PerSession; then you can do what you're trying to do (assuming you're self-hosting).
If you can't do this, then you'll have to develop a more complex solution like #PeterRitchie commented. First, your host: IIS is not designed to have long-running operations independent of requests, so I'll assume you're self-hosting. Next, you'll need a form of token (like a GUID) that will act as an identifier for a long-running operation. Your Start method will allocate a GUID and CancellationTokenSource and start the operation, and your Stop method will take a GUID and use that to look up the CancellationTokenSource and cancel the operation. You'll need a shared (static, threadsafe) dictionary to act as lookup.
If your host is IIS, then your solution gets more complex... :)
First, you'll need a backend that's not hosted in IIS. Common choices are an Azure worker role or a Win32 service. Next, you'll need a reliable communications mechanism: an Azure queue, MSMQ, WebSphere, etc. Then you can build your WCF-over-IIS service to have the Start method generate a GUID identifier and drop a message on the queue to start processing. The Stop method takes the GUID and drops a message on the queue to cancel processing. All other logic gets moved to the backend service.
From how you've asked, the client seems to be aware of the async nature of the request.
#StephenCleary and #PeterRitchie's points are excellent, but your first step is to re-do your service/contract to properly implement an async service and add the means of communicating back (to client) some information/handle to the long running operation.
The Framework contains several paradigms for asynchronous programming (already :-) )but when it comes to WCF, you kinda fall back to How to: Implement an Asynchronous Service Operation
That will provide some infrastructure, but not necessarily the ability to automatically cancel an operation.
Speaking strictly about the cancellation (as this is your question): you will have to extend whatever your solution ends up being for cancellation. At the minimum you need to add necessary logic to your service “worker” to monitor and honor the cancellation token.
Other considerations that you may expect to encounter: return result from cancellation; cancelling a task that has managed to complete (what of you updated the 1,000,000 records by the time the cancellation request came); exception handling (with task-based programming exceptions are not thrown, but bundled in the Task, or whatever other “vehicle” you use to describe the ongoing operation).
Related
We are implementing a .NET core 3.1 worker service and we need to register several hosted service to implement the required business.
This documentation explains that the StopAsync method of the IHostedService interface should be able to listen to cancellation requests issued after a default 5 seconds timeout.
Based on my understanding, the StopAsync method is called by the host during the shutdown process of the host itself.
In one of our hosted services we need to call an async method during the hosted service stop process; unfortunately this method does not support cancellation, so there is not a simple way for us to implement the StopAsync method in the proper way.
This is our scenario:
public sealed class MyHostedService : IHostedService
{
public async Task StopAsync(CancellationToken token)
{
_logger.LogInformation("Stopping the service....");
await DoSomethingAsync(); // this method does not support cancellation, so I can't pass the cancellation token here
_logger.LogInformation("Service was stopped");
}
}
What's going to happen at the host level in this scenario ?
Is this implementation "dangerous" in any sort ?
Is this going to block the host shutdown process in case of a long time needed to complete the execution of StopAsync ?
The simplest solution to this issue is implementing the class MyHostedService by deriving from the BackgroundService base class, but I would like to fully understand the possible implications of a long running StopAsync implementation at the host level.
Hosted Services are stopped in the reverse order from which they were started (which is also the order you add them to the ServiceCollection). You can extend the shutdown timeout by specifying HostOptions.ShutdownTimeout which otherwise defaults to five seconds:
services.Configure<HostOptions>(s => s.ShutdownTimeout = TimeSpan.FromMinutes(5));
When the Host is stopping its hosted services it does so in a loop, running over the services in reverse order. Once StopAsync is called for a particular service, the host will wait for StopAsync to return. So if that one service doesn't respect cancellation, the host will take a while to shutdown.
The host also calls ThrowIfCancellationRequested() in that loop prior to calling StopAsync on each service. What this means is that if the timeout elapses while the first service is stopping, it will finish completing* but before the next service's StopAsync is called an exception will be thrown. This will cause the shutdown process to abort, and for the process to exit ungracefully.
This abort can also happen after all services have been stopped if the timeout elapses in the very next instant (there is another ThrowIfCancellationRequested immediately after the loop). Anything bound to the Host/Application lifetimes will be skipped.
Moral of the story is to either ensure your services can stop quickly, don't require your services to need graceful termination, or increase the timeout. You can also make sure you add the services to the collection in such a way that the more important or quicker ones get stopped first (by adding them last) and your long running service gets stopped last (by adding it first).
* Assuming the service doesn't respect cancellation
Based on generic and web hosts shutdown timeout docs the host will just stop corresponding services ungracefully:
If the timeout period expires before all of the hosted services stop, any remaining active services are stopped when the app shuts down. The services stop even if they haven't finished processing. If services require additional time to stop, increase the timeout.
So long running StopAsync should not prevent the host from proceeding with shutdown. The only bad thing that can happen is that your DoSomethingAsync will be interrupted in some invalid state, if that is possible.
I am trying to implement a task in fire and forget manner.
Lets look at the below piece of code.
public IHttpActionResult Update(int id)
{
var updatedResult = _updater.update(id);
// fire and forget a task
sendEmailToUser();
return ok();
}
private async Task sendEmailToUser()
{
var httpclient = new HttpClient();
// assume the client is initiated with required url and other headers
await httpclient.postasync("some url");
}
Given the above code, can i safely assume that whenever Update endpoint is called, sendEmailToUser task is triggered and will be run to completion ?
No. You should almost never start any background threads in web application. HTTP is suppose to be stateless and the web server was designed with that in mind.
The server might be put into sleep state when there is no incoming request for a set period of time. During that time all the background execution will be halt including the one you had. It might and might not get resume when the next request comes in.
Or when IIS decides to recycle your App domain on a scheduled basis your thread will get killed too.
If you really need background tasks then do that using windows service or run it as a separate console application.
Under normal conditions, it's reasonable to expect that the task will run to completion. It will go on independently.
Your biggest concerns, in this case, should be about the web API not being terminated, and the task not throwing an exception.
But if OP needs to be 100% sure, there are other safer ways to code that.
I am developing a web-api that takes data from client, and saves it for later use. Now i have an external system that needs to know of all events, so i want to setup a notification component in my web-api.
What i do is, after data is saved, i execute a SendNotification(message) method in my new component. Meanwhile i don't want my client to wait or even know that we're sending notifications, so i want to return a 201 Created / 200 OK response as fast as possible to my clients.
Yes this is a fire-and-forget scenario. I want the notification component to handle all exception cases (if notification fails, the client of the api doesn't really care at all).
I have tried using async/await, but this does not work in the web-api, since when the request-thread terminates, the async operation does so aswell.
So i took a look at Task.Run().
My controller looks like so:
public IHttpActionResult PostData([FromBody] Data data) {
_dataService.saveData(data);
//This could fail, and retry strategy takes time.
Task.Run(() => _notificationHandler.SendNotification(new Message(data)));
return CreatedAtRoute<object>(...);
}
And the method in my NotificationHandler
public void SendNotification(Message message) {
//..send stuff to a notification server somewhere, syncronously.
}
I am relatively new in the C# world, and i don't know if there is a more elegant(or proper) way of doing this. Are there any pitfalls with using this method?
It really depends how long. Have you looked into the possibility of QueueBackgroundWorkItem as detailed here. If you want to implement a very fast fire and forget you also might want to consider a queue to pop these messages onto so you can return from the controller immediately. You'd then have to have something which polls the queue and sends out the notifications i.e. Scheduled Task, Windows service etc. IIRC, if IIS recycles during a task, the process is killed whereas with QueueBackgroundWorkItem there is a grace period for which ASP.Net will let the work item finish it's job.
I would take a look on Hangfire. It is fairly easy to setup, it should be able to run within your ASP.NET process and is easy to migrate to a standalone process in case your IIS load suddenly increases.
I experimented with Hangfire a while ago but in standalone mode. It has enough docs and easy to understand API.
I have a Windows service that spawns a set of child activities on separate threads and that should only terminate when all those activities have successfully completed. I do not know in advance how long it might take to terminate an activity after a stop signal is received. During OnStop(), I wait in intervals for that stop signal and keep requesting additional time for as long as the system is willing to grant it.
Here is the basic structure:
class MyService : ServiceBase
{
private CancellationTokenSource stopAllActivities;
private CountdownEvent runningActivities;
protected override void OnStart(string[] args)
{
// ... start a set of activities that signal runningActivities
// when they stop
// ... initialize runningActivities to the number of activities
}
protected override void OnStop()
{
stopAllActivities.Cancel();
while (!runningActivities.Wait(10000))
{
RequestAdditionalTime(15000); // NOTE: 5000 added for overhead
}
}
}
Just how much "overhead" should I be adding in the RequestAdditionalTime call? I'm concerned that the requests are cumulative, instead of based on the point in time when each RequestAdditionalTime call is made. If that's the case, adding overhead could result in the system eventually denying the request because it's too far out in the future. But if I don't add any overhead then my service could be terminated before it has a chance to request the next block of additional time.
This post wasn't exactly encouraging:
The MSDN documentation doesn’t mention this but it appears that the value specified in RequestAdditionalTime is not actually ‘additional’ time. Instead, it replaces the value in ServicesPipeTimeout. Worse still, any value greater than two minutes (120000 milliseconds) is ignored, i.e. capped at two minutes.
I hope that's not the case, but I'm posting this as a worst-case answer.
UPDATE: The author of that post was kind enough to post a very detailed reply to my comment, which I've copied below.
Lars, the short answer is no.
What I would say is that I now realise that Windows Services ought to be designed to start and terminate processing quickly when requested to do so.
As developers, we tend to focus on the implementation of the processing and then package it up and deliver it as a Windows Service.
However, this really isn’t the correct approach to designing Windows Services. Services must be able to respond quickly to requests to start and stop not only when an administrator making the request from the services console but also when the operating system is requesting a start as part of its start up processing or a stop because it is shutting down,
Consider what happens when Windows is configured to shut down when a UPS signals that the power has failed. It’s not appropriate for the service to respond with “I need a few more minutes…”.
It’s possible to write services that react quickly to stop requests even when they implement long running processing tasks. Usually a long running process will consist of batch processing of data and the processing should check if a stop has been requested at the level of the smallest unit of work that ensures data consistency.
As an example, the first service where I found the stop timeout was a problem involved the processing of a notifications queue on a remote server. The processing retrieved a notification from the queue, calling a web service to retrieve data related to the subject of the notification, and then writing a data file for processing by another application.
I implemented the processing as a timer driven call to a single method. Once the method is called it doesn’t return until all the notifications in the queue have been processed. I realised this was a mistake for a Windows Service because occasionally there might be tens of thousands of notifications in the queue and processing might take several minutes.
The method is capable of processing 50 notifications per second. So, what I should have done was implement a check to see if a stop had been requested before processing each notification. This would have allowed the method to return when it has completed the processing of a notification but before it has started to process the next notification. This would have ensured that the service responds quickly to a stop request and any pending notifications remained queued for processing when the service is restarted.
EDIT I realised my question was not stated clearly enough and have edited it heavily.
This is a bit of an open ended question so apologies in advance.
In a nutshell, I want to implement IIS-style asynchronous request processing in an Azure worker role.
It may be very simple or it may be insanely hard - I am looking for pointers to where to research.
While my implementation will use Azure Workers and Service Bus Queues, the general principle is applicable to any scenario where a worker process is listening for incoming requests and then servicing them.
What IIS does
In IIS there is a fixed-size threadpool. If you deal with all request synchronously then the maximum number of requests you can deal with in parallel == maxthreads. However, if you have to do slow external I/O to serve requests then this is highly inefficient because you can end up with the server being idle, yet have all threads tied up waiting for external I/O to complete.
From MSDN:
On the Web server, the .NET Framework maintains a pool of threads that are used to service ASP.NET requests. When a request arrives, a thread from the pool is dispatched to process that request. If the request is processed synchronously, the thread that processes the request is blocked while the request is being processed, and that thread cannot service another request.
This might not be a problem, because the thread pool can be made large enough to accommodate many blocked threads. However, the number of threads in the thread pool is limited. In large applications that process multiple simultaneous long-running requests, all available threads might be blocked. This condition is known as thread starvation. When this condition is reached, the Web server queues requests. If the request queue becomes full, the Web server rejects requests with an HTTP 503 status (Server Too Busy).
In order to overcome this issue, IIS has some clever logic that allows you to deal with requests asynchronously:
When an asynchronous action is invoked, the following steps occur:
The Web server gets a thread from the thread pool (the worker thread) and schedules it to handle an incoming request. This worker thread initiates an asynchronous operation.
The worker thread is returned to the thread pool to service another Web request.
When the asynchronous operation is complete, it notifies ASP.NET.
The Web server gets a worker thread from the thread pool (which might be a different thread from the thread that started the asynchronous operation) to process the remainder of the request, including rendering the response.
The important point here is when the asynchronous request returns, the return action is scheduled to run on one of the same pool of threads that serves the initial incoming requests. This means that the system is limiting how much work it is doing concurrently and this is what I would like to replicate.
What I want to do
I want to create a Worker role which will listen for incoming work requests on Azure Service Bus Queues and also potentially on TCP sockets. Like IIS I want to have a maxium threadpool size and I want to limit how much actual work the worker is doing in parallel; If the worker is busy serving existing requests - whether new incoming ones or the callbacks from previous async calls - I don't want to pick up any new incoming requests until some threads have been freed up.
It is not a problem to limit how many jobs I start concurrently - that is easy to control; It is limiting how many I am actually working on concurrently.
Let's assume a threadpool of 100 threads.
I get 100 requests to send an email come in and each email takes 5 seconds to send to the SMTP server. If I limit my server to only process 100 requests at the same time then my server will be unable to do anything else for 5 seconds, while the CPU is completely idle. So, I don't really mind starting to send 1,000 or 10,000 emails at the same time, because 99% of the "request process time" will be spent waiting for external I/O and my server will still be very quiet.
So, that particular scenario I could deal with by just keeping on accepting incoming requests with no limit (or only limit the start of the request until I fire off the async call; as soon as the BeginSend is called, I'll return and start serving another request).
Now, imagine instead that I have a type of request that goes to the database to read some data, does some heavy calculation on it and then writes that back to the database. There are two database requests there that should be made asynchronous but 90% of the request processing time will be spent on my worker. So, if I follow the same logic as above and keep start async calls and just letting the return do whatever it needs to get a thread to continue on then I will end up with a server that is very overloaded.
Somehow, what IIS does is make sure that when an async call returns it uses the same fixed-size thread pool. This means that if I fire off a lot of async calls and they then return and start using my threads, IIS will not accept new requests until those returns have finished. And that is perfect because it ensures a sensible load on the server, especially when I have multiple load-balanced servers and a queue system that the servers pick work from.
I have this sneaky suspicion that this might be very simple to do, there is just something basic I am missing. Or maybe it is insanely hard.
Creating a threadpool should be considered as independent of Windows Azure. Since a Worker Role instance is effectively Windows 2008 Server R2 (or SP2), there's nothing really different. You'd just need to set things up from your OnStart() or Run().
One thing you wanted to do was use queue length as a determining factor when scaling to more/less worker instances. Note that Service Bus Queues don't advertise queue length, where Windows Azure Queues (based on Storage, vs. Service Bus) do. With Windows Azure Queues, you'll need to poll synchronously for messages (whereas Service Bus Queues have long-polling operations). Probably a good idea to review the differences between Service Bus Queues and Windows Azure Queues, here.
Have you considered having a dedicated WCF instance (not WAS or IIS hosted) to buffer the long running requests? It will have its own dedicated app pool, with a separate Max value setting from IIS that won't contend with your ASP.NET HTTP requests. (HTTP requests are served by
Then use IIS Async methods to call WCF with the constrained app pool.
I've used the SmartThreadPool project in the past as a per-instance pool and, if I'm reading you correctly, it should have all the callback and worker-limiting functionality you need. My company actually has it running currently on Azure for the exact purpose you describe of reading message bus requests asynchronously.
I have been digging around in this and found that it is indeed relatively easy.
http://www.albahari.com/threading/ has got some good information and I actually ended up buying the book which that website is essentially promoting.
What I found out is that;
Your application has a ThreadPool available to it by default
You can limit the number of threads available in the ThreadPool
When you use QueueUserWorkItem or Task.Factory.StartNew the job you start run on a Thread in the ThreadPool
When you use one of the asynchronous IO calls in the framework (Begin... methods or WebcClient.DownloadStringAsync etc) the the callbacks will also run on a Thread from the ThreadPool (what happens with the IO request itself is outside the scope of this discussion).
So far, so good. The problem is that I can keep calling Task.Factory.StartNew as much as I like and the ThreadPool will simply queue up the work until there are free threads to service them. So, in the case of an Azure Worker, I could easily empty the Queue even though my worker is busy servicing existing requests (and callbacks from existing requests). That is the core of my problem. What I want is to not take anything out of the queue until I actually have some free threads to service the request.
This is a very simple example of how this could be achieved. In essence, I am using an AutoResetEvent to make sure that I don't start another task from the queue until the previous task has actually started. Granted, I do actually take stuff out of the queue before there is a free thread, but on balance this should avoid crazy overloads of the worker and allow me to spin up more workers to share the load.
ThreadPool.SetMaxThreads(5, 1000); // Limit to 5 concurrent threads
ThreadPool.SetMinThreads(5, 10); // Ensure we spin up all threads
var jobStart = new AutoResetEvent(true);
// The "listen" loop
while (true)
{
var job = this.jobQueue.Dequeue();
jobStart.WaitOne(); // Wait until the previous job has actually been started
Task.Factory.StartNew(
() =>
{
jobStart.Set(); // Will happen when the threadpool allocates this job to a thread
this.Download(job);
});
}
This can - and probably should - be made a lot more sophisticated, including having timeouts, putting the work item back in the queue if a thread can't be allocated within a reasonable time and so on.
An alternative would be to use ThreadPool.GetAvailableThreads to check if there are free threads before starting to listen to the queue but that feels rather more error prone.
Somehow, what IIS does is make sure that when an async call returns
it uses the same fixed-size thread pool.
This is not true: When your code runs in response to an HTTP-Request you decide on what threads the continuation function executes. Usually, this is the thread pool. And the thread pool is an appdomain-wide resource that is shared among all requests.
I think IIS does less "magic" than you think it does. All it does is to limit the number of parallel HTTP-requests and the backlog size. You decide what happens once you have been given control by ASP.NET.
If your code is not protected against overloading the server, you will overload the server even on IIS.
From what I understand you want to constrain the number of threads used for processing a certain type of message at the same time.
One approach would be to simply wrap the message processor, invoked on a new thread with something like
try
{
Interlocked.Increment(ref count)
Process(message);
}
finally
{
Interlocked.Decrement(ref count)
}
Before invoking the wrapper, simply check if the ‘count’ is less than your threshold count; and stop polling/handling more messages till the count is sufficiently lower.
EDIT Added more information based on comment
Frans, not sure why you see the infrastructure and business code being coupled. Once you place your business process to be serviced as a task on a new thread to run asynchronously, you need not worry about performing additional IO bound calls asynchronously. This is a simpler model to program in.
Here is what I am thinking.
// semi - pseudo-code
// Infrastructure – reads messages from the queue
// (independent thread, could be a triggered by a timer)
while(count < maxCount && (message = Queue.GetMessage()) != null)
{
Interlocked.Increment(ref count);
// process message asynchronously on a new thread
Task.Factory.StartNew(() => ProcessWrapper(message));
}
// glue / semi-infrastructure - deals with message deletion and exceptions
void ProcessWrapper(Message message)
{
try
{
Process(message);
Queue.DeleteMessage(message);
}
catch(Exception ex)
{
// Handle exception here.
// Log, write to poison message queue etc ...
}
finally
{
Interlocked.Decrement(ref count)
}
}
// business process
void Process(Message message)
{
// actual work done here
;
}