Develop Reference

High performance async monitoring tasks

I have a couple of hundred devices and I need to check their status every 5 seconds.
The API I'm using contains a blocking function that calls a dll and returns a status of a single device
string status = ReadStatus(int deviceID); // waits here until the status is returned
The above function usually returns the status in a couple of ms, but there will be situations where I might not get the status back for a second or more! Or even worse, one device might not respond at all.
I therefore need to introduce a form of asynchronicity to make sure that one device that doesn't respond doesn't impend all the others being monitored.
My current approach is as following
// triggers every 5 sec
public MonitorDevices_ElapsedInterval(object sender, ElapsedEventArgs elapsedEventArgs)
{
foreach (var device in lstDevices) // several hundred devices in the list
{
var task = device.ReadStatusAsync(device.ID, cts.Token);
tasks.Add(task);
}
// await all tasks finished, or timeout after 4900ms
await Task.WhenAny(Task.WhenAll(tasks), Task.Delay(4900, cts.Token));
cts.Cancel();
var devicesThatResponded = tasks.Where(t => t.Status == TaskStatus.RanToCompletion)
.Select(t => t.GetAwaiter().GetResult())
.ToList();
}
And below in the Device class
public async Task ReadStatusAsync(int deviceID, CancellationToken tk)
{
await Task.Delay(50, tk);
// calls the dll to return the status. Blocks until the status is return
Status = ReadStatus(deviceID);
}
I'm having several problems with my code
the foreach loops fires a couple of hundred tasks simultaneously, with the callback from the Task.Delay being served by a thread from the thread pool, each task taking a couple of ms.
I see this as a big potential bottleneck. Are there any better approaches?
This might be similar to what Stephen Cleary commented here, but he didn't provide an alternative What it costs to use Task.Delay()?
In case ReadStatus fails to return, I'm trying to use a cancellation token to cancel the thread that sits there waiting for the response... This doesn't seem to work.
await Task.Delay(50, tk)
Thread.Sleep(100000) // simulate the device not responding
I still have about 20 Worker Threads alive (even though I was expecting cts.Cancel() to kill them.

the foreach loops fires a couple of hundred tasks simultaneously
Since ReadStatus is synchronous (I'm assuming you can't change this), and since each one needs to be independent because they can block the calling thread, then you have to have hundreds of tasks. That's already the most efficient way.
Are there any better approaches?
If each device should be read every 5 seconds, then each device having its own timer would probably be better. After a few cycles, they should "even out".
await Task.Delay(50, tk);
I do not recommend using Task.Delay to "trampoline" non-async code. If you wish to run code on the thread pool, just wrap it in a Task.Run:
foreach (var device in lstDevices) // several hundred devices in the list
{
var task = Task.Run(() => device.ReadStatus(device.ID, cts.Token));
tasks.Add(task);
}
I'm trying to use a cancellation token to cancel the thread that sit there waiting for the response... This doesn't seem to work.
Cancellation tokens do not kill threads. If ReadStatus observes its cancellation token, then it should cancel; if not, then there isn't much you can do about it.
Thread pool threads should not be terminated; this reduces thread churn when the timer next fires.

As you can see in this Microsoft example page of a cancellation token, the doWork method is checking for cancellation on each loop. So, the loop has to start again to cancel out. In your case, when you simulate a long task, it never checks for cancellation at all when it's running.
From How do I cancel non-cancelable async operations?, it's saying at the end : "So, can you cancel non-cancelable operations? No. Can you cancel waits on non-cancelable operations? Sure… just be very careful when you do.". So it answers that we can't cancel it out.
What I would suggest is to use threads with a ThreadPool, you take the starting time of each one and you have an higher priority thread that looks if others bypass their maximum allowed time. If so, Thread.Interrupt().

Is Task.Delay Worth Cancellation?

I've recently reimplemented a whole bunch of async WCF service methods using the cancellation pattern I've seen described in a number of places - where you await a Task.WhenAny on a started task and a Task.Delay. Of course, the existing tasks aren't cancellable, but that will hopefully be addressed in a later release.
In my case, the default duration of the Task.Delay is governed by a service setting. In an overwhelming majority of cases, the outcome is that the hoped-for task completes in the requisite amount of time. The setting is often made generous.
Most of (but not all) the examples I've seen do not bother to cancel the Task.Delay. Is it so cheap it's not worth worrying about? I know that cancellation raises an exception. If I cancel the delay, should I be handling the exception?
Here's the method I've made that all the service methods are calling through:
private async Task<T> GetOrTimeout<T>( Task<T> task, [CallerMemberName] string caller = "" )
{
using ( var cts = new CancellationTokenSource( ) )
{
try
{
var timeout = GetDelay( cts.Token );
var first = await Task.WhenAny( task, timeout );
if ( first == timeout ) throw new TimeoutException( Properties.Resources.TimeoutOccurredInService.Fmt( caller ) );
cts.Cancel( ); //--> haven't been doing this. Should I?
return await task;
}
catch ( Exception ex )
{
throw LoggedFaultException( ex, caller );
}
}
}
...and the method that creates the delay would look like this:
private Task GetDelay( CancellationToken token )
{
return Task
.Delay( Properties.Settings.Default.ServiceMethodTimeout, token )
.ContinueWith( _ => { }, TaskContinuationOptions.ExecuteSynchronously );
}
If I don't cancel the delay, am I holding onto resources way longer than necessary? In particular, I'm worried about the instances that WCF spins up to invoke the service methods. I'm worried they'll cut into the concurrency parameters that the service was configured with. The timeout setting is pretty coarse. It seems wasteful to not cancel, but this is all pretty new stuff to me.
Since cancellation involves exceptions, and since I've been trained to not use exceptions to communicate state, this feels like I've painted myself into some awful anti-pattern that I don't fully understand. Perhaps Task.Delay isn't the right choice for me. It feels like I've made it more complicated than I should. Any light shed on the situation would be most appreciated.

First of all, this whole issue is probably negligible performance-wise and should only be considered otherwise after testing in a real environment.
However if we dive in, Task.Delay creates a task that is completed after a certain interval. It does so by creating a new System.Threading.Timer (which implements IDisposable) that completes the promise task after the interval using a ThreadPool thread.
If you use Task.Delay "a lot" you can have a considerable amount of wasted resources hanging around long after they're useful. If you also add any continuations to the Task.Delay task with a delegate that captures any references they too will hang around with no reason.
So yes, it's safer to cancel the task instead of letting it run out, though probably not by much.

Task.Delay is worth cancelling when you care about the shutdown speed of your app.
One example is asp.net web applications.
When the server recycles your web-app (when it's being live-updated for example) it needs everything to end fast. If you have tasks waiting in the background, especially registered via QueueBackgroundObject or a similar technique, it might take a while for the app to shut down.

Is it possible to use a CancellationToken with ExecuteReader?

The new async ExecuteReaderAsync takes a CancellationToken. Is there any way to cancel the old synchronous ExecuteReader?
In our case, all the data operations are synchronous on a background thread, so await is not an option. I don't want to start a second thread- Task.Run(() => command.ExecuteReaderAsync(token)).Result seems a waste just to be able to cancel from the UI thread.

Performance tests showed almost 2x performance benefit to a using a dedicated synchronous data-reading thread, compared to using the Begin or Async APIs with their thread pool continuations. (Since tens of millions of rows will be loaded in as many seconds, we prefer performance in this case.)
An extension method for convenient token passing:
public static SqlDataReader ExecuteReader(this SqlCommand command, CommandBehavior commandBehavior, CancellationToken cancellationToken)
{
try
{
using (cancellationToken.Register(command.Cancel))
return command.ExecuteReader(commandBehavior);
}
catch (SqlException) when (cancellationToken.IsCancellationRequested)
{
throw new OperationCanceledException(cancellationToken);
}
}
I did some spelunking with Reflector decompilation. The Begin and Async versions are both very frugal, but are both fully based on the TPL async. Because of that, there is thread pool dispatching for continuations on both.
This extension method has no threading overhead. The thread that calls Cancel on the token source will also call command.Cancel which will immediately cause a SqlException in the data thread.

I can't respond authoritatively to your edit but there's a few things you should know about your initial question:
Task.Run( ... ).Result is blocking; that syntax is a little misleading.
await Task.Run( () => command.ExecuteReaderAsync(token)); will block only the remainder of the executing method; allowing the rest to be treated as a callback.
await Task.Run( () => command.ExecuteReaderAsync(token), token); works as above but allows the task parallel library to honor to your cancellation token as well.
As to the main question, this msdn article suggests that ExecuteReaderAsync() is truly honoring that cancellationToken. Bear in mind there are a couple of methods in the framework that will not actually do that.

Cancelling a Thread due to hung Db call

I've designed and made a prototype application for a high performance, multi-threaded mail merge to run as a Windows Service (C#). This question refers to one sticky part of the problem, what to do if the process hangs on a database call. I have researched this a lot. I have read a lot of articles about thread cancellation and I ultimately only see one way to do this, thread.Abort(). Yes, I know, absolutely do not use Thread.Abort(), so I have been researching for days how to do it another way and as I see it, there is no alternative. I will tell you why and hopefully you can tell me why I am wrong.
FYI, these are meant as long running threads, so the TPL would make them outside the ThreadPool anyway.
TPL is just a nice wrapper for a Thread, so I see absolutely nothing a Task can do that a Thread cannot. It's just done differently.
Using a thread, you have two choices for stopping it.
1. Have the thread poll in a processing loop to see if a flag has requested cancellation and just end the processing and let the thread die. No problem.
2. Call Thread.Abort() (then catch the exception, do a Join and worry about Finally, etc.)
This is a database call in the thread, so polling will not work once it is started.
On the other hand, if you use TPL and a CancellationToken, it seems to me that you're still polling and then creating an exception. It looks like the same thing I described in case 1 with the thread. Once I start that database call (I also intend to put a async / await around it), there is no way I can test for a change in the CancellationToken. For that matter, the TPL is worse as calling the CancellationToken during a Db read will do exactly nothing, far less than a Thread.Abort() would do.
I cannot believe this is a unique problem, but I have not found a real solution and I have read a lot. Whether a Thread or Task, the worker thread has to poll to know it should stop and then stop (not possible when connected to a Db. It's not in a loop.) or else the thread must be aborted, throwing a ThreadAbortedException or a TaskCanceledException.
My current plan is to start each job as a longrunning thread. If the thread exceeds the time limit, I will call Thread.Abort, catch the exception in the thread and then do a Join() on the thread after the Abort().
I am very, very open to suggestions... Thanks, Mike
I will put this link, because it claims to do this, but I'm having trouble figuring it out and there are no replys to make me think it will work
multi-threading-cross-class-cancellation-with-tpl
Oh, this looked like a good possibility, but I don't know about it either Treating a Thread as a Service

You can't actually cancel the DB operation. The request is sent across the network; it's "out there" now, there's no pulling it back. The best you can really do is ignore the response that comes back, and continue on executing whatever code you would have executed had the operation actually completed. It's important to recognize what this is though; this isn't actually cancelling anything, it's just moving on even though you're not done. It's a very important distinction.
If you have some task, and you want it to instead become cancelled when you want it to be, you can create a continuation that uses a CancellationToken such that the continuation will be marked as canceled when the token indicates it should be, or it'll be completed when the task completes. You can then use that continuation's Task in place of the actual underlying tasks for all of your continuations, and the task will be cancelled if the token is cancelled.
public static Task WithCancellation(this Task task
, CancellationToken token)
{
return task.ContinueWith(t => t.GetAwaiter().GetResult(), token);
}
public static Task<T> WithCancellation<T>(this Task<T> task
, CancellationToken token)
{
return task.ContinueWith(t => t.GetAwaiter().GetResult(), token);
}
You can then take a given task, pass in a cancellation token, and get back a task that will have the same result except with altered cancellation semantics.

You have several other options for your thread cancellation. For example, your thread could make an asynchronous database call and then wait on that and on the cancellation token. For example:
// cmd is a SqlCommand object
// token is a cancellation token
IAsyncResult ia = cmd.BeginExecuteNonQuery(); // starts an async request
WaitHandle[] handles = new WaitHandle[]{token.WaitHandle, ia.AsyncWaitHandle};
var ix = WaitHandle.WaitAny(handles);
if (ix == 0)
{
// cancellation was requested
}
else if (ix == 1)
{
// async database operation is done. Harvest the result.
}
There's no need to throw an exception if the operation was canceled. And there's no need for Thread.Abort.
This all becomes much cleaner with Task, but it's essentially the same thing. Task handles common errors and helps you to do a better job fitting all the pieces together.
You said:
TPL is just a nice wrapper for a Thread, so I see absolutely nothing a Task can do that a Thread cannot. It's just done differently.
That's true, as far as it goes. After all, C# is just a nice wrapper for an assembly language program, so I see absolutely nothing a C# program can do that I can't do in assembly language. But it's a whole lot easier and faster to do it with C#.
Same goes for the difference between TPL or Tasks, and managing your own threads. You can do all manner of stuff managing your own threads, or you can let the TPL handle all the details and be more likely to get it right.

TPL.Dataflow message processing timeout?

Using .NET TPL DataFlow blocks.
Is there any way to timeout the processing of a message?
e.g.
lets say I have a BufferBlock<T>, is it possible to link that to another block that processes one message at a time (MaxDegreeOfParallellism 1) and force a timeout if the processing runs for too long?
Or is it even possible to do using the BufferBlock only?
I suspect I can use a cancellation token somehow and a delay, but not sure how this would be done.
Also, how expensive would such timeout be? would it add much overhead to the message processing time?

Many methods of BufferBlock<T> do accept CancellationToken, and I believe that would be the proper way of timing out an operation. E.g.:
var cts = new CancellationTokenSource(5000); // cancel in 5s
// Alternatively: cts.CancelAfter(5000);
try
{
var output = await bufferBlock.ReceiveAsync(cts.Token);
}
catch (Exception ex)
{
// check if ex is OperationCanceledException,
// which could be wrapped with AggregateException
}
IMO, the only way of evaluating its efficiency would be to run some profiling tests.
[UPDATE] Based upon the comments, if you're looking to time-out the pipeline processing, you can probably do that when you construct your ActionBlock object and provide it with an instance of ExecutionDataflowBlockOptions. At that point, you can supply DataflowBlockOptions.CancellationToken and use it in the same way as described above. Also, you could pass a CancellationToken into LinkTo as a part of DataflowLinkOptions.
Once you've provided the pipeline with a CancellationToken, you can track the status of ActionBlock.Completion/TransformBlock.Completion, which is a Task, so you can await it and catch a cancellation exception, or use ContinueWith with it (if that's what you mean under some way to tell if the "processing" of the message times out).
Disclaimer: I haven't tried this myself and would be interested to know whether it works as expected.