Develop Reference

How to handle multiple tasks running in parallel at different intervals inside a C# based Windows service?

I already have some experience in working with threads in Windows but most of that experience comes from using Win32 API functions in C/C++ applications. When it comes to .NET applications however, I am often not sure about how to properly deal with multithreading. There are threads, tasks, the TPL and all sorts of other things I can use for multithreading but I never know when to use which of those options.
I am currently working on a C# based Windows service which needs to periodically validate different groups of data from different data sources. Implementing the validation itself is not really an issue for me but I am unsure about how to handle all of the validations running simultaneously.
I need a solution for this which allows me to do all of the following things:
Run the validations at different (predefined) intervals.
Control all of the different validations from one place so I can pause and/or stop them if necessary, for example when a user stops or restarts the service.
Use the system ressources as efficiently as possible to avoid performance issues.
So far I've only had one similar project before where I simply used Thread objects combined with a ManualResetEvent and a Thread.Join call with a timeout to notify the threads about when the service is stopped. The logic inside those threads to do something periodically then looked like this:
while (!shutdownEvent.WaitOne(0))
{
if (DateTime.Now > nextExecutionTime)
{
// Do something
nextExecutionTime = nextExecutionTime.AddMinutes(interval);
}
Thread.Sleep(1000);
}
While this did work as expected, I've often heard that using threads directly like this is considered "oldschool" or even a bad practice. I also think that this solution does not use threads very efficiently as they are just sleeping most of the time. How can I achive something like this in a more modern and efficient way?
If this question is too vague or opinion-based then please let me know and I will try my best to make it as specific as possible.

Question feels a bit broad but we can use the provided code and try to improve it.
Indeed the problem with the existing code is that for the majority of the time it holds thread blocked while doing nothing useful (sleeping). Also thread wakes up every second only to check the interval and in most cases go to sleep again since it's not validation time yet. Why it does that? Because if you will sleep for longer period - you might block for a long time when you signal shutdownEvent and then join a thread. Thread.Sleep doesn't provide a way to be interrupted on request.
To solve both problems we can use:
Cooperative cancellation mechanism in form of CancellationTokenSource + CancellationToken.
Task.Delay instead of Thread.Sleep.
For example:
async Task ValidationLoop(CancellationToken ct) {
while (!ct.IsCancellationRequested) {
try {
var now = DateTime.Now;
if (now >= _nextExecutionTime) {
// do something
_nextExecutionTime = _nextExecutionTime.AddMinutes(1);
}
var waitFor = _nextExecutionTime - now;
if (waitFor.Ticks > 0) {
await Task.Delay(waitFor, ct);
}
}
catch (OperationCanceledException) {
// expected, just exit
// otherwise, let it go and handle cancelled task
// at the caller of this method (returned task will be cancelled).
return;
}
catch (Exception) {
// either have global exception handler here
// or expect the task returned by this method to fail
// and handle this condition at the caller
}
}
}
Now we do not hold a thread any more, because await Task.Delay doesn't do this. Instead, after specificed time interval it will execute the subsequent code on a free thread pool thread (it's more complicated that this but we won't go into details here).
We also don't need to wake up every second for no reason, because Task.Delay accepts cancellation token as a parameter. When that token is signalled - Task.Delay will be immediately interrupted with exception, which we expect and break from the validation loop.
To stop the provided loop you need to use CancellationTokenSource:
private readonly CancellationTokenSource _cts = new CancellationTokenSource();
And you pass its _cts.Token token into the provided method. Then when you want to signal the token, just do:
_cts.Cancel();
To futher improve the resource management - IF your validation code uses any IO operations (reads files from disk, network, database access etc) - use Async versions of said operations. Then also while performing IO you will hold no unnecessary threads blocked waiting.
Now you don't need to manage threads yourself anymore and instead you operatate in terms of tasks you need to perform, letting framework \ OS manage threads for you.

You should use Microsoft's Reactive Framework (aka Rx) - NuGet System.Reactive and add using System.Reactive.Linq; - then you can do this:
Subject<bool> starter = new Subject<bool>();
IObservable<Unit> query =
starter
.StartWith(true)
.Select(x => x
? Observable.Interval(TimeSpan.FromSeconds(5.0)).SelectMany(y => Observable.Start(() => Validation()))
: Observable.Never<Unit>())
.Switch();
IDisposable subscription = query.Subscribe();
That fires off the Validation() method every 5.0 seconds.
When you need to pause and resume, do this:
starter.OnNext(false);
// Now paused
starter.OnNext(true);
// Now restarted.
When you want to stop it all call subscription.Dispose().

Call an awaitable method inside a Thread in C#

In my C# 7.0 Code I like to use an old school Thread of type Thread to do some work. Inside this Thread, I need to use some async methods. What would be the best approach to call these methods?
Writing the thread function async does not make any sense.
this._networkListenerThread = new Thread(/* async here is not an option */() =>
{
while (!this._listenerCancellation.IsCancellationRequested)
{
try
{
// This does not compile
var packetBuffer = await this._commProxy.ReadAsync();
doSomethingMore();
}
}
}
If we go down the call stack, finally there will be this call:
// _socket is of type Android.Bluetooth.BluetoothSocket
// .InputStream of type System.IO.Stream
// ReadAsync only returns when data arrived on the stream
// or throws an exception when the connection is lost
var receivedBytes = await this._socket.InputStream.ReadAsync(buffer, 0, buffer.Length);
For those wondering why I want to use a Thread instead of Task: I want to give it a meaningful name to enhance debugging. I did not find a way to name a Task. Besides of this, this Thread runs almost as long as the application runs, therefore a Thread does make sense for me.

this Thread runs almost as long as the application runs
No, it doesn't. Because the work that it's doing is asynchronous. The thread runs long enough to check the status of the cancellation token, fire off ReadAsync (which, being asynchronous, will return basically immediately) and then it's done. The thread goes away, and it has no more work to do. That's the whole idea of asynchronous operations; being asynchronous means the operation returns to its caller pretty much immediately, and does whatever meaningful work it has to do after returning control back to the caller (in this case, since this is the top level method of the thread, returning control back means that the thread has finished executing and gets torn down).
So there just isn't much of any purpose in creating a new thread just to have it check a boolean value and start some operation that will go off and do work on its own. It's not that you should use a different way of getting a new thread to do work (like using Task.Run), but rather you shouldn't use any means of getting a new thread to do work, because you don't have any long running CPU bound work to do. The long running (non-CPU bound, by the look of it) work that you have is already asynchronous, so you can just call the method directly from whatever thread wants to start this work, and have it do it right in line.
If you simply want to have some value that you can share along an asynchronous operation's logical call context, there are of course tools that accomplish that, such as AsyncLocal. Creating a new thread wouldn't accomplish that, because as you finish starting the asynchronous operation you have your thread is dead and gone, and the continuations will be running in some other thread anyway.

How to covert async call to work as sync call in C#

I'm a newbie in C#, and I'm going to develop a small program using a third party network library to send the requests.
Suppose there have some requests (just simple strings) stored in the queue qTasks, and it will handle those requests one by one with the order as submitted, the queue can be updated during execution, and it should be stopped whenever there has error returned.
I can just use a for loop to call the send request command in the array one by one, but unfortunately the sendrequest command is an async method with callback OnStageChanged, and I need to check the result before sending the next request when the status is "Done".
I'm now using the following method to handle it:
In the main UI Thread,
// Put those request text in a queue names qTasks, then call a goNextTask() to process the request one by one.
// The queue can be updated by the UI thread at anytime, goNextTask will be called periodically to handle those pending request in the queue.
private void goNextTask(bool lastSuccess = true)
{
if (lastSuccess)
{
if (qTasks.Count > 0)
{
// continue to next request
string requestText = qTasks.Dequeue();
SendRequest(requestText, OnStageChangeHandler);
} else {
// Report for all request sent successfully
}
} else {
// stop and show error
}
}
The callback method OnStageChangeHandler will be called by the library whenever the stage changes, and it will have state "Done" when completed.
private void OnStageChangeHandler(object sender, StageChangeEventArgs e)
{
if (e.newState == SessionStates.Done)
{
// check result here
bool success = <...>
// then call the goNextTask in UI thread with the result of current request.
Application.Current.Dispatcher.BeginInvoke(
System.Windows.Threading.DispatcherPriority.Normal,
(Action)(() => goNextTask(success)));
}
}
Although it works fine now, I think it's a little bit stupid as it has a somewhat recursive flow (A -> B -> A -> B ->....).
I learnt that MS has improved the web request handling, so that it can work in sync mode.
I'd like to know if I can have a wrapper to make the above async call work as a sync call, so that it can be done in a simple flow as a loop like that:
while (qTaks.Count > 0)
{
if (!sendAndWaitReturn(qTasks.Dequeue())) {
// Report error and quit
}
}
// all tasks completed
This sendAndWaitReturn method will send the request, then wait for the status "Done", and then return the result.
I found some example that may use a control flag to indicate the status of the current request, and the callback function will update this control flag, while the UI thread will loop on this flag using a while loop:
while (!requestDone);
so that it will not continue to nextRequest until requestDone. But in this case, the UI will be blocked.
Is there any better way to convert the async call to work as a sync call without blocking the UI thread?

The difficulty you're going to run into is you have conflicting desires. On one hand, you want to avoid blocking the UI thread. On the other hand, you don't want to run things asynchronously and so you're going to block the UI thread.
You're going to have to pick one, and there's absolutely no reason to keep on doing things synchronously (especially in light of blocking the UI thread). If it hurts when you do that, don't do that.
You haven't specified, but I'm guessing that you're starting this processing from a button click event. Make the method invoked by that click event async. For example:
private async void StartProcessing_Click(object sender, EventArgs e)
{
await Task.Run(() => StartProcessing());
}
There, you've started processing and the UI thread isn't tied up.
The next thing is that, you're right, having the event behave in that cyclical manner is silly. The event is to notify someone that the state has changed, its goal isn't to manage queue policy. The queue should manage queue policy (or if you'd rather not abstract that out, the method that processes requests).
So how would you do that? Well, you've said that SendRequest hands the session object back to the caller. The caller is presumably the one who is orchestrating queue policy and determining whether or not to call SendRequest again.
Have the caller check the session object for validity and decide whether to keep going based on that.
Additionally, I'm unfamiliar with that particular library, but briefly glancing at the documentation it looks like there's also a SendRequestAndWait() method with the same signature and that sounds like it might better meet your needs.

BeginFlush seems to work synchronously

I'm trying to use HttpResponse BeginFlush and EndFlush methods in order to make the flush async, which means my worker thread won't being used while flushing to the stream.
However it seems that the BeginFlush methods run in synchronous way always.
I dig in Microsoft reference code and didn't find the reason for this behavior.
This is Microsoft implementation: http://referencesource.microsoft.com/#System.Web/HttpResponse.cs,f121c649c992c407
I checked the SupportsAsyncFlush flag and I'm getting true , so my environment actually supports the AsyncFlush.
Any idea?
This is a code snippet for trying to do the async flush, but I'm not getting to the "Different Threads" line - it is always the same thread that runs this code.
Context.Response.Write("Some message");
Context.Response.BeginFlush(
res =>
{
try
{
var previousThreadId = (int)res.AsyncState;
var thread2Id = Thread.CurrentThread.ManagedThreadId;
if (previousThreadId != thread2Id)
{
Console.WriteLine("Different Threads");
}
Context.Response.EndFlush(res);
}
catch (Exception e)
{
}
},
Thread.CurrentThread.ManagedThreadId);

The code is asynchronous, but it's not multithreaded. You're defining a callback; a method that will be run at some indeterminate point in the future when the flush finishes. That doesn't necessarily mean that it'll run on another thread.
There are also many implementations of abstract functionality in .NET where the behavior is defined as asynchronous, but the implementation is synchronous because that particular implementation expects to run so quickly as to not warrant asynchrony. This is true for a fair bit of .NET's file IO. If the writer you're using expects to be able to flush the buffer very quickly, it may not bother doing it asynchronously.

The effect you are seeing may be related to SycnhronizationContext used by ASP.NET. Depending on .NET version and whether your code executes under ASP.NET page or something else, the behavior may change. Generally, it is normal for the SynchronizationContext to execute the callback on the same thread which started the async operation. You can find more info here: https://msdn.microsoft.com/en-us/magazine/gg598924.aspx
In any case, you can check whether the callback is synchronous or not by checking weather the next line of your code (after BeginFlush) executes before the callback. That will indicate that the callback is indeed asynchronous.

Do asynchronous web service calls always call the AsyncCallback?

I'm making asynchronous web service calls from a C# app:
{
//Put UI in 'loading' state
...
//Now call web service
webServiceProxy.BeginMyMethod(param, new AsyncCallback(MyCallback), null);
}
private void MyCallback(IAsyncResult res)
{
...
//process result
// Put UI back in normal state (yes I'm marshalling back to the UI thread)
}
The main thread puts the app in a "waiting" mode, and then the end of the Callback function re-enables the controls. I'm seeing a bug that occasionally, the UI is stuck forever in the loading mode.
Now there may just be a bug in the callback code (there's quite a bit there), but my question to the community here is this:
Is "MyCallback" GUARANTEED to be called? Presuming that "BeginMyMethod" didn't throw an exception, can I be sure that MyCallback will be executed? I'm seeing a "CompletedSynchronously" and "IsCompleted" on the IAsyncResult returned by the BeginXXX functions, but I'm not sure if that's important or not.

Yes, the callback is guaranteed to be called. The callback is what permits asynchronous code using the Begin* / End* pattern to be written in a continuation-passing style.
You must call the corresponding End* method in your callback (normally, the first thing in the callback), however. It is how the asynchronous method signals an exception that may have occurred during the call, for one thing, as well as the way to get a result out (if any).
Coding the asynchronous pattern using anonymous delegates when using C# 2.0 is sometimes more elegant, and permits writing of the post-call continuation close to the initiation of the call, as well as permitting much easier data sharing through captured variables, providing that appropriate synchronization measures are used.
Ref: http://msdn.microsoft.com/en-us/library/ms228972.aspx:
Applications that can do other work while waiting for the results of an asynchronous operation should not block waiting until the operation completes. Use one of the following options to continue executing instructions while waiting for an asynchronous operation to complete:
Use an AsyncCallback delegate to process the results of the asynchronous operation in a separate thread. This approach is demonstrated in this topic.
[...]

The AsyncCallback will be called regardless of whether the operation was completed synchronously or asynchronously.

I believe the callback is not guaranteed to be called if the CompletedSynchronously property is true.