Develop Reference

How to handle multiple tasks running in parallel at different intervals inside a C# based Windows service?

I already have some experience in working with threads in Windows but most of that experience comes from using Win32 API functions in C/C++ applications. When it comes to .NET applications however, I am often not sure about how to properly deal with multithreading. There are threads, tasks, the TPL and all sorts of other things I can use for multithreading but I never know when to use which of those options.
I am currently working on a C# based Windows service which needs to periodically validate different groups of data from different data sources. Implementing the validation itself is not really an issue for me but I am unsure about how to handle all of the validations running simultaneously.
I need a solution for this which allows me to do all of the following things:
Run the validations at different (predefined) intervals.
Control all of the different validations from one place so I can pause and/or stop them if necessary, for example when a user stops or restarts the service.
Use the system ressources as efficiently as possible to avoid performance issues.
So far I've only had one similar project before where I simply used Thread objects combined with a ManualResetEvent and a Thread.Join call with a timeout to notify the threads about when the service is stopped. The logic inside those threads to do something periodically then looked like this:
while (!shutdownEvent.WaitOne(0))
{
if (DateTime.Now > nextExecutionTime)
{
// Do something
nextExecutionTime = nextExecutionTime.AddMinutes(interval);
}
Thread.Sleep(1000);
}
While this did work as expected, I've often heard that using threads directly like this is considered "oldschool" or even a bad practice. I also think that this solution does not use threads very efficiently as they are just sleeping most of the time. How can I achive something like this in a more modern and efficient way?
If this question is too vague or opinion-based then please let me know and I will try my best to make it as specific as possible.

Question feels a bit broad but we can use the provided code and try to improve it.
Indeed the problem with the existing code is that for the majority of the time it holds thread blocked while doing nothing useful (sleeping). Also thread wakes up every second only to check the interval and in most cases go to sleep again since it's not validation time yet. Why it does that? Because if you will sleep for longer period - you might block for a long time when you signal shutdownEvent and then join a thread. Thread.Sleep doesn't provide a way to be interrupted on request.
To solve both problems we can use:
Cooperative cancellation mechanism in form of CancellationTokenSource + CancellationToken.
Task.Delay instead of Thread.Sleep.
For example:
async Task ValidationLoop(CancellationToken ct) {
while (!ct.IsCancellationRequested) {
try {
var now = DateTime.Now;
if (now >= _nextExecutionTime) {
// do something
_nextExecutionTime = _nextExecutionTime.AddMinutes(1);
}
var waitFor = _nextExecutionTime - now;
if (waitFor.Ticks > 0) {
await Task.Delay(waitFor, ct);
}
}
catch (OperationCanceledException) {
// expected, just exit
// otherwise, let it go and handle cancelled task
// at the caller of this method (returned task will be cancelled).
return;
}
catch (Exception) {
// either have global exception handler here
// or expect the task returned by this method to fail
// and handle this condition at the caller
}
}
}
Now we do not hold a thread any more, because await Task.Delay doesn't do this. Instead, after specificed time interval it will execute the subsequent code on a free thread pool thread (it's more complicated that this but we won't go into details here).
We also don't need to wake up every second for no reason, because Task.Delay accepts cancellation token as a parameter. When that token is signalled - Task.Delay will be immediately interrupted with exception, which we expect and break from the validation loop.
To stop the provided loop you need to use CancellationTokenSource:
private readonly CancellationTokenSource _cts = new CancellationTokenSource();
And you pass its _cts.Token token into the provided method. Then when you want to signal the token, just do:
_cts.Cancel();
To futher improve the resource management - IF your validation code uses any IO operations (reads files from disk, network, database access etc) - use Async versions of said operations. Then also while performing IO you will hold no unnecessary threads blocked waiting.
Now you don't need to manage threads yourself anymore and instead you operatate in terms of tasks you need to perform, letting framework \ OS manage threads for you.

You should use Microsoft's Reactive Framework (aka Rx) - NuGet System.Reactive and add using System.Reactive.Linq; - then you can do this:
Subject<bool> starter = new Subject<bool>();
IObservable<Unit> query =
starter
.StartWith(true)
.Select(x => x
? Observable.Interval(TimeSpan.FromSeconds(5.0)).SelectMany(y => Observable.Start(() => Validation()))
: Observable.Never<Unit>())
.Switch();
IDisposable subscription = query.Subscribe();
That fires off the Validation() method every 5.0 seconds.
When you need to pause and resume, do this:
starter.OnNext(false);
// Now paused
starter.OnNext(true);
// Now restarted.
When you want to stop it all call subscription.Dispose().

Is there a way to limit the number of parallel Tasks globally in an ASP.NET Web API application?

I have an ASP.NET 5 Web API application which contains a method that takes objects from a List<T> and makes HTTP requests to a server, 5 at a time, until all requests have completed. This is accomplished using a SemaphoreSlim, a List<Task>(), and awaiting on Task.WhenAll(), similar to the example snippet below:
public async Task<ResponseObj[]> DoStuff(List<Input> inputData)
{
const int maxDegreeOfParallelism = 5;
var tasks = new List<Task<ResponseObj>>();
using var throttler = new SemaphoreSlim(maxDegreeOfParallelism);
foreach (var input in inputData)
{
tasks.Add(ExecHttpRequestAsync(input, throttler));
}
List<ResponseObj> resposnes = await Task.WhenAll(tasks).ConfigureAwait(false);
return responses;
}
private async Task<ResponseObj> ExecHttpRequestAsync(Input input, SemaphoreSlim throttler)
{
await throttler.WaitAsync().ConfigureAwait(false);
try
{
using var request = new HttpRequestMessage(HttpMethod.Post, "https://foo.bar/api");
request.Content = new StringContent(JsonConvert.SerializeObject(input, Encoding.UTF8, "application/json");
var response = await HttpClientWrapper.SendAsync(request).ConfigureAwait(false);
var responseBody = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
var responseObject = JsonConvert.DeserializeObject<ResponseObj>(responseBody);
return responseObject;
}
finally
{
throttler.Release();
}
}
This works well, however I am looking to limit the total number of Tasks that are being executed in parallel globally throughout the application, so as to allow scaling up of this application. For example, if 50 requests to my API came in at the same time, this would start at most 250 tasks running parallel. If I wanted to limit the total number of Tasks that are being executed at any given time to say 100, is it possible to accomplish this? Perhaps via a Queue<T>? Would the framework automatically prevent too many tasks from being executed? Or am I approaching this problem in the wrong way, and would I instead need to Queue the incoming requests to my application?

I'm going to assume the code is fixed, i.e., Task.Run is removed and the WaitAsync / Release are adjusted to throttle the HTTP calls instead of List<T>.Add.
I am looking to limit the total number of Tasks that are being executed in parallel globally throughout the application, so as to allow scaling up of this application.
This does not make sense to me. Limiting your tasks limits your scaling up.
For example, if 50 requests to my API came in at the same time, this would start at most 250 tasks running parallel.
Concurrently, sure, but not in parallel. It's important to note that these aren't 250 threads, and that they're not 250 CPU-bound operations waiting for free thread pool threads to run on, either. These are Promise Tasks, not Delegate Tasks, so they don't "run" on a thread at all. It's just 250 objects in memory.
If I wanted to limit the total number of Tasks that are being executed at any given time to say 100, is it possible to accomplish this?
Since (these kinds of) tasks are just in-memory objects, there should be no need to limit them, any more than you would need to limit the number of strings or List<T>s. Apply throttling where you do need it; e.g., number of HTTP calls done simultaneously per request. Or per host.
Would the framework automatically prevent too many tasks from being executed?
The framework has nothing like this built-in.
Perhaps via a Queue? Or am I approaching this problem in the wrong way, and would I instead need to Queue the incoming requests to my application?
There's already a queue of requests. It's handled by IIS (or whatever your host is). If your server gets too busy (or gets busy very suddenly), the requests will queue up without you having to do anything.

If I wanted to limit the total number of Tasks that are being executed at any given time to say 100, is it possible to accomplish this?
What you are looking for is to limit the MaximumConcurrencyLevel of what's called the Task Scheduler. You can create your own task scheduler that regulates the MaximumCongruencyLevel of the tasks it manages. I would recommend implementing a queue-like object that tracks incoming requests and currently working requests and waits for the current requests to finish before consuming more. The below information may still be relevant.
The task scheduler is in charge of how Tasks are prioritized, and in charge of tracking the tasks and ensuring that their work is completed, at least eventually.
The way it does this is actually very similar to what you mentioned, in general the way the Task Scheduler handles tasks is in a FIFO (First in first out) model very similar to how a ConcurrentQueue<T> works (at least starting in .NET 4).
Would the framework automatically prevent too many tasks from being executed?
By default the TaskScheduler that is created with most applications appears to default to a MaximumConcurrencyLevel of int.MaxValue. So theoretically yes.
The fact that there practically is no limit to the amount of tasks(at least with the default TaskScheduler) might not be that big of a deal for your case scenario.
Tasks are separated into two types, at least when it comes to how they are assigned to the available thread pools. They're separated into Local and Global queues.
Without going too far into detail, the way it works is if a task creates other tasks, those new tasks are part of the parent tasks queue (a local queue). Tasks spawned by a parent task are limited to the parent's thread pool.(Unless the task scheduler takes it upon itself to move queues around)
If a task isn't created by another task, it's a top-level task and is placed into the Global Queue. These would normally be assigned their own thread(if available) and if one isn't available it's treated in a FIFO model, as mentioned above, until it's work can be completed.
This is important because although you can limit the amount of concurrency that happens with the TaskScheduler, it may not necessarily be important - if for say you have a top-level task that's marked as long running and is in-charge of processing your incoming requests. This would be helpful since all the tasks spawned by this top-level task will be part of that task's local queue and therefor won't spam all your available threads in your thread pool.

When you have a bunch of items and you want to process them asynchronously and with limited concurrency, the SemaphoreSlim is a great tool for this job. There are two ways that it can be used. One way is to create all the tasks immediately and have each task acquire the semaphore before doing it's main work, and the other is to throttle the creation of the tasks while the source is enumerated. The first technique is eager, and so it consumes more RAM, but it's more maintainable because it is easier to understand and implement. The second technique is lazy, and it's more efficient if you have millions of items to process.
The technique that you have used in your sample code is the second (lazy) one.
Here is an example of using two SemaphoreSlims in order to impose two maximum concurrency policies, one per request and one globally. First the eager approach:
private const int maxConcurrencyGlobal = 100;
private static SemaphoreSlim globalThrottler
= new SemaphoreSlim(maxConcurrencyGlobal, maxConcurrencyGlobal);
public async Task<ResponseObj[]> DoStuffAsync(IEnumerable<Input> inputData)
{
const int maxConcurrencyPerRequest = 5;
var perRequestThrottler
= new SemaphoreSlim(maxConcurrencyPerRequest, maxConcurrencyPerRequest);
Task<ResponseObj>[] tasks = inputData.Select(async input =>
{
await perRequestThrottler.WaitAsync();
try
{
await globalThrottler.WaitAsync();
try
{
return await ExecHttpRequestAsync(input);
}
finally { globalThrottler.Release(); }
}
finally { perRequestThrottler.Release(); }
}).ToArray();
return await Task.WhenAll(tasks);
}
The Select LINQ operator provides an easy and intuitive way to project items to tasks.
And here is the lazy approach for doing exactly the same thing:
private const int maxConcurrencyGlobal = 100;
private static SemaphoreSlim globalThrottler
= new SemaphoreSlim(maxConcurrencyGlobal, maxConcurrencyGlobal);
public async Task<ResponseObj[]> DoStuffAsync(IEnumerable<Input> inputData)
{
const int maxConcurrencyPerRequest = 5;
var perRequestThrottler
= new SemaphoreSlim(maxConcurrencyPerRequest, maxConcurrencyPerRequest);
var tasks = new List<Task<ResponseObj>>();
foreach (var input in inputData)
{
await perRequestThrottler.WaitAsync();
await globalThrottler.WaitAsync();
Task<ResponseObj> task = Run(async () =>
{
try
{
return await ExecHttpRequestAsync(input);
}
finally
{
try { globalThrottler.Release(); }
finally { perRequestThrottler.Release(); }
}
});
tasks.Add(task);
}
return await Task.WhenAll(tasks);
static async Task<T> Run<T>(Func<Task<T>> action) => await action();
}
This implementation assumes that the await globalThrottler.WaitAsync() will never throw, which is a given according to the documentation. This will no longer be the case if you decide later to add support for cancellation, and you pass a CancellationToken to the method. In that case you would need one more try/finally wrapper around the task-creation logic. The first (eager) approach could be enhanced with cancellation support without such considerations. Its existing try/finally infrastructure is
already sufficient.
It is also important that the internal helper Run method is implemented with async/await. Eliding the async/await would be an easy mistake to make, because in that case any exception thrown synchronously by the ExecHttpRequestAsync method would be rethrown immediately, and it would not be encapsulated in a Task<ResponseObj>. Then the task returned by the DoStuffAsync method would fail without releasing the acquired semaphores, and also without awaiting the completion of the already started operations. That's another argument for preferring the eager approach. The lazy approach has too many gotchas to watch for.

WPF GUI Performance with a large number of parallel tasks

I developed a small client (WPF) to make some stress test on our systems. Essentially it has to call various methods on an Asp.Net WebApi endpoint in parallel.
Each time you press "Start" it generates 4000 tasks (Async - Await) in parallel with request to stress, waits until they all finish, then it does it again - until the user clicks the stop button. The GUI is decorated with a progress bar and some counters: requests in error, completed request, in progress requests. I obtain these informations because the object that makes the batch of stress requests exposes some events:
var stressTestTask = new stressTestTask(LogService, configuration);
stressTestTask.ErrorRequestCountChanged += stressTestTask_ErrorRequestCountChanged;
stressTestTask.GoodRequestCountChanged += stressTestTask_GoodRequestCountChanged;
stressTestTask.TryRequestCountChanged += stressTestTask_TryRequestCountChanged;
_executionCancellationToken = new CancellationTokenSource();
await Task.Run(
() => stressTestTask.ApiStressTestTask(_executionCancellationToken.Token),
_executionCancellationToken.Token);
The whole execution is started from an ICommand (MVVM):
private RelayCommand _startCommand;
public RelayCommand StartCommand
{
get
{
return _startCommand ?? (_startCommand = new RelayCommand(
async () =>
{
await StartStressTest();
}));
}
}
RelayCommand is an implementation of ICommand from the library Mvvm-Light.
What I don't understand is this behaviour: if I configure my batch of tasks with a "low" number of tasks, for example 2000, the GUI doesn't freeze while executing. If instead I choose 5000 tasks, after a while it freezes. If then I open another instance of the .exe of my client and I choose 2000 on each, the GUI is responsive in both.
My first question is: why opening one instance with x tasks is worse in terms of responsivness than opening n instances with x/n tasks? Is it something related to Windows Scheduler and the fact that in the first case I have only one process?
My second questions is: how can I address the problem to make everything work on a single GUI? I thought about making a console application with the single batch of stress tests and calling a command from the GUI for each instance I want, in order to generate a process for every batch.

Are you handling those API events by invoking to the UI context? If you have many invocations occurring you will flood the dispatcher with operations and cause the UI to hang and lag behind user input.
Try batching the UI updates.

My first question is: why opening one instance with x tasks is worse in terms of responsivness than opening n instances with x/n tasks?
Possibly because you are getting more events to handle on the UI thread. I guess your ErrorBetCountChanged, GoodRequestCountChanged and TryRequestCountChanged event handlers are invoked on the UI thread and a lot of events being raised may flood the UI thread.
As Gusdor suggets you should probably find a way of batching the updates. Take a look at the reactive extensions (Rx): http://www.introtorx.com/content/v1.0.10621.0/01_WhyRx.html.
It has a Buffer method that may come in handy: http://www.introtorx.com/content/v1.0.10621.0/13_TimeShiftedSequences.html.
It also has en Obervable.FromEvent method that you can use to convert an event into an IObservable: https://msdn.microsoft.com/en-us/library/hh229241(v=vs.103).aspx.
My second questions is: how can I address the problem to make everything work on a single GUI?
You need to find a way - one or anoher - of updating the UI less frequently. Batching the updates and events should be a good starting point. Raising less notifications is another option. Maybe you need to both.

how can I address the problem to make everything work on a single GUI?
Send API requests in "proper" async-await manner with only one thread.
private async Task SendStressRequests()
{
var tasks = new List<Task>();
for (int i = 0; i < 4000; i++)
{
var task = SendApiRequestAsync();
tasks.Add(task);
}
await Task.WhenAll(tasks);
// Update UI with results
}

Running class in the background

I have a win form that starts a mini server type thing to serve web pages to the local browser, now the problem is, is that when I start it the application obviously won't run because there is a loop that waits for requests, for every request I create a new thread. Now should I create a complete new thread for the entire process or is there another way? The class is in a separate dll file I have created. Alone it works perfectly as expected.

I suggest you take a look at the ThreadPool Class. It is an easy-to-use option for handling multiple threads:
The thread pool enables you to use threads more efficiently by providing your application with a pool of worker threads that are managed by the system.
To queue a method for execution simply use the QueueUserWorkItem Method:
ThreadPool.QueueUserWorkItem(state =>
{
// do some work!
});
If you realize that you need more active concurrent threads to serve your clients, call the SetMaxThreads Method:
ThreadPool.SetMaxThreads(50, 10);
All requests above those numbers for worker threads and I/O threads remain queued until thread pool threads become available.

There are two ways here:
Async server. More difficult and more performance. http://robjdavey.wordpress.com/2011/02/12/asynchronous-tcp-server-example/
One thread per client. Easy to write but not applicable if you have many clients. http://tech.pro/tutorial/704/csharp-tutorial-simple-threaded-tcp-server
don't use loop until requests

I would follow #Thomas suggestion, but adding waitHandles to your ThreadPool to manage the callback cycles.
WaitCallback classMethod1= new WaitCallback(DoClassMethod1);
bool isQueued = ThreadPool.QueueUserWorkItem(classMethod1, waitHandle[0]);
WaitCallback classMethod2= new WaitCallback(DoClassMethod2);
bool isQueued = ThreadPool.QueueUserWorkItem(classMethod2, waitHandle[1]);
// do this if you want to wait for all requests complated
if (WaitHandle.WaitAll(waitHandles, 5000, false))
// request completed, show your result.
else
// problem.
void DoClassMethod1(object state)
{
// do your work
ManualResetEvent mre = (ManualResetEvent)state;
mre.Set();
}

Stop main thread until WCF Call returns?

I am new to web services and started out with WCF. I have multiple web service calls, each of which are done asynchronously. But my problem is that the main thread should stop until all the web service calls return and when all the web service calls return only then it should proceed.
I tried two things here :
used a Boolean variable and an infinite loop to stop the main thread. I change the value in the Completed method of the web service call. It resulted in an infinite loop and the Completed never got called.
I made the web service call from the main thread and after making the web service call, I called the Join method on the main thread to stop this thread until the web service returns.
This is the code snippet :
ServerMonitoringBoardDataService.ServerMonitoringBoardDataServiceClient c = new ServerMonitoringBoardDataService.ServerMonitoringBoardDataServiceClient();
c.GetEnvironmentAndServersCompleted += new EventHandler<ServerMonitoringBoardDataService.GetEnvironmentAndServersCompletedEventArgs>(c_GetEnvironmentAndServersCompleted);
c.GetEnvironmentAndServersAsync();
void c_GetEnvironmentAndServersCompleted(object sender, ServerMonitoringBoardDataService.GetEnvironmentAndServersCompletedEventArgs e)
{
var x = e.Result;
}
The reason I am facing problems is that,the multiple web service calls returns data as lists and I have done some operations on this data and then displayed it on the UI.The web service calls are made in a static constructor,so as to fetch the data only once and manipulate and display it many time.
But what happens is that the main thread does not stop until the data is fetched and moves onto perform the operations,where I get a Null Exception.
Please suggest a way out for me and also why the above approaches didn't work.
Thanks in advance for any kind of help on this.

The simplest to code would be to use the async operation model (i.e. you call BeginXXX and it returns an IAsyncResult to you) and follow the procedure outlined here for all of your async calls. This model is supported out of the box if you used the service proxy generator that comes with Visual Studio.
You will start by making the async calls:
var async1 = service.BeginFoo();
var async2 = service.BeginBar();
// more similar calls
And then end them one by one -- order does not matter:
var result1 = service.EndFoo(async1);
var result2 = service.EndBar(async2);
// etc
Of course in practice you will need to add error handling, so it won't be quite as short.