Develop Reference

Parallel queued background tasks with hosted services in ASP.NET Core

I'm doing some tests with the new Background tasks with hosted services in ASP.NET Core feature present in version 2.1, more specifically with Queued background tasks, and a question about parallelism came to my mind.
I'm currently following strictly the tutorial provided by Microsoft and when trying to simulate a workload with several requests being made from a same user to enqueue tasks I noticed that all workItems are executed in order, so no parallelism.
My question is, is this behavior expected? And if so, in order to make the request execution parallel is it ok to fire and forget, instead of waiting the workItem to complete?
I've searched for a couple of days about this specific scenario without luck, so if anyone has any guide or examples to provide, I would be really glad.
Edit: The code from the tutorial is quite long, so the link for it is https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-2.1#queued-background-tasks
The method which executes the work item is this:
public class QueuedHostedService : IHostedService
{
...
public Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Queued Hosted Service is starting.");
_backgroundTask = Task.Run(BackgroundProceessing);
return Task.CompletedTask;
}
private async Task BackgroundProceessing()
{
while (!_shutdown.IsCancellationRequested)
{
var workItem =
await TaskQueue.DequeueAsync(_shutdown.Token);
try
{
await workItem(_shutdown.Token);
}
catch (Exception ex)
{
_logger.LogError(ex,
$"Error occurred executing {nameof(workItem)}.");
}
}
}
...
}
The main point of the question is to know if anyone out there could share the knowledge of how to use this specific technology to execute several work items at the same time, since a server can handle this workload.
I tried the fire and forget method when executing the work item and it worked the way I intended it to, several tasks executing in parallel at the same time, I 'm jut no sure if this is an ok practice, or if there is a better or proper way of handling this situation.

The code you posted executes the queued items in order, one at a time but also in parallel to the web server. An IHostedService is running per definition in parallel to the web server. This article provides a good overview.
Consider the following example:
_logger.LogInformation ("Before()");
for (var i = 0; i < 10; i++)
{
var j = i;
_backgroundTaskQueue.QueueBackgroundWorkItem (async token =>
{
var random = new Random();
await Task.Delay (random.Next (50, 1000), token);
_logger.LogInformation ($"Event {j}");
});
}
_logger.LogInformation ("After()");
We add ten tasks which will wait a random amount of time. If you put the code in a controller method the events will still be logged even after controller method returns. But each item will be executed in order so that the output looks like this:
Event 1
Event 2
...
Event 9
Event 10
In order to introduce parallelism we have to change the implementation of the BackgroundProceessing method in the QueuedHostedService.
Here is an example implementation that allows two Tasks to be executed in parallel:
private async Task BackgroundProceessing()
{
var semaphore = new SemaphoreSlim (2);
void HandleTask(Task task)
{
semaphore.Release();
}
while (!_shutdown.IsCancellationRequested)
{
await semaphore.WaitAsync();
var item = await TaskQueue.DequeueAsync(_shutdown.Token);
var task = item (_shutdown.Token);
task.ContinueWith (HandleTask);
}
}
Using this implementation the order of the events logged in no longer in order as each task waits a random amount of time. So the output could be:
Event 0
Event 1
Event 2
Event 3
Event 4
Event 5
Event 7
Event 6
Event 9
Event 8
edit: Is it ok in a production environment to execute code this way, without awaiting it?
I think the reason why most devs have a problem with fire-and-forget is that it is often misused.
When you execute a Task using fire-and-forget you are basically telling me that you do not care about the result of this function. You do not care if it exits successfully, if it is canceled or if it threw an exception. But for most Tasks you do care about the result.
You do want to make sure a database write went through
You do want to make sure a Log entry is written to the hard drive
You do want to make sure a network packet is sent to the receiver
And if you care about the result of the Task then fire-and-forget is the wrong method.
That's it in my opinion. The hard part is finding a Task where you really do not care about the result of the Task.

You can add the QueuedHostedService once or twice for every CPU in the machine.
So something like this:
for (var i=0;i<Environment.ProcessorCount;++i)
{
services.AddHostedService<QueuedHostedService>();
}
You can hide this in an extension method and make the concurrency level configurable to keep things clean.

Understanding fire and forget when using infinite loops

Can someone tell me what the best practice/proper way of doing this is?
I'm also using WPF, not a console or ASP.NET.
Using Listener to accept clients and spin off a new "thread" for each client that handles all the I/O and Exception catching for that client.
Method 1: Fire and forget, and just throw it into a variable to get rid of the warning.
public static async Task Start(CancellationToken token)
{
m_server = TcpListener.Create(33777);
m_server.Start();
running = true;
clientCount = 0;
// TODO: Add try... catch
while (!token.IsCancellationRequested)
{
var client = await m_server.AcceptTcpClientAsync().ConfigureAwait(false);
Client c = new Client(client);
var _ = HandleClientAsync(c);
}
}
Here's the Client Handler code:
public static async Task HandleClientAsync(Client c)
{
// TODO: add try...catch
while (c.connected)
{
string data = await c.reader.ReadLineAsync();
// Now we will parse the data and update variables accordingly
// Just Regex and some parsing that updates variables
ParseAndUpdate(data);
}
}
Method 2: The same thing... but with Task.Run()
var _ = Task.Run(() => HandleClientAsync());
Method 3: an intermediate non async function (doubt this is good. Should be Async all the way)
But this at least gets rid of the squiggly line without using the variable trick which kinda feels dirty.
while (!token.IsCancellationRequested)
{
var client = await m_server.AcceptTcpClientAsync().ConfigureAwait(false);
Client c = new Client(client);
NonAsync(c);
}
public static void NonAsync(VClient vc)
{
Task.Run(() => HandleClientAsync(vc));
}
Method 4: Make HandleClientAsync an Async void instead of Async Task (really bad)
public static async Task HandleClientAsync(Client c)
// Would change to
public static async Void HandleClientAsync(Client c)
Questions:
Is it any better to use Task.Run() When doing a fire and forget task?
Is it just accepted that you need to use the var _ = FireAndForget() trick to do fire and forget? I could just ignore the warning but something feels wrong about it.
If I wanted to update my UI from a Client, how would I do that? Would I just use a dispatcher?
Thanks guys

I've never been a fan of background workers which you expect to run for a long time, being run in a task. Tasks get scheduled to run on threads drawn from a pool. As you schedule these long running tasks, the thread pool gets smaller and smaller. Eventually all of the threads from the pool are busy running your tasks, and things get really slow and unmanageable.
My recommendation here? Use the Thread class and manage them yourselves. In this way, you keep your thread pool and the overhead for for tasks out of the picture.
Addendum - Producer Consumer Model
Another interesting question to consider: Do you really need a thread for every client? Threads are reasonably costly to create and maintain in terms of memory overhead, and if your client interaction is such that the client threads spend the vast majority of their time waiting around on something to do, then perhaps a producer consumer model is more suited to your use case.
Example:
Client connects on listening thread, gets put in a client queue
Worker thread responsible for checking to see if the clients need anything comes along through that queue every so often and checks - does the client have a new message to service? If so, it services all messages the client has, then moves on
In this way, you limit the number of threads working to just the number needed to manage the message queue. You can even get fancy and add worker threads dynamically based on how long its been since all the clients have been serviced.
If you insist
If you really like what you have going, I suggest refactoring what youre doing a bit so that rather than HandleClientAsync you do something more akin to CreateServiceForClient(c);
This could be a synchronous method that returns something like a ClientService. ClientService could then create the task that does what your HandleClientAsync does now, and store that task as a member. It could also provide methods like
ClientService.WaitUntilEnd()
and
ClientService.Disconnect() (which could set a cancellation token, also stored as a member variable)

Understanding async/await to manage multiple clients

I'm getting confused by await/async as I may still not get the point of its correct usage.
I have a simple WPF-UI and a ViewModel-method to start listening for clients which want to connect.
The following method is executed when the user clicks the button to start listening:
public void StartListening()
{
_tcpListener.Start(); // TcpListener
IsListening = true; // bool
Task.Factory.StartNew(DoStartListeningAsync, TaskCreationOptions.LongRunning);
}
The method DoStartListeningAsync which is called is defined like
private async Task DoStartListeningAsync()
{
while (IsListening)
{
using (var newClient = await _tcpListener.AcceptTcpClientAsync() /*.WithWaitCancellation(_cts.Token)*/)
{
apiClient = new ApiClient();
if(await apiClient.InitClientAsync()) // <-- here is the problem
{
// ... apiClient is now initialized
}
// ... do more and go back to await _tcpListener.AcceptTcpClientAsync()
}
}
}
The ApiClient class' InitClientAsync method is defined like:
public async Task<bool> InitClientAsync()
{
using (var requestStream = await _apiWebRequest.GetRequestStreamAsync())
{
_apiStreamWriter = new StreamWriter(requestStream);
}
// ... do somehing with the _apiStreamWriter
return true;
}
However, sometimes the InitClientAsync-call will get stuck at await _apiWebRequest.GetRequestStreamAsync() which then will freeze the execution of the DoStartListeningAsync-method at // <-- here is the problem.
In case the DoStartListeningAsync is stuck, no new connections will be handled which destroys my whole concept of handling multiple clients asynchronously.

Since you are using "await" keyword along the code path, you won't actually serve
multiple clients asynchronously.
The thing is, your code in the background thread will serve clients one by one. Take a deeper look - in the while loop you are getting request stream, wait till it is loaded, serve it, and then wait for other request stream.
async/await principle doesn't itself provide ability to serve multiple actions at the time. The only thing it is doing - prevents blocking current thread from being reusable by other code. So if going with async/await, you allow system yo use your current task thread, while it is waiting to other async action to complete (like _apiWebRequest.GetRequestStreamAsync()).
But since you are having one task, and you are waiting on every iteration of while loop - your code will work the same way, if you wrote it completely synchronous. The only profit is that you are using Task, and so .Net can reuse it's thread from thread pool while you are waiting for async actions to complete.
If you wan't to serve multiple clients asynchronously, you should either start multiple tasks, or don't wait till request is completely served - so actually remove some awaits from your code.
So you should move towards design, there you have one listening task/thread, that does nothing exept reading requests and putting it to the some queue. And having other tasks, that serve requests, reading it from the queue.
If I understood you correctly, you are using TcpListener under the hood. So what you need, is the loop where you accept new clients, and start serving them in the different thread/task without any waiting, so going directly to accepting other clients. But you can and should use async/await inside those handlers that serve clients.
Take a look at this answer - not completely your case (since I don't know all details of implementation), but just to get the idea.

Wrapping synchronous code into asynchronous call

I have a method in ASP.NET application, that consumes quite a lot of time to complete. A call to this method might occur up to 3 times during one user request, depending on the cache state and parameters that user provides. Each call takes about 1-2 seconds to complete. The method itself is synchronous call to the service and there is no possibility to override the implementation.
So the synchronous call to the service looks something like the following:
public OutputModel Calculate(InputModel input)
{
// do some stuff
return Service.LongRunningCall(input);
}
And the usage of the method is (note, that call of method may happen more than once):
private void MakeRequest()
{
// a lot of other stuff: preparing requests, sending/processing other requests, etc.
var myOutput = Calculate(myInput);
// stuff again
}
I tried to change the implementation from my side to provide simultaneous work of this method, and here is what I came to so far.
public async Task<OutputModel> CalculateAsync(InputModel input)
{
return await Task.Run(() =>
{
return Calculate(input);
});
}
Usage (part of "do other stuff" code runs simultaneously with the call to service):
private async Task MakeRequest()
{
// do some stuff
var task = CalculateAsync(myInput);
// do other stuff
var myOutput = await task;
// some more stuff
}
My question: Do I use the right approach to speed up the execution in ASP.NET application or am I doing unnecessary job trying to run synchronous code asynchronously?
Can anyone explain why the second approach is not an option in ASP.NET (if it is really not)?
Also, if such approach is applicable, do I need to call such method asynchronously if it is the only call we might perform at the moment (I have such case, when no other stuff there is to do while waiting for completion)?
Most of the articles in the net on this topic covers using async-await approach with the code, that already provides awaitable methods, but that's not my case. Here is the nice article describing my case, which doesn't describe the situation of parallel calls, declining the option to wrap sync call, but in my opinion my situation is exactly the occasion to do it.

It's important to make a distinction between two different types of concurrency. Asynchronous concurrency is when you have multiple asynchronous operations in flight (and since each operation is asynchronous, none of them are actually using a thread). Parallel concurrency is when you have multiple threads each doing a separate operation.
The first thing to do is re-evaluate this assumption:
The method itself is synchronous call to the service and there is no possibility to override the implementation.
If your "service" is a web service or anything else that is I/O-bound, then the best solution is to write an asynchronous API for it.
I'll proceed with the assumption that your "service" is a CPU-bound operation that must execute on the same machine as the web server.
If that's the case, then the next thing to evaluate is another assumption:
I need the request to execute faster.
Are you absolutely sure that's what you need to do? Are there any front-end changes you can make instead - e.g., start the request and allow the user to do other work while it's processing?
I'll proceed with the assumption that yes, you really do need to make the individual request execute faster.
In this case, you'll need to execute parallel code on your web server. This is most definitely not recommended in general because the parallel code will be using threads that ASP.NET may need to handle other requests, and by removing/adding threads it will throw the ASP.NET threadpool heuristics off. So, this decision does have an impact on your entire server.
When you use parallel code on ASP.NET, you are making the decision to really limit the scalability of your web app. You also may see a fair amount of thread churn, especially if your requests are bursty at all. I recommend only using parallel code on ASP.NET if you know that the number of simultaneous users will be quite low (i.e., not a public server).
So, if you get this far, and you're sure you want to do parallel processing on ASP.NET, then you have a couple of options.
One of the easier methods is to use Task.Run, very similar to your existing code. However, I do not recommend implementing a CalculateAsync method since that implies the processing is asynchronous (which it is not). Instead, use Task.Run at the point of the call:
private async Task MakeRequest()
{
// do some stuff
var task = Task.Run(() => Calculate(myInput));
// do other stuff
var myOutput = await task;
// some more stuff
}
Alternatively, if it works well with your code, you can use the Parallel type, i.e., Parallel.For, Parallel.ForEach, or Parallel.Invoke. The advantage to the Parallel code is that the request thread is used as one of the parallel threads, and then resumes executing in the thread context (there's less context switching than the async example):
private void MakeRequest()
{
Parallel.Invoke(() => Calculate(myInput1),
() => Calculate(myInput2),
() => Calculate(myInput3));
}
I do not recommend using Parallel LINQ (PLINQ) on ASP.NET at all.

I found that the following code can convert a Task to always run asynchronously
private static async Task<T> ForceAsync<T>(Func<Task<T>> func)
{
await Task.Yield();
return await func();
}
and I have used it in the following manner
await ForceAsync(() => AsyncTaskWithNoAwaits())
This will execute any Task asynchronously so you can combine them in WhenAll, WhenAny scenarios and other uses.
You could also simply add the Task.Yield() as the first line of your called code.

this is probably the easiest generic way in your case
return await new Task(
new Action(
delegate () {
// put your synchronous code here
}
)
);

How to ensure order of async operation calls?

[This appears to be a loooong question but I have tried to make it as clear as possible. Please have patience and help me...]
I have written a test class which supports an Async operation. That operation does nothing but reports 4 numbers:
class AsyncDemoUsingAsyncOperations
{
AsyncOperation asyncOp;
bool isBusy;
void NotifyStarted () {
isBusy = true;
Started (this, new EventArgs ());
}
void NotifyStopped () {
isBusy = false;
Stopped (this, new EventArgs ());
}
public void Start () {
if (isBusy)
throw new InvalidOperationException ("Already working you moron...");
asyncOp = AsyncOperationManager.CreateOperation (null);
ThreadPool.QueueUserWorkItem (new WaitCallback (StartOperation));
}
public event EventHandler Started = delegate { };
public event EventHandler Stopped = delegate { };
public event EventHandler<NewNumberEventArgs> NewNumber = delegate { };
private void StartOperation (object state) {
asyncOp.Post (args => NotifyStarted (), null);
for (int i = 1; i < 5; i++)
asyncOp.Post (args => NewNumber (this, args as NewNumberEventArgs), new NewNumberEventArgs (i));
asyncOp.Post (args => NotifyStopped (), null);
}
}
class NewNumberEventArgs: EventArgs
{
public int Num { get; private set; }
public NewNumberEventArgs (int num) {
Num = num;
}
}
Then I wrote 2 test programs; one as console app and another as windows form app. Windows form app works as expected when I call Start repeatedly:
But console app has hard time ensuring the order:
Since I am working on class library, I have to ensure that my library works correctly in all app models (Haven't tested in ASP.NET app yet). So I have following questions:
I have tested enough times and it appears to be working but is it OK to assume above code will always work in windows form app?
Whats the reason it (order) doesn't work correctly in console app? How can I fix it?
Not much experienced with ASP.NET. Will the order work in ASP.NET app?
[EDIT: Test stubs can be seen here if that helps.]

Unless I am missing something then given the code above I believe there is no way of guaranteeing the order of execution. I have never used the AsyncOperation and AsyncOperationManager class but I looked in reflector and as could be assumed AsyncOperation.Post uses the thread pool to execute the given code asynchronously.
This means that in the example you have provided 4 tasks will be queued to the thread pool synchronously in very quick succession. The thread pool will then dequeue the tasks in FIFO order (first in first out) but it's entirely possible for one of later threads to be scheduled before an earlier one or one of the later threads to complete before an earlier thread has completed its work.
Therefore given what you have there is no way to control the order in the way you desire. There are ways to do this, a good place to look is this article on MSDN.
http://msdn.microsoft.com/en-us/magazine/dd419664.aspx

I use a Queue you can then Enqueue stuff and Dequeue stuff in the correct order. This solved this problem for me.

The documentation for AsyncOperation.Post states:
Console applications do not synchronize the execution of Post calls. This can cause ProgressChanged events to be raised out of order. If you wish to have serialized execution of Post calls, implement and install a System.Threading.SynchronizationContext class.
I think this is the exact behavior you’re seeing. Basically, if the code that wants to subscribe to notifications from your asynchronous event wants to receive the updates in order, it must ensure that there is a synchronization context installed and that your AsyncOperationManager.CreateOperation() call is run inside of that context. If the code consuming the asynchronous events doesn’t care about receiving them in the correct order, it simply needs to avoid installing a synchronization context which will result in the “default” context being used (which just queues calls directly to the threadpool which may execute them in any order it wants to).
In the GUI version of your application, if you call your API from a UI thread, you will automatically have a synchronization context. This context is wired up to use the UI’s message queueing system which guarantees that posted messages are processed in order and serially (i.e., not concurrently).
In a Console application, unless if you manually install your own synchronization context, you will be using the default, non-synchronizing threadpool version. I am not exactly sure, but I don’t think that .net makes installing a serializing synchronization context very easy to do. I just use Nito.AsyncEx.AsyncContext from the Nito.AsyncEx nuget package to do that for me. Basically, if you call Nito.AsyncEx.AsyncContext.Run(MyMethod), it will capture the current thread and run an event loop with MyMethod as the first “handler” in that event loop. If MyMethod calls something that creates an AsyncOperation, that operation increments an “ongoing operations” counter and that loop will continue until the operation is completed via AsyncOperation.PostOperationCompleted or AsyncOperation.OperationCompleted. Just like the synchronization context provided by a UI thread, AsyncContext will queue posts from AsyncOperation.Post() in the order it receives them and run them serially in its event loop.
Here is an example of how to use AsyncContext with your demo asynchronous operation:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Starting SynchronizationContext");
Nito.AsyncEx.AsyncContext.Run(Run);
Console.WriteLine("SynchronizationContext finished");
}
// This method is run like it is a UI callback. I.e., it has a
// single-threaded event-loop-based synchronization context which
// processes asynchronous callbacks.
static Task Run()
{
var remainingTasks = new Queue<Action>();
Action startNextTask = () =>
{
if (remainingTasks.Any())
remainingTasks.Dequeue()();
};
foreach (var i in Enumerable.Range(0, 4))
{
remainingTasks.Enqueue(
() =>
{
var demoOperation = new AsyncDemoUsingAsyncOperations();
demoOperation.Started += (sender, e) => Console.WriteLine("Started");
demoOperation.NewNumber += (sender, e) => Console.WriteLine($"Received number {e.Num}");
demoOperation.Stopped += (sender, e) =>
{
// The AsyncDemoUsingAsyncOperation has a bug where it fails to call
// AsyncOperation.OperationCompleted(). Do that for it. If we don’t,
// the AsyncContext will never exit because there are outstanding unfinished
// asynchronous operations.
((AsyncOperation)typeof(AsyncDemoUsingAsyncOperations).GetField("asyncOp", BindingFlags.NonPublic|BindingFlags.Instance).GetValue(demoOperation)).OperationCompleted();
Console.WriteLine("Stopped");
// Start the next task.
startNextTask();
};
demoOperation.Start();
});
}
// Start the first one.
startNextTask();
// AsyncContext requires us to return a Task because that is its
// normal use case.
return Nito.AsyncEx.TaskConstants.Completed;
}
}
With output:
Starting SynchronizationContext
Started
Received number 1
Received number 2
Received number 3
Received number 4
Stopped
Started
Received number 1
Received number 2
Received number 3
Received number 4
Stopped
Started
Received number 1
Received number 2
Received number 3
Received number 4
Stopped
Started
Received number 1
Received number 2
Received number 3
Received number 4
Stopped
SynchronizationContext finished
Note that in my example code I work around a bug in AsyncDemoUsingAsyncOperations which you should probably fix: when your operation stops, you never call AsyncOperation.OperationCompleted or AsyncOperation.PostOperationCompleted. This causes AsyncContext.Run() to hang forever because it is waiting for the outstanding operations to complete. You should make sure that your asynchronous operations complete—even in error cases. Otherwise you might run into similar issues elsewhere.
Also, my demo code, to imitate the output you showed in the winforms and console example, waits for each operation to finish before starting the next one. That kind of defeats the point of asynchronous coding. You can probably tell that my code could be greatly simplified by starting all four tasks at once. Each individual task would receive its callbacks in the correct order, but they would all make progress concurrently.
Recommendation
Because of how AsyncOperation seems to work and how it is intended to be used, it is the responsibility of the caller of an asynchronous API that uses this pattern to decide if it wants to receive events in order or not. If you are going to use AsyncOperation, you should document that the asynchronous events will only be received in order by the caller if the caller has a synchronization context that enforces serialization and suggest that the caller call your API on either a UI thread or in something like AsyncContext.Run(). If you try to use synchronization primitives and whatnot inside of the delegate you call with AsyncOperation.Post(), you could end up putting threadpool threads in a sleeping state which is a bad thing when considering performance and is completely redundant/wasteful when the caller of your API has properly set up a synchronization context already. This also enables the caller to decide that, if it is fine with receiving things out of order, that it is willing to process events concurrently and out of order. That may even enable speedup depending on what you’re doing. Or you might even decide to put something like a sequence number in your NewNumberEventArgs in case the caller wants both concurrency and still needs to be able to assemble the events into order at some point.