In my ASP.Net Core 6 application, a BackgroundService task called MqttClientService runs a MQTTNet client that handles incoming mqqt messages and responds with a message to indicate it was successful.
I have gotten the sample console app from the MQTTNet repo to work using Console.ReadLine(), however this feels like a hack for my use case. Is there a better way to keep the BackgroundService handling incoming messages without restarting constantly?
There is an example with Asp.Net Core and MQTTNet version 3, but it uses handles implemented by interfaces rather than async events that the library now uses: the MQTTNet's Upgrading Guide.
Any information will be appreciated, thank you.
MqttClientService.cs in Services/
using MQTTnet;
using MQTTnet.Client;
using System.Text;
namespace MqttClientAspNetCore.Services
{
public class MqttClientService : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
await Handle_Received_Application_Message();
}
}
public static async Task Handle_Received_Application_Message()
{
var mqttFactory = new MqttFactory();
using (var mqttClient = mqttFactory.CreateMqttClient())
{
var mqttClientOptions = new MqttClientOptionsBuilder()
.WithTcpServer("test.mosquitto.org")
.Build();
// Setup message handling before connecting so that queued messages
// are also handled properly.
mqttClient.ApplicationMessageReceivedAsync += e =>
{
Console.WriteLine("### RECEIVED APPLICATION MESSAGE ###");
Console.WriteLine($"+ Payload = {Encoding.UTF8.GetString(e.ApplicationMessage.Payload)}");
// Publish successful message in response
var applicationMessage = new MqttApplicationMessageBuilder()
.WithTopic("keipalatest/1/resp")
.WithPayload("OK")
.Build();
mqttClient.PublishAsync(applicationMessage, CancellationToken.None);
Console.WriteLine("MQTT application message is published.");
return Task.CompletedTask;
};
await mqttClient.ConnectAsync(mqttClientOptions, CancellationToken.None);
var mqttSubscribeOptions = mqttFactory.CreateSubscribeOptionsBuilder()
.WithTopicFilter(f =>
{
f.WithTopic("keipalatest/1/post");
f.WithAtLeastOnceQoS();
})
.Build();
await mqttClient.SubscribeAsync(mqttSubscribeOptions, CancellationToken.None);
Console.WriteLine("MQTT client subscribed to topic.");
// The line below feels like a hack to keep background service from restarting
Console.ReadLine();
}
}
}
}
Program.cs
using MqttClientAspNetCore.Services;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddHostedService<MqttClientService>();
var app = builder.Build();
// To check if web server is still responsive
app.MapGet("/", () =>
{
return "Hello World";
});
app.Run();
There's no need for Console.ReadLine or even the loop. The BackgroundService application code won't terminate when ExecuteAsync returns. If you want the application to terminate when ExecuteAsync terminates you have to actually tell it to through the IApplicationLifecycle interface.
I've found this the hard way the first time I tried using a Generic host for a command line tool. Which seemed to hang forever ....
ExecuteAsync can be used to set up the MQTT client and the event handler and just let them work. The code terminates only when StopAsync is called. Even then, this is done by signaling a cancellation token, not by aborting some worker thread.
The client itself can be built in the constructor, eg using configuration settings. Only ConnectAsync needs to be called in ExecuteAsync.
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await _client.ConnectAsync(_clientOptions, CancellationToken.None);
_logger.LogInformation("Connected");
await _client.SubscribeAsync(_subscriptionOptions, CancellationToken.None);
_logger.LogInformation("Subscribed");
}
The service code stops when StopAsync is called and the cancellation token is triggered. stoppingToken.Register could be used to call _client.DisconnectAsync when that happens, but Register doesn't accept an asynchronous delegate. A better option is to override StopAsync itself :
public virtual async Task StopAsync(CancellationToken cancellationToken)
{
await _client.DisconnectAsync();
await base.StopAsync(cancellationToken);
}
The constructor can create the client and register the message handler
public class MqttClientService : BackgroundService
{
ILogger<MqttClientService> _logger;
IMqttClient _client=client;
MqttClientOptions _clientOptions;
MqttSubscriptionOptions _subscriptionOptions;
string _topic;
public MqttClientService(IOptions<MyMqttOptions> options,
ILogger<MqttClientService> logger)
{
_logger=logger;
_topic=options.Value.Topic;
var factory = new MqttFactory();
_client = factory.CreateMqttClient();
_clientOptions = new MqttClientOptionsBuilder()
.WithTcpServer(options.Value.Address)
.Build();
_subscriptionOptions = factory.CreateSubscribeOptionsBuilder()
.WithTopicFilter(f =>
{
f.WithTopic(options.Value.Topic);
f.WithAtLeastOnceQoS();
})
.Build();
_client.ApplicationMessageReceivedAsync += HandleMessageAsync;
}
Received messages are handled by the HandleMessageAsync method :
async Task HandleMessageAsync(ApplicationMessageProcessedEventArgs e)
{
var payload=Encoding.UTF8.GetString(e.ApplicationMessage.Payload);
_logger.LogInformation("### RECEIVED APPLICATION MESSAGE ###\n{payload}",payload);
var applicationMessage = new MqttApplicationMessageBuilder()
.WithTopic(_topic)
.WithPayload("OK")
.Build();
await _client.PublishAsync(applicationMessage, CancellationToken.None);
_logger.LogInformation("MQTT application message is published.");
}
Finally, since BackgroundService implements IDisposable, we can use Dispose to dispose the _client instance :
public void Dispose()
{
Dispose(true);
}
protected virtual Dispose(bool disposing)
{
if(disposing)
{
_client.Dispose();
base.Dispose();
}
_client=null;
}
If your service has nothing else useful to do, it can just wait for the CancellationToken to fire:
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
try
{
await Handle_Received_Application_Message(stoppingToken);
}
catch (OperationCanceledException) { }
}
public static async Task Handle_Received_Application_Message(CancellationToken cancellationToken)
{
...
Console.WriteLine("MQTT client subscribed to topic.");
await Task.Delay(Timeout.Infinite, cancellationToken);
}
Related
I am working on a Realtime Application where I have to consume messages from Kafka and process the message and create a status dictionary to display on webpage. The problem is that while Kafka is running as BackgroundService in my Application, The ControllerBase class is not working or say my app doesn't launch localhost:5000 or so.
using (var consumer = new ConsumerBuilder<string, string>(
(IEnumerable<KeyValuePair<string, string>>)configuration).Build())
{
consumer.Subscribe(topic);
try
{
var message = consumer.Consume(cts.Token);
string consumedMessage = result.Message.Value.ToString();
}
catch (OperationCanceledException)
{
// Ctrl-C was pressed.
}
finally
{
consumer.Close();
}
}
}
return Task.CompletedTask;
}
Running this following service class in Background as soon as i comment out the consume part the localhost:5000 launches and if consume is present it doesn't.
Your consumer needs to run in separate thread or it will block the main thread and thus the startup process.
Run your consumer in a function of the service, which you call asynchronously in the ExecuteAsync function.
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await Task.Factory.StartNew(
() =>
{
while (!stoppingToken.IsCancellationRequested)
{
YourConsumeFunction();
}
}
);
}
OK so this is my first attempt at using a queue really and I'm confident there are some obvious issues here.
This code however has been working locally for me but am having issues having deployed to my test azure environment. At best it runs once but often not at all. The code is being hooked up using:
services.AddHostedService<ServiceBusListener>();
and then this is my main code:
public class ServiceBusListener : BackgroundService
{
private readonly QueueSettings _scoreUpdatedQueueSettings;
private readonly IEventMessageHandler _eventMessageHandler;
private QueueClient _queueClient;
public ServiceBusListener(IOptions<QueueSettings> scoreUpdatedQueueSettings,
IEventMessageHandler eventMessageHandler)
{
_eventMessageHandler = eventMessageHandler;
_scoreUpdatedQueueSettings = scoreUpdatedQueueSettings.Value;
}
public override Task StartAsync(CancellationToken cancellationToken)
{
_queueClient = new QueueClient(_scoreUpdatedQueueSettings.ServiceBusConnectionString,
_scoreUpdatedQueueSettings.QueueName);
var messageHandlerOptions = new MessageHandlerOptions(_eventMessageHandler.ExceptionReceivedHandler)
{
MaxConcurrentCalls = 1,
AutoComplete = false
};
_queueClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
return Task.CompletedTask;
}
public override Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
public async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
await _eventMessageHandler.ProcessMessagesAsync(message, token);
await _queueClient.CompleteAsync(message.SystemProperties.LockToken);
}
protected override Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
}
return Task.CompletedTask;
}
}
As mentioned locally this works fine, every time a message hits the queue my relevant functions run.
It feels as though some code should be living in ExecuteAsync but I don't want to create a handler every x do I?
For context this is running on a wep app in azure, I've done it like that as we have an api that can be hit to manage some of the related data.
There seems to be little around this on the net so any help would be appreciated.
TIA
I have a program that needs to terminate when an IHostedService background task encounters a certain scenario. I was hoping to do this by just throwing an exception in the background task that would get kicked up to the main function. I could then trigger the cancellation token to kill other background tasks.
My problem is that when I throw the exception, it kills the task and that's all. Everything else keeps running. Is there a way to do this, or a better way to do what I'm trying to do? Is there another way to have a backgrounds task trigger the common CancellationToken?
I included a simplified version of my issue in the code below.
If I comment out the await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken); line, the exception does what I want and I can trigger the CancellationToken. When it is in place, the task stops, but the program does not.
NOTE: In my messier code I have more IHostedServices running, which is why I'm trying to trigger cancelSource.Cancel()
public class Program
{
public static async Task Main(string[] args)
{
using (var cancelSource = new CancellationTokenSource())
{
try
{
await new HostBuilder()
.ConfigureServices((hostContext, services) =>
{
services.AddHostedService<TestService>();
})
.Build()
.RunAsync(cancelSource.Token);
}
catch (Exception E)
{
cancelSource.Cancel();
}
}
}
}
public class TestService : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
await Task.Delay(TimeSpan.FromSeconds(1), stoppingToken);
Console.WriteLine("loop 1");
throw new ApplicationException("OOPS!!");
}
}
}
Commenting the only line in the ExecuteAsync method with await operator makes your code run synchronously. If we look at the sources of BackgroundService.StartAsync, we can see that it checks for _executingTask.IsCompleted and it returns task that will contain your exception in case we don't have any await in ExecuteAsync method, otherwise it will return Task.CompletedTask and you won't be able to catch this exception from ExecuteAsync in Main method.
You can manage your services with IApplicationLifetime that can be injected in all your background services. For example, you can catch exception within ExecuteMethod and call ApplicationLifetime.StopApplication.
Example:
static async Task Main(string[] args)
{
await new HostBuilder()
.ConfigureServices((hostContext, services) =>
{
services.AddHostedService<TestService>();
services.AddHostedService<TestService2>();
})
.Build()
.RunAsync();
Console.WriteLine("App stoped");
}
Service 1
public class TestService : BackgroundService
{
private readonly IApplicationLifetime _applicationLifetime;
public TestService(IApplicationLifetime applicationLifetime)
{
_applicationLifetime = applicationLifetime;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
try
{
while (!_applicationLifetime.ApplicationStopping.IsCancellationRequested)
{
await Task.Delay(TimeSpan.FromSeconds(1), _applicationLifetime.ApplicationStopping);
Console.WriteLine("running service 1");
throw new ApplicationException("OOPS!!");
}
}
catch (ApplicationException)
{
_applicationLifetime.StopApplication();
}
}
}
Service 2
public class TestService2 : BackgroundService
{
private readonly IApplicationLifetime _applicationLifetime;
public TestService2(IApplicationLifetime applicationLifetime)
{
_applicationLifetime = applicationLifetime;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
try
{
while (!_applicationLifetime.ApplicationStopping.IsCancellationRequested)
{
await Task.Delay(100, _applicationLifetime.ApplicationStopping);
Console.WriteLine("running service 2");
}
}
catch (ApplicationException)
{
_applicationLifetime.StopApplication();
}
}
}
Output:
running service 2
running service 2
running service 1
App stoped
I'm using mvc with .net core, I need to run a task on start, and stop it when application stops. In Startup.cs I registered events for application start and stop. The problem is, I don't know how to run a task that has to be run in a specific class from startup. The task looks like this:
public void PreventStatusChange()
{
while (forceStatusChange)
{
foreach (var ext in GetExtensions())
{
ext.Status = StatusType.Available;
}
Thread.Sleep(1000);
}
}
Variable forceStatusChange is declared in the same class, so I don't see it from my Startup.cs. What is the best way to do it?
You need to create a class that implements IHostedService. This interface defines just two methods, StartAsync which is called when the application starts and StopAsync which is called when it terminates.
You need to register it as a hosted service with :
services.AddHostedService<TimedHostedService>();
Be careful to use AddHostedService, NOT AddSingleton. If you use AddSingleton the runtime won't know to call StartAsync and StopAsync when appropriate.
The article Background tasks with hosted services in ASP.NET Core shows how to implement a service using a timer :
internal class TimedHostedService : IHostedService, IDisposable
{
private readonly ILogger _logger;
private Timer _timer;
public TimedHostedService(ILogger<TimedHostedService> logger)
{
_logger = logger;
}
public Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Timed Background Service is starting.");
_timer = new Timer(DoWork, null, TimeSpan.Zero,
TimeSpan.FromSeconds(10));
return Task.CompletedTask;
}
private void DoWork(object state)
{
_logger.LogInformation("Timed Background Service is working.");
}
public Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Timed Background Service is stopping.");
_timer?.Change(Timeout.Infinite, 0);
return Task.CompletedTask;
}
public void Dispose()
{
_timer?.Dispose();
}
}
There's nothing particularly interesting in this code - just start a timer when StartAsync is called and stop it on StopAsync
Cancelling long-running tasks
When the runtime needs to recycle or stop, it will call the StopAsync method on all hosted services, wait a while for them to finish gracefully and then warn them to cancel immediatelly. After a short while it will go ahead and terminate the app or recycle it.
The cancellationToken parameter is used to signal that the service should stop immediatelly. Typically, that means that you'd have to write your own code to check it, warn your own task to terminate, wait for all of them to finish etc, similar to the code shown in this article
This is pretty much boilerplate though, which is why the BackgroundService class can be used to create a class that only needs to implement ExecuteAsync(CancellationToken). Starting and stopping that task is provided by BackgroundService, eg :
public class PollingService : BackgroundService
{
private readonly ILogger _logger;
public PollingService(ILogger<PollingService> logger)
{
_logger = logger;
}
protected async override Task ExecuteAsync(
CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
try
{
await DoSomething(cancellationToken);
await Task.Delay(1000,cancellationToken);
}
catch (Exception ex)
{
_logger.LogError(ex,
$"Error occurred executing {nameof(workItem)}.");
}
}
_logger.LogInformation("Queued Hosted Service is stopping.");
}
}
In this case Task.Delay() itself will be cancelled as soon as the runtime raises the cancellation token. DoSomething() itself should be implemented in a way that checks the cancellation token, eg pass it to any asynchronous method that accepts it as a parameter, test the IsCancellationRequested property on every loop and exit it it's raised.
For example :
protected async override Task ExecuteAsync(
CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
try
{
foreach (var ext in GetExtensions())
{
//Oops, time to cancel
if(cancellationToken.IsCancellationRequested)
{
break;
}
//Otherwise, keep working
ext.Status = StatusType.Available;
}
await Task.Delay(1000,cancellationToken);
}
catch (Exception ex)
{
...
}
}
_logger.LogInformation("Hosted Service is stopping.");
}
You can use BackgroundService
public class LongRunningService : BackgroundService
{
public LongRunningService()
{
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested && forceStatusChange)
{
foreach (var ext in GetExtensions())
{
ext.Status = StatusType.Available;
}
await Task.Delay(1000, stoppingToken);
}
}
protected override async Task StopAsync (CancellationToken stoppingToken)
{
// Run your graceful clean-up actions
}
}
And register it:
public void ConfigureServices(IServiceCollection services)
{
...
services.AddSingleton<IHostedService, LongRunningService>();
...
}
I have implemented a Kafka event bus as a singleton service in Net Core. The service itself is configured with Autofac in Startup.cs. The service has a Listen() method:
public void Listen()
{
using(var consumer = new Consumer<Null, string>(_config, null, new StringDeserializer(Encoding.UTF8)))
{
consumer.Subscribe(new string[] { "business-write-topic" });
consumer.OnMessage += (_, msg) =>
{
Console.WriteLine($"Topic: {msg.Topic} Partition: {msg.Partition} Offset: {msg.Offset} {msg.Value}");
consumer.CommitAsync(msg);
};
while (true)
{
consumer.Poll(100);
}
}
}
My understanding is that in order for this method to constantly listen for messages during the lifetime of the application, i have to call it in Program.cs from the web host by somehow getting the ServiceProvider associated with the host, then retrieving an instance of the service, and calling the method.
I have configured my Program.cs from the default Net Core 2.1 template to the following:
public class Program
{
public static void Main(string[] args)
{
var host = CreateWebHost(args);
host.Run();
}
public static IWebHost CreateWebHost(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseStartup<Startup>()
.Build();
}
Beyond having the host available so i can somehow access the services, i don't know where to go from here. I have searched for similar questions and read around in the official docs but i can't seem to figure out how to access the service so that i can call the Listen() method.
Is this the "go-to" way of accomplishing my goal? If so, how do i proceed? And if not - that is - if this kind of task is commonly accomplished in another way, how do i go about it?
Edit:
The answer below is still perfectly valid. There is a base-class called BackgroundService provided by Microsoft that can be used where you only need to implement ExecuteAsync(CancellationToken stoppingToken) rather than the whole interface of IHostedService. You can find it here. For that you will need to install the package Microsoft.Extensions.Hosting.
Previous and still valid answer:
I would suggest to use IHostedService. IHostedService implementations are registered as singletons and they run the whole time until the server shuts down.
Create this base class
public abstract class HostedService : IHostedService
{
private Task executingTask;
private CancellationTokenSource cancellationTokenSource;
public Task StartAsync(CancellationToken cancellationToken)
{
this.cancellationTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
this.executingTask = this.ExecuteAsync(this.cancellationTokenSource.Token);
return this.executingTask.IsCompleted ? this.executingTask : Task.CompletedTask;
}
public async Task StopAsync(CancellationToken cancellationToken)
{
if (this.executingTask == null)
{
return;
}
this.cancellationTokenSource.Cancel();
await Task.WhenAny(this.executingTask, Task.Delay(-1, cancellationToken));
}
protected abstract Task ExecuteAsync(CancellationToken cancellationToken);
}
Then create the consumer-host
public class ConsumerHost : HostedService
{
protected override async Task ExecuteAsync(CancellationToken cancellationToken)
{
using (var consumer = new Consumer<Null, string>(_config, null, new StringDeserializer(Encoding.UTF8)))
{
consumer.Subscribe(new string[] {"business-write-topic"});
consumer.OnMessage += (_, msg) =>
{
Console.WriteLine(
$"Topic: {msg.Topic} Partition: {msg.Partition} Offset: {msg.Offset} {msg.Value}");
consumer.CommitAsync(msg);
};
while (!cancellationToken.IsCancellationRequested) // will make sure to stop if the application is being shut down!
{
consumer.Poll(100);
await Task.Delay(TimeSpan.FromSeconds(10), cancellationToken);
}
}
}
}
Now in your startup-class in the ConfigureService method add the singleton
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<IHostedService, ConsumerHost>();
}
This service will now kick in when the webhost finished building and stop when you shutdown the server. No need to manually trigger it, let the webhost do it for you.
I think BackgroundService is what you need.
public class ListnerBackgroundService : BackgroundService
{
private readonly ListnerService service;
public ListnerBackgroundService(ListnerService service)
{
this.service = service;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
service.Listen();
return Task.CompletedTask;
}
}
And register it:
public void ConfigureServices(IServiceCollection services)
{
...
services.AddSingleton<IHostedService, ListnerBackgroundService>();
...
}