Develop Reference

How to keep BackgroundService task receiving mqtt messages without Console.ReadLine()

In my ASP.Net Core 6 application, a BackgroundService task called MqttClientService runs a MQTTNet client that handles incoming mqqt messages and responds with a message to indicate it was successful.
I have gotten the sample console app from the MQTTNet repo to work using Console.ReadLine(), however this feels like a hack for my use case. Is there a better way to keep the BackgroundService handling incoming messages without restarting constantly?
There is an example with Asp.Net Core and MQTTNet version 3, but it uses handles implemented by interfaces rather than async events that the library now uses: the MQTTNet's Upgrading Guide.
Any information will be appreciated, thank you.
MqttClientService.cs in Services/
using MQTTnet;
using MQTTnet.Client;
using System.Text;
namespace MqttClientAspNetCore.Services
{
public class MqttClientService : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
await Handle_Received_Application_Message();
}
}
public static async Task Handle_Received_Application_Message()
{
var mqttFactory = new MqttFactory();
using (var mqttClient = mqttFactory.CreateMqttClient())
{
var mqttClientOptions = new MqttClientOptionsBuilder()
.WithTcpServer("test.mosquitto.org")
.Build();
// Setup message handling before connecting so that queued messages
// are also handled properly.
mqttClient.ApplicationMessageReceivedAsync += e =>
{
Console.WriteLine("### RECEIVED APPLICATION MESSAGE ###");
Console.WriteLine($"+ Payload = {Encoding.UTF8.GetString(e.ApplicationMessage.Payload)}");
// Publish successful message in response
var applicationMessage = new MqttApplicationMessageBuilder()
.WithTopic("keipalatest/1/resp")
.WithPayload("OK")
.Build();
mqttClient.PublishAsync(applicationMessage, CancellationToken.None);
Console.WriteLine("MQTT application message is published.");
return Task.CompletedTask;
};
await mqttClient.ConnectAsync(mqttClientOptions, CancellationToken.None);
var mqttSubscribeOptions = mqttFactory.CreateSubscribeOptionsBuilder()
.WithTopicFilter(f =>
{
f.WithTopic("keipalatest/1/post");
f.WithAtLeastOnceQoS();
})
.Build();
await mqttClient.SubscribeAsync(mqttSubscribeOptions, CancellationToken.None);
Console.WriteLine("MQTT client subscribed to topic.");
// The line below feels like a hack to keep background service from restarting
Console.ReadLine();
}
}
}
}
Program.cs
using MqttClientAspNetCore.Services;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddHostedService<MqttClientService>();
var app = builder.Build();
// To check if web server is still responsive
app.MapGet("/", () =>
{
return "Hello World";
});
app.Run();

There's no need for Console.ReadLine or even the loop. The BackgroundService application code won't terminate when ExecuteAsync returns. If you want the application to terminate when ExecuteAsync terminates you have to actually tell it to through the IApplicationLifecycle interface.
I've found this the hard way the first time I tried using a Generic host for a command line tool. Which seemed to hang forever ....
ExecuteAsync can be used to set up the MQTT client and the event handler and just let them work. The code terminates only when StopAsync is called. Even then, this is done by signaling a cancellation token, not by aborting some worker thread.
The client itself can be built in the constructor, eg using configuration settings. Only ConnectAsync needs to be called in ExecuteAsync.
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await _client.ConnectAsync(_clientOptions, CancellationToken.None);
_logger.LogInformation("Connected");
await _client.SubscribeAsync(_subscriptionOptions, CancellationToken.None);
_logger.LogInformation("Subscribed");
}
The service code stops when StopAsync is called and the cancellation token is triggered. stoppingToken.Register could be used to call _client.DisconnectAsync when that happens, but Register doesn't accept an asynchronous delegate. A better option is to override StopAsync itself :
public virtual async Task StopAsync(CancellationToken cancellationToken)
{
await _client.DisconnectAsync();
await base.StopAsync(cancellationToken);
}
The constructor can create the client and register the message handler
public class MqttClientService : BackgroundService
{
ILogger<MqttClientService> _logger;
IMqttClient _client=client;
MqttClientOptions _clientOptions;
MqttSubscriptionOptions _subscriptionOptions;
string _topic;
public MqttClientService(IOptions<MyMqttOptions> options,
ILogger<MqttClientService> logger)
{
_logger=logger;
_topic=options.Value.Topic;
var factory = new MqttFactory();
_client = factory.CreateMqttClient();
_clientOptions = new MqttClientOptionsBuilder()
.WithTcpServer(options.Value.Address)
.Build();
_subscriptionOptions = factory.CreateSubscribeOptionsBuilder()
.WithTopicFilter(f =>
{
f.WithTopic(options.Value.Topic);
f.WithAtLeastOnceQoS();
})
.Build();
_client.ApplicationMessageReceivedAsync += HandleMessageAsync;
}
Received messages are handled by the HandleMessageAsync method :
async Task HandleMessageAsync(ApplicationMessageProcessedEventArgs e)
{
var payload=Encoding.UTF8.GetString(e.ApplicationMessage.Payload);
_logger.LogInformation("### RECEIVED APPLICATION MESSAGE ###\n{payload}",payload);
var applicationMessage = new MqttApplicationMessageBuilder()
.WithTopic(_topic)
.WithPayload("OK")
.Build();
await _client.PublishAsync(applicationMessage, CancellationToken.None);
_logger.LogInformation("MQTT application message is published.");
}
Finally, since BackgroundService implements IDisposable, we can use Dispose to dispose the _client instance :
public void Dispose()
{
Dispose(true);
}
protected virtual Dispose(bool disposing)
{
if(disposing)
{
_client.Dispose();
base.Dispose();
}
_client=null;
}

If your service has nothing else useful to do, it can just wait for the CancellationToken to fire:
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
try
{
await Handle_Received_Application_Message(stoppingToken);
}
catch (OperationCanceledException) { }
}
public static async Task Handle_Received_Application_Message(CancellationToken cancellationToken)
{
...
Console.WriteLine("MQTT client subscribed to topic.");
await Task.Delay(Timeout.Infinite, cancellationToken);
}

Masstransit Problem adding Payload to request/response in Unit Testing, alternativly using Pipes in Unit Tests

currently I have the problem that I want to write unit tests for Masstransit in .NET. My request/response consumer has some consumer filters, one of these filters are generating extra data as message payload and attaching this to the request message. In order to test my consumer in a unit test I would like to add the Payload.
Q1) Is it possible to add the payload to the request message
Q2) Alternativly, can I make a mocking filter and set it as consumer filter in the pipeline? (Which sets the payload)
This is my latest attempt:
public class ContactCommandConsumerTest
{
[Fact]
public async Task CreateContactOnUserRequestConsumer_RequestConsumer_IsAttached()
{
var harness = new InMemoryTestHarness { TestTimeout = TimeSpan.FromSeconds(5) };
[...]
var consumer = harness.Consumer<CreateContactOnUserRequestCommandConsumer>(() => new CreateContactOnUserRequestCommandConsumer(loggerConsumer, mapper,kontakteintragRep,machineTime));
var pipe = Pipe.New<PipeContext>(x => x.UseFilter(new MockFilter<PipeContext>()));
// harness.Consumer<CreateContactOnUserRequestCommandConsumer>();
await harness.Start();
try
{
harness.Bus.ConnectConsumePipe<CreateContactOnUserRequestCommandConsumer>(pipe);
var requestClient = await harness.ConnectRequestClient<CreateContactOnUserRequestCommand>();
var response = await requestClient.GetResponse<AcceptedResponse, FaultedResponse>(new
{
EntityInfo = "Vb48cc135-4593-4b96-bb29-2cf136b3d1ee",
});
Assert.True(consumer.Consumed.Select<CreateContactOnUserRequestCommand>().Any());
Assert.True(harness.Sent.Select<FaultedResponse>().Any());
}
finally
{
await harness.Stop();
}
}
}
internal class MockFilter<T> : IFilter<T> where T: class, PipeContext
{
public void Probe(ProbeContext context)
{
context.CreateFilterScope("mock");
}
public async Task Send(T context, IPipe<T> next)
{
context.GetOrAddPayload(() => new ContextUserPayload() { ContextUser = new Guid("dc6e091f-669e-45b3-9dd6-a36316f70527") });
await next.Send(context);
}
}
I tried to build a pipe and add it to "harness.bus.ConnectConsumerPipe". But the mock filter is never called ???

You use use the OnConfigureInMemoryBus event on the InMemoryTestHarness to add your filter to the bus endpoint.
Similar to:
harness.OnConfigureInMemoryBus += configurator =>
{
configurator.UseFilter(...);
}
To add a filter to the request, use:
using RequestHandle<TRequest> requestHandle = requestClient.Create(message, cancellationToken);
requestHandle.UseFilter(...);
return await requestHandle.GetResponse<TResponse>().ConfigureAwait(false);

Service Fabric service with WCF communication deployment

Here is a simple Service Fabric stateless service with WCF communication and it's client - a console app. It works well on the local cluster, client gets responce from the service. But I don't know how to communicate with a service if I deploy it in the cloud. What should I do to access it from console app?
SF Stateless service with WCF communications:
Contract:
[ServiceContract]
public interface IPresidentialService
{
[OperationContract]
Task<string> GetInfo();
}
Service:
internal sealed class PresidentialService : StatelessService, IPresidentialService
{
public PresidentialService(StatelessServiceContext context) : base(context)
{
}
public Task<string> GetInfo() => Task.FromResult($"Node {Context.NodeContext.NodeName} operating");
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
return new[]
{
new ServiceInstanceListener(context =>
new WcfCommunicationListener<IPresidentialService>(wcfServiceObject: this, serviceContext: context,
endpointResourceName: "WcfServiceEndpoint",
listenerBinding: WcfUtility.CreateTcpListenerBinding()))
};
}
}
}
Client console app:
WCF client:
public class PresidentialServiceClient : ServicePartitionClient<WcfCommunicationClient<IPresidentialService>>
{
public PresidentialServiceClient(
ICommunicationClientFactory<WcfCommunicationClient<IPresidentialService>> communicationClientFactory,
Uri serviceUri, ServicePartitionKey partitionKey = null,
TargetReplicaSelector targetReplicaSelector = TargetReplicaSelector.Default, string listenerName = null,
OperationRetrySettings retrySettings = null) : base(communicationClientFactory, serviceUri, partitionKey,
targetReplicaSelector, listenerName, retrySettings)
{
}
public Task<string> GetInfo() => InvokeWithRetryAsync(client => client.Channel.GetInfo());
}
Client App:
private static void Main(string[] args)
{
var binding = WcfUtility.CreateTcpClientBinding();
var partitionResolver = ServicePartitionResolver.GetDefault();
var wcfClientFactory =
new WcfCommunicationClientFactory<IPresidentialService>(binding,
servicePartitionResolver: partitionResolver);
var serviceUri = new Uri("fabric:/Application5/PresidentialService");
var client = new PresidentialServiceClient(wcfClientFactory, serviceUri, ServicePartitionKey.Singleton);
do
{
Console.WriteLine(client.GetInfo().Result);
Console.ReadKey();
} while (true);
}
Added to ServiceManifest.xml:
<Endpoints>
<Endpoint Name="WcfServiceEndpoint" />
</Endpoints>
UPDATE
Changed ServicePartitionResolver:
var partitionResolver = new ServicePartitionResolver("sfapp.westeurope.cloudapp.azure.com:19000");
Still not works.
UPDATE
Added a load balancer rule for TCP port 777.

When the service is running in the cloud, you can't use the default resolver.
The default ServicePartitionResolver assumes that the client is
running in same cluster as the service. If that is not the case,
create a ServicePartitionResolver object and pass in the cluster
connection endpoints.
Try something like
ServicePartitionResolver resolver = new ServicePartitionResolver("mycluster.cloudapp.azure.com:19000");
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-communication-wcf

Aborting WCF proxy that has non-cached ChannelFactory with CancellationToken causes deadlock

So here's how it is.
There is a WCF service. I've generated a proxy for it by "Add Service Reference" with task-based operations.
Endpoint address for that service might change for different users. I have no control over this service, vendor just does that this way.
Then I wrap that service into another type and through that type interaction with WCF service occurs.
It all looks like this:
//Generated code
public partial class MyServiceClient: System.ServiceModel.ClientBase<IMyService>, IMyService
{
public async Task<ResultDataContractType> GetResultsAsync(ArgsDataContractType args)
{
return base.Channel.GetResultsAsync(args);
}
}
...
...
...
//end of generated code
public class ClientFactory
{
public static IMyService CreateClient(string url)
{
var binding = new BasicHttpBinding();
var address = new EndpointAddress(url);
var client = new MyServiceClient(binding, address);
return client;
}
}
public class Wrapper()
{
public async Task<ResultType> GetResultsAsync(string url, ArgsType args, CancelationToke cancelationToken)
{
var client = ClientFactory.CreateClient(url);
try
{
cancellationToken.Register(target =>
{
var communicationObject = target as ICommunicationObject;
if (communicationObject != null)
{
communicationObject.Abort();
}
}, client);
ArgsDataContractType requestArgs = MapArgs(args);
ResultDataContractType result = await client.GetResultsAsync(args);
}
}
}
public class Consumer
{
public async void DoWork()
{
var args = new ArgsType
{
...
};
var cts = new CancellationTokenSource()
var wrapper = new Wrapper();
Task<ResultType> task = wrapper.GetResultsAsync("http://someaddress.com", args, cts.Token);
cts.Cancel(); //This is made intentionaly, normaly there is timeout timespan for token source
await task;
...
...
...
}
}
Consumer is actually the NUnit unit test, but calling the same code from ASP.NET application would also end up in a deadlock. It gets stuck on await task;
What I have noticed, that if I would set MyServiceClient.CacheSetting = CacheSetting.AlwaysOn; will make that code run without deadlocking.
Also, if I would configure MyServiceClient from App.config or Web.config will make code running without deadlocking. But if I would set MyServiceClient.CacheSetting = CacheSetting.AlwaysOff; before instantiating MyServiceClient this code will deadlock.
Also, configuring awaiter like this:
ResultDataContractType result = await client.GetResultsAsync(args).ConfigureAwait(false)
Will make code run without deadlocking.
Could you please enlighten me with any idea why's that deadlock doesn't happens when ChannelFactory for MyServiceClient is cached, and will happen if it is not cached?

TPL dataflow blocks inside a WCF duplex

I am new writer to SO, pls bear with me.
I have a WCF service with a duplex service contract. This service contract has an operation contact that suppose to do long data processing. I am constrained to limit the number of concurrent data processing to let's say max 3. My problem is that after the data processing I need to get back to the same service instance context so I call back my initiator endpoint passing the data processing result. I need to mention that due to various reasons I am constrained to TPL dataflows and WCF duplex.
Here is a demo to what I wrote so far
In a console library I simulate WCF calls
class Program
{
static void Main(string[] args)
{
// simulate service calls
Enumerable.Range(0, 5).ToList().ForEach(x =>
{
new System.Threading.Thread(new ThreadStart(async () =>
{
var service = new Service();
await service.Inc(x);
})).Start();
});
}
}
Here is what suppose to be the WCF service
// service contract
public class Service
{
static TransformBlock<Message<int>, Message<int>> transformBlock;
static Service()
{
transformBlock = new TransformBlock<Message<int>, Message<int>>(x => Inc(x), new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 3
});
}
static Message<int> Inc(Message<int> input)
{
System.Threading.Thread.Sleep(100);
return new Message<int> { Token = input.Token, Data = input.Data + 1 };
}
// operation contract
public async Task Inc(int id)
{
var token = Guid.NewGuid().ToString();
transformBlock.Post(new Message<int> { Token = token, Data = id });
while (await transformBlock.OutputAvailableAsync())
{
Message<int> message;
if (transformBlock.TryReceive(m => m.Token == token, out message))
{
// do further processing using initiator service instance members
// something like Callback.IncResult(m.Data);
break;
}
}
}
}
public class Message<T>
{
public string Token { get; set; }
public T Data { get; set; }
}
The operation contract is not really necessary to be async, but I needed the OutputAvailableAsync notification.
Is this a good approach or is there a better solution for my scenario?
Thanks in advance.

First, I think you shouldn't use the token the way you do. Unique identifiers are useful when communicating between processes. But when you're inside a single process, just use reference equality.
To actually answer your question, I think the (kind of) busy loop is not a good idea.
A simpler solution for asynchronous throttling would be to use SemaphoreSlim. Something like:
static readonly SemaphoreSlim Semaphore = new SemaphoreSlim(3);
// operation contract
public async Task Inc(int id)
{
await Semaphore.WaitAsync();
try
{
Thread.Sleep(100);
var result = id + 1;
// do further processing using initiator service instance members
// something like Callback.IncResult(result);
}
finally
{
Semaphore.Release();
}
}
If you really want to (or have to?) use dataflow, you can use TaskCompletionSource for synchronization between the operation and the block. The operation method would wait on the Task of the TaskCompletionSource and the block would set it when it finished computation for that message:
private static readonly ActionBlock<Message<int>> Block =
new ActionBlock<Message<int>>(
x => Inc(x),
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 3
});
static void Inc(Message<int> input)
{
Thread.Sleep(100);
input.TCS.SetResult(input.Data + 1);
}
// operation contract
public async Task Inc(int id)
{
var tcs = new TaskCompletionSource<int>();
Block.Post(new Message<int> { TCS = tcs, Data = id });
int result = await tcs.Task;
// do further processing using initiator service instance members
// something like Callback.IncResult(result);
}