I'm writing an app which reads messages from rabbitmq. A message contains a user's name and a correlation ID. There is a message handler which performs some calculations based on the message's payload and writes logs as well. A log entry must contain a user's name and a correlation ID from the message.
So, for each message, a new instance of MessageProcessor is created and a new instance of ExecutionContext should be created as well.
I'm using Microsoft.DependencyInjection which comes with .net core and I can't achieve the desired behaviour.
I tried to use Replace method but it did not work.
MessageProcessor.cs
internal sealed class MessageProcessor
{
public MessageProcessor(ILogger logger)
{
}
public async Task Process(Message message)
{
// do something useful
this.logger.WriteInfo("Message has been processed");
}
}
Logger.cs
internal sealed class Logger
{
public Logger(ExecutionContext context)
{
//context contains UserName and CorrelationID
}
}
Is it a good idea to rebind ExecutionContext for each message? Are there better solutions?
I wouldn't recommend creating a new instance of MessageProcessor with every arrived message. It is better to use the one instance of the processor and just delegate messages:
//register your processor in DI container
services.AddSingleton<MessageProcessor>();
var consumer = new EventingBasicConsumer(Model);
consumer.Received += OnMessageReceived;
Model.BasicConsume(
queue: queueName,
autoAck: true,
consumer: consumer);
//...
private void OnMessageReceived(object sender, BasicDeliverEventArgs e)
{
var message = e.Body; //here's your message in bytes
//deserialize into your model
IFormatter formatter = new BinaryFormatter();
using (MemoryStream stream = new MemoryStream(obj.Body))
{
myMessage = (Message) formatter.Deserialize(stream);
}
//...
}
RabbitMQ operates with bytes, not with JSON/XML/...
Can you post your code of Model class and how the connection to Rabbit looks like?
Related
I'm trying to write a notification system between a server and multiple clients using gRPC server streaming in protobuf-net.grpc (.NET Framework 4.8).
I based my service off of this example. However, if I understand the example correctly, it is only able to handle a single subscriber (as _subscriber is a member variable of the StockTickerService class).
My test service looks like this:
private readonly INotificationService _notificationService;
private readonly Channel<Notification> _channel;
public ClientNotificationService(INotificationService notificationService)
{
_notificationService = notificationService;
_notificationService.OnNotification += OnNotification;
_channel = Channel.CreateUnbounded<Notification>();
}
private async void OnNotification(object sender, Notification notification)
{
await _channel.Writer.WriteAsync(notification);
}
public IAsyncEnumerable<Notification> SubscribeAsync(CallContext context = default)
{
return _channel.AsAsyncEnumerable(context.CancellationToken);
}
INotificationService just has an event OnNotification, which is fired when calling its Notify method.
I then realized that System.Threading.Channels implements the Producer/Consumer pattern, but I need the Publisher/Subscriber pattern. When trying it out, indeed only one of the clients gets notified, instead of all of them.
It would also be nice if the server knew when a client disconnects, which seems impossible when returning _channel.AsAsyncEnumerable.
So how can I modify this in order to
serve multiple clients, with all of them being notified when OnNotification is called
and log when a client disconnects?
For 1, you'd need an implementation of a publisher/subscriber API; each call to SubscribeAsync will always represent a single conversation between gRPC endpoints, so you'll need your own mechanism for broadcasting that to multiple consumers. Maybe RX is worth investigating there
For 2, context.CancellationToken should be triggered by client-disconnect
Many thanks to Marc Gravell
I rewrote the NotificationService like this, using System.Reactive.Subjects (shortened) - no need for an event, use an Action instead:
public class NotificationService<T>
{
private readonly Subject<T> _stream = new Subject<T>();
public void Publish(T notification)
{
_stream.OnNext(notification);
}
public IDisposable Subscribe(Action<T> onNext)
return _stream.Subscribe(onNext);
}
}
My updated ClientNotificationService, which is exposed as a code-first gRPC service:
public class ClientNotificationService : IClientNotificationService
{
private readonly INotificationService<Notification> _notificationService;
public ClientNotificationService(INotificationService<Notification> notificationService)
{
_notificationService = notificationService;
}
public async IAsyncEnumerable<Notification> SubscribeAsync(CallContext context = default)
{
try
{
Channel<Notification> channel = Channel.CreateUnbounded<Notification>(
new UnboundedChannelOptions { SingleReader = true, SingleWriter = true });
CancellationTokenSource cts = CancellationTokenSource.CreateLinkedTokenSource(context.CancellationToken);
using (_notificationService.Subscribe(n => channel.Writer.WriteAsync(n, cts.Token)))
{
await foreach (Notification notification in channel.AsAsyncEnumerable(cts.Token))
{
yield return notification;
}
}
}
finally
{
// canceled -> log, cleanup, whatever
}
}
}
Note: Solution provided by OP on question section.
I have an interface IRabbitSender and the implementation RabbitSender
public class RabbitSender : IRabbitSender(){
public RabbitSender(string connection, string queue){
}
public void Send (object info){ // send message to specific queue }
}
I need different instances of the RabbitSender that will send information for different queues/connections.
But I only know which instance choose in runtime.
How can I do the DI? Actually, I have this, but I don't know how to distinct both and how to resolve the dependency in runtime.
services.AddTransient<IRabbitSender>(s => new RabbitSender(connection1, queueName1));
services.AddTransient<IRabbitSender>(s => new RabbitSender(connection2, queueName1));
There are several approaches to take here. Here are two to consider.
Consider making the interface generic:
IRabbitSender<TMessage>
This allows the consumer specifying which message to send, and in your configuration you can map message types to queues (tip: try keeping message names and queue names in sync as a convention; that drastically simplifies your life):
// Handy extension method
public static AddSender<TMessage>(
this IServiceCollection services, string con, string queue)
{
services.AddSingleton<IRabbitSender<TMessage>(new RabbitSender(con, queue);
}
// Registrations
services.AddSender<ShipOrder>(connection1, queueName1);
services.AddSender<CancelOrder>(connection2, queueName1);
Inject the full list queues into RabbitSender
Another option is to inject the mapping of messages types to queue information into RabbitSender. For instance:
public class RabbitSender : IRabbitSender {
private Dictionary<Type, (string connection, string queue)> senders;
public RabbitSender(Dictionary<Type, (string connection, string queue)> senders){
this.senders = senders;
}
public void Send(object info) {
var queueInfo = this.senders[info.GetType];
// TODO: Use queue info to send message to a queue
}
}
// Registration
servies.AddSingleton<IRabbitSender>(new RabbitSender(new Dictionary
{
{ typeof(ShipOrder), (connection1, queueName1) }
{ typeof(CancelOrder), (connection2, queueName1) }
}
You need to new up your RabbitSender class when you register instance it should be unique to. For example:
services.AddTransient<IRequireUniqueRabbit>(_ => {
return new RequireUniqueRabbit(new RabbitSender(connString1, queue1))
});
services.AddTransient<IRequireUniqueRabbit2>(_ => {
return new RequireUniqueRabbit2(new RabbitSender(connString2, queue2))
});
PS: Think about how you register classes holding the RabbitMQ connection, if you register them as transient or scoped they will dispose rabbitMQ connection each time they are disposed, most likely you want to avoid that
I have a custom LogEntry class that is designed for easy serialization. There are some log entries which will occur when doing an operation on an object that I want to send to the user. I also want to send those same entries to the console/whatever serilog sinks that are configured. My current approach looks like this:
public static void Info(this Case c, ILogger log, string message, params object[] values)
{
log.Information(message, values);
var formattedMessage = string.Empty; // TODO: use serilog to get the string.
// This is what I'm asking for help on!
var entry = new LogEntry
{
LogLevel = LogLevel.Info,
Message = formattedMessage,
PhaseType = c.CurrentPhase // <- it would be convenient if I could enrich
// the current serilog log with this info,
// but I don't know how to do that either.
};
c.Log.Add(entry);
}
Where my Case class is a POCO ready to be sent into newtonsoft for serializing. For the sake of completeness, the Case class contains this definition:
public class Case
{
// ...
public List<LogEntry> Log { get; set; } = new List<LogEntry>();
}
Perhaps my approach is entirely wrong. Hopefully I've given enough context to explain what I'm trying to accomplish. If this question takes me down a happier path: how do I create a custom temporary sink for an ILogger instance?
One option is to collect the Serilog LogEvents created by a logger call and use them to construct the rendered messages.
Here's an executable sketch of the general idea.
using System;
using System.Collections.Generic;
using Serilog;
using Serilog.Core;
using Serilog.Events;
// dotnet add package Serilog.Sinks.Console
class LogEventCollection : ILogEventEnricher
{
// Note, this is not threadsafe
public List<LogEvent> Events { get; } = new List<LogEvent>();
public void Enrich(LogEvent logEvent, ILogEventPropertyFactory _)
{
Events.Add(logEvent);
}
}
class Program
{
static void Main(string[] args)
{
Log.Logger = new LoggerConfiguration()
.WriteTo.Console()
.CreateLogger();
var collection = new LogEventCollection();
// Create an `ILogger` with the collector wired in. This
// also works with `LogContext.Push()`.
var collected = Log.ForContext(collection);
collected.Information("Hello");
collected.Information("World");
// One entry for each call above
foreach (var evt in collection.Events)
{
Console.WriteLine(evt.RenderMessage(null));
}
}
}
Output:
[14:23:34 INF] Hello
[14:23:34 INF] World
Hello
World
I would like to create one web job (listener) to listen to all queues in a storage. If there is any new message, then it triggers a handler.
Azure WebJobs SDK offers a solution which only listens to one queue:
public class Functions
{
// This function will get triggered/executed when a new message is written
// on an Azure Queue called queue.
public static async Task ProcessQueueMessage(
[QueueTrigger("%test%")] CloudQueueMessage message,
IBinder binder)
{
//do some stuff
}
}
This approach is good, but I need:
1) to listen to different queues
2) to inject a class to this class which I think I can't
So I am thinking of creating my own listener. I want to create several threats and each threat listens to one queue. Then when i run the web job it starts all threats.
I wonder if anyone can suggest a better solution. Code sample would be really good to have.
Thanks
You don't need to create your own listener unless you really want to. The Azure Webjobs SDK does the heavy lifting for you already.
Here's an example Functions.cs that can process data from different queues.
You can inject services into Functions.cs so that different queues are processed by different services if you want to.
private readonly IMyService _myService;
//You can use Dependency Injection if you want to.
public Functions(IMyService myService)
{
_myService = myService;
}
public void ProcessQueue1([QueueTrigger("queue1")] string item)
{
//You can get the message as a string or it can be strongly typed
_myService.ProcessQueueItem1(item);
}
public void ProcessQueue2([QueueTrigger("queue2")] MyObject item)
{
//You can get the message as a string or it can be strongly typed
_myService.ProcessQueueItem2(item);
}
Hope this helps
As #lopezbertoni suggested I created two methods in Functions and I've used IJobActivator to inject classes to Functions. See example below:
public class Program
{
static void Main()
{
var builder = new ContainerBuilder();
builder.RegisterType<MyMessageHandler>().As<IMessageHandler>();
builder.RegisterType<Functions>()
.InstancePerDependency();
var host = new JobHost(new JobHostConfiguration
{
JobActivator = new AutofacJobActivator(builder.Build())
});
host.RunAndBlock();
}
}
public class AutofacJobActivator : IJobActivator
{
private readonly IContainer _container;
public AutofacJobActivator(IContainer container)
{
_container = container;
}
public T CreateInstance<T>()
{
return _container.Resolve<T>();
}
}
public class Functions
{
private IMessageHandler _myService;
//You can use Dependency Injection if you want to.
public Functions(IMessageHandler myService)
{
_myService = myService;
}
// This function will get triggered/executed when a new message is written
// on an Azure Queue called queue.
public async Task ProcessQueueMessage1(
[QueueTrigger("test1")] CloudQueueMessage message,
IBinder binder)
{
_myService.HandleMessage(message.AsString);
Console.WriteLine("ProcessQueueMessage1 was run");
await Task.CompletedTask;
}
public async Task ProcessQueueMessage2(
[QueueTrigger("test2")] CloudQueueMessage message,
IBinder binder)
{
_myService.HandleMessage(message.AsString);
Console.WriteLine("ProcessQueueMessage2 was run");
await Task.CompletedTask;
}
}
I'm using NServiceBus 4.6 and Serilog. I've configured NServiceBus to use Serilog via:
global::NServiceBus.SetLoggingLibrary.Custom(new SeriLoggerFactory());
The factory itself is very simple as well:
public class SeriLoggerFactory : ILoggerFactory
{
public ILog GetLogger(Type type)
{
return new SeriLoggerAdapter(Log.ForContext(type));
}
public ILog GetLogger(string name)
{
var contextLogger = Log.ForContext("SourceContext", name);
return new SeriLoggerAdapter(contextLogger);
}
}
I'm definitely getting log entries related to NServiceBus, but one thing that's missing is the exception details when a message is processed but an exception is thrown. I can see the exception information in the NServiceBus message headers (either directly by viewing the message in the error queue, or via Service Insight), but the message that is logged by NServiceBus is missing most relevant information:
Message with '0d255d19-85f9-4915-a27c-a41000da12ed' id has failed FLR
and will be handed over to SLR for retry attempt 1
or
SLR has failed to resolve the issue with message
0d255d19-85f9-4915-a27c-a41000da12ed and will be forwarded to the
error queue at MYERRORQUEUE
Not having any details about the root exception makes debugging a bit difficult. It requires the developer go open up Service Insight, or open up a tool to view the message in the queue itself. Both are cumbersome, and both lack any extensiblity.
For instance, Serilog allows you to create ILogEventEnricher classes that can log special details about the exception in question - stuff that's not logged by a simple .ToString on the exception. Without NServiceBus actually logging my exceptions, I have no way of extracting these details.
What am I missing here?
NServiceBus has a class named NServiceBus.Faults.ErrorsNotifications which contains the following observables:
MessageSentToErrorQueue
MessageHasFailedAFirstLevelRetryAttempt
MessageHasBeenSentToSecondLevelRetries
You can subscribe to these observables when the endpoint starts, like in the following example which logs an error messeages are sent to first level retry:
public class GlobalErrorHandler : IWantToRunWhenBusStartsAndStops
{
private readonly ILogger _logger;
private readonly BusNotifications _busNotifications;
readonly List<IDisposable> _notificationSubscriptions = new List<IDisposable>();
public GlobalErrorHandler(ILogger logger, BusNotifications busNotifications)
{
_logger = logger;
_busNotifications = busNotifications;
}
public void Start()
{
_notificationSubscriptions.Add(_busNotifications.Errors.MessageHasFailedAFirstLevelRetryAttempt.Subscribe(LogWhenMessageSentToFirstLevelRetry));
}
public void Stop()
{
foreach (var subscription in _notificationSubscriptions)
{
subscription.Dispose();
}
}
private void LogWhenMessageSentToFirstLevelRetry(FirstLevelRetry message)
{
var properties = new
{
MessageType = message.Headers["NServiceBus.EnclosedMessageTypes"],
MessageId = message.Headers["NServiceBus.MessageId"],
OriginatingMachine = message.Headers["NServiceBus.OriginatingMachine"],
OriginatingEndpoint = message.Headers["NServiceBus.OriginatingEndpoint"],
ExceptionType = message.Headers["NServiceBus.ExceptionInfo.ExceptionType"],
ExceptionMessage = message.Headers["NServiceBus.ExceptionInfo.Message"],
ExceptionSource = message.Headers["NServiceBus.ExceptionInfo.Source"],
TimeSent = message.Headers["NServiceBus.TimeSent"]
};
_logger.Error("Message sent to first level retry. " + properties, message.Exception);
}
}
The observable is implemented by using Reactive Extensions, so you will have to install the NuGet package Rx-Core for this to work.