Update oct. 20, 2021
It seems that publishing to a queue works as expected. When I publish to a queue, the message is persisted on the queue. Conversely, when I publish to a topic, the message is not persisted.
Updated
I have added a simple console app that reproduces the same behavior down below.
I am trying to send a message to a topic in service bus from an Azure Function. I have tried this with a managed identity using Mass Transit. I have also tried this with a shared access key using the Azure.Messaging.ServiceBus nuget package. Both methods complete without exception, but the message is not in the topic.
This is what I see when sending from my function:
Additional settings on the topic:
I am able to put messages on the topic using the Service Bus Explorer within the Azure portal.
There are no subscriptions on this topic. I did have one setup as a test earlier, but it has since been deleted.
Mass Transit Setup (in Startup.cs)
private void ConfigureMassTransit(IServiceCollection services, IConfiguration config) {
const string KEY_QUEUE_SERVER = "REDACTED";
const string EMAIL_RETRY_TOPIC = "REDACTED";
const string EMAIL_SENT_TOPIC = "REDACTED";
services.AddMassTransit(x => {
x.UsingAzureServiceBus((context, cfg) => {
cfg.Host(new Uri(config[KEY_QUEUE_SERVER]), host => {
host.TokenProvider = TokenProvider.CreateManagedIdentityTokenProvider();
});
cfg.Message<EmailSentEvent>(m => m.SetEntityName(EMAIL_SENT_TOPIC));
cfg.Message<TransactionEmailFailedEvent>(m => m.SetEntityName(EMAIL_RETRY_TOPIC));
cfg.ConfigureEndpoints(context);
});
});
}
MassTransitQueueAdapter.cs
public class MassTransitQueueAdapter : IQueueAdapter {
#region attributes
private readonly IBus _bus;
#endregion
#region ctor
public MassTransitQueueAdapter(IBus bus) {
_bus = bus;
}
#endregion
#region methods
public void PublishFailure(TransactionEmailFailedEvent failedEvent) {
_bus.Publish(failedEvent);
}
public void PublishSuccess(EmailSentEvent sentEvent) {
_bus.Publish(sentEvent);
}
#endregion
}
ServiceBusQueueAdapter.cs
public class ServiceBusQueueAdapter : IQueueAdapter {
#region attributes
private readonly QueueContext _context;
#endregion
#region ctor
public ServiceBusQueueAdapter(QueueContext context) {
_context = context;
}
#endregion
#region methods
private static ServiceBusClient BuildClient(string connectionString) => new ServiceBusClient(connectionString);
public void PublishFailure(TransactionEmailFailedEvent failedEvent) {
throw new System.NotImplementedException();
}
public void PublishSuccess(EmailSentEvent sentEvent) {
ServiceBusClient client = BuildClient(_context.SentTopicConnectionString);
ServiceBusSender sender = client.CreateSender(_context.SentTopicName);
Task.Run(() => sender.SendMessageAsync(new ServiceBusMessage(JsonConvert.SerializeObject(sentEvent))));
}
#endregion
}
Simple Console App
class Program {
static void Main() {
string cs = "Endpoint=sb://REDACTED.servicebus.windows.net/;SharedAccessKeyName=test_with_manage;SharedAccessKey=REDACTED;";
ServiceBusClient client = new ServiceBusClient(cs);
ServiceBusSender sender = client.CreateSender("test_1");
sender.SendMessageAsync(new ServiceBusMessage("Hello World!"))
.Wait();
}
}
The problem here is with my understanding of topics in Azure Service Bus. I was expecting the Topic to act as a type of storage for messages. I was wrong in that assumption. The topic will only forward messages to subscriptions. The subscription can either persist the message or forward to another topic or queue.
With that knowledge in mind, I was able to get my Service Bus specific implementations working. I apparently still have a bit to learn about Mass Transit.
Related
Trying to send messages to all the partition in round-robin fashion but all the messages are going into the last partition. Can anyone please help me with that?
I am using Confluent.Kafka nuget package. My producer Configuration -
"ProducerConfiguration": {
"bootstrap.servers": "localhost:9092"
}
And my Kafka producer class -
public class Publisher
{
private ProducerConfig _producerConfig;
public Publisher(IOptions<ApplicationSetting> _applicationSetting)
{
var producerConfig = _applicationSetting.Value.KafkaConfiguration.ProducerConfiguration;
_producerConfig = new ProducerConfig(producerConfig);
}
public async Task<DeliveryResult<Null, TValue>> Publish<TValue>(TValue message, string topic = null)
{
using var producer = new ProducerBuilder<Null, TValue>(_producerConfig)
.SetValueSerializer(new JsonSerializer<TValue>())
.Build();
var topicName = String.IsNullOrEmpty(topic) ? message.GetType().Name : topic;
return await producer.ProduceAsync(topicName, new Message<Null, TValue>() { Value = message });
}
}
Publishing the message like -
public class DemoHandler : IRequestHandler<SendMail, string>
{
private readonly Publisher _publisher;
public DemoHandler(Publisher publisher)
{
_publisher = publisher;
}
public async Task<string> Handle(SendMail message, CancellationToken cancellationToken)
{
await _publisher.Publish(message);
return "Message sent";
}
}
All the messages are going to the last partition only -
Thanks in advance.
I added this answer
https://stackoverflow.com/a/71791802/628908
I faced this same issue. In this case the issue will be because you're re-creating a new IProducer instance each time you call your Publish method
If you could make sure the IProducer instance is the same every time you call Publish, that will let the producer to decide which partition send the message to.
We are trying to achieve a scenario where user can subscribe to notifications via channels.
For example, user can subscribe to notifications via Email, SMS, Real Time Notifications, etc.
For now, in order to achieve email notifications, I am implementing the IRealTimeNotifier in this way:
public class EmailNotifier : IRealTimeNotifier, ITransientDependency
{
private readonly IEmailSender _emailSender;
private readonly ISettingManager _settingManager;
public IAbpSession AbpSession { get; set; }
public EmailNotifier(IEmailSender emailSender, ISettingManager settingManager, IAbpSession abpSession)
{
this._emailSender = emailSender;
this._settingManager = settingManager;
this.AbpSession = NullAbpSession.Instance;
}
public async Task SendNotificationsAsync(UserNotification[] userNotifications)
{
List<Task> sendEmails = new List<Task>();
string fromAddress = this._settingManager.GetSettingValueForTenant(EmailSettingNames.Smtp.UserName, Convert.ToInt32(this.AbpSession.TenantId));
foreach (UserNotification notification in userNotifications)
{
notification.Notification.Data.Properties.TryGetValue("toAddresses", out object toAddresses);
notification.Notification.Data.Properties.TryGetValue("subject", out object sub);
notification.Notification.Data.Properties.TryGetValue("body", out object content);
notification.Notification.Data.Properties.TryGetValue("toAddresses", out object toAddresses);
List<string> toEmails = toAddresses as List<string> ;
string subject = Convert.ToString(sub);
string body = Convert.ToString(content);
toEmails.ForEach(to =>
{
sendEmails.Add(this._emailSender.SendAsync(fromAddress, to, subject, body));
});
}
await Task.Run(() =>
{
sendEmails.ForEach(async task =>
{
try
{
await task.ConfigureAwait(false);
}
catch (Exception)
{
// TODO #1: Add email to background job to be sent later.
// TODO #2: Add circuit breaker to email sending mechanism
/*
Email sending is failing because of two possible reasons.
1. Email addresses are wrong.
2. SMTP server is down.
3. Break the circuit while the SMTP server is down.
*/
// TODO #3 (Optional): Notify tenant admin about failure.
// TODO #4: Remove throw statement for graceful degradation.
throw;
}
});
});
}
}
The problem is with the IAbpSession, whether I inject it via property or constructor, at the time of execution of this notifier, the response has already been returned and the TenantId in this session is null, so the email is being sent with host configurations and not tenant configuration.
Similarly, I need to implement IRealTimeNotifier for SMS. I think I can reuse the SignalRRealTimeNotifier from ABP, but the tenantId in session is being set to null.
This is where the publisher is being called:
public class EventUserEmailer : IEventHandler<EntityCreatedEventData<Event>>
{
public ILogger Logger { get; set; }
private readonly IEventManager _eventManager;
private readonly UserManager _userManager;
private readonly IAbpSession _abpSession;
private readonly INotificationPublisher _notiticationPublisher;
public EventUserEmailer(
UserManager userManager,
IEventManager eventManager,
IAbpSession abpSession,
INotificationPublisher notiticationPublisher)
{
_userManager = userManager;
_eventManager = eventManager;
_notiticationPublisher = notiticationPublisher;
_abpSession = abpSession;
Logger = NullLogger.Instance;
}
[UnitOfWork]
public virtual void HandleEvent(EntityCreatedEventData<Event> eventData)
{
// TODO: Send email to all tenant users as a notification
var users = _userManager
.Users
.Where(u => u.TenantId == eventData.Entity.TenantId)
.ToList();
// Send notification to all subscribed uses of tenant
_notiticationPublisher.Publish(AppNotificationNames.EventCreated);
}
}
Can anybody recommend a better way? Or point anything out that we are doing wrong here.
I have not thought about how to handle DI of these particular notifiers yet. For testing purposes, I have given a named injection in my module like this:
IocManager.IocContainer.Register(
Component.For<IRealTimeNotifier>()
.ImplementedBy<EmailNotifier>()
.Named("Email")
.LifestyleTransient()
);
Current ABP version: 3.7.1
This is the information I have until now. If anything is needed, you can ask in comments.
// Send notification to all subscribed uses of tenant
_notificationPublisher.Publish(AppNotificationNames.EventCreated);
If you publish notifications like this, ABP's default implementation enqueues a background job.
There is no session in a background job.
You can get the tenantId like this:
var tenantId = userNotifications[0].Notification.TenantId;
using (AbpSession.Use(tenantId, null))
{
// ...
}
I am creating a RabbitBus.Bus with the RabbitBus.BusBuilder in my RabbitAdapter class.
public class RabbitAdapter
{
private Bus _bus;
public RabbitAdapter()
{
// The exchange and queue values are the same as what I see in RabbitMQ in browser
_bus = new BusBuilder()
.Configure(ctx => ctx.Consume<StatusUpdate>()
.WithExchange("exchange")
.WithQueue("Log"))
.Build();
}
public void Init()
{
// The [url] and [port] values are the same as what I see in browser
_bus.Connect("amqp://guest:guest#[url]:[port]/#/", TimeSpan.FromSeconds(10));
_bus.Subscribe<StatusUpdate>(OnHandle);
}
private void OnHandle(IMessageContext<StatusUpdate> statusUpdateContext)
{
Console.WriteLine(statusUpdateContext.Id);
}
public void Start()
{
}
}
I know that I'm probably just missing something here. The _connectionFactory is not null in the Bus, but the _connection is. It seems to timeout, I've even tried making the timeout one minute.
What you miss is although the console listens on 15672, the actual server listens on 5672 port.
I wish to process related messages in a batch e.g. processing the events CustomerCreated and PreferredCustomer with the same handler (same instance) within the same scope/transaction using the Rebus service bus.
The same handler is handling both messages/events:
class CustomerHandler : IHandleMessages<CustomerCreated>, IHandleMessages<PreferredCustomer>
{
Customer Customer { get; set; }
public CustomerHandler() {
Customer = new Customer();
}
public void Handle(CustomerCreated message) {
Customer.Name = message.Name;
Console.WriteLine(Customer);
}
public void Handle(PreferredCustomer message) {
Customer.Rebate = message.Rebate;
Console.WriteLine(Customer);
}
}
When sending the messages I use the batch operation (transport messages in NServiceBus)
bus.Advanced.Batch.Publish(
new CustomerCreated() { Name = "Anders" },
new PreferredCustomer() { Rebate = 10 });
To control the lifetime of the handler, I use Windsor Castle’s Scoped lifestyle
_container.Register(
Component.For<IHandleMessages<CustomerCreated>, IHandleMessages<PreferredCustomer>>)
.ImplementedBy<CustomerHandler>().LifestyleScoped());
And a custom UnitOfWorkManager that instanciates the ScopedUnitOfWork
class CustomUnitOfWorkManager : IUnitOfWorkManager
{
private readonly IWindsorContainer _container;
public CustomUnitOfWorkManager(IWindsorContainer container) {
_container = container;
}
public IUnitOfWork Create() {
return new ScopedUnitOfWork(_container);
}
}
class ScopedUnitOfWork : IUnitOfWork
{
private readonly IDisposable _scope;
public ScopedUnitOfWork(IWindsorContainer container) {
// Begin transaction
_scope = container.BeginScope();
}
public void Dispose() {
_scope.Dispose();
}
public void Commit() {
// Commit transaction
Console.WriteLine("Commiting");
}
public void Abort() {
// Rollback transaction
Console.WriteLine("Aborting!!!");
}
}
Finally configured Rebus to use the CustomUnitOfWorkManager
var bus = Configure.With(new WindsorContainerAdapter(_container))
.Transport(t => t.UseMsmqAndGetInputQueueNameFromAppConfig())
.MessageOwnership(d => d.FromRebusConfigurationSection())
.Events(x => x.AddUnitOfWorkManager(new CustomUnitOfWorkManager(_container)))
.CreateBus()
.Start();
Is this the correct approach?
My limited testing shows that this should work. I am even able to expand this to include transaction management against the data store within the ScopedUnitOfWork too.
Sounds like you have it nailed :)
If you get the correct commit/rollback behavior, then I'd say it's fine and dandy.
If you're interested in an alternative, you might want to take a look at the PerTransportMessage Castle Windsor scope accessor - it can be used like this:
container.Register(
Component
.For<ISomething>()
.ImplementedBy<Whatever>()
.LifestyleScoped<PerTransportMessage>()
);
which should be able to achieve the exact same behavior.
When creating a NServiceBus SendOnly endpoint, the purpose is to just fire-and-forget, i.e. just send a message and then someone else will take care of it. Which seems like the thing I need. I dont want any communication between the bus and the system handling messages. System "A" wants to notify system "B" about something.
Well the creation of an SendOnly endpoint if very straightforward but what about the system listening for messages from an SendOnly endpoint.
I'm trying to set up a listener in a commandline project that will handle messages. The messages get sent to the queue but they doesnt get handled by system "B".
Is this the wrong approach? Is a bus overkill for this type of functionality?
System A:
public class Program
{
static void Main(string[] args)
{
var container = new UnityContainer();
var bus = Configure.With()
.UnityBuilder(container)
.JsonSerializer()
.Log4Net()
.MsmqTransport()
.UnicastBus()
.SendOnly();
while(true)
{
Console.WriteLine("Send a message");
var message = new Message(Console.ReadLine());
bus.Send(message);
}
}
}
System B:
class Program
{
static void Main(string[] args)
{
var container = new UnityContainer();
var bus = Configure.With()
.UnityBuilder(container)
.JsonSerializer()
.Log4Net()
.MsmqTransport()
.UnicastBus()
.LoadMessageHandlers()
.CreateBus()
.Start();
Console.WriteLine("Waiting for messages...");
while(true)
{
}
}
}
public class MessageHandler : IHandleMessages<Message>
{
public void Handle(Message message)
{
Console.WriteLine(message.Data);
}
}
public class Message : IMessage
{
public Message()
{
}
public Message(string data)
{
Data = data;
}
public string Data { get; set; }
}
In the MessageEndpointMappings you need to update it as follows:
Replace DLL with the name of the assembly containing your messages (e.g. "Messages")
Change the Endpoint to the name of the queue which System B is reading from (You can check the queue name by looking in the MSMQ snapin under private queues).
<add Messages="Messages" Endpoint="SystemB" />
NServiceBus 3 automatically creates the queue name based upon the namespace of the hosting assembly.
Additionally, you may want to look at using the NServiceBus.Host to host your handlers instead of your own console application.