I have a RMQ Cluster setup on Amazon EC2. Now when I try to create a RabbitMQ host by setting the cluster members, I get the Broker Unreachable exception every time. However, if I comment that UseCluster method, things start working fine. But I am not able to enjoy full benefits of cluster in that case.
Following is the piece of code I am working with:
foreach(var server in servers)
{
try
{
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(server), hst =>
{
hst.Username(settings.UserName);
hst.Password(settings.Password);
hst.UseCluster(c =>
{
c.ClusterMembers = servers;
});
});
//Check if the connection is reachable
host.Settings.GetConnectionFactory().CreateConnection();
registrationAction?.Invoke(cfg, host);
});
AddBusToContainer(container, bus);
return bus;
}
catch (BrokerUnreachableException ex)
{
EventLog.WriteEntry($"Error trying to connect to {server}", $"{ex}", EventLogEntryType.Error);
continue;
}
}
In the above code, if I comment the UseCluster implementation, I am able to successfully test the connection. But I receive the BrokerUnreachableException with the above code.
Does anyone have any idea about how to configure RMQ Cluster when creating a MassTransit bus? It would be very helpful.
Thanks in advance.
To configure a cluster, you should only configure a single bus instance (you're configuring multiple in your example). I don't know what version of MassTransit you're using (it looks older, given the syntax), but this represents the current approach if you're hand-coding the bus.
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host("cluster", hst =>
{
hst.Username(settings.UserName);
hst.Password(settings.Password);
hst.UseCluster(c =>
{
foreach(var server in servers)
c.Node(server);
});
});
registrationAction?.Invoke(cfg);
});
AddBusToContainer(container, bus);
return bus;
Though I would suggest using the newer syntax with your container, to ensure components are properly registered.
container.AddMassTransit(x =>
{
x.UsingRabbitMq(cfg =>
{
cfg.Host("cluster", hst =>
{
hst.Username(settings.UserName);
hst.Password(settings.Password);
hst.UseCluster(c =>
{
foreach(var server in servers)
c.Node(server);
});
});
registrationAction?.Invoke(cfg);
});
})
Related
I feel like I didn't fully understand how Rebus works...
I have a system where an API should queue messages in a SQL table, and a Worker Service that will do the processing of the messages. I noticed that the publisher tries to route the messages to a handler and if it does not find a proper handler it raises an exception and moves the message to the error queue... I just want my API to queue the message and let the Worker Service to get the message when ready...
My API configuration is:
services.AddRebus((configure) => configure
.Logging(l => l.NLog())
.Transport(t => t.UseSqlServer(transportOptions, "QueueMsgs"))
.Options(o => o.SetNumberOfWorkers(1))
.Options(o => o.SimpleRetryStrategy(maxDeliveryAttempts: 10))
);
And I try to send messages like:
await _bus.Send(user);
If I debug the Worker Service at the same time, everything works perfectly, but my goal is to not need the Worker Service to be active to keep the messages in the queue...
What can I do?
I tried to use publish to queue the message
await _bus.Publish(user);
And adding the Subscription:
services.AddRebus((configure) => configure
.Logging(l => l.NLog())
.Transport(t => t.UseSqlServer(transportOptions, "QueueMsgs"))
.Options(o => o.SetNumberOfWorkers(1))
.Options(o => o.SimpleRetryStrategy(maxDeliveryAttempts: 10))
.Subscriptions(s => s.StoreInSqlServer(sqlConnectionString, "QueueMsgsSubs"))
//.Options(o => )
);
I'm quite lost, to be honest.
Please any help will be highly appreciated. Thank you!!
As I suspected, the fault was on me, I didn't have the correct configuration:
This is for the service configuration:
public static IServiceCollection AddRebusInjection(this IServiceCollection services, string sqlConnectionString)
{
string queueTable = "QueueMsgs";
services.AddRebus((configure) => configure
.Logging(l => l.NLog())
.Transport(t => t.UseSqlServerAsOneWayClient(sqlConnectionString))
.Options(o => o.SimpleRetryStrategy(maxDeliveryAttempts: 10, errorQueueAddress: "queueErrors"))
.Routing(r => r.TypeBased()
.Map<UserDTO>(queueTable)
.Map<StudyRoleDTO>(queueTable))
);
return services;
}
And we need to send messages with .Send:
[HttpPost, Route("RegisterAdminUser")]
public async Task<IActionResult> SetAdminUser(SetAdminUserRequest request)
{
_logger.LogTrace($"Received a SetAdminUser request: {JsonConvert.SerializeObject(request, Formatting.None)}");
var user = new UserDTO()
{
Name = request.LastName +", "+request.FirstName,
FirstName = request.FirstName,
LastName = request.LastName,
Email = request.Email,
UserGUID = request.UserGuid,
AdminType = await _userService.GetAdminTypeAsync(request.Roles)
};
await _userService.CheckUserAsync(user);
await _bus.Send(user);
return Accepted();
}
And that's it. Now my messages are nice and comfy waiting for the consumer to get them :)
I'm using masstransit to consume messages from an azure service bus. It's going greate for nom but I need now to add filter rules to my subscription.
I found some posts on the subject, like this one:
Is there a way to define a Azure Service Bus rule/filter when setting up a consumer?
but without many success...
My subscription is created properly when configuring my consumers like this, but it has the $Default 1=1 filter.
cfg.SubscriptionEndpoint<MyMessage>(mySubscription, cfg =>
{
se.Consumer<MyConsumer>(x => x.UseConcurrentMessageLimit(1));
});
I would like to add a different filter, but when I do this, the creation of the subscription seems to fail silently
cfg.SubscriptionEndpoint<MyMessage>(mySubscription, cfg =>
{
cfg.Rule = new CreateRuleOptions
{
Name = "Receiver filter",
Filter = new SqlRuleFilter("receiver='all'")
};
se.Consumer<MyConsumer>(x => x.UseConcurrentMessageLimit(1));
});
I'm I missing something?
I found my mistake... Everything is fine except for one thing. The rule name does not support spaces.
cfg.Rule = new CreateRuleOptions
{
Name = "ReceiverFilter", // instead of "Receiver filter"
Filter = new SqlRuleFilter("receiver='all'")
};
Consuming a message published with NServiceBus, using IQueueClient/Functions-ServiceBusTrigger (Microsoft.Azure.ServiceBus)
I'm working in a WebJob using .NET Core and Microsoft.Azure.ServiceBus to consume a message that has been published by a separate service using NServiceBus. My initial approach with this WebJob was to use a class Functions.cs with a method ProcessMessage that uses the attribute ServiceBusTrigger
Below is an example of how my Function.cs looks like:
public class Functions
{
public Task ProcessAuditLogMessage([ServiceBusTrigger("MyQueue")]
Message message)
{
var messageBody = Encoding.UTF8.GetString(message.Body);
var auditLogMessage = JsonConvert
.DeserializeObject<MyMessage>(messageBody);
_logger.Information("Hello World");
return Task.CompletedTask;
}
}
In Program.cs, I have:
class Program
{
static async Task Main()
{
var serviceCollection = new ServiceCollection();
serviceCollection.AddWebJobs(o => o.AllowPartialHostStartup = true);
var builder = new HostBuilder()
.UseServiceProviderFactory(
new AutofacServiceProviderFactory(serviceCollection))
.ConfigureContainer<ContainerBuilder>(b =>
{
BuildAutofacContainer();
})
.ConfigureWebJobs(b =>
{
b.AddServiceBus(o =>
{
o.ConnectionString = configProvider.AuditLogServiceBus;
});
});
var host = builder.Build();
using (host)
{
await host.RunAsync();
}
}
private static IContainer BuildAutofacContainer(IServiceColletion
serviceCollection)
{
...
}
}
I was expecting that this logic would consume the messages published in the Queue but so far the messages get moved to the Dead-letter count with a DeadLetterReason: MaxDeliveryCountExceeded and the error description Message could not be consumed after 10 delivery attempts which gives me the indication that at least there is an attempt to get these messages delivered to my consumer.
Unfortunately that's all I have in terms of error messages/logs (I'm in the process to set up some logs from my Azure Dashboard and see if I can get more detailed logs)
Has anyone come across the scenario o having to consume messages, that have been published with NServiceBus, using Microsoft.Azure.ServiceBus instead of NServiceBus (on the consumer side). Maybe I'm missing something...
I'm using Azure Service Bus as transport for MassTransit and I'm trying to send message(command) to queue:
var sendEndpoint = await busControl.GetSendEndpoint(sericeUri);
sendEndpoint.Send<ISimpleRequest>(new SimpleRequest(customerId));
Also I try to connect to this queue by two consumers:
var _busControl = Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
var host = cfg.Host("...", h =>
{
h.OperationTimeout = TimeSpan.FromMinutes(1);
});
cfg.ReceiveEndpoint(host, "queueName",
e => { e.Consumer<RequestConsumer>(); });
cfg.UseServiceBusMessageScheduler();
});
The same code with same queue name for second consumer.
After I send message only one consumer get the response. So how I can config this to work with two or more consumers?
If you want to get two copies of the message, one for each consumer, you should use two separate queues and Publish the message. In this case, MassTransit will send it to the topic, and each queue will receive a copy forwarded from the topic.
I'm trying to listen to the error queue to process failed messages but I can't seem to get MassTransit not to set bindings on message that i want it to listen to within the configuration. The configuration is below and is using v3 of MassTransit:
var hostAddress = new Uri("rabbitmq://localhost/");
var username = "guest";
var password = "guest";
_busControl = MassTransit.Bus.Factory.CreateUsingRabbitMq(configurator =>
{
var host = configurator.Host(hostAddress, h =>
{
h.Username(username);
h.Password(password);
});
configurator.ReceiveEndpoint(host, "myqueue_error",
endpointConfigurator =>
{
endpointConfigurator.Handler<SomeMessage>(context =>
{
return Console.Out.WriteLineAsync("Woop");
});
});
});
In the above example it will set bindings up for anything that publishes SomeMessage and direct them in to the myqueue_error which I only want messages going in to this queue which has been forward from the service that are failing. Is there anyway to consume messages from a queue but tell MassTransit not to get bindings up for them?
Update - Potential Solution
It seems that I don't need to setup a ReceiveEndpoint but I can just rename the controlbus to accept the message that I care about, This will then be able to process these messages without creating exchange bindings to the messages.
Below is the altered code, not sure if this is an ideal way but it works
var hostAddress = new Uri("rabbitmq://localhost/");
var username = "guest";
var password = "guest";
_busControl = MassTransit.Bus.Factory.CreateUsingRabbitMq(configurator =>
{
configurator.Host(hostAddress, h =>
{
h.Username(username);
h.Password(password);
});
// We need to make the queue look like the error queue
configurator.BusQueueName = $"{_queue}_error";
configurator.Durable = true;
configurator.AutoDelete = false;
configurator.SetQueueArgument("x-expires", null);
});
var connectHandle = _busControl.ConnectHandler<SomeMessage>(context => Console.Out.WriteLineAsync("Woop"));
_busHandle = _busControl.Start();
_busHandle.Ready.Wait();
// Wait
// Clean up
connectHandle.Disconnect();
_busHandle.Stop
From a lot of digging around I found a better solution which I totally missed from the documentation.
It seems that we can listen to messages by subscribing consumers to listen to Fault This works perfect for what I've been trying to achieve and we can also keep the error queues in tack.
http://docs.masstransit-project.com/en/mt3/usage/exceptions.html#handling-exceptions
So the final bit of configuration that i settle with is the following:
var hostAddress = new Uri("rabbitmq://localhost/");
var username = "guest";
var password = "guest";
_busControl = MassTransit.Bus.Factory.CreateUsingRabbitMq(configurator =>
{
var host = configurator.Host(hostAddress, h =>
{
h.Username(username);
h.Password(password);
});
configurator.ReceiveEndpoint(host, "error_listener",
endpointConfigurator =>
{
endpointConfigurator.Handler<Fault<SomeMessage>>(context =>
{
return Console.Out.WriteLineAsync("Woop");
});
});
});