Develop Reference

MassTransit could not send message to queue who has x-message-ttl, received none but current is the value

I'm using MS newly. I'm thinking to build something like this. I have two queues, ConsumeQueue and DelayQueue.
DelayQueue has no consumer and defined dead letter exchange ConsumeQueue.
DelayQueue "x-message-ttl" queue argument setted for example 10 seconds.
Consumer consuming ConsumeQueue and process data. Handle unsuccessful datas and send message to DelayQueue.
After 10 seconds message moving to dead-letter ConsumeQueue and consuming less data and process.
note: I don't want to scheduled message plugin.
Startup configuration, please note if it can be better:
services.AddMassTransit(configure =>
{
configure.AddBus(factory =>
{
return Bus.Factory.CreateUsingRabbitMq(mqConfig =>
{
mqConfig.Host(eventBusOptions.Host, hostConfigurator =>
{
hostConfigurator.Username(eventBusOptions.Username);
hostConfigurator.Password(eventBusOptions.Password);
});
mqConfig.ReceiveEndpoint($"ConsumeQueue", endpointConfig =>
{
endpointConfig.Consumer<ConsumeMessage>(factory);
});
mqConfig.ReceiveEndpoint($"DelayedQueue", endpointConfig =>
{
endpointConfig.SetQueueArgument("x-message-ttl", TimeSpan.FromSeconds(10));
endpointConfig.BindDeadLetterQueue($"{nameof(FundValueConsumer)}", $"{nameof(FundValueConsumer)}", configure: a =>
{
a.Lazy = true;
a.Durable = true;
});
});
});
});
});
And inside ConsumeQueue consumer, the send message to DelayQueue part:
...
var endpoint = await _busControl.GetSendEndpoint(new Uri("queue:DelayedQueue"));
await endpoint.Send(retryCodeMessage);
//Also i tried this too
await endpoint.Send(retryCodeMessage, (context) =>
{
context.TimeToLive = TimeSpan.FromSeconds(15);
});
...
When send the message line throwing:
The AMQP operation was interrupted: AMQP close-reason,
initiated by Peer, code=406, text='PRECONDITION_FAILED
- inequivalent arg 'x-message-ttl' for queue 'DelayedQueue' in vhost 'dev':
received none but current is the value '10000' of type 'signedint'', classId=50, methodId=10
What should i do? What i did, it could be wrong. Please show the correct implementation of the plan I want

Configure Deduplication on Azure Service Bus Queue with MassTransit

I have found the following question (How to configure the RequiresDuplicateDetection for AzureServiceBus topics) about how to set the RequiresDuplicationDetection property when configuring a publish topic from a producer application in MassTransit. However, I have not been able to find out how to do it for commands that are transmitted to a queue with Send rather than Publish.
Additionally, I have found that when configuring a consumer of one the queues in question I can set the property easily, as shown below. This however is not ideal for my use case, if possible I would much rather the producer set this property when it starts and creates the queue.
cfg.ReceiveEndpoint(queue, e =>{
e.RequiresDuplicateDetection = true;
e.ConfigureConsumer<JobEventConsumer>(registrationContext, consumerConfig =>{
consumerConfig.UseMessageRetry(r =>{
r.Interval(10, TimeSpan.FromMilliseconds(200));
r.Ignore<ValidationException>();
});
});
});
Update: After a bit more investigation I have also found that setting the property to true at the global config level doesn't seem to work either. Code shown below
class Program {
static async Task Main(string[] args) {
EndpointConvention.Map<ExtractionRequest>(new Uri("queue:test-queue"));
var busControl = Bus.Factory.CreateUsingAzureServiceBus(cfg =>{
cfg.Host("My connection string");
cfg.RequiresDuplicateDetection = true;
cfg.EnablePartitioning = true;
});
await busControl.StartAsync();
try {
do {
string value = await Task.Run(() =>{
Console.WriteLine("Enter message (or quit to exit)");
Console.Write("> ");
return Console.ReadLine();
});
if ("quit".Equals(value, StringComparison.OrdinalIgnoreCase)) break;
await busControl.Send<ExtractionRequest>(new {});
}
while (true);
}
finally {
await busControl.StopAsync();
}
}
}
public interface ExtractionRequest {}
Any advice on how to turn RequiresDuplicationDetection on for a queue from the producer is welcomed.
Thanks in advance, James.

You can't set queue properties from a message sender, it's the responsibility of the receive endpoint.
The receive endpoint is the responsible component because it's declaring the queue and related attributes.
The reason publish is different is because topics can be configured by the producer, since there may be multiple consumer subscriptions on a single topic.

MassTransit: Initialize consumer constructor with IRequestClient

1) Hi. I'm learning MassTransit with RabbitMQ, but stuck with using Request/Respond. I read a lot of articles and try to write console app using MassTransit documentation. But still can't find any information about initializing consumer with IRequestClient interface. Here is My code:
static void Main(string[] args){
var serviceAddress = new Uri("loopback://localhost/notification.service");
var requestTimeout = TimeSpan.FromSeconds(120);
var bus = BusConfigurator.ConfigureBus((cfg, host) =>
{
cfg.ReceiveEndpoint(host, RabbitMqConstants.NotificationServiceQueue, e =>
{
e.Consumer(() => new OrderRegisteredConsumer(???));
});
});
IRequestClient<ISimpleRequest, ISimpleResponse> client = new MessageRequestClient<ISimpleRequest, ISimpleResponse>(bus, serviceAddress, requestTimeout);
bus.Start();
Console.WriteLine("Listening for Order registered events.. Press enter to exit");
Console.ReadLine();
bus.Stop();
}
And my consumer
public class OrderRegisteredConsumer: IConsumer<IOrderRegisteredEvent>
{
private static IBusControl _bus;
IRequestClient<ISimpleRequest, ISimpleResponse> _client;
public OrderRegisteredConsumer(IRequestClient<ISimpleRequest, ISimpleResponse> client)
{
_client = client;
}
public async Task Consume(ConsumeContext<IOrderRegisteredEvent> context)
{
await Console.Out.WriteLineAsync($"Customer notification sent: Order id {context.Message.OrderId}");
ISimpleResponse response = await _client.Request(new SimpleRequest(context.Message.OrderId.ToString()));
Console.WriteLine("Customer Name: {0}", response.CustomerName);
}
}
How can I put my client inside
e.Consumer(() => new OrderRegisteredConsumer(???));
2) I Also try to find some information about Request/Respond in sagas, but, unfortunately, all I find is https://github.com/MassTransit/MassTransit/issues/664
I will appreciate If someone have an example of using this in sagas, or if someone could provide some links, where I can read about this more.

You need the client variable to be available, but the client doesn't need to be ready at the moment you configure the endpoint. endpoint.Consumer does not instantiate the consumer straight away, it just needs a factory delegate, which will instantiate the consumer when a message comes for this consumer.
Since the delegate is a reference type, you can assign it later in your code.
So this would work:
IRequestClient<ISimpleRequest, ISimpleResponse> client;
var bus = BusConfigurator.ConfigureBus((cfg, host) =>
{
cfg.ReceiveEndpoint(host, RabbitMqConstants.NotificationServiceQueue, e =>
{
e.Consumer(() => new OrderRegisteredConsumer(client));
});
});
client = new MessageRequestClient<ISimpleRequest, ISimpleResponse>(
bus, serviceAddress, requestTimeout);

RabbitMQ channel leak when bus was failed to start

I have a problem with MassTransit (Using MassTransit 3.5.7 (via Nuget), RabbitMQ 3.6.10, Erlang 19.0)
Looks like that MassTransit not clear up RabbitMQ channels, when bus is failed to start.
Here is my test program
using System;
using System.Threading;
using MassTransit;
namespace TestSubscriber
{
class Program
{
static void Main()
{
IBusControl busControl = null;
var failCount = 0;
var busNotInitialised = true;
//Keep RabbitMQ switched off for a few iterations of this loop, then switch it on.
while (busNotInitialised)
{
busControl = Bus.Factory.CreateUsingRabbitMq(sbc =>
{
var host = sbc.Host(new Uri("rabbitmq://localhost/"), h =>
{
h.Username("guest");
h.Password("guest");
});
sbc.ReceiveEndpoint(host, "some_queue", endpoint =>
{
endpoint.Handler<string>(async context =>
{
await Console.Out.WriteLineAsync($"Received: {context.Message}");
});
});
});
try
{
busControl.Start();
busNotInitialised = false;
}
catch (Exception)
{
Console.WriteLine($"Attempt:{++failCount} failed.");
//wait some time
Thread.Sleep(5000);
}
}
//At this point, using RabbitMq's management web page, you will see failCount + 1 channels.
busControl.Stop();
//At this point, using RabbitMq's management web page, you will see failCount channels.
Console.ReadLine();
}
}
}
It continuously tries to create a service bus that uses RabbitMQ.
The program breaks out of the loop as soon as the service bus is created successfully.
After running the program for a couple of minutes (with stopped RabbitMQ) and stopping on a breakpoint, I can see a lot of worker threads (one for each failed service bus creation attempt).
After startup RabbitMQ, all of those "dangling" connect threads will connect to RabbitMQ.
If I try to shut down the bus, the latest connection ( that belongs to the bus that was successfully created), is closed. All of the other dangling connections are still connected to RabbitMQ.
Problem here is that those dangling threads, when connected, reads data from the queue which leads to data loss.
Is there any way to get around this problem?

Microsoft Azure Service Bus Queue working as FIFO

I'm developing two WebJobs for azure: One which will be putting messages in the Service Bus Queue using a topic and another which is subscribed to the ServiceBusTrigger using the same topic.
The messages are sent to the service bus queue correctly but when run the WebJob subscribed to the ServiceBusTrigger those messages are not being processed in FIFO basis.
The code for the WebJob which puts messages in the service bus queue is the following:
NamespaceManager namespaceManager = NamespaceManager.Create();
// Delete if exists
if (namespaceManager.TopicExists("SampleTopic"))
{
namespaceManager.DeleteTopic("SampleTopic");
}
TopicDescription td = new TopicDescription("SampleTopic");
td.SupportOrdering = true;
TopicDescription myTopic = namespaceManager.CreateTopic(td);
SubscriptionDescription myAuditSubscription = namespaceManager.CreateSubscription(myTopic.Path, "ImporterSubscription");
TopicClient topicClient = TopicClient.Create("SampleTopic");
for(int i = 1; i <= 10; i++)
{
var message = new BrokeredMessage("message"+i);
topicClient.Send(message);
}
topicClient.Close();
The WebJob which is subscrited to the service bus trigger has the following code:
namespace HO.Importer.Azure.WebJob.TGZProcessor
{
public class Program
{
static void Main(string[] args)
{
JobHostConfiguration config = new JobHostConfiguration();
config.UseServiceBus();
JobHost host = new JobHost(config);
host.RunAndBlock();
}
public static void WriteLog([ServiceBusTrigger("SampleTopic", "ImporterSubscription")] string message,
TextWriter logger)
{
Console.WriteLine(message));
}
}
}
How can I achieve to process the messages fromo the queue as FIFO?
Thanks in advance!

Use SessionId or PartitionKey, that will ensure the message is handled by the same message broker.
See: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-partitioning
"SessionId: If a message has the BrokeredMessage.SessionId property set, then Service Bus uses this property as the partition key. This way, all messages that belong to the same session are handled by the same message broker. This enables Service Bus to guarantee message ordering as well as the consistency of session states."

While Azure Service Bus provides FIFO feature(Sessions), it is better not to assume this kind of behavior with a broker based queuing system. Ben Morris had a good post Don’t assume message ordering in Azure Service Bus on the fact that assuming ordering with asynchronous messaging is almost a fallacy and reasons for that.