Develop Reference

Azure Service Bus Receive Messages continuously when ever new message placed in web application [duplicate]

I am using Azure.Messaging.ServiceBus nuget package to work with Azure service bus. We have created a topic and a subscription. The subscription has 100+ messages. We want to read all the message and continue to read message as they arrive.
Microsoft.Azure.ServiceBus package (deprecated now) provided RegisterMessageHandler which use to process every incoming message. I am not able to find similar option under Azure.Messaging.ServiceBus nuget package.
I am able to read one message at a time but I have to call await receiver.ReceiveMessageAsync(); every time manually.

To receive multiple messages (a batch), you should use ServiceBusReceiver.ReceiveMessagesAsync() (not plural, not singular 'message'). This method will return whatever number of messages it can send back. To ensure you retrieve all 100+ messages, you'll need to loop until no messages are available.
If you'd like to use a processor, that's also available in the new SDK. See my answer to a similar question here.

As suggested by #gaurav Mantri, I used ServiceBusProcessor class to implement event based model for processing messages
public async Task ReceiveAll()
{
string connectionString = "Endpoint=sb://sb-test-today.servicebus.windows.net/;SharedAccessKeyName=manage;SharedAccessKey=8e+6SWp3skB3Aedsadsadasdwz5DU=;";
string topicName = "topicone";
string subscriptionName = "subone";
await using var client = new ServiceBusClient(connectionString, new ServiceBusClientOptions
{
TransportType = ServiceBusTransportType.AmqpWebSockets
});
var options = new ServiceBusProcessorOptions
{
// By default or when AutoCompleteMessages is set to true, the processor will complete the message after executing the message handler
// Set AutoCompleteMessages to false to [settle messages](https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-transfers-locks-settlement#peeklock) on your own.
// In both cases, if the message handler throws an exception without settling the message, the processor will abandon the message.
AutoCompleteMessages = false,
// I can also allow for multi-threading
MaxConcurrentCalls = 1
};
await using ServiceBusProcessor processor = client.CreateProcessor(topicName, subscriptionName, options);
processor.ProcessMessageAsync += MessageHandler;
processor.ProcessErrorAsync += ErrorHandler;
await processor.StartProcessingAsync();
Console.ReadKey();
}
public async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
Console.WriteLine(body);
// we can evaluate application logic and use that to determine how to settle the message.
await args.CompleteMessageAsync(args.Message);
}
public Task ErrorHandler(ProcessErrorEventArgs args)
{
// the error source tells me at what point in the processing an error occurred
Console.WriteLine(args.ErrorSource);
// the fully qualified namespace is available
Console.WriteLine(args.FullyQualifiedNamespace);
// as well as the entity path
Console.WriteLine(args.EntityPath);
Console.WriteLine(args.Exception.ToString());
return Task.CompletedTask;
}

MassTransit could not send message to queue who has x-message-ttl, received none but current is the value

I'm using MS newly. I'm thinking to build something like this. I have two queues, ConsumeQueue and DelayQueue.
DelayQueue has no consumer and defined dead letter exchange ConsumeQueue.
DelayQueue "x-message-ttl" queue argument setted for example 10 seconds.
Consumer consuming ConsumeQueue and process data. Handle unsuccessful datas and send message to DelayQueue.
After 10 seconds message moving to dead-letter ConsumeQueue and consuming less data and process.
note: I don't want to scheduled message plugin.
Startup configuration, please note if it can be better:
services.AddMassTransit(configure =>
{
configure.AddBus(factory =>
{
return Bus.Factory.CreateUsingRabbitMq(mqConfig =>
{
mqConfig.Host(eventBusOptions.Host, hostConfigurator =>
{
hostConfigurator.Username(eventBusOptions.Username);
hostConfigurator.Password(eventBusOptions.Password);
});
mqConfig.ReceiveEndpoint($"ConsumeQueue", endpointConfig =>
{
endpointConfig.Consumer<ConsumeMessage>(factory);
});
mqConfig.ReceiveEndpoint($"DelayedQueue", endpointConfig =>
{
endpointConfig.SetQueueArgument("x-message-ttl", TimeSpan.FromSeconds(10));
endpointConfig.BindDeadLetterQueue($"{nameof(FundValueConsumer)}", $"{nameof(FundValueConsumer)}", configure: a =>
{
a.Lazy = true;
a.Durable = true;
});
});
});
});
});
And inside ConsumeQueue consumer, the send message to DelayQueue part:
...
var endpoint = await _busControl.GetSendEndpoint(new Uri("queue:DelayedQueue"));
await endpoint.Send(retryCodeMessage);
//Also i tried this too
await endpoint.Send(retryCodeMessage, (context) =>
{
context.TimeToLive = TimeSpan.FromSeconds(15);
});
...
When send the message line throwing:
The AMQP operation was interrupted: AMQP close-reason,
initiated by Peer, code=406, text='PRECONDITION_FAILED
- inequivalent arg 'x-message-ttl' for queue 'DelayedQueue' in vhost 'dev':
received none but current is the value '10000' of type 'signedint'', classId=50, methodId=10
What should i do? What i did, it could be wrong. Please show the correct implementation of the plan I want

Configure Deduplication on Azure Service Bus Queue with MassTransit

I have found the following question (How to configure the RequiresDuplicateDetection for AzureServiceBus topics) about how to set the RequiresDuplicationDetection property when configuring a publish topic from a producer application in MassTransit. However, I have not been able to find out how to do it for commands that are transmitted to a queue with Send rather than Publish.
Additionally, I have found that when configuring a consumer of one the queues in question I can set the property easily, as shown below. This however is not ideal for my use case, if possible I would much rather the producer set this property when it starts and creates the queue.
cfg.ReceiveEndpoint(queue, e =>{
e.RequiresDuplicateDetection = true;
e.ConfigureConsumer<JobEventConsumer>(registrationContext, consumerConfig =>{
consumerConfig.UseMessageRetry(r =>{
r.Interval(10, TimeSpan.FromMilliseconds(200));
r.Ignore<ValidationException>();
});
});
});
Update: After a bit more investigation I have also found that setting the property to true at the global config level doesn't seem to work either. Code shown below
class Program {
static async Task Main(string[] args) {
EndpointConvention.Map<ExtractionRequest>(new Uri("queue:test-queue"));
var busControl = Bus.Factory.CreateUsingAzureServiceBus(cfg =>{
cfg.Host("My connection string");
cfg.RequiresDuplicateDetection = true;
cfg.EnablePartitioning = true;
});
await busControl.StartAsync();
try {
do {
string value = await Task.Run(() =>{
Console.WriteLine("Enter message (or quit to exit)");
Console.Write("> ");
return Console.ReadLine();
});
if ("quit".Equals(value, StringComparison.OrdinalIgnoreCase)) break;
await busControl.Send<ExtractionRequest>(new {});
}
while (true);
}
finally {
await busControl.StopAsync();
}
}
}
public interface ExtractionRequest {}
Any advice on how to turn RequiresDuplicationDetection on for a queue from the producer is welcomed.
Thanks in advance, James.

You can't set queue properties from a message sender, it's the responsibility of the receive endpoint.
The receive endpoint is the responsible component because it's declaring the queue and related attributes.
The reason publish is different is because topics can be configured by the producer, since there may be multiple consumer subscriptions on a single topic.

How to purge messages for Service Bus Topic Subscription

Just wondering the best way (even if via Portal, Powershell, or C#) to purge the messages off of a Service Bus Topic's Subscription.
Imagine we have a topic with 4 subscriptions, and we only want to purge the messages from one of the subscriptions.
I have a feeling the only way may be to read the messages in a while loop, but hoping for something better.
UPDATE:
Apart from using code, you can use the Server Explorer as suggested in the answer - right click subscription and purge messages:

You can most certainly do it via code. If you're using Service Bus SDK, you could do something like the following:
static void PurgeMessagesFromSubscription()
{
var connectionString = "Endpoint=sb://account-name.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=access key";
var topic = "topic-name";
var subscription = "subscription-name";
int batchSize = 100;
var subscriptionClient = SubscriptionClient.CreateFromConnectionString(connectionString, topic, subscription, ReceiveMode.ReceiveAndDelete);
do
{
var messages = subscriptionClient.ReceiveBatch(batchSize);
if (messages.Count() == 0)
{
break;
}
}
while (true);
}
What this code will do is fetch messages from the subscription (100 at a time) in Receive & Delete mode so that as soon as messages are fetched, they are deleted from the subscription automatically.
I believe Service Bus Explorer tool also has the capability to purge messages. You can use that as well instead of writing the code.

If you have a lot of messages and can tolerate a bit of downtime on subscriber side, it might be faster to just drop the subscription and create a new one with the same name.

Thank you #Gaurav Mantri, I used slightly changed code without the batch option with version 5.2.0 of Microsoft.Azure.ServiceBus Nuget Package:
var connectionString = "Endpoint=sb://";
var topic = "topic";
var subscription = "subscription";
var subscriptionClient = new SubscriptionClient(connectionString, topic, subscription, ReceiveMode.ReceiveAndDelete);
subscriptionClient.RegisterMessageHandler(
(message, token) =>
{
Console.WriteLine($"Received message: SequenceNumber:
{message.SystemProperties.SequenceNumber}");
return Task.CompletedTask;
},
(exceptionEvent) =>
{
Console.WriteLine("Exception = " + exceptionEvent.Exception);
return Task.CompletedTask;
});

Out-of-order delivery acknowledgements in RabbitMQ

In manual-acknowledgement mode (with noAck=false), messages may be acknowledged after being received and processed. Using IBasicConsumer, the .NET/C# API Guide suggests that this may be done, one-by-one, when a message is consumed:
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (_, e) =>
{
// process the message...
channel.BasicAck(e.DeliveryTag, false);
};
string consumerTag = channel.BasicConsume(queueName, false, consumer);
In my particular scenario, however, I need to consume several messages before I'm able to process them. And, in order to retain durability, I would have to acknowledge previous messages only later on (outside the scope of when they were received). These acknowledgements might also happen in a different order from the order in which messages are received.
var consumer = new EventingBasicConsumer(channel);
var buffer = new List<BasicDeliverEventArgs>();
consumer.Received += (_, e) =>
{
buffer.Add(e);
if (TryProcess(buffer, out IList<BasicDeliverEventArgs> subset))
{
foreach (var p in subset)
{
channel.BasicAck(p.DeliveryTag, false);
buffer.Remove(p);
}
}
};
string consumerTag = channel.BasicConsume(queueName, false, consumer);
When I try this I only receive a few messages and then something blocks, which leads me to thinking that RabbitMQ does not support this. I've also used the channel's BasicQos method to bump up the prefetchCount to hundreds, but the number of messages I receive is still only 4 or 5.
Does RabbitMQ support out-of-order delivery acknowledgements? And if not, is there a better technique I can apply that uses RabbitMQ to provide the same durability?