Just wondering the best way (even if via Portal, Powershell, or C#) to purge the messages off of a Service Bus Topic's Subscription.
Imagine we have a topic with 4 subscriptions, and we only want to purge the messages from one of the subscriptions.
I have a feeling the only way may be to read the messages in a while loop, but hoping for something better.
UPDATE:
Apart from using code, you can use the Server Explorer as suggested in the answer - right click subscription and purge messages:
You can most certainly do it via code. If you're using Service Bus SDK, you could do something like the following:
static void PurgeMessagesFromSubscription()
{
var connectionString = "Endpoint=sb://account-name.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=access key";
var topic = "topic-name";
var subscription = "subscription-name";
int batchSize = 100;
var subscriptionClient = SubscriptionClient.CreateFromConnectionString(connectionString, topic, subscription, ReceiveMode.ReceiveAndDelete);
do
{
var messages = subscriptionClient.ReceiveBatch(batchSize);
if (messages.Count() == 0)
{
break;
}
}
while (true);
}
What this code will do is fetch messages from the subscription (100 at a time) in Receive & Delete mode so that as soon as messages are fetched, they are deleted from the subscription automatically.
I believe Service Bus Explorer tool also has the capability to purge messages. You can use that as well instead of writing the code.
If you have a lot of messages and can tolerate a bit of downtime on subscriber side, it might be faster to just drop the subscription and create a new one with the same name.
Thank you #Gaurav Mantri, I used slightly changed code without the batch option with version 5.2.0 of Microsoft.Azure.ServiceBus Nuget Package:
var connectionString = "Endpoint=sb://";
var topic = "topic";
var subscription = "subscription";
var subscriptionClient = new SubscriptionClient(connectionString, topic, subscription, ReceiveMode.ReceiveAndDelete);
subscriptionClient.RegisterMessageHandler(
(message, token) =>
{
Console.WriteLine($"Received message: SequenceNumber:
{message.SystemProperties.SequenceNumber}");
return Task.CompletedTask;
},
(exceptionEvent) =>
{
Console.WriteLine("Exception = " + exceptionEvent.Exception);
return Task.CompletedTask;
});
Related
I am using Azure.Messaging.ServiceBus nuget package to work with Azure service bus. We have created a topic and a subscription. The subscription has 100+ messages. We want to read all the message and continue to read message as they arrive.
Microsoft.Azure.ServiceBus package (deprecated now) provided RegisterMessageHandler which use to process every incoming message. I am not able to find similar option under Azure.Messaging.ServiceBus nuget package.
I am able to read one message at a time but I have to call await receiver.ReceiveMessageAsync(); every time manually.
To receive multiple messages (a batch), you should use ServiceBusReceiver.ReceiveMessagesAsync() (not plural, not singular 'message'). This method will return whatever number of messages it can send back. To ensure you retrieve all 100+ messages, you'll need to loop until no messages are available.
If you'd like to use a processor, that's also available in the new SDK. See my answer to a similar question here.
As suggested by #gaurav Mantri, I used ServiceBusProcessor class to implement event based model for processing messages
public async Task ReceiveAll()
{
string connectionString = "Endpoint=sb://sb-test-today.servicebus.windows.net/;SharedAccessKeyName=manage;SharedAccessKey=8e+6SWp3skB3Aedsadsadasdwz5DU=;";
string topicName = "topicone";
string subscriptionName = "subone";
await using var client = new ServiceBusClient(connectionString, new ServiceBusClientOptions
{
TransportType = ServiceBusTransportType.AmqpWebSockets
});
var options = new ServiceBusProcessorOptions
{
// By default or when AutoCompleteMessages is set to true, the processor will complete the message after executing the message handler
// Set AutoCompleteMessages to false to [settle messages](https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-transfers-locks-settlement#peeklock) on your own.
// In both cases, if the message handler throws an exception without settling the message, the processor will abandon the message.
AutoCompleteMessages = false,
// I can also allow for multi-threading
MaxConcurrentCalls = 1
};
await using ServiceBusProcessor processor = client.CreateProcessor(topicName, subscriptionName, options);
processor.ProcessMessageAsync += MessageHandler;
processor.ProcessErrorAsync += ErrorHandler;
await processor.StartProcessingAsync();
Console.ReadKey();
}
public async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
Console.WriteLine(body);
// we can evaluate application logic and use that to determine how to settle the message.
await args.CompleteMessageAsync(args.Message);
}
public Task ErrorHandler(ProcessErrorEventArgs args)
{
// the error source tells me at what point in the processing an error occurred
Console.WriteLine(args.ErrorSource);
// the fully qualified namespace is available
Console.WriteLine(args.FullyQualifiedNamespace);
// as well as the entity path
Console.WriteLine(args.EntityPath);
Console.WriteLine(args.Exception.ToString());
return Task.CompletedTask;
}
I have a application in which I want to have 1 durable and 1 non-durable queue in Active MQ Artemis.
For connecting to this message bus I use amqpnetlite.
var source = new Source()
{
};
if (durable)
{
source.Address = amqpAddressConverter.GetSubscriberAddress(address, useLoadBalancing);
source.Durable = 1;
source.ExpiryPolicy = new Symbol("never");
source.DistributionMode = new Symbol("copy");
}
else
{
source.Address = amqpAddressConverter.GetSubscriberAddress(address);
source.Durable = 0;
source.ExpiryPolicy = "never";
}
var receiverLink = new ReceiverLink(session, linkName, source, null);
So this is my receiver link. As shown I set the Durable uint of the Source which will given into the ReceiverLink.
Because as I saw in the Active MQ Artemis documentation, that the Durable is a boolean but within the amqpnetlite library it is an uint my understanding is that everything over 0 should be true and 0 should be false.
At first the behaviour was very strange: Even when the Aretemis Web interface was shown a queue as durable it would be deleted as soon as no consumer would be connected.
I found this:
ActiveMQ Artemis queue deleted after shutdown of consuming client
which describes that even durable queues get deleted because of the default behaviour.
So I manipulated the broker.xml and set AUTO-DELETE-QUEUE to false.
Since then the behaviour completly switched:
Both (durable = 1 and durable = 0) queues are being still there after the connection disconnected.
So how to create a durable and a non-durable connection correctly?
The Artemis source carries an example in .NET that creates a durable topic subscription and also shows how to later recover it using AmqpNetLite.
One key thing many folks miss is that your client needs to use a unique container ID analogous to the JMS Client ID concept.
For Queue specific subscriptions the client should indicate in the link capabilities that it wants a Queue based address created as the default is a multicast Queue which won't behave the same.
Source source = new Source() {
Address = address,
Capabilities = new Symbol[] {"queue"},
};
vs topic specific source configuration:
Source source = new Source() {
Address = address,
Capabilities = new Symbol[] {"topic"},
};
Using MessageReceiver only first time getting messages from topic using Azure Function after that next time when execute Function though I have messages into service bus topic still getting null into variable messages.
Using below code to get messages from Service Bus Topic , whats the cause?
string serviceBusConnectionString = Environment.GetEnvironmentVariable("ConnectionString");
var messageReceiver = new MessageReceiver(serviceBusConnectionString, Environment.GetEnvironmentVariable("topicName/subscriptions/subscriptionName"), ReceiveMode.PeekLock, null, 500);
var messages = await messageReceiver.ReceiveAsync(500, TimeSpan.FromSeconds(1));
i can't understand this - after every 15 minutes when LA call FA then
FA should able to read new available messages but it is coming null
is there anything related with lock duration ?
var messages = await messageReceiver.ReceiveAsync(500, TimeSpan.FromSeconds(1));
will wait until it receives one or more messages and will then return. Any messages that come in later will not be read. If you want to do that (and I want to doubt that this is a good idea inside an Azure Function), put this in a loop:
while(/* some condition */)
{
messages = await messageReceiver.ReceiveAsync(500, TimeSpan.FromSeconds(1));
// do something with the received messages
}
If i didn't understand it wrong, you use a timer trigger logic app to call the function to get the service bus message and you always get the old message and you don't want it.
You always get the old message because your ReceiveMode is PeekLock, about this ReceiveMode you could check this description:Receive mode. After complete the message, the message wouldn't be deleted, that's why you always get the old message.
First way is you could change it to ReceiveAndDelete when you create MessageReceiver like the below code.
string serviceBusConnectionString = Environment.GetEnvironmentVariable("servicebuscon");
var messageReceiver = new MessageReceiver(serviceBusConnectionString, "myqueue", ReceiveMode.ReceiveAndDelete, null, 500);
var tempMessages = await messageReceiver.ReceiveAsync(500, TimeSpan.FromSeconds(1));
Note: if your function go exception or other errors and the message doesn't get processed completely, this message will be deleted with this mode.
The other way is after process the message call the CompleteAsync method, after process the message will be deleted.
You could refer to the below code.
var messageReceiver = new MessageReceiver(serviceBusConnectionString, "myqueue", ReceiveMode.PeekLock, null, 500);
var tempMessages = await messageReceiver.ReceiveAsync(500, TimeSpan.FromSeconds(1));
foreach (Message m1 in tempMessages)
{
log.LogInformation($"C# HTTP trigger function processed message: {Encoding.UTF8.GetString(m1.Body)}");
await messageReceiver.CompleteAsync(m1.SystemProperties.LockToken);
}
Enable Batch Operation should be false.
But this is also working for random times - not a reliable solution for the problem.
Check the below screenshot:
I'm developing two WebJobs for azure: One which will be putting messages in the Service Bus Queue using a topic and another which is subscribed to the ServiceBusTrigger using the same topic.
The messages are sent to the service bus queue correctly but when run the WebJob subscribed to the ServiceBusTrigger those messages are not being processed in FIFO basis.
The code for the WebJob which puts messages in the service bus queue is the following:
NamespaceManager namespaceManager = NamespaceManager.Create();
// Delete if exists
if (namespaceManager.TopicExists("SampleTopic"))
{
namespaceManager.DeleteTopic("SampleTopic");
}
TopicDescription td = new TopicDescription("SampleTopic");
td.SupportOrdering = true;
TopicDescription myTopic = namespaceManager.CreateTopic(td);
SubscriptionDescription myAuditSubscription = namespaceManager.CreateSubscription(myTopic.Path, "ImporterSubscription");
TopicClient topicClient = TopicClient.Create("SampleTopic");
for(int i = 1; i <= 10; i++)
{
var message = new BrokeredMessage("message"+i);
topicClient.Send(message);
}
topicClient.Close();
The WebJob which is subscrited to the service bus trigger has the following code:
namespace HO.Importer.Azure.WebJob.TGZProcessor
{
public class Program
{
static void Main(string[] args)
{
JobHostConfiguration config = new JobHostConfiguration();
config.UseServiceBus();
JobHost host = new JobHost(config);
host.RunAndBlock();
}
public static void WriteLog([ServiceBusTrigger("SampleTopic", "ImporterSubscription")] string message,
TextWriter logger)
{
Console.WriteLine(message));
}
}
}
How can I achieve to process the messages fromo the queue as FIFO?
Thanks in advance!
Use SessionId or PartitionKey, that will ensure the message is handled by the same message broker.
See: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-partitioning
"SessionId: If a message has the BrokeredMessage.SessionId property set, then Service Bus uses this property as the partition key. This way, all messages that belong to the same session are handled by the same message broker. This enables Service Bus to guarantee message ordering as well as the consistency of session states."
While Azure Service Bus provides FIFO feature(Sessions), it is better not to assume this kind of behavior with a broker based queuing system. Ben Morris had a good post Don’t assume message ordering in Azure Service Bus on the fact that assuming ordering with asynchronous messaging is almost a fallacy and reasons for that.
I am trying to receive messages in batch from the ServiceBus using the ReceiveBatch method in the MessageReceiver:
IEnumerable<BrokeredMessage> messages;
var messagingfactory = MessagingFactory.CreateFromConnectionString("ConnectionString");
var msgrcvr = messagingfactory.CreateMessageReceiver("queueName", ReceiveMode.ReceiveAndDelete);
messages = msgrcvr.ReceiveBatch(20, timeoutInSecs);
I have checked that my queue contains 20 messages using the Service Bus Explorer.
This code returns only one message in the messages structure. Is there some property I am missing?
This is only a partial-answer or work-around; the following code reliably gets all elements, but doesn't use the "ReceiveBatch"; note, as far as I can discern, Peek(i) operates on a one-based index. Also: depending on which server one is running on, if you are charged by the message pull, this may (or may not) be more expensive, so use at your own risk:
List<BrokeredMessage> dlIE = new List<BrokeredMessage>();
BrokeredMessage potentialMessage = null;
int loopCount = 1;
while ((potentialMessage = deadletterSubscriptionClient.Peek(loopCount)) != null)
{
dlIE.Add(potentialMessage); loopCount++;
}