I have a application in which I want to have 1 durable and 1 non-durable queue in Active MQ Artemis.
For connecting to this message bus I use amqpnetlite.
var source = new Source()
{
};
if (durable)
{
source.Address = amqpAddressConverter.GetSubscriberAddress(address, useLoadBalancing);
source.Durable = 1;
source.ExpiryPolicy = new Symbol("never");
source.DistributionMode = new Symbol("copy");
}
else
{
source.Address = amqpAddressConverter.GetSubscriberAddress(address);
source.Durable = 0;
source.ExpiryPolicy = "never";
}
var receiverLink = new ReceiverLink(session, linkName, source, null);
So this is my receiver link. As shown I set the Durable uint of the Source which will given into the ReceiverLink.
Because as I saw in the Active MQ Artemis documentation, that the Durable is a boolean but within the amqpnetlite library it is an uint my understanding is that everything over 0 should be true and 0 should be false.
At first the behaviour was very strange: Even when the Aretemis Web interface was shown a queue as durable it would be deleted as soon as no consumer would be connected.
I found this:
ActiveMQ Artemis queue deleted after shutdown of consuming client
which describes that even durable queues get deleted because of the default behaviour.
So I manipulated the broker.xml and set AUTO-DELETE-QUEUE to false.
Since then the behaviour completly switched:
Both (durable = 1 and durable = 0) queues are being still there after the connection disconnected.
So how to create a durable and a non-durable connection correctly?
The Artemis source carries an example in .NET that creates a durable topic subscription and also shows how to later recover it using AmqpNetLite.
One key thing many folks miss is that your client needs to use a unique container ID analogous to the JMS Client ID concept.
For Queue specific subscriptions the client should indicate in the link capabilities that it wants a Queue based address created as the default is a multicast Queue which won't behave the same.
Source source = new Source() {
Address = address,
Capabilities = new Symbol[] {"queue"},
};
vs topic specific source configuration:
Source source = new Source() {
Address = address,
Capabilities = new Symbol[] {"topic"},
};
Related
Just wondering the best way (even if via Portal, Powershell, or C#) to purge the messages off of a Service Bus Topic's Subscription.
Imagine we have a topic with 4 subscriptions, and we only want to purge the messages from one of the subscriptions.
I have a feeling the only way may be to read the messages in a while loop, but hoping for something better.
UPDATE:
Apart from using code, you can use the Server Explorer as suggested in the answer - right click subscription and purge messages:
You can most certainly do it via code. If you're using Service Bus SDK, you could do something like the following:
static void PurgeMessagesFromSubscription()
{
var connectionString = "Endpoint=sb://account-name.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=access key";
var topic = "topic-name";
var subscription = "subscription-name";
int batchSize = 100;
var subscriptionClient = SubscriptionClient.CreateFromConnectionString(connectionString, topic, subscription, ReceiveMode.ReceiveAndDelete);
do
{
var messages = subscriptionClient.ReceiveBatch(batchSize);
if (messages.Count() == 0)
{
break;
}
}
while (true);
}
What this code will do is fetch messages from the subscription (100 at a time) in Receive & Delete mode so that as soon as messages are fetched, they are deleted from the subscription automatically.
I believe Service Bus Explorer tool also has the capability to purge messages. You can use that as well instead of writing the code.
If you have a lot of messages and can tolerate a bit of downtime on subscriber side, it might be faster to just drop the subscription and create a new one with the same name.
Thank you #Gaurav Mantri, I used slightly changed code without the batch option with version 5.2.0 of Microsoft.Azure.ServiceBus Nuget Package:
var connectionString = "Endpoint=sb://";
var topic = "topic";
var subscription = "subscription";
var subscriptionClient = new SubscriptionClient(connectionString, topic, subscription, ReceiveMode.ReceiveAndDelete);
subscriptionClient.RegisterMessageHandler(
(message, token) =>
{
Console.WriteLine($"Received message: SequenceNumber:
{message.SystemProperties.SequenceNumber}");
return Task.CompletedTask;
},
(exceptionEvent) =>
{
Console.WriteLine("Exception = " + exceptionEvent.Exception);
return Task.CompletedTask;
});
I'm developing two WebJobs for azure: One which will be putting messages in the Service Bus Queue using a topic and another which is subscribed to the ServiceBusTrigger using the same topic.
The messages are sent to the service bus queue correctly but when run the WebJob subscribed to the ServiceBusTrigger those messages are not being processed in FIFO basis.
The code for the WebJob which puts messages in the service bus queue is the following:
NamespaceManager namespaceManager = NamespaceManager.Create();
// Delete if exists
if (namespaceManager.TopicExists("SampleTopic"))
{
namespaceManager.DeleteTopic("SampleTopic");
}
TopicDescription td = new TopicDescription("SampleTopic");
td.SupportOrdering = true;
TopicDescription myTopic = namespaceManager.CreateTopic(td);
SubscriptionDescription myAuditSubscription = namespaceManager.CreateSubscription(myTopic.Path, "ImporterSubscription");
TopicClient topicClient = TopicClient.Create("SampleTopic");
for(int i = 1; i <= 10; i++)
{
var message = new BrokeredMessage("message"+i);
topicClient.Send(message);
}
topicClient.Close();
The WebJob which is subscrited to the service bus trigger has the following code:
namespace HO.Importer.Azure.WebJob.TGZProcessor
{
public class Program
{
static void Main(string[] args)
{
JobHostConfiguration config = new JobHostConfiguration();
config.UseServiceBus();
JobHost host = new JobHost(config);
host.RunAndBlock();
}
public static void WriteLog([ServiceBusTrigger("SampleTopic", "ImporterSubscription")] string message,
TextWriter logger)
{
Console.WriteLine(message));
}
}
}
How can I achieve to process the messages fromo the queue as FIFO?
Thanks in advance!
Use SessionId or PartitionKey, that will ensure the message is handled by the same message broker.
See: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-partitioning
"SessionId: If a message has the BrokeredMessage.SessionId property set, then Service Bus uses this property as the partition key. This way, all messages that belong to the same session are handled by the same message broker. This enables Service Bus to guarantee message ordering as well as the consistency of session states."
While Azure Service Bus provides FIFO feature(Sessions), it is better not to assume this kind of behavior with a broker based queuing system. Ben Morris had a good post Don’t assume message ordering in Azure Service Bus on the fact that assuming ordering with asynchronous messaging is almost a fallacy and reasons for that.
I am trying to receive messages in batch from the ServiceBus using the ReceiveBatch method in the MessageReceiver:
IEnumerable<BrokeredMessage> messages;
var messagingfactory = MessagingFactory.CreateFromConnectionString("ConnectionString");
var msgrcvr = messagingfactory.CreateMessageReceiver("queueName", ReceiveMode.ReceiveAndDelete);
messages = msgrcvr.ReceiveBatch(20, timeoutInSecs);
I have checked that my queue contains 20 messages using the Service Bus Explorer.
This code returns only one message in the messages structure. Is there some property I am missing?
This is only a partial-answer or work-around; the following code reliably gets all elements, but doesn't use the "ReceiveBatch"; note, as far as I can discern, Peek(i) operates on a one-based index. Also: depending on which server one is running on, if you are charged by the message pull, this may (or may not) be more expensive, so use at your own risk:
List<BrokeredMessage> dlIE = new List<BrokeredMessage>();
BrokeredMessage potentialMessage = null;
int loopCount = 1;
while ((potentialMessage = deadletterSubscriptionClient.Peek(loopCount)) != null)
{
dlIE.Add(potentialMessage); loopCount++;
}
Since we are experiencing some trouble with IBM's Websphere MQ using XMS.net (Windows service that sometimes seems to give up listening for messages on a queue) we would like to create a simple application to monitor the depths of some queues (or number of messages on the queue) to be able to alert someone when the queue depth exceeds a certain threshold. This application would be launched by the task scheduler on a specific interval and would "read out" for X queues their queue depth (and maybe some other statistics).
Our windows service is using the following code and I was hoping I could reuse that same "knowledge" for our "monitoring" application.
AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(CurrentDomain_UnhandledException);
//Read config values
string QueueManager = ConfigurationManager.AppSettings["queuemanager"];
string Channel = ConfigurationManager.AppSettings["channel"];
string Queue = ConfigurationManager.AppSettings["queue"];
string HostIP = ConfigurationManager.AppSettings["host"];
int Port = int.Parse(ConfigurationManager.AppSettings["port"]);
//Create connection
var factoryfactory = XMSFactoryFactory.GetInstance(XMSC.CT_WMQ);
var connectionfactory = factoryfactory.CreateConnectionFactory();
connectionfactory.SetStringProperty(XMSC.WMQ_QUEUE_MANAGER, QueueManager);
connectionfactory.SetStringProperty(XMSC.WMQ_HOST_NAME, HostIP);
connectionfactory.SetIntProperty(XMSC.WMQ_PORT, Port);
connectionfactory.SetStringProperty(XMSC.WMQ_CHANNEL, Channel);
connectionfactory.SetIntProperty(XMSC.WMQ_BROKER_VERSION, XMSC.WMQ_BROKER_V2);
connectionfactory.SetIntProperty(XMSC.WMQ_CONNECTION_MODE, XMSC.WMQ_CM_CLIENT_UNMANAGED);
Console.WriteLine("Creating connection");
var connection = connectionfactory.CreateConnection();
connection.ExceptionListener = new ExceptionListener(OnXMSExceptionReceived);
//Create a_session
Console.WriteLine("Creating sessions");
var session = connection.CreateSession(false, AcknowledgeMode.ClientAcknowledge);
//Create queue
Console.WriteLine("Creating queue");
var queue = session.CreateQueue(string.Format("queue://{0}/{1}", QueueManager, Queue));
I have browsed through the properties of session, queue etc. but, ofcourse, there are no "current queue depth" properties. I could use GetIntProperty() or GetLongProperty() on these objects but I don't know which constant to use for that (I have seen IBM.XMS.MQC.MQIA_CURRENT_Q_DEPTH but that contains an int and Get...Property() expects a string as parameter).
Long story short: how would I go about retrieving a queues depth with the above code as a starting-point? Is it at all possible using XMS.Net?
I was able to solve it using, as Shashi suggested, the MQ API. For this you need to reference amqmdnet.dll (C:\Program Files (x86)\IBM\WebSphere MQ\bin\amqmdnet.dll) and use the following (example) code. Please note that this is a simple example, no exception handling etc. is included.
using System;
using System.Collections;
using System.Configuration;
using IBM.WMQ;
namespace Test
{
class Program
{
static void Main(string[] args)
{
//Connection properties
var properties = new Hashtable();
properties.Add(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES_CLIENT);
properties.Add(MQC.CHANNEL_PROPERTY, "SOME.CHANNEL.TCP");
properties.Add(MQC.HOST_NAME_PROPERTY, "12.34.56.78");
properties.Add(MQC.PORT_PROPERTY, 1416);
var qmgr = new MQQueueManager("MYQMGR", properties);
Console.WriteLine("Local : {0}", GetQueueDepth(qmgr, "FOO.LOCALQ"));
Console.WriteLine("Report : {0}", GetQueueDepth(qmgr, "FOO.REPORTQ"));
}
public static int GetQueueDepth(MQQueueManager queuemgr, string queue)
{
return queuemgr.AccessQueue(queue,
MQC.MQOO_INPUT_AS_Q_DEF +
MQC.MQOO_FAIL_IF_QUIESCING +
MQC.MQOO_INQUIRE).CurrentDepth;
}
}
}
This performs way better than my initial "workaround".
Using XMS .NET queue depth can't be determined. Queue depth is specific to messaging providers and not JMS/XMS, so you will need to use MQ APIs to get the queue depth. You could use MQ .NET API to find the queue depth. MQQueue.CurrentDepth will give the number of message in the queue.
IMO it would be good to investigate why XMS .NET service stopped listening for messages than write another program to monitor queue depth.
I have a Use Case where I need to queue a select number of messages when the current queue length drops below a specified value. Since I'm running in Azure, I'm trying to use the RetrieveApproximateMessageCount() method to get the current message count. Everytime I call this I get an exception stating StorageClientException: The specified queue does not exist.. Here is a review of what I've done:
Created the queue in the portal and have successfully queued messages to it.
Created the storage account in the portal and it is in the Created/Online state
Coded the query as follows (using http and https options):
var storageAccount = new CloudStorageAccount(
new StorageCredentialsAccountAndKey(_messagingConfiguration.StorageName.ToLower(),
_messagingConfiguration.StorageKey), false);
var queueClient = storageAccount.CreateCloudQueueClient();
var queue = queueClient.GetQueueReference(queueName.ToLower());
int messageCount;
try
{
messageCount = queue.RetrieveApproximateMessageCount();
}
catch (Exception)
{
//Booom!!!!! in every case
}
// ApproximateMessageCount is always null
messageCount = queue.ApproximateMessageCount == null ? 0 : queue.ApproximateMessageCount.Value;
I've confirmed the name is cased correctly with not special characters, numbers, or spaces and the resulting queue Url appears as though its correct formed based on the API documentations (e.g. http://myaccount.queue.core.windows.net/myqueue)
Can anyone help shed some light on what I'm doing wrong.
EDIT
I've confirmed that using the MessageFactory I can create a QueueClient and then enqueue/dequeue messages successfully. When I use the CloudStorageAccount the queue is never present so the counts and GetMessage routines never work. I am guessing these are not the same thing??? Assuming, I'm correct, what I need is to measure the length of the Service Bus Queue. Is that possible?
RetrieveApproximateMessageCount() has been deprecated
if you want to use ApproximateMessageCount to get result try this
CloudQueue q = queueClient.GetQueueReference(QUEUE_NAME);
q.FetchAttributes();
qCnt = q.ApproximateMessageCount;
The CloudQueue method has been deprecated (along with the v11 SDK).
The following snippet is the current replacement (from the Azure Docs)
//-----------------------------------------------------
// Get the approximate number of messages in the queue
//-----------------------------------------------------
public void GetQueueLength(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];
// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);
if (queueClient.Exists())
{
QueueProperties properties = queueClient.GetProperties();
// Retrieve the cached approximate message count.
int cachedMessagesCount = properties.ApproximateMessagesCount;
// Display number of messages.
Console.WriteLine($"Number of messages in queue: {cachedMessagesCount}");
}
}
https://learn.microsoft.com/en-us/azure/storage/queues/storage-dotnet-how-to-use-queues?tabs=dotnet#get-the-queue-length