Question
How do i get the dead letter queue length without receiving each message and counting how many message I received?
My Current Implementation
public int GetDeadLetterQueueCount()
{
//Ref:http://stackoverflow.com/questions/22681954/how-do-you-access-the-dead-letter-sub-queue-on-an-azure-subscription
MessagingFactory factory = MessagingFactory.CreateFromConnectionString(CloudConnectionString);
QueueClient deadLetterClient = factory.CreateQueueClient(QueueClient.FormatDeadLetterPath(_QueueClient.Path), ReceiveMode.PeekLock);
BrokeredMessage receivedDeadLetterMessage;
List<string> lstDeadLetterQueue = new List<string>();
// Ref: https://code.msdn.microsoft.com/Brokered-Messaging-Dead-22536dd8/sourcecode?fileId=123792&pathId=497121593
// Log the dead-lettered messages that could not be processed:
while ((receivedDeadLetterMessage = deadLetterClient.Receive(TimeSpan.FromSeconds(10))) != null)
{
lstDeadLetterQueue.Add(String.Format("DeadLettering Reason is \"{0}\" and Deadlettering error description is \"{1}\"",
receivedDeadLetterMessage.Properties["DeadLetterReason"],
receivedDeadLetterMessage.Properties["DeadLetterErrorDescription"]));
var locktime = receivedDeadLetterMessage.LockedUntilUtc;
}
return lstDeadLetterQueue.Count;
}
Problem with implementation
Because I am receiving each message in peek and block mode, the messages have a lock duration set. During this time i cannot receive or even see the messages again until this time period has timed out.
There must be an easier way of just getting the count without having to poll the queue?
I do not want to consume the messages either, i would just like the count of the total amount.
You can use the NamespaceManager's GetQueue() method which has a MessageCountDetails property, which in turn has a DeadLetterMessageCount property. Something like:
var namespaceManager = Microsoft.ServiceBus.NamespaceManager.CreateFromConnectionString("<CONN_STRING>");
var messageDetails = namespaceManager.GetQueue("<QUEUE_NAME>").MessageCountDetails;
var deadLetterCount = messageDetails.DeadLetterMessageCount;
Related
So I am busy writing an Watchdog for message queue installed on one of my servers. I have a application ( 3rd party ) that is listening on the queue and processing the messages. I now want to do a count on it and if message reach example 1500 I send a email. So all my code works except that I need to close the 3rd party app to use the message queue. What I do is I get all the queue names that exist. Work fine.
public void GetPrivateQueues()
{
MessageQueue[] QueueList =
MessageQueue.GetPrivateQueuesByMachine(".");
foreach (MessageQueue queueItem in QueueList)
{
i++;
myPrivateQueues.Add(queueItem.Path);
Count(queueItem.Path);
}
return;
}
So when I do the count of the queue like this
public void Count(String path)
{
MessageQueue queue = new MessageQueue(path);
MessageEnumerator messageEnumerator = queue.GetMessageEnumerator2();
int iii = 0;
while (messageEnumerator.MoveNext())
{
iii++;
}
myPrivateQueuesCount.Add(iii);
return;//i;
}
I get the error.
System.Messaging.MessageQueueException (0x80004005): Sharing violation resulted from queue being open already for exclusive receive.
How can I go about reading the queue to do a count without trying to get exclusive access on it. I just need to count it.
Thank you
Regards
I used performance Counter to read the queue.
Working like a dream now!
Added the catch exception. This is for when the queue is blank. I write a 0. Performance counter gives error on blank queue.
public void Count(String path)
{
path = path.Remove(0, 21);
try
{
PerformanceCounter queueCounter = new PerformanceCounter(
"MSMQ Queue",
"Messages in Queue",
#path);
Console.WriteLine("Queue contains {0} messages",
queueCounter.NextValue().ToString());
myPrivateQueuesCount.Add((int)queueCounter.NextValue());
}
catch (Exception exc)
{
myPrivateQueuesCount.Add(0);
}
return;
}
I am trying out Azure Service Bus queue. I have the below code:
Queue send:
string strConnectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"];
var namespaceManager = NamespaceManager.CreateFromConnectionString(strConnectionString);
if (!namespaceManager.QueueExists("Test"))
{
QueueDescription qD = new QueueDescription("Test");
qD.DefaultMessageTimeToLive = new TimeSpan(05, 00, 00);
qD.LockDuration = new TimeSpan(00, 02, 30);
qD.MaxSizeInMegabytes = 5120;
namespaceManager.CreateQueue(qD);
}
if (namespaceManager.QueueExists("Test"))
{
QueueClient client = QueueClient.CreateFromConnectionString(strConnectionString, "Test", ReceiveMode.PeekLock);
var qMessage = Console.ReadLine();
using (MemoryStream strm = new MemoryStream(Encoding.UTF8.GetBytes(qMessage)))
{
BrokeredMessage bMsg = new BrokeredMessage(strm);
bMsg.MessageId = Guid.NewGuid().ToString();
bMsg.TimeToLive = new TimeSpan(05, 00, 00);
client.Send(bMsg);
Console.WriteLine("Message sent");
}
}
Console.ReadLine();
The receive code:
string strConnectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"];
var namespaceManager = NamespaceManager.CreateFromConnectionString(strConnectionString);
if (namespaceManager.QueueExists("Test"))
{
QueueClient client = QueueClient.CreateFromConnectionString(strConnectionString, "Test",ReceiveMode.PeekLock);
if (client != null)
{
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
options.AutoRenewTimeout = TimeSpan.FromSeconds(31);
client.OnMessage((message) =>
{
Console.WriteLine(message.State.ToString());
Console.WriteLine("Message Id: " + message.MessageId);
Stream stream = message.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
Console.WriteLine("Message: " + reader.ReadToEnd());
Console.WriteLine("***************");
message.Abandon();
});
Console.ReadLine();
}
}
I see that whenever I call Abandon, the message is getting DeadLettered. My assumption was that it should get Active and can be picked up by another client.
Your understanding of BrokeredMessage.Abandon Api is correct. It is intended to abandon the peek-lock acquired on the message (but NOT abandon the message itself) and hence, makes it available for other receivers to pick the Message up.
Here's how we envisioned different states of a peek-lock'ed message:
Basics first
The 'Why': If Customers need Competing-Consumer (Job-Queue) semantics - where they need multiple workers to simultaneously process different messages from a Queue with Exactly-Once guarantee - then they use the ReceiveMode.PeekLock. In this model, each worker (the queue receiver) needs a way to communicate the Progress of its Current message (Job) to other workers. Hence, brokeredMessage provides 4 functions to express the states.
The 'What':
if a message is successfully processed by the current Worker - call BrokeredMessage.Complete()
if the BrokeredMessage cannot be processed by the current worker, and want the processing to be retried on another Worker - then, Abandon the message. But, the catch here is: lets say, there are 2 workers and each of them thinks that the other one can process this message and calls Abandon - soon they will end up in an Infinite loop of retry'ing to process just that message! So, to avoid this situation, we provided a Configuration called MaxDeliveryCount on QueueDescription. This setting guards the limit on the number of times the message is delivered from the Queue to receiver. In the above example, Each time you received (and abandoned) the message, the 'deliveryCount' on the ServiceBus service is incremented. When it reaches 10 - the message has hit max no. of deliveries and hence, will be deadlettered.
if the current receiver (worker) knows for sure, that, this message cannot be processed, BrokeredMessage.DeadLetter(). The goal here is to let the consuming application Audit the dead-lettered messages regularly.
if the current receiver (worker) cannot process this message, but, knows that this message can be processed at a later point of time BrokeredMessage.Defer().
HTH!
Sree
I am trying to receive messages in batch from the ServiceBus using the ReceiveBatch method in the MessageReceiver:
IEnumerable<BrokeredMessage> messages;
var messagingfactory = MessagingFactory.CreateFromConnectionString("ConnectionString");
var msgrcvr = messagingfactory.CreateMessageReceiver("queueName", ReceiveMode.ReceiveAndDelete);
messages = msgrcvr.ReceiveBatch(20, timeoutInSecs);
I have checked that my queue contains 20 messages using the Service Bus Explorer.
This code returns only one message in the messages structure. Is there some property I am missing?
This is only a partial-answer or work-around; the following code reliably gets all elements, but doesn't use the "ReceiveBatch"; note, as far as I can discern, Peek(i) operates on a one-based index. Also: depending on which server one is running on, if you are charged by the message pull, this may (or may not) be more expensive, so use at your own risk:
List<BrokeredMessage> dlIE = new List<BrokeredMessage>();
BrokeredMessage potentialMessage = null;
int loopCount = 1;
while ((potentialMessage = deadletterSubscriptionClient.Peek(loopCount)) != null)
{
dlIE.Add(potentialMessage); loopCount++;
}
I have a Use Case where I need to queue a select number of messages when the current queue length drops below a specified value. Since I'm running in Azure, I'm trying to use the RetrieveApproximateMessageCount() method to get the current message count. Everytime I call this I get an exception stating StorageClientException: The specified queue does not exist.. Here is a review of what I've done:
Created the queue in the portal and have successfully queued messages to it.
Created the storage account in the portal and it is in the Created/Online state
Coded the query as follows (using http and https options):
var storageAccount = new CloudStorageAccount(
new StorageCredentialsAccountAndKey(_messagingConfiguration.StorageName.ToLower(),
_messagingConfiguration.StorageKey), false);
var queueClient = storageAccount.CreateCloudQueueClient();
var queue = queueClient.GetQueueReference(queueName.ToLower());
int messageCount;
try
{
messageCount = queue.RetrieveApproximateMessageCount();
}
catch (Exception)
{
//Booom!!!!! in every case
}
// ApproximateMessageCount is always null
messageCount = queue.ApproximateMessageCount == null ? 0 : queue.ApproximateMessageCount.Value;
I've confirmed the name is cased correctly with not special characters, numbers, or spaces and the resulting queue Url appears as though its correct formed based on the API documentations (e.g. http://myaccount.queue.core.windows.net/myqueue)
Can anyone help shed some light on what I'm doing wrong.
EDIT
I've confirmed that using the MessageFactory I can create a QueueClient and then enqueue/dequeue messages successfully. When I use the CloudStorageAccount the queue is never present so the counts and GetMessage routines never work. I am guessing these are not the same thing??? Assuming, I'm correct, what I need is to measure the length of the Service Bus Queue. Is that possible?
RetrieveApproximateMessageCount() has been deprecated
if you want to use ApproximateMessageCount to get result try this
CloudQueue q = queueClient.GetQueueReference(QUEUE_NAME);
q.FetchAttributes();
qCnt = q.ApproximateMessageCount;
The CloudQueue method has been deprecated (along with the v11 SDK).
The following snippet is the current replacement (from the Azure Docs)
//-----------------------------------------------------
// Get the approximate number of messages in the queue
//-----------------------------------------------------
public void GetQueueLength(string queueName)
{
// Get the connection string from app settings
string connectionString = ConfigurationManager.AppSettings["StorageConnectionString"];
// Instantiate a QueueClient which will be used to manipulate the queue
QueueClient queueClient = new QueueClient(connectionString, queueName);
if (queueClient.Exists())
{
QueueProperties properties = queueClient.GetProperties();
// Retrieve the cached approximate message count.
int cachedMessagesCount = properties.ApproximateMessagesCount;
// Display number of messages.
Console.WriteLine($"Number of messages in queue: {cachedMessagesCount}");
}
}
https://learn.microsoft.com/en-us/azure/storage/queues/storage-dotnet-how-to-use-queues?tabs=dotnet#get-the-queue-length
I am using Rabbit MQ in C#. This is my scenario
A separate process publishes messages to the queue
Client has to read set of N messages from queue
Process the N messages
Acknowledge the N messages
Under the same channel, I receive the messages and then process them and then acknowledge them. The server process keeps publishing messages. The problem I am facing is, when I try to get next set of messages, they do not come in the same order as it was published by the publishing process. The messages come in a random order. Only the first set of messages come in the correct order.
Does any one what is going wrong here? Is creating a new channel to access the next set of messages not right? Below is the sample code:
while (true)
{
using (IModel getChannel = MQConnection.CreateModel())
{
// Create a consumer
QueueingBasicConsumer consumer = CreateQueueConsumer(getChannel, exchangeName, queueName);
int numberOfMessages = 100;
// Next Recieve
List<object> msgSet = GetNextSetOfMessages(consumer, getChannel, exchangeName, queueName, numberOfMessages, out finalDeliverytag);
// Do some processing
if (finalDeliverytag > 0)
AckFinishedMessages(exchangeName, queueName, finalDeliverytag, getChannel);
if (finalDeliverytag == 0)
break;
}
}
Kindly help. Thanks!