Develop Reference

Cannot construct MessageQueue object for journal queue

I am unable to construct a MessageQueue object to access a target journal queue used by one of my applications.
When I execute
MessageQueue journalQ = new MessageQueue("my_computer\\private$\\test queue")
I can construct a messagequeue.
I have verified that test queue exists and it has a journal queue that is enabled and has a message.
When I execute
MessageQueue journalQ = new MessageQueue("my_computer\\private$\\test queue\\Journal$")
I cannot construct the queue, when looking at my locals I get MessageQueueException exceptions thrown saying The specified format name does not support the requested operation. For example, a direct queue format name cannot be deleted.
I have tried multiple different path fomrat combinations, both the full "Format=OS" and more dissected strings, and ";journal" rather than "\journal"
string journalPath = (messageQueueList[choiceInt].Path.Split(':')[2].Split('$')[0]
+ "$"
+ messageQueueList[choiceInt].Path.Split('$')[1]
+ "\\Journal$"
);
journalQ = new MessageQueue(journalPath);

You cannot directly send messages to a journal queue, you can only receive from those. Accessing sending-related members of a journal's MessageQueue object will throw the exception you cited in your question. Take the following example code:
static void Main(string[] args)
{
// Create queue
string queueName = "computer_name\\private$\\test queue";
MessageQueue.Create(queueName);
// Create MessageQueue object and enable journal
MessageQueue queue = new MessageQueue(queueName);
queue.Formatter = new XmlMessageFormatter(new Type[] { typeof(string) });
queue.UseJournalQueue = true;
Console.WriteLine($"queue.CanRead = {queue.CanRead}");
Console.WriteLine($"queue.CanWrite = {queue.CanWrite}");
// Send message
queue.Send("test message");
// Receive message from queue
// The message is copied to the journal now
Console.WriteLine($"queue.Receive(): {(string)queue.Receive().Body}");
// Receive message again from journal
MessageQueue journal = new MessageQueue(queueName + "\\Journal$");
journal.Formatter = new XmlMessageFormatter(new Type[] { typeof(string) });
Console.WriteLine($"journal.CanRead = {journal.CanRead}");
Console.WriteLine($"journal.CanWrite = {journal.CanWrite}");
Console.WriteLine($"journal.Receive(): {(string)journal.Receive().Body}");
}
This will create a new message queue and enable the matching journal queue. A sent message is stored in the message queue; after receiving it, it is moved to the journal queue. You can then receive this message a second time from the journal queue.
The program's output is:
queue.CanRead = True
queue.CanWrite = True
queue.Receive(): test message
journal.CanRead = True
journal.CanWrite = False
journal.Receive(): test message

How to retrieve Dead Letter Queue count?

Question
How do i get the dead letter queue length without receiving each message and counting how many message I received?
My Current Implementation
public int GetDeadLetterQueueCount()
{
//Ref:http://stackoverflow.com/questions/22681954/how-do-you-access-the-dead-letter-sub-queue-on-an-azure-subscription
MessagingFactory factory = MessagingFactory.CreateFromConnectionString(CloudConnectionString);
QueueClient deadLetterClient = factory.CreateQueueClient(QueueClient.FormatDeadLetterPath(_QueueClient.Path), ReceiveMode.PeekLock);
BrokeredMessage receivedDeadLetterMessage;
List<string> lstDeadLetterQueue = new List<string>();
// Ref: https://code.msdn.microsoft.com/Brokered-Messaging-Dead-22536dd8/sourcecode?fileId=123792&pathId=497121593
// Log the dead-lettered messages that could not be processed:
while ((receivedDeadLetterMessage = deadLetterClient.Receive(TimeSpan.FromSeconds(10))) != null)
{
lstDeadLetterQueue.Add(String.Format("DeadLettering Reason is \"{0}\" and Deadlettering error description is \"{1}\"",
receivedDeadLetterMessage.Properties["DeadLetterReason"],
receivedDeadLetterMessage.Properties["DeadLetterErrorDescription"]));
var locktime = receivedDeadLetterMessage.LockedUntilUtc;
}
return lstDeadLetterQueue.Count;
}
Problem with implementation
Because I am receiving each message in peek and block mode, the messages have a lock duration set. During this time i cannot receive or even see the messages again until this time period has timed out.
There must be an easier way of just getting the count without having to poll the queue?
I do not want to consume the messages either, i would just like the count of the total amount.

You can use the NamespaceManager's GetQueue() method which has a MessageCountDetails property, which in turn has a DeadLetterMessageCount property. Something like:
var namespaceManager = Microsoft.ServiceBus.NamespaceManager.CreateFromConnectionString("<CONN_STRING>");
var messageDetails = namespaceManager.GetQueue("<QUEUE_NAME>").MessageCountDetails;
var deadLetterCount = messageDetails.DeadLetterMessageCount;

RabbitMQ only listens to the first message on a queue

I'm having an issue with my Rabbit queues that is currently only reacting to the first message in queue, after that any other messages being pushed are being ignored.
I start with instantiating the connection and declaring the queue in my IQueueConnectionProvider:
var connectionFactory = new ConnectionFactory() { HostName = hostName };
var connection = _connectionFactory.CreateConnection();
var channel = connection.CreateModel();
That IQueueConnectionProvider is then used in my IQueueListener as a dependency with just one method:
public void ListenToQueue(string queue)
{
var channel = _queueConnectionProvider.GetQueue();
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
string path = #"d:\debug.log.txt";
File.AppendAllLines(path, new List<string>() {"MESSAGE RECEIVED", Environment.NewLine });
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
channel.BasicAck(ea.DeliveryTag, false);
};
channel.BasicConsume(queue, true, consumer);
}
My log file ends up being just one line "MESSAGE RECEIVED", however I can see in the Rabbit ui interface that my other services are pushing the messages to that queue just fine.
Is there something I'm missing here?

This was a dumb mistake... yet again.
channel.BasicConsume(queue, false, consumer);
This was what I needed. I want to manually acknowledge my messages, therefore noAck needs to be false;

The code works fine! Have tested with my queue, and was able to get "MESSAGE RECEIVED" 9 times in the log file; since I had 9 messages in my queue.
I tried without this line of code, and it worked fine
var channel = _queueConnectionProvider.GetQueue();

Azure service bus queue Message deadlettered after Message.Abandon

I am trying out Azure Service Bus queue. I have the below code:
Queue send:
string strConnectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"];
var namespaceManager = NamespaceManager.CreateFromConnectionString(strConnectionString);
if (!namespaceManager.QueueExists("Test"))
{
QueueDescription qD = new QueueDescription("Test");
qD.DefaultMessageTimeToLive = new TimeSpan(05, 00, 00);
qD.LockDuration = new TimeSpan(00, 02, 30);
qD.MaxSizeInMegabytes = 5120;
namespaceManager.CreateQueue(qD);
}
if (namespaceManager.QueueExists("Test"))
{
QueueClient client = QueueClient.CreateFromConnectionString(strConnectionString, "Test", ReceiveMode.PeekLock);
var qMessage = Console.ReadLine();
using (MemoryStream strm = new MemoryStream(Encoding.UTF8.GetBytes(qMessage)))
{
BrokeredMessage bMsg = new BrokeredMessage(strm);
bMsg.MessageId = Guid.NewGuid().ToString();
bMsg.TimeToLive = new TimeSpan(05, 00, 00);
client.Send(bMsg);
Console.WriteLine("Message sent");
}
}
Console.ReadLine();
The receive code:
string strConnectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"];
var namespaceManager = NamespaceManager.CreateFromConnectionString(strConnectionString);
if (namespaceManager.QueueExists("Test"))
{
QueueClient client = QueueClient.CreateFromConnectionString(strConnectionString, "Test",ReceiveMode.PeekLock);
if (client != null)
{
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
options.AutoRenewTimeout = TimeSpan.FromSeconds(31);
client.OnMessage((message) =>
{
Console.WriteLine(message.State.ToString());
Console.WriteLine("Message Id: " + message.MessageId);
Stream stream = message.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
Console.WriteLine("Message: " + reader.ReadToEnd());
Console.WriteLine("***************");
message.Abandon();
});
Console.ReadLine();
}
}
I see that whenever I call Abandon, the message is getting DeadLettered. My assumption was that it should get Active and can be picked up by another client.

Your understanding of BrokeredMessage.Abandon Api is correct. It is intended to abandon the peek-lock acquired on the message (but NOT abandon the message itself) and hence, makes it available for other receivers to pick the Message up.
Here's how we envisioned different states of a peek-lock'ed message:
Basics first
The 'Why': If Customers need Competing-Consumer (Job-Queue) semantics - where they need multiple workers to simultaneously process different messages from a Queue with Exactly-Once guarantee - then they use the ReceiveMode.PeekLock. In this model, each worker (the queue receiver) needs a way to communicate the Progress of its Current message (Job) to other workers. Hence, brokeredMessage provides 4 functions to express the states.
The 'What':
if a message is successfully processed by the current Worker - call BrokeredMessage.Complete()
if the BrokeredMessage cannot be processed by the current worker, and want the processing to be retried on another Worker - then, Abandon the message. But, the catch here is: lets say, there are 2 workers and each of them thinks that the other one can process this message and calls Abandon - soon they will end up in an Infinite loop of retry'ing to process just that message! So, to avoid this situation, we provided a Configuration called MaxDeliveryCount on QueueDescription. This setting guards the limit on the number of times the message is delivered from the Queue to receiver. In the above example, Each time you received (and abandoned) the message, the 'deliveryCount' on the ServiceBus service is incremented. When it reaches 10 - the message has hit max no. of deliveries and hence, will be deadlettered.
if the current receiver (worker) knows for sure, that, this message cannot be processed, BrokeredMessage.DeadLetter(). The goal here is to let the consuming application Audit the dead-lettered messages regularly.
if the current receiver (worker) cannot process this message, but, knows that this message can be processed at a later point of time BrokeredMessage.Defer().
HTH!
Sree

In C#, how can I process all RabbitMQ messages currently on the queue?

The basic RabbitMQ tutorial gives an example of how to retrieve messages continuously from a queue:
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("hello", false, false, false, null);
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume("hello", true, consumer);
Console.WriteLine(" [*] Waiting for messages." +
"To exit press CTRL+C");
while (true)
{
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
}
}
}
What I want to do is retrieve all messages which have been placed onto the queue and then stop.
Here are two examples that would solve my problem
If I start my code at 1pm, I want to process all the messages that have been placed on the queue before 1pm.
OR
If I start my code at 13:00:00, and it takes 10 seconds for my code to run, I don't mind if it includes messages placed on the queue between 13:00:00 and 13:00:10, as long as it stops as soon as the queue is empty.
I realize that I can probably put a time stamp in my message and check for that, or I could fiddle with timeout values, but I was wondering if there's any built in way to do this properly.
Thanks in advance.

From the comments it seems that RabbitMQ is not meant for batch processing, so it hasn't been designed for this purpose.
I've also noticed that, when testing the DequeueNoWait method, or attempting to Dequeue with a zero timeout, they didn't work at all, and simply returned null.
The following solution uses QueueDeclare to get a count of existing messages and doesn't require a time stamp or a hacky timeout:
var factory = new ConnectionFactory { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
var queueDeclareResponse = channel.QueueDeclare(Constants.QueueName, false, false, false, null);
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(Constants.QueueName, true, consumer);
Console.WriteLine(" [*] Processing existing messages.");
for (int i = 0; i < queueDeclareResponse.MessageCount; i++)
{
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
}
Console.WriteLine("Finished processing {0} messages.", queueDeclareResponse.MessageCount);
Console.ReadLine();
}
}

What you can do:
1.) First add a timestamp to the message object
2.) Reject the message if the timestamp is not valid. It is an integrated feature of rabbitmq
Reference