I have a few questions on coding in RabbitMQ... I'm not new to this world and have got questions based on a design provided to me for implementation...
If I send BasicAck or BasicAack from Consumer, does it only remove
the corresponding message from the queue or will it deliver Ack to
the Publisher?
How do I ensure that Publisher sends message to Server only when Consumer is ready for processing?
The design says Publisher needs to wait and know when the processing of Consumer is completed to do certain task on client side (depending on success / failure).
I have tried below code but dequeue immediately removed message from queue without I send any Ack or Nack. I'm confused
publisher code:
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("test", durable, false, false, null);
channel.TxSelect();
var properties = channel.CreateBasicProperties();
properties.SetPersistent(true);
string message = "Hello World!";
var body = Encoding.UTF8.GetBytes(message);
channel.BasicPublish("", "test", properties, body);
channel.TxCommit();
Console.WriteLine(" [x] Sent {0}", message);
}
}
Consumer code
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("test", durable, false, false, null);
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume("test", true, consumer);
Console.WriteLine(" [*] Waiting for messages." +
"To exit press CTRL+C");
while (true)
{
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
}
}
}
Note: I realized that channel.BasicConsume("test", true, consumer); has noAck to true. I changed this to channel.BasicConsume("test", false, consumer);
I can see the message is removed from the queue when I used channel.BasicAck(deliveryTag: ea.DeliveryTag, multiple: false); But, how does Publisher know that the Consumer processed it successfully?
How do I ensure that Publisher sends message to Server only when Consumer is ready for processing?
You can't. And more important, you shouldn't. The point of using a messaging architecture is to forget about this kind of problems. Check this.
Also, RabbitMQ will store this messages for you until someone will be ready to process them if the queue is durable.
But, how does Publisher know that the Consumer processed it successfully?
No, will not. The ack is only between RabbitMQ and your consumer or between RabbitMQ and your producer. Check this for some details on ack/nack.
What you want to achieve here, if I can understand you correctly, is a kind of "chatty" architecture, where the consumer is also a producer of "response messages" for the publisher, messages like "hey, I'm done with message XX, everything is ok".
The easies way to do this is to set your consumer to be also a producer, and your producer to be also a consumer. You simply need to add a guid or an unique id of some sort to the message, and when you're done with it, you will send a message on another queue with that id as content to inform the original publisher (which is a consumer of this "response" queue) that the work has completed successfully.
Hope it helps :)
This is kind of a duplicated question. Check this thread, which is a similar question with a correct answer: Why doesn't Channel.waitForConfirmsOrDie block?
Or you can use the RPC pattern
https://www.rabbitmq.com/tutorials/tutorial-six-dotnet.html
Related
Here is what I have so far. I've been able to successfully publish a message to a queue and see that it is there via RabbitMQ's management console.
However, when I try to receive it, it does not seem to trigger the callback function at all.
Here is the relevant code.
MessageQueue mq = new MessageQueue();
mq.Receive("task_queue", (model, ea) => {
var message = Encoding.UTF8.GetString(ea.Body);
System.Diagnostics.Debug.WriteLine(message);
});
Here is my Receive function in the MessageQueue class:
public void Receive(string queueName, EventHandler<BasicDeliverEventArgs> onReceived)
{
using (IConnection connection = GetConnection(myLocalhost))
{
using (IModel channel = connection.CreateModel())
{
channel.QueueDeclare(queue: queueName, durable: true, exclusive: false, autoDelete: false, arguments: null);
// Don't dispatch a new message to a consumer until it has processed and acknowledged the previous one.
channel.BasicQos(prefetchSize: 0, prefetchCount: 1, global: false);
var consumer = new EventingBasicConsumer(channel); // non-blocking
// Set the event to be executed when receiving a message
consumer.Received += onReceived;
// Register a consumer to listen to a specific queue.
channel.BasicConsume(queue: queueName, autoAck: true, consumer: consumer);
}
}
}
When I try to run the Receive function while there is something in the queue, nothing is printed to my output window.
Can anyone help me on this?
UPDATE
I took the code in the Receive function and placed it in the same file as the code that calls it. Still no luck. That rules out a scoping issue I think. I also tried setting the Received field to an actual event delegate (instead of a onReceive function and had that call another function in which I put a breakpoint. That function is never hit leading me to believe that my event delegate callback is never being called at all.
I'm at a loss as to why this is. The message is still being consumed from the queue as the RabbitMQ management console shows me. I've also tried renaming the queue to something else to make sure no other phantom services are consuming from the same queue. No cigar.
UPDATE 2
I tried extracting the two using statements and calling my Receive function inside there in order to keep the scope but that didn't work either. I even extracted the code in the whole Receive block out to a main function and now it doesn't even consume from the queue.
Looking at your code above, you have a pretty straightforward problem.
The instant after you call channel.BasicConsume, the whole thing (connection/channel) goes out of scope and is immediately disposed/destroyed via the using statement.
To prevent this from happening, you need to have an infinite loop immediately following the channel.BasicConsume, with appropriate logic of course to exit when you shut down the program.
while (_isRunning & channel.IsOpen) {
Thread.Sleep(1);
// Other application logic here; e.g. periodically break out of the
// loop to prevent unacknowledged messages from accumulating in the system
// (if you don't, random effects will guarantee that they eventually build up)
}
BACKGROUND INFO
I have a queue (for emails) in RabbitMQ, and want to build a consumer for it. The queue is used by another .NET app for sending emails to customers. I wanted the emailing logic to sit outside of the .NET app, and also have the benefits of durability ...etc that RabbitMQ offers.
ISSUE
The .NET app is able to publish/push emails onto the queue, but I have difficulty building the consumer! Here's my code for the consumer:
// A console app that would be turned into a service via TopShelf
public void Start()
{
using (_connection = _connectionFactory.CreateConnection())
{
using (var model = _connection.CreateModel())
{
model.QueueDeclare(_queueName, true, false, false, null);
model.BasicQos(0, 1, false);
var consumer = new EventingBasicConsumer(model);
consumer.Received += (channelModel, ea) =>
{
var message = (Email) ea.Body.DeSerialize(typeof(Email));
Console.WriteLine("----- Email Processed {0} : {1}", message.To, message.Subject);
model.BasicAck(ea.DeliveryTag, false);
};
var consumerTag = model.BasicConsume(_queueName, false, consumer);
}
}
}
The code above should be able to grab messages off the queue and process them (according to this official guide), but this isn't happening.
The problem is premature connection disposal. People often think that BasicConsume is a blocking call, but it is not. It will return almost immediately, and the very next statement is disposing (closing) of channel and connection which of course will cancel your subscription. So to fix - store connection and model in private fields and dispose them only when you are done with queue consumption.
You said queue is used by another .Net app, is that another consumer? If that is another consumer then can you please confirm which exchange you are using? If you want multiple consumers to pick up the message then please go ahead with "FanOut" exchange
I have active queue which will have all messages from Publisher. My Consumer reads those message and Acks/Nacks depending on the message processing result.
while (true)
{
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
var processed = ProcessMessage(message)
if (processed)
channel.BasicAck(deliveryTag: ea.DeliveryTag, multiple: false);
else
channel.BasicNack(deliveryTag: ea.DeliveryTag, multiple: false, requeue: true);
}
My questions are
Is setting true for requeue parameter when it is nacked correct?
Or do we need to create another queue for Retry?
Let us say, if I want to move the message to DLX after retrying for 10 times? How do I do it? Is it C# code or can a rule be defined on the queue?
How do I know that a message is retried for 10 times? Does RabbitMQ provide any mechanism or do I need to manually design message object to contain retry count?
Thanks for your inputs
Starting with release 3.5.2, RabbitMQ automatically adds a header to dead-letterred messages with informations such as:
the queue(s) which saw the message
the reason(s) it was dead-letterred
the number of times it was dead-letterred
timestamps
Look at the "Dead-Lettered Messages" section near the end of the DLX documentation for more details.
If you use an older version of RabbitMQ, then #Franklin's solution should work.
if you set requeue to false then it will go to any DeadLetter Exchange assigned to the Queue. True will requeue the message.
What I have done for retry attempts is to Create a Hold Exchange and Queue. If you want to retry a message Return a positive Ack to the Queue, Add a RetryAttepmts Header to the Message then Publish it to the HoldQueue Exchange with a timeout value. Set the Hold Queue Dead Letter Exchange to an exchange that will send the message to the original Queue. Then Check the header and nack if the retry attempts are too large.
I am new to activemq. T want to ask a question about the topics of Activemq. I succeed to get a message from a queue. Also I can send message to topic/Queue, but I can't get a message from Topic.
I have tried using Java Code. The result is the same.
The following is my core code:
connection.ClientId = clientId;
connection.Start();
using (ISession session = connection.CreateSession())
{
ITopic topic = new Apache.NMS.Commands.Topic(topicName);
IDestination destination = SessionUtil.GetDestination(session, topicName,
DestinationType.Topic);
using (IMessageConsumer consumer = **session.CreateDurableConsumer**(topic, "news", null, false))
{
**consumer.Listener += new MessageListener(consumer_Listener);**
//**IMessage iMsg = consumer.Receive();**
// if (iMsg != null)//{
// ITextMessage msg = (ITextMessage)iMsg;
// return msg.Text;
// }
//else
//return iMsg;
}
}
I also using: IMessage iMsg = consumer.Receive();
IMsg always null（topicname has messages. How can I consume topic's message?
The Messages would need to have been sent after the Topic consumer was created. A Topic is fire and forget, if there are no consumers then the message is discarded. Any consumer that comes online will only receive message sent after that time unless it is either a Durable Topic consumer or a Queue consumer.
In the case of a durable consumer you must have created an instance of it so there is a subscription record before those message were sent to the Topic. So I would guess your problem is that you didn't subscribe this consumer before and so the Broker was not storing any Messages for it.
I was so stupid about the phrase "using".Beacause I use "using" open connection and session. when the code block was excuted, the connnection/session is disappear. Now I dont use "using" block to cerate connection. just like normal code. It works. also I build "Global.asax" file. The program can listener Topic once started up. At the same time, I write a function to colse the connection.I tested. Once a message was sent to the topic, the Onessage() function would be exectued.
just resolve my problem.maybe you would have better answer.Thanks Tim.
I do not know why the processing of page does not stops when it dequeue. I am using Dot net/C#. How can I show the messages If this deadlock does not stops?
Moreover I have seen in tutorial to create a exchange and then create a dynamic queue and bind it in Tutorial no. 3. but how come in this code, I have taken it from rabbitMQ c# Examples but they have not used this.
I can not find the random created queue and then binding with exchange.
Emit Code is given over here::https://github.com/rabbitmq/rabbitmq-tutorials/blob/master/dotnet/EmitLog.cs
and In receive, If we have not created any queue while emit/send log , then why we are creating that here ::
https://github.com/rabbitmq/rabbitmq-tutorials/blob/master/dotnet/ReceiveLogs.cs
and most of the all deadlock in this code where there is written Dequeue.
using (var channel = connection.CreateModel())
{
channel.ExchangeDeclare("logs", "fanout");
var queueName = channel.QueueDeclare();
channel.QueueBind(queueName, "logs", "");
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(queueName, true, consumer);
Console.WriteLine(" [*] Waiting for logs." +
"To exit press CTRL+C");
while (true)
{
var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] {0}", message);
}
}
1)One more question, I am using exchange to send the message to all users. IS it right or wrong? Because I am not getting anything after reading tutorials also.they are confusing.
I want to write a message in a text box then I want that whenever any user clicks on receive message, he/she should receive that message.Can this be possible using Exchanges?
2)And For everytime/everytime when page will open, I have to create a new Queue/Exchange?
Please Help me.