We have been using RabbitMQ as messaging service in the project. We will be pushing message into a queue and which will be received at the message consumer and will be trying to make entry into database. Once the values entered into the database we will be sending positive acknowledgement back to the RabbitMQ server if not we will be sending negative acknowledgement.
I have created Message Consumer as Windows service.Message has been successfully entered and well received by the message consumer(Made entry in table)but with an exception log "Shared Queue closed".
Please find the code block.
while (true)
{
try
{
if (!Connection.IsOpen || !Channel.IsOpen)
{
CreateConnection(existConnectionConfig, QueueName);
consumer = new QueueingBasicConsumer(Channel);
consumerTag=Channel.BasicConsume(QueueName,false,consumer);
}
BasicDeliverEventArgs e = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
IBasicProperties props = e.BasicProperties;
byte[] body = e.Body;
bool ack = onMessageReceived(body);
if (ack == true)
{
Channel.BasicAck(e.DeliveryTag, false);
}
else
Channel.BasicNack(e.DeliveryTag, false, true);
}
catch (Exception ex)
{
//Logged the exception in text file where i could see the
//message as "Shared queue closed"
}
}
I have surfed in net too but couldn't able to what the problem. It will be helpful if anyone able to help me out.
Thanks in advance,
Selva
In answer to your question, I have experienced the same problems when my Web Client has reset the connection due to App Pool recycling or some other underlying reason the connection has been dropped that appears beyond your scope. You may need to build in a retry mechanism to cope with this.
You might want to look at MassTransit. I have used this with RabbitMQ and it makes things a lot easier by effectively providing a management layer to RabbitMQ. MassTransit takes away the headache of retry mechanisms - see Connection management. It also provides a nice multi threaded concurrent consumer configuration.
This has the bonus of your implementation being more portable - you could easily change things to MSMQ should the requirement come along.
Related
BACKGROUND INFO
I have a queue (for emails) in RabbitMQ, and want to build a consumer for it. The queue is used by another .NET app for sending emails to customers. I wanted the emailing logic to sit outside of the .NET app, and also have the benefits of durability ...etc that RabbitMQ offers.
ISSUE
The .NET app is able to publish/push emails onto the queue, but I have difficulty building the consumer! Here's my code for the consumer:
// A console app that would be turned into a service via TopShelf
public void Start()
{
using (_connection = _connectionFactory.CreateConnection())
{
using (var model = _connection.CreateModel())
{
model.QueueDeclare(_queueName, true, false, false, null);
model.BasicQos(0, 1, false);
var consumer = new EventingBasicConsumer(model);
consumer.Received += (channelModel, ea) =>
{
var message = (Email) ea.Body.DeSerialize(typeof(Email));
Console.WriteLine("----- Email Processed {0} : {1}", message.To, message.Subject);
model.BasicAck(ea.DeliveryTag, false);
};
var consumerTag = model.BasicConsume(_queueName, false, consumer);
}
}
}
The code above should be able to grab messages off the queue and process them (according to this official guide), but this isn't happening.
The problem is premature connection disposal. People often think that BasicConsume is a blocking call, but it is not. It will return almost immediately, and the very next statement is disposing (closing) of channel and connection which of course will cancel your subscription. So to fix - store connection and model in private fields and dispose them only when you are done with queue consumption.
You said queue is used by another .Net app, is that another consumer? If that is another consumer then can you please confirm which exchange you are using? If you want multiple consumers to pick up the message then please go ahead with "FanOut" exchange
I have a C# application that sets up numerous MQ listeners (multiple threads and potentially multiple servers each with their own listeners). There are some messages that will come off the queue that I will want to leave on the queue, move on to the next message on the MQ, but then under some circumstances I will want to go back to re-read those messages...
var connectionFactory = XMSFactoryFactory.GetInstance(XMSC.CT_WMQ).CreateConnectionFactory();
connectionFactory.SetStringProperty(XMSC.WMQ_HOST_NAME, origination.Server);
connectionFactory.SetIntProperty(XMSC.WMQ_PORT, int.Parse(origination.Port));
connectionFactory.SetStringProperty(XMSC.WMQ_QUEUE_MANAGER, origination.QueueManager);
connectionFactory.SetStringProperty(XMSC.WMQ_CHANNEL, origination.Channel);
var connection = connectionFactory.CreateConnection(null, null);
_connections.Add(connection);
var session = connection.CreateSession(false, AcknowledgeMode.ClientAcknowledge); //changed to use ClientAcknowledge so that we will leave the message on the MQ until we're sure we're processing it
_sessions.Add(session);
var destination = session.CreateQueue(origination.Queue);
_destinations.Add(destination);
var consumer = session.CreateConsumer(destination);
_consumers.Add(consumer);
Logging.LogDebugMessage(Constants.ListenerStart);
connection.Start();
ThreadPool.QueueUserWorkItem((o) => Receive(forOrigination, consumer));
Then I have...
if (OnMQMessageReceived != null)
{
var message = consumer.Receive();
var identifier = string.Empty;
if (message is ITextMessage)
{
//do stuff with the message here
//populates identifier from the message
}
else
{
//do stuff with the message here
//populates identifier from the message
}
if (!string.IsNullOrWhiteSpace(identifier)&& OnMQMessageReceived != null)
{
if( some check to see if we should process the message now)
{
//process message here
message.Acknowledge(); //this really pulls it off of the MQ
//here is where I want to trigger the next read to be from the beginning of the MQ
}
else
{
//We actually want to do nothing here. As in do not do Acknowledge
//This leaves the message on the MQ and we'll pick it up again later
//But we want to move on to the next message in the MQ
}
}
else
{
message.Acknowledge(); //this really pulls it off of the MQ...its useless to us anyways
}
}
else
{
Thread.Sleep(0);
}
ThreadPool.QueueUserWorkItem((o) => Receive(forOrigination, consumer));
So a couple of questions:
If I do not acknowledge the message it stays on the MQ, right?
If the message is not acknowledged then by default when I read from the MQ again with the same listener it reads the next one and does not go to the beginning, right?
How do I change the listener so that the next time I read I start at the beginning of the queue?
Leaving messages on a queue is an anti-pattern. If you don't want to or cannot process the message at a certain point of your logic, then you have a number of choices:
Get it off the queue and put to another queue/topic for a delayed/different processing.
Get it off the queue and dump to a database, flat file - whatever, if you want to process it outside of messaging flow, or don't want to process at all.
If it is feasible, you may want to change the message producer so it doesn't mix the messages with different processing requirements in the same queue/topic.
In any case, do not leave a message on the queue, and always move forward to the next message. This will make the application way more predictable and easier to reason about. You will also avoid all kinds of performance problems. If your application is or may ever become sensitive to the sequence of message delivery, then manual acknowledgement of selected messages will be at odds with it too.
To your questions:
The JMS spec is vague regarding the behavior of unacknowledged messages - they may be delivered out of order, and it is undefined when exactly when they will be delivered. Also, the acknowledge method call will acknowledge all previously received and unacknowledged messages - probably not what you had in mind.
If you leave messages behind, the listener may or may not go back immediately. If you restart it, it of course will start afresh, but while it is sitting there waiting for messages it is implementation dependent.
So if you try to make your design work, you may get it kind of work under certain circumstances, but it will not be predictable or reliable.
I've tried checking the server:port with telnet and I'm getting the expected results. So either writer.Write() or reader.ReadLine() isn't working cause I get nothing from the server.
TcpClient socket = new TcpClient(hostname, port);
if (!socket.Connected) {
Console.WriteLine("Failed to connect!");
return;
}
TextReader reader = new StreamReader(socket.GetStream());
TextWriter writer = new StreamWriter(socket.GetStream());
writer.Write("PING");
writer.Flush();
String line = null;
while ((line = reader.ReadLine()) != null) {
Console.WriteLine(line);
}
Console.WriteLine("done");
EDIT: I might have found the issue. This code was based off examples I found on the web. I tried another irc server: open.ircnet.net:6669 and I got a response:
:openirc.snt.utwente.nl 020 * :Please wait while we process your connection.
It seems as if I probably need to run the reader in a Thread so it can just constantly wait for a response. However it does seem weird that the program got caught on the while loop without ever printing done to the console.
I think you need to provide further details. I'm just going to assume that because you can easily telnet to the server using the same port your problem lies in the evaluation of the Connected property...
if (!socket.Connected) {
Console.WriteLine("Failed to connect!");
return;
}
this is wrong because Microsoft clearly specifies in the documentation that the Connected property is not reliable
Because the Connected property only reflects the state of the connection as of the most recent operation, you should attempt to send or receive a message to determine the current state. After the message send fails, this property no longer returns true. Note that this behavior is by design. You cannot reliably test the state of the connection because, in the time between the test and a send/receive, the connection could have been lost. Your code should assume the socket is connected, and gracefully handle failed transmissions.
That said, you should not use this property to determine the state of the connection. Needless to say that using this property to control the flow of your console app will result in unexpected results.
Suggestion
Remove the evaluation of the Connected property
Wrap your GetStream and Write method calls in a try/catch block to handle network communication errors
reader.ReadLine() will just wait for any data to arrive. If no data arrive, it seems to hang. That's a feature of tcp (I don't like it either). You need to find out how the end of the message is defined and stop based on that end criterion. Be careful, the end of message identifier may be split into two or more lines...
RFC for ping says that the server may not respond to it & such connections has to be closed after a time. Please check the RFC: https://www.rfc-editor.org/rfc/rfc1459#section-4.6.2
I am new to activemq. T want to ask a question about the topics of Activemq. I succeed to get a message from a queue. Also I can send message to topic/Queue, but I can't get a message from Topic.
I have tried using Java Code. The result is the same.
The following is my core code:
connection.ClientId = clientId;
connection.Start();
using (ISession session = connection.CreateSession())
{
ITopic topic = new Apache.NMS.Commands.Topic(topicName);
IDestination destination = SessionUtil.GetDestination(session, topicName,
DestinationType.Topic);
using (IMessageConsumer consumer = **session.CreateDurableConsumer**(topic, "news", null, false))
{
**consumer.Listener += new MessageListener(consumer_Listener);**
//**IMessage iMsg = consumer.Receive();**
// if (iMsg != null)//{
// ITextMessage msg = (ITextMessage)iMsg;
// return msg.Text;
// }
//else
//return iMsg;
}
}
I also using: IMessage iMsg = consumer.Receive();
IMsg always null（topicname has messages. How can I consume topic's message?
The Messages would need to have been sent after the Topic consumer was created. A Topic is fire and forget, if there are no consumers then the message is discarded. Any consumer that comes online will only receive message sent after that time unless it is either a Durable Topic consumer or a Queue consumer.
In the case of a durable consumer you must have created an instance of it so there is a subscription record before those message were sent to the Topic. So I would guess your problem is that you didn't subscribe this consumer before and so the Broker was not storing any Messages for it.
I was so stupid about the phrase "using".Beacause I use "using" open connection and session. when the code block was excuted, the connnection/session is disappear. Now I dont use "using" block to cerate connection. just like normal code. It works. also I build "Global.asax" file. The program can listener Topic once started up. At the same time, I write a function to colse the connection.I tested. Once a message was sent to the topic, the Onessage() function would be exectued.
just resolve my problem.maybe you would have better answer.Thanks Tim.
I'm trying to put a Message back into an MSMQ when an exception is thrown. The following code appears to work but the Message is not put back in the queue?
Message msg = null;
try
{
MessageQueue MQueue = new MessageQueue(txtMsgQPath.Text);
msg = MQueue.ReceiveById(txtQItemToRead.Text);
lblMsgRead.Text = msg.Body.ToString(); // This line throws exception
}
catch (Exception ex)
{
lblMsgRead.Text = ex.Message;
if (msg != null)
{
MessageQueue MQ = new MessageQueue(txtMsgQPath.Text);
MQ.Send(msg);
}
}
Couple of points: The best way to do this would be using a transaction spanning both queues; that way you'll know you won't lose a message.
The second part of it is to be careful about how the queues are created and how you submit messages to the second queue. In particular, MSMQ sometimes appears to "fail silently" when sending a message (though in reality an error message is recorded elsewhere in the dead letter queues), particularly if the transactional options of the send don't match the transactional nature of the target queue.
Is it really your intention to send that message back to the originator? Sending it back to yourself is very dangerous, you'll just bomb again, over and over.
I believe that you're looking to "Peek" at the message. Use: MessageQueue.Peek and if you succeed, then consume the message.
I managed to get the code above to work by creating a new queue and pointing the code at the new queue.
I then compared the 2 queues and noticed that the new queue was multicast (the first queue wasn't) and the new queue had a label with the first didn't. Otherwise the queues appeared to be the same.