I'm new to RabbitMQ and trying to write to a Queue and verify the message was sent. If it fails I need to know about it.
I made a fake queue to watch it fail but no matter what I see no execptions and when I am looking for a ack I always get one. I never see the BasicNack.
I'm not even sure i'm the BasicAcks is the way to go.
private void button1_Click(object sender, EventArgs e)
{
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare("task_queue", true, false, false, null);
var message = ("Helllo world");
var body = Encoding.UTF8.GetBytes(message);
channel.ConfirmSelect();
var properties = channel.CreateBasicProperties();
properties.SetPersistent(true);
properties.DeliveryMode = 2;
channel.BasicAcks += channel_BasicAcks;
channel.BasicNacks += channel_BasicNacks;
//fake queue should be task_queue
channel.BasicPublish("", "task_2queue", true, properties, body);
channel.WaitForConfirmsOrDie();
Console.WriteLine(" [x] Sent {0}", message);
}
}
}
void channel_BasicNacks(IModel model, BasicNackEventArgs args)
{
}
void channel_BasicAcks(IModel model, BasicAckEventArgs args)
{
}
For those looking for a C# answer - here is what you need.
https://rianjs.net/2013/12/publisher-confirms-with-rabbitmq-and-c-sharp
Something like this: (BasicAcks attaches an event handler - there is also BasicNacks)
using (var connection = FACTORY.CreateConnection())
{
var channel = connection.CreateModel();
channel.ExchangeDeclare(QUEUE_NAME, ExchangeType.Fanout, true);
channel.QueueDeclare(QUEUE_NAME, true, false, false, null);
channel.QueueBind(QUEUE_NAME, QUEUE_NAME, String.Empty, new Dictionary<string, object>());
channel.BasicAcks += (sender, eventArgs) =>
{
//implement ack handle
};
channel.ConfirmSelect();
for (var i = 1; i <= numberOfMessages; i++)
{
var messageProperties = channel.CreateBasicProperties();
messageProperties.SetPersistent(true);
var message = String.Format("{0}\thello world", i);
var payload = Encoding.Unicode.GetBytes(message);
Console.WriteLine("Sending message: " + message);
channel.BasicPublish(QUEUE_NAME, QUEUE_NAME, messageProperties, payload);
channel.WaitForConfirmsOrDie();
}
}
You need a Publisher Confirms
as you can read you can implement:
The transaction:
ch.txSelect(); <-- start transaction
ch.basicPublish("", QUEUE_NAME,
MessageProperties.PERSISTENT_BASIC,
"nop".getBytes());
ch.txCommit();<--commit transaction
The message is stored to the queue and to the disk.
This way can be slow, if you need performance you shouldn't use it.
You can use the Streaming Lightweight Publisher Confirms, using:
ch.setConfirmListener(new ConfirmListener() {
public void handleAck(long seqNo, boolean multiple) {
if (multiple) {
unconfirmedSet.headSet(seqNo+1).clear();
} else {
unconfirmedSet.remove(seqNo);
}
}
public void handleNack(long seqNo, boolean multiple) {
// handle the lost messages somehow
}
I hope it helps
Ok, you always get the ACK for your message sent because "Every time message is delivered to Default Exchange Successfully."
PS: You are not sending message directly to Queue, Once Exchange recevis the message it gives you ACK then it route the message to all bound queue using the routing keys if any.
Related
I am publishing json messages into a queue in rabbitmq and it is workin properly. But facing an issue that i want to consume all data in the issued queue (as a chat app) and i have to use all messages.
For example I have 9 items in the queue as below
{"Sender":123,"Message":"Test Message-1","Group":1}
{"Sender":123,"Message":"Test Message-2","Group":1}
{"Sender":123,"Message":"Test Message-3","Group":1}
{"Sender":123,"Message":"Test Message-4","Group":1}
{"Sender":567,"Message":"Test Message-5","Group":21}
{"Sender":123,"Message":"Test Message-6","Group":1}
{"Sender":456,"Message":"Test Message-7","Group":1}
{"Sender":456,"Message":"Test Message-8","Group":1}
{"Sender":123,"Message":"Test Message-9","Group":1}
These all messages are stored in queue as i want. But when i try to collect them with an api call as below it won't work properly. Sometimes getting data but sometimes don't get any data and acked the list. So is there any way to get all or limited data into an object or array in c#. Because all examples are consuming messages into the Console. I need to get as a collection.
public IList<string> GetMessageFromQueue(string _key, bool AutoAck = false)
{
var _list = new List<string>();
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: _key,
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var response = channel.QueueDeclarePassive(_key);
var _test= response.MessageCount;
var _test2 = response.ConsumerCount;
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body.ToArray();
var message = Encoding.UTF8.GetString(body);
_list.Add(message);
};
//if (_list.Count == 0)
// AutoAck = false;
channel.BasicConsume(queue: _key,
autoAck: AutoAck,
consumer: consumer);
}
return _list;
}
And My Controller
public IActionResult Collect(){
_queueClient.GetMessageFromQueue("myKey",true);
}
This method olsa clears the queue because of BasicConsume's autoack property. I tried to use basicAck also.
What is the best way to get messages to an object array for next operations in rabbitmq/c#.
It seems to me that your function GetMessageFromQueue is going through the motions of setting everything up, but then exiting from the function straight away without waiting for your Received function to collect all the messages.
So for example, this the the inline function you setup to collect the messages from the queue:
consumer.Received += (model, ea) =>
{
var body = ea.Body.ToArray();
var message = Encoding.UTF8.GetString(body);
_list.Add(message);
};
...but then 2 lines later you simply exit the function straight away without waiting for your Recieved function to added all the messages to your list.
// exit function straight away!
return _list;
I notice in your sample code, that you can get a count of the messages held on the queue. This is good because it means you know how many messages to expect to recieve.
var _test= response.MessageCount;
So, one thing you could try and do is add a ManualResetEventSlim or SemaphoreSlim to wait at the bottom of your function until its signaled and then return (there might be better ways of doing this but thats the idea that popped in my head right now)
For example, at the top of the fucntion create a ManualResetEventSlim event
var msgsRecievedGate = new ManualResetEventSlim(false);
and then before exiting the function wait for it to be set.
msgsRecievedGate.Wait();
Something like this:
public IList<string> GetMessageFromQueue(string _key, bool AutoAck = false)
{
var _list = new List<string>();
// Setup synchronization event.
var msgsRecievedGate = new ManualResetEventSlim(false);
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: _key,
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var response = channel.QueueDeclarePassive(_key);
var msgCount = response.MessageCount;
var msgRecieved = 0;
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
msgRecieved++;
var body = ea.Body.ToArray();
var message = Encoding.UTF8.GetString(body);
_list.Add(message);
if ( msgRecieved == msgCount )
{
// Set signal here
msgsRecievedGate.Set();
// exit function
return;
}
};
channel.BasicConsume(queue: _key,
autoAck: AutoAck,
consumer: consumer);
}
// Wait here until all messages are retrieved
msgsRecievedGate.Wait();
// now exit function!
return _list;
}
Please beware. I have not tested the above code so your mileage my vary.
I have different message types on the queue and I'm trying to create multiple C# consumers that each just take a certain type off the queue.
Apparently, I'm supposed to be able to specify the routing key, like shown here, to get just those messages:
channel.QueueBind(queue: queueName, exchange: "SandboxEventBus", routingKey: "MessageEvent");
No matter why I try, I get all of the messages from the queue and not just the ones with that routing key. What am I missing?
This is the class:
public static class RabbitReceiver
{
public static void PullFromQueue(string caller, int sleepTimeInMilliseconds)
{
string hostName = "localhost";
string queueName = "Sandbox.v1";
var factory = new ConnectionFactory { HostName = hostName, UserName = "guest", Password = "guest", Port = 6003 };
var connection = factory.CreateConnection();
var channel = connection.CreateModel();
var arguments = new Dictionary<string, object>();
arguments.Add("x-message-ttl", 864000000); // Not sure why I have to specify this. Got an exception if I didn't.
//arguments.Add("topic", "MessageEvent");
channel.BasicQos(0, 1, false);
// Enabling these lines doesn't change anything. It was part of my effort of trying different things.
//channel.ExchangeDeclare("SandboxEventBus", "topic", durable: true); // topic is case-sensitive
//channel.QueueDeclare(queue: queueName, durable: true, exclusive: false, autoDelete: false, arguments: arguments);
channel.QueueBind(queue: queueName, exchange: "SandboxEventBus", routingKey: "MessageEvent");
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, eventArgs) =>
{
var body = eventArgs.Body.ToArray();
var message = Encoding.UTF8.GetString(body);
Console.WriteLine($"Message: {message}");
Console.WriteLine($"{caller} - Pausing {sleepTimeInMilliseconds} ms before getting next message (so we can watch the Rabbit dashboard update).");
Thread.Sleep(sleepTimeInMilliseconds);
// This should probably go in a finally block.
channel.BasicAck(eventArgs.DeliveryTag, false);
};
channel.BasicConsume(queue: queueName, autoAck: false, consumer: consumer);
}
}
Here is the publishing code. I'm simply publishing two different message types.
public class MessagePublisherService : BackgroundService
{
private readonly ICapPublisher _capBus;
public MessagePublisherService(ICapPublisher capPublisher)
{
_capBus = capPublisher;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
// This publishes messages. To get the messages off the queue, run the RabbitMQ project in this solution.
try
{
// To have a different event, add this.
await _capBus.PublishAsync(nameof(SnuhEvent), new SnuhEvent());
Console.WriteLine($"Published message {nameof(SnuhEvent)}.");
for (int i = 1; i <= 10; i++)
{
//_capBus.Publish(nameof(MessageEvent), new MessageEvent(i));
await _capBus.PublishAsync(nameof(MessageEvent), new MessageEvent(i));
Console.WriteLine($"Published message {i}.");
}
}
catch (Exception ex)
{
Console.WriteLine("In catch block of publisher.");
Console.WriteLine(ex);
}
}
}
And here is the queue. Notice each message type shows as a different routing key.
Here is the RMQ code in Startup.cs:
public void ConfigureServices(IServiceCollection services)
{
services.AddCap(x =>
{
x.UseSqlServer("Server=localhost; Database=Sandbox; Integrated Security=true;");
x.DefaultGroup = "Sandbox";
x.FailedRetryCount = 3;
x.FailedRetryInterval = 2; // CAP will invoke the subscriber again if not finished in N seconds.
x.UseRabbitMQ(conf =>
{
conf.ExchangeName = "SandboxEventBus";
conf.ConnectionFactoryOptions = x =>
{
x.Uri = BuildRabbitUri();
};
});
});
services.AddHostedService<MessagePublisherService>();
}
public static Uri BuildRabbitUri()
{
string protocol = "amqp";
string userName = "guest";
string password = "guest";
string host = "localhost";
string port = "6003";
return new Uri(protocol + "://" + userName + ":" + password + "#" + host + ":" + port);
}
Using a topic exchange, I would like to have a publish/subscribe messaging pattern with the following features:
Have "publisher confirms" implemented.
Have the consumer acknowledge each message as well once it has processed it.
Use routing keys to route messages to one or more consumers.
Have persistent consumer queues so if the consumer application is down temporarily, it can pick up messages from its queue when it comes back up.
So I have created 2 console applications (Send and Receive) to test the above.
Send
static void Main(string[] args)
{
Console.WriteLine(" Type [exit] to exit.");
Publisher publisher = new Publisher();
do
{
var userInput = Console.ReadLine();
if (userInput == "exit")
{
break;
}
publisher.SendMessageToBroker("localhost", "main", "user.update", userInput);
} while (true);
}
Publisher
public class Publisher
{
const string ExchangeType = "topic";
Dictionary<ulong, string> unConfirmedMessageTags = new Dictionary<ulong, string>();
public void SendMessageToBroker(string host, string exchangeName, string routingKey, string message)
{
var factory = new ConnectionFactory() { HostName = host };
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.BasicAcks += (sender, ea) => OnBasicAcks(ea.Multiple, ea.DeliveryTag);
channel.BasicNacks += (sender, ea) => OnBasicNacks(ea.Multiple, ea.DeliveryTag);
channel.ConfirmSelect();
channel.ExchangeDeclare(exchangeName, ExchangeType);
var body = Encoding.UTF8.GetBytes(message);
var properties = channel.CreateBasicProperties();
properties.Persistent = true;
unConfirmedMessageTags.TryAdd(channel.NextPublishSeqNo, message);
channel.BasicPublish(exchange: exchangeName,
routingKey: routingKey,
basicProperties: properties,
body: body);
Console.WriteLine(" [x] Sent {0}", message);
}
}
private void OnBasicNacks(bool multiple, ulong deliveryTag)
{
if (multiple)
{
Console.WriteLine("Messages with delivery tag LESS THAN {0} have been LOST and must be resent.", deliveryTag);
}
else
{
Console.WriteLine("Message with delivery tag {0} has been LOST and must be resent.", deliveryTag);
}
}
private void OnBasicAcks(bool multiple, ulong deliveryTag)
{
if (multiple)
{
var confirmed = unConfirmedMessageTags.Where(k => k.Key <= deliveryTag);
foreach (var entry in confirmed)
{
unConfirmedMessageTags.Remove(entry.Key);
Console.WriteLine("Message with delivery tag {0} has been confirmed and deleted.", entry.Key);
}
}
else
{
unConfirmedMessageTags.Remove(deliveryTag);
Console.WriteLine("Message with delivery tag {0} has been confirmed and deleted.", deliveryTag);
}
}
}
}
Receive
static void Main(string[] args)
{
const string ExchangeName = "main";
const string QueueName = "q1";
const string ExchangeType = "topic";
const string RoutingKey = "user.update";
var factory = new ConnectionFactory() { HostName = "localhost" };
using(var connection = factory.CreateConnection())
using(var channel = connection.CreateModel())
{
channel.ExchangeDeclare(ExchangeName, ExchangeType);
channel.QueueDeclare(queue: QueueName,
durable: true,
autoDelete: false,
exclusive: false,
arguments: null);
channel.QueueBind(QueueName, ExchangeName, RoutingKey);
//channel.BasicQos(prefetchSize: 0, prefetchCount: 1, global: false);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) => Basic_Ack(channel, ea.DeliveryTag, ea.Body);
channel.BasicConsume(queue: QueueName, autoAck: false, consumer: consumer);
Console.WriteLine(" Press [enter] to exit.");
Console.ReadLine();
}
}
private static void Basic_Ack(IModel channel, ulong deliveryTag, ReadOnlyMemory<byte> body)
{
var message = Encoding.UTF8.GetString(body.ToArray());
Console.WriteLine(" [x] Received {0}", message);
Thread.Sleep(2000);
channel.BasicAck(deliveryTag: deliveryTag, multiple: false);
Console.WriteLine(" [x] Processed {0}", message);
}
}
The problem is the OnBasicAcks in my Send program only gets called once for the first message.
For anyone else out there who may hit this issue, I was opening a connection and channel (virtual connection) for every publish which is discouraged:
"Connections are meant to be long-lived. Opening a connection for every operation (e.g. publishing a message) would be very inefficient and is highly discouraged."
Also see here:
"Publisher confirms are enabled at the channel level with the ConfirmSelect method... This method must be called on every channel that you expect to use publisher confirms. Confirms should be enabled just once, not for every message published."
Switching to using a long-lived connection solved the issue for me.
I'm trying to convert my RabbitMQ micro-service to a windows service. I have used TopShelf for the conversion. My RabbitMQ micro-service works perfectly fine on its own but when I run it as a service it no longer receives messages. In my public static void Main(string[] args) I have:
HostFactory.Run(host =>
{
host.Service<PersonService>(s =>
{
s.ConstructUsing(name => new PersonService());
s.WhenStarted(tc => tc.Start());
s.WhenStopped(tc => tc.Stop());
});
host.SetDescription("Windows service that provides database access totables.");
host.SetDisplayName("Service");
host.SetServiceName("Service");
});
}
Then in my PersonService class I have
public void Start() {
ConsumeMessage();
}
And finally my ConsumeMessage function:
private static void ConsumeMessage() {
MessagingConfig.SetInstance(new MessagingConstants());
IMessageFactory pmfInst = MessageFactory.Instance;
//message worker
var factory = new ConnectionFactory() {
HostName = MessagingConfig.Instance.GetBrokerHostName(),
UserName = MessagingConfig.Instance.GetBrokerUserName(),
Password = MessagingConfig.Instance.GetBrokerPassword()
};
var connection = factory.CreateConnection();
using (var channel = connection.CreateModel()) {
channel.QueueDeclare(queue: MessagingConfig.Instance.GetServiceQueueName(),
durable: true,
exclusive: false,
autoDelete: false,
arguments: null);
channel.BasicQos(0, 1, false);
var consumer = new EventingBasicConsumer(channel);
channel.BasicConsume(queue: MessagingConfig.Instance.GetServiceQueueName(),
noAck: false,
consumer: consumer);
Console.WriteLine("Service.");
Console.WriteLine(" [x] Awaiting RPC requests");
// Code Below Is Not Executed In Service
consumer.Received += (model, ea) => {
string response = null;
var body = ea.Body;
var props = ea.BasicProperties;
var replyProps = channel.CreateBasicProperties();
replyProps.CorrelationId = props.CorrelationId;
string receivedMessage = null;
try {
receivedMessage = Encoding.UTF8.GetString(body);
response = ProcessMessage(receivedMessage);
}
catch (Exception e) {
// Received message is not valid.
WinLogger.Log.Error(
"Errror Processing Message: " + receivedMessage + " :" + e.Message);
response = "";
}
finally {
var responseBytes = Encoding.UTF8.GetBytes(response);
channel.BasicPublish(exchange: "", routingKey: props.ReplyTo,
basicProperties: replyProps, body: responseBytes);
channel.BasicAck(deliveryTag: ea.DeliveryTag,
multiple: false);
}
};
Console.ReadLine();
}
Looking at A similar SO question it looks like it has something to do with the return the Windows Service is wanting, but I'm not sure of how to call ConsumeMessage so consumer.Received += (model, ea) => {...}; is executed.
EDIT: It looks like my blocking mechanism Console.ReadLine(); is ignored by the service so it just continues on and disposes of the message consumer. So how do I block there for messages to be received?
You code uses using construct, which means when your OnStart method returns, your channel will actually be disposed. The docs suggest to do your initialization on OnStart, so create your channel and consumer there, but don't use using:
this.connection = factory.CreateConnection();
this.channel = connection.CreateModel();
this.consumer = new EventingBasicConsumer(this.channel);
Then those objects will continue to exist after OnStart method is finished. You should dispose of them in the OnStop method.
I am learning RabbitMq with .NET. According to the tutorial, simplest implementation of consumer looks like :
public class Receive
{
public static void Main()
{
var factory = new ConnectionFactory() { HostName = "localhost" };
using(var connection = factory.CreateConnection())
using(var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: "hello",
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(" [x] Received {0}", message);
};
channel.BasicConsume(queue: "hello",
noAck: true,
consumer: consumer);
Console.WriteLine(" Press [enter] to exit.");
Console.ReadLine();
}
}
}
And it works correctly. However, I wanted to refactor it: let define functionality of receiver in separate method. It looks like :
public class Recieve
{
private ConnectionFactory factory;
public void ConsumeSimpleMessage(string queueName = "default")
{
using(var connection = factory.CreateConnection())
{
using(var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: queueName, durable: false,
exclusive: false, autoDelete: false, arguments: null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
Console.WriteLine(message);
};
channel.BasicConsume(queue: queueName,
noAck: true,
consumer: consumer);
}
}
}
public Recieve(string hostName = "localhost")
{
factory = new ConnectionFactory() { HostName = hostName };
}
}
And when I try to call this method in Main():
class Program
{
static void Main(string[] args)
{
Recieve reciever = new Recieve();
reciever.ConsumeSimpleMessage();
Console.ReadLine();
}
}
here it doesn't work. It show nothing. However, messages will be deleted, meaning that they were recieved. Why does it happen? Is there anything I didn't know about Event Handing?
Try to see if it works without doing using, Or if you want to use the using statement keep the Console.Read() inside the using statement and see if that works. You can keep the connection and channel open and don't have to close it manually.
If you really want to debug then you can put the breakpoint on the consumer.Receive and see if you can see the message staying unacked. That way you know when the message will be deleted.
Also I usually recommend using tracer for rabbitmq as it logs all the messages coming in the server which makes it easy to trace.