RabbitMQ Missing data in body after dequeue - c#

I'm working with RabbitMQ on someone else's project and having trouble with dequeuing and missing data.
The data is all there as a string when I publish, and it's also on there correctly on the RabbitMQ queue. When I pull the data off, buts of the data is there like the User ID, but the rest of it is gone. I've looked throughout the code and I'm fairly positive that its something going on with RabbitMQ, and its happening when I dequeue. Any help would be greatly appreciated. Thanks.
Here is the code right before the publish.
private bool sendJobToMQ(EncodeJobModel job, string p_correlation_id, string p_request_routing_key)
{
JavaScriptSerializer ser = new JavaScriptSerializer();
StringBuilder sb_job = new StringBuilder();
ser.Serialize(job, sb_job);
string rpc_reply_queue;
ConnectionFactory factory = new ConnectionFactory();
factory.HostName = HOST_NAME;
factory.VirtualHost = VHOST_NAME;
factory.UserName = USERNAME;
factory.Password = PASSWORD;
IConnection rabconn = factory.CreateConnection();
IModel sender_channel = rabconn.CreateModel();
try
{
sender_channel.ExchangeDeclare(EXCHANGE_NAME, ExchangeType.Direct, true, false, null);
}
catch (Exception err)
{
logger.Error("Error Declaring Exchange " + EXCHANGE_NAME + ": " + err.ToString());
return false;
}
try
{
sender_channel.QueueDeclare(REQUEST_QUEUE, true, false, false, null);
}
catch (Exception err)
{
logger.Error("Error QueueDeclare (" + REQUEST_QUEUE + " true, false, false, null): " + err.ToString());
return false;
}
try
{
sender_channel.QueueBind(REQUEST_QUEUE, EXCHANGE_NAME, REQUEST_ROUTING_KEY, null);
}
catch (Exception err)
{
logger.Error("Error QueueBind (" + REQUEST_QUEUE + " -> " + EXCHANGE_NAME + " " + REQUEST_ROUTING_KEY + ", null): " + err.ToString());
return false;
}
//rpc_reply_queue = sender_channel.QueueDeclare("rq_" + job.encodejob_id.ToString(), false, false, true, null);
//////bind the rpc reply queue to the exchange via a routing key (I appended _routingkey to signify this)
//sender_channel.QueueBind(rpc_reply_queue, EXCHANGE_NAME, rpc_reply_queue + "_routingkey");
//// Not sure what the props object is for yet but you can try to pass null in the mean time - Steve "Apeshit" Han
BasicProperties props = new BasicProperties();
props.CorrelationId = p_correlation_id;
//props.ReplyTo = rpc_reply_queue;
try
{
sender_channel.BasicPublish(EXCHANGE_NAME, REQUEST_ROUTING_KEY, props, Encoding.UTF8.GetBytes(sb_job.ToString()));
}
And the code for the dequeue.
QueueingBasicConsumer consumer = new QueueingBasicConsumer(p_channel);
string consumerTag = p_channel.BasicConsume(p_queue, false, consumer);
if (_is_console && Environment.UserInteractive)
Console.WriteLine("Listening...");
while (m_Listen)
{
try
{
//get the properties of the message, including the ReplyTo queue, to which we can append '_routingkey' (designated by me), to reply with messages
BasicDeliverEventArgs e;
Object message;
if (!consumer.Queue.Dequeue(4000, out message)) {
// we do not wait to indefinitely block on waiting for the queue
// if nothing in queue continue loop iteration and wait again
continue;
}
// cast as necessary back to BasicDeliverEventArgs
e = (BasicDeliverEventArgs)message;
IBasicProperties props = e.BasicProperties;
//get the Correlation ID sent by the client to track the job
string client_correlation_id = props.CorrelationId;
// I left out the reply_to field in the wizard, it can be set back in ApiEncodeServiceDefault - Steve "Smurfing Smurf" Han
//string reply_to = props.ReplyTo;
//get the body of the request
byte[] body = e.Body;
string body_result = Encoding.UTF8.GetString(body);
bool redelivered = e.Redelivered;
The e.Body string is missing data.

why continue if you don't have any message
it is better to block until you receive a message otherwise the process is not interesting (work with no data?)
try like this
QueueingBasicConsumer consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(queueName, null, consumer);
while (m_Listen) {
try {
RabbitMQ.Client.Events.BasicDeliverEventArgs e =
(RabbitMQ.Client.Events.BasicDeliverEventArgs)
consumer.Queue.Dequeue();
IBasicProperties props = e.BasicProperties;
byte[] body = e.Body;
// ... process the message
channel.BasicAck(e.DeliveryTag, false);
} catch (OperationInterruptedException ex) {
// The consumer was removed, either through
// channel or connection closure, or through the
// action of IModel.BasicCancel().
break;
}
}

Related

Identify connection failure/internet issue while listening to RabbitMQ queue C#

I'm developing WPF application to listen to queue in RabbitMQ and do some work in the method CaptureDataFromMessage() when consume message from RMQ queue.
To keep listening RabbitMQ queue, the application using the while loop.
When application starts, I check rabbitMQ connection[CheckRabbitMQStatus()] and if the connection opens then the application will start to listen to the queue, else error message will display in txtOutput(display notification area).
When there is no internet or exception occur while application open & listening to the RabbitMQ queue then I need to provide informative messages to txtOutput. I was trying to check using a connection.IsOpen inside the while loop, even though there is no connection, but it provides true.
So my effort wasn't a success, [but the application working fine & connects itself when an internet connection is back and consuming messages]
1. Please suggest to me how to identify connection lost & connection establish again event to display in txtOutput while application running.
2. Is it OK to use thread inside message receiver method.
This is my code.
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
if (CheckRabbitMQStatus())
{
Thread _threadHandleMessage;
_threadHandleMessage = new Thread(() =>
{
QueueListener();
});
_threadHandleMessage.Start();
}
}
private ConnectionFactory CreateConnectionFactory()
{
#region rabbitMQ Configurtions
string HOSTNAME = "RabbitMQServer";
string USERNAME = "RabbitMQUsername";
string RabbitMQPassword = "RabbitMQPassword";
string VHOST = "RabbitMQVHost";
#endregion
return new ConnectionFactory() { HostName = HOSTNAME, UserName = USERNAME, Password = DecryptRabbitMQPassword, VirtualHost = VHOST };
}
private void QueueListener()
{
try
{
string QueueName = "RabbitMQQueueName";
var factory = CreateConnectionFactory();
using (var connection = factory.CreateConnection())
{
if (connection.IsOpen)
{
this.Dispatcher.Invoke(() =>
{
txtOutput.Text = (string.IsNullOrEmpty(txtOutput.Text) ? "" : txtOutput.Text + "\n") + "[" + DateTime.Now.ToString("MM/dd/yyyy HH:mm:ss") + "]" + " Application has started.";
});
}
else
{
this.Dispatcher.Invoke(() =>
{
txtOutput.Text = (string.IsNullOrEmpty(txtOutput.Text) ? "" : txtOutput.Text + "\n") + "[" + DateTime.Now.ToString("MM/dd/yyyy HH:mm:ss") + "]" + " Application has stopped due to connection failure.";
});
}
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: QueueName,
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var consumer = new EventingBasicConsumer(channel);
channel.BasicQos(prefetchSize: 0, prefetchCount: 1, global: false);
channel.BasicConsume(queue: QueueName,
autoAck: false,
consumer: consumer);
HandleMessage handleMessage = new HandleMessage(this.Dispatcher, txtOutput);
Thread _threadPrintMessage;
consumer.Received += (model, ea) =>
{
_threadPrintMessage = new Thread(() =>
{
handleMessage.CaptureDataFromMessage(ea.Body);
});
_threadPrintMessage.Start();
channel.BasicAck(deliveryTag: ea.DeliveryTag, multiple: false);
};
while (true)
{
if (connection.IsOpen)
{
// I tried to capture netwok availability from here & else part
// But even there is no internet connection in the computer or queue deleted after connection open, still connection states as OPEN
}
else
{
//
}
}
}
}
}
catch (Exception ex)
{
log.Error(ex);
this.Dispatcher.Invoke(() =>
{
txtOutput.Text = (string.IsNullOrEmpty(txtOutput.Text) ? "" : txtOutput.Text + "\n") + "[" + DateTime.Now.ToString("MM/dd/yyyy HH:mm:ss") + "]" + " Error has occured while processing.";
});
}
}
private bool CheckRabbitMQStatus()
{
var factory = CreateConnectionFactory();
IConnection conn = null;
try
{
conn = factory.CreateConnection();
conn.Close();
conn.Dispose();
return true;
}
catch (Exception ex)
{
log.Error(ex);
string mesage ="";
if (ex.Message == "None of the specified endpoints were reachable")
{
mesage = " Application has not started due to connection failure.";
}
else {
mesage = " Application has not started due to a failure.";
}
this.Dispatcher.Invoke(() =>
{
txtOutput.Text = (string.IsNullOrEmpty(txtOutput.Text) ? "" : txtOutput.Text + "\n") + "[" + DateTime.Now.ToString("MM/dd/yyyy HH:mm:ss") + "]" + mesage;
});
return false;
}
}
}
}

C# Console application pause working until right click on Console

I'm building a c# console application which read messages from MSMQ(Microsoft Message Queuing), I expected it to run forever but after a day running it stop reading message from MSMQ until I right click on its Console, my problem seems to be alittle bit similar to this: "Console Application "pausing" c#". Bellow is the function that I'm using:
private static void downloadHandler(object args)
{
while (true)
{
while (CURRENT_ACTIVE_THREAD < MAX_THREAD)
{
DownloadController.WriteLog("Current active Thread = " + CURRENT_ACTIVE_THREAD);
Console.WriteLine("Current active Thread = " + CURRENT_ACTIVE_THREAD);
Thread.Sleep(1000);
MessageQueue messageQueue;
if (MessageQueue.Exists(messageQueuePath))
{
messageQueue = new MessageQueue(messageQueuePath);
Message requestMessage = messageQueue.Receive();
try
{
requestMessage.Formatter = new BinaryMessageFormatter();
string msg = requestMessage.Body.ToString();
if (!string.IsNullOrEmpty(msg))
{
DownloadController.WriteLog("received message with message = " + msg);
CURRENT_ACTIVE_THREAD += 1;
RequestDownload request = new RequestDownload();
request = JsonConvert.DeserializeObject<RequestDownload>(msg);
DownloadController.WriteLog("received message with contentId = " + request.contentId + "from message queue | title= " + request.contentTitle + " | url = " + request.baseURL);
DownloadController downloader = new DownloadController();
Thread t = new Thread(new ParameterizedThreadStart(downloader.findURLandDownload));
object[] objs = new object[2];
objs[0] = request;
objs[1] = "1";
t.Start(objs);
Console.WriteLine("received message with contentId = " + request.contentId);
}
}
catch (Exception ex)
{
CURRENT_ACTIVE_THREAD -= 1;
Console.WriteLine("Exception: " + ex.Message);
DownloadController.WriteLog("There is exception while trying to read message from message queue, Exception = " + ex.Message);
}
}
}
}
}
So,could anyone please tell me what the problem is? Why this happening?
It might be you're while loop. I had while loops freeze or break my applications before. Might i suggest using timers instead ? I have some links you could use for reference:
c# wait for a while without blocking
https://msdn.microsoft.com/en-us/library/system.timers.timer(v=vs.110).aspx
I do hope this fixes the problem you're having !
greatings,
Ramon.

Why does a method that calls Ping.Send() on an invalid URL die with an unhandled exception?

The following method calls Ping.Send(). When I pass an invalid URL, Send() dies and an unhandled exception happens. What is the cause of this?
private void ping()
{
comboBox3.Visible = false;
listBox2.Items.Clear();
// check the url if it is null
if (string.IsNullOrEmpty(textBox1.Text) || textBox1.Text == "")
{
listBox2.Items.Add("Please use valid IP or web address!!");
comboBox3.Visible = false;
coloring_red_tab4();
}
else
{
// do the ping
coloring_green_tab4();
for (int i = 0; i < numericUpDown1.Value; i++)
{
string s;
s = textBox1.Text;
string data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
byte[] buffer = Encoding.ASCII.GetBytes(data);
int timeout = 120;
Ping p = new Ping();
PingOptions options = new PingOptions();
options.DontFragment = true;
//pingexception was unhalded (if the url wrong here is the error)
PingReply r = p.Send(s, timeout, buffer, options);
// if it's true url
if (r.Status == IPStatus.Success)
{
listBox2.Items.Add("Ping to " + s.ToString() + "[" + r.Address.ToString() + "]" + " (Successful) "
+ "Bytes =" + r.Buffer.Length + " TTL=" + r.Options.Ttl + " Response delay = " + r.RoundtripTime.ToString() + " ms " + "\n");
label91.Text = r.Address.ToString();
}
else
{
// just to know the ip for the website if they block the icmp protocol
listBox2.Items.Add(r.Status);
IPAddress[] ips;
ips = Dns.GetHostAddresses(textBox1.Text);
foreach (IPAddress ip in ips)
{
label91.Text = ip.ToString();
}
}
}
}
}
The exception is unhandled because you do not handle it. Whenever you call a .Net library method, you need to check its documentation to see what exceptions it throws, and decide which, if any, you want to handle at that level of code. Here is the relevant portion of the documentation for Ping.Send(), which I am including as an image so you will be able to recognize these sections going forward:
Notice that the documentation states that a PingException can occur if
An exception was thrown while sending or receiving the ICMP messages. See the inner exception for the exact exception that was thrown.
Thus it's clear from the documentation that many errors from Ping() will be reported as thrown exceptions rather than reported by setting PingReply.Status != IPStatus.Success. So you need to modify your code to be something like the following:
public static bool TryPing(string hostNameOrAddress, out string pingStatusMessage, out string pingAddressMessage)
{
if (String.IsNullOrWhiteSpace(hostNameOrAddress))
{
pingStatusMessage = "Missing host name";
pingAddressMessage = "";
return false;
}
var data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
var buffer = Encoding.ASCII.GetBytes(data);
var timeout = 120;
using (var p = new Ping())
{
var options = new PingOptions();
options.DontFragment = true;
try
{
var r = p.Send(hostNameOrAddress, timeout, buffer, options);
if (r.Status == IPStatus.Success)
{
pingStatusMessage = "Ping to " + hostNameOrAddress.ToString() + "[" + r.Address.ToString() + "]" + " (Successful) "
+ "Bytes =" + r.Buffer.Length + " TTL=" + r.Options.Ttl + " Response delay = " + r.RoundtripTime.ToString() + " ms " + "\n";
pingAddressMessage = r.Address.ToString();
return true;
}
else
{
// just to know the ip for the website if they block the icmp protocol
pingStatusMessage = r.Status.ToString();
var ips = Dns.GetHostAddresses(hostNameOrAddress);
pingAddressMessage = String.Join(",", ips.Select(ip => ip.ToString()));
return false;
}
}
catch (PingException ex)
{
pingStatusMessage = string.Format("Error pinging {0}: {1}", hostNameOrAddress, (ex.InnerException ?? ex).Message);
pingAddressMessage = hostNameOrAddress;
return false;
}
}
}
Here I have extracted a utility method from the user interface code and also properly disposed of the Ping instance after it is no longer needed.
Then
TryPing(#"www.google.com", out pingStatusMessage, out pingAddressMessage);
Gives
Ping to www.google.com[146.115.8.83] (Successful) Bytes =32 TTL=62 Response delay = 8 ms
While
TryPing(#"www.kdjf98rglkfgjldkfjgdl;fge8org.com", out pingStatusMessage, out pingAddressMessage);
Gives
Error pinging www.kdjf98rglkfgjldkfjgdl;fge8org.com: No such host is known

Rabbitmq message is not appearing in the queue from c#

I am executing below code successfully without any errors but when I am executing list_queue command it gives me no queue. Dont know where the problem is.
The below code is not giving error at all. My machine is windows 7 64x, installed AMPQ runtime 64bit and using 32x RabbitMq server v.2.8.1 .
private void createExchange(string message)
{
var queName = Convert.ToString(ConfigurationManager.AppSettings["clientQueue"]);
var exchangeName = Convert.ToString(ConfigurationManager.AppSettings["clientExchange"]);
var hostName = Convert.ToString(ConfigurationManager.AppSettings["host"]);
_logger.Info("entered in create exchange");
_logger.Info("queName: " + queName);
_logger.Info("exchangeName: " + exchangeName);
_logger.Info("hostName: " + hostName);
try
{
var connectionFactory = new ConnectionFactory();
connectionFactory.HostName = "localhost";
connectionFactory.UserName = "user1";
connectionFactory.Password = "userpassword";
using (IConnection connection =
connectionFactory.CreateConnection())
{
_logger.Info("Conncection created");
using (IModel model = connection.CreateModel())
{
_logger.Info("Model is created");
model.ExchangeDeclare(exchangeName, ExchangeType.Fanout, true);
model.QueueDeclare(queName, true, true, false, null);
model.QueueBind(queName, exchangeName, "", new Dictionary<string, object>());
IBasicProperties basicProperties = model.CreateBasicProperties();
model.BasicPublish(exchangeName, "", false, false,
basicProperties, Encoding.UTF8.GetBytes(message));
_logger.Info("message: " + message);
_logger.Info("message published");
}
}
}
catch(Exception ex)
{
_logger.Info("Error in create exchange");
_logger.Info("InnerException:" + ex.InnerException);
_logger.Info("Message:" + ex.Message);
_logger.Info("StackTrace:" + ex.StackTrace);
}
}
It appears as though you are declaring an 'exclusive' queue. Exclusive queues are exclusive to the connection and as such cannot be operated upon (including being listed) from another connection.
Try
Model.QueueDeclare(queName, true, **false**, false, null)
If you want to be able to list it or do anything else with if from another connection.

ASP.net Caching

I have implemented asp.net caching. But I am getting weird results
Unlike most caching where you are trying to avoid the amount of hits to the DB. I am trying to avoid any hits to the DB by the user. This is b/c the amount of time the fist page takes to load. It is basically a dashboard with a lot of charts and long running queries
I tried several techniques
1) Have the cache time very long and have a schedule process expire and get new cache.
2) and on RemoveCallback
In the second option I have all the cache go through a static class I created. The purpose is as it expires to refresh the data. Here is what I am calling.
Cache.Insert(dbCacheString, dtNetwork, null, DateTime.Now.AddHours(2),
System.Web.Caching.Cache.NoSlidingExpiration, CacheItemPriority.High,
new CacheItemRemovedCallback(CacheManager.CacheRemovedCallback));
public static class CacheManager
{
private static Hashtable times = new Hashtable();
private static bool isRefreshingCache = false;
public static void CacheRemovedCallback(String key, object value,
CacheItemRemovedReason removedReason)
{
RefreshCache(key);
}
public static void StartCache()
{
string lcUrl = "http://localhost/ratingspage/";
// *** Establish the request
try
{
WebClient client = new WebClient();
client.Credentials = new NetworkCredential("xxx", "xxx",
"xxx");
byte[] myDataBuffer = client.DownloadData(lcUrl);
}
catch (Exception ex)
{
ErrHandler.WriteError(ex.Message + "\n" +
ex.StackTrace.ToString());
LogUtil.LogDebugMessages(ex.Message + ":" +
ex.StackTrace.ToString());
}
}
public static void RefreshCache(string key)
{
string controlname = "";
if ( key.ToLower().StartsWith("control:") )
{
string[] tmp = key.Split(':');
if (tmp.Length > 1)
controlname = tmp[1];
else
return;
}
else
return;
string lcUrl = "http://localhost/ratingspage/Admin/" + "
"LoadControl.aspx?CachingSpider=true&Control=" + controlname;
string lcHtml = isRefreshingCache.ToString();
// *** Establish the request
if (!isRefreshingCache)
{
isRefreshingCache = true;
lcHtml = isRefreshingCache.ToString();
try
{
WebClient client = new WebClient();
client.Credentials = new NetworkCredential("xxx",
"xxx", "xxx");
byte[] myDataBuffer = client.DownloadData(lcUrl);
lcHtml = Encoding.ASCII.GetString(myDataBuffer);
}
catch (Exception ex)
{
lcHtml = ex.Message;
isRefreshingCache = false;
ErrHandler.WriteError(ex.Message + "\n" +
ex.StackTrace.ToString());
LogUtil.LogDebugMessages(ex.Message + ":" +
ex.StackTrace.ToString());
}
isRefreshingCache = false;
}
MailMessage mail = new MailMessage(
new MailAddress("jgiblin#univision.net"),
new MailAddress("jgiblin#univision.net"));
mail.Subject = "Cache Expire: " + key;
mail.Body = string.Format("The Key {0} has expired at {1}",
key, DateTime.Now.ToShortDateString() + " " +
DateTime.Now.ToShortTimeString()) + "\nRefreshing Cache: " +
lcHtml;
SmtpClient smtp = new SmtpClient("mercury.utg.uvn.net");
mail.IsBodyHtml = false;
try
{
smtp.Send(mail);
}
catch (Exception ex)
{
ErrHandler.WriteError(ex.Message + "\n" +
ex.StackTrace.ToString());
LogUtil.LogDebugMessages(ex.Message + ":" +
ex.StackTrace.ToString());
}
}
}
for some reason, when I go to the page. Someone times the data is cached and sometimes it is not. Is there something wrong here
I tried app fabric but since the server does not have iis 7 I am not able to use that
Looks to me like you may have a race condition in your RefreshCache call. Check out this great answer for suggestions on how to handle synchronization:
C# version of java's synchronized keyword?
If I were you, I'd simplify my code. You only really need:
when the application starts up, load the cache
when the cache expires, reload it
if a user tries to access the cached object and misses, sleep their thread for a second and check again
The amount of data you can get into the cache may be the issue.
If the cache is full then the data will obviously not be cached.
You can check your memory allocation using the process monitor, have a look in "Cache Total Entries."

Categories

Resources