Develop Reference

Azure Service Bus - Receive Messages with OnMessage() Method

Following the MS documentation it was not difficult to receive message(s) from subscription. However, if I'd like my application to receive a message every time new message is posted - a constant polling. Hence the OnMessage() method of the SubscriptionClient class.
MS documentation says: "...When calling OnMessage, the client starts an internal message pump that constantly polls the queue or subscription. This message pump consists of an infinite loop that issues a Receive() call. If the call times out, it issues the next Receive() call. ..."
But when the application is running, the moment OnMessage() method is called only latest message(s) is received. When new messages are posted the constant polling does not seem to be working. After trying many different approaches the only way I could make this work and have the application react the moment new message is received is to place the code into a separate task with infinite loop. This seems totally wrong on so many levels! (see code below).
Can anyone help me to correct my code or post a working sample to accomplish the same functionality without the loop? Thank you!
public void ReceiveMessageFromSubscription(string topicName, string subscriptionFilter)
{
var newMessage = new MessageQueue();
int i = 0;
Task listener = Task.Factory.StartNew(() =>
{
while (true)
{
SubscriptionClient Client = SubscriptionClient.CreateFromConnectionString(connectionString, topicName, subscriptionFilter);
Dictionary<string, string> retrievedMessage = new Dictionary<string, string>();
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
options.AutoRenewTimeout = TimeSpan.FromMinutes(1);
Client.OnMessage((message) =>
{
try
{
retrievedMessage.Add("messageGuid", message.Properties["MessageGuid"].ToString());
retrievedMessage.Add("instanceId", message.Properties["InstanceId"].ToString());
retrievedMessage.Add("pId", message.Properties["ProcessId"].ToString());
retrievedMessage.Add("processKey", message.Properties["ProcessKey"].ToString());
retrievedMessage.Add("message", message.Properties["Message"].ToString());
newMessage.AnnounceNewMessage(retrievedMessage); // event ->
message.Complete(); // Remove message from subscription.
}
catch (Exception ex)
{
string exmes = ex.Message;
message.Abandon();
}
}, options);
retrievedMessage.Clear();
i++;
Thread.Sleep(3000);
}
});
}

Your code has a few issues to iron out -
It falls through and I assume your application then exits - or at
least the thread that is listening for the messages terminates.
Your while loop keeps repeating the code to hook up the message handler,
you only need to do this once.
You need a way to keep the call stack alive and prevent your app from garbage collecting your object.
The below should see you on your way to success. Good luck.
ManualResetEvent CompletedResetEvent = new ManualResetEvent(false);
SubscriptionClient Client;
public void ReceiveMessagesFromSubscription(string topicName, string subscriptionFilter, string connectionString)
{
Task listener = Task.Factory.StartNew(() =>
{
// You only need to set up the below once.
Client = SubscriptionClient.CreateFromConnectionString(connectionString, topicName, subscriptionFilter);
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
options.AutoRenewTimeout = TimeSpan.FromMinutes(1);
options.ExceptionReceived += LogErrors;
Client.OnMessage((message) =>
{
try
{
Trace.WriteLine("Got the message with ID {0}", message.MessageId);
message.Complete(); // Remove message from subscription.
}
catch (Exception ex)
{
Trace.WriteLine("Exception occurred receiving a message: {0}" + ex.ToString());
message.Abandon(); // Failed. Leave the message for retry or max deliveries is exceeded and it dead letters.
}
}, options);
CompletedResetEvent.WaitOne();
});
}
/// <summary>
/// Added in rudimentary exception handling .
/// </summary>
/// <param name="sender">The sender.</param>
/// <param name="ex">The <see cref="ExceptionReceivedEventArgs"/> instance containing the event data.</param>
private void LogErrors(object sender, ExceptionReceivedEventArgs ex)
{
Trace.WriteLine("Exception occurred in OnMessage: {0}" + ex.ToString());
}
/// <summary>
/// Call this to stop the messages arriving from subscription.
/// </summary>
public void StopMessagesFromSubscription()
{
Client.Close(); // Close the message pump down gracefully
CompletedResetEvent.Set(); // Let the execution of the listener complete and terminate gracefully
}
Alternatively you can chunk the message off in a more traditional fashion yourself using ReceiveBatch:
var messages = await queueClient.ReceiveBatchAsync(10, TimeSpan.FromSeconds(30),
cancellationToken);

EasyNetQ with Windows Service Exception

I have been running a simple Windows Service, with a EasyNetQ,
but when i try to publish a message I have been receiving an Exception.
Exception:
Pubisher confirms timed out after 10 seconds waiting for ACK or NACK from sequence
Here are my publish tests:
try {
var queue = _bus.Advanced.QueueDeclare("api-request-history");
_bus.Send(queue.Name, message);
} catch (Exception e) {
Logger.FatalException(string.Format("Message were not queued in queue: {0}", queue.Name), e);
OnErrorOccurred(e);
}
Here is my Subscription tests:
public override void Subscribe(Action<IReceiveRegistration> execution) {
var queue = _bus.Advanced.QueueDeclare("api-request-history");
try {
while (_bus.IsConnected) {
_bus.Receive(queue.Name, execution);
}
} catch (Exception e) {
Logger.ErrorException("Error occurred", e);
} finally {
if (_bus != null) {
_bus.Dispose();
}
}
}
Here is the Invokation of Subscribe:
_queueConsumer.Subscribe(x => x.Add<ApiRequestHistory>(message => {
Logger.Info("Initializing subscribtion sequence!");
Logger.Info("Message data is being saved...");
Execute();
Logger.Info("Message data saving is completed!");
Thread.Sleep(100);
wait.Set();
}));
wait.Wait();

I think you are subscribing the wrong way. You shouldn't be calling _bus.Receive(queueName) more than once for the same queue (from EasyNETQ docs):
Note: You probably do not want to call bus.Receive more than once for the same queue. This will create a new consumer on the queue and RabbitMQ will round-robin between them. If you are consuming different types on different Receive calls (and thus different consumers), some of your messages will end up on the error queue because EasyNetQ will not find a handler for your message type associated with the consumer on which it is consumed.
Just do it once and your consumers (per type) will be registered.
This is specially true as it seems you are using publisher confirms. I don't know your scenario fully well, but usually that is not needed unless you have a transactional RPC call model.
Check the docs on publisher confirms.

WCF CommunicationException When Subscribed Clients Abort Abnormally

In a WCF publish/subscribe setup, I currently have an Unsubscribe() method in place to gracefully disconnect clients from the WCF host when the client is closed or needs to stop listening; however, this does not handle cases in which the client aborts forcefully or abnormally, such as the computer itself losing power. If a client application dies in such a way, then its channel remains and the following error is received at the publisher the next time it tries to send out messages:
ExceptionDetail> was caught
The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it has been Aborted.
Clients subscribe anonymously, and the publisher follows a multicasting structure (any subscribed clients/channels should receive the message). Although I am able to catch the exception, I do not know how to single out the faulty channel from this point in the code in order to dispose of it and allow other clients to continue receiving messages. My publishing code looks similar to the following:
public static void Publish(DateTime sendTimeStamp, DataTable sendTable)
{
InstanceContext context = new InstanceContext(null, new PublishStatus());
MessagesClient publishingClient = new MessagesClient(context);
try {
publishingClient.PublishMessage(sendTimeStamp, sendTable);
if (publishingClient.State != CommunicationState.Faulted)
publishingClient.Close();
else
publishingClient.Abort();
}
catch (CommunicationException ex)
{
// This is where the error is caught
}
catch (TimeoutException ex)
{
publishingClient.Abort();
}
catch (Exception ex)
{
publishingClient.Abort();
throw ex;
}
}
Is it possible to isolate the faulty channel from this point (at which the exception first picks up on the issue) and dispose of it so that the publishing service itself can continue to send messages?

After some trial and error as well as exception research, an additional try-catch block in my WCF host was able to unsubscribe incorrectly aborted clients and keep the error from coming back to the publishing service. Posting a simple version here in case someone else stumbles on the same type of problem:
public static event MessageEventHandler MessageEvent;
public delegate void MessageEventHandler(object sender, ServiceEventArgs e);
IClientContract callback = null;
MessageEventHandler messageHandler = null;
public void Subscribe()
{
callback = OperationContext.Current.GetCallbackChannel<IClientContract>();
messageHandler = new MessageEventHandler(Publish_NewMessageEvent);
MessageEvent += messageHandler;
}
public void Unsubscribe()
{
MessageEvent -= messageHandler;
}
public void PublishMessage(DateTime timeStamp, DataTable table)
{
ServiceEventArgs se = new ServiceEventArgs();
se.timeStamp = timeStamp;
se.table = table;
MessageEvent(this, se);
}
public void Publish_NewMessageEvent(object sender, ServiceEventArgs e)
{
try
{
// This callback was causing the error, as the client would no longer exist but the channel would still be open and trying to receive the message
callback.ReceiveMessage(e.timeStamp, e.table);
}
catch
{
// Unsubscribe the dead client.
Unsubscribe();
}
}

What is better approach to listen in MultiThreading Service?

I am relatively new both to MSMQ and Threading in .NET. I have to create a service which listen in different threads, via TCP and SNMP, several network Devices and all this stuff run in dedicated threads, but here also is required to listen on MSMQ Queue from another applications.
I am analyzing another similar projects and there is used next logic:
private void MSMQRetrievalProc()
{
try
{
Message mes;
WaitHandle[] handles = new WaitHandle[1] { exitEvent };
while (!exitEvent.WaitOne(0, false))
{
try
{
mes = MyQueue.Receive(new TimeSpan(0, 0, 1));
HandleMessage(mes);
}
catch (MessageQueueException)
{
}
}
}
catch (Exception Ex)
{
//Handle Ex
}
}
MSMQRetrievalThread = new Thread(MSMQRetrievalProc);
MSMQRetrievalThread.Start();
But in another service (message dispatcher) I used asynchronous messages' reading based on MSDN Example:
public RootClass() //constructor of Main Class
{
MyQ = CreateQ(#".\Private$\MyQ"); //Get or create MSMQ Queue
// Add an event handler for the ReceiveCompleted event.
MyQ.ReceiveCompleted += new
ReceiveCompletedEventHandler(MsgReceiveCompleted);
// Begin the asynchronous receive operation.
MyQ.BeginReceive();
}
private void MsgReceiveCompleted(Object source, ReceiveCompletedEventArgs asyncResult)
{
try
{
// Connect to the queue.
MessageQueue mq = (MessageQueue)source;
// End the asynchronous Receive operation.
Message m = mq.EndReceive(asyncResult.AsyncResult);
// Process received message
// Restart the asynchronous Receive operation.
mq.BeginReceive();
}
catch (MessageQueueException Ex)
{
// Handle sources of MessageQueueException.
}
return;
}
Does asynchronous handling suppose that every message will be handled in other than main thread?
Could and need this (2nd) approach be put in separate thread?
Please advice better approach or some simple alternatives.
Messages arrival in Queue doesn't have some rule-defined behavior. It may be that for long time no nay message will arrive or in one second there my arrive many (up to 10 or even more) messages. Based on actions defined in some message it will need to delete/change some objects having running threads.

I highly recommend using WCF for MSMQ.
http://msdn.microsoft.com/en-us/library/ms789048.aspx
This allows you to both asynchronous handle the incoming calls using the WCF threading model which allows for throttling, capping, retries, etc...

Invalid Operation Exception In Calling Async Method from inside a Parallel.ForEach Loop

I have inherited a windows service that processes a large number of e-mails in a queue. Sounds simple, Grab queue, send e-mail, if SmtpClient.SendAsync does not return an error from the call back then flag the e-mail in the DB as being sent.. I am using a Semaphore to waitone on the thread so multiple calls can be made to the Async Send method of the SMTP Client. This is the only way I can get the status and per Microsoft docs it has to finish the operation before another call can be made async. So now for the fun part. I decided to use a Parallel.ForEach to get he queue like so. This method is called in the Windows Service OnStart. Please note I have tried calling this method on a separate Thread and get the same results.
I am thinking that either A, I am missing something obvious, due to my lack of knowledge on threading, or something is flat bugged. Most likely A.
private static void ProcessEmailQueue()
{
List<EmailQueue> emailQueue =
_repository.Select<EmailQueue>().Where(x => x.EmailStatuses.EmailStatus == "Pending").ToList();
Parallel.ForEach(emailQueue, message =>
{
_smtpMail.FromAddress = message.FromAddress;
_smtpMail.ToAddress = message.ToAddress;
_smtpMail.Subject = message.Subject;
_smtpMail.SendAsHtml = message.IsHtml > 0;
_smtpMail.MessageBody = message.MessageBody;
_smtpMail.UserToken = message.EmailQueueID;
bool sendStatus = _smtpMail.SendMessage();
// THIS BLOWS UP with InvalidOperation Exception
});
}
Here is the SMTP Method being called from withing the loop.
public bool SendMessage()
{
mailSendSemaphore = new Semaphore(0, 10); // This is defined as private static Semaphore mailSendSemaphore;
try
{
var fromAddress = new MailAddress(FromAddress);
var toAddress = new MailAddress(ToAddress);
using (var mailMessage = new MailMessage(fromAddress, toAddress))
{
mailMessage.Subject = Subject;
mailMessage.IsBodyHtml = SendAsHtml;
mailMessage.Body = MessageBody;
Envelope = mailMessage;
smtp.SendCompleted += smtp_SendCompleted;
smtp.SendAsync(mailMessage, UserToken);
mailSendSemaphore.WaitOne();
return _mailSent;
}
}
catch (Exception exception)
{
_logger.Error(exception);
return _mailSent;
}
}
CALLBACK For Smtp Send
private void smtp_SendCompleted(object sender, AsyncCompletedEventArgs e)
{
if (e.Cancelled)
{
}
if (e.Error != null)
{
}
else
{
_mailSent = true;
}
mailSendSemaphore.Release(2);
}
Here is the Exception, took a few to get it for some odd reason.
System.InvalidOperationException was unhandled by user code
Message=An asynchronous call is already in progress. It must be completed or canceled before you can call this method.
Source=System
StackTrace:
at System.Net.Mail.SmtpClient.SendAsync(MailMessage message, Object userToken)
at DFW.Infrastructure.Communications.SmtpMail.SendMessage() in SmtpMail.cs:line 71
at EmaiProcessorService.EmailQueueService.b_0(EmailQueue message) in Service1.cs:line 57
at System.Threading.Tasks.Parallel.<>c_DisplayClass2d2.<ForEachWorker>b__23(Int32 i)
at System.Threading.Tasks.Parallel.<>c__DisplayClassf1.b__c()
InnerException:
Seems my waitone is getting obliterated by System.Threading.Tasks.Parallel

Okay, now that we've got the error text, it seems fairly clear:
Message=An asynchronous call is already in progress. It must be completed or canceled before you can call this method.
This concurs with the documentation:
Two simple options:
Create a fixed number of clients, and a queue of messages to send. Make each client take a message from the queue each time it finishes, until the queue is empty. BlockingCollection<T> is good for this.
Create a new SmtpClient per message. This could cause you to effectively launch a DOS attack on your SMTP server, which isn't ideal.
To be honest, it's not really clear why you're using SendAsync when you're then just waiting for the message to be sent anyway...

I'm not clear on why you're using a Semaphore here, but you're almost certainly using it incorrectly. You're creating a new semaphore instance for each call to SendMessage. Also, you're calling WaitOne on it once, and then calling Release(2), so eventually you'll have more releases than acquires. That's probably what causes your InvalidOperationException.
It doesn't do you any good to parallelize processing of the email queue, since you can only send one message at a time. And trying to do it asynchronously inside of the Parallel.Foreach is just more needless complication.
You're better off using something like ThreadPool.QueueUserWorkItem, and having a simple loop that sends one message at a time.
List<EmailQueue> emailQueue =
_repository.Select<EmailQueue>().Where(x => x.EmailStatuses.EmailStatus == "Pending").ToList();
ThreadPool.QueueUserWorkItem(ProcessEmailQueue, emailQueue);
void ProcessEmailQueue(object state)
{
List<EmailQueue> emailQueue = (List<EmailQueue>)state;
foreach (var message in EmailQueue)
{
// Format and send message here.
}
}
Alternatively, you can do the same thing with a Task. The point is that you just need a single thread to process the queue sequentially. Since you can't send more than one message at a time, Parallel.ForEach doesn't do you any good.
EDIT:
If you need to do multiple sends at a time, you can probably modify your original code. First, initialize the semaphore at class scope:
private static Semaphore mailSendSemaphore = new Semaphore(10, 10);
Then, in your SendMessage method:
bool SendMessage()
{
// acquire semaphore. This will block until there's a slot available.
mailSendSemaphore.WaitOne();
try
{
// do all your processing here, including sending the message.
// use Send rather than SendAsync
}
finally
{
mailSendSemaphore.Release();
}
}
There's no need to use SendAsync.