Matching failed messages to their exceptions in NServiceBus using MSMQ - c#

When setting up NServiceBus with MSMQ using the standard IServer config options, you define:
an input queue
an error queue.
When your NServiceBus Message Handler fails handling the message for whatever reason, it throws an exception and moves the message to the error queue.
Is the message in the error queue the exact same message that was in the input queue? If so, which I imagine it is, is there any way to know why those messages failed? Is there any metadata attached to them that might contain the original exception that was thrown?
Being able to do this would be especially useful in scenarios when your Handler is set to retry a number of times greater than one. This is because even if fatal errors might be thrown in the Handler, and logged, they're not really Fatal as such until they go in the error queue, because that's when they've actually failed.
Any ideas?
cheers

It's an exact copy of the message that is send to the error q. The message id and source queue is stored in the headers to enable the message to be replayed. In 2.5 there is no good way to get the exception details for the failed message so you have to correlate the message id with entries the logfiles. The fact that NSB reties for you will often cause the same logmessage to be displayed multiple times for a message so make sure to use the last entry.
This is of course not very user friendly and has been fixed in the upcoming 3.0 where you can register Failure managers that let's you hook into NSB to get this info. The default Failure Manager will put the exception details in a headers so that you can easy get at them by looking at the failed message.

Related

Azure WebJob Service Bus requeue message infront of the queue on error

I have a WebJob getting messages from Service bus which is working fine. If the webjob fails to finish or throws exception the message is sent back in the queue which is fine in itself.
I have set MaxDequeueCount to 10 and it is not a problem if it fails in some cases as it will try to process it again. But the problem is that the message seems to be sent to the bottom of the queue and other messages are handled before we get back to the first failed one.
I would like to handle one message at a time because I might have multiple updates on the same entity coming in a row. If the order changes it would update the entity in wrong order.
Is if it is possible to send the message back infront of the queue on error or continue working on the same message until we reach the MaxDequeueCount?
Ideally, you should not be relying on message order.
Given your requirement, you could potentially go with the FIFO semantics of Azure Service Bus Sessions. When a message is handled within a session and message is aborted, it will be handled once again rather than go to the end of the queue. You need to keep in mind the following:
Can only process one message at a time
Requires session to be used
If message is not completed and not aborted, it will be left hanging in the queue and will be picked up when a session is restarted.

Why is message put into a dead-letter queue (MSMQ)?

Preamble:
I have a web application, which sends MSMQ messages (with UseDeadLetterQueue = true) from time to time. I can also see that some of those messages are put into a system dead letter queue. However, I can't see the failure reason directly from "Computer Management" console.
I found the following resource:
https://msdn.microsoft.com/en-us/library/ms789035(v=vs.110).aspx,
which allows to check message status and failure. But it's usable only in WCF scope (as it uses OperationContext).
Basically checking message status and failure reason is what I need, but how can I configure a WCF service to listen to a dead-letter queue and track all the messages put inside, regardless of sender?
Or is there any other (non-WCF) way to get reason of why specific message was put into a dead letter queue?
The linked article you found tells you how to create a WCF service that takes messages off the DLQ. Note that it tells you you must use AddressFilterMode = AddressFilterMode.Any to pull all messages off regardless of sender.
Note: by the time the message ends up on the DLQ, you will not have information about the reason this occurred. Regardless of whether or not you're using WCF that information isn't stored in the queue.
If for example your recipient is a WCF listener using the MSMQ binding any exception thrown will cause the message to fail to get delivered. MSMQ will retry for a while and then eventually give up and put the message in the DLQ. So if you wanted to know the original reason you need to add some tracing in your WCF service to catch and trace thrown exceptions.

Calling Abandon on an Azure Service Bus re-queues the message at the back rather than the front of the queue

I'm using an Azure Service Bus Queue with Session based messaging enabled. To consume from the queue I register an IMessageSessionAsyncHandler and then process the message in the OnMessageAsync method.
This issue I'm seeing is that if I abandon a message for whatever reason, rather than being received again immediately, I receive the next message in the session and only after processing that message, do I receive the first message again (assuming only two messages in the session).
As an example, lets say I have a queue with 2 messages on it, both with the same SessionId. The two messages have sequence numbers of 1 and 2 respectively. I start receiving and get message with sequence 1, as expected. If I then abandon this message using message.Abandon (the reason for abandoning is irrelevant), I immediately get the next message in the session (sequence number 2). Only after handling (or abandoning) this second message, do I get the first message again.
This behaviour I'm seeing isn't what I'd expect from abandoning a message and isn't consistent with other ways of using the queue. I've tested this same example in the following scenarios
without the use of an IMessageSessionAsyncHandler and instead just manually accepting a message session.
without the use of sessions and instead just having two independent messages on the queue.
In both scenarios, I see the expected bahaviour, in that when I abandon a message it is always guaranteed to be the next message received, unless the max delivery count is exceeded and it is dead-lettered.
My question is this: Is the behaviour I'm seeing with the use of an IMessageSessionAsyncHandler expected, or is this a bug in the Service Bus Library? If this is not a bug, can anyone give me an explaination for why this behaves different to the other ways of receiving?
When you Register a session handler on the Queueclient, Prefetch is turned on internally to improve latency and throughput of the receivers. Unfortunately for the IMessageSessionAsyncHandler scenario this behavior cannot be overriden. One option is to abandon the Session itself when you encounter a message in a session which needs to be abandoned, this will ensure that the messages are delivered in order.

Quartz.NET JobListener for failed jobs

In my Quartz.NET based application, I have a JobListener that audits all jobs executed to an audit table. However, I also want a listener to detect whenever any sort of error occurs, so I can catch this and email an email address that there is an error, and maybe find the specific error (i.e. my program moves files around, so an error could be a path doesn't exist).
How do I listen for failed jobs, and also is it possible to detect whether there is a huge number of errors, and in that case only send a few emails rather than an email for every single error?
To detect whenever any sort of error occurs you will want to implement a listener, either a Job listener or a Trigger listener, or probably you'll want both: http://quartznet.sourceforge.net/tutorial/lesson_7.html
You can attach the listeners by calling methods on the IScheduler object you create.
What we do to roll-up multiple emails is use a logging system, specifically NLog. We use a BufferingWrapper around a Mail target so that the error log is sent after some specified number of events has been logged (e.g., 200) or after a specified timeout period has elapsed after the last logged error (e.g., 2 minutes).

NServiceBus exception handling and message retry mechanism

We are planning to use NServiceBus in our application for dispatching messages.
In our case each message has timeToLive property, defining period of time, in which this message should be processed.
For the case if message handling was unsuccessful in first attempt, our plan is to move it to specific retry storage (retry queue) and than retry message (with some timeouts between retries) while it is successfully handled or timeToLive is expired.
In case if timeToLive is expired, we plan to log message content and discard message.
Actually, this retry behaviour is mostly determined by protocol, which we are implementing.
Is there any ways to achieve such a behaviour with NServiceBus? I see, that unsuccessful messages goes to specific error queue. Is it possible to create a separate bus, pointing to error queue?
I would suggest that you have a separate process that monitors the error queue perform retries according to the logic you describes. Take a look at the ReturnToSourceQueue tool that comes with nservicebus for inspiration:
http://nservicebus.svn.sourceforge.net/viewvc/nservicebus/trunk/src/tools/management/Errors/ReturnToSourceQueue/NServiceBus.Tools.Management.Errors.ReturnToSourceQueue/Class1.cs?view=markup
I have a blog post on how to handle failures that might give you some ideas as well:
http://andreasohlund.net/2010/03/15/errorhandling-in-a-message-oriented-world/
Hope this helps!

Categories

Resources