I'm using RabbitMQ in C# with the EasyNetQ library. I'm using a pub/sub pattern here. I still have a few issues that I hope anyone can help me with:
When there's an error while consuming a message, it's automatically moved to an error queue. How can I implement retries (so that it's placed back on the originating queue, and when it fails to process X times, it's moved to a dead letter queue)?
As far as I can see there's always 1 error queue that's used to dump messages from all other queues. How can I have 1 error queue per type, so that each queue has its own associated error queue?
How can I easily retry messages that are in an error queue? I tried Hosepipe, but it justs republishes the messages to the error queue instead of the originating queue. I don't really like this option either because I don't want to be fiddling around in a console. Preferably I'd just program against the error queue.
Anyone?
The problem you are running into with EasyNetQ/RabbitMQ is that it's much more "raw" when compared to other messaging services like SQS or Azure Service Bus/Queues, but I'll do my best to point you in the right direction.
Question 1.
This will be on you to do. The simplest way is that you can No-Ack a message in RabbitMQ/EasyNetQ, and it will be placed at the head of the queue for you to retry. This is not really advisable because it will be retried almost immediately (With no time delay), and will also block other messages from being processed (If you have a single subscriber with a prefetch count of 1).
I've seen other implementations of using a "MessageEnvelope". So a wrapper class that when a message fails, you increment a retry variable on the MessageEnvelope and redeliver the message back onto the queue. YOU would have to do this and write the wrapping code around your message handlers, it would not be a function of EasyNetQ.
Using the above, I've also seen people use envelopes, but allow the message to be dead lettered. Once it's on the dead letter queue, there is another application/worker reading items from the dead letter queue.
All of these approaches above have a small issue in that there isn't really any nice way to have a logarithmic/exponential/any sort of increasing delay in processing the message. You can "hold" the message in code for some time before returning it to the queue, but it's not a nice way around.
Out of all of these options, your own custom application reading the dead letter queue and deciding whether to reroute the message based on an envelope that contains the retry count is probably the best way.
Question 2.
You can specify a dead letter exchange per queue using the advanced API. (https://github.com/EasyNetQ/EasyNetQ/wiki/The-Advanced-API#declaring-queues). However this means you will have to use the advanced API pretty much everywhere as using the simple IBus implementation of subscribe/publish looks for queues that are named based on both the message type and subscriber name. Using a custom declare of queue means you are going to be handling the naming of your queues yourself, which means when you subscribe, you will need to know the name of what you want etc. No more auto subscribing for you!
Question 3
An Error Queue/Dead Letter Queue is just another queue. You can listen to this queue and do what you need to do with it. But there is not really any out of the box solution that sounds like it would fit your needs.
I've implemented exactly what you describe. Here are some tips based on my experience and related to each of your questions.
Q1 (how to retry X times):
For this, you can use IMessage.Body.BasicProperties.Headers. When you consume a message off an error queue, just add a header with a name that you choose. Look for this header on each message that comes into the error queue and increment it. This will give you a running retry count.
It's very important that you have a strategy for what to do when a message exceeds the retry limit of X. You don't want to lose that message. In my case, I write the message to disk at that point. It gives you lots of helpful debugging information to come back to later, because EasyNetQ automatically wraps your originating message with error info. It also has the original message so that you can, if you like, manually (or maybe automated, through some batch re-processing code) requeue the message later in some controlled way.
You can look at the code in the Hosepipe utility to see a good way of doing this. In fact, if you follow the pattern you see there then you can even use Hosepipe later to requeue the messages if you need to.
Q2 (how to create an error queue per originating queue):
You can use the EasyNetQ Advanced Bus to do this cleanly. Use IBus.Advanced.Container.Resolve<IConventions> to get at the conventions interface. Then you can set the conventions for the error queue naming with conventions.ErrorExchangeNamingConvention and conventions.ErrorQueueNamingConvention. In my case I set the convention to be based on the name of the originating queue so that I get a queue/queue_error pair of queues every time I create a queue.
Q3 (how to process messages in the error queues):
You can declare a consumer for the error queue the same way you do any other queue. Again, the AdvancedBus lets you do this cleanly by specifying that the type coming off of the queue is EasyNetQ.SystemMessage.Error. So, IAdvancedBus.Consume<EasyNetQ.SystemMessage.Error>() will get you there. Retrying simply means republishing to the original exchange (paying attention to the retry count you put in the header (see my answer to Q1, above), and information in the Error message that you consumed off the error queue can help you find the target for republishing.
I know this is an old post but - just in case it helps someone else - here is my self-answered question (I needed to ask it because existing help was not enough) that explains how I implemented retrying failed messages on their original queues. The following should answer your question #1 and #3. For #2, you may have to use the Advanced API, which I haven't used (and I think it defeats the purpose of EasyNetQ; one might as well use RabbitMQ client directly). Also consider implementing IConsumerErrorStrategy, though.
1) Since there can be multiple consumers of a message and all may not need to retry a msg, I have a Dictionary<consumerId, RetryInfo> in the body of the message, as EasyNetQ does not (out of the box) support complex types in message headers.
public interface IMessageType
{
int MsgTypeId { get; }
Dictionary<string, TryInfo> MsgTryInfo {get; set;}
}
2) I have implemented a class RetryEnabledErrorMessageSerializer : IErrorMessageSerializer that just updates the TryCount and other information every time it is called by the framework. I attach this custom serializer to the framework on a per-consumer basis via the IoC support provided by EasyNetQ.
public class RetryEnabledErrorMessageSerializer<T> : IErrorMessageSerializer where T : class, IMessageType
{
public string Serialize(byte[] messageBody)
{
string stringifiedMsgBody = Encoding.UTF8.GetString(messageBody);
var objectifiedMsgBody = JObject.Parse(stringifiedMsgBody);
// Add/update RetryInformation into objectifiedMsgBody here
// I have a dictionary that saves <key:consumerId, val: TryInfoObj>
return JsonConvert.SerializeObject(objectifiedMsgBody);
}
}
And in my EasyNetQ wrapper class:
public void SetupMessageBroker(string givenSubscriptionId, bool enableRetry = false)
{
if (enableRetry)
{
_defaultBus = RabbitHutch.CreateBus(currentConnString,
serviceRegister => serviceRegister.Register<IErrorMessageSerializer>(serviceProvider => new RetryEnabledErrorMessageSerializer<IMessageType>(givenSubscriptionId))
);
}
else // EasyNetQ's DefaultErrorMessageSerializer will wrap error messages
{
_defaultBus = RabbitHutch.CreateBus(currentConnString);
}
}
public bool SubscribeAsync<T>(Func<T, Task> eventHandler, string subscriptionId)
{
IMsgHandler<T> currMsgHandler = new MsgHandler<T>(eventHandler, subscriptionId);
// Using the msgHandler allows to add a mediator between EasyNetQ and the actual callback function
// The mediator can transmit the retried msg or choose to ignore it
return _defaultBus.SubscribeAsync<T>(subscriptionId, currMsgHandler.InvokeMsgCallbackFunc).Queue != null;
}
3) Once the message is added to the default error queue, you can have a simple console app/windows service that periodically republishes existing error messages on their original queues. Something like:
var client = new ManagementClient(AppConfig.BaseAddress, AppConfig.RabbitUsername, AppConfig.RabbitPassword);
var vhost = client.GetVhostAsync("/").Result;
var aliveRes = client.IsAliveAsync(vhost).Result;
var errQueue = client.GetQueueAsync(Constants.EasyNetQErrorQueueName, vhost).Result;
var crit = new GetMessagesCriteria(long.MaxValue, Ackmodes.ack_requeue_false);
var errMsgs = client.GetMessagesFromQueueAsync(errQueue, crit).Result;
foreach (var errMsg in errMsgs)
{
var innerMsg = JsonConvert.DeserializeObject<Error>(errMsg.Payload);
var pubInfo = new PublishInfo(innerMsg.RoutingKey, innerMsg.Message);
pubInfo.Properties.Add("type", innerMsg.BasicProperties.Type);
pubInfo.Properties.Add("correlation_id", innerMsg.BasicProperties.CorrelationId);
pubInfo.Properties.Add("delivery_mode", innerMsg.BasicProperties.DeliveryMode);
var pubRes = client.PublishAsync(client.GetExchangeAsync(innerMsg.Exchange, vhost).Result, pubInfo).Result;
}
4) I have a MessageHandler class that contains a callback func. Whenever a message is delivered to the consumer, it goes to the MessageHandler, which decides if the message try is valid and calls the actual callback if so. If try is not valid (maxRetriesExceeded/the consumer does not need to retry anyway), I ignore the message. You can choose to Dead Letter the message in this case.
public interface IMsgHandler<T> where T: class, IMessageType
{
Task InvokeMsgCallbackFunc(T msg);
Func<T, Task> MsgCallbackFunc { get; set; }
bool IsTryValid(T msg, string refSubscriptionId); // Calls callback only
// if Retry is valid
}
Here is the mediator function in MsgHandler that invokes the callback:
public async Task InvokeMsgCallbackFunc(T msg)
{
if (IsTryValid(msg, CurrSubscriptionId))
{
await this.MsgCallbackFunc(msg);
}
else
{
// Do whatever you want
}
}
Here, I have implemented a Nuget package (EasyDeadLetter) for this purpose, which can be easily implemented with the minimum changes in any project.
All you need to do is follow the four steps :
First of all, Decorate your class object with QeueuAttribute
[Queue(“Product.Report”, ExchangeName = “Product.Report”)]
public class ProductReport { }
The second step is to define your dead-letter queue with the same QueueAttribute and also inherit the dead-letter object from the Main object class.
[Queue(“Product.Report.DeadLetter”, ExchangeName =
“Product.Report.DeadLetter”)]
public class ProductReportDeadLetter : ProductReport { }
Now, it’s time to decorate your main queue object with the EasyDeadLetter attribute and set the type of dead-letter queue.
[EasyDeadLetter(DeadLetterType =
typeof(ProductReportDeadLetter))]
[Queue(“Product.Report”, ExchangeName = “Product.Report”)]
public class ProductReport { }
In the final step, you need to register EasyDeadLetterStrategy as the default error handler (IConsumerErrorStrategy).
services.AddSingleton<IBus>
(RabbitHutch.CreateBus(“connectionString”,
serviceRegister =>
{
serviceRegister.Register<IConsumerErrorStrategy,
EasyDeadLetterStrategy>();
}));
That’s all. from now on any failed message will be moved to the related dead-letter queue.
See more detail here :
GitHub Repository
NuGet Package
Related
If the processing of an Azure Service Bus message depends on another resource, e.g. an API or a database service, and this resource is not available, not calling CompleteMessageAsync() is not an option, because the message will be immediately received again until the Max Delivery Count is reached, and then put into the DLQ. If an API is down for maintenance, we want to wait a bit before retrying.
One of the answers to this question has the general steps for deferring and receiving deferred messages. This is a little better than Microsoft's documentation, but not enough for me to understand the intent of the API, and how it is to be implemented in a hosted service that basically sits in ServiceBusProcessor.StartProcessingAsync all day long.
This is the basic structure of my service:
public class ServiceBusWatcher : IHostedService, IDisposable
{
public Task StartAsync(CancellationToken stoppingToken)
{
ReceiveMessagesAsync();
return Task.CompletedTask;
}
private async void ReceiveMessagesAsync()
{
ServiceBusClient client = new ServiceBusClient(connectionString);
processor = client.CreateProcessor(queueName, new ServiceBusProcessorOptions());
processor.ProcessMessageAsync += MessageHandler;
await processor.StartProcessingAsync();
}
async Task MessageHandler(ProcessMessageEventArgs args)
{
// a dependency is not available that allows me to process a message. so:
await args.DeferMessageAsync(args.Message);
Once the message is deferred, it is my understanding that the processor will not get to it anymore (or will it?). Instead, I have to use ReceiveDeferredMessageAsync() to receive it, along with the sequence number of the originally received message.
In my case, it will make sense to wait minutes or hours before trying again.
This could be done with a separate service that uses a timer and an explicit call to ReceiveDeferredMessageAsync(), as opposed to using a ServiceBusProcessor. I also suppose that the deferred message sequence numbers will have to be persisted in non-volatile storage so that they don't get lost.
Does this sound like a viable approach? I don't like having to remember its sequence numbers so that I can get to a message later. It goes against everything that using a message queue brings to the table in the first place.
Or, instead of deferring, I could just post a new "internal" message with the sequence number and use the ScheduledEnqueueTimeUtc property to delay receiving it. Once I receive this message, I could call ReceiveDeferredMessageAsync() with that sequence number to get to the original message. This seems elegant at the surface, but messages could quickly multiply if there is a longer outage of a dependency.
Another idea that could work without another service: I could complete and repost the payload of the message and set ScheduledEnqueueTimeUtc to a time in the future, as described in another answer to the question I mentioned earlier. Assuming that this works (Microsoft's documentation does not mention what this property is for), it seems simple and clean, and I like simple.
How have you solved this? Is there a better/preferred way that balances low complexity with high robustness without requiring a large amount of code?
Deferring a message works when you know what message you want to retrieve later and your receiver will have the message sequence number saved to retrieve the deferred message. If the receiver has no ability to save message sequence number, the delaying the message is a better option. Delaying a message will mean to copy the original message data into a newly scheduled one and completing the original message. That way the consumer doesn't have to neither hold on to the message sequence number nor initiate the retrieval of a specific message.
I have some code that uses the Service Bus Event Data, and I suspect that I need to use the offset property as, currently, my program is (or seems to be) re-running the same Event Hub data over and over again.
My code is as follows:
public class EventHubListener : IEventProcessor
{
private static EventHubClient _eventHubClient;
private const string EhConnectionStringNoPath = "Endpoint=...";
private const string EhConnectionString = EhConnectionStringNoPath + ";...";
private const string EhEntityPath = "...";
public void Start()
{
_eventHubClient = EventHubClient.CreateFromConnectionString(EhConnectionString);
EventHubConsumerGroup defaultConsumerGroup = _eventHubClient.GetDefaultConsumerGroup();
EventHubDescription eventHub = NamespaceManager.CreateFromConnectionString(EhConnectionStringNoPath).GetEventHub(EhEntityPath);
foreach (string partitionId in eventHub.PartitionIds)
{
defaultConsumerGroup.RegisterProcessor<EventHubListener>(new Lease
{
PartitionId = partitionId
}, new EventProcessorCheckpointManager());
Console.WriteLine("Processing : " + partitionId);
}
}
public Task ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> messages)
{
foreach (EventData eventData in messages)
{
string bytes = Encoding.UTF8.GetString(eventData.GetBytes());
MyData data = JsonConvert.DeserializeObject<MyData>(bytes);
As I get the same messages over and over again, I suspect that I need to do something like this:
string bytes = Encoding.UTF8.GetString(eventData.GetBytes(), eventData.Offset, eventData.SerializedSizeInBytes - eventData.Offset);
However, Offset is a string, even though it seems to be a numeric value ("12345" for example). The documentation on context.CheckPointAsync() made it seem like that might be the answer; however, issuing that at the end of the loop seems to make no difference.
So, I have a two part question:
What is offset? Is it what I think it is (i.e. a numeric marker to a point in the stream) and, if so, why is it a string?
Why would I be getting the same messages over again? As I understand Event Hubs, although they guarantee at least once, once a Checkpoint has been issues, I shouldn't be getting the same messages back.
EDIT:
After a while of messing about, I've come up with something that avoids this problem; however, I certainly wouldn't claim it's a solution:
var filteredMessages =
messages.Where(a => a.EnqueuedTimeUtc >= _startDate)
.OrderBy(a => a.EnqueuedTimeUtc);
Using the EventProcessorHost seemed to actually make the problem worse; that is, not only were historical events being replayed, but they seemed to be replayed in a random order.
EDIT:
I came across this excellent article by #Mikhail, which does seem to address my exact issue. However; and presumably the root of my problem (or one of them, assuming this is correct, then I'm unsure why using the EventProcessorHost doesn't just work out of the box as #Mikhail said himself in the comments). However, the ServiceBus version of ICheckpointManager only has a single interface method:
namespace Microsoft.ServiceBus.Messaging
{
public interface ICheckpointManager
{
Task CheckpointAsync(Lease lease, string offset, long sequenceNumber);
}
}
Your title should be event hub, rather than service bus. For your question:
Although event hub has similar design as Kafka, but one big difference is that you should manage offsets by yourself. Event hub broker has completely no idea about your consumer group's offset.
So event hub sdk provide some help class to store offset in storage account, but you still need to call checkpoint manually after processing the message.
What is offset? Is it what I think it is (i.e. a numeric marker to a point in the stream) and, if so, why is it a string?
The offset is the pointer within a stream. The offset of an event changes as events gets removed from your Event Hub when the Message Retention policy has elapsed. So a message that was once at offset 10, maybe at offset 0 several days later because older messages were dropped from the stream. This has a good diagram: Event Hubs: Stream Offsets.
Why would I be getting the same messages over again? As I understand Event Hubs, although they guarantee at least once, once a Checkpoint has been issues, I shouldn't be getting the same messages back.
You may be getting the same messages again if you are using the low-level EventReceiver offset since messages expire from the Event Hub when the Message Retention policy elapses (ie. Default is 1 day). Sequence number is a better field to leverage because it does not change.
When checkpointing succeeds, it tells us the last event that was successfully processed, so you shouldn't be getting the same event back because when the client starts, it'll create a stream to a position in the event stream after that event. You can file an issue on GitHub.
EventProcessorHost is helpful as it tries to balance the processing of partitions between the number of instances running. (ie. Consider a 6 partition Event Hub. If you have 2 EventProcessorHosts connected to the same Event Hub reading with the same consumer group, they'll end up balancing the processing of those partitions with 3 each.) It also reconnects when there are transient failures like network loss.
It supports checkpointing to durable storage like Azure Storage Blob. Here is a sample: Process Events using an EventProcessorClient
We currently have a NServiceBus 5 system, which contains two recurring Sagas. Since they act as dispatcher to periodically pull multiple sorts of data from an external system, we're using the Timeouts to trigger this: We created a generic and empty class called ExecuteTask, which is used by the Saga to handle the timeout.
public class ScheduleSaga1 : Saga<SchedulerSagaData>,
IAmStartedByMessages<StartScheduleSaga1>,
IHandleMessages<StopSchedulingSaga>,
IHandleTimeouts<ExecuteTask>
And the other Saga is almost identically defined:
public class ScheduleSaga2: Saga<SchedulerSagaData>,
IAmStartedByMessages<StartScheduleSaga2>,
IHandleMessages<StopSchedulingSaga>,
IHandleTimeouts<ExecuteTask>
The timeout is handled equally in both Sagas:
public void Handle(StartScheduleSaga1 message)
{
if (_schedulingService.IsDisabled())
{
_logger.Info($"Task '{message.TaskName}' is disabled!");
}
else
{
Debugger.DoDebug($"Scheduling '{message.TaskName}' started!");
Data.TaskName = message.TaskName;
// Check to avoid that if the saga is already started, don't initiate any more tasks
// as those timeout messages will arrive when the specified time is up.
if (!Data.IsTaskAlreadyScheduled)
{
// Setup a timeout for the specified interval for the task to be executed.
Data.IsTaskAlreadyScheduled = true;
// Send the first Message Immediately!
SendMessage();
// Set the timeout
var timeout = _schedulingService.GetTimeout();
RequestTimeout<ExecuteTask>(timeout);
}
}
}
public void Timeout(ExecuteTask state)
{
if (_schedulingService.IsDisabled())
{
_logger.Info($"Task '{Data.TaskName}' is disabled!");
}
else
{
SendMessage();
// Action that gets executed when the specified time is up
var timeout = _schedulingService.GetTimeout();
Debugger.DoDebug($"Request timeout for Task '{Data.TaskName}' set to {timeout}!");
RequestTimeout<ExecuteTask>(timeout);
}
}
private void SendMessage()
{
// Send the Message to the bus so that the handler can handle it
Bus.Send(EndpointConfig.EndpointName, Activator.CreateInstance(typeof(PullData1Request)));
}
Now the problem: Since both Sagas are requesting Timeouts for ExecuteTask, it gets dispatched to both Sagas!
Even worse, it seems like the stateful Data in the Sagas gets messed up, since both Sagas are sending both message.
Therefore, it seems like the Timeouts are getting sent to all the Saga Instances which are requesting it.
But looking at the example https://docs.particular.net/samples/saga/simple/ there is no special logic regarding multiple Saga instances and their state.
Is my assumption correct? If this is the case, what are the best practices to have multiple Sagas requesting and receiving Timeouts?
The only reason I can think of when this is happening is that they share the same identifier to uniquely identify the saga instance.
Both ScheduleSaga1 and ScheduleSaga2 are using the same SchedulerSagaData for storing state. NServiceBus sees an incoming message and tries to retrieve the state, based on the unique identifier in the incoming message. If both StartScheduleSaga1 and StartScheduleSaga2 come in with identifier 1 for example, NServiceBus will search for saga state in the table SchedulerSagaData with unique identifier 1.
Both ScheduleSaga1 and ScheduleSaga2 will then share the same row!!!
Timeouts are based on SagaId in the TimeoutEntity table. Because both sagas share the same SagaId, it's logical they are both executed once the timeout arrives.
At the minimum you should not reuse the identifier to schedule tasks. It's probably better to not share the same class for storing saga state. Also easier to debug.
I have a C# application that sets up numerous MQ listeners (multiple threads and potentially multiple servers each with their own listeners). There are some messages that will come off the queue that I will want to leave on the queue, move on to the next message on the MQ, but then under some circumstances I will want to go back to re-read those messages...
var connectionFactory = XMSFactoryFactory.GetInstance(XMSC.CT_WMQ).CreateConnectionFactory();
connectionFactory.SetStringProperty(XMSC.WMQ_HOST_NAME, origination.Server);
connectionFactory.SetIntProperty(XMSC.WMQ_PORT, int.Parse(origination.Port));
connectionFactory.SetStringProperty(XMSC.WMQ_QUEUE_MANAGER, origination.QueueManager);
connectionFactory.SetStringProperty(XMSC.WMQ_CHANNEL, origination.Channel);
var connection = connectionFactory.CreateConnection(null, null);
_connections.Add(connection);
var session = connection.CreateSession(false, AcknowledgeMode.ClientAcknowledge); //changed to use ClientAcknowledge so that we will leave the message on the MQ until we're sure we're processing it
_sessions.Add(session);
var destination = session.CreateQueue(origination.Queue);
_destinations.Add(destination);
var consumer = session.CreateConsumer(destination);
_consumers.Add(consumer);
Logging.LogDebugMessage(Constants.ListenerStart);
connection.Start();
ThreadPool.QueueUserWorkItem((o) => Receive(forOrigination, consumer));
Then I have...
if (OnMQMessageReceived != null)
{
var message = consumer.Receive();
var identifier = string.Empty;
if (message is ITextMessage)
{
//do stuff with the message here
//populates identifier from the message
}
else
{
//do stuff with the message here
//populates identifier from the message
}
if (!string.IsNullOrWhiteSpace(identifier)&& OnMQMessageReceived != null)
{
if( some check to see if we should process the message now)
{
//process message here
message.Acknowledge(); //this really pulls it off of the MQ
//here is where I want to trigger the next read to be from the beginning of the MQ
}
else
{
//We actually want to do nothing here. As in do not do Acknowledge
//This leaves the message on the MQ and we'll pick it up again later
//But we want to move on to the next message in the MQ
}
}
else
{
message.Acknowledge(); //this really pulls it off of the MQ...its useless to us anyways
}
}
else
{
Thread.Sleep(0);
}
ThreadPool.QueueUserWorkItem((o) => Receive(forOrigination, consumer));
So a couple of questions:
If I do not acknowledge the message it stays on the MQ, right?
If the message is not acknowledged then by default when I read from the MQ again with the same listener it reads the next one and does not go to the beginning, right?
How do I change the listener so that the next time I read I start at the beginning of the queue?
Leaving messages on a queue is an anti-pattern. If you don't want to or cannot process the message at a certain point of your logic, then you have a number of choices:
Get it off the queue and put to another queue/topic for a delayed/different processing.
Get it off the queue and dump to a database, flat file - whatever, if you want to process it outside of messaging flow, or don't want to process at all.
If it is feasible, you may want to change the message producer so it doesn't mix the messages with different processing requirements in the same queue/topic.
In any case, do not leave a message on the queue, and always move forward to the next message. This will make the application way more predictable and easier to reason about. You will also avoid all kinds of performance problems. If your application is or may ever become sensitive to the sequence of message delivery, then manual acknowledgement of selected messages will be at odds with it too.
To your questions:
The JMS spec is vague regarding the behavior of unacknowledged messages - they may be delivered out of order, and it is undefined when exactly when they will be delivered. Also, the acknowledge method call will acknowledge all previously received and unacknowledged messages - probably not what you had in mind.
If you leave messages behind, the listener may or may not go back immediately. If you restart it, it of course will start afresh, but while it is sitting there waiting for messages it is implementation dependent.
So if you try to make your design work, you may get it kind of work under certain circumstances, but it will not be predictable or reliable.
Current Setup includes a windows service which picks up a message from the local queue and extracts the information and puts in to my SQL database.According to my design
Service picks up the message from the queue.(I am using Peek() here).
Sends it to the database.
If for some reason i get an exception while saving it to the database the message is back into the queue,which to me is reliable.
I am logging the errors so that a user can know what's the issue and fix it.
Exception example:If the DBconnection is lost during saving process of the messages to the database then the messages are not lost as they are in the queue.I don't comit untill i get an acknowledgement from the DB that the message is inserted .So a user can see the logs and make sure that the DBconnection exists and every thing would be normal and we dont lose any messages in the queue.
But looking into another scenario:The messages I would be getting in the queue are from a 3rd party according a standard schema.The schema would remain same and there is no change in that.But i have seen some where i get some format exceptions and since its not committed the message is back to the queue.At this point this message would be a bottle neck for me as the same messages is picked up again and tries to process the message.Every time the service would pick up the same message and gets the same exception.So this loops infinitely unless that message is removed or put that message last in the queue.
Looking at removing the message:As of now if i go based on the format exception...then i might be wrong since i might encounter some other exceptions in the future .
Is there a way i can put this messages back to the queue last in the list instead beginning of the queue.
Need some advice on how to proceed further.
Note:Queue is Transactional .
As far as I'm aware, MSMQ doesn't automatically dump messages to fail queues. Either way you handle it, it's only a few lines of code (Bill, Michael, and I recommend a fail queue). As far as a fail queue goes, you could simple create one named .\private$\queuename_fail.
Surviving poison messages in MSMQ is a a decent article over this exact topic, which has an example app and source code at the end.
private readonly MessageQueue _failQueue;
private readonly MessageQueue _messageQueue;
/* Other code here (cursor, peek action, run method, initialization etc) */
private void dumpToFailQueue(Message message)
{
var oldId = message.Id;
_failQueue.Send(message, MessageQueueTransactionType.Single);
// Remove the poisoned message
_messageQueue.ReceiveById(oldId);
}
private void moveToEnd(Message message)
{
var oldId = message.Id;
_messageQueue.Send(message, MessageQueueTransactionType.Single);
// Remove the poisoned message
_messageQueue.ReceiveById(oldId);
}