Develop Reference

Azure Service Bus Receive Messages continuously when ever new message placed in web application [duplicate]

I am using Azure.Messaging.ServiceBus nuget package to work with Azure service bus. We have created a topic and a subscription. The subscription has 100+ messages. We want to read all the message and continue to read message as they arrive.
Microsoft.Azure.ServiceBus package (deprecated now) provided RegisterMessageHandler which use to process every incoming message. I am not able to find similar option under Azure.Messaging.ServiceBus nuget package.
I am able to read one message at a time but I have to call await receiver.ReceiveMessageAsync(); every time manually.

To receive multiple messages (a batch), you should use ServiceBusReceiver.ReceiveMessagesAsync() (not plural, not singular 'message'). This method will return whatever number of messages it can send back. To ensure you retrieve all 100+ messages, you'll need to loop until no messages are available.
If you'd like to use a processor, that's also available in the new SDK. See my answer to a similar question here.

As suggested by #gaurav Mantri, I used ServiceBusProcessor class to implement event based model for processing messages
public async Task ReceiveAll()
{
string connectionString = "Endpoint=sb://sb-test-today.servicebus.windows.net/;SharedAccessKeyName=manage;SharedAccessKey=8e+6SWp3skB3Aedsadsadasdwz5DU=;";
string topicName = "topicone";
string subscriptionName = "subone";
await using var client = new ServiceBusClient(connectionString, new ServiceBusClientOptions
{
TransportType = ServiceBusTransportType.AmqpWebSockets
});
var options = new ServiceBusProcessorOptions
{
// By default or when AutoCompleteMessages is set to true, the processor will complete the message after executing the message handler
// Set AutoCompleteMessages to false to [settle messages](https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-transfers-locks-settlement#peeklock) on your own.
// In both cases, if the message handler throws an exception without settling the message, the processor will abandon the message.
AutoCompleteMessages = false,
// I can also allow for multi-threading
MaxConcurrentCalls = 1
};
await using ServiceBusProcessor processor = client.CreateProcessor(topicName, subscriptionName, options);
processor.ProcessMessageAsync += MessageHandler;
processor.ProcessErrorAsync += ErrorHandler;
await processor.StartProcessingAsync();
Console.ReadKey();
}
public async Task MessageHandler(ProcessMessageEventArgs args)
{
string body = args.Message.Body.ToString();
Console.WriteLine(body);
// we can evaluate application logic and use that to determine how to settle the message.
await args.CompleteMessageAsync(args.Message);
}
public Task ErrorHandler(ProcessErrorEventArgs args)
{
// the error source tells me at what point in the processing an error occurred
Console.WriteLine(args.ErrorSource);
// the fully qualified namespace is available
Console.WriteLine(args.FullyQualifiedNamespace);
// as well as the entity path
Console.WriteLine(args.EntityPath);
Console.WriteLine(args.Exception.ToString());
return Task.CompletedTask;
}

Filter Duplicate Messages IoT Device

I am working on a console application program that receives messages from an Azure IoT Device and is using a timer to receive the messages every two seconds. Upon receiving a message the device sends out a call to complete it. The issue that I am experiencing is that the message does not complete in Azure before the device receives it again, which results in the message getting reprocessed. I have tried to filter incoming messages when it is the same message coming in multiple times, but the messages are coming in with the same message id whether or not they are duplicate messages or new messages. I do not have access to control the incoming message's message id field and make it unique, but that would solve the problem. The sequence number is unique for every message that comes in whether it is duplicate or not so I cannot use that as a filter either. Is there a way to filter a message to see if it is a duplicate without the message id field?
//Within Program.cs > Main():
_timer = new Timer(Operations, null, 0, _timerInterval); //_timerInterval is set to 2000
//Within Initialize class used to setup device client:
//Fully qualified namespace for DeviceClient:
//Microsoft.Azure.Devices.Client.DeviceClient
string connectionString = "code removed for example";
var deviceClient = DeviceClient.CreateFromConnectionString(connectionString);
//Within Operations class:
var message = await deviceClient.ReceiveAsync();
if (message != null && !string.IsNullOrEmpty(message?.MessageId))
{
//Filtering message based on MessageId
if (_memoryCache.Get(message.MessageId) == null)
{
_memoryCache.Set(message.MessageId, message.MessageId, DateTimeOffset.UtcNow.AddMinutes(10));
await deviceClient.CompleteAsync(message);
//Processing message
await ProcessMessage(message);
}
else
{
await deviceClient.RejectAsync(message);
}
}

You can use Microsoft.Azure.Devices.Client.Message package to retrieve the Device client message values.
Use the IOT explicit unique id in message to check the duplicates while receiving.
Follow the below code to check the duplicate value
List<string> FinalResponse = new List<string>();
Microsoft.Azure.Devices.Client.Message Response = await deviceClient.ReceiveAsync(TimeSpan.FromSeconds(10));
if (Response == null)
{
await Task.Delay(10).ConfigureAwait(false);
continue;
}
//here you can use the explicit properties like message id or correlation Id
Trace.WriteLine(Response.MessageId.ToString());
await this.deviceClient.CompleteAsync(Response);
var content = Encoding.UTF8.GetString(Response.GetBytes());
FinalResponse.Add(content);
Either you can use above one or use below conditions
Create a List and ADD all values to the list which get from Device
Add condition to ignore If any duplicates occur while insert into list.
Then send unduplicated values to Azure.

C# Azure ServiceBus / Azure SQL - works locally, does not seem to work when deployed to Azure

Just starting to work with Azure.
Have a simple C# .NET Core app, which connects to Azure ServiceBus, reads messages, and writes them to Azure SQL database.
Works locally just fine - connects to remote Azure Service Bus Queue, reads messages, connect to remote Azure SQL db, writes records.
Same exact app, when deployed to Azure as a WebApp, appears to "run", but no longer reads messages from Services Bus, and no longer writes anything to Azure SQL.
Here is the entire app (i.e. Program.cs):
using System;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure.ServiceBus;
using System.Data.SqlClient;
namespace ServiceBusReader
{
class Program
{
const string ServiceBusConnectionString = "SB_CONNECTION_STRING";
const string QueueName = "BasicQueue";
static IQueueClient queueClient;
static SqlConnection connection = null;
public static async Task Main(string[] args)
{
System.Diagnostics.Trace.TraceError("Inside Main function...");
queueClient = new QueueClient(ServiceBusConnectionString, QueueName);
Console.WriteLine("======================================================");
System.Diagnostics.Trace.TraceError("======================================================");
Console.WriteLine("Press ENTER key to exit after receiving all the messages.");
Console.WriteLine("======================================================");
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder();
builder.DataSource = "XXXX.database.windows.net";
builder.UserID = "USERID";
builder.Password = "PASSWORD";
builder.InitialCatalog = "mySampleDatabase";
connection = new SqlConnection(builder.ConnectionString);
connection.Open();
// Register the queue message handler and receive messages in a loop
RegisterOnMessageHandlerAndReceiveMessages();
Console.ReadKey();
await queueClient.CloseAsync();
}
static void RegisterOnMessageHandlerAndReceiveMessages()
{
// Configure the message handler options in terms of exception handling, number of concurrent messages to deliver, etc.
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
// Maximum number of concurrent calls to the callback ProcessMessagesAsync(), set to 1 for simplicity.
// Set it according to how many messages the application wants to process in parallel.
MaxConcurrentCalls = 1,
// Indicates whether the message pump should automatically complete the messages after returning from user callback.
// False below indicates the complete operation is handled by the user callback as in ProcessMessagesAsync().
AutoComplete = false
};
// Register the function that processes messages.
queueClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
}
static async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
// Process the message.
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
string query = "INSERT INTO [SalesLT].[messages] (message) VALUES(#Message)";
SqlCommand cmd = new SqlCommand(query, connection);
System.Diagnostics.Trace.TraceError(Encoding.UTF8.GetString(message.Body));
cmd.Parameters.AddWithValue("#Message", Encoding.UTF8.GetString(message.Body));
cmd.ExecuteNonQuery();
Console.WriteLine("Records Inserted Successfully...");
System.Diagnostics.Trace.TraceError("Records Inserted Successfully...");
// Complete the message so that it is not received again.
// This can be done only if the queue Client is created in ReceiveMode.PeekLock mode (which is the default).
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
// Note: Use the cancellationToken passed as necessary to determine if the queueClient has already been closed.
// If queueClient has already been closed, you can choose to not call CompleteAsync() or AbandonAsync() etc.
// to avoid unnecessary exceptions.
}
// Use this handler to examine the exceptions received on the message pump.
static Task ExceptionReceivedHandler(ExceptionReceivedEventArgs exceptionReceivedEventArgs)
{
Console.WriteLine($"Message handler encountered an exception {exceptionReceivedEventArgs.Exception}.");
var context = exceptionReceivedEventArgs.ExceptionReceivedContext;
Console.WriteLine("Exception context for troubleshooting:");
Console.WriteLine($"- Endpoint: {context.Endpoint}");
Console.WriteLine($"- Entity Path: {context.EntityPath}");
Console.WriteLine($"- Executing Action: {context.Action}");
return Task.CompletedTask;
}
}
}
Should I do anything differently in this app in order to make it work in Azure?

It seems that you are trying to deploy a webjob to web app. May I know if you have set it to Continuous type? If you have set it to Continuous, the webjob will automatically run after deployment.
By default, the type is Triggered and you need to manually start the webjob from portal.

How can I consume RabbitMq messages with different type of consumers in .Net?

MessageType: "PublishX"
Consumers:
Type1ConsumerX
Type2ConsumerX
Type3ConsumerX
All of the consumers must catch messages immediately, but consume synchronously inside themselves..
For example there are 100 "PublishX" messages in the queue. Type1ConsumerX consumed 30 messages (synchronously), Type2ConsumerX consumed 50 messages(synchronously) , Type3ConsumerX consumed 100 messages(synchronously).
How can I know the message is consumed by "all type of consumers" ?
Could RabbitMQ/MassTransit PUSH messages to consumers?
Could RabbitMQ/MassTransit push messages (merging them) with intervals (1s) for decrease network traffic?
Could RabbitMQ/MassTransit push same messages to the different type of Consumers?

If I've understood the question correctly you just need to set up a basic pub/sub pattern. This will allow you to deliver the same message to multiple consumers.
Example publisher:
public static void PublishMessageToFanout()
{
var factory = new ConnectionFactory { HostName = "localhost" };
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.ExchangeDeclare("messages", "fanout");
var message = new Message { Text = "This is a message to send" };
var json = JsonConvert.SerializeObject(message);
var body = Encoding.UTF8.GetBytes(json);
channel.BasicPublish("messages", string.Empty, null, body);
}
}
Example consumers:
SubscribeToMessages("sms-messages", (s) => Console.WriteLine("SMS Message: {0}", s));
SubscribeToMessages("email-messages", (s) => Console.WriteLine("Email Message: {0}", s));
public static void SubscribeToMessages(string queueName, Action<string> messageAction)
{
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
channel.ExchangeDeclare("messages", "fanout");
channel.QueueDeclare(queueName, true, false, false, null);
channel.QueueBind(queueName, "messages", string.Empty);
var consumer = new QueueingBasicConsumer(channel);
channel.BasicConsume(queueName, true, consumer);
while (true)
{
var ea = consumer.Queue.Dequeue();
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
messageAction(message);
}
}
}
If you run the SubscribeToMessages loops in separate processes or console apps you'll see that they both print out the message whenever you call the PublishMessageToFanout. You'll also see that both queues exist in RabbitMQ Management under Queues.

Regarding the MassTransit part of your question
RabbitMQ/MassTransit PUSH messages to consumers?
Yes, MassTransit publishes messages to the bus, and then a consumer processes them
Could RabbitMQ/MassTransit push messages (merging them) with intervals (1s) for decrease network traffic?
Don't know if there is a feature for this, you could write your own but you would have to be very careful about losing the messages.
Could RabbitMQ/MassTransit push same messages to the different type of Consumers?
Yes, multiple consumers can consume the same type of message.
I've written a simple hello world app that shows the basics - http://nodogmablog.bryanhogan.net/2015/04/mass-transit-with-rabbitmq-hello-world/

Windows Non-WCF Service Moving Transacted MSMQ Message to Failed Queue

I have a legacy Windows service running on Server 2008 that reads messages from a Transactional MSMQ Queue. This is not configured as a WCF service.
We are wanting to improve the handling of failed and poison messages in code (C# 4.0) by catching custom exceptions and sending the related message to a separate 'failed' or 'poison' queue depending upon the type of exception thrown.
I can't get the Catch code to send the message to the separate queue - it disappears from the original queue (as desired!) but doesn't show up in the 'failed' queue.
For testing all of the queues have no Authentication required and permissions are set to allow Everyone to do everything.
Clearly something is missing or wrong and I suspect it is transaction related, but I can't see it. Or perhaps this is not possible the way I am trying to do it ?
Any guidance / suggestions appreciated!
Our simplified PeekCompleted Event code:
private void MessageReceived(object sender, PeekCompletedEventArgs e)
{
using (TransactionScope txnScope = new TransactionScope())
{
MyMessageType currentMessage = null;
MessageQueue q = ((MessageQueue)sender);
try
{
Message queueMessage = q.EndPeek(e.AsyncResult);
currentMessage = (FormMessage)queueMessage.Body;
Processor distributor = new Processor();
Processor.Process(currentMessage); // this will throw if need be
q.ReceiveById(e.Message.Id);
txnScope.Complete();
q.BeginPeek();
}
catch (MyCustomException ex)
{
string qname = ".\private$\failed";
if (!MessageQueue.Exists(qname)){
MessageQueue.Create(qname , true);
}
MessageQueue fq = new MessageQueue(qname){
Formatter = new BinaryMessageFormatter()
};
System.Messaging.Message message2 = new System.Messaging.Message{
Formatter = new BinaryMessageFormatter(),
Body = currentMessage,
Label = "My Failed Message";
};
fq.Send(message2); //send to failed queue
q.ReceiveById(e.Message.Id); //off of original queue
txnScope.Complete(); // complete the trx
_queue.BeginPeek(); // next or wait
}
//other catches handle any cases where we want to tnxScope.Dispose()
EDIT : October 8, 2013
Hugh's answer below got us on the right track. Inside the Catch block the Failed Queue was already created as transactional
MessageQueue.Create(qname , true);
but the Send needed a TransactionType parameter
fq.Send(message2,MessageQueueTransactionType.Single);
That did the trick. Thanks Hugh!

If the message is disappearing from your original queue then that means your code is reaching the second scope.Complete() (in your catch block).
This means the problem has to do with your send to the error queue.
I would suggest that you need to create the queue as transactional because you are sending from within a scope.
MessageQueue fq = new MessageQueue(qname, true){
Formatter = new BinaryMessageFormatter()
};
Then you need to do a transactional send:
fq.Send(message2, Transaction.Current);
See if this works.