I'm trying to read a message off the queue (notified rather than polling) using C# azure function.
I'm currently pushing a message onto the queue using a console app for now, see below.
There seems to be two different products, azure service bus and a queue within a storage account. The following code uses the later, I'm not sure how to read the message of the queue?
Many thanks,
Code example
StorageCredentials creds = new StorageCredentials(accountname, accountkey);
CloudStorageAccount account = new CloudStorageAccount(creds, useHttps: true);
CloudQueueClient queueClient = account.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference("test-queue");
queue.CreateIfNotExists();
while (true)
{
CloudQueueMessage message = new CloudQueueMessage(JsonConvert.SerializeObject("my message"));
queue.AddMessage(message);
}
Update
After the comments and following links as suggested by experts trying to help I have tried the following code sample to push a message onto Azure Service Bus (i.e not a queue in a storage account)
namespace ConsoleApp1
{
using System;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.ServiceBus;
class Program
{
const string ServiceBusConnectionString = "Endpoint=sb://xxx.servicebus.windows.net/;SharedAccessKeyName=xxx;SharedAccessKey=xxx";
const string QueueName = "name_of_queue";
static IQueueClient queueClient;
public static async Task Main(string[] args)
{
const int numberOfMessages = 10;
queueClient = new QueueClient(ServiceBusConnectionString, QueueName);
await SendMessagesAsync(numberOfMessages);
await queueClient.CloseAsync();
}
static async Task SendMessagesAsync(int numberOfMessagesToSend)
{
for (var i = 0; i < numberOfMessagesToSend; i++)
{
string messageBody = $"Message {i}";
var message = new Message(Encoding.UTF8.GetBytes(messageBody));
Console.WriteLine($"Sending message: {messageBody}");
// Send the message to the queue
try
{
await queueClient.SendAsync(message); // this line
}
catch (Exception ex)
{
throw ex;
}
}
}
}
}
When I run the code above the console window doesn't do anything, no error message just nothing..! strange.. Looking in the azure service bus overview it active message count is zero.
I'm using this sample project but the queueClient.SendAsync never returns back. Do I need to set something up in azure, permissions perhaps?
https://github.com/Azure/azure-service-bus
Error eventually received
A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond
I can see requests within the portal service bus screen
Is these what you want?
1.If you want to get message from azure service bus queue:
using System;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure.ServiceBus;
namespace ConsoleApp2
{
class Program
{
const string ServiceBusConnectionString = "xxxxxx";
const string QueueName = "xxx";
static IQueueClient queueClient;
public static async Task Main(string[] args)
{
queueClient = new QueueClient(ServiceBusConnectionString, QueueName);
RegisterOnMessageHandlerAndReceiveMessages();
Console.ReadKey();
await queueClient.CloseAsync();
}
static void RegisterOnMessageHandlerAndReceiveMessages()
{
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
MaxConcurrentCalls = 10,
AutoComplete = false
};
queueClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
}
static async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
}
static Task ExceptionReceivedHandler(ExceptionReceivedEventArgs exceptionReceivedEventArgs)
{
Console.WriteLine($"Message handler encountered an exception {exceptionReceivedEventArgs.Exception}.");
var context = exceptionReceivedEventArgs.ExceptionReceivedContext;
Console.WriteLine("Exception context for troubleshooting:");
Console.WriteLine($"- Endpoint: {context.Endpoint}");
Console.WriteLine($"- Entity Path: {context.EntityPath}");
Console.WriteLine($"- Executing Action: {context.Action}");
return Task.CompletedTask;
}
}
}
Before receive from the service bus queue, I have put message in the queue.
This is the result:
2.If you want to get message from azure storage queue:
using System;
using Microsoft.Azure;
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Queue;
namespace ReceiveMessageFromAzureStorageQueue
{
class Program
{
static void Main(string[] args)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("xxxxxx");
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference("xxx");
CloudQueueMessage peekedMessage = queue.PeekMessage();
queue.FetchAttributes();
int? cachedMessageCount = queue.ApproximateMessageCount;
Console.WriteLine("Number of messages in queue: " + cachedMessageCount);
for(int i=0; i<cachedMessageCount; i++) {
System.Threading.Thread.Sleep(10);
CloudQueueMessage retrievedMessage = queue.GetMessage(TimeSpan.FromMilliseconds(10));
Console.WriteLine(retrievedMessage.AsString);
queue.DeleteMessage(retrievedMessage);
}
Console.WriteLine("Already Read.");
}
}
}
Still put messages in the azure storage queue before receive them.
This is the result:
Please let me know if you have more doubts.
Related
Just starting to work with Azure.
Have a simple C# .NET Core app, which connects to Azure ServiceBus, reads messages, and writes them to Azure SQL database.
Works locally just fine - connects to remote Azure Service Bus Queue, reads messages, connect to remote Azure SQL db, writes records.
Same exact app, when deployed to Azure as a WebApp, appears to "run", but no longer reads messages from Services Bus, and no longer writes anything to Azure SQL.
Here is the entire app (i.e. Program.cs):
using System;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure.ServiceBus;
using System.Data.SqlClient;
namespace ServiceBusReader
{
class Program
{
const string ServiceBusConnectionString = "SB_CONNECTION_STRING";
const string QueueName = "BasicQueue";
static IQueueClient queueClient;
static SqlConnection connection = null;
public static async Task Main(string[] args)
{
System.Diagnostics.Trace.TraceError("Inside Main function...");
queueClient = new QueueClient(ServiceBusConnectionString, QueueName);
Console.WriteLine("======================================================");
System.Diagnostics.Trace.TraceError("======================================================");
Console.WriteLine("Press ENTER key to exit after receiving all the messages.");
Console.WriteLine("======================================================");
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder();
builder.DataSource = "XXXX.database.windows.net";
builder.UserID = "USERID";
builder.Password = "PASSWORD";
builder.InitialCatalog = "mySampleDatabase";
connection = new SqlConnection(builder.ConnectionString);
connection.Open();
// Register the queue message handler and receive messages in a loop
RegisterOnMessageHandlerAndReceiveMessages();
Console.ReadKey();
await queueClient.CloseAsync();
}
static void RegisterOnMessageHandlerAndReceiveMessages()
{
// Configure the message handler options in terms of exception handling, number of concurrent messages to deliver, etc.
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
// Maximum number of concurrent calls to the callback ProcessMessagesAsync(), set to 1 for simplicity.
// Set it according to how many messages the application wants to process in parallel.
MaxConcurrentCalls = 1,
// Indicates whether the message pump should automatically complete the messages after returning from user callback.
// False below indicates the complete operation is handled by the user callback as in ProcessMessagesAsync().
AutoComplete = false
};
// Register the function that processes messages.
queueClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
}
static async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
// Process the message.
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
string query = "INSERT INTO [SalesLT].[messages] (message) VALUES(#Message)";
SqlCommand cmd = new SqlCommand(query, connection);
System.Diagnostics.Trace.TraceError(Encoding.UTF8.GetString(message.Body));
cmd.Parameters.AddWithValue("#Message", Encoding.UTF8.GetString(message.Body));
cmd.ExecuteNonQuery();
Console.WriteLine("Records Inserted Successfully...");
System.Diagnostics.Trace.TraceError("Records Inserted Successfully...");
// Complete the message so that it is not received again.
// This can be done only if the queue Client is created in ReceiveMode.PeekLock mode (which is the default).
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
// Note: Use the cancellationToken passed as necessary to determine if the queueClient has already been closed.
// If queueClient has already been closed, you can choose to not call CompleteAsync() or AbandonAsync() etc.
// to avoid unnecessary exceptions.
}
// Use this handler to examine the exceptions received on the message pump.
static Task ExceptionReceivedHandler(ExceptionReceivedEventArgs exceptionReceivedEventArgs)
{
Console.WriteLine($"Message handler encountered an exception {exceptionReceivedEventArgs.Exception}.");
var context = exceptionReceivedEventArgs.ExceptionReceivedContext;
Console.WriteLine("Exception context for troubleshooting:");
Console.WriteLine($"- Endpoint: {context.Endpoint}");
Console.WriteLine($"- Entity Path: {context.EntityPath}");
Console.WriteLine($"- Executing Action: {context.Action}");
return Task.CompletedTask;
}
}
}
Should I do anything differently in this app in order to make it work in Azure?
It seems that you are trying to deploy a webjob to web app. May I know if you have set it to Continuous type? If you have set it to Continuous, the webjob will automatically run after deployment.
By default, the type is Triggered and you need to manually start the webjob from portal.
I have an Azure IOThub that contains one edge device. Within this edge device I have several modules running and I can change the twin property of any individual module by connecting to it with it's connection string.
Now I would like the module to do something when it's twin property is changed but the module doesn't have access to it's connection string and it shouldn't because it shouldn't need to connect to itself.
How can a module detect it's twin property change without having a connection string?
I have followed this tutorial but this uses a connection string to detect changes: https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-csharp-csharp-module-twin-getstarted#create-a-module-identity
Here's the code for this module:
using System;
using Microsoft.Azure.Devices.Client;
using Microsoft.Azure.Devices.Shared;
using System.Threading.Tasks;
using Newtonsoft.Json;
namespace DataSyncService
{
class Program
{
private const string ModuleConnectionString = "CONNECTION STRING";
private static ModuleClient Client = null;
static void ConnectionStatusChangeHandler(ConnectionStatus status,
ConnectionStatusChangeReason reason)
{
Console.WriteLine("Connection Status Changed to {0}; the reason is {1}",
status, reason);
}
static void Main(string[] args)
{
Microsoft.Azure.Devices.Client.TransportType transport =
Microsoft.Azure.Devices.Client.TransportType.Amqp;
try
{
Client =
ModuleClient.CreateFromConnectionString(ModuleConnectionString, transport);
Client.SetConnectionStatusChangesHandler(ConnectionStatusChangeHandler);
// I want to set this callback but this requires a client and the client requires
// a connection string.
Client.SetDesiredPropertyUpdateCallbackAsync(OnDesiredPropertyChanged, null).Wait();
Console.WriteLine("Retrieving twin");
var twinTask = Client.GetTwinAsync();
twinTask.Wait();
var twin = twinTask.Result;
Console.WriteLine(JsonConvert.SerializeObject(twin.Properties));
Console.WriteLine("Sending app start time as reported property");
TwinCollection reportedProperties = new TwinCollection();
reportedProperties["DateTimeLastAppLaunch"] = DateTime.Now;
Client.UpdateReportedPropertiesAsync(reportedProperties);
}
catch (AggregateException ex)
{
Console.WriteLine("Error in sample: {0}", ex);
}
Console.WriteLine("Waiting for Events. Press enter to exit...");
Console.ReadLine();
Client.CloseAsync().Wait();
}
private static async Task OnDesiredPropertyChanged(TwinCollection desiredProperties,
object userContext)
{
Console.WriteLine("desired property change:");
Console.WriteLine(JsonConvert.SerializeObject(desiredProperties));
Console.WriteLine("Sending current time as reported property");
TwinCollection reportedProperties = new TwinCollection
{
["DateTimeLastDesiredPropertyChangeReceived"] = DateTime.Now
};
await Client.UpdateReportedPropertiesAsync(reportedProperties).ConfigureAwait(false);
}
}
}
When you create a new module in Visual Studio Code, it generates a template module for you that will show you how it's done. I'll include the important bit below:
static async Task Init()
{
MqttTransportSettings mqttSetting = new MqttTransportSettings(TransportType.Mqtt_Tcp_Only);
ITransportSettings[] settings = { mqttSetting };
// Open a connection to the Edge runtime
ModuleClient ioTHubModuleClient = await ModuleClient.CreateFromEnvironmentAsync(settings);
await ioTHubModuleClient.OpenAsync();
Console.WriteLine("IoT Hub module client initialized.");
// Register callback to be called when a message is received by the module
await ioTHubModuleClient.SetInputMessageHandlerAsync("input1", PipeMessage, ioTHubModuleClient);
}
The way this works is because the Azure IoT Edge runtime will create the module as a Docker container with the connection settings as environment variables. The module client uses those variables to connect to IoT Hub when you call
ModuleClient ioTHubModuleClient = await
ModuleClient.CreateFromEnvironmentAsync(settings);
There is a good sample on this Microsoft tutorial that also covers listening for Twin updates.
using System;
using System.Text;
using System.Threading.Tasks;
namespace eNtsaIOTMqttApp
{
class Program
{
// you can get DeviceConnectionString from your IoT hub
private const string DeviceConnection = "HostName=eNstaIot.azure-devices.net;DeviceId=GcobaniTesting1;SharedAccessKey=*********";
static ServiceClient serviceClient;
static void Main(string[] args)
{
serviceClient = serviceClient.CreateFromConnectionString(DeviceConnectionString);
Program prog = new Program();
}
public Program() {
DeviceClient deviceClient = DeviceClient.CreateFromConnectionString(DeviceConnectionString);
SendEvent().Wait();
ReceiveCommands(deviceClient).Wait();
}
// This method is responsible for sending Event to Iot Hub.
static async Task SendEvent()
{
string dataBuffer = "IOT in 90 seconds";
Microsoft.Azure.Devices.Message eventMessage = new Microsoft.Azure.Devices.Messages(Encoding.ASCII);
await serviceClient.SendAysnc("GcobaniTesting1", eventMessage);
}
// this method is responbile for receive message on the IOT hub.
async TaskReceiveCommands(DeviceClient deviceClient)
{
Console.WriteLine("\nDevice waiting for IoT hub command...\n");
Microsoft.Azure.Devices.Client.Message receiveMessage;
string messageData;
while (true)
{
receiveMessage = await deviceClient.ReceiveAsync(TimeSpan.FromSeconds(1));
if(receiveMessage != null)
{
messageData = Encoding.ASCII.GetString(receiveMessage.GetBytes());
Console.WriteLine("\t{0}> Message received: {1}", DateTime.Now.ToLocalTime(),messageData);
await deviceClient.CompleteAsync(receiveMessage);
}
}
}
}
}
Why am i not receive the message on my console when i am running this console application? I do get iot device, but when i send message from iot hub i dont get on visual studio.
is there some useful link to this rule?
This line is wrong:
serviceClient = serviceClient.CreateFromConnectionString(DeviceConnectionString);
You must use the service-side connection string to create the ServiceClient, for example the IoTHubOwner connection string. You are trying to create your service client with the device connection string. This should actually throw an error.
See here for guidance on connection strings: https://devblogs.microsoft.com/iotdev/understand-different-connection-strings-in-azure-iot-hub/
I am wondering how to resend messages that have faulted with EasyNetQ, programmatically, without using HosePipe, that is to resend to their original target queue using their original sending exchange.
Is it possible, how?
The solution I came up with is:
public static class AdvancedBusExtensions
{
public static async Task ResendErrorsAsync(this IAdvancedBus source, string errorQueueName)
{
var errorQueue = await source.QueueDeclareAsync(errorQueueName);
var message = await source.GetMessageAsync(errorQueue);
while (message != null)
{
var utf8Body = Encoding.UTF8.GetString(message.Body);
var error = JsonConvert.DeserializeObject<Error>(utf8Body);
var errorBodyBytes = Encoding.UTF8.GetBytes(error.Message);
var exchange = await source.ExchangeDeclareAsync(error.Exchange, x =>
{
// This can be adjusted to fit the exchange actual configuration
x.AsDurable(true);
x.AsAutoDelete(false);
x.WithType("topic");
});
await source.PublishAsync(exchange, error.RoutingKey, true, error.BasicProperties, errorBodyBytes);
message = await source.GetMessageAsync(errorQueue);
}
}
}
I have an Azure worker role that is responsible for checking 4 service bus queues. Currently, I just the looping method to manually check the queues.
while(true)
{
//loop through my queues to check for messages
}
With the Azure SDK 2.0 came the ability to listen for messages rather than polling for them. But Every example I've seen uses a console app with Console.ReadKey(). Is there a way to have the worker role sit and wait on messages too?
I tried:
public override void Run()
{
_queueProcessors.ForEach(x => x.OnMessage(Process);
}
where _queueProcessors is a list of QueueClients and Process is a private method that handles the messages. However, the worker role would register them and then restart.
So anyone know how to make a queue client sit and wait on a message?
Following is a code sample for this:
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;
using Microsoft.WindowsAzure.ServiceRuntime;
using System.Diagnostics;
using System.Net;
using System.Threading;
namespace WorkerRoleWithSBQueue1
{
public class WorkerRole : RoleEntryPoint
{
// The name of your queue
const string QueueName = "demoapp";
ManualResetEvent CompletedEvent = new ManualResetEvent(false);
// QueueClient is thread-safe. Recommended that you cache
// rather than recreating it on every request
QueueClient Client;
public override void Run()
{
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = true; // Indicates if the message-pump should call complete on messages after the callback has completed processing.
options.MaxConcurrentCalls = 1; // Indicates the maximum number of concurrent calls to the callback the pump should initiate
options.ExceptionReceived += LogErrors; // Allows users to get notified of any errors encountered by the message pump
Trace.WriteLine("Starting processing of messages");
// Start receiveing messages
Client.OnMessage((receivedMessage) => // Initiates the message pump and callback is invoked for each message that is recieved, calling close on the client will stop the pump.
{
try
{
// Process the message
Trace.WriteLine("Processing Service Bus message: " + receivedMessage.SequenceNumber.ToString());
}
catch
{
// Handle any message processing specific exceptions here
}
}, options);
CompletedEvent.WaitOne();
}
private void LogErrors(object sender, ExceptionReceivedEventArgs e)
{
if (e.Exception != null)
{
Trace.WriteLine("Error: " + e.Exception.Message);
}
}
public override bool OnStart()
{
// Set the maximum number of concurrent connections
ServicePointManager.DefaultConnectionLimit = 12;
// Create the queue if it does not exist already
Trace.WriteLine("Creating Queue");
string connectionString = "*** provide your connection string here***";
var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.QueueExists(QueueName))
{
namespaceManager.CreateQueue(QueueName);
}
// Initialize the connection to Service Bus Queue
Client = QueueClient.CreateFromConnectionString(connectionString, QueueName);
Trace.WriteLine("Sending messages...");
// populate some messages
for (int ctr = 0; ctr < 10; ctr++)
{
Client.Send(new BrokeredMessage());
}
return base.OnStart();
}
public override void OnStop()
{
// Close the connection to Service Bus Queue
Client.Close();
CompletedEvent.Set(); // complete the Run function
base.OnStop();
}
}
}