Consuming a message published with NServiceBus, using IQueueClient/Functions-ServiceBusTrigger (Microsoft.Azure.ServiceBus)
I'm working in a WebJob using .NET Core and Microsoft.Azure.ServiceBus to consume a message that has been published by a separate service using NServiceBus. My initial approach with this WebJob was to use a class Functions.cs with a method ProcessMessage that uses the attribute ServiceBusTrigger
Below is an example of how my Function.cs looks like:
public class Functions
{
public Task ProcessAuditLogMessage([ServiceBusTrigger("MyQueue")]
Message message)
{
var messageBody = Encoding.UTF8.GetString(message.Body);
var auditLogMessage = JsonConvert
.DeserializeObject<MyMessage>(messageBody);
_logger.Information("Hello World");
return Task.CompletedTask;
}
}
In Program.cs, I have:
class Program
{
static async Task Main()
{
var serviceCollection = new ServiceCollection();
serviceCollection.AddWebJobs(o => o.AllowPartialHostStartup = true);
var builder = new HostBuilder()
.UseServiceProviderFactory(
new AutofacServiceProviderFactory(serviceCollection))
.ConfigureContainer<ContainerBuilder>(b =>
{
BuildAutofacContainer();
})
.ConfigureWebJobs(b =>
{
b.AddServiceBus(o =>
{
o.ConnectionString = configProvider.AuditLogServiceBus;
});
});
var host = builder.Build();
using (host)
{
await host.RunAsync();
}
}
private static IContainer BuildAutofacContainer(IServiceColletion
serviceCollection)
{
...
}
}
I was expecting that this logic would consume the messages published in the Queue but so far the messages get moved to the Dead-letter count with a DeadLetterReason: MaxDeliveryCountExceeded and the error description Message could not be consumed after 10 delivery attempts which gives me the indication that at least there is an attempt to get these messages delivered to my consumer.
Unfortunately that's all I have in terms of error messages/logs (I'm in the process to set up some logs from my Azure Dashboard and see if I can get more detailed logs)
Has anyone come across the scenario o having to consume messages, that have been published with NServiceBus, using Microsoft.Azure.ServiceBus instead of NServiceBus (on the consumer side). Maybe I'm missing something...
Related
I have the following services implemented on Azure:
1x Azure SignalR Service (Serverless) ASRS
2x Azure Functions (Serverless) HubFuncDown & HubFuncUp
On the ASRS I have defined TWO UpStream URLs, one to HubFuncDown Azure Function and the other to HubFuncUp. Using the URL pattern defined in the docs:
HubFuncDown Contains the following method which instructs the device app to disable itself:
[FunctionName(nameof(DisableDevice))]
public async Task DisableDevice([SignalRTrigger] InvocationContext invocationContext, string deviceId, ILogger logger) {
await Clients.User(deviceId).SendAsync(DisableDeviceTarget, new NewMessage(invocationContext, deviceId));
}
And HubFuncUp Contains the following method:
[FunctionName(nameof(DeviceDisabled))]
public async Task DeviceDisabled([SignalRTrigger] InvocationContext invocationContext, string deviceId, ILogger logger) {
// .. Updates DBContext, sends alerts etc
}
I'm trying to tell HubFuncUp that this client is now in a disabled state, the code to do that on the (UWP) client is:
var connection = new HubConnectionBuilder()
.WithUrl("https://hubfuncup.azurewebsites.net/api", options => {
options.AccessTokenProvider = () => Auth();
})
.ConfigureLogging(logging => {
logging.AddProvider(new SerilogLoggerProvider());
logging.SetMinimumLevel(LogLevel.Debug);
})
.WithAutomaticReconnect(new RetryPolicy())
.Build();
And to tell HubFuncUp of the state:
await connection.InvokeAsync("UpdateDeviceState", new DisabledDeviceMessage { DeviceId = 123, State = States.Disabled });
But each time I'm receiving a 404 Error from the call to InvokeAsync("UpdateDeviceState"..). It seems no matter what I do I cannot connect the one ASRS to two Azure Functions using multiple Upstream URLs.
Am I correct in thinking I need to utilise a seperate Azure SignalR Service (which doubles my cost) or can I connect the two Functions to the one SignalR Service via routing on the Upstream URLs?
I have 2 Web APIs developed on ASP.NET Core. The idea is: the WebAPI_1 sends a message to the Azure Service Bus and then WebAPI_2 has to catch this moment and read it shortly after the message is sent. I understand how to catch this moment is I have a console app instead of WebAPI_2, but I am not sure how to subscribe WebAPI_2 on such the event happening in Azure Service Bus.
Below is the code where I have WebAPI_1 and the Console App.
WebAPI_1 (Sender):
public class QueueService : IQueueService
{
private readonly IConfiguration _config;
public QueueService(IConfiguration config)
{
_config = config;
}
public async Task SendMessageAsync<T>(T serviceBusMessage, string queueName)
{
var queueClient = new QueueClient(_config.GetConnectionString("AzureServiceBus"), queueName);
string messageBody = JsonSerializer.Serialize(serviceBusMessage);
var message = new Message(Encoding.UTF8.GetBytes(messageBody));
await queueClient.SendAsync(message);
}
}
And this is how I send it:
await queue.SendMessageAsync(obj, "myqueue");
And this is the Console App (Receiver):
Main(){
queueClient = new QueueClient(connectionString, queueName);
var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
MaxConcurrentCalls = 1,
AutoComplete = false
};
queueClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
}
private static async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
var jsonString = Encoding.UTF8.GetString(message.Body);
Model obj = JsonSerializer.Deserialize<Model>(jsonString);
Console.WriteLine($"Person Received: { obj.Field1} { obj.Field2}");
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
}
But I want WebAPI_2 to be able to receive the messages instead of the Console App.
Please advise.
Receiving messages requires a continuous job. ASP.NET Core Controller, as you've probably found out, is not the right place as it's not running continuously and is intended to respond to the request. For a continuous execution, a background service, or task, is the right option. ASP.NET Core has an option to run a BackgroundService that could be used for exactly what you need.
There are multiple blog posts with the details in case you want to get some inspiration:
Getting Started With Azure Service Bus Queues And ASP.NET Core Background Services
Using An ASP.NET Core IHostedService To Run Azure Service Bus Subscriptions and Consumers
I'm looking at incorporating Azure SignalR functionality into my .net core Blazor web application. To this end i've been following this tutorial - Azure Signalr Serverless. This is working fine - i have a project running the Azure functions app and can start up two browsers and have a chat session. What i'm trying to do is add the ability to receive these message notifications from the Azure signalR hub that's been configured into my Blazor app. I've added the following code in Index.razor.cs that mimics the javascript code in the example client:
public class IndexComponent : ComponentBase
{
private HubConnection _connection;
public string Message;
protected override Task OnInitializedAsync()
{
_connection = new HubConnectionBuilder()
.WithUrl("http://localhost:7071/api")
.Build();
_connection.On<string, string>("ReceiveMessage", (user, message) =>
{
Message = $"Got message {message} from user {user}";
this.StateHasChanged();
});
_connection.StartAsync();
return base.OnInitializedAsync();
}
}
The example javascript code btw is:
const connection = new signalR.HubConnectionBuilder()
.withUrl(`${apiBaseUrl}/api`)
.configureLogging(signalR.LogLevel.Information)
.build();
connection.on('newMessage', newMessage);
connection.onclose(() => console.log('disconnected'));
console.log('connecting...');
connection.start()
.then(() => data.ready = true)
.catch(console.error);
So the problem is that my Blazor app never receives any message notifications sent from the javascript chat clients (so the _connection.On handler is never hit). What am i missing in my Blazor code ?
Ok so this is what i needed to do to get it to work in my Blazor app:
_connection.On<object>("newMessage", update =>
{
Console.WriteLine(update);
//Message = update;
});
I needed to subscribe to the 'newMessage' target (since that's the JS is sending on) and also the type that's being posted isn't a string but a JObject type which i would need to deserialize to the correct type.
I have a RMQ Cluster setup on Amazon EC2. Now when I try to create a RabbitMQ host by setting the cluster members, I get the Broker Unreachable exception every time. However, if I comment that UseCluster method, things start working fine. But I am not able to enjoy full benefits of cluster in that case.
Following is the piece of code I am working with:
foreach(var server in servers)
{
try
{
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(server), hst =>
{
hst.Username(settings.UserName);
hst.Password(settings.Password);
hst.UseCluster(c =>
{
c.ClusterMembers = servers;
});
});
//Check if the connection is reachable
host.Settings.GetConnectionFactory().CreateConnection();
registrationAction?.Invoke(cfg, host);
});
AddBusToContainer(container, bus);
return bus;
}
catch (BrokerUnreachableException ex)
{
EventLog.WriteEntry($"Error trying to connect to {server}", $"{ex}", EventLogEntryType.Error);
continue;
}
}
In the above code, if I comment the UseCluster implementation, I am able to successfully test the connection. But I receive the BrokerUnreachableException with the above code.
Does anyone have any idea about how to configure RMQ Cluster when creating a MassTransit bus? It would be very helpful.
Thanks in advance.
To configure a cluster, you should only configure a single bus instance (you're configuring multiple in your example). I don't know what version of MassTransit you're using (it looks older, given the syntax), but this represents the current approach if you're hand-coding the bus.
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host("cluster", hst =>
{
hst.Username(settings.UserName);
hst.Password(settings.Password);
hst.UseCluster(c =>
{
foreach(var server in servers)
c.Node(server);
});
});
registrationAction?.Invoke(cfg);
});
AddBusToContainer(container, bus);
return bus;
Though I would suggest using the newer syntax with your container, to ensure components are properly registered.
container.AddMassTransit(x =>
{
x.UsingRabbitMq(cfg =>
{
cfg.Host("cluster", hst =>
{
hst.Username(settings.UserName);
hst.Password(settings.Password);
hst.UseCluster(c =>
{
foreach(var server in servers)
c.Node(server);
});
});
registrationAction?.Invoke(cfg);
});
})
I create a bot, called picturesaver, using Microsoft's Bot Framework, I added a GroupMe channel, and I have it hosted in Azure. The bot works perfectly, saving pictures to Google Drive.
However, the bot gives an error saying "Service Error:POST to picturesaver timed out after 15s" Is it possible to extend the timeout time? Or even stop the bot from posting anything at all. Could this be an Azure issue or is it a GroupMe issue?
If your bot performs an operation that takes longer than 15 seconds to process a message, you can process the message on another thread, and acknowledge the call right away. Something like:
public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
{
if (activity.Type == ActivityTypes.Message)
{
if ([determine if this will take > 15s])
{
// process the message asyncronously
Task.Factory.StartNew(async () => await Conversation.SendAsync(activity, () => new Dialogs.RootDialog()));
}
else
{
//process the message normally
await Conversation.SendAsync(activity, () => new Dialogs.RootDialog());
}
}
return Request.CreateResponse(HttpStatusCode.OK); //ack the call
}
This will avoid the 15 second timeout between connector and bot.
Edit: the above will not scale, and is just using a Task.Factory. Please refer to https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-howto-long-operations-guidance for the recommended guidance on processing long operations from a bot.
The Bot Connector service has a 15s timeout so you need to make sure any async API calls are handled in that timeframe, or make sure your bot responds with some kind of message if it's waiting for some other operation to complete. Currently the 15s timeout cannot be modified.
The solution to process the message on another thread, and acknowledge the call right away is good only for a bot on an App Service.
But as for a Functions Bot doing so will finish the Azure Function if I immediately return from this method.
I tried it. The Azure Function stops running, and the real response to the chat never comes. So it's not a solution at all for the Function Bots.
I ended up with this code for a Functions Bot, which resolves this problem.
Using Azure Queues
public static class Functions
{
[FunctionName("messages")]
[return: Queue("somequeue")]
public static async Task<MessagePayload> Messages([HttpTrigger
(WebHookType = "genericJson")]HttpRequestMessage req) =>
// return from this Azure Function immediately to avoid timeout warning message
// in the chat.
// just put the request into "somequeue".
// We can't pass the whole request via the Queue, so pass only what we need for
// the message to be processed by Bot Framework
new MessagePayload
{
RequestUri = req.RequestUri,
Content = await req.Content.ReadAsStringAsync(),
AuthScheme = req.Headers.Authorization.Scheme,
AuthParameter = req.Headers.Authorization.Parameter
};
// Do the actual message processing in another Azure Function, which is
// triggered by a message enqueued in the Azure Queue "somequeue"
[FunctionName("processTheMessage")]
public static async Task ProcessTheMessage([QueueTrigger("somequeue")]
MessagePayload payload, TraceWriter logger)
{
// we don't want the queue to process this message 5 times if it fails,
// so we won't throw any exceptions here at all, but we'll handle them properly.
try
{
// recreate the request
var request = new HttpRequestMessage
{
Content = new StringContent(payload.Content),
RequestUri = payload.RequestUri
};
request.Headers.Authorization = new
AuthenticationHeaderValue(payload.AuthScheme, payload.AuthParameter);
// initialize dependency injection container, services, etc.
var initializer = new SomeInitializer(logger);
initializer.Initialize();
// handle the request in a usual way and reply back to the chat
await initializer.HandleRequestAsync(request);
}
catch (Exception ex)
{
try
{
// TODO: handle the exception
}
catch (Exception anotherException)
{
// swallow any exceptions in the exceptions handler?
}
}
}
}
[Serializable]
public class MessagePayload
{
public string Content { get; set; }
public string AuthParameter { get; set; }
public string AuthScheme { get; set; }
public Uri RequestUri { get; set; }
}
(Be sure to use different Azure Queues for local development with Bot Framework emulator and for a cloud-deployed Function App. Otherwise, the messages sent to your bot from real customers may be processed locally while you are debugging on your machine)
Using an HTTP request
Of course, the same can be done without using an Azure Queue with a direct call to another Azure Function's public URL - https://<my-bot>.azurewebsites.net/api/processTheMessage?code=<function-secret>. This call has to be done on another thread, without waiting for the result in the messages function.
[FunctionName("messages")]
public static async Task Run([HttpTrigger(WebHookType = "genericJson")]
HttpRequestMessage req)
{
// return from this Azure Function immediately to avoid timeout warning message
// in the chat.
using (var client = new HttpClient())
{
string secret = ConfigurationManager.AppSettings["processMessageHttp_secret"];
// change the RequestUri of the request to processMessageHttp Function's
// public URL, providing the secret code, stored in app settings
// with key 'processMessageHttp_secret'
req.RequestUri = new Uri(req.RequestUri.AbsoluteUri.Replace(
req.RequestUri.PathAndQuery, $"/api/processMessageHttp?code={secret}"));
// don't 'await' here. Simply send.
#pragma warning disable CS4014
client.SendAsync(req);
#pragma warning restore CS4014
// wait a little bit to ensure the request is sent. It will not
// send the request at all without this line, because it would
// terminate this Azure Function immediately
await Task.Delay(500);
}
}
[FunctionName("processMessageHttp")]
public static async Task ProcessMessageHttp([HttpTrigger(WebHookType = "genericJson")]
HttpRequestMessage req,
Microsoft.Extensions.Logging.ILogger log)
{
// first and foremost: initialize dependency
// injection container, logger, services, set default culture/language, etc.
var initializer = FunctionAppInitializer.Initialize(log);
// handle the request in a usual way and reply back to the chat
await initializer.HandleRequest(req);
}