Develop Reference

Is it possible to listen to a service bus with multiple azure functions?

I'm trying to listen to a service bus that can contain messages with multiple event types. When a message arrives I want to process it based on the event type. I have 2 azure functions to handle the different event type messages, but when one function receives the message the other one is not triggered. Is it possible to trigger both of them and let them decide which one processes the message?
Here is some sample code, these are in 2 separate projects:
[FunctionName("Function1")]
public async Task RunAsync(
[ServiceBusTrigger("queueName", Connection = "connectionName")]
Message message,
MessageReceiver messageReceiver,
ILogger log)
{
var body = Encoding.Default.GetString(message.Body);
var messageType = _helper.GetMessageType(body);
if (messageType is not MessageType.Type1)
return;
// business logic
await messageReceiver.CompleteAsync(message.SystemProperties.LockToken);
}
[FunctionName("Function2")]
public async Task RunAsync(
[ServiceBusTrigger("queueName", Connection = "connectionName")]
Message message,
MessageReceiver messageReceiver,
ILogger log)
{
var body = Encoding.Default.GetString(message.Body);
var messageType = _helper.GetMessageType(body);
if (messageType is not MessageType.Type2)
return;
// business logic
await messageReceiver.CompleteAsync(message.SystemProperties.LockToken);
}

With Azure Service Bus Queues, it is not possible as a Queue can only have a single consumer.
A better approach would be to use Azure Service Bus Topics and Subscriptions. You can send the message to a Topic and then create Subscription Filtering rules so that a message goes to an appropriate Subscription.
You can then have 2 Azure Functions each listening to a separate Subscription.

In addition to Gaurav's answer, you can use the Azure Service Bus ForwardTo feature, see the following example:

There are few approach and each depends on your requirement.
As Suggested by #Gaurav , Topic is best solution where two subscription to topic ( one is function1 and another function2) and it works most of the case. Also keep in mind that it is individual subscriber to process message and it get failed or success depends on that subscription only and not across subscription.
Now if you still want to stick to Queue then also there is approach. In this you have to pass extra metadata in message like you are doing with type.
var body = Encoding.Default.GetString(message.Body);
var messageType = _helper.GetMessageType(body);
if (messageType is MessageType.Type1)
{
// Call handler/method that process Type1 message.
}
else if (messageType is MessageType.Type2)
{
// Call handler/method that process Type2 message.
}

Azure Service bus - pass parameter to message handler

In Azure Service Bus I need to listen for messages arriving from multiple subscriptions from different services busses at once.
To do this I created a list that contains objects with a connection string, a topic, a subscription name and some other information (the list is called 'jobs').
For each item in this list I am then creating a different task that creates the ServiceBusClient and the processor.
var jobs = GetAllServiceBusTopics();
Parallel.ForEach(jobs, async job =>
{
var client = new ServiceBusClient(job.Environment.ServiceBusConnectionString);
var options = new ServiceBusProcessorOptions();
var processor = client.CreateProcessor(job.Environment.TopicName, _subscriptionName, new ServiceBusProcessorOptions());
try
{
processor.ProcessMessageAsync += MessageHandler;
//Pass the job object somehow to the "MessageHandler" below.
processor.ProcessErrorAsync += ErrorHandler;
await processor.StartProcessingAsync();
Console.WriteLine("Wait for a minute and then press any key to end the processing");
Console.ReadKey();
Console.WriteLine("\nStopping the receiver...");
await processor.StopProcessingAsync();
Console.WriteLine("Stopped receiving messages");
}
finally
{
await processor.DisposeAsync();
await client.DisposeAsync();
}
});
And the handler that is called if a new message arrives:
static async Task MessageHandler(ProcessMessageEventArgs args)
{
//I need the "job" object from my loop above here.
}
How the concept generally works I learned on this website of Microsoft.
My first question:
Is this approach okay, or am I running in the wrong direction? Can I do it like this?
But even if this is okay, I have another more important task:
I need to pass the "job" object from my loop somehow to the message handler - as a parameter.
But I have currently no idea how to archvie this. Any proposals on this?

Is this approach okay, or am I running in the wrong direction? Can I do it like this?
Yes, you can do this. One thing to keep in mind is that you instantiate multiple ServiceBusClient instances, each causing a new connection to be established rather than using the same connection. I don't know how big the number of topics (jobs) might be but if it's large, you'll end up with connections starvation.
I need to pass the "job" object from my loop somehow to the message handler - as a parameter. But I have currently no idea how to archvie this. Any proposals on this?
That's not how ServiceBusProcessor is designed. It doesn't receive anything other than the incoming message that needs to be processed. If you need to have a job ID, that should be part of the message payload/metadata. If you need to know the entity it arrived from, you could add a subscription filter action to add a custom header with the identifier. An alternative approach would require wrapping the ServiceBusProcessor to retain the job ID/subscription identifier and use that in the event handler.

Correct way to publish and subscribe to an explicit service bus topic with MassTransit?

In scenarios where you want to share a Azure Service Bus namespace (to save cost etc), it's helpful to be able to explicitly set queue and topic names rather than rely on MassTransit conventions. Whilst this is straightforward with queues, we've run into difficulty with topics.
The MassTransit documentation is sparse in this area.
It's currently recommending to use a ReceiveEndpoint which appears to forward messages from a topic subscription onto a queue and then subscribe to that queue. We have got this setup working but it seems wasteful when we could just subscribe to the topic directly?
cfg.ReceiveEndpoint(
host,
"my-queue-1",
e =>
{
e.Subscribe("my-topic-1", "sub-1");
e.ConfigureConsumer<MyConsumer>(provider);
e.MaxConcurrentCalls = 24;
});
I've discovered SubscriptionEndpoint which appears to do just this. We'd like to subscribe to a specifically named topic 'my-topic-1' with a named subscription 'sub-1'. We can replace the ReceiveEndpoint with a SubscriptionEndpoint as follows:
cfg.SubscriptionEndpoint(
host,
"sub-1",
"my-topic-1",
e =>
{
e.ConfigureConsumer<MyConsumer>(provider);
e.MaxConcurrentCalls = 24;
});
We'd like the publisher to be in a separate application, what should its bus config and publish code look like? The docs hint that the Publish Topology can be used to supply an explicit Service Bus topic name. We've tried setting the TopicDescription.Path as follows:
Bus.Factory.CreateUsingAzureServiceBus(..., cfg =>
{
cfg.Publish<IMyMessage>(x =>
{
x.TopicDescription.Path = "my-topic-1";
});
});
await bus.Publish<IMyMessage>(message);
But the message is not getting through to the consumer.
It seems this is a issue with publishing. Although the publisher creates the topic 'my-topic-1' it still publishes the message to the topic 'my.namespace/imymessage'. Looking at the source of ServiceBusMessagePublishTopology I can see why this is the case (_topicDescription vs _topicConfigurator) but it doesn't seem obvious how to override the topic's Path with the topic path hardcoded to the messageTopology's EntityName (#L42) and the then only a setter on BasePath available.

IBM.XMS IMessageConsumer - won't release - Can't unsubscribe

First: the issue
I am unable to successfully do these steps on an asynchronous durable subscriber against ibm's mq topics.
IMessageConsumer.MessageListener = null;
IMessageConsumer.Close();
IMessageConsumer.Dispose();
ISession.Unsubscribe(topicPath);
I get IBM.WMQ.MQException {"2428"}
"Failed to unsubscribe from topic X using MQCLOSE.
There might have been a problem removing the subscription because it is being used by a message consumer.
Make sure any message consumers using this subscription are closed before unsubscribing. Please see the linked exception for more information."
Second: Specifics
So I'm using IBMs MQ infrastructure. Specifically their Topic implementation for distributed publisher/subscriber implementation. I'm wrapping the whole thing in a .NET WEB API (MVC5) project to abstract from the rest of the organization all the MQ specifics.
For the moment ignore all fail over, etc stuff. When a client calls in I first see if we have an ISession for the caller. If we do I use the existing, if not I make a new from a shared connection factory. Then I create a new destination for consumer. Next I create durable subscription for the specific topic path. Then I add a MessageListener and add the whole mess to an in memory cache. Other tracking also occurring but not important to this discussion.
using IBM.XMS;
// Subscribe
MQConnectionFactory f = (MQConnectionFactory)fact;
SessionClass newSession = new SessionClass(); // Contains an ISession, IDestination, IMessageConsumer
newSession.Session = f.CreateSession();
newSession.Destination = newSession.Session.CreateTopic(MQConnectionFactory.FormatTopic(path));
newSession.Consumer = newSession.Session.CreateDurableSubscriber(newSession.Destination, subId.ToString());
newSession.Consumer.MessageListener = new MessageListener(MessageHandler);
// Message Handler - Some "details" removed
private void MessageHandler(IMessage msg)
{
string topic = msg.GetStringProperty(MQConstants.TOPIC);
DateTime timestamp = DateTime.FromBinary(msg.GetLongProperty(MQConstants.DATETIME));
List<KeyValuePair<string, object>> parms = msg.GetStringProperty(MQConstants.PARAMETERS).FromBase64();
object payload = msg.GetObjectProperty(MQConstants.PAYLOAD);
Publication publication = new Publication()
{
MessageTimestamp = timestamp,
Topic = topic,
Parameters = parms
};
Callback.Notify(publication))
_log.DebugFormat("Message delivered to {0}", msg.JMSMessageID);
msg.Acknowledge();
}
// And for unsubscribe
SessionClass s = _destinations[fullPath];
s.Consumer.MessageListener = null;
s.Consumer.Close();
s.Consumer.Dispose();
s.Session.Unsubscribe(s.Destination.Name);
_destinations.Remove(fullPath);
So all the subscribe and message handling works fine. The UnSubscribe always fails when I hit the
s.Session.Unsubscribe(s.Destination.Name)
saying it's "in use" basically. I've attempted Stopping the connection prior to the unsubscribe to no effect as well.
Anyone have any ideas? I'm completely unable to remove any subscriptions from MQ after creating them as a result.
thanks

So after reviewing documentation more and consulting more with other people it's a matter of not the most clear MQ documentatin.
What needs passed to the Unsubscribe method is the ID passed to the CreateDurableSubscriber method
So....using the code above the only change is to the Unsubscribe call.
s.Session.Unsubscribe(subId.ToString());
And then all is good.

Cant consume message from Topic in activemq

I am new to activemq. T want to ask a question about the topics of Activemq. I succeed to get a message from a queue. Also I can send message to topic/Queue, but I can't get a message from Topic.
I have tried using Java Code. The result is the same.
The following is my core code:
connection.ClientId = clientId;
connection.Start();
using (ISession session = connection.CreateSession())
{
ITopic topic = new Apache.NMS.Commands.Topic(topicName);
IDestination destination = SessionUtil.GetDestination(session, topicName,
DestinationType.Topic);
using (IMessageConsumer consumer = **session.CreateDurableConsumer**(topic, "news", null, false))
{
**consumer.Listener += new MessageListener(consumer_Listener);**
//**IMessage iMsg = consumer.Receive();**
// if (iMsg != null)//{
// ITextMessage msg = (ITextMessage)iMsg;
// return msg.Text;
// }
//else
//return iMsg;
}
}
I also using: IMessage iMsg = consumer.Receive();
IMsg　always null（topicname has messages. How can I consume topic's message?

The Messages would need to have been sent after the Topic consumer was created. A Topic is fire and forget, if there are no consumers then the message is discarded. Any consumer that comes online will only receive message sent after that time unless it is either a Durable Topic consumer or a Queue consumer.
In the case of a durable consumer you must have created an instance of it so there is a subscription record before those message were sent to the Topic. So I would guess your problem is that you didn't subscribe this consumer before and so the Broker was not storing any Messages for it.

I was so stupid about the phrase "using".Beacause I use "using" open connection and session. when the code block was excuted, the connnection/session is disappear. Now I dont use "using" block to cerate connection. just like normal code. It works. also I build "Global.asax" file. The program can listener Topic once started up. At the same time, I write a function to colse the connection.I tested. Once a message was sent to the topic, the Onessage() function would be exectued.
just resolve my problem.maybe you would have better answer.Thanks Tim.