I am using the library Silverback and Apache Kafka as the message broker.
I have a two message classes:
Animal (base class) which is published to / consumed from the "animals" Kafka topic
Dog (extends Animal, but doesn't have any additional property) which is published to / consumed from the "dogs" Kafka topic
When I publish the message of type Dog, Silverback publishes one message in "animals" and another message in "dogs".
I was expecting the message of type Dog to be published only on the "dogs" topic.
Related
We are using MassTransit with RabbitMQ and part of our implementation includes an outbox pattern.
Now i'm trying to create a docker container whose only purpose is to dispatch messages from outboxes in several databases.
The container gets a list of connection strings to the various databases and then starts to dispatch messages from their outboxes.
Currently we store the following information in our outbox (with examples):
MessageType: SomeNamespace.SomeType, SomeContract
MessageBody: {"SomeProperty":"MyValue"}
TransmitMethod: Send/Publish
QueueName: SomeQueueName
My question is if it's possible to dispatch these messages without having access to the contract types?
I can add more information to the table if needed to make this happen.
You can look at how the MassTransit message scheduler support for Quartz.NET captures and ultimately sends the message on the transport. In this case, it's saving the serialized message from the transport and reloading the JSON into the message body at serialization time.
You might also find useful details in the relational outbox draft PR.
I have a service bus topic subscription model . I am in control of designing the sender component to a topic. however receiver is a remote server whose code i cannot control. Now the tricky part, is i need to somehow possibly know some stats from service bus without really having to ask the remote server to do additional work.
For eg.
1)Last message processed (it's content)
2)Last message completed succesfully - Time and content.
This is for basic troubleshooting on my word to know, that message has atleast been recieved by the receiver.
Is it possible to do this?
What does it mean "Last message processed" and "Last message completed succesfully". If you have a constant stream of messages, at what point you would determine what's a last message.
What you're asking is somewhat in violation of the pub/sub concept. The whole point of topics and subscriptions is to decouple publishers and subscribers.
This is for basic troubleshooting on my word to know, that message has atleast been recieved by the receiver.
When messages are sent to the subscription queue, they are either consumed or eventually end up in the dead-letter queue. If they are in the dead-letter queue, you'll know the reason. In case they are consumed, you will have to trust the consumer it knows what it's doing. Any time of "reply" or "acknowledgement" goes against the concept of events, where you broadcast of something that has happened and should not carry if it was received or not.
I'm currently building a system using MassTransit and RabbitMQ as my messaging layer. I'm trying to find a way to have a Consumer that listens on all messages of all types on the bus. This is for our audit logging framework and we want to log all of the events going across the message bus.
Is there a way to do this in MassTransit?
You would need to add some type of auditing interface to your message that could be subscribed for auditing purposes. For example, if you were to create a base interface:
public interface IAuditable
{
DateTime Timestamp {get;}
string Username {get}
}
Or whatever properties must be commonly available for auditing. Then you can subscribe to that interface and get a copy of every message. Or you could make it an empty interface and just audit message headers. But the messages would need to implement it and publish it to get a copy.
This seems like a generally bad idea, since you're creating copies of the messages all over the place...
Another approach would be to add an observer to message consumption and use that observer to either write to the audit storage or to send a message to an audit queue and let that asynchronous consumer to write to the audit storage.
The thing is, if you're auditing every message, and every message is sending an audit message, make sure you don't observer your audit consumer or you'll die the infinite death.
The observer option is my favorite, since it not only logs the message, but allows the disposition (success/fault) to be captured, as well as the host which consumed the message, processing duration, etc.
MassTransit has build in support for auditing
See this link:
https://masstransit-project.com/advanced/audit.html
So you're better use their built in functionallity instead of creating observers and other hacks
Two main parts need to be saved for each message to provide complete audit:
The message itself
Metadata
Message metadata includes:
Message id
Message type
Context type (Send, Publish or Consume)
Conversation id
Correlation id
Initiator id
Request id (for request/response)
Source address
Destination address
Response address (for request/response)
Fault address
I am finding out that even when my NSB process does not handle messages for say DTOXXX, it is still sending an auto-subscribe message to the publisher queue for DTOXXX.
This is not the desired behavior. I want the process to publish and subscribe to messages for DTOYYY, but any communication using DTOXXX is strictly send only.
If that wasn't clear enough I have 2 assemblies that contains my DTO. I Want to establish a pub/sub bus, but only for assemblies in YYY.dll. As for the DTOs in the other assembly, I want the communication to be done via SEND only (not pub sub).
The problem I am running across is that NSB is sending auto subscribe message to the other process even though:
There is no handler for the DTOs in the XXX assembly. It's being referenced only so that YYY NSB can send messages to XXX NSB.
The communication between the 2 modules are strictly SEND only. This is done to promote low coupling given the actual use case & biz requirement.
How can I set up my modules properly? That is I need to somehow tell NSB, to auto subscribe for messages but only for the ones in a given namespace/assembly.
You can define your own rules for which messages are considered commands/events (or plain messages) by implementing your own DefiningEventsAs in the configure interface. Nsb will only autosubscribe to events. That may help you for your usecase...
There are a couple of ways to handle this, the first being that you can turn of auto-subscribe and subscribe manually. This is done via .DoNotAutoSubscribe() in you endpoint config. From there you will resolve and instance of IBus and then subscribe explicitly to the messages you care about.
The second way is to separate your messages from all other code into different assemblies and only map the events(pub/sub) to the Publisher via the app.config file.
Is There any way I can read raw xml message from MSMQ using NService bus before it parses it to corresponding class object.
I am getting message in Handle(Class obj). This is working fine but I want to log the xml from raw message body which NService parses to class object .
Have a look at transport message mutators. You'll be able to inspect both the Body and Headers of the message
#Chris-Bednarski's answer is correct, but I wanted to add that NServiceBus has auditing built-in which takes the complete messages and passes it on to another queue, so you don't have to do that yourself.
As of version 4, there is another process which feeds off of that queue and persists those messages into RavenDB as well as a UI (called ServiceInsight) that enables you to see everything that flowed through your system. You can find it here:
http://particular.net/ServiceInsight