I have been playing around with pubsub and so far it looks good for what I need (a basic game experiment).
From a Javascript perspective and mobile (via Appcelerator's Titanium) I can really see the value of using pubsub.
However, I need to write a server app in c#/.NET (although open to other ideas) to listen to the subscriber queue I have, and process the messages.. which involves some decision making etc, and then possibly writing another message to the publish queue for example.
So far I have played with RX (Reactive Extensions) for C# which listen on my subscribe channel. So far so good, I see the messages come in, although for now I just wrote a C# console app to test.
My question is would the best way to wait and listen for pubsub subscriber messages be to write a windows service app? or is there another technique more appropriate? obviously at some possible point I might have to scale the server to 2-3 servers, however given the nature of pubsub queue/messaging, I don't see a problem if I had some load-balancing etc.
Any ideas welcome!
Use Service Bus. When cloud is good for you, than Azure Service Bus. When not then nServiceBus. Take a look also for RabbitMQ, it's AMQP framework and is able to do more then pubsub. Also rabbit has multiple clients on multiple platorms. For example one of approaches purely for JavaScript is RabitMQ + Node.js + WebSockets.
All clients and devtools, and articles about RabbitMQ for different platforms and languages are here.
There is also special RabbitMQ binding for .NET, find it here.
NServiceBus PubSub explanation is here. It's .NET service bus, but is not such free as RabbitMQ. Anyway RabbitMQ is platform agnostic.
Any of service buses implementations already has PubSub, that is the reason they exist. Therefore there is no reason to implement, what is already implemented
Related
RabbitMQ allows for 'Quorum Queues'. As far as I have read in the documentation, 'quorum' queues allow queues to be replicated on all nodes within a rabbit cluster, whereas 'classic' queues host a specific queue on a specific node. I understand that there will be a higher latency when using 'quorum' over 'classic' queues.
I use ServiceStack to talk to RabbitMQ. The exchanges and the queues are created automatically - based around my requests and responses, and this all works well.
I am writing software for use in a highly available environment - I am writing C# code, using .NET 6 in a Linux environment (docker containers running in K8s), and am using ServiceStack 6.0.2. I would like to use 'quorum' queues rather than 'classic' queues if possible to help prevent message loss if one of the rabbit nodes in the cluster goes down.
Is it possible for ServiceStack to create 'quorum' queues? Having read through the documentation, searched SO, searched the ServiceStack forums, general web searching and experimentation in a stand-alone application, I can find no obvious way of creating these types of queues automatically via ServiceStack. By the looks of it, the queues are registered with various features, but always seem to be created as 'classic' queues.
Furthermore, will there be any problem with using ServiceStack and 'quorum' queues? The RabbitMQ documentation suggests that A client library that can use regular mirrored queues will be able to use quorum queues., but I am unclear if this is the case with ServiceStack.
No ServiceStack doesn't support creating Rabbit MQ Quorum Queues.
ServiceStack MQ is a messaging abstraction over multiple MQ implementations to enable alternative Reply and OneWay endpoints for invoking your Services.
You'll need to utilize the MQ libraries directly when you need additional MQ-specific features beyond this.
I have an application written in Python (PIKA) and C# (official NuGet package). Those applications are publishing new messages into RabbitMQ queues.
Until now, I used this syntax to publish a new message into the queue:
model.BasicPublish(exchange, routingKey, basicProperties, body);
I found that BasicPublish function always returns with success. I also read in RabbitMQ documentation that in case of broker destroyed, the messages that didn't send yet will be removed without sending it to RabbitMQ.
I want to avoid the loss of messages. So, I found 3 options to submit those messages to publish:
Transaction - Very slow.
Confirmation - I found it tricky to implement in a multi-threaded environment.
with REST API - What do you think about that?
I think that it will be ideal for me yo use REST API for inserting messages into queues.
The Question:
The way that I found to send a message with API is to send POST message to this endpoint:
http://localhost:15672/api/exchanges/vhost/amq.default/publish
As you can see, this port (15672) belongs to the RabbitMQ management system.
Is this the right way to use RabbitMQ with REST API?
Can I trust the RabbitMQ management system in the production environment?
Can you recommend an alternative to REST API that will accept to message enqueue immediately after insertion (blocking)?
No, don't use the HTTP API. It is not intended for production use for publishing or consuming messages.
You must use publisher confirms. The technique described in this tutorial applies to the .NET client library as well.
You could also investigate libraries written on top of the official .NET library that may correctly implement publisher confirms for you. EasyNetQ is one such library.
Another good resource to read with concern to 100% reliability is this blog post.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
I would like to find a solution to create a pub/sub medium for 2 microservices to talk to each other,
I am aware i can use some third parties E.g Redis, RabbitMQ
Implementing event-based communication between microservices (integration events)
The challenge lies on the client is unable to allow install any third parties tool due to security reason.
The messageQueue server in Windows won't be allowed to use too.
I can only use the applications that is only existed in the server.
Therefore i am asking if there is anyway that i can create one simple app using windows service.
It is a one-to-many relationship. I have one service that will be dealing with data, once if there is any update, it will publish to those services that is subsribed to it.
It seems my problem could be similar with
.NET Scalable Pub/Sub service implementation
WCF Pub/Sub with subscriber caching(link is dead on the WCF pub-sub)
but i dont see any critical solutions.
I was thinking to use data notifications that MSSQL offers as last alternatives, but seems like it could cause a bottle neck when the applications get scale up.
The internet is so much flooded with articles using third parties tool.
Thanks
Check out Rebus library, that allows using different transport methods to send end receive messages in just a line of code (so in the future you can change it without effort).
You could use SQL Server or try to develop your own transport method
I have a straightforward, existing ASP.NET MVC web solution. The server-based stuff writes information to a database. I am now going to integrate/synchronize this system with a number of other 3rd-party systems. I want to separate the integration processing from the existing core processing, leaving the existing system as untouched as possible.
My plan is as follows:
whenever a database write occurs on the core system server I will write a message to an MSMQ Queue.
an entirely separate server-based windows service will poll the MSMQ, look at the message and will write messages to one or more 'outbound' sync MSMQ queues.
other windows services will monitor the 'outbound' sync queues, and will talk to the 3rd-party systems as necessary, managing the outbound synchronization.
I have a couple of questions:
Should I have a single windows service doing all this, or should I have several services, one central 'routing' one and one for each 3rd-party system?
Should I use WCF for any of this. Does that buy me anything, given that the 'trigger' for writing to the initial queue is already 'happening' on a server-based process?
Thanks very much.
To answer your questions:
Should I have a single windows service doing all this
Definitely not. What if you want to scale out the routing service, or relocate it?
Should I use WCF
If you have your heart set on msmq then the only advantage WCF gives you is it provides a convenient, proven way to design and host your service endpoints, and an alternative to mucking around in System.Messaging. I would say at this stage it doesn't matter that much.
Does that buy me anything
Not sure what you mean, but as Wiktor says in his post, you could chose not to use vanilla .Net or WCF and choose a service bus type framework such as masstransit or nservicebus.
The benefit here is it abstracts you away from the messaging sub-system so you could in theory move away from msmq in the future to rabbitmq or azure queues.
First, a separate windows service is always safer than any attempt to integrate this with your asp.net runtime.
Second, do not write anything by yourself. Use
http://code.google.com/p/masstransit/
It is straightforward and does everything you need. Reference the library from their nuget package, read some tutorials and you will love it.
Is it recommended to use NServiceBus (or any service bus library) solely for the purpose of publishing messages on the client side?
I've been looking at a handful of sample open source projects and they all seem to have one thing in common. All projects are publishing messages and getting consumers to handle the specific message (or command).
Essentially, I'm looking to decouple my actions by sending out messages and have a handler take care of it. This would all be done locally on the client.
I think NServiceBus and equivalent tools are too heavy on the client side and often requires extensive configuration.