I would like to find a solution to create a pub/sub medium for 2 microservices to talk to each other,
I am aware i can use some third parties E.g Redis, RabbitMQ
Implementing event-based communication between microservices (integration events)
The challenge lies on the client is unable to allow install any third parties tool due to security reason.
The messageQueue server in Windows won't be allowed to use too.
I can only use the applications that is only existed in the server.
Therefore i am asking if there is anyway that i can create one simple app using windows service.
It is a one-to-many relationship. I have one service that will be dealing with data, once if there is any update, it will publish to those services that is subsribed to it.
It seems my problem could be similar with
.NET Scalable Pub/Sub service implementation
WCF Pub/Sub with subscriber caching(link is dead on the WCF pub-sub)
but i dont see any critical solutions.
I was thinking to use data notifications that MSSQL offers as last alternatives, but seems like it could cause a bottle neck when the applications get scale up.
The internet is so much flooded with articles using third parties tool.
Thanks
Check out Rebus library, that allows using different transport methods to send end receive messages in just a line of code (so in the future you can change it without effort).
You could use SQL Server or try to develop your own transport method
Related
We currently have a solution, using web services (implement via WCF) where client software periodically call a service to retrieve a list of new items waiting for them, then for each item, calls a separate service to do the actual download. This is the typical message polling scenario, and is relatively simple in implementation and trouble free. It does not give a near real time messaging solution however. For real time messaging you'd want more of a push notification architecture.
Because of security concerns we do not want clients to expose interfaces that our system can call because then each client needs to secure that exposed interface among other concerns.
Id like to explore the option of the client still initiating the connection to the server as we do today, but instead of polling, we maintain the connection and once the client has completed the connection, the server is now able to push messages to the client.
Our clients base may be implementing their solution on a variety of platforms, from Linux to Windows, .Net to PHP, etc. Additionally, clients vendors have a varying degree of technical capabilities so complexity of the solution is a factor here. Can SOAP or REST services be used in this type of architecture? Are there other technologies I should be looking at? Our server side part of the solution would need to be a .net one.
I have a straightforward, existing ASP.NET MVC web solution. The server-based stuff writes information to a database. I am now going to integrate/synchronize this system with a number of other 3rd-party systems. I want to separate the integration processing from the existing core processing, leaving the existing system as untouched as possible.
My plan is as follows:
whenever a database write occurs on the core system server I will write a message to an MSMQ Queue.
an entirely separate server-based windows service will poll the MSMQ, look at the message and will write messages to one or more 'outbound' sync MSMQ queues.
other windows services will monitor the 'outbound' sync queues, and will talk to the 3rd-party systems as necessary, managing the outbound synchronization.
I have a couple of questions:
Should I have a single windows service doing all this, or should I have several services, one central 'routing' one and one for each 3rd-party system?
Should I use WCF for any of this. Does that buy me anything, given that the 'trigger' for writing to the initial queue is already 'happening' on a server-based process?
Thanks very much.
To answer your questions:
Should I have a single windows service doing all this
Definitely not. What if you want to scale out the routing service, or relocate it?
Should I use WCF
If you have your heart set on msmq then the only advantage WCF gives you is it provides a convenient, proven way to design and host your service endpoints, and an alternative to mucking around in System.Messaging. I would say at this stage it doesn't matter that much.
Does that buy me anything
Not sure what you mean, but as Wiktor says in his post, you could chose not to use vanilla .Net or WCF and choose a service bus type framework such as masstransit or nservicebus.
The benefit here is it abstracts you away from the messaging sub-system so you could in theory move away from msmq in the future to rabbitmq or azure queues.
First, a separate windows service is always safer than any attempt to integrate this with your asp.net runtime.
Second, do not write anything by yourself. Use
http://code.google.com/p/masstransit/
It is straightforward and does everything you need. Reference the library from their nuget package, read some tutorials and you will love it.
I'm at the investigative stage for Workflow Service and WPF.
Having State Machine WF Service hosted in IIS and one or more WPF clients talking to the WF Service sounds reasonable choice so far.
However, although days of reading and research it isn't clear to me what would be the best strategy for tracking the transfer between states from WPF app.
There're numerous samples of tracking participants but most of them are based on One process scenario.
So I am thinking of a structure as below.
A server-side WCF operation that any client calls to register its client-side endpoint
A custom tracking participant that goes through all registered client-side endpoints and sends TrackingRecord at it's Track() method.
Advantage of this approach is that it allows real time update of the states without extra layers like ETW. Another advantage is that it allows decoupling of the logic (or maybe model layer) from the presentation layer.
Can anyone share the opinion over the above structure?
I would also welcome any suggestions for achieving the goal.
[EDIT]
To make my idea above more detailed and clear, below steps would be a typical usage.
1) (WPF client) contains and opens a WCF endpoint for receiving TrackRecords.
2) (WF Service) opens a WCF operation (or a simple WF instance with a Receive message) that registers client-side address to an internal store.
3) (WF Service) a custom tracking participant is created and added that will send TrackingRecord to the registered clients' endpoints.
4) (client) connects to the above service and hands out client-side endpoint mentioned at step 1 and consequently receives TrackingRecords.
[ EDIT 2 ]
To put my goal in simple terms, I'd like to know
1) the most efficient way of tracking the StateMachine's state on a WF Service (IIS) + WPF or any types of client app through TrackingParticipant.
2) if my suggestion can be improved
Meanwhile, I have implemented this and works good so far. I also added MvvM Light framework's messaging feature at the client-side so that it propagate the received message to the models easily.
You might take a look at SignalR instead of trying to force WCF to become a pub/sub platform which is not it's strength. There is an example on my blog with the visual tracking example with the tracking participant split out from the tracking application so it's not all in one process. That blog also has links to two other blogs where similar things were done, but all using a messaging architecture more suitable to events like this.
http://panmanphil.wordpress.com/2012/11/05/slides-and-sample-from-the-chippewa-valley-code-camp/
There is an existing mechanism that wraps a lot of the functionality that you are suggesting (If I understand your needs correctly). If you need to utilize a WCF service to communicate in a bidirectional way (i.e. PUSH data to connected clients) I would suggest leveraging the PollingDuplex Binding.
I have used PollingDuplex in the past with various Silverlight clients to exchange data, and I have read articles like this one describing the steps to produce the same behavior in WPF space.
This approach will automate much of the endpoint registration and tracking logic that you apparently are thinking of doing manually.
I hope this helps.
I have been playing around with pubsub and so far it looks good for what I need (a basic game experiment).
From a Javascript perspective and mobile (via Appcelerator's Titanium) I can really see the value of using pubsub.
However, I need to write a server app in c#/.NET (although open to other ideas) to listen to the subscriber queue I have, and process the messages.. which involves some decision making etc, and then possibly writing another message to the publish queue for example.
So far I have played with RX (Reactive Extensions) for C# which listen on my subscribe channel. So far so good, I see the messages come in, although for now I just wrote a C# console app to test.
My question is would the best way to wait and listen for pubsub subscriber messages be to write a windows service app? or is there another technique more appropriate? obviously at some possible point I might have to scale the server to 2-3 servers, however given the nature of pubsub queue/messaging, I don't see a problem if I had some load-balancing etc.
Any ideas welcome!
Use Service Bus. When cloud is good for you, than Azure Service Bus. When not then nServiceBus. Take a look also for RabbitMQ, it's AMQP framework and is able to do more then pubsub. Also rabbit has multiple clients on multiple platorms. For example one of approaches purely for JavaScript is RabitMQ + Node.js + WebSockets.
All clients and devtools, and articles about RabbitMQ for different platforms and languages are here.
There is also special RabbitMQ binding for .NET, find it here.
NServiceBus PubSub explanation is here. It's .NET service bus, but is not such free as RabbitMQ. Anyway RabbitMQ is platform agnostic.
Any of service buses implementations already has PubSub, that is the reason they exist. Therefore there is no reason to implement, what is already implemented
I have developed a windows service which reads data from a database, the database is populated via a ASP.net MVC application.
I have a requirement to make the service re-load the data in memory by issuing a select query to the database. This re-load will be triggered by the web app. I have thought of a few ways to accomplish this e.g. Remoting, MSMQ, or simply making the service listen on a socket for the reload command.
I am just looking for suggestions as to what would be the best approach to this.
How reliable does the notification has to be? If a notification is lost (lets say the communication pipe has a hickup in a router and drops the socket), will the world end come or is business as usual? If the service is down, do notifications from the web site ned to be queued up for when it starts up, or they can e safely dropped?
The more reliable you need it to be, the more you have to go toward a queued solution (MSMQ). If reliability is not an issue, then you can choose from the mirirad of non-queued solutions (remoting, TCP, UDP broadcast, HTTP call etc).
Do you care at all about security? Do you fear an attacker my ping your 'refresh' to death, causing at least a DoS if not worse? Do you want to authenticate the web site making the 'refresh' call? Do you need privacy of the notifications (ie. encryption)? UDP is more difficult to secure (no session).
Does the solution has to allow for easy deployment, configuration and management on the field (ie. is a standalone, packaged, product) or is a one time deployment that can be fixed 'just-in-time' if something changes?
Withous knowing the details of all these factors, is dififcult to say 'use X'. At least one thing is sure: remoting is sort of obsolete by now.
My recommendation would be to use WCF, because of the ease of changing bindings on-the-fly, so you can test various configurations (TCP, net pipe, http) w/o any code change.
BTW, have you considered using Query Notifications to detect data changes, instead of active notifications from the web site? I reckon this is a shot in the dark, but equivalent active cache support exists on many databases.
Simply host a WCF service inside the Windows Service. You can use netTcpBinding for the binding, which will use binary over TCP/IP. This will be much simpler than sockets, yet easier to develop and maintain.
I'd use standard TCP sockets - this will survive all sorts of moving of components, and minimize configuration issues IMHO.