I have a client server setup where i am sending the messages to the server over http(I am using WCF service hosted as windows service.)In my current setup as I have seen some messages are lost when the connection is down between client and server ,for the sake of reliability i have decided to use MSMQ.So the client sends the messages to the queue and the server continuously polls the queue I need some design decision to be taken before i will developing.
Which one would be the best bet windows service or wcf service(hosted as windows service) ?
Are there any advantages apart from windows service which WCF has if i have to develop such a service where all it has to do is continuusly read messages from the queue and does some processing.I would be using a private queue which is transnactional .
I would suggest avoiding writing code that directly integrates with MSMQ and instead using an existing service bus to do that for you (like NServiceBus or MassTransit). Rolling-your-own messaging layer may work for the simplest of systems but as requirements change over time you will need a fuller featured service bus. The service buses that I mentioned do the following things in a developer friendly way:
Messaging patterns (fire-and-forget, request-response, publish subscribe)
Message serialization
Message routing
Failure/Retry logic (i.e. a message handler is supposed to update a database, but the database is down how do you handle this?)
Long running processes (also called sagas)
These are just a few of the things you will be writing before long if you go the roll-your-own route.
Related
There is an async method in WPF viewmodel
var asyncTask = await Service.Execute();
Execute method is a method of a microservice, which is executed asynchronyously.
Is there any way to send the task status of execute method from microservice to WPF viewmodel ?
There are a couple of options, but it all depends a bit on your setup.
The idea is to have a callback to your application, one way or the other. This can be a direct response to the application, the application listening to a shared notification system or by polling.
Remote Procedure Call (RPC)
WebSockets / notification system
Polling
Message queuing like MSMQ, RabbitMQ, SQS, Azure Service Bus
These methods all have pro's and cons. Depending on the requirements you need to pick a suited one.
Let me elaborate.
Remote Procedure Call (RPC)
In this setup the micro service (MS) needs to to communicate back to the UI application. This usually is a bad idea, because you need to make the MS aware of the Client application - and also, you'll need connectivity inbound to the client application.
The Client applicationto host RPC endpoints to make this work.
Normally this is not really an option.
WebSockets / notification system
Basically this is a similar method as above. However WebSockets or a notification system like Azure or AWS SNS is likely to overcome the burden of the incoming connection.
I'll add some documentation for some libraries.
Polling
This is by far the easiest to implement Client side, but harder in MS side.
Since it's a long running task, after initiation on could send back the ID of the job, (in a HTTP ACCEPTED way (assuming you use HTTP to interact with the MS), and periodically request the status of the given job at the MS.
Message queuing like MSMQ, RabbitMQ, SQS, Azure Service Bus
This is unlikely, but I want to name it for completeness.
Often MS inter process communication is done over a Message Queuing (MQ) mechanism. You could hook up the application onto this MQ system and let it listen to an event/message broadcasted by the MS.
I'd like to make an informed choice towards a simple publish/subscribe architecture.
So I'm wondering: what does the Service Bus add that MSMQ can't do?
What are the drawbacks of the Service Bus?
Thx for enlightening me!
The main functional difference is Service Bus provides out of the box support for message exchange semantics such as topic based routing via publish-subscribe.
MSMQ on the other hand is a lightweight store-and-forward queuing system, which supports point-to-point one way messaging.
Service Bus:
depends on SQL Server, and
is a broker. This may be considered a drawback.
If you are looking at pub-sub frameworks then a popular one at the moment (free in single threaded mode) is NServiceBus, which sits on top of MSMQ, though has swap-able transport.
Pros
Service Bus allows you to publish over tcp and http which is cool, and gives you greater
decoupling.
Service Bus is a sql database so your Disaster Recovery is WAY simpler and a lot cheaper to implement.
Cons
Service Bus is centralised, and MSMQ is federated, so potentially more scalable. Although you can scale out with more nodes in WSB.
You need a live connection to the central bus before you can publish. So MSMQ being federated (on each machine) makes it more available to clients.
However people are using MSMQ as a local store with Service Bus, so publish locally, then push it over to the bus when a connection is available.
We are having a good experience with Service Bus instead of MSMQ at the moment.
I need to build a system that is similar to a pub/sub system. It is composed of multiple sub-systems or services running in separate executables or as a Windows Services.
The sub-systems are:
The pub/sub service
A pub/sub service managing communications between the internal sub-systems and the users.
A user can have multiple channels open (A web page connected to a SignalR service, a mobile device connected to a duplex WCF service, etc.).
The service should manage all the channels of an user and be able to send information to them on demand based on topics or specific users.
The service must support multiple transports like SignalR, WCF, or others ...
Worker services
A worker that runs as a Windows Service and sends information to the users using the pub/sub service.
The SignalR and WCF host
The SignalR service and WCF service will be hosted on IIS
My questions are
As the sub-systems run in separate processes, how do I communicate between the pub/sub service and the other sub-systems like (the workers and IIS). The communication must be really fast. Do I use named-pipes, is it fast enough ?
An example; The worker tells the pub/sub system to send a message to a user, the pub/sub systems checks the channels opened for the user (let's say a SignalR channel), then in turn it must notify the SignalR service running in IIS to send the message to the user's browser.
Do you know of implementations of similar systems ?
Observations
I cannot use third-party service-bus services (Azure ..). And even with that .. I can't see a solutions to the problems above.
The service must be very scalable and high-demand proof.
If the question is how to bridge SignalR with other transports there are several solutions.
On a single server you could just connect them up with the Reactive framework's own pubsub mechanism which is neatly encapsulated in the Subject class.
If you need to scale out to multiple servers you'll want to use an existing service bus or perhaps roll your own simple one using SQL server and a SqlDependency.
You could also use SignalR as a client on one server communicating with the other servers to copy messages between them.
I recommend you look into some of the existing Service Bus technologies for .NET.
There is an article which clearly explains the possible mechanism of how to incorporate a pub/sub design pattern in your .NET application. The answer lies in using a .NET In-Memory distributed cache and use its clustering capabilities as a publish subscribe medium. Since it's clustered therefore you won't have to worry about down-times as well.
Basically you'll be using the Application Initiated Custom Events
Register your events
public void OnApplicationEvent(object notifId, object data)
{
...
}
_cache.CustomEvent += new CustomEventCallback(this.OnApplicationEvent);
And fire those events whenever you need to
_cache.RaiseCustomEvent("NotificationID", DateTime.Now);
Pub/Sub design pattern in a .NET distributed cache
Full disclosure: I work for Alachisoft
Suppose, I want to scale out (add more boxes) some WCF service. This looks pretty easy, set up load balancer that calls WCF services on multiple boxes using for example round robin algorithm.
However how to deal with situation when a WCF service have callback contract. When a client connects to some particular box, it receives events only raised by this computer WCF service instance. And I want client to receive events that were raised by any WCF service instance in group (cluster).
What is the best way to make WCF service know about events raised by other WCF service instances?
Some ideas: Multicast, broadcast, WCF NetPeerTcpBinding, Single server that subscribes to all WCF services in cluster (acting as event aggregate).
UPDATE: I have managed to create test system, using NetPeerTCPBinding as a mechanism to share events across servers. I haven't made a benchmark yet, but I feel that WCF P2P is to heavy for this tusk, I'm gonna implement UDP broadcast based event sharing system.
I would implement this by setting up a MSMQ queue that each server can subscribe to, and when an event occurs that the other servers need to know about, the service can publish it.
I use a library called NServiceBus to make this entire process simple. NServiceBus is a full-featured library that uses MSMQ (among other transports) to create pub/sub messaging buses, which would exactly solve your problem. It is easy to use and has a fluent interface for configuration, subscription, and publishing.
I will come back and edit this post later with an example, but the NServiceBus website has plenty of documentation to get you started until then.
Have you considered messaging? Sounds ideal.
I haven't been able to find a clear answer to this problem. Is there a good way to subscribe to a MSMQ through the internet? Ideally I need security both in authentication and encryption for this connection. But I would like the subscriber to act just like any other client that would be subscribed on the local network. I believe I have a couple of options here
Expose the MSMQ ports publicly
Put the MSMQ behind some type of WCF service (not sure if that works for a subscriber)
What other options do I have? We're sitting in a .NET environment and the main problem domain that is trying to be solved is to change the remote connections from a pulling system to an event based system to reduce the load on the main server.
One way is to use a queue ON the Internet.
I work at Microsoft and my team owns MSMQ and we also own the Windows Azure Service Bus service. For the scenario you describe you may want to take a look at using a Service Bus Queue, which has not only the advantage of being reachable for Internet senders but also eliminates the need to create inbound firewall rules on the receive side.
More here: http://www.windowsazure.com/en-us/develop/net/how-to-guides/service-bus-queues/
The most natural option will be to use MSMQ over http, which is a feature of MSMQ:
http://msdn.microsoft.com/en-us/magazine/cc164041.aspx
The alternative would be to create an http WCF service possibly with duplex polling and use WS-Routing to an MSMQ WCF service.
Checkout the Gateway feature of NServiceBus.