I need to build a system that is similar to a pub/sub system. It is composed of multiple sub-systems or services running in separate executables or as a Windows Services.
The sub-systems are:
The pub/sub service
A pub/sub service managing communications between the internal sub-systems and the users.
A user can have multiple channels open (A web page connected to a SignalR service, a mobile device connected to a duplex WCF service, etc.).
The service should manage all the channels of an user and be able to send information to them on demand based on topics or specific users.
The service must support multiple transports like SignalR, WCF, or others ...
Worker services
A worker that runs as a Windows Service and sends information to the users using the pub/sub service.
The SignalR and WCF host
The SignalR service and WCF service will be hosted on IIS
My questions are
As the sub-systems run in separate processes, how do I communicate between the pub/sub service and the other sub-systems like (the workers and IIS). The communication must be really fast. Do I use named-pipes, is it fast enough ?
An example; The worker tells the pub/sub system to send a message to a user, the pub/sub systems checks the channels opened for the user (let's say a SignalR channel), then in turn it must notify the SignalR service running in IIS to send the message to the user's browser.
Do you know of implementations of similar systems ?
Observations
I cannot use third-party service-bus services (Azure ..). And even with that .. I can't see a solutions to the problems above.
The service must be very scalable and high-demand proof.
If the question is how to bridge SignalR with other transports there are several solutions.
On a single server you could just connect them up with the Reactive framework's own pubsub mechanism which is neatly encapsulated in the Subject class.
If you need to scale out to multiple servers you'll want to use an existing service bus or perhaps roll your own simple one using SQL server and a SqlDependency.
You could also use SignalR as a client on one server communicating with the other servers to copy messages between them.
I recommend you look into some of the existing Service Bus technologies for .NET.
There is an article which clearly explains the possible mechanism of how to incorporate a pub/sub design pattern in your .NET application. The answer lies in using a .NET In-Memory distributed cache and use its clustering capabilities as a publish subscribe medium. Since it's clustered therefore you won't have to worry about down-times as well.
Basically you'll be using the Application Initiated Custom Events
Register your events
public void OnApplicationEvent(object notifId, object data)
{
...
}
_cache.CustomEvent += new CustomEventCallback(this.OnApplicationEvent);
And fire those events whenever you need to
_cache.RaiseCustomEvent("NotificationID", DateTime.Now);
Pub/Sub design pattern in a .NET distributed cache
Full disclosure: I work for Alachisoft
Related
I would like to find a solution to create a pub/sub medium for 2 microservices to talk to each other,
I am aware i can use some third parties E.g Redis, RabbitMQ
Implementing event-based communication between microservices (integration events)
The challenge lies on the client is unable to allow install any third parties tool due to security reason.
The messageQueue server in Windows won't be allowed to use too.
I can only use the applications that is only existed in the server.
Therefore i am asking if there is anyway that i can create one simple app using windows service.
It is a one-to-many relationship. I have one service that will be dealing with data, once if there is any update, it will publish to those services that is subsribed to it.
It seems my problem could be similar with
.NET Scalable Pub/Sub service implementation
WCF Pub/Sub with subscriber caching(link is dead on the WCF pub-sub)
but i dont see any critical solutions.
I was thinking to use data notifications that MSSQL offers as last alternatives, but seems like it could cause a bottle neck when the applications get scale up.
The internet is so much flooded with articles using third parties tool.
Thanks
Check out Rebus library, that allows using different transport methods to send end receive messages in just a line of code (so in the future you can change it without effort).
You could use SQL Server or try to develop your own transport method
I have an ASP.NET MVC application hosted in Azure.
This application is complemented with a desktop application that also has WCF services for communicating with III party interfaces. WCF are hosted locally.
There are thousands of clients using the desktop application at different geographical locations.
Till now, every desktop application used to talk to web app using api with the help of WCF.
This was limited to on demand from the desktop application.
Whenever desktop application feels the need to talk to web app, it used the way of web api from WCF.
Now, what I want is:-
To access the different desktop applications(typically called sites), from azure depending upon the need.
This is required on account of an online ordering system that is through web app/mobile app.
I do not want to keep polling from desktop application to know about if any new order is there for this site.
I feel it would be better if I can play from other side.
Also, keeping in mind that IP of sites will not be fixed. There may be issue with firewall. NAT may translate resource identifier differently.
Can service bus in azure may be of any help, but what confuses me is that every desktop application is having its own WCF service and order should reach the respective site only.
Any type of ideas on this would be appreciated.
According to your description, Service Bus messaging is a perfect way to achieve this.
More information about Service Bus Messaging, we can refer to: Service Bus queues, topics, and subscriptions
In addition, We can also use RabbitMQ or ZeroMQ which is similar with Service Bus Messaging because both of them are free. You can choose an best way to realize your requirements.
About differences between ZeroMQ and RabbitMQ:
ZeroMQ has better performance, but it is built in the case of allowing message data loss to apply to high throughput / low latency applications. Unlike ZeroMQ, RabbitMQ fully implements the AMQP protocol, which is similar to mailbox services, supporting message persistence, transaction, congestion control, load balancing and so on, making RabbitMQ have a more extensive application scenario.
Function RabbitMQ ZeroMQ
Message persistence Support Not Support
Transaction Support Not Support
performance Low High
stability High Low
Support for AMQP protocol Support Not Support
Application scenario Data loss is not allowed High throughput
More information about RabbitMQ and ZeroMQ, we can refer to:
RabbitMQ
ZeroMQ
If you are able to modify the desktop applications, implementing a websockets connection with SignalR might be worth a look. The desktop applications sign up with a SignalR hub you provide.
You can then push data to the clients from, for example an ASP.NET MVC app. It works very reliable and handles lots of connections well. It is typically used for realtime web communication but might be useful in your case, too.
The downside is probably, that the desktop app needs to initially sign up to a hub to receive push messages.
I have a website running as an Azure Web App which is configured to use SignalR and the Azure Service Bus scaleout backplane. All working well, and clients connect via the /signalr URL and I can push data from the server to the connected clients.
I also have several console apps which periodically run and output new data. I would also like to push the new data to the web-connected clients.
How can I tie in my non-web applications with SignalR? All the examples I see assume every server in the scaleout cluster are web-facing. Is there some special setup I need to do to have external processes join the SignalR cluster and be able to push data to web-connected clients over the SignalR websocket connection without having to make them web apps as well?
There are two options.
First Option
You can connect your "console apps" to the signalR backplane, and these apps will be able to call client's methods with something like
var hub = GlobalHost.ConnectionManager.GetHubContext<MyHub>();
hub.Clients.All.doSomething("blah-blah");
You can use any port for the signalR config but same backplane configuration and same Hubs. You can use Owin for self hosting. Nobody will connect to these console apps.
Disadvantages:
1) Your console applications needs to reference SignalR.SelfHosting and your hubs classes. This is not good from architecture point of view.
2) Your console apps will listen some ports (since they're signalR servers). Theoretically somebody could connect to this hubs and do something. This is not good neither from architecture point of view no from security.
Second Option
You can implement Hub method like "PushSomeDataToClients" and call it from your console apps using SignalR client as mentioned in #bartbje comment.
Pros: no disadvantages from the first option.
Cons: You need to implement some security stuff to prevent anybody outside the system to call this method. SignalR has a lot of stuff to do this, so just google. For example you can create separate Hub for inter-system communications.
Third Option
Interact with web-server apps with another way then SignalR. Probably you're already using something like rabbitMq or any type of service bus. Also you can implement it using separate ApiController at your web-server app. But it seems to be closer to the second option.
Due to me, I would probably choose the third option since it's clean from architecture point of view.
I have a client server setup where i am sending the messages to the server over http(I am using WCF service hosted as windows service.)In my current setup as I have seen some messages are lost when the connection is down between client and server ,for the sake of reliability i have decided to use MSMQ.So the client sends the messages to the queue and the server continuously polls the queue I need some design decision to be taken before i will developing.
Which one would be the best bet windows service or wcf service(hosted as windows service) ?
Are there any advantages apart from windows service which WCF has if i have to develop such a service where all it has to do is continuusly read messages from the queue and does some processing.I would be using a private queue which is transnactional .
I would suggest avoiding writing code that directly integrates with MSMQ and instead using an existing service bus to do that for you (like NServiceBus or MassTransit). Rolling-your-own messaging layer may work for the simplest of systems but as requirements change over time you will need a fuller featured service bus. The service buses that I mentioned do the following things in a developer friendly way:
Messaging patterns (fire-and-forget, request-response, publish subscribe)
Message serialization
Message routing
Failure/Retry logic (i.e. a message handler is supposed to update a database, but the database is down how do you handle this?)
Long running processes (also called sagas)
These are just a few of the things you will be writing before long if you go the roll-your-own route.
I need multiple clients that talk to a WCF service. The WCF service also must be able to connect to any one of the clients also.
So - it sounds like the server and the clients need to have both a WCF server and client built into each one.
Is this correct or is there some way to do this?
I was looking at NetPeerTcpBinding, but that is obsolete. To be fair I'm not sure if that is a valid solution either.
Background:
I plan to have a Windows service installed on hundreds of machines in our network with a WCF service and a WCF client built in.
I will have one Windows service installed on a server with a WCF service and a client built in.
I will have a Windows Forms application
I will have a database
The clients on the network will connect to the service running on the server in order to insert some information on the database.
The user will use the Windows Forms application to connect to the Windows service on the server and this Windows service will connect to the relevant client on the factory floor (to allow remote browsing of files and folders).
Hence I believe the machines on the floor and the server both require a WCF cleint and service built in.
The reason people are recommending wsHttpDualBinding is because it is in itself a secure and interoperable binding that is designed for use with duplex service contracts that allows both services and clients to send and receive messages.
The type of communication mentioned 'duplex' has several variations. Half and Full are the simplest.
Half Duplex: Works like a walkie-talkie, one person may speak at any given time.
Full Duplex: Like a phone, any person may speak at any given time.
Each will introduce a benefit and a problem, they also provide ways to build this communication more effectively based upon your needs.
I'm slightly confused, but I'll attempt to clarify.
You have an assortment of approaches that may occur here, a Windows Communication Foundation (WCF) Service requires the following:
Address
Binding
Contract
Those are essentially the "ABC's" for WCF. The creation of those depicts a picture like this:
As you can see the Service will contain:
Host
Service
Client
The host houses the service which the client will consume so those service methods perform a desired task. An example representation:
As you see Client-1 is going through the Internet (HTTP, HTTPS, etc.) then will hit the Host, which will have the service perform those tasks.
Now Client-n is consuming the service locally, so it is talking over (TCP, etc.) as an example.
The easiest way to remember: One service can be consumed by however many clients require those methods to perform a task. You can create very complex models using a service-oriented architecture (SOA).
All WCF is, is a mean to connect your application to a host or
centralized location you may not have access to.
As you can see in the above image, the Client communicates through a Service to the Host. Which performs a series of task. WCF will talk over an array of protocols. Hopefully this will provide a better understanding of how WCF is structured.
There are a lot of tutorials and even post to get you started. Some excellent books such as "WCF Step by Step".
Essentially your looking for an asynchronous full duplex connection, or a synchronous full duplex service. As mentioned above, your task in essence is the point of a Service.
The question: How does this work best?
It will boil down to your design. There are limitations and structures that you will need to adhere to to truly optimize it for your goal.
Such obstacles may be:
Server Load
Communication Path
Security
Multiple Clients Altering UI / Same Data
Etc.
The list continues and continues. I'd really look up tutorials or a few books on WCF. Here are a few:
WCF Step by Step
WCF Multi-Tier Development
WCF Service Development
They will help you work with the service structure to adhere to your desired goal.
Remember the "ABCs" for the most success with WCF.
Use wsDualHttpBinding if you want your service communicate with your clients.
Read WS Dual HTTP.
You might want to try out creating a WCF service using netTcpBinding. It will work for your requirements. You can use the article How to: Use netTcpBinding with Windows Authentication and Transport Security in WCF Calling from Windows Forms as a start:
Also, there are many examples included within the WCF Samples package which you can use.