I have a server process bound to a port that receives network packets at essentially random intervals. When a packet is received it is parsed and an object representing this packet's data is created. I would like to be able to 'push' this data object to any number, 0..n, client processes running on the same machine. The clients will always be on the localhost.
A client process is only interested in data objects created and pushed by the server since it was launched. This is also a one-way only flow of information. Client's do not need to communicate with the server they just need to receive any new data objects from the server.
The server and client processes are both written in C# using the .Net framework.
Given this setup, what method of IPC would you use to get this to work? My current plan is to serialize the data object and write it to a named pipe that clients read from. Is this the way to go? Also worth noting is that speed isn't a critical factor.
I solved this using WCF callbacks. Clients 'subscribe' to the server, the server then iterates over the subscribed callbacks and calls them with the data to be pushed. When a client process is ended it issues an unsubscribe.
There are loads of examples of this on the net that are fairly easy to follow. For anyone interested these links might help.
http://msdn.microsoft.com/en-us/magazine/cc163537.aspx
http://dotnetaddict.dotnetdevelopersjournal.com/wcf_alarmclock.htm
http://idunno.org/archive/2008/05/29/wcf-callbacks-a-beginners-guide.aspx
Related
Scenario: Want to braodcast message to multiple clients from server.
Need: I have a server and multiple clients connected to that server. If Admin want to update server then server will broadcast message to all the clients connected that "Please logout...server is being updated".
Please if any one have any idea regarding the message broadcasting reply asap. I am using TcpServer
Language : c#
For TCP I use a thread that is initialized from the main server protocol thread; when a change occurs to the main data structure, which needs to be sent to each client, a static boolean is set and sends that relevant data to the client, which also has a thread that is implemented off the main client protocol thread to listen and implement those changes.
I use a token passing system to lock access into the data structure, so a client can only make changes when their token is called. I tag each token, so that the static var isn't set to false until each user has received the data to be changed.
This is more a challenge in logic than utilizing a single line of code. Regardless any library interface would have to use similar logic in the sense it must contact (n) number of clients.
In my case, this usage reduces client calls and allows for the passing of only data that has been changed. Unlike an initial load where the entire data structure is passed, but I do something similar that is also connected to traversing the data structure and saving it to hard disk. This means a reduction in data that is passed over the network and keeps the main protocol thread open longer so that those that really need access get it.
edit: so that no one complains, I'm talking straight TCP/IP and not UDP, where the connection needs to be established. UDP does use a multicast concept and setting it up is rather easy. Beware, it is prone to being a security risk, and even though it does provide some efficiency, it's scope of use should be viewed as limited.
I'm very interested in how financial data is streamed from server to client. I often here the term 'push-pull' used. I wondered if someone could give me an example (preferably in Java, C# or perhaps Javascript) how this is actually achieved? Whenever I have written amateur hobby projects at home I often end up querying a URL (containing the price) and continuously calling this within a while(true) loop, with a thread.sleep(x), even if the price doesnt change.
Thanks in advance
Don't know what you mean with 'streaming financial data', but the concept of push/pull is not limited to the financial sector :)
Generally speaking, pull strategy means that the client is actively getting data through a pre-defined communication channel (in your case a socket to an existing and known URL) and polling this channel for new information.
In contrast to that you have the push strategy where you are notified of any changes and you supply the communication channel and register it with the connection's partner. E.g. you have a webservice and your connection partner will post information to that webservice whenever he sees fit. See http://en.wikipedia.org/wiki/Observer_pattern for this concept.
Hope this helps a bit.
If client is working over HTTP the push is always initiated by client, i.e. client requests new updates and server sends them. If client is thin client (i.e. application running in browser) the modern way is to use AJAX to retrieve data without refreshing the page. But again the initiative is at client side, but user just does not see it. It is done on scheduled basis using javascript.
The most "real time" approach is using HTTP tunneling technique: client performs HTTP GET to special URL mapped to servlet that does not close the connection. It just holds it open. When it has something to send to client it writes to stream. So, you get server-to-client push, but still initial connection was performed by client.
You were doing pull. Pulling is when the clients request the data from the server and the server acts on that request.
If the server would have send you data upon receiving new data, that would have been push.
So the difference is: push is initiated by the server and pull is initiated by the client.
Financial data is often transmitted with software like TIBCO Rendezvous. A publisher sends the message to a daemon and listeners that subscribed that subject gets the message from the daemon.
Websockets
EventSource
These are the two web-based PUSH techniques.
As for browser support:
Chrome/Safari/Firefox6 support Both of those.
Opera supports EventSource and Websockets but has the latter disabled by default.
Firefox 4 supports websockets but has it disabled by default.
IE<10 supports neither, IE10 might support one if your lucky
There are many pull techniques, including HTTP and ajax.
I need to create a server process which can push high frequency data (1000 updates per second) to around 50 client. I'm thinking the best way you do this is using async sockets with the SocketAsyncEventArgs type.
The client -> server connections will be long running at least several days to indefinite. I plan to have a server process listening and the clients connect and the server starts pushing the data to the clients.
Can someone point me to or show me an example of how to do this? I can't find any example showing a server process pushing an object to a client.
EDIT: This is over a gigibit LAN. Using windows server with 16 cores and 24gb ram
thanks
First, some more requirements from your side is required. You have server with lots of muscle, but it will fail miserably if you don't do what has to be done.
can the client live without some of the data? I mean, does the stream of the data need to reach other side in proper order, without any drops?
how big is 'the data'? few bytes or?
fact: scheduling interval on windows is 10 msec.
fact: no matter WHEN you send, clients will receive it depending on lots of stuff - network config, number of routers in-between, client processor load, and so on. so you need some kind of timestamping here
Depending on all this, you could design a priority queue with one thread servicing it and sending out UDP datagrams for each client. Also, since (4) is in effect, you can 'clump' some of your data together and have 10 updates per second of 100 data.
If you want to achieve something else, then LAN will be required here with lots of quality network equipment.
If you want to use .NET Sockets to create this server-client project, then this is a good outline of what's needed:
Since the server will be transferring data to several clients simultaneously, you'll need to use the asynchronous Socket.Beginxxx methods or the SocketAsyncEventArgs class.
You'll have clients connect to your server. The server will accept those connections and then add the newly connected client to an internal clients list.
You'll have a thread running within the server, that periodically sends notifications to all sockets in the clients list. If any exceptions/errors occurs while sending data to a socket, then that client is removed from the list.
You'll have to make sure that access to the clients list is synchronized since the server is a multithreaded application.
You don't need to worry about buffering your send data since the TCP stack takes care of that. If you do not want to buffer your data at all (i.e. have the socket send data immediately), then set Socket.NoDelay to true.
It doesn't seem like you need any data from your clients, but if you do, you'd have to make sure your server has a Socket.BeginReceive loop if using Socket.BeginXXX pattern or Socket.ReceiveAsync method if using SocketAsyncEventArgs.
Once you have the connection and transmission of data between server and client going, you then need to worry about serialization and deserialization of objects between client and server.
Serialization which occurs on the server is easy, since you can use the BinaryFormatter or other encoders to encode your object and dump the data onto the socket.
Deserialization on the other hand, which occurs on the client, can be pretty complex because an object can span multiple packets and you can have multiple objects in one packet. You essentially need a way to identify the beginning and end of an object within the stream, so that you can pluck out the object data and deserialize it.
One way to do this is to embed your data in a well known protocol, like HTTP, and send it using that format. Unfortunately, this also means you'd have to write a HTTP parser at the client. Not an easy task.
Another way is to leverage an existing encoding scheme like Google's protocol buffers. This approach would require learning how to use the protocol buffers stack.
You can also embed the data in an XML structure and then have a stream-to-XML decoder on your client side. This is probably the easiest approach but the least efficient.
As you can see, this is not an easy project, but you can get started with the Socket.BeginSend examples here and here, and the SocketAsyncEventArgs example here
Other Tips:
You can improve the reliability of your communication by having the client maintain two connections to the server for redundancy purposes. The reason being that TCP connections take a while to establish, so if one fails, you can still receive data from the other one while the client attempts to reconnect the failed connection.
You can look into using TCPClient class for the client implementation, since it's mostly reading a stream from a network connection.
What about rendezvous or 29 west? It would save reinventing the wheel. Dunno about amqp or zeromq they might work fine too....
I'm using c# to design a client-server application. I'm still a beginner, and am learning the ropes of c# and OO. Right now, I wrote on a piece of paper a few ideas. Essentially, I would create a class "client", which contains all the details (sockets, etc). The client class would be created and stored in an array which would be used by the server in a loop. If 100 clients are connected, would the memory used be significant?
I guess the server would loop through every client in the array checking for a "dataSend" flag that would then flag the server to create a NetworkStream object to the client.
Should I create a networkstream object upon connection of the client and close on connection?
If anyone could point me in the direction of writing my own client-server software, it would be appreciated.
Cam, what you've described isn't quite real client/server design, as the two sides are tightly coupled in your scenario, sharing an array of objects. Think about it instead in terms of requests and responses. The client makes a request to the server over the network, and the server returns a response over the network to the client. They share two things: a common network connection and knowledge of the interface that the server exposes.
The Web is a great, familiar instance of this pattern. The client, your browser, composes an HTTP Request and sends it over a network connection to the server. The server interprets the request and sends an HTTP response back to client. Each knows how to interpret the HTTP standard. That is the link between them, nothing else.
I'd suggest starting out with implementing a very simple request/response. For instance, the client sends a request of 'TIME' and the server responds back with the current time, and the request 'DATE' responds back with the current date. By having a simple protocol to implement, you can concentrate on learning the mechanics of .NET's networking classes.
An array of client classes representing each client connected to the server is a good way to handle this. You might also want to have a member class representing the actual socket and network code, its keeps it cleaner.
Pro tip: Check out the source code to some MUDs.
Cam, we need a bit more detail about what you are trying to achieve to give you a really good answer. Generally, I'd suggest you make use of WCF. You simply need to create a service, define an interface for your operations, and then as many clients can consume that interface as you wish. It's pretty easy in practice.
Something like this would be appropriate:
Like you said, a class Named client to hold client socket.
Maintain a table on server with client's ip:port as key, and client object as value. This will help you keep track of connected clients.
Use asynchronous send and receive for clients. So rather then you iterating through clients, every client will receive data, do the job and respond back to the connected client.
Imagine a WinForms client app that displays fairly complex calculated data fetched from a server app with .Net Remoting over a HTTPChannel.
Since the client app might be running for a whole workday, I need a method to notify the client that new data is available so the user is able to start a reload of the data when he needs to.
Currently I am using remoted .Net events, serializing the event to the client and then rethrowing the event on the side of the client.
I am not very happy with this setup and plan to reimplement it.
Important for me is:
.Net 2.0 based technology
easy of use
low complexity
robust enough to survive a server or client restart still functional
When limited to .Net 2.0, how would you implement such a feature? What technologies / libraries would you use?
I am looking for inspiration on how to attack the problem.
Edit:
The client and server exist in the same organisation, typically a LAN, perhaps a WAN/VPN situation.
This mechanism should only make the client aware that there is new data available. I'd like to keep remoting for getting the actual data to the client since that is working pretty well. MSMQ comes with windows, doesn't it? So it should be ok to use it, but I'm open to any alternative.
I've implemented a similar notification mechanism using MSMQ. The client machine opens a local, public queue, and then advises the server of it's queue name. When changes occur, the server pushes notifications into all the client queues that it's be made aware of. This way the client will know that data is ready, even if it wasn't running when the notification was sent.
The only downside is that it requires MSMQ on the clients, so this may not work if you don't have that kind of control over your client's machines.
For an extra level of redundancy (for example, if a client machine is completely down, and therefore the client queue is unavailable) you could queue notifications on the server prior to dissemination to clients. Notifications in the server queues are only removed when the client is successfully contacted (or perhaps after 3 failed attempts, etc.)
Also in that regard, if the server fails to deliver messages to a client a measured number of times, over a measured period of time, then support entities are notified, error alerts go out, and the client queue is removed from the list of destinations. When I say "measured" I mean a frequency/duration that makes sense to the setting. In my case, it was 5 retries with 5 minute intervals between attempts.
It might also make sense to have the client "renew" it's notification subscription at intervals. If a renewal doesn't occur, then eventually the client queue is removed from the destination list by a "groomer" process in the service.
It sounds as though you need to implement a message-queue based solution. Easy to implement, can survive reboots, and the technology is mature both on the server (MSMQ, MGQSeries) and on the client (System.Messaging)
If you can't find anything built-in and assuming you know the address of all the clients, you could send them a UDP message when data changes. Using UdpClient, this is very easy. The datagram doesn't even need to contain any data if the client app can assume that any UDP data on a certain port means it needs to get new data from the server.
If necessary, you can even make this a broadcast packet (if you don't know who the clients are and they are on the same subnet as the server), so long as the server isn't too "chatty".
Whatever solution you decide on, I would urge you to avoid having the clients poll. This will create a lot of unecessary network traffic and still won't perform all that well.
I would usually use a UI timer on the client to periodically hit the server to see if there was new or updated data. (Assuming you have a mechanism to identify that you have new data like time stamps for new rows, or file time stamps, or a table with last-calculated dates, etc)
That way the server doesn't have to know about the clients. The clients can check at their leisure, etc.