I need to connect to thousands of clients over TCP on a proprietary protocol to acquire data cyclically. I need to write a .NET server application in C#.
The first attempt was to create for each tcp socket an own thread, which works but needs a lot of cpu usage.
I found out that it would be a better idea to use the .NET threadpool instead. As far as I understand (http://msdn.microsoft.com/en-us/library/ms973903.aspx) I could use timers in order to get each socket acquire the data cyclically in a given period (like 1 sec). This does not work for me because the sockets time out once the connection was openende because there are a lot of more sockets which have to be opened before it's the open sockets turn again.
Another try was using asynchronous callbacks. This would work for me but I don't know how to get the sockets acquire data cyclically???
Try using Socket's high performance API which allows simultaneously receiving data on a very large number of sockets, without using one thread per socket. At the bottom of the article there's a link to a complete sample. There's also a sample in the MSDN article for the SocketAsyncEventArgs class.
Why not populate a queue with the addresses you need to poll and have your thread pool take items off the queue to process?
Once you are done with an item push it to the back of the queue.
Related
I have an application which connects to a third party server let’s call it Server-A. I have been given four different ports i.e.
4000, 40001, 40002, 40003. On each port I can create 20 connections so I can create 80 total connections with server-A. I want to create a service layer that should communicate with server-A on mentioned ports. The technology will be asp.net C#.
The problem statement
1- Application should be non-blocking/asynchronous to entertain 10 to 20 million request per day
2- Whenever the service layer starts it create 20 connections on each port. (Total 80 connections)
2- All connections should remain connected/alive 24/7 and reconnect whenever any connections drops/disconnects. It will send a heartbeat message in idle time.
My Questions
How can I manage these connection? Should I add those to a static list one by one when a TCP socket is successful?
How can I know that a certain connection is dropped/disconnected?
How can I send certain requests on different ports? Let’s say if a>b send it on port 4000 else if a<=b send it on 4001
How can I make it asynchronous?
For an initial start I created a single TCP connection on single port and it works as expected. Then I replicated the same code for other port, but I know it is very bad approach and I have to copy same code 80 times to make 80 connections. I want a clean and scalable way to achieve it, so that in future may be I increase the connection to 100 or more.
Is there any framework which I can use?
Any help would be greatly appraised.
#Kartoos Khan, i have made some services with those requirements and using asynchronous methods is the best way to create high performance services in C#, because:
It does not block IO peripherals, as can be sockets.
Minimize the threads and improve the performance to it.
Let me recommend you the book Writing High-Performance .NET Code. The chapter 4, Asynchronous Programming has the information that you need to know to improve the performance.
In my experience those are my recommendations:
Create a main threat to handle the main program.
Create a class to handle the Socket Server, which implements an asynchronous process to accept connections, using the methods BeginAccept and EndAccept, here is a sample of how to use it.
Create another class to handled the socket connections, which has as a property the Socket object.
2.1 create a method to start the Reading process, which will be called by the Server class to start the communication between the endpoints. This methos will start the process of read in an asynchronous way.
2.2 To read and write in an asyncrhonous way, it is necessary to get the NetworkStream from the socket, and use the methods BeginRead and EndRead, to receive data, and BegineWrite and EndWrite to send data. Here there is the documentation.
In the case that your service only needs to connect to a Server, ignore the step 1and implement the Client class to start the connection to an specific EndPoint.
Use a collection class, as can be a Dictionary, Key-Value-Pair collection, to store each Client Class and use the socket ID as the key to access to each Client Class.
Due each Client Socket handles it own socket, i use to implements a way to reconnect at the same Client Socket, i this way each Client is responsable for itself.
The main program will be responsable to create each Client Server and set the EndPoint of each client, as you need, and start to connect each of them. In this case, TCPClient allow you begin an asynchronous process for connect, using the methods BeginConnect and EndConnect.
Here you can see more details about this issue.
I hope this might be useful for you.
To handle such a large volume of traffic you need to do a few things.
Assumptions
You are connecting to another client’s server.
You have a large volume of web traffic from either multiple machines or from multiple working processes on any given machine.
You know how to create TCP client server objects and handle the connections.
For less than 80 worker threads across your servers:
Because each thread processes synchronously, you only need to use a single connection for each thread.
If no single web server is running more than 20 worker processes, then you can designate a single port for each server to use. Stick the port in your web.config file as a variable and use that when creating connections. You will never hit the limit.
Store your connection in a shared object that the entire app can use (could put this in your BLL layer) and if you have a connection error, re-create a new connection on that thread.
For more than 80 worker threads across your servers:
Do the same as the last step but at this point you either need to negotiate for more connections or you will add a new layer in between your application and the server you wish to reach.
This second layer acts as a broker for the two sides and can manage a pool of connections instead and draws off a connection each time you need to access Server-A and puts it back into the pool when finished.
Anytime you connect to the broker application, spawn a new thread to do the processing until the connection is dropped or closed.
Keep track of your open connections and viola you can have as many clients as you need but your bottleneck will be those 80 connections out even if you have hundreds or thousands in.
Good day all!
I am working on a open source server for a game that is closed source - the game operates using TCP/IP sockets (instead of UDP, doh...) So being a connection based protocal, I am contrained to work with this.
My current program structure (dumbed down):
Core Thread
Receive new connection and create a new client object.
Client Object
IOloop (runs on its own thread)
Get data from socket, process packets. (one packet at a time)
Send data buffered from other threads (one packet at a time)
The client will send data immediately (no delay) when it is it's own thread.
I am noticing a huge flaw with the program, mainly it sends data very slow. Because I send data packet by packet, synchronously. (Socket.Send(byte[] buffer))
I would like to send data almost immediately, without delay - asynchronously.
I have tried creating a new thread every time I wanted to send a packet (so each packet sends on it's own managed thread) but this was a huge mess.
My current system uses a synchronous sending with nagle algorithm disabled - but this has the flaw of bottlenecking - send one packet, send operation blocks until TCP to confirms, then send the next... I can issue easily 10 packets every 100ms, and if the packets take 400ms to send, this backs up and breaks. Of course on my Local host I don't get this issue.
So, my question: What would be the best way to send multiple small packets of data? I am thinking of merging the data to send at the end of every IO thread loop to one large byte buffer, to reduce the back and forth delay - but the obvious problem here is this undermines the nagle algorithm avoidance which I had hoped would stop the delays.
How does a synchronous send work? Does it block until the data is confirmed correctly received by the recipient as I believe? Is there a way I can do this without waiting for confirmation? I do understand the packets must all go in order and correctly (as per the protocol specification).
I have done some research and I'm now using the current system:
Async send/receive operations with no threading, using AsyncCallbacks.
So I have the server start a read operation, then callback until done, when a packet is received, it is processed, then a new read operation will begin...
This method drastically reduces system overheads - thread pools, memory etc.
I have used some methods from the following and customised to my liking: https://msdn.microsoft.com/en-us/library/bew39x2a(v=vs.110).aspx
Very efficient, highly recommend this approach. For any TCP/IP network application low lag is very important and async callbacks is the best way to do it.
i have a TCP server software. I open a thread for every TCP client. (This is not important why i open a thread for every TCP client.) I want to kick a TCP client any time and want to kill its thread. How can i find the thread of TCP client that i want to kick out of server?
Thank you.
You need to keep track of all TcpClient's in existence. For example, use a list to store them. That way you have access to them.
I need to create a server process which can push high frequency data (1000 updates per second) to around 50 client. I'm thinking the best way you do this is using async sockets with the SocketAsyncEventArgs type.
The client -> server connections will be long running at least several days to indefinite. I plan to have a server process listening and the clients connect and the server starts pushing the data to the clients.
Can someone point me to or show me an example of how to do this? I can't find any example showing a server process pushing an object to a client.
EDIT: This is over a gigibit LAN. Using windows server with 16 cores and 24gb ram
thanks
First, some more requirements from your side is required. You have server with lots of muscle, but it will fail miserably if you don't do what has to be done.
can the client live without some of the data? I mean, does the stream of the data need to reach other side in proper order, without any drops?
how big is 'the data'? few bytes or?
fact: scheduling interval on windows is 10 msec.
fact: no matter WHEN you send, clients will receive it depending on lots of stuff - network config, number of routers in-between, client processor load, and so on. so you need some kind of timestamping here
Depending on all this, you could design a priority queue with one thread servicing it and sending out UDP datagrams for each client. Also, since (4) is in effect, you can 'clump' some of your data together and have 10 updates per second of 100 data.
If you want to achieve something else, then LAN will be required here with lots of quality network equipment.
If you want to use .NET Sockets to create this server-client project, then this is a good outline of what's needed:
Since the server will be transferring data to several clients simultaneously, you'll need to use the asynchronous Socket.Beginxxx methods or the SocketAsyncEventArgs class.
You'll have clients connect to your server. The server will accept those connections and then add the newly connected client to an internal clients list.
You'll have a thread running within the server, that periodically sends notifications to all sockets in the clients list. If any exceptions/errors occurs while sending data to a socket, then that client is removed from the list.
You'll have to make sure that access to the clients list is synchronized since the server is a multithreaded application.
You don't need to worry about buffering your send data since the TCP stack takes care of that. If you do not want to buffer your data at all (i.e. have the socket send data immediately), then set Socket.NoDelay to true.
It doesn't seem like you need any data from your clients, but if you do, you'd have to make sure your server has a Socket.BeginReceive loop if using Socket.BeginXXX pattern or Socket.ReceiveAsync method if using SocketAsyncEventArgs.
Once you have the connection and transmission of data between server and client going, you then need to worry about serialization and deserialization of objects between client and server.
Serialization which occurs on the server is easy, since you can use the BinaryFormatter or other encoders to encode your object and dump the data onto the socket.
Deserialization on the other hand, which occurs on the client, can be pretty complex because an object can span multiple packets and you can have multiple objects in one packet. You essentially need a way to identify the beginning and end of an object within the stream, so that you can pluck out the object data and deserialize it.
One way to do this is to embed your data in a well known protocol, like HTTP, and send it using that format. Unfortunately, this also means you'd have to write a HTTP parser at the client. Not an easy task.
Another way is to leverage an existing encoding scheme like Google's protocol buffers. This approach would require learning how to use the protocol buffers stack.
You can also embed the data in an XML structure and then have a stream-to-XML decoder on your client side. This is probably the easiest approach but the least efficient.
As you can see, this is not an easy project, but you can get started with the Socket.BeginSend examples here and here, and the SocketAsyncEventArgs example here
Other Tips:
You can improve the reliability of your communication by having the client maintain two connections to the server for redundancy purposes. The reason being that TCP connections take a while to establish, so if one fails, you can still receive data from the other one while the client attempts to reconnect the failed connection.
You can look into using TCPClient class for the client implementation, since it's mostly reading a stream from a network connection.
What about rendezvous or 29 west? It would save reinventing the wheel. Dunno about amqp or zeromq they might work fine too....
We're creating a c# app that needs to communicate with one other system via TCP/IP sockets. We expect to receive about 1-2 incoming transactions per second, each message averaging around 10k in size (text and 1 image).
We'll then do some processing (could take anywhere from 100 milliseconds to 3 seconds, depending on a few variables), and then send a response back of around 1k.
In the examples I've looked at, some are multi-threaded. For this application, would it be better to make it single or multi-threaded? If multi-threaded is recommended, roughly what would the different threads do?
(not specific to C#)
Having done it both ways (extreme performance was not the deciding factor), I much prefer the 1-thread-per-connection approach.
Listener Thread
The job of this thread is to listen on the socket for incoming connections, accept them, and spawn a new connection thread (giving it the connected socket).
Connection Threads
These threads (one per connection) handle all of the communication with the connected socket. They may also handle the processing of requests if it is synchronous (you will need to look into that for your specific app).
When the connection dies, this thread dies as well.
Management Threads
If cleanup, or periodic maintenance need performed, these can all run in their own threads.
Just keep in mind locking (obviously):
How much data do connections need to share? Make sure all of your resources are correctly locked when accessing, and that you do not have any deadlocks or race conditions. That is more of a "general threading" topic however.
If you're expecting multiple connections you'll need multiple threads. Each thread will be given streams for a particular client which it will need to handle separately.
I think the Silverlight policy server is a great first time example of a multithreaded server app. Though, it uses the Socket class instead of the TcpListener.
I would accept sockets and use async calls. Allows you to accept multiple connections and avoids creating a thread for every connection.
Basically create a socket with the listener
Socket socket = tcpListener.AcceptSocket();
and Socket.BeginReceive to start receiving data.
I think it's important to define what is meant by the word 'connection.'
If the other system creates a new connection to your TcpListener each time a transaction is sent, then this would be considered multiple connections, and it would make sense to have a dedicated thread to process these incoming connection requests. If this is the case, ignore everything beyond this and use gahooa's suggestion solution.
On the other hand, if the other system simply establishes a connection and sends all transactions over that same connection, then there's really no point in processing the connection request in a separate thread (since there is only one connection). If this is the case, then I would suggest accepting the incoming request once and reading the socket asynchronously (as opposed to polling the socket for data). Each time you receive a complete transaction, throw it "over the wall" to a transaction processing thread. When the processing is complete and the "answer" is computed, throw it "over the wall" to a response thread that sends the result back to the other system. In this scenario, there are basically four threads:
Main thread
Read thread
Processing thread
Write thread
The Main thread creates the TcpListener and waits until the connection is established. At that point, it simply initiates the asynchronous read and waits until the program ends (ManualResetEvent.WaitOne()).
The Read thread is the asynchronous thread that services the reading from the NetworkStream.
The Processing thread receives transactions from the Read thread and does whatever processing is necessary.
The Write thread takes whatever responses are generated by the Processing thread and writes them to the NetworkStream.
According to this link, you do not have to synchronize the reading and writing from the same NetworkStream as long as the reading and writing are done in separate threads. You can use a generic List or Queue to move data between the threads. I'd create one for the Read-to-Processing data and one for the Processing-to-Write data. Just be sure to synchronize access to them using the SyncRoot property.