I have a TcpListener listening for incoming connections, and now i basically want to ask if it is better to process the client communication in the same thread or start a new one; so if there is a best practice.
I intentionally didn't add the try-catch blocks and other handling to keep the question simple and clear.
Method 1:
while(true)
{
TcpClient client = listener.AcceptTcpClient();
processData(client);
}
Method 2:
while(true)
{
TcpClient client = listener.AcceptTcpClient();
new Thread(() => processData(client)).Start();
}
Method 3:
while(true)
{
TcpClient client = listener.AcceptTcpClient();
Thread t = new Thread(() => processData(client));
t.Start();
t.Join();
}
The code was written like Method 1 before, but processData randomly threw ThreadAbortExceptions, which shut down the entire server thread (probably because of some timeout with the client, could not exactly find the source of the Exception as the code runs on a .NET Compact framework on an Embedded Compact 2013 machine).
i basically want to ask if it is better to process the client communication in the same thread or start a new one; so if there is a best practice.
When you are using threads with sockets you could block the I/O channels each time that the thread which handle the socket is paused. And, your first case is better than third case, because you the program doesn't need to change of thread.
So, in my experience, the best way to process multiple socket is using a single thread and its asyncrhonous methods. It is faster and secure than multiple threads by sockets. With this technic you are optimize the machine resources and avoid the threads competing for them.
Check my answer to see how to use asynchronous methods into the socket. In this case I attended 15k socket request by hour and the the use of the processor does not exceed 1%.
Related
I'm using C# with .NET Framework 4.5. I'm writing a server application that should be able to connect to arbitrarily many clients at once. So I have one thread that will listen for connections, and then it will send the connection to a background thread to go into a loop waiting for messages. Since the number of supportable client connections should be very high, spawning a new thread for every connection won't work. Instead what I need is a thread pool. However, I don't want to use the system thread pool because these threads will essentially be blocked in a call to Socket.Select indefinitely, or at least for the life of the connections they host.
So I think I need a custom ThreadPool that I can explicitly round-robin the connections over to. How to achieve this in C#?
There's no point in using threads for this - that's just wasting resources.
Instead, you want to use asynchronous I/O. Now, ignoring all the complexities involved with networking, the basic idea is something like this:
async Task Loop()
{
using(var client = new TcpClient())
{
await client.ConnectAsync(IPAddress.Loopback, 6536).ConfigureAwait(false);
var stream = client.GetStream();
byte[] outBuf = new byte[4096];
// The "message loop"
while (true)
{
// Asynchronously wait for the message
var read = await stream.ReadAsync(outBuf, 0, outBuf.Length);
// Handle the message, asynchronously send replies, whatever...
}
}
}
You can run this method for each of the connection you're making (without using await - very important). The thread handling that particular socket will be released on every await - the continuation of that await will be posted on a thread-pool thread.
The result being, the amount of threads used at any given time will tend to self-balance with the available CPU cores etc., while you can easily service thousands of connections at a time.
Do not use the exact code I posted. You need to add tons of error-handling code etc. Handling networking safely and reliably is very tricky.
I am thinking of having socket.listen() on a separate thread for my server, does it pause the thread?
I want to have my thread to listen, talk with the client, end client connection, listen again.
The Listen method does not block the current thread, it merely sets up the Socket to accept new connections. Methods like Accept in synchronous mode will block the thread until a connection is received.
Note: The MSDN documentation for methods on Socket are pretty good about listing whether or not they block in the Remarks section. Example
Socket::Accept
I'm trying to make a stunnel clone in C# just for fun. The main loop goes something like this (ignore the catch-everything-and-do-nothing try-catches just for now)
ServicePointManager.ServerCertificateValidationCallback = Validator;
TcpListener a = new TcpListener (9999);
a.Start ();
while (true) {
Console.Error.WriteLine ("Spinning...");
try {
TcpClient remote = new TcpClient ("XXX.XX.XXX.XXX", 2376);
SslStream ssl = new SslStream(remote.GetStream(), false, new RemoteCertificateValidationCallback(Validator));
ssl.AuthenticateAsClient("mirai.ca");
TcpClient user = a.AcceptTcpClient ();
new Thread (new ThreadStart(() => {
Thread.CurrentThread.IsBackground = true;
try{
forward(user.GetStream(), ssl); //forward is a blocking function I wrote
}catch{}
})).Start ();
} catch {
Thread.Sleep (1000);
}
}
I found that if I do the remote SSL connection, as I did, before waiting for the user, then when the user connects the SSL is already set up (this is for tunneling HTTP so latency is pretty important). On the other hand, my server closes long-inactive connections, so if no new connection happens in, say, 5 minutes, everything locks up.
What is the best way?
Also, I observe my program generating as much as 200 threads, which of course means that context-switching overhead is pretty big and sometimes results in the whole thing just blocking for seconds, even with just one user tunneling through the program. My forward function goes, in a gist, like
new Thread(new ThreadStart(()=>in.CopyTo(out))).Start();
out.CopyTo(in);
of course with lots of error handling to prevent broken connections from holding up forever. This seems to stall a lot though. I can't figure how to use asynchronous methods like BeginRead which should help according to google.
For any kind of proxy server (including an stunnel clone), opening the backend connection after you accept the frontend connection is clearly much simpler to implement.
If you pre-open backend connections in anticipation of receiving frontend connections, you can certainly save an RTT (which is good for latency), but you have to deal with the issue you hinted at: the backend will close idle connections. At any time that you receive a frontend connections, you run the risk that the backend connection that you are about to associate with this frontend connection and which has been opened some time ago is too old to use and may be closed by the backend. You will have to manage a pool of currently open backend connections and periodically close and refresh them when they become idle for too long. There is even a race condition where if the backend decided the connection has been idle too long and decides to close it but the proxy server receives a new frontend connection at the same time, the frontend may decide to forward a request through the backend connection while the backend is closing this connection. That means that you must be able to know a priori how long backend connections can be idle for before the backend will close them (you must know what the timeout values that are configured on the backend are set to) so you can give them up just before the backend will decide they are too old.
So in summary: pre-opening backend connections will save an RTT versus opening them only on demand, but it is a lot of work, including subtle connection pool management that it quite tough to implement bug-free. Up to you to judge if the extra complexity is worth it.
By the way, concerning your comment about handling several hundred simultaneous connections, I recommend implementing such an I/O-bound program as a proxy server based around an event loop instead of based around threads. Basically, you use non-blocking sockets and process events in a single thread (e.g. "this socket has new data waiting to be forwarded to the other side") instead of spawning a thread for each connection (which can get expensive both in thread creation and context switches). In order to scale such an event-based model to multiple CPU cores, you can start a small number of parallel threads of processes (more or less one per CPU core) which each handle many hundreds (or thousands) of simultaneous connections.
I have a server that has several clients C1...Cn to each of which there is a TCP connection established. There are less than 10,000 clients.
The message protocol is request/response based, where the server sends a request to a client and then the client sends a response.
The server has several threads, T1...Tm, and each of these may send requests to any of the clients. I want to make sure that only one of these threads can send a request to a specific client at any one time, while the other threads wanting to send a request to the same client will have to wait.
I do not want to block threads from sending requests to different clients at the same time.
E.g. If T1 is sending a request to C3, another thread T2 should not be able to send anything to C3 until T1 has received its response.
I was thinking of using a simple lock statement on the socket:
lock (c3Socket)
{
// Send request to C3
// Get response from C3
}
I am using asynchronous sockets, so I may have to use Monitor instead:
Monitor.Enter(c3Socket); // Before calling .BeginReceive()
And
Monitor.Exit(c3Socket); // In .EndReceive
I am worried about stuff going wrong and not letting go of the monitor and therefore blocking all access to a client. I'm thinking that my heartbeat thread could use Monitor.TryEnter() with a timeout and throw out sockets that it cannot get the monitor for.
Would it make sense for me to make the Begin and End calls synchronous in order to be able to use the lock() statement? I know that I would be sacrificing concurrency for simplicity in this case, but it may be worth it.
Am I overlooking anything here? Any input appreciated.
My answer here would be a state machine per socket. The states would be free and busy:
If socket is free, the sender thread would mark it busy and start sending to client and waiting for response.
You might want to setup a timeout on that wait just in case a client gets stuck somehow.
If the state is busy - the thread sleeps, waiting for signal.
When that client-related timeout expires - close the socket, the client is dead.
When a response is successfully received/parsed, mark the socket free again and signal/wakeup the waiting threads.
Only lock around socket state inquiry and manipulation, not the actual network IO. That means a lock per socket, plus some sort of wait primitive like a conditional variables (sorry, don't remember what's really available in .NET)
Hope this helps.
You certainly can't use the locking approach that you've described. Since your system is primarily asynchronous, you can't know what thread operations will be running on. This means that you may call Exit on the wrong thread (and have a SynchronizationLockException thrown), or some other thread may call Enter and succeed even though that client is "in use", just because it happened to get the same thread that Enter was originally called on.
I'd agree with Nikolai that you need to hold some additional state alongside each socket to determine whether it is currently in use or not. You woud of course need locking to update this shared state.
I am currently creating a Windows Service that will create TCP connections to multiple machines (same socket on all machines) and then listen for 'events' from those machines. I am attempting to write the code to create a connection and then spawn a thread that listens to the connection waiting for packets from the machine, then decode the packets that come through, and call a function depending on the payload of the packet.
The problem is I'm not entirely sure how to do that in C#. Does anyone have any helpful suggestions or links that might help me do this?
Thanks in advance for any help!
Depending on how many concurrent clients you plan on supporting, a thread-per-connection architecture will probably break down very quickly. Reason being, each thread requires significant resources. By default each .NET thread gets 1MB of stack space so that's 1MB per connection plus any overhead.
Instead when supporting multiple connected clients typically you will use the asynchronous methods (see here also) which are very efficient because Windows will use "completion ports" which basically free up the thread to do other things while waiting on some event to complete.
For this you would look at methods such as BeginAccept, BeginReceive, BeginSend, etc.
A simpler approach which also avoids making blocking calls and avoids multiple threads is to use the Socket.Select method in a loop. This allows a single thread to service multiple sockets. The thread can only physically read or write to a single socket at a time but the idea is that you are checking the state of multiple sockets which may or may not contain data to read.
In any case, the thread-per-connection approach is much simpler to get your head around at first, but it does have significant scalability problems. I would suggest doing that first with the synchronous methods like Accept, Receive, Send, etc. Then later on refactor your code to use the asynchronous methods so that you don't exhaust the server's memory.
You can have asynchronous receive for every socket connection and decode the data coming from other machines to perform your tasks (You can find some useful information about asynchronous methods here: http://msdn.microsoft.com/en-us/library/2e08f6yc.aspx).
To create a connection, you can do:
Socket sock = Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
sock.Connect(new IPEndPoint(address, port));
Same way you can create multiple connections and keep there references in a List of Dictionary (whichever you prefer).
For receiving data asynchronously on a socket, you can do:
sock.BeginReceive(buffer, 0, buffer.len, SocketFlags.None, new AsyncCallback(OnDataReceived), null);
The method OnDataReceived will be called when receive operation completes.
In OnDataReceived method you have to do:
void OnDataReceived(IAsyncResult result)
{
int dataReceived = sock.EndReceive(result);
//Make sure that dataReceived is equal to amount of data you wanted to receive. If it is
//less then the data you wanted, you can do synchronous receive to read remaining data.
//When all of the data is recieved, call BeginReceive again to receive more data
//... Do your decoding and call the method ...//
}
I hope this helps you.
Have a single thread that runs the accept() to pick up new connections. For each new connection you get, spawn a worker thread using the thread pool.
I don't know if it's possible in your situation but have you thought about using a WCF service that gets called by the multiple machines ? You can host this in a custom windows service or IIS. It will consume very little resource while waiting for events and it's much simpler to code than all that low level scary socket stuff. It's automatically async. You get nice messages to your service rather than a packet you need to deserialize and/or parse. You can use any number of protocols such as REST or binary.
You will of course need to create the process on the other end that sends the messages.
Just a thought...
Cheers