Develop Reference

C# tcp async listener gets stuck on my on_receive callback after client closes socket

I've got a listener socket that accepts, receives and sends as a TCP server typically does. I've given my accept and receive code below, it's not that different from the example on Microsoft's documentation. The main difference is that my server doesn't kill a connection after it stops receiving data (I don't know if this is a bad design or not?).
private void on_accept(IAsyncResult xResult)
{
Socket listener = null;
Socket handler = null;
TStateObject state = null;
Task<int> consumer = null;
try
{
mxResetEvent.Set();
listener = (Socket)xResult.AsyncState;
handler = listener.EndAccept(xResult);
state = new TStateObject()
{
Socket = handler
};
consumer = async_input_consumer(state);
OnConnect?.Invoke(this, handler);
handler.BeginReceive(state.Buffer, 0, TStateObject.BufferSize, 0, new AsyncCallback(on_receive), state);
}
catch (SocketException se)
{
if (se.ErrorCode == 10054)
{
on_disconnect(state);
}
}
catch (ObjectDisposedException)
{
return;
}
catch (Exception ex)
{
System.Console.WriteLine("Exception in TCPServer::AcceptCallback, exception: " + ex.Message);
}
}
private void on_receive(IAsyncResult xResult)
{
Socket handler = null;
TStateObject state = null;
try
{
state = xResult.AsyncState as TStateObject;
handler = state.Socket;
int bytesRead = handler.EndReceive(xResult);
UInt16 id = TClientRegistry.GetIdBySocket(handler);
TContext context = TClientRegistry.GetContext(id);
if (bytesRead > 0)
{
var buffer_data = new byte[bytesRead];
Array.Copy(state.Buffer, buffer_data, bytesRead);
state.BufferBlock.Post(buffer_data);
}
Array.Clear(state.Buffer, 0, state.Buffer.Length);
handler.BeginReceive(state.Buffer, 0, TStateObject.BufferSize, 0, new AsyncCallback(on_receive), state);
}
catch (SocketException se)
{
if(se.ErrorCode == 10054)
{
on_disconnect(state);
}
}
catch (ObjectDisposedException)
{
return;
}
catch (Exception ex)
{
System.Console.WriteLine("Exception in TCPServer::ReadCallback, exception: " + ex.Message);
}
}
This code is used to connect to an embedded device and works (mostly) fine. I was investigating a memory leak and trying to speed up the process a bit by replicating exactly what the device does (our connection speeds are in the realm of about 70kbps to our device, and it took an entire weekend of stress testing to get the memory leak to double the memory footprint of the server).
So I wrote a C# program to replicate the data transactions, but I've run into an issue where when I disconnect the test program, the server gets caught in a loop where it endlessly has its on_receive callback called. I was under the impression that BeginReceive wouldn't be triggered until something was received, and it seems to call on_receive, ends the receiving like an async callback should do, process the data, and then I want the connection to await more data so I call BeginReceive again.
The part of my test program where the issue occurs is in here:
private static void read_write_test()
{
mxConnection = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
mxConnection.Connect("12.12.12.18", 10);
if (mxConnection.Connected)
{
byte[] data = Encoding.ASCII.GetBytes("HANDSHAKESTRING"); //Connect string
int len = data.Length;
mxConnection.Send(data);
data = new byte[4];
len = mxConnection.Receive(data);
if (len == 0 || data[0] != '1')
{
mxConnection.Disconnect(false);
return;
}
}
//Meat of the test goes here but isn't relevant
mxConnection.Shutdown(SocketShutdown.Both);
mxConnection.Close();
}
Up until the Shutdown(SocketShutdown.Both) call, everything works as expected. When I make that call however, it seems like the server never gets notification that the client has closed the socket and gets stuck in a loop of endlessly trying to receive. I've done my homework and I think I am closing my connection properly as per this discussion. I've messed around with the disconnect section to just do mxConnection.Disconnect(false) as well, but the same thing occurs.
When the device disconnects from the server, my server catches a SocketException with error code 10054, which documentation says:
Connection reset by peer.
An existing connection was forcibly closed
by the remote host. This normally results if the peer application on
the remote host is suddenly stopped, the host is rebooted, the host or
remote network interface is disabled, or the remote host uses a hard
close (see setsockopt for more information on the SO_LINGER option on
the remote socket). This error may also result if a connection was
broken due to keep-alive activity detecting a failure while one or
more operations are in progress. Operations that were in progress fail
with WSAENETRESET. Subsequent operations fail with WSAECONNRESET.
I've used this to handle the socket being closed and has worked well for the most part. However, with my C# test program, it doesn't seem like it works the same way.
Am I missing something here? I'd appreciate any input. Thanks.

The main difference is that my server doesn't kill a connection after it stops receiving data (I don't know if this is a bad design or not?).
Of course it is.
it seems like the server never gets notification that the client has closed the socket and gets stuck in a loop of endlessly trying to receive
The server does get notification. It's just that you ignore it. The notification is that your receive operation returns 0. When that happens, you just call BeginReceive() again. Which starts a new read operation. Which…returns 0! You just keep doing that over and over again.
When a receive operation returns 0, you're supposed to complete the graceful closure (with a call to Shutdown() and Close()) that the remote endpoint started. Do not try to receive again. You'll just keep getting the same result.
I strongly recommend you do more homework. A good place to start would be the Winsock Programmer's FAQ. It is a fairly old resource and doesn't address .NET at all. But for the most part, the things that novice network programmers are getting wrong in .NET are the same things that novice Winsock programmers were getting wrong twenty years ago. The document is still just as relevant today as it was then.
By the way, your client-side code has some issues as well. First, when the Connect() method returns successfully, the socket is connected. You don't have to check the Connected property (and in fact, should never have to check that property). Second, the Disconnect() method doesn't do anything useful. It's used when you want to re-use the underlying socket handle, but you should be disposing the Socket object here. Just use Shutdown() and Close(), per the usual socket API idioms. Third, any code that receives from a TCP socket must do that in a loop, and make use of the received byte-count value to determine what data has been read and whether enough has been read to do anything useful. TCP can return any positive number of bytes on a successful read, and it's your program's job to identify the start and end of any particular blocks of data that were sent.

You missed this in the documentation for EndReceive() and Receive():
If the remote host shuts down the Socket connection with the Shutdown method, and all available data has been received, the Receive method will complete immediately and return zero bytes.
When you read zero bytes, you still start another BeginReceive(), instead of shutting down:
if (bytesRead > 0)
{
var buffer_data = new byte[bytesRead];
Array.Copy(state.Buffer, buffer_data, bytesRead);
state.BufferBlock.Post(buffer_data);
}
Array.Clear(state.Buffer, 0, state.Buffer.Length);
handler.BeginReceive(state.Buffer, 0, TStateObject.BufferSize, 0, new AsyncCallback(on_receive), state);
Since you keep calling BeginReceive on a socket that's 'shutdown', you're going to keep getting callbacks to receive zero bytes.
Compare with the example from Microsoft in the documentation for EndReceive():
public static void Read_Callback(IAsyncResult ar){
StateObject so = (StateObject) ar.AsyncState;
Socket s = so.workSocket;
int read = s.EndReceive(ar);
if (read > 0) {
so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read));
s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0,
new AsyncCallback(Async_Send_Receive.Read_Callback), so);
}
else{
if (so.sb.Length > 1) {
//All of the data has been read, so displays it to the console
string strContent;
strContent = so.sb.ToString();
Console.WriteLine(String.Format("Read {0} byte from socket" +
"data = {1} ", strContent.Length, strContent));
}
s.Close();
}
}

C# Server - Socket not connecting and out of memory exception

I am trying to implement a simple TCP server and I basically copied the example on MSDN sans a couple of lines and tried to make it work. I have an external client trying to connect already.
This is my code:
IPHostEntry ipHostInfo = Dns.Resolve(Dns.GetHostName());
IPEndPoint localEP = new IPEndPoint(ipHostInfo.AddressList[0], 4001);
Socket listener = new Socket(localEP.Address.AddressFamily,
SocketType.Stream, ProtocolType.Tcp);
try
{
listener.Bind(localEP);
listener.Listen(1000);
while (true)
{
listener.BeginAccept(new AsyncCallback(AcceptCnxCallback), listener);
}
}
catch (Exception e)
{
//Log here
}
This is my callback:
private void AcceptCnxCallback(IAsyncResult iar)
{
MensajeRecibido msj = new MensajeRecibido();
Socket server = (Socket)iar.AsyncState;
msj.workSocket = server.EndAccept(iar);
}
And this is the information of one of the incoming packages:
TCP:[SynReTransmit #1727889]Flags=......S., SrcPort=57411, DstPort=4001, PayloadLen=0, Seq=673438964, Ack=0, Win=5840 ( Negotiating scale factor 0x4 ) = 5840
Source: 10.0.19.65 Destination: 10.0.19.59
I basically have two issues:
If I use the while loop I get an OutOfMemoryException
I never do manage to connect to the client
Any tips on either of the two problems? Thank you in advance!

Your problem is, that you use asynchronous calls all the time. There is no wait mechanism or similar, so generally you are just creating new asynchronous callbacks in an infinite loop.
For a basic TCP I would recommend to use the simple approach and use the synchronous methods.
Accept() is blocking, so the program flow will stop until there is an ingoing connection.
while (true)
{
Socket s = listener.Accept();
buffer = new byte[BUFFER_SIZE];
s.Receive(buffer);
//Do something
s.Send(...);
}
Noe that this is just a basic example. If you want to keep your connection you might consider a new Thread for each accepted Socket, that continoues with receiving and sending data.

First problem
You are using an infinite loop to call an async method.
try it like this:
listener.BeginAccept(new AsyncCallback(AcceptCnxCallback), listener);
//add your code here (this part will be executed wile the listner is waiting for a connection.
while (true)
{
Thread.Sleep(100);
}
and change the Callbackmethod to:
private void AcceptCnxCallback(IAsyncResult iar)
{
MensajeRecibido msj = new MensajeRecibido();
Socket server = (Socket)iar.AsyncState;
msj.workSocket = server.EndAccept(iar);
//call again the listener after you get a message
listener.BeginAccept(new AsyncCallback(AcceptCnxCallback), listener);
}

Concurrent Clients issue in C# Sockets

I am working on a client server application, Windows Server and Linux Client. I was testing my server with multiple concurrent clients. I tried just 20 concurrent connections from client, and i noticed that some requests were not processed despite all 20 requests were the same. They went into the queue and for some reason when their turn comes client was shutdown (Client connect timeout is 5 sec).
Then I added a Thread.Sleep(1000), to check if it is really asynchronous but then i realized it does not process other request until timeout. Despite the fact
It is asynchronous
ManualResetEvent was set before going to sleep.
Now I am wondering what Am I missing here, as this happens with concurrent connections mostly?
public static void StartServer(IPAddress ipAddr, int port)
{
//IPEndPoint serverEndPoint = new IPEndPoint(ipAddr, port);
IPEndPoint serverEndPoint = new IPEndPoint(IPAddress.Any, port);
Socket clientListener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
try
{
clientListener.Bind(serverEndPoint);
clientListener.Listen(500);
Console.WriteLine("-- Server Listening: {0}:{1}",ipAddr,port);
while (true)
{
resetEvent.Reset();
Console.WriteLine("|| Waiting for connection");
clientListener.BeginAccept(new AsyncCallback(AcceptConnection), clientListener);
resetEvent.WaitOne();
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
public static void AcceptConnection(IAsyncResult ar)
{
// Get the socket that handles the client request.
Socket listener = (Socket)ar.AsyncState;
Socket handler = listener.EndAccept(ar);
// Signal the main thread to continue.
resetEvent.Set();
// Create the state object.
JSStateObject state = new JSStateObject();
state.workSocket = handler;
if (handler.Connected)
{
Console.WriteLine("** Connected to: {0}", handler.RemoteEndPoint.ToString());
state.workingDirectory = JSUtilityClass.CreatetTemporaryDirectry();
try
{
Thread.Sleep(1000);
Receive(state);
}
catch (Exception e)
{
handler.Shutdown(SocketShutdown.Both);
handler.Close();
Console.WriteLine(e.Message);
}
}
}

I created a test that sends 100 connection attempts and found a few things slowing it down.
Why is it so slow?
I put a breakpoint in AcceptConnection to look at the callstack, this is it
ConsoleApplication1.exe!ConsoleApplication1.Program.AcceptConnection(System.IAsyncResult ar) Line 62 C#
System.dll!System.Net.LazyAsyncResult.Complete(System.IntPtr userToken) + 0x69 bytes
System.dll!System.Net.ContextAwareResult.CaptureOrComplete(ref System.Threading.ExecutionContext cachedContext, bool returnContext) + 0xab bytes
System.dll!System.Net.ContextAwareResult.FinishPostingAsyncOp(ref System.Net.CallbackClosure closure) + 0x3c bytes
System.dll!System.Net.Sockets.Socket.BeginAccept(System.AsyncCallback callback, object state) + 0xe3 bytes
ConsoleApplication1.exe!ConsoleApplication1.Program.StartServer(System.Net.IPAddress ipAddr, int port) Line 48 + 0x32 bytes C#
So the callback AcceptConnection is running from the same thread that BeginAccept was called from. I had a look at FinishPostingAsyncOp with reflector and it's using the async pattern where if there is already a socket operation in the queue waiting to be processed, it'll do so on the current thread, otherwise if there isn't anything pending, it'll process in a different thread later on, e.g.
SocketAsyncEventArgs sae = new SocketAsyncEventArgs();
sae.Completed += new EventHandler<SocketAsyncEventArgs>(SocketOperation_Completed);
if (!clientListener.AcceptAsync(sae))
AcceptConnection(clientListener, sae); // operation completed synchronously, process the result
else
// operation will complete on a IO completion port (different thread) which we'll handle in the Completed event
So as you observed the program is effectively completely synchronous in this scenario, and with the 1 second Thread.Sleep it's going to take at least 100 seconds to accept all the connections, by which time most of them will timeout.
The solution
Even though BeginAccept method summary says
Begins an asynchronous operation to accept an incoming connection
attempt.
It turns out there is more to the story
From MSDN http://msdn.microsoft.com/en-AU/library/system.net.sockets.socket.beginaccept.aspx
BeginAccept(Int32, AsyncCallback, Object)
Begins an asynchronous operation to accept an
incoming connection attempt and receives the first block of data sent
by the client application.
So it's performing a read operation with a short timeout before firing the callback. You can disable this by specifying the receiveSize of 0. Change
clientListener.BeginAccept(new AsyncCallback(AcceptConnection), clientListener);
to
clientListener.BeginAccept(0, new AsyncCallback(AcceptConnection), clientListener);
That speeds it up, and if we remove the Thread.Sleep(1000) from AcceptConnection then all the connections are accepted really fast.
If you leave the Thread.Sleep(1000) in there to simulate work load or just for testing then you may want to prepare the server to handle such a load by doing
int minWorkerThreads = 0;
int minCompletionPortThreads = 0;
ThreadPool.GetMinThreads(out minWorkerThreads, out minCompletionPortThreads);
ThreadPool.SetMinThreads(minWorkerThreads, 100);
Where 100 is the amount of threads you want readily available to handle socket operations.
Just one other thing, it's a matter of personal preference but just so you know you might like to call BeginAccept from within AcceptConnection which removes the need for that while loop.
i.e. change this
while (true)
{
resetEvent.Reset();
Console.WriteLine("|| Waiting for connection");
clientListener.BeginAccept(new AsyncCallback(AcceptConnection), clientListener);
resetEvent.WaitOne();
}
to this
Console.WriteLine("|| Waiting for connection");
clientListener.BeginAccept(new AsyncCallback(AcceptConnection), clientListener);
and put another BeginAccept in AcceptConnection
public static void AcceptConnection(IAsyncResult ar)
{
// Get the socket that handles the client request.
Socket listener = (Socket)ar.AsyncState;
// start another listening operation
listener.BeginAccept(new AsyncCallback(AcceptConnection), listener);
... the rest of the method
}

Async client handle receive?

I was told several times that async is better or that I should use async instead of sync sockets and as such started learning it, but am already having difficult.
I've got the basic feel of how the callback works and how to establish a connection.
I am using this msdn code as reference!
A few problems I am having with the code:
Currently that code will connect to the server, send a text, read the response and exit. How do I do so I can keep receiving the data until either the server disconnects me and/or I end it by myself ? I am not much sure on how I should do it, if I would need to wrap it on a thread with while or simple call that Receive again once the ReceiveCallback is done.
Another things I've noticed is when it connects, the socket is assigned to client but the code itself is always reassigning the client socket which I don't understand very well compared to the sync socket we have a main socket that we are always consulting etc.
I am not sure on how old the reference I am using is but would appreciate if you could help me with examples of what I have pointed out as it is easier for me to understand.
UPDATE:
private void SetupRecieveCallback(Socket sock)
{
new Thread(
delegate()
{
while (isReceiving)
{
_receiveQueue.Reset();
try
{
AsyncCallback recieveData = new AsyncCallback(OnRecievedData);
sock.BeginReceive(m_byBuff, 0, m_byBuff.Length, SocketFlags.None, recieveData, sock);
}
catch (Exception ex)
{
_logger.Error("Setup Recieve Callback failed! " + ex.Message);
}
_receiveQueue.WaitOne();
}
}
).Start();
/*
// The original code
try
{
AsyncCallback recieveData = new AsyncCallback(OnRecievedData);
sock.BeginReceive(m_byBuff, 0, m_byBuff.Length, SocketFlags.None, recieveData, sock);
}
catch (Exception ex)
{
_logger.Error("Setup Recieve Callback failed! " + ex.Message);
}
*/
}

Simply call BeginReceive() again in the callback to keep receiving. When the server breaks the connection then your callback will be called and EndReceive() throws an ObjectDisposedException. That's how you know to stop calling BeginReceive().
Second question is harder to decode (ask only one). I'm guessing you are puzzled about this statement:
private static void ConnectCallback(IAsyncResult ar) {
try {
// Retrieve the socket from the state object.
Socket client = (Socket) ar.AsyncState;
// etc..
No reassigning the socket is happening here. The code simply retrieves a reference to the original socket. Which is a useful technique, it allows this callback to be used by more than one connection. The ar.AsyncState value got to be the socket by this statement:
client.BeginConnect( remoteEP,
new AsyncCallback(ConnectCallback), client);
Note how client is passed to the AsyncCallback constructor. The exact same client that's retrieved in the callback. Any object can be passed.

What happens if you break out of a Lock() statement?

I'm writing a program which listens to an incoming TcpClient and handles data when it arrives. The Listen() method is run on a separate thread within the component, so it needs to be threadsafe. If I break out of a do while loop while I'm within a lock() statement, will the lock be released? If not, how do I accomplish this?
Thanks!
(Any other advice on the subject of Asynchronous TCP Sockets is welcome as well.)
private void Listen()
{
do
{
lock (_clientLock)
{
if (!_client.Connected) break;
lock (_stateLock)
{
if (!_listening) break;
if (_client.GetStream().DataAvailable) HandleData();
}
}
Thread.Sleep(0);
} while (true);
}

Yes. The lock statement translates into a try/finally clause. In C# 4, for example, a lock statement like so:
lock(obj)
{
// body
}
roughly translates (taken from Eric Lippert's blog here) to:
bool lockWasTaken = false;
var temp = obj;
try
{
Monitor.Enter(temp, ref lockWasTaken);
{
// body
}
}
finally
{
if (lockWasTaken)
Monitor.Exit(temp);
}
When the execution leaves the scope of the lock {}, the underlying lock will be released automatically. This will happen no matter how you exit scope (break/return/etc), since the call to Monitor.Exit is wrapped, internally, inside of the finally block of a try/finally.

Yes, the lock will be released. You can use ILDASM or Reflector to look at the actual generated code. The lock statement is shorthand for the following code (roughly).
Monitor.Enter(_client);
try
{
// do your stuff
}
finally {
Monitor.Exit(_client);
}
Notice the finally block is always executed.

Because you asked for other advice...I noticed that you are nesting locks. This, by itself, is not necessarily a bad thing. But, it is one my red flags I watch out for. There is the possibility of a deadlock if you ever acquire those two locks in a different order in another part of your code. I am not saying there is anything wrong with your code. It is just something else to watch out for because it is easy to get wrong.

Once you exit the lock{}, it will unlock what you have locked (it's just like a using statement in that regard). It doesn't matter where you exit (the beginning, the end, or the middle), it's that you left the scope of the lock at all. Think about what would happen if you raised an exception in the middle.

To answer the other half of your question:
Any other advice on the subject of Asynchronous TCP Sockets is welcome as well
Simply put I wouldn't manage this in the fashion demonstrated by your original post. Rather seek help from the System.Net.Sockets.TcpClient and the System.Net.Sockets.TcpListener classes. Use the async calls like BeginAcceptSocket(...) and BeginRead(...) and allow the ThreadPool to do it's job. It's really pretty easy to put together that way.
You should be able to achieve all the server behavior you desire without ever coding the dreaded words "new Thread" :)
Here is a basic example of the idea, minus the idea of graceful shutdown, exception handling ect:
public static void Main()
{
TcpListener listener = new TcpListener(new IPEndPoint(IPAddress.Loopback, 8080));
listener.Start();
listener.BeginAcceptTcpClient(OnConnect, listener);
Console.WriteLine("Press any key to quit...");
Console.ReadKey();
}
static void OnConnect(IAsyncResult ar)
{
TcpListener listener = (TcpListener)ar.AsyncState;
new TcpReader(listener.EndAcceptTcpClient(ar));
listener.BeginAcceptTcpClient(OnConnect, listener);
}
class TcpReader
{
string respose = "HTTP 1.1 200\r\nContent-Length:12\r\n\r\nHello World!";
TcpClient client;
NetworkStream socket;
byte[] buffer;
public TcpReader(TcpClient client)
{
this.client = client;
socket = client.GetStream();
buffer = new byte[1024];
socket.BeginRead(buffer, 0, 1024, OnRead, socket);
}
void OnRead(IAsyncResult ar)
{
int nBytes = socket.EndRead(ar);
if (nBytes > 0)
{
//you have data... do something with it, http example
socket.BeginWrite(
Encoding.ASCII.GetBytes(respose), 0, respose.Length, null, null);
socket.BeginRead(buffer, 0, 1024, OnRead, socket);
}
else
socket.Close();
}
}
For a much more complicated example of how to do this see the SslTunnel Library I wrote a while ago.