Develop Reference

TcpListener and NetworkStream doesn't get data

I am trying to receive a message from an equipment. This equipment is an authentication terminal, and it will send the message as soon as the user set his credentials.
Also, the manual of the equipment says the message will be sent in the ILV format, standing I for identification, L for length and V for value.
a normal message would be:
I -> 0x00 (byte 0 indicating success)
L -> 0x04 0x00 (two bytes for length, being 4 the length in this case)
V -> 0x35 0x32 0x38 0x36 (the message itself)
The message is sent in TCP protocol, so I created a socket using the TcpListener class, following this sample from Microsoft:
https://msdn.microsoft.com/en-us/library/system.net.sockets.tcplistener(v=vs.110).aspx
new Thread(() =>
{
TcpListener server = null;
try
{
Int32 port = 11020;
IPAddress localAddr = IPAddress.Parse("192.168.2.2");
server = new TcpListener(localAddr, port);
server.Server.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
server.Start();
byte[] bytes = new byte[256];
String data = null;
while (true)
{
TcpClient client = server.AcceptTcpClient();
data = null;
NetworkStream stream = client.GetStream();
int i = 0;
while((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
// this code is never reached as the stream.Read above runs for a while and receive nothing
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
}
client.Close();
}
}
catch (SocketException ex)
{
// Actions for exceptions.
}
finally
{
server.Stop();
}
}).Start();
If the stream.Read is removed, then the code flows (although I got nothing either), however if I put any stream.Read statement the execution holds for a while like it was waiting for some response, and then it ends with no response, all bytes read is zero.
I am running Wireshark on the computer and the data is being sent.
Anybody knows what I am doing wrong?

I think the problem is right in the Read method, which halts until the buffer fills completely, but that's obviously won't happen anytime.
In that while-loop, you should spin by checking for available data first, then read them and append to the byte-array. Moreover, since the loop become too "tight", it's better to yield the control to the task scheduler for a bit.
Here is an example:
while (true)
{
if (stream.DataAvailable)
{
int count = stream.Read(bytes, i, bytes.Length);
i += count;
// ...add something to detect the end of message
}
else
{
Thread.Sleep(0);
}
}
It's worthwhile to notice that there's no timeout, but that's a very common feature in order to quit the loop when no data (or broken) are incoming.
Furthermore, the above pattern is not the best way to accumulate data, because you may get an exception when too many bytes are received. Consider a spurious stream of 300 bytes: that will overflow the available buffer. Either use the "bytes" as a temporary buffer for what the Read method gives, or provide a safety check before calling the Read method (so that you may provide the best byte count to read).
Useful links here:
https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.networkstream.read?view=netframework-4.7.1#System_Net_Sockets_NetworkStream_Read_System_Byte___System_Int32_System_Int32_

Where is data sent by UDP stored?

I had never used UDP before, so I gave it a go. To see what would happen, I had the 'server' send data every half a second, and the client receive data every 3 seconds. So even though the server is sending data much faster than the client can receive, the client still receives it all neatly one by one.
Can anyone explain why/how this happens? Where is the data buffered exactly?
Send
class CSimpleSend
{
CSomeObjServer obj = new CSomeObjServer();
public CSimpleSend()
{
obj.changedVar = varUpdated;
obj.threadedChangeSomeVar();
}
private void varUpdated(int var)
{
string send = var.ToString();
byte[] packetData = System.Text.UTF8Encoding.UTF8.GetBytes(send);
string ip = "127.0.0.1";
int port = 11000;
IPEndPoint ep = new IPEndPoint(IPAddress.Parse(ip), port);
Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
client.SendTo(packetData, ep);
Console.WriteLine("Sent Message: " + send);
Thread.Sleep(100);
}
}
All CSomeObjServer does is increment an integer by one every half second
Receive
class CSimpleReceive
{
CSomeObjClient obj = new CSomeObjClient();
public Action<string> showMessage;
Int32 port = 11000;
UdpClient udpClient;
public CSimpleReceive()
{
udpClient = new UdpClient(port);
showMessage = Console.WriteLine;
Thread t = new Thread(() => ReceiveMessage());
t.Start();
}
private void ReceiveMessage()
{
while (true)
{
//Thread.Sleep(1000);
IPEndPoint remoteIPEndPoint = new IPEndPoint(IPAddress.Any, port);
byte[] content = udpClient.Receive(ref remoteIPEndPoint);
if (content.Length > 0)
{
string message = Encoding.UTF8.GetString(content);
if (showMessage != null)
showMessage("Recv:" + message);
int var_out = -1;
bool succ = Int32.TryParse(message, out var_out);
if (succ)
{
obj.alterSomeVar(var_out);
Console.WriteLine("Altered var to :" + var_out);
}
}
Thread.Sleep(3000);
}
}
}
CSomeObjClient stores the variable and has one function (alterSomeVar) to update it
Ouput:
Sent Message: 1
Recv:1
Altered var to :1
Sent Message: 2
Sent Message: 3
Sent Message: 4
Sent Message: 5
Recv:2
Altered var to :2
Sent Message: 6
Sent Message: 7
Sent Message: 8
Sent Message: 9
Sent Message: 10
Recv:3
Altered var to :3

The operating system kernel maintains separate send and receive buffers for each UDP and TCP socket. If you google SO_SNDBUF and SO_RCVBUF you'll find lots of information about them.
When you send data, it is copied from your application space into the send buffer. From there it is copied to the network interface card, and then onto the wire. The receive side is the reverse: NIC to receive buffer, where it waits until you read it. Additionally copies and buffering can also occur, depending on the OS.
It is critical to note that the sizes of these buffers can vary radically. Some systems might default to as little as 4 kilobytes, while others give you 2 megabytes. You can find the current size using getsockopt() with SO_SNDBUF or SO_RCVBUF and likewise set it using setsockopt(). But many systems limit the size of the buffer, sometimes to arbitrarily small amounts. This is typically a kernel value like net.core.wmem_max or net.core.rmem_max, but the exact reference will vary by system.
Also note that setsockopt() can fail even if you request an amount less than the supposed limit. So to actually get a desired size, you need to repeatedly call setsockopt() using decreasing amounts until it finally succeeds.
The following page is a Tech Note from my company which touches on this topic a little bit and provides references for some common systems: http://www.dataexpedition.com/support/notes/tn0024.html

It looks to me like the UdpClient-Class provides a buffer for received data. Try using a socket directly. You might also want to set that sockets ReceiveBufferSize to zero, even though I believe it is only used for TCP connections.

Sockets starts to slow down and not respond

i am developing a server (with c#) and a client (with flash, actionscript 3.0) application. Server sends data (datas are arround 90 bytes) to clients continuously and clients behave according to data they received (data is json formatted)
for a while, everything works as expected but after some time passed, clients start to receive messages laggy. they keep waiting for some time and then they behave according to last message (some messages lost). after some time passed clients starts to wait and process all the messages at the same time. I could not figured out what causing this. My network condition is stable.
here is some part of my c# code, sending message:
public void Send(byte[] buffer)
{
if (ClientSocket != null && ClientSocket.Connected)
{
ClientSocket.BeginSend(buffer, 0, buffer.Length, 0, WriteCallback, ClientSocket);
}
}
private void WriteCallback(IAsyncResult result)
{
//
}
and some part of my client, receiving message (actionscript)
socket.addEventListener(ProgressEvent.SOCKET_DATA, onResponse);
function onResponse(e:ProgressEvent):void {
trace(socket.bytesAvailable);
if(socket.bytesAvailable > 0) {
try
{
var serverResponse:String = socket.readUTFBytes(socket.bytesAvailable);
....
I hope i could explain my problem. How should i optimize my code? What can be causing lags. Thanks.

You really need to give more detail as to how you're setting up the socket (is it TCP or UDP?)
Assuming it's a TCP socket, then it would appear that your client relies on each receive call returning the same number of bytes that were sent by the server's Send() call. This is however not the case, and could well be the cause of your issues if a message is only being partially received on the client, or multiple messages are received at once.
For example, the server may send a 90 byte message in a single call, but your client may receive it in one 90-byte receive, or two 45-byte chunks, or even 90 x 1-byte chunks, or anything in between. Multiple messages sent by the server may also be partially combined when received by the client. E.g. two 90-byte messages may be received in a single 180-byte chunk, or a 150-byte and a 30-byte chunk, etc. etc.
You need therefore to provide some kind of framing on your messages so that when the stream of data is received by the client, it can be reliably reconstructed into individual messages.
The most basic framing mechanism would be to prefix each message sent with a fixed-length field indicating the message size. you may be able to get away with a single byte if you can guarantee that your messages will never be > 255 bytes long, which will simplify the receiving code.
On the client side, you first need to receive the length prefix, and then read up to that many bytes off the socket to construct the message data. If you receive fewer than the required number of bytes, your receiving code must wait for more data (appending it to the partially-received message when it is eventually received) until it has a complete message of the.
Once the full message is received it can be processed as you are currently.
Unfortunately I don't know ActionScript, so can't give you an example of the client-side code, but here's how you might write the server and client framing in C#:
Server side:
public void SendMessage(string message)
{
var data = Encoding.UTF8.GetBytes(message);
if (data.Length > byte.MaxValue) throw new Exception("Data exceeds maximum size");
var bufferList = new[]
{
new ArraySegment<byte>(new[] {(byte) data.Length}),
new ArraySegment<byte>(data)
};
ClientSocket.Send(bufferList);
}
Client side:
public string ReadMessage()
{
var header = new byte[1];
// Read the header indicating the data length
var bytesRead = ServerSocket.Receive(header);
if (bytesRead > 0)
{
var dataLength = header[0];
// If the message size is zero, return an empty string
if (dataLength == 0) return string.Empty;
var buffer = new byte[dataLength];
var position = 0;
while ((bytesRead = ServerSocket.Receive(buffer, position, buffer.Length - position, SocketFlags.None)) > 0)
{
// Advance the position by the number of bytes read
position += bytesRead;
// If there's still more data to read before we have a full message, call Receive again
if (position < buffer.Length) continue;
// We have a complete message - return it.
return Encoding.UTF8.GetString(buffer);
}
}
// If Receive returns 0, the socket has been closed, so return null to indicate this.
return null;
}

c# async sockets, read all buffer

I have a client that will send a lots of data to server from different threads.
The packet uses the following format:
PACKET_ID
CONTENT
END_OF_PACKET_INDICATOR
I have the following onDataRecieved function:
public void OnDataReceived(IAsyncResult asyn)
{
SocketPacket socketData = (SocketPacket)asyn.AsyncState;
int iRx = 0;
iRx = socketData.m_currentSocket.EndReceive(asyn);
char[] chars = new char[iRx + 1];
System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder();
int charLen = d.GetChars(socketData.dataBuffer, 0, iRx, chars, 0);
MessageBox.Show("Incoming data: " + socketData.dataBuffer.Length.ToString() + " from socket(" + socketData.socket_id + ")");
char[] PACKET_END_IDENTIFIER = { (char)2, (char)1, (char)2, (char)1 };
for (int i = 0; i < iRx; i++)
{
GLOBAL_BUFFER.Add(chars[iRx]);
}
if (PacketEndReached(chars, PACKET_END_IDENTIFIER))
{
// done reading the data
PROCESS_PACKET(GLOBAL_BUFFER);
}
WaitForData(socketData.m_currentSocket, socketData.socket_id);
}
My socket buffer size is set to 100. If I send 1000 bytes, they would be split up in 10 chunks and onDataRecieved would get triggered 10 times.
All I need to do is keep reading the data into buffer for each individual packet sent my client until PacketEndReached gets triggered
then pass the buffer to another function that will process the data.
If I define a GLOBAL_BUFFER for storing incoming data, then if client sends data from multiple threads, wouldn't the data get mixed up? I need a way to read all the data for each individual packet sent my client.
Thanks!
UPDATE:
This is my current class:
public partial class TCP_SERVER
{
const int MAX_CLIENTS = 3000;
const int MAX_SOCKET_BUFFER_SIZE = 10;
public AsyncCallback pfnWorkerCallBack;
private Socket m_mainSocket;
private Socket[] m_workerSocket = new Socket[MAX_CLIENTS];
private int m_clientCount = 0;
public GLOBAL_BUFFER;
public void StartServer(int listen_port)
public void OnClientConnect(IAsyncResult asyn)
public void ProcessIncomingData(char[] INCOMING_DATA, int CLIENT_ID)
public void OnDataReceived(IAsyncResult asyn)
}
As you can see GLOBAL_BUFFER is defined 'globally'. If client sends packet_1 that takes 10 seconds to send and at the same time packet_2 that takes 2 secs to send data would get mixed up. I need to collect data for each packet individually.

If at all possible, I would recommend allowing each client thread to have their own connection to the server. Doing so will help the Winsock stack differentiate the messages from each thread and avoid any bleeding of packets between messages. This will effectively allow you to benefit from the stack's ability to decipher which messages (and message segements) are intended to be grouped together before passing them to your application as a complete message.
The message design you describe while very primitive can only work (reliably) if you separate your threads to different connections (or otherwise provide a guarantee that only a single message will be sent from the client at a time). You employee a very primitive message framing technique to your communication which will aide in your effort to determining message boundries but the reason it is failing is because socketData.m_currentSocket.EndReceive(asyn); will only tell you the number of bytes received when the event is raised (not necessarily the total number of bytes in the message). Rather than relying on it to tell you how many bytes have been read, I'd suggest reading the incoming message incrementally from a loop within your async handler reading very small message segments until it discovers your end of message byte sequence. Doing so will tell your event when to quit reading and to pass the data on to something else to process it.
The way I typically approach message framing is to have a before message eye-catcher (some value that will rarely if ever be seen in the messaging), followed by the length of the message (encoded to taste, I personally use binary encoding for it's efficiency), , the message content, and finally a second eye-catcher at the end of your message. The eye-catchers serve as logical queues for message breaks in the protocol and the message length tells your server explicitly how many bytes to wait for. Doing it this way, you are guaranteed to receive the number of bytes necessary (if you don't it is a problem so discard and/or throw exception) and it provides a very explicit boundary between messages that you can code to, which allows intermittent spot checking and validation.

Simply use Dictionary<String,List<Char>> to replace your current GLOBAL_BUFFER,
store different PACKET_ID data into different List.
I strongly recommend you a perfect Socket Framework SuperSocket, you needn't write any socket code, it will significantly improve your development efficiency.

Windows socket.Send data isn't received until socket.Close

I'm developing a server application that asynchronously accepts TCP connections (BeginAccept/EndAccept) and data (BeginReceive/EndReceive). The protocol requires an ACK to be sent whenever the EOM character is found before it will send the next message. The accept and receive are working but the sending app is not receiving the ACK (sent synchronously).
private void _receiveTransfer(IAsyncResult result)
{
SocketState state = result.AsyncState as SocketState;
int bytesReceived = state.Socket.EndReceive(result);
if (bytesReceived == 0)
{
state.Socket.Close();
return;
}
state.Offset += bytesReceived;
state.Stream.Write(state.Buffer, 0, bytesReceived);
if (state.Buffer[bytesReceived - 1] == 13)
{
// process message
Messages.IMessage message = null;
try
{
var value = state.Stream.ToArray();
// do some work
var completed = true;
if (completed)
{
// send positive ACK
var ackMessage = string.Format(ack, message.TimeStamp.ToString("yyyyMMddhhmm"), message.MessageType, message.Id, "AA", message.Id);
var buffer = ASCIIEncoding.ASCII.GetBytes(ackMessage);
int bytesSent = state.Socket.Send(buffer, 0, buffer.Length, SocketFlags.None);
}
else
{
// send rejected ACK
var ackMessage = string.Format(ack, message.TimeStamp.ToString("yyyyMMddhhmm"), message.MessageType, message.Id, "AR", message.Id);
state.Socket.Send(ASCIIEncoding.ASCII.GetBytes(ackMessage));
}
}
catch (Exception e)
{
// log exception
// send error ACK
if (message != null)
{
var ackMessage = string.Format(ack, DateTime.Now.ToString("yyyyMMddhhmm"), message.MessageType, message.Id, "AE", message.Id);
state.Socket.Send(ASCIIEncoding.ASCII.GetBytes(ackMessage));
}
}
}
state.Socket.BeginReceive(state.Buffer, 0, state.Buffer.Length, SocketFlags.None, new AsyncCallback(_receiveTransfer), state);
}
The state.Socket.Send returns the correct number of bytes but the data isn't received until the socket is disposed.
Suggestions are appreciated.

you shouldn't do anything synchronous from async completion routines. Under load you can end up hijacking all IO completion threads from the thread pool and severly hurt performance, up to and including complete IO deadlock. So don't send ACKs synchronously from async callback.
protocols and formats that use preambles are easier to manage that those that use terminators. Ie. write the length of the message in the fixed size message header as opposed to detecting a terminator \0x13. Of course, this applies if the protocol is under your control to start with.
As for your question, you didn't specify if the same code as you posted is also on the client side too.

How long are you giving it? The network stack can buffer, and that could delay transmition. From MSDN:
To increase network efficiency, the
underlying system may delay
transmission until a significant
amount of outgoing data is collected.
A successful completion of the Send
method means that the underlying
system has had room to buffer your
data for a network send.
You might want to try flushing using the IOControl method.
edit
Actually, the IOControl flush will kill the buffer. You may want to check out the Two Generals Problem to see if your protocol will have some inherent problems.

try setting TCP_NODELAY socket option

Have you set the NoDelay property on the socket to true? When set to false (the default), data is buffered for up to 200 milliseconds before it's sent. The reason is to reduce network traffic by limiting the number of packets that are sent. Setting NoDelay to true will force the data to be sent sooner.