Develop Reference

Multiple Client Sockets from one instance of a program connecting different devices- working really slow

I have built application that connects with help of TCP Sockets to 4 devices.
For that i created an TCP class with asynchronous methods to send and receive data.
public delegate void dataRec(string recStr);
public event dataRec dataReceiveEvent;
public Socket socket;
public void Connect(string IpAdress, int portNum)
{
socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
IPEndPoint epServer = new IPEndPoint(IPAddress.Parse(IpAdress), portNum);
socket.Blocking = false;
AsyncCallback onconnect = new AsyncCallback(OnConnect);
m_sock.BeginConnect(epServer, onconnect, socket);
}
public void SetupRecieveCallback(Socket sock)
{
try
{
AsyncCallback recieveData = new AsyncCallback(OnRecievedData);
sock.BeginReceive(m_byBuff, 0, m_byBuff.Length, SocketFlags.None, recieveData, sock);
}
catch (Exception ex)
{
//nevermind
}
}
public void OnRecievedData(IAsyncResult ar)
{
// Socket was the passed in object
Socket sock = (Socket)ar.AsyncState;
try
{
int nBytesRec = sock.EndReceive(ar);
if (nBytesRec > 0)
{
string sRecieved = Encoding.ASCII.GetString(m_byBuff, 0, nBytesRec);
OnAddMessage(sRecieved);
SetupRecieveCallback(sock);
}
else
{
sock.Shutdown(SocketShutdown.Both);
sock.Close();
}
}
catch (Exception ex)
{
//nevermind
}
}
public void OnAddMessage(string sMessage)
{
if (mainProgram.InvokeRequired)
{
{
scanEventCallback d = new scanEventCallback(OnAddMessage);
mainProgram.BeginInvoke(d, sMessage);
}
}
else
{
dataReceiveEvent(sMessage);
}
}
I have 4 devices with 4 different IP's and Port's that i send data, and from which i receive data.
So i created 4 different instances of a class mentioned.
When i receive data i call callback functions to do their job with the data i received (OnAddMessage event).
The connection with devices is really good, latency is like: 1-2ms~ (its in internal network).
Functions i call by callbacks are preety fast, each function is not more than 100ms.
The problem is it is working really slow, and its not caused by callback functions.
For each data i send to device, i receive one message from it.
When i start sending them, and stop after like 1 minute of working, the program keep receiving data for like 4-5 sec, even when i turn off devices- its like some kind of lag, that i receive data, that should be delivered a lot earlier.
It looks like something is working really slow.
Im getting like 1 message per second from each device, so it shouldnt be a big deal.
Any ideas what else i should do or set, or what actually could slow me down ?

You haven't posted all the relevant code, but here are some things to pay attention to:
With a network sniffer, like Wireshark or tcpdump, you can see what is actually going on.
Latency it not the only relevant factor for "connection speed". Look also at throughput, packet loss, re-transmissions, etc..
Try to send and receive in large chunks. Sending and receive only single bytes is slow because it has a lot of overhead.
The receiver should read data faster than the sender can send it, or else internal buffers (OS, network) will fill up.
Try to avoid a "chatty" protocol, basically synchronous request/reply, if possible.
If you have a chatty protocol, you can get better performance by disabling the Nagle algorithm. The option to disable this algorithm is often called "TCP no delay" or similar.
Don't close/reopen the connection for each message. TCP connection setup and teardown has quite some overhead.
If you have long standing open TCP connections, close the connection when the connection is idle for some time, for example several minutes.

C# Socket.Receive() malfunctions when TCP package is larger than MTU

I'm using C# Socket class to send/receive TCP messages to/from our server by short connection. The pseudo-code (exclude some false tolerant logic to make it clear) is as followings.
class MyTCPClient {
private Socket mSocket;
MyTCPClient() {
mSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
}
void Connect(ipEndpoint, sendTimeOut, receiveTimeOut) {
mSocket.Connect(ipEndpoint);
mSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.SendTimeout, sendTimeOut);
mSocket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveTimeout, receiveTimeOut);
}
void Send(msg) {
mSocket.Send(msg);
}
void Receive(data) {
ByteStreamWriter bsw = new ByteStreamWriter();
int totalLen = 0;
while (true) {
byte[] temp = new byte[1024];
int len = mSocket.Receive(temp, SocketFlags.None);
System.Threading.Thread.Sleep(50); // This is the weirdest part
if (len <= 0) break;
totalLen += len;
bsw.WriteBytes(temp);
}
byte[] buff = bsw.GetBuffer();
data = new byte[totalLen];
Array.Copy(buff, 0, data, 0, totalLen);
}
~MyTCPClient() {
mSocket.Close();
}
}
I was using this class to request the same message from our server several times by short connection, and the following things occurred (only when the message size was larger than one MTU -- 1500 bytes).
If the "Sleep(50)" was commented out, the most of the times (90%) I received wrong "data", specifically to say, the "totalLen" was right, but the "data" was wrong.
If I replaced "Sleep(50)" to "Sleep(10)", about half of the times I received wrong "data", and "totalLen" was also right always.
If I used "Sleep(50)", occasionally, I received wrong "data".
I can guarantee that, every time, our server sent the right data, and the client received the right data too at TCP layer (I used WireShark to monitor all messages through the port I used). Is there anyone who can help to answer why my C# code cannot get the right data?
Also, if I use mSocket.Available instead of the return value of Socket.Receive() to judge the while loop, I would always get the right data and data length...

You don't truncate temp to len bytes before writing it. All the bytes after len in temp have not been filled with current data and thus contain nonsensical, stale date.
When copying to another stream you could use stream.Write(temp, 0, len)

Out of memory exception when receiving large amounts of data on TCP connection

I have an application that receives data from GPRS clients in the field on a TCP connection. From time to time the GPRS client devices loses connection and buffers the data, when the connection is restored all the buffered data is send to the TCP connection, cauing my application to throw a System.OutOfMemoryException.
I presume this is because the data received is bigger than my buffer size (which is set to int.MaxValue).
How do I prevent my application to run out of memory?
How do I make sure that I do not lose any of the incomming data?
Below is the code used to listen for, and handle incomming data
public void Listen(string ip, int port)
{
_logger.Debug("All.Tms.SensorDataServer : SensorDataListener : Listen");
try
{
var listener = new TcpListener(IPAddress.Parse(ip), port);
listener.Start();
while (true)
{
var client = listener.AcceptTcpClient();
client.SendBufferSize = int.MaxValue;
var thread = new Thread(() => ReadAndHandleIncommingData(client));
thread.IsBackground = true;
thread.Start();
}
}
catch (Exception ex)
{
_logger.Error("TMS.Sensor.SensorDataServer : SensorDataListener : Error : ", ex);
}
}
and
private void ReadAndHandleIncommingData(TcpClient connection)
{
try
{
var stream = connection.GetStream();
var data = new byte[connection.ReceiveBufferSize];
var bytesRead = stream.Read(data, 0, System.Convert.ToInt32(connection.ReceiveBufferSize));
var sensorDataMapper = new SensorDataMapperProvider().Get(data);
if (sensorDataMapper != null)
{
_sensorDataHandler.Handle(sensorDataMapper.Map(data));
}
}
catch (Exception ex)
{
_logger.Error("TMS.Sensor.SensorDataServer : SensorDataListener : ReadAndHandleIncommingData : Error : ", ex);
}
finally
{
try
{
connection.Close();
}
catch(Exception ex)
{
_logger.Error("All.Tms.SensorDataServer : SensorDataListener : ReadAndHandleIncommingData : Error : ", ex);
}
}
}

About buffers
OutOfMemoryExceptions are thrown when there is no sequential memory left.
In your case, that means connection.ReceiveBufferSize is too big to keep in memory in one piece. Not because the data received is bigger than your buffer size.
You can use a smaller, fixed buffer to get the received bytes, append it somewhere and use the same buffer to receive the rest of the data, until you have it all.
One thing to look out for is the collection you use to store the received data. You can't use List<byte> for example because it stores its elements in a single array under the hood which makes no difference than getting everything in one go - like you do now.
You can see MemoryTributary, a stream implementation meant to replace MemoryStream. You can copy your stream to this one and keep it as a stream. This page also contains a lot of information that can help you understand the causes of the OutOfMemoryExceptions.
In addition, I wrote a buffer manager to provide fixed sized buffers a while ago. You can find that in Code Review, here.
About threads
Creating a thread for every single connection is brutal. They cost you 1 MB each so you should either use ThreadPool or better, IOCP (via asynchronous methods of the Socket class).
You may want to look into these for common pitfalls and best practices about socket programming:
Stephen Cleary (the blog) | TCP/IP .NET Sockets FAQ
Stack Overflow | How to write a scalable Tcp/Ip based server
Code Project | C# SocketAsyncEventArgs High Performance Socket Code

Use a fixed receive buffer, and parse data incrementally

Where is data sent by UDP stored?

I had never used UDP before, so I gave it a go. To see what would happen, I had the 'server' send data every half a second, and the client receive data every 3 seconds. So even though the server is sending data much faster than the client can receive, the client still receives it all neatly one by one.
Can anyone explain why/how this happens? Where is the data buffered exactly?
Send
class CSimpleSend
{
CSomeObjServer obj = new CSomeObjServer();
public CSimpleSend()
{
obj.changedVar = varUpdated;
obj.threadedChangeSomeVar();
}
private void varUpdated(int var)
{
string send = var.ToString();
byte[] packetData = System.Text.UTF8Encoding.UTF8.GetBytes(send);
string ip = "127.0.0.1";
int port = 11000;
IPEndPoint ep = new IPEndPoint(IPAddress.Parse(ip), port);
Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
client.SendTo(packetData, ep);
Console.WriteLine("Sent Message: " + send);
Thread.Sleep(100);
}
}
All CSomeObjServer does is increment an integer by one every half second
Receive
class CSimpleReceive
{
CSomeObjClient obj = new CSomeObjClient();
public Action<string> showMessage;
Int32 port = 11000;
UdpClient udpClient;
public CSimpleReceive()
{
udpClient = new UdpClient(port);
showMessage = Console.WriteLine;
Thread t = new Thread(() => ReceiveMessage());
t.Start();
}
private void ReceiveMessage()
{
while (true)
{
//Thread.Sleep(1000);
IPEndPoint remoteIPEndPoint = new IPEndPoint(IPAddress.Any, port);
byte[] content = udpClient.Receive(ref remoteIPEndPoint);
if (content.Length > 0)
{
string message = Encoding.UTF8.GetString(content);
if (showMessage != null)
showMessage("Recv:" + message);
int var_out = -1;
bool succ = Int32.TryParse(message, out var_out);
if (succ)
{
obj.alterSomeVar(var_out);
Console.WriteLine("Altered var to :" + var_out);
}
}
Thread.Sleep(3000);
}
}
}
CSomeObjClient stores the variable and has one function (alterSomeVar) to update it
Ouput:
Sent Message: 1
Recv:1
Altered var to :1
Sent Message: 2
Sent Message: 3
Sent Message: 4
Sent Message: 5
Recv:2
Altered var to :2
Sent Message: 6
Sent Message: 7
Sent Message: 8
Sent Message: 9
Sent Message: 10
Recv:3
Altered var to :3

The operating system kernel maintains separate send and receive buffers for each UDP and TCP socket. If you google SO_SNDBUF and SO_RCVBUF you'll find lots of information about them.
When you send data, it is copied from your application space into the send buffer. From there it is copied to the network interface card, and then onto the wire. The receive side is the reverse: NIC to receive buffer, where it waits until you read it. Additionally copies and buffering can also occur, depending on the OS.
It is critical to note that the sizes of these buffers can vary radically. Some systems might default to as little as 4 kilobytes, while others give you 2 megabytes. You can find the current size using getsockopt() with SO_SNDBUF or SO_RCVBUF and likewise set it using setsockopt(). But many systems limit the size of the buffer, sometimes to arbitrarily small amounts. This is typically a kernel value like net.core.wmem_max or net.core.rmem_max, but the exact reference will vary by system.
Also note that setsockopt() can fail even if you request an amount less than the supposed limit. So to actually get a desired size, you need to repeatedly call setsockopt() using decreasing amounts until it finally succeeds.
The following page is a Tech Note from my company which touches on this topic a little bit and provides references for some common systems: http://www.dataexpedition.com/support/notes/tn0024.html

It looks to me like the UdpClient-Class provides a buffer for received data. Try using a socket directly. You might also want to set that sockets ReceiveBufferSize to zero, even though I believe it is only used for TCP connections.

C# Socket BeginReceive / EndReceive capturing multiple messages

Problem:
When I do something like this:
for (int i = 0; i < 100; i++)
{
SendMessage( sometSocket, i.ToString());
Thread.Sleep(250); // works with this, doesn't work without
}
With or without the sleep the server logs sending of separate messages. However without the sleep the client ends up receiving multiple messages in single OnDataReceived so the client will receive messages like:
0,
1,
2,
34,
5,
678,
9 ....
Server sending Code:
private void SendMessage(Socket socket, string message)
{
logger.Info("SendMessage: Preparing to send message:" + message);
byte[] byteData = Encoding.ASCII.GetBytes(message);
if (socket == null) return;
if (!socket.Connected) return;
logger.Info("SendMessage: Sending message to non " +
"null and connected socket with ip:" + socket.RemoteEndPoint);
// Record this message so unit testing can very this works.
socket.Send(byteData);
}
Client receiving code:
private void OnDataReceived(IAsyncResult asyn)
{
logger.Info("OnDataReceived: Data received.");
try
{
SocketPacket theSockId = (SocketPacket)asyn.AsyncState;
int iRx = theSockId.Socket.EndReceive(asyn);
char[] chars = new char[iRx + 1];
System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder();
int charLen = d.GetChars(theSockId.DataBuffer, 0, iRx, chars, 0);
System.String szData = new System.String(chars);
logger.Info("OnDataReceived: Received message:" + szData);
InvokeMessageReceived(new SocketMessageEventArgs(szData));
WaitForData(); // .....
Socket Packet:
public class SocketPacket
{
private Socket _socket;
private readonly int _clientNumber;
private byte[] _dataBuffer = new byte[1024]; ....
My hunch is it's something to do with the buffer size or its just the between the OnDataReceived and EndReceive we're getting multiple messages.
Update: It turns out when I put a Thread.Sleep at the start of OnDataReceived it gets every message. Is the only solution to this wrapping my message in a prefix of length and an string to signify the end?

This is expected behaviour. A TCP socket represents a linear stream of bytes, not a sequence of well-delimited “packets”. You must not assume that the data you receive is chunked the same way it was when it was sent.
Notice that this has two consequences:
Two messages may get merged into a single callback call. (You noticed this one.)
A single message may get split up (at any point) into two separate callback calls.
Your code must be written to handle both of these cases, otherwise it has a bug.

There is no need to abandon Tcp because it is stream oriented.
You can fix the problems that you are having by implementing message framing.
See
http://blogs.msdn.com/malarch/archive/2006/06/26/647993.aspx
also:
http://nitoprograms.blogspot.com/2009/04/message-framing.html

TCP sockets don't always send data right away -- in order to minimize network traffic, TCP/IP implementations will often buffer the data for a bit and send it when it sees there's a lull (or when the buffer's full).
If you want to ensure that the messages are processed one by one, you'll need to either set socket.NoDelay = true (which might not help much, since data received may still be bunched up together in the receive buffer), implement some protocol to separate messages in the stream (like prefixing each message with its length, or perhaps using CR/LF to separate them), or use a message-oriented protocol like SCTP (which might not be supported without additional software) or UDP (if you can deal with losing messages).