Develop Reference

C# - Seems to be receiving two TCP packets at once?

I'm making a chatroom application, and I send TCP packets to communicate between the server and the client.
I have the following code:
string returnMessage = "[EVT]USERSUCCESS";
bytes = Encoding.ASCII.GetBytes(returnMessage);
info.WriteToStream(bytes);
foreach (ConnectionInfo con in connections)
{
info.WriteToStream(bytes);
bytes = Encoding.ASCII.GetBytes("[EVT]USERJOIN;" + username);
con.WriteToStream(bytes);
}
However, when the client reads this, the response is:
[EVT]USERSUCCESS[EVT]USERJOIN;Name
Which seems as though it is receiving both packets at once..?
This is the code I have for receiving:
static void ServerListener()
{
while (true)
{
byte[] bytes = new byte[1024];
int numBytes = stream.Read(bytes, 0, bytes.Length);
string message = Encoding.ASCII.GetString(bytes, 0, numBytes);
if (HandleResponse(message) && !WindowHasFocus())
{
player.Play();
}
}
}
Which is run as a separate thread. HandleResponse() is fully working.
Thanks in advance!

Tcp is a stream protocol not a packet protocol.
You could get the bytes in mutiple packets or a single packet like you are doing.
What you need to do is put a byte that siginifies the end of a packet (such as the null byte)
Psudeo code:
Have a buffer that you add every byte received to.
If their is a null byte in the buffer then split the buffer at the null byte
Handle the left side as a complete packet. Keep the right side in the buffer until you get a new null byte.
Advanced TCP:
You may want to add your own error checking in your "packet"

How to read unknown data length in a TCP Client

I am relativity new to C#. In my TCP client have the following function which sends data to the server and returns the response:
private static TcpClient tcpint = new TcpClient(); //Already initiated and set up
private static NetworkStream stm; //Already initiated and set up
private static String send(String data)
{
//Send data to the server
ASCIIEncoding asen = new ASCIIEncoding();
byte[] ba = asen.GetBytes(data);
stm.Write(ba, 0, ba.Length);
//Read data from the server
byte[] bb = new byte[100];
int k = stm.Read(bb, 0, 100);
//Construct the response from byte array to string
StringBuilder sb = new StringBuilder();
for (int i = 0; i < k; i++)
{
sb.Append(bb[i].ToString());
}
//Return server response
return sb.ToString();
}
As you can see here, when I am reading the response from the server, I am reading it into a fix byte[] array of length 100 bytes.
byte[] bb = new byte[100];
int k = stm.Read(bb, 0, 100);
What do i do if the response from the server is more than 100 bytes? How can I read the data without me knowing what the max length of data form the server will be?

Typically, where there is not some specific intrinsic size of something, tcp protocols explicitly send the length of objects they are sending. One possible method for illustration:
size_t data_len = strlen(some_data_blob);
char lenstr[32];
sprintf(lenstr, "%zd\n", data_len);
send(socket, lenstr, strlen(lenstr));
send(socket, some_data_blob, data_len);
then when the receiver reads the length string, it knows exactly how mush data should follow (good programming practice is to trust but verify though -- if there is more or less data really sent -- say by an 'evil actor' -- you need to be prepared to handle that).

Not with respect to C# but a general answer on writing TCP application:
TCP is steam based protocol. It does not maintain message boundaries. So, the applications using TCP should take care of choosing the right method of data exchange between server and client. Its becomes more paramount if multiple messages gets sent and received on one connection.
One widely used method is to prepend the data message with the length bytes.
Ex:
[2 byte -length field][Actual Data].
The receiver of such data (be it server or client needs to decode length field, wait for until such event where as many bytes are received or raise an alarm on timeout and give up.
Another protocol that can be used is to have applications maintain message boundaries.
Ex:
`[START-of-MSG][Actual Data][END-of-MSG]
The reciever has to parse the data for Start-byte and End-byte (predefined by application protocol) and treat anything in between as data of interest.

hello i solved it with a list, i don't know the size of the complete package but i can read it in parts
List<byte> bigbuffer = new List<byte>();
byte[] tempbuffer = new byte[254];
//can be in another size like 1024 etc..
//depend of the data as you sending from de client
//i recommend small size for the correct read of the package
NetworkStream stream = client.GetStream();
while (stream.Read(tempbuffer, 0, tempbuffer.Length) > 0) {
bigbuffer.AddRange(tempbuffer);
}
// now you can convert to a native byte array
byte[] completedbuffer = new byte[bigbuffer.Count];
bigbuffer.CopyTo(completedbuffer);
//Do something with the data
string decodedmsg = Encoding.ASCII.GetString(completedbuffer);
I do this whith images and looks good, i thik than you dont know the size of the data if the porpouse is read a complete source with a unknow size

I was looking around for an answer to this, and noticed the Available property was added to TcpClient. It returns the amount of bytes available to read.
I'm assuming it was added after most of the replies, so I wanted to share it for others that may stumble onto this question.
https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.tcpclient.available?view=netframework-4.8

Proper way to receive network data in loop

I am writing a C# client application which will connect to the server written in python. My question is about receiving data in loop. The application structure is all about client asks server -> server responds to client. Everything works fine when the message is lower that actual buffer size (set in server). For example: server side buffer: 1024, client buffer size: 256, data length < 1kb. I run my application with following code:
int datacounter = 0;
byte[] recived = new byte[256];
StringBuilder stb = new StringBuilder();
serverStream.ReadTimeout = 1500;
try
{
while ((datacounter = serverStream.Read(recived, 0, 256)) > 0)
{
Console.WriteLine("RECIVED: " + datacounter.ToString());
stb.append(System.Text.Encoding.UTF8.GetString(recived, 0, datacounter));
}
}
catch { Console.WriteLine("Timeout!"); }
Then the application receives data in 4 loops (256 bytes each):
RECIVED: 256
RECIVED: 256
RECIVED: 256
RECIVED: 96
And then the timeout ticks, that ends the transmission and pass the complete data to later analysis (from stb object). I don't think using timeout is proper, but i don't know any other way to do this.
However, this way it works. Here we go with example, that does not:
server side buffer: 1024, client side buffer: 256, data length ~ 8kbytes (python side sends data in loop).
RECIVED: 256
RECIVED: 256
RECIVED: 256
RECIVED: 256
Then the timeout ticks (and obviosly the data is incomplete - got 1kb of 8kb). Sometimes the loop even ends after 1 run with 28 recived bytes and thats all before timeout. Python says that the data has been send properly. Here's the way i create the socket and serverStream object:
TcpClient clientSocket = new TcpClient();
clientSocket.Connect("x.y.z.x", 1234);
NetworkStream serverStream = clientSocket.GetStream();
Its not the TcpClient fault. Tried the same with clear sockets, created like:
new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
However that works similar. Is there a way, to make my loop work without timeout, receiving all data? I would like to keep the socket synchronous if possible.

I don't think there's anything wrong with your receive code functionally. I put together a test, and the receiver gets as much as you can send it (eg. 8 MBs), as long as you keep sending without 1.5 seconds pause, before timing out.
So it looks like your server is simply not sending "fast" enough.
To answer your question, timing is not the typical way of knowing when you have received a full message. One common, simple way of determining when a full message is received is to prefix the length of the full message on the sending side (eg. 4-byte int). Then on the receive side, first read 4 bytes, decode to the length, and then read that many more bytes.
You could also consider appending a message termination string, such as Environment.NewLine, to the end of your message. This has the advantage that you could call StreamReader.ReadLine(), which will block until the full message is received. This only works if the termination can NOT be included in the message itself.
If you can't alter the server protocol, is there any other way of knowing you have received a full message? (eg. checking for a NewLine at the end of the message, an XML end tag, or some other pattern.) If not, perhaps you could wait for the server to disconnect, otherwise it looks like you would be forced to find the right timing balance.
I am including the test code below in case you want to play around with it.
Server/Sending Side:
IPAddress localAddr = IPAddress.Parse("127.0.0.1");
TcpListener server = new TcpListener(localAddr, 13579);
server.Start();
TcpClient clientSocket = server.AcceptTcpClient();
NetworkStream stream = clientSocket.GetStream();
int bytesSent = 0;
int bytesToSend = 1 << 25;
int bufferSize = 1024;
string testMessage = new string('X', bufferSize);
byte[] buffer = UTF8Encoding.UTF8.GetBytes(testMessage);
while (bytesSent < bytesToSend)
{
int byteToSendThisRound = Math.Min(bufferSize, bytesToSend - bytesSent);
stream.Write(buffer, 0, byteToSendThisRound);
bytesSent += byteToSendThisRound;
}
Client/Receiving Side:
TcpClient client = new TcpClient("127.0.0.1", 13579);
NetworkStream serverStream = client.GetStream();
int totalBytesReceived = 0;
int datacounter = 0;
byte[] recived = new byte[256];
StringBuilder stb = new StringBuilder();
serverStream.ReadTimeout = 1500;
try
{
while ((datacounter = serverStream.Read(recived, 0, 256)) > 0)
{
totalBytesReceived += 256;
Console.WriteLine("RECIVED: {0}, {1}", datacounter, totalBytesReceived);
stb.Append(System.Text.Encoding.UTF8.GetString(recived, 0, datacounter));
}
}
catch { Console.WriteLine("Timeout!"); }

Why don't you dump the exception that makes your code go into the catch branch and find out? :)
catch (Exception ex) { Console.WriteLine("Timeout because of... " + ex.Message); }
--EDIT
Sorry I didn't see the timout. The question you're asking is if there's a way to do it without timeout. Yes, don't set any timeout and check if received number of bytes is smaller than the buffer size.
That is:
while ((datacounter = serverStream.Read(recived, 0, 256)) > 0)
{
Console.WriteLine("RECIVED: " + datacounter.ToString());
stb.append(System.Text.Encoding.UTF8.GetString(recived, 0, datacounter));
if(datacounter < 256) //you're good to go
break;
}

For anyone else who needs help with this
Just to add to Chi_Town_Don's answer, make sure you use stb.ToString() outside of the loop. And I've found that nothing will print out unless the loop breaks out. To do that if(!serverStream.DataAvailable()){break;} works wonders. That way you don't need to pass in the packet size or some other convoluted condition.

Stream.Read is combining two different reads

i have a simplistic file server\client application ive written in c#. but i commonly run into the problem that my stream writes two different reads into a single buffer. i have a synchronized stream, still isnt helping. any suggestions? thanks!
System.Threading.Thread.Sleep(25);
receive_fspos = new byte[30];
int bytesread = stream_1.Read(receive_fspos, 0, receive_fspos.Length);//this is where it gets combined
if (bytesread == 0)
{
finished = true;
System.Threading.Thread.Sleep(25);
}
string string_1 = utf.GetString(receive_fspos).TrimEnd(new char[] { (char)0 });
int fsposition = (int)Convert.ToInt64(string_1);
bytestosend = fsposition;
filestream.Position = fsposition;
byte[] buffer_1 = new byte[bufsize];
int bytesreadfromfs = filestream.Read(buffer_1, 0, buffer_1.Length);
stream_1.Write(buffer_1, 0, buffer_1.Length);
Console.Write("\rSent " + fsposition + " / " + length + " bytes");
finished = true;

I would not recommend writing your own stream method if you do not fully understand it.
The problem that you are having is because the incoming data is a stream of bytes that does not give you a way of knowing how many bytes in length that the message is.
In the code below you are stating that you would like to read "receive_fspos.Length" bytes of the stream. Since "receive_fspos.Length" is 30, the amount of bytes that will be read will be anywhere from 0 to 30.
If there is only 15 bytes that have been received by the connection. It will give you 15 bytes. If the message was 20 bytes long. Then the message is now split up into different segments.
If the first message was 4 bytes and the second message is 12 bytes. Now you have 2 messages and a set of 16 blank bytes at the end. Even worse those 16 "blank" bytes could be the beginning of a third message coming in to the stream.
If the message is 50 bytes long. Then you will only receive half of the message. Now you would need to add the bytes that were read to a seperate buffer. Read from the stream again. Then repeat this until you have determined that you have read the exact amount of bytes that are needed to complete the entire message. Then concat all of the read bytes back to a single byte[].
receive_fspos = new byte[30];
int bytesread = stream_1.Read(receive_fspos, 0, receive_fspos.Length);//this is where it gets combined
Instead of rolling your own loop please use the BCL methods. It sounded like you are using strings so this would be the preferred method.. I would suggest the following.
using(NetworkStream networkStream = tcpClient.GetStream())
using(StreamReader streamReader = new StreamReader(networkStream))
using(StreamWriter streamWriter = new StreamWriter(networkStream))
{
networkStream.ReadTimeout = timeout; //Set a timeout to stop the stream from reading indefinately
//To receive a string
string incomingString = stream.ReadLine();
//To send a string
stream.WriteLine(messageToSend);
stream.Flush();
}
Your answer clarified that you are trying to send a file. For this I would recommend sending an array of bytes[]. Using this method you can send anything that can be serialized. This includes a file. Please note that the size of the file is limited since it must be kept in memory. To write a larger file you would want to save the data in chunks as it is being streamed in.
//Please note that if the file size is large enough. It may be preferred to use a stream instead of holding the entire file in memory.
byte[] fileAsBytes = File.ReadAllBytes(fileName);
using(NetworkStream networkStream = tcpClient.GetStream())
using(BinaryReader binaryReader = new BinaryReader(networkStream))
using(BinaryWriter binaryWriter = new BinaryWriter(networkStream))
{
networkStream.ReadTimeout = timeout; //Set a timeout to stop the stream from reading indefinately
//To receive a byte array
int incomingBytesLength = BinaryReader.ReadInt32(); //The header is 4 bytes that lets us know how large the incoming byte[] is.
byte[] incomingBytes = BinaryReader.ReadBytes(incomingBytesLength);
//To send a byte array
BinaryWriter.Write(fileAsBytes.Length); //Send a header of 4 bytes that lets the listener know how large the incoming byte[] is.
BinaryWriter.Write(fileAsBytes);
}

got it working, code > 30000 chars :\
it is a little messy but hey, it's functional.
server : https://www.dropbox.com/s/2wyccxpjbja10z3/Program.cs?m
client : https://www.dropbox.com/s/yp78nx4ubacsz6f/Program.cs?m

Data loss TCP IP C# [duplicate]

This question already has answers here:
Receiving data in TCP
(10 answers)
Closed 2 years ago.
Here's my code:
private void OnReceive(IAsyncResult result)
{
NetStateObject state = (NetStateObject)result.AsyncState;
Socket client = state.Socket;
int size = client.EndReceive(result);
byte[] data = state.Buffer;
object data = null;
using (MemoryStream stream = new MemoryStream(data))
{
BinaryFormatter formatter = new BinaryFormatter();
data = formatter.Deserialize(stream);
}
//todo: something with data
client.BeginReceive(
state.Buffer,
0,
NetStateObject.BUFFER_SIZE,
SocketFlags.None,
OnReceive,
state
);
}
state.Buffer has a maximum size of NetStateObject.BUFFER_SIZE (1024). Firstly, is this too big or too small? Second, if I send something larger than that, my deserialize messes up because the object it is trying to deserialize doesnt have all the information (because not all the data was sent). How do I make sure that all my data has been received before I try to construct it and do something with it?
Completed Working Code
private void OnReceive(IAsyncResult result)
{
NetStateObject state = (NetStateObject)result.AsyncState;
Socket client = state.Socket;
try
{
//get the read data and see how many bytes we received
int bytesRead = client.EndReceive(result);
//store the data from the buffer
byte[] dataReceived = state.Buffer;
//this will hold the byte data for the number of bytes being received
byte[] totalBytesData = new byte[4];
//load the number byte data from the data received
for (int i = 0; i < 4; i++)
{
totalBytesData[i] = dataReceived[i];
}
//convert the number byte data to a numan readable integer
int totalBytes = BitConverter.ToInt32(totalBytesData, 0);
//create a new array with the length of the total bytes being received
byte[] data = new byte[totalBytes];
//load what is in the buffer into the data[]
for (int i = 0; i < bytesRead - 4; i++)
{
data[i] = state.Buffer[i + 4];
}
//receive packets from the connection until the number of bytes read is no longer less than we need
while (bytesRead < totalBytes + 4)
{
bytesRead += state.Socket.Receive(data, bytesRead - 4, totalBytes + 4 - bytesRead, SocketFlags.None);
}
CommandData commandData;
using (MemoryStream stream = new MemoryStream(data))
{
BinaryFormatter formatter = new BinaryFormatter();
commandData = (CommandData)formatter.Deserialize(stream);
}
ReceivedCommands.Enqueue(commandData);
client.BeginReceive(
state.Buffer,
0,
NetStateObject.BUFFER_SIZE,
SocketFlags.None,
OnReceive,
state
);
dataReceived = null;
totalBytesData = null;
data = null;
}
catch(Exception e)
{
Console.WriteLine("***********************");
Console.WriteLine(e.Source);
Console.WriteLine("***********************");
Console.WriteLine(e.Message);
Console.WriteLine("***********************");
Console.WriteLine(e.InnerException);
Console.WriteLine("***********************");
Console.WriteLine(e.StackTrace);
}
}

TCP is a stream protocol. It has no concept of packets. A single write call can be sent in multiple packets, and multiple write calls can be put into the same packet. So you need to implement your own packetizing logic on top of TCP.
There are two common ways to packetize:
Delimiter characters, this is usually used in text protocols, with the new-line being a common choice
Prefix the length to each packet, usually a good choice with binary protocols.
You store the size of a logical packet at the beginning of that packet. Then you read until you received enough bytes to fill the packet and start deserializing.

How do I make sure that all my data has been received before I try to construct it and do something with it?
You have to implement some protocol so you know.
While TCP is reliable, it does not guarantee that the data from single write at one end of the socket will appear as a single read at the other end: retries, packet fragmentation and MTU can all lead to data being received in different sized units by the receiver. You will get the data in the right order.
So you need to include some information when sending that allows the receiver to know when it has the complete message. I would also recommend including what kind of message and what version of the data (this will form the basis of being able to support different client and server versions together).
So the sender sends:
- Message type
- Message version
- Message size (in bytes)
And the receiver will loop, performing a read with a buffer and appending this to a master buffer (MemoryStream is good for this). Once the complete header is received it knows when the complete data has been received.
(Another route is to include some pattern as an "end of message" marker, but then you need to handle the same sequence of bytes occurring in the content—hard to do if the data is binary rather than text.)