Develop Reference

Stream.Read is combining two different reads

i have a simplistic file server\client application ive written in c#. but i commonly run into the problem that my stream writes two different reads into a single buffer. i have a synchronized stream, still isnt helping. any suggestions? thanks!
System.Threading.Thread.Sleep(25);
receive_fspos = new byte[30];
int bytesread = stream_1.Read(receive_fspos, 0, receive_fspos.Length);//this is where it gets combined
if (bytesread == 0)
{
finished = true;
System.Threading.Thread.Sleep(25);
}
string string_1 = utf.GetString(receive_fspos).TrimEnd(new char[] { (char)0 });
int fsposition = (int)Convert.ToInt64(string_1);
bytestosend = fsposition;
filestream.Position = fsposition;
byte[] buffer_1 = new byte[bufsize];
int bytesreadfromfs = filestream.Read(buffer_1, 0, buffer_1.Length);
stream_1.Write(buffer_1, 0, buffer_1.Length);
Console.Write("\rSent " + fsposition + " / " + length + " bytes");
finished = true;

I would not recommend writing your own stream method if you do not fully understand it.
The problem that you are having is because the incoming data is a stream of bytes that does not give you a way of knowing how many bytes in length that the message is.
In the code below you are stating that you would like to read "receive_fspos.Length" bytes of the stream. Since "receive_fspos.Length" is 30, the amount of bytes that will be read will be anywhere from 0 to 30.
If there is only 15 bytes that have been received by the connection. It will give you 15 bytes. If the message was 20 bytes long. Then the message is now split up into different segments.
If the first message was 4 bytes and the second message is 12 bytes. Now you have 2 messages and a set of 16 blank bytes at the end. Even worse those 16 "blank" bytes could be the beginning of a third message coming in to the stream.
If the message is 50 bytes long. Then you will only receive half of the message. Now you would need to add the bytes that were read to a seperate buffer. Read from the stream again. Then repeat this until you have determined that you have read the exact amount of bytes that are needed to complete the entire message. Then concat all of the read bytes back to a single byte[].
receive_fspos = new byte[30];
int bytesread = stream_1.Read(receive_fspos, 0, receive_fspos.Length);//this is where it gets combined
Instead of rolling your own loop please use the BCL methods. It sounded like you are using strings so this would be the preferred method.. I would suggest the following.
using(NetworkStream networkStream = tcpClient.GetStream())
using(StreamReader streamReader = new StreamReader(networkStream))
using(StreamWriter streamWriter = new StreamWriter(networkStream))
{
networkStream.ReadTimeout = timeout; //Set a timeout to stop the stream from reading indefinately
//To receive a string
string incomingString = stream.ReadLine();
//To send a string
stream.WriteLine(messageToSend);
stream.Flush();
}
Your answer clarified that you are trying to send a file. For this I would recommend sending an array of bytes[]. Using this method you can send anything that can be serialized. This includes a file. Please note that the size of the file is limited since it must be kept in memory. To write a larger file you would want to save the data in chunks as it is being streamed in.
//Please note that if the file size is large enough. It may be preferred to use a stream instead of holding the entire file in memory.
byte[] fileAsBytes = File.ReadAllBytes(fileName);
using(NetworkStream networkStream = tcpClient.GetStream())
using(BinaryReader binaryReader = new BinaryReader(networkStream))
using(BinaryWriter binaryWriter = new BinaryWriter(networkStream))
{
networkStream.ReadTimeout = timeout; //Set a timeout to stop the stream from reading indefinately
//To receive a byte array
int incomingBytesLength = BinaryReader.ReadInt32(); //The header is 4 bytes that lets us know how large the incoming byte[] is.
byte[] incomingBytes = BinaryReader.ReadBytes(incomingBytesLength);
//To send a byte array
BinaryWriter.Write(fileAsBytes.Length); //Send a header of 4 bytes that lets the listener know how large the incoming byte[] is.
BinaryWriter.Write(fileAsBytes);
}

got it working, code > 30000 chars :\
it is a little messy but hey, it's functional.
server : https://www.dropbox.com/s/2wyccxpjbja10z3/Program.cs?m
client : https://www.dropbox.com/s/yp78nx4ubacsz6f/Program.cs?m

C# How to send more than one byte streams using TCPClient?

I'm trying to make Login feature by using TCP Client. I have two forms: Client-side and Server-side.
The Client-side handles user input while Server-side connect to database.
The problem is the reader result, which always combine both inputs into one long string like this:
myusernamemypassword
Here's part of the sender of client-side:
byte[] byteUsername = Encoding.Unicode.GetBytes(username);
byte[] bytePassword = Encoding.Unicode.GetBytes(password);
NetworkStream stream = client.GetStream();
stream.Write(username, 0, byteUsername.Length);
stream.Write(password, 0, bytePassword.Length);
//if offset != 0, the code always return ArgumentOutOfRangeException
And the reader in server-side:
return Encoding.Unicode.GetString(buffer, 0, buffer.Length)
After long-search I found the solution, but it can only handle two strings; the third+ string will be combined together with the second string. I need to send at least 4 strings for other feature.
Here's the updated reader code:
List<string> list = new List<string>();
int totalRead = 0;
do
{
int read = client.GetStream().Read(buffer, totalRead, buffer.Length - totalRead);
totalRead += read;
list.Add(Encoding.Unicode.GetString(buffer, 0, totalRead));
} while (client.GetStream().DataAvailable);
I don't quite understand this code. How can it knows which bytes are part of the first string? The size of Read() parameter is length-totalRead which is length - 0, it should return the whole buffer right?
Any solution guys?
Thanks before

You should prefix each string with its length (in bytes, not characters) as a 4-byte integer.
This way, the server will know how many bytes to read into each string.

NetworkStream.Read() and SSL

Was just wondering if anybody can look at the following code and explain to me why the length variable returns 0:
textToBeEncrypted = Encoding.ASCII.GetString(buffer);
txtEncryptedText = AESEncryption(textToBeEncrypted, key, true);
byte[] encText = Encoding.ASCII.GetBytes(txtEncryptedText);
NetworkStream stream = s.GetStream();
stream.Write(encText, 0, PACKET_SIZE);
s.ReceiveTimeout = Timeout;
int length = stream.Read(buffer, 0, PACKET_SIZE);
if (length == PACKET_SIZE)
{
string decText = Encoding.ASCII.GetString(encText);
txtDecryptedText = AESDecryption(decText, key, true);
buffer = Encoding.ASCII.GetBytes(txtDecryptedText);
retval = Decode();
}
After I've encoded everything using AES, I'm writing out 1366 bytes of data in encText (PACKET_SIZE is 1036). I get no complaints regarding the Send; the data is sent out happily. When it tries to read it back in, however, length is always 0, meaning I don't get to enter the decode statement bracket. Any ideas? (retval is a string, before anyone asks)

If the length is zero from this:
int length = stream.Read(buffer, 0, PACKET_SIZE);
it means the other machine has closed their outbound socket (your inbound socket), and no more data is ever going to be available.
You should also be very careful about this:
if (length == PACKET_SIZE)
{...}
There is absolutely no guarantee about what you read. What you should do here is buffer the data until you have an entire message (frame), and then process what you buffered. In particular, if the other end sends less than PACKET_SIZE bytes, your code is guaranteed to do nothing. Even if the other end sent exactly PACKET_SIZE bytes, it would be pretty unlikely to arrive in exactly a single chunk of PACKET_SIZE bytes.
For example:
int length;
MemoryStream ms = new MemoryStream();
while((length = stream.Read(buffer, 0, buffer.Length)) > 0) {
ms.Write(buffer, 0, length); // append what we just recieved
// now: could check `ms` to see if we have a "frame" here...
}
//...or you could just process the entire recieved data here

Strange behaviour of networkstream.read() in C#

When i write to a network stream two seperate byte array, i sometimes dont get the first byte array.
Why is that?
For eg This fails, the header is not received, sometimes by Read() on other side
byte[] header = msg.getByteHeader();
byte[] data = msg.getByteData();
clientStream.Write(header, 0, header.Length);
clientStream.Write(data, 0, data.Length);
clientStream.Flush();
however this succeeds
NetworkStream clientStream = tcpClient.GetStream();
byte[] header = msg.getByteHeader();
byte[] data = msg.getByteData();
int pos = 0;
Array.Copy(header, 0, message, pos, header.Length);
pos += header.Length;
Array.Copy(data, 0, message, pos, data.Length);
clientStream.Write(message, 0, message.Length);
This is how my Read() looks
try
{
//blocks until a client sends a message
bytesRead = clientStream.Read(message, 0, 4);
//string stringData = Encoding.ASCII.GetString(message, 0, bytesRead);
len = BitConverter.ToInt32(message, 0);
//MessageBox.Show(len.ToString());
bytesRead = clientStream.Read(message, 0, 5 + len);
}

i believe this is a timing issue. There's a lag between when you first open socket communication and when you can read the first data from the buffer. It's not instantaneous. You can query the DataAvailable boolean status of network stream before attempting to read. If there's no DataAvailable, Sleep the thread for say 100 ms and then try reading again.

You could troubleshoot it by commenting out the second Write and see if your server is sent any data.
Your reading mechanism does look very very fragile and I'd agree with Simon Fox that it does not look like it's correct. Why is the second read asking for len + 5 bytes? I would have thought it would only be len bytes since the first read was for the 4 header bytes.
If I was you I'd add a delimiter to the start of your header transmission. That will allow your receivers to scan for that to determine the start of a packet. With TCP you will often get fragemented transmissions or multiple transmissions bundled into the same packet. Things will go wrong once you deploy to real networks such as the internet if you are always relying on getting exactly the number of bytes you request.
Either that or switch to UDP where you can rely on having one transmission per packet.

Aren't you overwritting what you read in the first call to read with the second call? The second argument to Read is the offset at which to start storing the data read, both calls use 0 so the second overwrites the first...

TCP Framing with Binary Protocol

Hey, I'm having an issue seperating packets using a custom binary protocol.
Currently the server side code looks like this.
public void HandleConnection(object state)
{
TcpClient client = threadListener.AcceptTcpClient();
NetworkStream stream = client.GetStream();
byte[] data = new byte[4096];
while (true)
{
int recvCount = stream.Read(data, 0, data.Length);
if (recvCount == 0) break;
LogManager.Debug(Utility.ToHexDump(data, 0, recvCount));
//processPacket(new MemoryStream(data, 0, recvCount));
}
LogManager.Debug("Client disconnected");
client.Close();
Dispose();
}
I've been watching the hex dumps of the packets, and sometimes the entire packet comes in one shot, let's say all 20 bytes. Other times it comes in fragmented, how do I need to buffer this data to be able to pass it to my processPacket() method correctly. I'm attempting to use a single byte opcode header only, should I add something like a (ushort)contentLength to the header aswell? I'm trying to make the protocol as lightweight as possible, and this system won't be sending very large packets(< 128 bytes).
The client side code I'm testing with is, as follows.
public void auth(string user, string password)
{
using (TcpClient client = new TcpClient())
{
client.Connect(IPAddress.Parse("127.0.0.1"), 9032);
NetworkStream networkStream = client.GetStream();
using (BinaryWriter writer = new BinaryWriter(networkStream))
{
writer.Write((byte)0); //opcode
writer.Write(user.ToUpper());
writer.Write(password.ToUpper());
writer.Write(SanitizationMgr.Verify()); //App hash
writer.Write(Program.Seed);
}
}
}
I'm not sure if that could be what's messing it up, and binary protocol doesn't seem to have much info on the web, especially where C# is involved. Any comment's would be helpful. =)
Solved with this, not sure if it's correct, but it seems to give my handlers just what they need.
public void HandleConnection(object state)
{
TcpClient client = threadListener.AcceptTcpClient();
NetworkStream stream = client.GetStream();
byte[] data = new byte[1024];
uint contentLength = 0;
var packet = new MemoryStream();
while (true)
{
int recvCount = stream.Read(data, 0, data.Length);
if (recvCount == 0) break;
if (contentLength == 0 && recvCount < headerSize)
{
LogManager.Error("Got incomplete header!");
Dispose();
}
if(contentLength == 0) //Get the payload length
contentLength = BitConverter.ToUInt16(data, 1);
packet.Write(data, (int) packet.Position, recvCount); //Buffer the data we got into our MemStream
if (packet.Length < contentLength + headerSize) //if it's not enough, continue trying to read
continue;
//We have a full packet, pass it on
//LogManager.Debug(Utility.ToHexDump(packet));
processPacket(packet);
//reset for next packet
contentLength = 0;
packet = new MemoryStream();
}
LogManager.Debug("Client disconnected");
client.Close();
Dispose();
}

You should just treat it as a stream. Don't rely on any particular chunking behaviour.
Is the amount of data you need always the same? If not, you should change the protocol (if you can) to prefix the logical "chunk" of data with the length in bytes.
In this case you're using BinaryWriter on one side, so attaching a BinaryReader to the NetworkStream returned by TcpClient.GetStream() would seem like the easiest approach. If you really want to capture all the data for a chunk at a time though, you should go back to my idea of prefixing the data with its length. Then just loop round until you've got all the data.
(Make sure you've got enough data to read the length though! If your length prefix is 4 bytes, you don't want to read 2 bytes and miss the next 2...)