Develop Reference

C# NetworkStream.DataAvailable seems to be unreliable

I have an app that uses a TCP socket to exchange byte arrays which in most cases contain JSON string data. What I'm experiencing is that, for larger messages and less than ideal network conditions, use of NetworkStream.DataAvailable does NOT seem to be a reliable way to detect an end of message. It seems that in some cases DataAvailable is set to false even when only part of the message has been transmitted by peer (which is using TcpClient.GetStream().Write(data, 0, data.Length). This results in incomplete data being passed back to the app, which in the case of a JSON message, means deserialization fails.
I've tried two implementations which exhibit the same issue:
Implementation 1:
byte[] Data;
byte[] buffer = new byte[2048];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = ClientStream.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
BytesRead += read;
if (!ClientStream.DataAvailable) break;
}
Data = ms.ToArray();
}
Implementation 2:
byte[] Data;
using (MemoryStream ms = new MemoryStream())
{
byte[] buffer = new byte[2048];
do
{
int read = ClientStream.Read(buffer, 0, buffer.Length);
if (read > 0)
{
ms.Write(buffer, 0, read);
BytesRead += read;
}
}
while (ClientStream.DataAvailable);
Data = ms.ToArray();
}
It seems one solution that works really well but is completely sub-optimal is to add a Thread.Sleep in case NetworkStream.DataAvailable is false (while inside the loop) to allow data to be delivered. However this severely limits overall IOPS which I would like to avoid, i.e.
Implementation 3 (works, but suboptimal)
byte[] Data;
using (MemoryStream ms = new MemoryStream())
{
byte[] buffer = new byte[2048];
do
{
int read = ClientStream.Read(buffer, 0, buffer.Length);
if (read > 0)
{
ms.Write(buffer, 0, read);
BytesRead += read;
}
if (!ClientStream.DataAvailable) System.Threading.Thread.Sleep(250);
}
while (ClientStream.DataAvailable);
Data = ms.ToArray();
}
I'd really like to find a way to remain in the loop until all of the data is delivered. As I mentioned, I'm doing a simple write operation on the client from zero to data length, so I'm not thinking there is an issue there.
Has anyone had any experience like this before and a recommendation?

It seems .DataAvailable is indeed reliable and that, since the data arrives over the network potentially at a rate slower than data is read from the stream, .DataAvailable can flip-flop between the start and end of what my application thinks is a message.
I'm answering and closing this as I believe the only solutions to this are:
1) add an over-arching receive timeout value, and perform a thread.sleep in the read loop, and expiring the operation once the receive timeout is reached
2) implement some mechanism of indicating the data payload size - either explicitly or by creating a system of metadata headers - to indicate how much data should be read, and exiting after that much data has been read or the operation has timed out
These two are the best I could come up with and seem to be validated by the likes of other TCP-based protocols like HTTP and generally any other RPC out there.
Hopefully this saves someone some time.

Losing characters over socket in string

I'm sending a message over a socket.
On the client side i'm assembling the message using StringBuilder
StringBuilder sb = new StringBuilder(message);
sb.Insert(0, (char)11);
sb.Append((char)28);
sb.Append((char)13);
Sending it from client to server
Byte[] data = new Byte[1024];
data = Encoding.ASCII.GetBytes(message.ToString());
NetworkStream stream = client.GetStream();
stream.Write(data, 0, data.Length);
Server Side
StringBuilder message = new StringBuilder(Encoding.ASCII.GetString(bytesReceived, 0, bytesReceived.Length));
I then want to check to see if my message is contained within the correct container but for some reason the last 2 characters equal 0 in the check instead of the correct 28 and 13.
if (((int)messsage[message.Length - 2] == 28) && ((int)message[message.Length - 1] == 13))
Thanks in advance for any help
Added Data that was asked for
byte[] bytes = new byte[1024];
NetworkStream stream = tcpClient.GetStream();
stream.Read(bytes, 0, bytes.Length);

Stream.Read will read up to bytes.Length bytes, it's return value will tell you how many bytes it actually read.
If that is not enough, then you will need to call Stream.Read() again.
Also, bytes.Length will always return the length of the array, not the number of bytes read.
Looking at your sending code, you probably want to read as much as you can from the stream, append what was read to the string builder, then check to see if the last 2 characters are 28 and 13, and if they are then you have your complete message.

C# How to send more than one byte streams using TCPClient?

I'm trying to make Login feature by using TCP Client. I have two forms: Client-side and Server-side.
The Client-side handles user input while Server-side connect to database.
The problem is the reader result, which always combine both inputs into one long string like this:
myusernamemypassword
Here's part of the sender of client-side:
byte[] byteUsername = Encoding.Unicode.GetBytes(username);
byte[] bytePassword = Encoding.Unicode.GetBytes(password);
NetworkStream stream = client.GetStream();
stream.Write(username, 0, byteUsername.Length);
stream.Write(password, 0, bytePassword.Length);
//if offset != 0, the code always return ArgumentOutOfRangeException
And the reader in server-side:
return Encoding.Unicode.GetString(buffer, 0, buffer.Length)
After long-search I found the solution, but it can only handle two strings; the third+ string will be combined together with the second string. I need to send at least 4 strings for other feature.
Here's the updated reader code:
List<string> list = new List<string>();
int totalRead = 0;
do
{
int read = client.GetStream().Read(buffer, totalRead, buffer.Length - totalRead);
totalRead += read;
list.Add(Encoding.Unicode.GetString(buffer, 0, totalRead));
} while (client.GetStream().DataAvailable);
I don't quite understand this code. How can it knows which bytes are part of the first string? The size of Read() parameter is length-totalRead which is length - 0, it should return the whole buffer right?
Any solution guys?
Thanks before

You should prefix each string with its length (in bytes, not characters) as a 4-byte integer.
This way, the server will know how many bytes to read into each string.

SslStream read all bytes

When i am try to read from SslStream function Read() is never end if i am not set connection timeout but if i do i've got timeout exception. This guy have the same problem http://msdn.microsoft.com/en-us/library/system.net.security.sslstream.read.aspx. I don't what to do here the code
public byte[] ReadBytes()
{
this.bufferGlobal.Clear();
byte[] buffer = new byte[this.bufferSize];
int recv = this.stream.Read(buffer, 0, buffer.Length);
while (recv != 0)
{
addBytes(buffer, ref bufferGlobal, recv);
recv = this.stream.Read(buffer, 0, buffer.Length);
}
return (byte[])this.bufferGlobal.ToArray(typeof(byte));
}
Thx in advance.
UPD:
i think i find the answer. I can set read timeout on SslStream equal one, thats value does not make sense for socket alive (its mean you can download huge files and don't worry about SslStream he wouldn't close connection or interupt recieve data). I just testing this solution but seems works fine. Thx everybody.

You are reading from a network stream, which means you will not encounter the end of stream until the other side closes its half of the connection. It isn't enough for it to stop sending data. So, make your other program close the connection after it has sent all data it intends to.

The stream won't end until the connection is closed. This is the nature of all streams of an unknown length.
You either need to know the length ahead of time or you need to keep reading until the connection is closed. A common way is to transmit the byte length of the stream as a 64bit int. So the first 8 bytes of your stream is read into an int64, then the rest is the data.
Typically one reads in a stream one "buffer" at a time.
pseudo code
while(!Stream.end)
{
i = Stream.Read(buffer, buffer.length)
DestStream.Write(buffer, i)
}

bit late but I am using msdn code with following modification
if (sb.ToString().IndexOf("a OK") != -1 || sb.ToString().IndexOf("a BAD") != -1 || sb.ToString().IndexOf("a NO") != -1)
{
break;
}

please use this working patch if it works for you:
int bytesRead = 0;
string chunkString = "";
do
{
byte[] responseBuffer = new byte[CHUNKSIZE];
bytesRead = sslStream.Read(responseBuffer, 0, responseBuffer.Length);
ApplicationHelper.ApplicationLogger.WriteInfoLog("Bytes Received " + bytesRead);
if (bytesRead > 0)
{
chunkString = Encoding.UTF8.GetString(responseBuffer, 0, bytesRead);
responseXML += chunkString;
}
}
while (!chunkString.EndsWith(EOF));

In c# client/server program Data that sent from server is changed when received at client

Server side code:
byte[] size = new byte[4];
size = BitConverter.GetBytes(fileData.Length);
stream.Write(size, 0, 4);
Client side code:
byte[] size = new byte[4];
ReadWholeArray(s, size);
int fileSize = BitConverter.ToInt32(size, 0);
Definition of ReadWholeArray method:
public static void ReadWholeArray(Stream stream, byte[] data)
{
int offset = 0;
int remaining = data.Length;
while (remaining > 0)
{
int read = stream.Read(data, offset, remaining);
if (read <= 0)
throw new EndOfStreamException
(String.Format("End of stream reached with {0} bytes left to read", remaining));
remaining -= read;
offset += read;
}
}
The Program sends fileData.Length (the value for this instance is 2422) from server. On receiving this data at client side the value of received data is -772097985
Why the sent data is not received without alteration in value? What is the problem?

Okay, simple diagnostics to start with: log the individual contents of the byte array at both ends, so you can see what's going on there.
That way you can see if it's the binary data which is getting corrupted in your communication protocol, or whether BitConverter is causing your problem. For example, you could have a big-endian BitConverter at one end, and a little-endian BitConverter at the other. That seems unlikely, but it's possible - particularly if one of your server or client is running Mono rather than .NET itself.
If that does turn out to be the problem, you might want to use my EndianBitConverter class from MiscUtil, which lets you specify the endianness.
If the problem is in the communications layer, you quite possibly want to install Wireshark to see what's happening at the network level. Are you sure you've read all the data you're meant to have read so far, for example? (If you've only read 15 bytes before this, and the size is written at offset 16, then obviously you'll get the "extra" byte first.)

This works fine:
private void button2_Click(object sender, RoutedEventArgs e)
{
MemoryStream ms = new MemoryStream();
byte [] original = BitConverter.GetBytes((int)2224); // 176, 8, 0, 0
ms.Write(original, 0, original.Length);
ms.Seek(0, SeekOrigin.Begin);
byte [] data = new byte[4];
int count = ms.Read(data, 0, 4); // count is 4, data is 176, 8, 0, 0
int fileSize = BitConverter.ToInt32(data, 0); // is 2224
return;
}
Can you use WireShark or something to intercept the bytes? What kind of connection are you using? Could more data be being sent (i.e. telnet control characters at the start of the stream)? Can you debug each end and verify these values or write the byte array contents to a log file? By the way calling this: "byte[] size = new byte[4];" is wasteful because BitConverter.GetBytes() returns a new array.