Develop Reference

C# Read from SslStream continuously (long connection, last for up to days) and Efficiently without infinite loop

I am completely new to C#, and need to encrypt the data sent and received between client and server, after googled it for two days, learnt the best way is to use SslStream, some answers I found give good examples but they all somehow assume we just need to read one message and then close the connection, which is totally not my case, I have to read whenever a user triggers his device to send a message through the persistent connection.
one example from Microsoft documentation:
static string ReadMessage(SslStream sslStream)
{
// Read the message sent by the client.
// The client signals the end of the message using the
// "<EOF>" marker.
byte [] buffer = new byte[2048];
StringBuilder messageData = new StringBuilder();
int bytes = -1;
do
{
// Read the client's test message.
bytes = sslStream.Read(buffer, 0, buffer.Length);
// Use Decoder class to convert from bytes to UTF8
// in case a character spans two buffers.
Decoder decoder = Encoding.UTF8.GetDecoder();
char[] chars = new char[decoder.GetCharCount(buffer,0,bytes)];
decoder.GetChars(buffer, 0, bytes, chars,0);
messageData.Append (chars);
// Check for EOF or an empty message. <------ In my case,I don't have EOF
if (messageData.ToString().IndexOf("<EOF>") != -1)
{
break;
}
} while (bytes !=0);
return messageData.ToString();
}
and other answers actually tell me how to continuously read from a SslStream, but they are using infinite loop to do it, on the server side, there could be thousands clients connected to it, so the possible poor performance concerns me,like this one :
Read SslStream continuously in C# Web MVC 5 project
So I want to know if there is a better way to continuously read from a persistent SslStream connection.
I know with bare socket I can use SocketAsyncEventArgs to know when there is new data ready, I hope I could do this with SslStream, probably I misunderstand something, any ideas would be appreciated, thanks in advance.

Here's my shot at it. Instead of looping forever, I chose recursion. This method will return immediately but will fire an event when EOF is hit and continue to keep reading:
public static void ReadFromSSLStreamAsync(
SslStream sslStream,
Action<string> result,
Action<Exception> error,
StringBuilder stringBuilder = null)
{
const string EOFToken = "<EOF>";
stringBuilder = stringBuilder ?? new StringBuilder();
var buffer = new byte[4096];
try
{
sslStream.BeginRead(buffer, 0, buffer.Length, asyncResult =>
{
// Read all bytes avaliable from stream and then
// add them to string builder
{
int bytesRead;
try
{
bytesRead = sslStream.EndRead(asyncResult);
}
catch (Exception ex)
{
error?.Invoke(ex);
return;
}
// Use Decoder class to convert from bytes to
// UTF8 in case a character spans two buffers.
var decoder = Encoding.UTF8.GetDecoder();
var buf = new char[decoder.GetCharCount(buffer, 0, bytesRead)];
decoder.GetChars(buffer, 0, bytesRead, buf, 0);
stringBuilder.Append(buf);
}
// Find the EOFToken, if found copy all data before the token
// and send it to event, then remove it from string builder
{
int tokenIndex;
while((tokenIndex = stringBuilder.ToString().IndexOf(EOFToken)) != -1)
{
var buf = new char[tokenIndex];
stringBuilder.CopyTo(0, buf, 0, tokenIndex);
result?.Invoke(new string(buf));
stringBuilder.Remove(0, tokenIndex + EOFToken.Length);
}
}
// Continue reading...
ReadFromSSLStreamAsync(sslStream, result, error, stringBuilder);
}, null);
}
catch(Exception ex)
{
error?.Invoke(ex);
}
}
You could call it as so:
ReadFromSSLStreamAsync(sslStream, sslData =>
{
Console.WriteLine($"Finished: {sslData}");
}, error =>
{
Console.WriteLine($"Errored: {error}");
});
It's not TaskAsync, so you don't have to await on it. But it is asynchronous so your thread can go on to do other things.

Consider checking out the following asnwer. SSLStream was derived from the Stream class therefore the ReadAsnyc method can be used. Code below, read until the <EOF> delimiter characters then return with the received message as string.
internal static readonly byte[] EOF = Encoding.UTF8.GetBytes("<EOF>");
internal static async Task<string> ReadToEOFAsync(Stream stream)
{
byte[] buffer = new byte[8192];
using (MemoryStream memoryStream = new MemoryStream())
{
long eofLength = EOF.LongLength;
byte[] messageTail = new byte[eofLength];
while (!messageTail.SequenceEqual(EOF))
{
int bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length);
await memoryStream.WriteAsync(buffer, 0, bytesRead);
Array.Copy(memoryStream.GetBuffer(), memoryStream.Length - eofLength, messageTail, 0, eofLength);
}
// Truncate the EOF tail from the data stream
byte[] result = new byte[memoryStream.Length - eofLength];
Array.Copy(memoryStream.GetBuffer(), 0, result, 0, result.LongLength);
return Encoding.UTF8.GetString(result);
}
}
The received messages was appended to the memoryStream. The first Array.Copy copies the message tail from the buffer. If the message tail is euqals to the <EOF> then it stops reading from the stream. Second copy is to ensure truncating the delimiter characters from the message.
Note: There is a more sophisticated way of slicing using Span introduced in .NET Core 2.1.

C# TCPClient-Server doesn't always send the full data

I'm currently working on a TCPClient and Server. Lately I added an encryption for the messages, and had no trouble. Once I started noticing that I'm getting a weird error like this:
But It's totally random, and no idea why. It happens at larger messages, but as I said, not always.
Checking the byte[] length at the server side says 1920 (Sometimes it says 1920 on the client too, and thats the point when i dont have error)
On client it says a lot lesser.
I actually think that sometimes the client doesn't receive the full byte that It should, this is how I do It:
Client:
byte[] bb = new byte[12288];
int k = stm.Read(bb, 0, 12288);
string message = Encoding.UTF8.GetString(bb, 0, k);
MessageBox.Show(message.Length.ToString()); // This sometimes says 1460, and 1920
message = Encrypter.DecryptData(message); // Error here If the length is not 1920
Server:
bmsg = Encrypter.EncryptData(((int)Codes.XYZEnum) + "=" + data);
Logger.Log(bmsg.Length.ToString()); // Original msg, always says 1920
s.Send(asen.GetBytes(bmsg));
s.Close();
What could be the problem? Should I try async sending?
SOLUTION:
Server code, took me a little while to make it cool:
System.Net.Sockets.Socket s = myList.AcceptSocket(); // Accept the connection
Stream stream = new NetworkStream(s); // Create the stream object
byte[] leng = new byte[4]; // We will put the length of the upcoming message in a 4 length array
int k2 = s.Receive(leng); // Receive the upcoming message length
if (BitConverter.IsLittleEndian)
{
Array.Reverse(leng);
}
int upcominglength = (BitConverter.ToInt32(leng, 0)); // Convert it to int
byte[] b = ByteReader(upcominglength, stream); // Create the space for the bigger message, read all bytes until the received length!
string message = Encoding.UTF8.GetString(b, 0, b.Length); // Convert it to string!
internal byte[] ByteReader(int length, Stream stream)
{
byte[] data = new byte[length];
using (MemoryStream ms = new MemoryStream())
{
int numBytesRead;
int numBytesReadsofar = 0;
while (true)
{
numBytesRead = stream.Read(data, 0, data.Length);
numBytesReadsofar += numBytesRead;
ms.Write(data, 0, numBytesRead);
if (numBytesReadsofar == length)
{
break;
}
}
return ms.ToArray();
}
}
Client code, and it is working nicely!:
var result = tcpclnt.BeginConnect(User.IP, User.Port, null, null);
var success = result.AsyncWaitHandle.WaitOne(TimeSpan.FromSeconds(3)); // Connect with timeout
if (!success)
{
return "Failed to connect!";
}
Stream stm = tcpclnt.GetStream(); // get the stream
UTF8Encoding asen = new UTF8Encoding();
byte[] ba = asen.GetBytes(msg); // get the bytes of the message we are sending
byte[] intBytes = BitConverter.GetBytes(ba.Length); // Get the length of that in bytes
if (BitConverter.IsLittleEndian)
{
Array.Reverse(intBytes);
}
stm.Write(intBytes, 0, intBytes.Length); // Write the length in the stream!
stm.Flush(); // Clear the buffer!
stm.Write(ba, 0, ba.Length); // Write the message we are sending!
// If we have answers....
byte[] bb = new byte[10000];
int k = stm.Read(bb, 0, 10000);
string mmessage = Encoding.UTF8.GetString(bb, 0, k);
// If we have answers....
tcpclnt.Close(); // Close the socket

Because only 8Kb can to send by once packet. if you have large data you need use cycle.

How to get all data from NetworkStream

I am trying to read all data present in the buffer of the Machine connected through TCP/IP but i don't know why i am not getting all data ,some data is getting Missed.
Here is the code that i am using ..
using (NetworkStream stream = client.GetStream())
{
byte[] data = new byte[1024];
int numBytesRead = stream.Read(data, 0, data.Length);
if (numBytesRead > 0)
{
string str= Encoding.ASCII.GetString(data, 0, numBytesRead);
}
}
Please tell me what i am missing to get all the data from the machine.
Thanks in advance..

The problem with your code is that you will not get all the data if the data size is bigger than the buffer size (1024 bytes in your case) so you have to Read the stream inside the loop. Then you can Write all the data inside a MemoryStream until the end of the NetworkStream.
string str;
using (NetworkStream stream = client.GetStream())
{
byte[] data = new byte[1024];
using (MemoryStream ms = new MemoryStream())
{
int numBytesRead ;
while ((numBytesRead = stream.Read(data, 0, data.Length)) > 0)
{
ms.Write(data, 0, numBytesRead);
}
str = Encoding.ASCII.GetString(ms.ToArray(), 0, (int)ms.Length);
}
}

This example from MSDN: NetworkStream.DataAvailable shows how you can use that property to do so:
// Examples for CanRead, Read, and DataAvailable.
// Check to see if this NetworkStream is readable.
if(myNetworkStream.CanRead)
{
byte[] myReadBuffer = new byte[1024];
StringBuilder myCompleteMessage = new StringBuilder();
int numberOfBytesRead = 0;
// Incoming message may be larger than the buffer size.
do{
numberOfBytesRead = myNetworkStream.Read(myReadBuffer, 0, myReadBuffer.Length);
myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead));
}
while(myNetworkStream.DataAvailable);
// Print out the received message to the console.
Console.WriteLine("You received the following message : " +
myCompleteMessage);
}
else
{
Console.WriteLine("Sorry. You cannot read from this NetworkStream.");
}

Try this:
private string GetResponse(NetworkStream stream)
{
byte[] data = new byte[1024];
using (MemoryStream memoryStream = new MemoryStream())
{
do
{
stream.Read(data, 0, data.Length);
memoryStream.Write(data, 0, data.Length);
} while (stream.DataAvailable);
return Encoding.ASCII.GetString(memoryStream.ToArray(), 0, (int)memoryStream.Length);
}
}

Try this code:
using (NetworkStream stream = client.GetStream())
{
while (!stream.DataAvailable)
{
Thread.Sleep(20);
}
if (stream.DataAvailable && stream.CanRead)
{
Byte[] data = new Byte[1024];
List<byte> allData = new List<byte>();
do
{
int numBytesRead = stream.Read(data,0,data.Length);
if (numBytesRead == data.Length)
{
allData.AddRange(data);
}
else if (numBytesRead > 0)
{
allData.AddRange(data.Take(numBytesRead));
}
} while (stream.DataAvailable);
}
}
Hope this helps, it should prevent that you miss any data sended to you.

The synchronous method sometimes does not display the request body. Using the asynchronous method stably displays request body.
string request = default(string);
StringBuilder sb = new StringBuilder();
byte[] buffer = new byte[client.ReceiveBufferSize];
int bytesCount;
if (client.GetStream().CanRead)
{
do
{
bytesCount = client.GetStream().ReadAsync(buffer, 0, buffer.Length).Result;
sb.Append(Encoding.UTF8.GetString(buffer, 0, bytesCount));
}
while(client.GetStream().DataAvailable);
request = sb.ToString();
}

TCP itself does not have any ways to define "end of data" condition. This is responsibility of application level portocol.
For instance see HTTP request description:
A client request (consisting in this case of the request line and only one header field) is followed by a blank line, so that the request ends with a double newline
So, for request end of data is determined by two newline sequences. And for response:
Content-Type specifies the Internet media type of the data conveyed by the HTTP message, while Content-Length indicates its length in bytes.
The response content size is specified in header before data.
So, it's up to you how to encode amount of data transferred at once - it can be just first 2 or 4 bytes in the beginning of the data holding total size to read or more complex ways if needed.

for my scenario, the message itself was telling the length of subsequent message. here is the code
int lengthOfMessage=1024;
string message = "";
using (MemoryStream ms = new MemoryStream())
{
int numBytesRead;
while ((numBytesRead = memStream.Read(MessageBytes, 0, lengthOfMessage)) > 0)
{
lengthOfMessage = lengthOfMessage - numBytesRead;
ms.Write(MessageBytes, 0, numBytesRead);
}
message = Encoding.ASCII.GetString(ms.ToArray(), 0, (int)ms.Length);
}

#George Chondrompilas answer is correct but instead of writing it by yourself you can use CopyTo function which does the same :
https://stackoverflow.com/a/65188160/4120180

What is the correct way to read from NetworkStream in .NET

I've been struggling with this and can't find a reason why my code is failing to properly read from a TCP server I've also written. I'm using the TcpClient class and its GetStream() method but something is not working as expected. Either the operation blocks indefinitely (the last read operation doesn't timeout as expected), or the data is cropped (for some reason a Read operation returns 0 and exits the loop, perhaps the server is not responding fast enough). These are three attempts at implementing this function:
// this will break from the loop without getting the entire 4804 bytes from the server
string SendCmd(string cmd, string ip, int port)
{
var client = new TcpClient(ip, port);
var data = Encoding.GetEncoding(1252).GetBytes(cmd);
var stm = client.GetStream();
stm.Write(data, 0, data.Length);
byte[] resp = new byte[2048];
var memStream = new MemoryStream();
int bytes = stm.Read(resp, 0, resp.Length);
while (bytes > 0)
{
memStream.Write(resp, 0, bytes);
bytes = 0;
if (stm.DataAvailable)
bytes = stm.Read(resp, 0, resp.Length);
}
return Encoding.GetEncoding(1252).GetString(memStream.ToArray());
}
// this will block forever. It reads everything but freezes when data is exhausted
string SendCmd(string cmd, string ip, int port)
{
var client = new TcpClient(ip, port);
var data = Encoding.GetEncoding(1252).GetBytes(cmd);
var stm = client.GetStream();
stm.Write(data, 0, data.Length);
byte[] resp = new byte[2048];
var memStream = new MemoryStream();
int bytes = stm.Read(resp, 0, resp.Length);
while (bytes > 0)
{
memStream.Write(resp, 0, bytes);
bytes = stm.Read(resp, 0, resp.Length);
}
return Encoding.GetEncoding(1252).GetString(memStream.ToArray());
}
// inserting a sleep inside the loop will make everything work perfectly
string SendCmd(string cmd, string ip, int port)
{
var client = new TcpClient(ip, port);
var data = Encoding.GetEncoding(1252).GetBytes(cmd);
var stm = client.GetStream();
stm.Write(data, 0, data.Length);
byte[] resp = new byte[2048];
var memStream = new MemoryStream();
int bytes = stm.Read(resp, 0, resp.Length);
while (bytes > 0)
{
memStream.Write(resp, 0, bytes);
Thread.Sleep(20);
bytes = 0;
if (stm.DataAvailable)
bytes = stm.Read(resp, 0, resp.Length);
}
return Encoding.GetEncoding(1252).GetString(memStream.ToArray());
}
The last one "works", but it certainly looks ugly to put a hard-coded sleep inside the loop considering that sockets already support read timeouts! Do I need to setup some property(ies) on the TcpClient of the NetworkStream? Does the problem resides in the server? The server don't close the connections, it is up to the client to do so. The above is also running inside the UI thread context (test program), maybe it has something to do with that...
Does someone know how to properly use NetworkStream.Read to read data until no more data is available? I guess what I'm wishing for is something like the old Win32 winsock timeout properties... ReadTimeout, etc. It tries to read until the timeout is reached, and then return 0... But it sometimes seem to return 0 when data should be available (or on the way.. can Read return 0 if is available?) and it then blocks indefinitely on the last read when data is not available...
Yes, I'm at a loss!

Networking code is notoriously difficult to write, test and debug.
You often have lots of things to consider such as:
what "endian" will you use for the data that is exchanged (Intel x86/x64 is based on little-endian) - systems that use big-endian can still read data that is in little-endian (and vice versa), but they have to rearrange the data. When documenting your "protocol" just make it clear which one you are using.
are there any "settings" that have been set on the sockets which can affect how the "stream" behaves (e.g. SO_LINGER) - you might need to turn certain ones on or off if your code is very sensitive
how does congestion in the real world which causes delays in the stream affect your reading/writing logic
If the "message" being exchanged between a client and server (in either direction) can vary in size then often you need to use a strategy in order for that "message" to be exchanged in a reliable manner (aka Protocol).
Here are several different ways to handle the exchange:
have the message size encoded in a header that precedes the data - this could simply be a "number" in the first 2/4/8 bytes sent (dependent on your max message size), or could be a more exotic "header"
use a special "end of message" marker (sentinel), with the real data encoded/escaped if there is the possibility of real data being confused with an "end of marker"
use a timeout....i.e. a certain period of receiving no bytes means there is no more data for the message - however, this can be error prone with short timeouts, which can easily be hit on congested streams.
have a "command" and "data" channel on separate "connections"....this is the approach the FTP protocol uses (the advantage is clear separation of data from commands...at the expense of a 2nd connection)
Each approach has its pros and cons for "correctness".
The code below uses the "timeout" method, as that seems to be the one you want.
See http://msdn.microsoft.com/en-us/library/bk6w7hs8.aspx. You can get access to the NetworkStream on the TCPClient so you can change the ReadTimeout.
string SendCmd(string cmd, string ip, int port)
{
var client = new TcpClient(ip, port);
var data = Encoding.GetEncoding(1252).GetBytes(cmd);
var stm = client.GetStream();
// Set a 250 millisecond timeout for reading (instead of Infinite the default)
stm.ReadTimeout = 250;
stm.Write(data, 0, data.Length);
byte[] resp = new byte[2048];
var memStream = new MemoryStream();
int bytesread = stm.Read(resp, 0, resp.Length);
while (bytesread > 0)
{
memStream.Write(resp, 0, bytesread);
bytesread = stm.Read(resp, 0, resp.Length);
}
return Encoding.GetEncoding(1252).GetString(memStream.ToArray());
}
As a footnote for other variations on this writing network code...when doing a Read where you want to avoid a "block", you can check the DataAvailable flag and then ONLY read what is in the buffer checking the .Length property e.g. stm.Read(resp, 0, stm.Length);

Setting the underlying socket ReceiveTimeout property did the trick. You can access it like this: yourTcpClient.Client.ReceiveTimeout. You can read the docs for more information.
Now the code will only "sleep" as long as needed for some data to arrive in the socket, or it will raise an exception if no data arrives, at the beginning of a read operation, for more than 20ms. I can tweak this timeout if needed. Now I'm not paying the 20ms price in every iteration, I'm only paying it at the last read operation. Since I have the content-length of the message in the first bytes read from the server I can use it to tweak it even more and not try to read if all expected data has been already received.
I find using ReceiveTimeout much easier than implementing asynchronous read... Here is the working code:
string SendCmd(string cmd, string ip, int port)
{
var client = new TcpClient(ip, port);
var data = Encoding.GetEncoding(1252).GetBytes(cmd);
var stm = client.GetStream();
stm.Write(data, 0, data.Length);
byte[] resp = new byte[2048];
var memStream = new MemoryStream();
var bytes = 0;
client.Client.ReceiveTimeout = 20;
do
{
try
{
bytes = stm.Read(resp, 0, resp.Length);
memStream.Write(resp, 0, bytes);
}
catch (IOException ex)
{
// if the ReceiveTimeout is reached an IOException will be raised...
// with an InnerException of type SocketException and ErrorCode 10060
var socketExept = ex.InnerException as SocketException;
if (socketExept == null || socketExept.ErrorCode != 10060)
// if it's not the "expected" exception, let's not hide the error
throw ex;
// if it is the receive timeout, then reading ended
bytes = 0;
}
} while (bytes > 0);
return Encoding.GetEncoding(1252).GetString(memStream.ToArray());
}

As per your requirement, Thread.Sleep is perfectly fine to use because you are not sure when the data will be available so you might need to wait for the data to become available. I have slightly changed the logic of your function this might help you little further.
string SendCmd(string cmd, string ip, int port)
{
var client = new TcpClient(ip, port);
var data = Encoding.GetEncoding(1252).GetBytes(cmd);
var stm = client.GetStream();
stm.Write(data, 0, data.Length);
byte[] resp = new byte[2048];
var memStream = new MemoryStream();
int bytes = 0;
do
{
bytes = 0;
while (!stm.DataAvailable)
Thread.Sleep(20); // some delay
bytes = stm.Read(resp, 0, resp.Length);
memStream.Write(resp, 0, bytes);
}
while (bytes > 0);
return Encoding.GetEncoding(1252).GetString(memStream.ToArray());
}
Hope this helps!

SharpZipLib Deflater creates bad data

Original compressed data can be correctly inflated back. However, if I inflate data, deflate, and again inflate, resulting data are incorrect (e.g. simple data extraction, its modification and again compression - only now when testing no modification occurs, so I can test it).
Resulting data are somehow "damaged". The starting (about) 40 bytes are OK, and then "block" of incorrect data follows (remnants of original data are still there, but many bytes are missing).
Changing compression level doesn't help (except setting NO_COMPRESSION creates somehow incomplete stream).
Question is simple: why is that happening?
using ICSharpCode.SharpZipLib.Zip.Compression;
public byte[] Inflate(byte[] inputData)
{
Inflater inflater = new Inflater(false);
using (var inputStream = new MemoryStream(inputData))
using (var ms = new MemoryStream())
{
var inputBuffer = new byte[4096];
var outputBuffer = new byte[4096];
while (inputStream.Position < inputData.Length)
{
var read = inputStream.Read(inputBuffer, 0, inputBuffer.Length);
inflater.SetInput(inputBuffer, 0, read);
while (inflater.IsNeedingInput == false)
{
var written = inflater.Inflate(outputBuffer, 0, outputBuffer.Length);
if (written == 0)
break;
ms.Write(outputBuffer, 0, written);
}
if (inflater.IsFinished == true)
break;
}
inflater.Reset();
return ms.ToArray();
}
}
public byte[] Deflate(byte[] inputData)
{
Deflater deflater = new Deflater(Deflater.BEST_SPEED, false);
deflater.SetInput(inputData);
deflater.Finish();
using (var ms = new MemoryStream())
{
var outputBuffer = new byte[65536 * 4];
while (deflater.IsNeedingInput == false)
{
var read = deflater.Deflate(outputBuffer);
ms.Write(outputBuffer, 0, read);
if (deflater.IsFinished == true)
break;
}
deflater.Reset();
return ms.ToArray();
}
}
Edit: My bad, by mistake I rewrote first several bytes of the original compressed data. This isn't SharpZipLib fault, but mine.

I know this is a tangential answer, but the exact same thing happened to me, I abandoned SharpZipLib and went to DotNetZip :
http://dotnetzip.codeplex.com/
Easier API, no corrupt or strange byte order files.