Develop Reference

C# Read from SslStream continuously (long connection, last for up to days) and Efficiently without infinite loop

I am completely new to C#, and need to encrypt the data sent and received between client and server, after googled it for two days, learnt the best way is to use SslStream, some answers I found give good examples but they all somehow assume we just need to read one message and then close the connection, which is totally not my case, I have to read whenever a user triggers his device to send a message through the persistent connection.
one example from Microsoft documentation:
static string ReadMessage(SslStream sslStream)
{
// Read the message sent by the client.
// The client signals the end of the message using the
// "<EOF>" marker.
byte [] buffer = new byte[2048];
StringBuilder messageData = new StringBuilder();
int bytes = -1;
do
{
// Read the client's test message.
bytes = sslStream.Read(buffer, 0, buffer.Length);
// Use Decoder class to convert from bytes to UTF8
// in case a character spans two buffers.
Decoder decoder = Encoding.UTF8.GetDecoder();
char[] chars = new char[decoder.GetCharCount(buffer,0,bytes)];
decoder.GetChars(buffer, 0, bytes, chars,0);
messageData.Append (chars);
// Check for EOF or an empty message. <------ In my case,I don't have EOF
if (messageData.ToString().IndexOf("<EOF>") != -1)
{
break;
}
} while (bytes !=0);
return messageData.ToString();
}
and other answers actually tell me how to continuously read from a SslStream, but they are using infinite loop to do it, on the server side, there could be thousands clients connected to it, so the possible poor performance concerns me,like this one :
Read SslStream continuously in C# Web MVC 5 project
So I want to know if there is a better way to continuously read from a persistent SslStream connection.
I know with bare socket I can use SocketAsyncEventArgs to know when there is new data ready, I hope I could do this with SslStream, probably I misunderstand something, any ideas would be appreciated, thanks in advance.

Here's my shot at it. Instead of looping forever, I chose recursion. This method will return immediately but will fire an event when EOF is hit and continue to keep reading:
public static void ReadFromSSLStreamAsync(
SslStream sslStream,
Action<string> result,
Action<Exception> error,
StringBuilder stringBuilder = null)
{
const string EOFToken = "<EOF>";
stringBuilder = stringBuilder ?? new StringBuilder();
var buffer = new byte[4096];
try
{
sslStream.BeginRead(buffer, 0, buffer.Length, asyncResult =>
{
// Read all bytes avaliable from stream and then
// add them to string builder
{
int bytesRead;
try
{
bytesRead = sslStream.EndRead(asyncResult);
}
catch (Exception ex)
{
error?.Invoke(ex);
return;
}
// Use Decoder class to convert from bytes to
// UTF8 in case a character spans two buffers.
var decoder = Encoding.UTF8.GetDecoder();
var buf = new char[decoder.GetCharCount(buffer, 0, bytesRead)];
decoder.GetChars(buffer, 0, bytesRead, buf, 0);
stringBuilder.Append(buf);
}
// Find the EOFToken, if found copy all data before the token
// and send it to event, then remove it from string builder
{
int tokenIndex;
while((tokenIndex = stringBuilder.ToString().IndexOf(EOFToken)) != -1)
{
var buf = new char[tokenIndex];
stringBuilder.CopyTo(0, buf, 0, tokenIndex);
result?.Invoke(new string(buf));
stringBuilder.Remove(0, tokenIndex + EOFToken.Length);
}
}
// Continue reading...
ReadFromSSLStreamAsync(sslStream, result, error, stringBuilder);
}, null);
}
catch(Exception ex)
{
error?.Invoke(ex);
}
}
You could call it as so:
ReadFromSSLStreamAsync(sslStream, sslData =>
{
Console.WriteLine($"Finished: {sslData}");
}, error =>
{
Console.WriteLine($"Errored: {error}");
});
It's not TaskAsync, so you don't have to await on it. But it is asynchronous so your thread can go on to do other things.

Consider checking out the following asnwer. SSLStream was derived from the Stream class therefore the ReadAsnyc method can be used. Code below, read until the <EOF> delimiter characters then return with the received message as string.
internal static readonly byte[] EOF = Encoding.UTF8.GetBytes("<EOF>");
internal static async Task<string> ReadToEOFAsync(Stream stream)
{
byte[] buffer = new byte[8192];
using (MemoryStream memoryStream = new MemoryStream())
{
long eofLength = EOF.LongLength;
byte[] messageTail = new byte[eofLength];
while (!messageTail.SequenceEqual(EOF))
{
int bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length);
await memoryStream.WriteAsync(buffer, 0, bytesRead);
Array.Copy(memoryStream.GetBuffer(), memoryStream.Length - eofLength, messageTail, 0, eofLength);
}
// Truncate the EOF tail from the data stream
byte[] result = new byte[memoryStream.Length - eofLength];
Array.Copy(memoryStream.GetBuffer(), 0, result, 0, result.LongLength);
return Encoding.UTF8.GetString(result);
}
}
The received messages was appended to the memoryStream. The first Array.Copy copies the message tail from the buffer. If the message tail is euqals to the <EOF> then it stops reading from the stream. Second copy is to ensure truncating the delimiter characters from the message.
Note: There is a more sophisticated way of slicing using Span introduced in .NET Core 2.1.

How do I make sure I receive the whole message if it doesn't fit in the buffer in TCP?

I recently started to work with TCP in C#. I'm now at the point where I want the client to receive the data sent by the server.
I know that there is no guarantee for the client to receive all data at once. If the size of the data sent is bigger than the buffer's size at the client's side, then the data will be sent in parts. So my question is: how can I store all my received data in a byte array, and then convert it to the actual message when all is received?
I've set the buffer size to 1, so I can see what happens when all sent data doesn't fit in the buffer. Here are my methods in which I call stream.BeginRead() in Client.cs:
// Deliberately setting the buffer size to 1, to simulate what happens when the message doesn't fit in the buffer.
int bufferSize = 1;
byte[] receiveBuffer;
private void ConnectCallback(IAsyncResult result)
{
client.EndConnect(result);
Console.WriteLine("Connected to server.");
stream = client.GetStream();
// At this point, the client is connected, and we're expecting a message: "Welcome!"
receiveBuffer = new byte[bufferSize];
stream.BeginRead(receiveBuffer, 0, receiveBuffer.Length, new AsyncCallback(ReadCallback), stream);
}
private void ReadCallback(IAsyncResult result)
{
int bytesLength = stream.EndRead(result);
// Should be "Welcome!". But of course it's "W", because the bufferSize is 1.
string message = Encoding.UTF8.GetString(receiveBuffer, 0, bytesLength);
Console.WriteLine("Received message: {0}", receivedMessage);
// Reset the buffer and begin reading a new message, which will be "e".
// However, I want the whole message ("Welcome!") in one byte array.
receiveBuffer = new byte[bufferSize];
stream.BeginRead(receiveBuffer, 0, receiveBuffer.Length, ReadCallback, null);
}
This is the output when sending the message "Welcome!":
Connected to server.
Received message: W
Received message: e
Received message: l
Received message: c
Received message: o
Received message: m
Received message: e
Received message: !
Should I temporary store the data until the whole message has arrived, and then convert that to a string?
Follow up question: What if 2 messages are sent closely after each other, for example Welcome! and then What's your name? How do I distinguish the two messages then?

Should I temporary store the data until the whole message has arrived, and then convert that to a string?
Yes, exactly.
Follow up question: What if 2 messages are sent closely after each other, for example Welcome! and then What's your name? How do I distinguish the two messages then?
The general approach is to send the length of the message before the message itself. That way the receiving end will know when it has received a complete package.

As 500 - Internal Server Error already pointed out, you use a buffer for it. Here is some Code example:
Receiving:
while (true) //you should use a bool variable here to stop this on disconnect.
{
byte[] bytes;
bytes = ReadNBytes(ns, 4);
//read out the length field we know is there, because the server always sends it.
int msgLenth = BitConverter.ToInt32(bytes, 0);
bytes = ReadNBytes(ns, msgLenth);
//working with the buffer...
if (bytes.Length > 0)
{
try
{
//do stuff here. bytes contains your complete message.
}
catch (Exception e) { Log(e.Message); }
}
}
public static byte[] ReadNBytes(NetworkStream stream, int n)
{
byte[] buffer = new byte[n];
try
{
int bytesRead = 0;
int chunk;
while (bytesRead < n)
{
chunk = stream.Read(buffer, (int)bytesRead, buffer.Length - (int)bytesRead);
if (chunk == 0)
{
// error out
Log("Unexpected disconnect");
stream.Close();
}
bytesRead += chunk;
}
}
catch (Exception e) { Log(e.Message); }
return buffer;
}
To send stuff use something linke this:
public static void SendObject(NetworkStream ns, byte[] data)
{
byte[] lengthBuffer = BitConverter.GetBytes(data.Length);
ns.Write(lengthBuffer, 0, lengthBuffer.Length);
ns.Write(data, 0, data.Length);
}
I hope that helped!

How to properly send and receive complete data using sockets [duplicate]

I am working on socket C#. I've implemented a client server application using socket, but the problem is that the client doesn't receive all data sent by the server.
Here is the client application code. What should I do so that it would receive all data sent by the server?
strRecieved = "";
Socket soc = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
IPEndPoint endPoint = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 9001);
soc.Connect(endPoint);
byte[] msgBuffer = Encoding.Default.GetBytes(path);
soc.Send(msgBuffer, 0, msgBuffer.Length, 0);
byte[] buffer = new byte[2000];
int rec = soc.Receive(buffer);
strRecieved = String.Format(Encoding.Default.GetString(buffer));

First of all. If you're implementing some kind of streaming feature ( tcp/udp/file ) you should consider using some kind of protocol.
What is a protocol? It's just a scheme to use when streaming data. Example:
[4Bytes - length][lengthBytes - message][1Byte - termination indicator]
Knowing the protocol you can read all of the incoming bytes simply as such :
byte[] buffer = new byte[4];
stream.ReadBytes(buffer, 0, 4); // cast that to int and read the rest
int packetLen = BitConverter.ToInt32(buffer, 0);
buffer = new byte[packetLen];
stream.ReadBytes(buffer, 0, buffer.Length); // all bytes that was sent
Remember that you have to subtract thease 4 bytes in the length before sending the message.
EDIT:
Simple example on how to send and receive data using shared protocol.
// sender.cs
string _stringToSend = "some fancy string";
byte[] encodedString = Encoding.UTF8.GetBytes(_stringToSend);
List<byte> buffer = new List<byte>();
buffer.AddRange(BitConverter.GetBytes(encodedString.Length));
buffer.AddRange(encodedString);
netStream.WriteBytes(buffer.ToArray(), 0, buffer.Count);
// netStream sent message in protocol [#LEN - 4Bytes][#MSG - #LENBytes]
// simply speaking something like: 5ABCDE
// receiver.cs
byte[] buffer = new byte[sizeof(int)];
netStream.ReadBytes(buffer, 0, buffer.Length);
// receiver got the length of the message eg. 5
int dataLen = BitConverter.ToInt32(buffer, 0);
buffer = new byte[dataLen];
// now we can read an actual message because we know it's length
netStream.ReadBytes(buffer, 0, buffer.Length);
string receivedString = Encoding.UTF8.GetString(buffer);
// received string is equal to "some fancy string"
Making it simpler
This technique forces you to use desired protocol which in this example will be :
First 4 bytes sizeof(int) are indicating the length of the incoming packet
Every byte further is your packet until the end.
So right now you should make ProtocolHelper object:
public static class ProtocolHelper
{
public byte[] PackIntoProtocol(string message)
{
List<byte> result = new List<byte>();
byte[] messageBuffer = Encoding.UTF8.GetBytes(message);
result.AddRange(BitConverter.GetBytes(messageBuffer.Length), 0); // this is the first part of the protocol ( length of the message )
result.AddRange(messageBuffer); // this is actual message
return result.ToArray();
}
public string UnpackProtocol(byte[] buffer)
{
return Encoding.UTF8.GetString(buffer, 0, buffer.Length);
}
}
Now ( depending on method you've chosen to read from network ) you have to send and receive your message.
// sender.cs
string meMessage = "network message 1";
byte[] buffer = ProtocolHelper.PackIntoProtocol(meMessage);
socket.Send(buffer, 0, buffer.Length, 0);
// receiver.cs
string message = string.Empty;
byte[] buffer = new byte[sizeof(int)]; // or simply new byte[4];
int received = socket.Receive(buffer);
if(received == sizeof(int))
{
int packetLen = BitConverter.ToInt32(buffer);// size of our message
buffer = new byte[packetLen];
received = socket.Receive(buffer);
if( packetLen == received ) // we have full buffer
{
message = PacketHelper.UnpackProtocol(buffer);
}
}
Console.WriteLine(message); // output: "network message 1"

You're limiting the size of received messages by 2KB as you're using new byte[2000].
I think you could either:
Size up you buffer to meet you message's size needs; and/or
Split you message into more than one socket messages.
Given that 4-8K is a good size for buffering socket messages and assuming RAM size is not a issue I would start with that, say, new byte[8000].
Also, you can send socket messages splitted in chunks. Maybe this is a good idea for the case. For example, if you have msg as the message (or object) you want to send:
private static async Task SendAnswer(Message msg, WebSocket socket)
{
var answer = System.Text.Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(msg).ToCharArray());
var bufferSize = 8000;
var endOfMessage = false;
for (var offset = 0; offset < answer.Length; offset += bufferSize)
{
if (offset + bufferSize >= answer.Length)
{
bufferSize = answer.Length - offset;
endOfMessage = true;
}
await socket.SendAsync(new ArraySegment<byte>(answer, offset, bufferSize),
WebSocketMessageType.Text, endOfMessage, CancellationToken.None);
}
}
And when receiving, you can also split the reception in chunks, so you can control you buffer (and therefore you memory consumption). After hanlding the whole message, you should wait for another message from the client to do more stuff. Source
private async Task ReceiveMessage(WebSocket webSocket)
{
var buffer = new byte[8000];
var result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
while (!result.CloseStatus.HasValue)
{
string msg = Encoding.UTF8.GetString(new ArraySegment<byte>(buffer, 0, result.Count).ToArray());
while (!result.EndOfMessage)
{
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
msg += Encoding.UTF8.GetString(new ArraySegment<byte>(buffer, 0, result.Count).ToArray());
}
//At this point, `msg` has the whole message you sent, you can do whatever you want with it.
// [...]
//After you handle the message, wait for another message from the client
result = await webSocket.ReceiveAsync(buffer, CancellationToken.None);
}
await webSocket.CloseAsync(result.CloseStatus.Value, result.CloseStatusDescription, CancellationToken.None);
}

Sending and receiving images via a socket

I have a C# desktop app. It connects to another PC on my network which is a UWP C# app.
I am trying to send an image or 2 to my listening socket and to test this I get the listening socket to send me the image back.
The trouble is that even though my server recieves all the bytes that were orginally sent the recieved image back to the client is not of the same size.
To make this even more weird is sometimes the returned bytes are correct and I get the whole image and when I attempt to send 2 images the 1st one is OK and the 2nd one is not.
Then it will/can revert back to no images being sent back correctly.
I think is maybe to do with the async/await parts bit I am not sure how.
This is my server code:
using (IInputStream input = args.Socket.InputStream)
{
byte[] data = new byte[BufferSize];
IBuffer buffer = data.AsBuffer();
uint dataRead = BufferSize;
while (dataRead == BufferSize)
{
await input.ReadAsync(buffer, BufferSize, InputStreamOptions.Partial);
requestInBytes.AddRange(data.Take((int) buffer.Length));
dataRead = buffer.Length;
}
}
var ct = requestInBytes.Count;
I then trip out the header info:
int counter = 0;
counter = requestCommand[0].Length;
counter = counter + requestCommand[1].Length;
counter = counter + requestCommand[2].Length;
counter = counter + requestCommand[3].Length;
counter = counter + requestCommand[4].Length;
counter = counter + requestCommand[5].Length;
counter = counter + 6;
Now I extract the image:
var imgBody = new byte[totalBytes.Length- counter];
System.Buffer.BlockCopy(totalBytes, counter, imgBody, 0, imgBody.Length);
byteArray = imgBody;
And send just the image back:
using (IOutputStream output = args.Socket.OutputStream)
{
using (Stream response = output.AsStreamForWrite())
{
MemoryStream stream = new MemoryStream(byteArray);
await response.WriteAsync(byteArray, 0, byteArray.Length);
await response.FlushAsync();
}
}
This is my client code:
StringBuilder sb = new StringBuilder();
foreach (var gallery in Shared.CurrentJobGallery)
{
try
{
sb.Clear();
sb.Append(GeneralTags.ACTION_ADD);
sb.Append(Shared.DLE);
sb.Append("GALLERY");
sb.Append(Shared.DLE);
sb.Append(Shared.CurrentClientId);
sb.Append(Shared.DLE);
sb.Append(gallery.Title);
sb.Append(Shared.DLE);
sb.Append(gallery.Description);
sb.Append(Shared.DLE);
sb.Append(jobRef);
sb.Append(Shared.DLE);
byte[] galleryHdr = Encoding.UTF8.GetBytes(sb.ToString());
byte[] byteArray = new byte[galleryHdr.Length + gallery.ImageData.Length];
Buffer.BlockCopy(galleryHdr, 0, byteArray, 0, galleryHdr.Length);
Buffer.BlockCopy(gallery.ImageData, 0, byteArray, galleryHdr.Length, gallery.ImageData.Length);
List<byte> requestInBytes2 = new List<byte>();
System.Diagnostics.Debug.WriteLine("SENT: " + gallery.ImageData.Length.ToString());
using (TcpClient clientSocket = new TcpClient())
{
await clientSocket.ConnectAsync(GeneralTags.RASPBERRY_PI_IP_ADDRESS, GeneralTags.RASPBERRY_PI_PORT);
using (NetworkStream serverStream = clientSocket.GetStream())
{
List<byte> requestInBytes = new List<byte>();
serverStream.Write(byteArray, 0, byteArray.Length);
serverStream.Flush();
int i;
Byte[] bytes = new Byte[1024];
do
{
i = serverStream.Read(bytes, 0, bytes.Length);
byte[] receivedBuffer = new byte[i];
Array.Copy(bytes, receivedBuffer, i);
requestInBytes2.AddRange(receivedBuffer);
} while (serverStream.DataAvailable);
}
}
using (MemoryStream ms = new MemoryStream())
{
System.Diagnostics.Debug.WriteLine("BACK: " + requestInBytes2.Count.ToString());
ms.Write(requestInBytes2.ToArray(), 0, requestInBytes2.ToArray().Length);
Shared.ViewImage(Image.FromStream(ms, true));
}
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.ToString());
}
}

Your problem is that TCP sockets are based around streams, not packets. It's true that "on the wire" everything is a packet, but when you're using TCP, you have no control over how the data is split up into packets or is reassembled into a stream.
In particular, this line of code is incorrect:
await input.ReadAsync(buffer, BufferSize, InputStreamOptions.Partial);
According to the docs, you must use the buffer returned from ReadAsync. Also note that this buffer may be a partial image, and it's up to your code to detect that situation, read more if necessary, and append those blocks together. Also, the buffer may contain part of one image and part of the next image; it's also up to your code to detect that and handle it correctly.
For this reason, most TCP applications use some form of message framing (described in more detail on my blog). Note that getting this right is surprisingly hard.
I strongly recommend that you use SignalR instead of raw TCP sockets. SignalR handles message framing for you, and it is capable of self-hosting (i.e., it does not require ASP.NET).

Can I read the stream from TcpClient in a better way?

I have a system where in one end I have a device communicating with a module via RS-232.
The module is connected to a PC via TCP, and translates TCP messages to RS-232 and vice versa.
This is how I do it:
I read out every byte on the stream
build a string
compare the latter part of the string to a delimiter
then I stop reading the stream (and fire an event, though not shown here).
My current code for handling this is
string delimiter = "\r\n";
byte[] reply = new byte[1];
string replyString = string.Empty;
bool readOk = true;
int dl = delimiter.Length;
bool delimiterReached = false;
do
{
try
{
stream.Read(reply, 0, 1);
}
catch (Exception e)
{
readOk = false;
break;
}
replyString += Encoding.ASCII.GetString(reply, 0, reply.Length);
int rl = replyString.Length;
if (rl > dl)
{
string endString = replyString.Substring(rl-dl, dl);
if (endString.Equals(delimiter))
delimiterReached = true;
}
} while (!delimiterReached);
where stream is the TcpClient.GetStream()
I don't much care for the constant string building, so I was wondering if there is a better way of doing this?

Once the TCP connection is established, wrap the TcpClient stream in a StreamReader:
var reader = new StreamReader( stream, Encoding.ASCII );
Since your delimiter is \r\n, StreamReader.ReadLine will read a complete message:
var reply = reader.ReadLine();
Be careful to create the StreamReader once per TCP connection; it may read and buffer additional data from the underlying TCP stream for efficiency.