Develop Reference

Retrieving entire line from a socket in C#?

I have a simple client-server system sending plain text - though only commands that have been approved. The server is a Python system - and I've confirmed proper connections.
However, the client is C# - in Unity. Searching for examples, I stumbled across this bit of code. It does seem to do what I want, however, only partially:
public String readSocket()
{
if (!socketReady)
return "";
if (theStream.DataAvailable)
return theReader.ReadLine();
return "";
}
The strings I am sending end with \n, but I'm only getting half the message like this:
Message A:
claim_2
Message B:
_20_case
claim_1
I know this probably has to do with how I'm directly reading the line but I cannot find any better examples - strangely enough, everyone seems to point back at this snippet even when multiple people point out the problems.
Can anything be done to fix this bit of code properly?
In case it helps, I'm sending the information (from my Python server) out like this:
action = str(command) + "_" + str(x) + "_" + str(userid) + "_" + str(user)
cfg.GameSendConnection.sendall((action + "\n").encode("utf-8"))

When you do sockets programming, it is important to note that data might not be
available in one piece. In fact, this is exactly what you are seeing. Your
messages are being broken up.
So why does ReadLine not wait until there's a line to read?.
Here's some simple sample code:
var stream = new MemoryStream();
var reader = new StreamReader(stream);
var writer = new StreamWriter(stream) { AutoFlush = true };
writer.Write("foo");
stream.Seek(0, SeekOrigin.Begin);
Console.WriteLine(reader.ReadLine());
Note that there is no newline at the end. Still, the output of this little
snippet is foo.
ReadLine returns the string up to the first line break or until there is no
more data to read. The exception being reading from a stream that has no more
data to read, then it returns null.
When a NetworkStream has its DataAvailable property return true, it has
data. But as mentioned before, there is no guarantee whatsoever about what that
data is. It might be a single byte. Or a part of a message. Or a full message
plus part of the next message. Note that depending on the encoding, it could
even be possible to receive only part of a character. Not all character
encodings have all characters be at most a single byte. This includes UTF-8, which cfg.GameSendConnection.sendall((action + "\n").encode("utf-8")) sends.
How to solve this? Read bytes, not lines. Put them in some buffer. After every
read, check if the buffer contains a newline. If it does, you now have a full
message to handle. Remove the message up to and including the newline from the
buffer and keep appending new data to it until the next newline is received. And
so on.

This is how I process the entire line in my similar application, which is a very simple code, and your code may be different, but you can get the idea.
private string incompleteRecord = "";
public void ReadSocket()
{
if (_networkStream.DataAvailable)
{
var buffer = new byte[8192];
var receivedString = new StringBuilder();
do
{
int numberOfBytesRead = _networkStream.Read(buffer, 0, buffer.Length);
receivedString.AppendFormat("{0}", Encoding.UTF8.GetString(buffer, 0, numberOfBytesRead));
} while (_networkStream.DataAvailable);
var bulkMsg = receivedString.ToString();
// When you receive data from the socket, you can receive any number of messages at a time
// with no guarantee that the last message you receive will be complete.
// You can receive only part of a complete message, with next part coming
// with the next call. So, we need to save any partial messages and add
// them to the beginning of the data next time.
bulkMsg = incompleteRecord + bulkMsg;
// clear incomplete record so it doesn't get processed next time too.
incompleteRecord = "";
// loop though the data breaking it apart into lines by delimiter ("\n")
while (bulkMsg.Length > 0)
{
var newLinePos = bulkMsg.IndexOf("\n");
if (newLinePos > 0)
{
var line = bulkMsg.Substring(0, newLinePos);
// Do whatever you want with your line here ...
// ProcessYourLine(line)
// Move to the next message.
bulkMsg = bulkMsg.Substring(line.Length + 1);
}
else
{
// there are no more newline delimiters
// so we save the rest of the message (if any) for processing with the next batch of received data.
incompleteRecord = bulkMsg;
bulkMsg = "";
}
}
}
}

Stream.Read is combining two different reads

i have a simplistic file server\client application ive written in c#. but i commonly run into the problem that my stream writes two different reads into a single buffer. i have a synchronized stream, still isnt helping. any suggestions? thanks!
System.Threading.Thread.Sleep(25);
receive_fspos = new byte[30];
int bytesread = stream_1.Read(receive_fspos, 0, receive_fspos.Length);//this is where it gets combined
if (bytesread == 0)
{
finished = true;
System.Threading.Thread.Sleep(25);
}
string string_1 = utf.GetString(receive_fspos).TrimEnd(new char[] { (char)0 });
int fsposition = (int)Convert.ToInt64(string_1);
bytestosend = fsposition;
filestream.Position = fsposition;
byte[] buffer_1 = new byte[bufsize];
int bytesreadfromfs = filestream.Read(buffer_1, 0, buffer_1.Length);
stream_1.Write(buffer_1, 0, buffer_1.Length);
Console.Write("\rSent " + fsposition + " / " + length + " bytes");
finished = true;

I would not recommend writing your own stream method if you do not fully understand it.
The problem that you are having is because the incoming data is a stream of bytes that does not give you a way of knowing how many bytes in length that the message is.
In the code below you are stating that you would like to read "receive_fspos.Length" bytes of the stream. Since "receive_fspos.Length" is 30, the amount of bytes that will be read will be anywhere from 0 to 30.
If there is only 15 bytes that have been received by the connection. It will give you 15 bytes. If the message was 20 bytes long. Then the message is now split up into different segments.
If the first message was 4 bytes and the second message is 12 bytes. Now you have 2 messages and a set of 16 blank bytes at the end. Even worse those 16 "blank" bytes could be the beginning of a third message coming in to the stream.
If the message is 50 bytes long. Then you will only receive half of the message. Now you would need to add the bytes that were read to a seperate buffer. Read from the stream again. Then repeat this until you have determined that you have read the exact amount of bytes that are needed to complete the entire message. Then concat all of the read bytes back to a single byte[].
receive_fspos = new byte[30];
int bytesread = stream_1.Read(receive_fspos, 0, receive_fspos.Length);//this is where it gets combined
Instead of rolling your own loop please use the BCL methods. It sounded like you are using strings so this would be the preferred method.. I would suggest the following.
using(NetworkStream networkStream = tcpClient.GetStream())
using(StreamReader streamReader = new StreamReader(networkStream))
using(StreamWriter streamWriter = new StreamWriter(networkStream))
{
networkStream.ReadTimeout = timeout; //Set a timeout to stop the stream from reading indefinately
//To receive a string
string incomingString = stream.ReadLine();
//To send a string
stream.WriteLine(messageToSend);
stream.Flush();
}
Your answer clarified that you are trying to send a file. For this I would recommend sending an array of bytes[]. Using this method you can send anything that can be serialized. This includes a file. Please note that the size of the file is limited since it must be kept in memory. To write a larger file you would want to save the data in chunks as it is being streamed in.
//Please note that if the file size is large enough. It may be preferred to use a stream instead of holding the entire file in memory.
byte[] fileAsBytes = File.ReadAllBytes(fileName);
using(NetworkStream networkStream = tcpClient.GetStream())
using(BinaryReader binaryReader = new BinaryReader(networkStream))
using(BinaryWriter binaryWriter = new BinaryWriter(networkStream))
{
networkStream.ReadTimeout = timeout; //Set a timeout to stop the stream from reading indefinately
//To receive a byte array
int incomingBytesLength = BinaryReader.ReadInt32(); //The header is 4 bytes that lets us know how large the incoming byte[] is.
byte[] incomingBytes = BinaryReader.ReadBytes(incomingBytesLength);
//To send a byte array
BinaryWriter.Write(fileAsBytes.Length); //Send a header of 4 bytes that lets the listener know how large the incoming byte[] is.
BinaryWriter.Write(fileAsBytes);
}

got it working, code > 30000 chars :\
it is a little messy but hey, it's functional.
server : https://www.dropbox.com/s/2wyccxpjbja10z3/Program.cs?m
client : https://www.dropbox.com/s/yp78nx4ubacsz6f/Program.cs?m

Invalid paramether in Image.FromStream(MemoryStream)

im trying to send an image via network stream, i have a sendData and Getdata functions
and i always get an invalid parameter when using the Image.FromStream function
this is my code :
I am Getting the pic from the screen, then converting it to a byte[]
Inserting it to a Memory stream that i send via a networkStream.
private void SendData()
{
StreamWriter swWriter = new StreamWriter(this._nsClient);
// BinaryFormatter bfFormater = new BinaryFormatter();
// this method
lock (this._secLocker)
{
while (this._bShareScreen)
{
// Check if you need to send the screen
if (this._bShareScreen)
{
MemoryStream msStream = new MemoryStream();
this._imgScreenSend = new Bitmap(this._imgScreenSend.Width, this._imgScreenSend.Height);
// Send an image code
swWriter.WriteLine(General.IMAGE);
swWriter.Flush();
// Copy image from screen
this._grGraphics.CopyFromScreen(0, 0, 0, 0, this._sizScreenSize);
this._imgScreenSend.Save(msStream, System.Drawing.Imaging.ImageFormat.Jpeg);
msStream.Seek(0, SeekOrigin.Begin);
// Create the pakage
byte[] btPackage = msStream.ToArray();
// Send its langth
swWriter.WriteLine(btPackage.Length.ToString());
swWriter.Flush();
// Send the package
_nsClient.Write(btPackage, 0, btPackage.Length);
_nsClient.Flush();
}
}
}
}
private void ReciveData()
{
StreamReader srReader = new StreamReader(this._nsClient);
string strMsgCode = String.Empty;
bool bContinue = true;
//BinaryFormatter bfFormater = new BinaryFormatter();
DataContractSerializer x = new DataContractSerializer(typeof(Image));
// Lock this method
lock (this._objLocker)
{
while (bContinue)
{
// Get the next msg
strMsgCode = srReader.ReadLine();
// Check code
switch (strMsgCode)
{
case (General.IMAGE):
{
// Read bytearray
int nSize = int.Parse(srReader.ReadLine().ToString());
byte[] btImageStream = new byte[nSize];
this._nsClient.Read(btImageStream, 0, nSize);
// Get the Stream
MemoryStream msImageStream = new MemoryStream(btImageStream, 0, btImageStream.Length);
// Set seek, so we read the image from the begining of the stream
msImageStream.Position = 0;
// Build the image from the stream
this._imgScreenImg = Image.FromStream(msImageStream); // Error Here

Part of the problem is that you're using WriteLine() which adds Environment.NewLine at the end of the write. When you just call Read() on the other end, you're not dealing with that newline properly.
What you want to do is just Write() to the stream and then read it back on the other end.
The conversion to a string is strange.
What you're doing, when transferring an image, is sending an array of bytes. All you need to do is send the length of the expected stream and then the image itself, and then read the length and the byte array on the other side.
The most basic and naive way of transferring a byte array over the wire is to first send an integer that represents the length of the array, and read that length on the receiving end.
Once you now know how much data to send/receive, you then send the array as a raw array of bytes on the wire and read the length that you previously determined on the other side.
Now that you have the raw bytes and a size, you can reconstruct the array from your buffer into a valid image object (or whatever other binary format you've just sent).
Also, I'm not sure why that DataContractSerializer is there. It's raw binary data, and you're already manually serializing it to bytes anyway, so that thing isn't useful.
One of the fundamental problems of network programming using sockets and streams is defining your protocol, because the receiving end can't otherwise know what to expect or when the stream will end. That's why every common protocol out there either has a very strictly defined packet size and layout or else does something like sending length/data pairs, so that the receiving end knows what to do.
If you implement a very simple protocol such as sending an integer which represents array length and reading an integer on the receiving end, you've accomplished half the goal. Then, both sender and receiver are in agreement as to what happens next. Then, the sender sends exactly that number of bytes on the wire and the receiver reads exactly that number of bytes on the wire and considers the read to be finished. What you now have is an exact copy of the original byte array on the receiving side and you can then do with it as you please, since you know what that data was in the first place.
If you need a code example, I can provide a simple one or else there are numerous examples available on the net.

Trying to keep it short:
the Stream.Read function (which you use) returns an int that states how many bytes were read, this is return to you so you could verify that all the bytes you need are received.
something like:
int byteCount=0;
while(byteCount < nSize)
{
int read = this._nsClient.Read(btImageStream, byteCount, nSize-byteCount);
byteCount += read;
}
this is not the best code for the job

C# How to send more than one byte streams using TCPClient?

I'm trying to make Login feature by using TCP Client. I have two forms: Client-side and Server-side.
The Client-side handles user input while Server-side connect to database.
The problem is the reader result, which always combine both inputs into one long string like this:
myusernamemypassword
Here's part of the sender of client-side:
byte[] byteUsername = Encoding.Unicode.GetBytes(username);
byte[] bytePassword = Encoding.Unicode.GetBytes(password);
NetworkStream stream = client.GetStream();
stream.Write(username, 0, byteUsername.Length);
stream.Write(password, 0, bytePassword.Length);
//if offset != 0, the code always return ArgumentOutOfRangeException
And the reader in server-side:
return Encoding.Unicode.GetString(buffer, 0, buffer.Length)
After long-search I found the solution, but it can only handle two strings; the third+ string will be combined together with the second string. I need to send at least 4 strings for other feature.
Here's the updated reader code:
List<string> list = new List<string>();
int totalRead = 0;
do
{
int read = client.GetStream().Read(buffer, totalRead, buffer.Length - totalRead);
totalRead += read;
list.Add(Encoding.Unicode.GetString(buffer, 0, totalRead));
} while (client.GetStream().DataAvailable);
I don't quite understand this code. How can it knows which bytes are part of the first string? The size of Read() parameter is length-totalRead which is length - 0, it should return the whole buffer right?
Any solution guys?
Thanks before

You should prefix each string with its length (in bytes, not characters) as a 4-byte integer.
This way, the server will know how many bytes to read into each string.

How to Encode an Image for .NET Sockets

I'm creating a simple proxy server. The problem I'm facing is that when I request an image, instead of getting (first 4 bytes):
ÿØÿà
I get:
����
I'm using char array to store the data. Do I need to use a byte instead of char?
I'm using async sockets, this is what I have in onRecieve callback:
System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder();
int charLen = d.GetChars(socketData.dataBuffer, 0, iRx, chars, 0);
Can anyone please point me at the right direction?
UPDATE:
I did use the socketData.buffer suggestion, but now all I get is numbers, no characters. This is what I'm doing:
Collecting all the data from socketData.buffer and place it into an List
Loop through the list and write out all the data to browser, like this:
Stream clientStream = client.GetStream();
Stream outStream = clientStream;
StreamWriter myResponseWriter = new StreamWriter(outStream);
for(int i=0; i<myList.Count; i++){
myResponseWriter.Write(myList[i]);
}
myResponseWrite.Flush();
Output I'm getting is 728484804749464932504848327975131067108105101110116583278101..
EDIT2:
BinaryWriter solved my second problem.

Convert.ToBase64String Method and its inverse FromBase64String can help you go between byte arrays and strings, if that's how you want to transfer the data, say if you were planning to attach it to a plain text email for instance.