Develop Reference

clear buffer contents in C#

I am working on a program where I continously send control commands to control the AR.Drone. I am writing the contents to a buffer and sending to a socket.
When I execute the program the drone is not behaving as required. I think this may be due to contents queued in the buffer that were not sent.
So I want to clear the contents of buffer before sending any new command. How can I do this?
In C we can use flush(). What is it in C#?
My code is:
Socket sending_socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
IPAddress send_to = IPAddress.Parse("192.168.1.1");
IPEndPoint sending_end_point = new IPEndPoint(send_to, 5556);
string buff1;
byte[] buff2;
I am writing a string to a text file upon each button press. I then read the text file to retrieve the content and based upon the content send the corresponding command.And in text file each time I am overwriting the previous string so there will be only one string in text file.
switch (choice1)
{
case "takeoff":
{
System.Console.WriteLine("send take-off command");
buff1 = null;
buff2 = null;
buff1 = String.Format("AT*REF={0},290718208\r", seq);
buff2 = Encoding.ASCII.GetBytes(buff1);
sending_socket.SendTo(buff2, sending_end_point);
seq = seq + 1;
break;

So I want to clear the contents of buffer before sending any new command.
In the code shown, you are creating a new buffer each time. There is no need to clear / reset / flush this.
You mention a Socket; in that case, there's a good chance this is Nagle. Try setting:
socket.NoDelay = true;
Note that if you do this, it is your job to perform any necessary buffering prior to sending. Avoid things like:
netStream.Write(oneByte);
netStream.Write(threeBytes);
netStream.Write(twoBytes);
as this will lead to packet fragmentation. If you have existing code like that, a pragmatic fix is to wrap a NetworkStream in a BufferedStream, and Flush() that at the end of logical messages.

Trying to receive a file using a Socket

I'm trying to stream a file over a socket, and write it to a file as I receive it (so that it can be opened in other applications). It mostly works except for near the end where (I'm guessing) it blocks because it's trying to read an entire 8 KB chunk when there isn't that much left to receive. I'm at a loss trying to figure out how to fix this. I know the total amount of bytes I'm supposed to receive, if that helps. Thanks!
using (var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)) {
socket.Connect(e.IPAddress, e.Port);
using (var fs = new FileStream(e.Filename, FileMode.OpenOrCreate, FileAccess.Write, FileShare.Read)) {
byte[] buffer = new byte[8192];
int read;
while ((read = socket.Receive(buffer)) > 0) {
fs.Write(buffer, 0, read);
}
}
}

From the MSDN...
If no data is available for reading, the Receive method will block until data is available, unless a time-out value was set by using Socket.ReceiveTimeout. If the time-out value was exceeded, the Receive call will throw a SocketException. If you are in non-blocking mode, and there is no data available in the in the protocol stack buffer, the Receive method will complete immediately and throw a SocketException. You can use the Available property to determine if data is available for reading. When Available is non-zero, retry the receive operation.
The Socket.Receive() method blocks because the connection is still open on the other end, even if there isn't any more data available. There are a couple ways to get around this, one which requres you to know the size of the file beforehand and stopping once you've read that many bytes Another would be using a non-blocking socket (by setting the Socket.Blocking property to false) and getting the data available from the Socket.Available property. The latter requires a little more finesse in writing your code since read operations won't halt your code anymore.
If you know the size of the data beforehand, you can compare that to how much you've read so far and stop once you've reached that size:
//If you know the size beforehand, set it as an int and compare
// it to what you have so far
int totalBytes = 9001;
using (var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
{
socket.Connect(e.IPAddress, e.Port);
using (var fs = new FileStream(e.Filename, FileMode.OpenOrCreate, FileAccess.Write, FileShare.Read))
{
byte[] buffer = new byte[8192];
int read;
int bytesSoFar = 0; //Use this to keep track of how many bytes have been read
do
{
read = socket.Receive(buffer);
fs.Write(buffer, 0, read);
bytesSoFar += read;
} while (bytesSoFar < totalBytes);
}
}
Keep in mind that this second example isn't very robust and will not work when trying to send multiple files using the same socket. But for just a simple 1 file transfer, this should work fine.

TCP transfer not finishing: "An existing connection was forcibly closed by the remote host"?

I have a client application that uses a TcpClient to send a file to a server application that uses a TcpListener to receive it. Sometimes, the file transfers fine. But at other times, the transfer starts but does not finish. After I have read in a varying number of bytes on the server, I get an IOException with the message: "An existing connection was forcibly closed by the remote host."
On the client side, I create a header byte array containing data about my file, including the total size in bytes and a few other bits of data. I combine this byte array with that of the file, and send it to the server.
TcpClient fileClient = new TcpClient();
fileClient.Connect("mydomain.com", 7728);
NetworkStream clientStream = fileClient.GetStream();
ASCIIEncoding encoder = new ASCIIEncoding();
FileStream fs = File.Open(filePath, FileMode.Open);
string header = filePath + "|" + fs.Length.ToString() + "|" + this.ID); //ID = short string of mine
header = header.PadRight(512, '*');
byte[] sizeArray = encoder.GetBytes(header);
byte[] fileBuffer = new byte[fs.Length];
fs.Read(fileBuffer, 0, Convert.ToInt32(fs.Length));
fs.Close();
byte[] buffer = Combine(sizeArray, fileBuffer); //Combine = method I use to combine byte arrays
clientStream.Write(buffer, 0, buffer.Length);
clientStream.Flush();
On the server side, I create a TcpListener, and start listening. When I get a connection, I handle it, read my header array, the first 512 bytes, determine the size of the new file, and then use this code to read the file / rest of the bytes:
int newSize = ... //The size that I sent in my header array
byte[] fileArray = new byte[newSize];
int off = 0;
while (true)
{
try
{
off += clientStream.Read(fileArray, off, fileArray.Length - off);
}
catch (Exception ex)
{
off = newSize; // Enables partial receive on error, not total loss
}
if (off >= newSize)
{
break;
}
}
This is where it sometimes - about 25% of the time - goes wrong. I will get the exception at the off += clientStream.Read(fileArray, off, fileArray.Length - off) line. I surrounded it with a try catch, which enables the application to still get part of the file even when this error occurs. However, I need to be able to get the full file all the time. What am I doing wrong with this code?
The files I am transferring are JPEG images around 100KB in size, and generally take no more than two seconds to transfer on even the slowest of connections. Changing the timeout values for the clients and the listener does nothing to help.

using python to send an image over TCP to c# server

I am working on a project that uses a python script to send an image taken with a webcam to a c# webserver using sockets. I am currently sending strings to the server from the python client using code like:
info = bytearray("Text to send", "UTF-8")
socket.send(info)
to send data to the c# server and it functions perfectly for text and numbers. I run into an issue when trying to encode the data read from the .bmp into "UTF-8", as trying this returns an error of not being able to encode certain characters into UTF-8.
I was wondering if anyone had any idea of a way to encode this that c# will be able to recognize, or, if there is a better way of trying to implement this process, I am all ears.
A couple of options I have come up with would be to 1 - use something like google drive to save the image to, or an FTP server and then have the c# server retrieve it from there or 2 - create a packet system containing the RGB values and recreating the image from those pixel values on the server.
Thanks for your help.
EDIT: I have tried sending the file this way
data = bytearray("123456789","UTF-8")
file = open("image.bmp", "rb")
data += file.read()
socket.send(data)
and was able to successfully retreive the string "123456789", but the data after this is garbage. I have also implemented sending the size of the file before sending the data and that size number is retrieved fine, but the img data saves as a black bmp.
Edit 2 :
Here is the server and client code I am using to try and recreate the image using a memory stream. I the client code is using the process mentioned by hcalves.
Client
if __name__ == "__main__":
sock = socket.socket()
sock.connect(("localhost", 50839))
with open("image.bmp", "rb") as fd:
buf = fd.read(1024)
while (buf):
sock.send(buf)
buf = fd.read(1024)
sock.close()
Server
Socket client = server.AcceptSocket();
NetworkStream stream = new NetworkStream(client);
byte[] imgData = new byte[1024];
MemoryStream memstream = new MemoryStream();
stream.Read(imgData,0, 1024);
int counter = 0;
while (stream.DataAvailable)
{
memstream.Write(imgData, 0, 1024);
stream.Read(imgData, 0, 1024);
counter = counter + 1024;
}
memstream.Seek(0, SeekOrigin.Begin);
using (Stream file = File.OpenWrite("img.bmp"))
{
byte[] buffer = new byte[8*1024];
int len;
while ((len = memstream.Read(buffer, 0, buffer.Length)) > 0)
{
file.Write(buffer,0,len);
}
}

You shouldn't need more than this recipe:
import socket
if __name__ == "__main__":
sock = socket.socket()
sock.connect(("localhost", 50839))
with open("data.bin", "rb") as fd:
buf = fd.read(1024)
while (buf):
sock.send(buf)
buf = fd.read(1024)
sock.close()
For practical reasons, you can treat str objects (the result of fd.read) as raw data, you don't need any further crazy encoding. Just iterate the file and send over the socket. Test by running this server which just echoes to stdout with python server.py > data2.bin:
import socket
import sys
if __name__ == "__main__":
sock = socket.socket()
sock.bind(("localhost", 50839))
sock.listen(1)
client, address = sock.accept()
buf = client.recv(1024)
while (buf):
sys.stdout.write(buf)
buf = client.recv(1024)
client.close()
sock.close()
A checksum shows the binary data is sent correctly:
% md5 data.bin data2.bin
MD5 (data.bin) = 8b3280072275badf3e53a6f7aae0b8be
MD5 (data2.bin) = 8b3280072275badf3e53a6f7aae0b8be
Your C# server should be able to accept this data as is. If it doesn't work, it's because your server is expecting something in particular, not just raw data.

C# TCP file transfer - Images semi-transferred

I am developing a TCP file transfer client-server program. At the moment I am able to send text files and other file formats perfectly fine, such as .zip with all contents intact on the server end. However, when I transfer a .gif the end result is a gif with same size as the original but with only part of the image showing as if most of the bytes were lost or not written correctly on the server end.
The client sends a 1KB header packet with the name and size of the file to the server. The server then responds with OK if ready and then creates a fileBuffer as large as the file to be sent is.
Here is some code to demonstrate my problem:
// Serverside method snippet dealing with data being sent
while (true)
{
// Spin the data in
if (streams[0].DataAvailable)
{
streams[0].Read(fileBuffer, 0, fileBuffer.Length);
break;
}
}
// Finished receiving file, write from buffer to created file
FileStream fs = File.Open(LOCAL_FOLDER + fileName, FileMode.CreateNew, FileAccess.Write);
fs.Write(fileBuffer, 0, fileBuffer.Length);
fs.Close();
Print("File successfully received.");
// Clientside method snippet dealing with a file send
while(true)
{
con.Read(ackBuffer, 0, ackBuffer.Length);
// Wait for OK response to start sending
if (Encoding.ASCII.GetString(ackBuffer) == "OK")
{
// Convert file to bytes
FileStream fs = new FileStream(inPath, FileMode.Open, FileAccess.Read);
fileBuffer = new byte[fs.Length];
fs.Read(fileBuffer, 0, (int)fs.Length);
fs.Close();
con.Write(fileBuffer, 0, fileBuffer.Length);
con.Flush();
break;
}
}
I've tried a binary writer instead of just using the filestream with the same result.
Am I incorrect in believing successful file transfer to be as simple as conversion to bytes, transportation and then conversion back to filename/type?
All help/advice much appreciated.

Its not about your image .. It's about your code.
if your image bytes were lost or not written correctly that's mean your file transfer code is wrong and even the .zip file or any other file would be received .. It's gonna be correpted.
It's a huge mistake to set the byte buffer length to the file size. imagine that you're going to send a large a file about 1GB .. then it's gonna take 1GB of RAM .. for an Idle transfering you should loop over the file to send.
This's a way to send/receive files nicely with no size limitation.
Send File
using (FileStream fs = new FileStream(srcPath, FileMode.Open, FileAccess.Read))
{
long fileSize = fs.Length;
long sum = 0; //sum here is the total of sent bytes.
int count = 0;
data = new byte[1024]; //8Kb buffer .. you might use a smaller size also.
while (sum < fileSize)
{
count = fs.Read(data, 0, data.Length);
network.Write(data, 0, count);
sum += count;
}
network.Flush();
}
Receive File
long fileSize = // your file size that you are going to receive it.
using (FileStream fs = new FileStream(destPath, FileMode.Create, FileAccess.Write))
{
int count = 0;
long sum = 0; //sum here is the total of received bytes.
data = new byte[1024 * 8]; //8Kb buffer .. you might use a smaller size also.
while (sum < fileSize)
{
if (network.DataAvailable)
{
{
count = network.Read(data, 0, data.Length);
fs.Write(data, 0, count);
sum += count;
}
}
}
}
happy coding :)

When you write over TCP, the data can arrive in a number of packets. I think your early tests happened to fit into one packet, but this gif file is arriving in 2 or more. So when you call Read, you'll only get what's arrived so far - you'll need to check repeatedly until you've got as many bytes as the header told you to expect.
I found Beej's guide to network programming a big help when doing some work with TCP.

As others have pointed out, the data doesn't necessarily all arrive at once, and your code is overwriting the beginning of the buffer each time through the loop. The more robust way to write your reading loop is to read as many bytes as are available and increment a counter to keep track of how many bytes have been read so far so that you know where to put them in the buffer. Something like this works well:
int totalBytesRead = 0;
int bytesRead;
do
{
bytesRead = streams[0].Read(fileBuffer, totalBytesRead, fileBuffer.Length - totalBytesRead);
totalBytesRead += bytesRead;
} while (bytesRead != 0);
Stream.Read will return 0 when there's no data left to read.
Doing things this way will perform better than reading a byte at a time. It also gives you a way to ensure that you read the proper number of bytes. If totalBytesRead is not equal to the number of bytes you expected when the loop is finished, then something bad happened.

Thanks for your input Tvanfosson. I tinkered around with my code and managed to get it working. The synchronicity between my client and server was off. I took your advice though and replaced read with reading a byte one at a time.