I've been trying to send a file from a client to a server application using the TCPClient class in C#. Before I send the actual data, I send some additional information like the exact file size and the file name, so the server application knows how much there is to read. The funny thing is everything was fine when I tested it on 127.0.0.1 - as soon as I replaced the IP address with the actual one, the server could only read about 1,5 KByte of the data that was sent. It still gets the filename and the file size, but theres no way it's retrieving the actual data.
For testing purposes, I replaced the image I was going to send with a simple string and the transmission went alright, so I suppose there is a problem with sending and receiving the data chunks, but I'm not getting any exceptions on the client side either.
Anyone got an idea? Cheers!
Edit:
Thanks so far, this is what I have got codewise. For the client:
IPAddress ipAddress = IPAddress.Parse("xx.xx.xx.xx");
int port = 3003;
int bufferSize = 1024;
TcpClient client = new TcpClient();
NetworkStream netStream;
// Connect to server
try
{
client.Connect(new IPEndPoint(ipAddress, port));
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
netStream = client.GetStream();
// Read bytes from image
byte[] data = File.ReadAllBytes("C:\\Users\\Dan\\Desktop\\asdf.jpg");
// Build the package
byte[] dataLength = BitConverter.GetBytes(data.Length);
byte[] package = new byte[4 + data.Length];
dataLength.CopyTo(package, 0);
data.CopyTo(package, 4);
// Send to server
int bytesSent = 0;
int bytesLeft = package.Length;
while (bytesLeft > 0)
{
int nextPacketSize = (bytesLeft > bufferSize) ? bufferSize : bytesLeft;
netStream.Write(package, bytesSent, nextPacketSize);
bytesSent += nextPacketSize;
bytesLeft -= nextPacketSize;
}
// Clean up
netStream.Close();
client.Close();
And the server:
TcpListener listen = new TcpListener(3003);
TcpClient client;
int bufferSize = 1024;
NetworkStream netStream;
int bytesRead = 0;
int allBytesRead = 0;
// Start listening
listen.Start();
// Accept client
client = listen.AcceptTcpClient();
netStream = client.GetStream();
// Read length of incoming data
byte[] length = new byte[4];
bytesRead = netStream.Read(length, 0, 4);
int dataLength = BitConverter.ToInt32(length,0);
// Read the data
int bytesLeft = dataLength;
byte[] data = new byte[dataLength];
while (bytesLeft > 0)
{
int nextPacketSize = (bytesLeft > bufferSize) ? bufferSize : bytesLeft;
bytesRead = netStream.Read(data, allBytesRead, nextPacketSize);
allBytesRead += bytesRead;
bytesLeft -= bytesRead;
}
// Save image to desktop
File.WriteAllBytes("C:\\Users\\Dan\\Desktop\\tcpimage.jpg", data);
// Clean up
netStream.Close();
client.Close();
About 1.5 KiB sounds like 1500 bytes, "the largest allowed by Ethernet at the network layer". This is the maximum transmission unit (mtu) forcing your network stack to split your file into several small packets.
You need to call the NetworkStream.Read in a loop to read every packet arrived. There's example code of this at MSDN.
Combine this with the default behavior of .NET; consolidating smaller packets to reduce the amount of packets sent, and you'll also see this behavior when sending smaller packets. This can be controlled with ServicePointManager.UseNagleAlgorithm or by using smaller scoped socket options.
Ok, don't know what I'm doing here, but in case of anyone uses this as referrence.
I got rid of unnecessary copying and done a very important improvement. Your way to calc nextPacketSize is not complete, since there could be less data, than 1024 avaliable, and you'll get extra nulls in between these chunks (I had a lot of headache with that, now so happy to figure out)
I made them as function, needed for several files, so here's code:
client
this one is almost the same
public void sendData(byte[] data, NetworkStream stream)
{
int bufferSize = 1024;
byte[] dataLength = BitConverter.GetBytes(data.Length);
stream.Write(dataLength, 0, 4);
int bytesSent = 0;
int bytesLeft = data.Length;
while (bytesLeft > 0)
{
int curDataSize = Math.Min(bufferSize, bytesLeft);
stream.Write(data, bytesSent, curDataSize);
bytesSent += curDataSize;
bytesLeft -= curDataSize;
}
}
server
public byte[] getData(TcpClient client)
{
NetworkStream stream = client.GetStream();
byte[] fileSizeBytes = new byte[4];
int bytes = stream.Read(fileSizeBytes, 0, 4);
int dataLength = BitConverter.ToInt32(fileSizeBytes, 0);
int bytesLeft = dataLength;
byte[] data = new byte[dataLength];
int bufferSize = 1024;
int bytesRead = 0;
while (bytesLeft > 0)
{
int curDataSize = Math.Min(bufferSize, bytesLeft);
if (client.Available < curDataSize)
curDataSize = client.Available; //This saved me
bytes = stream.Read(data, bytesRead, curDataSize);
bytesRead += curDataSize;
bytesLeft -= curDataSize;
}
return data;
}
I used some of your code for a test networking project I'm working on. I tweaked a couple things to fit the requirements of my project, but one alteration I needed to make to the original code is to add a "-4" to the line that was setting the bytesLeft variable. After that the code worked. My test file is 52KB & was transmitted successfully.
// Read the data
int bytesLeft = dataLength-4;
byte[] data = new byte[dataLength];
Related
private byte[] ResponseAsync(byte[] buffer, string ip, int port)
{
byte[] buffer_ = new byte[10000]; // receiving buffer 10KB
TcpClient client = new TcpClient(ip, port);
NetworkStream stream = client.GetStream();
stream.Write(buffer, 0, buffer.Length);
//await write;
int i = 0;
while (stream.DataAvailable)
{
MessageBox.Show(i.ToString());
i = stream.Read(buffer_, 0, 1024);
}
return buffer_.Take(i).ToArray();
}
the code was async but I thought I was doing somethign wrong so made it synchrone
You're overwriting the beginning of your buffer each time you do a read and will only return the data read in the last iteration of the while loop.
As such you'll need to increment i with the amount of data read and then use that as the offset when copying data in to your buffer.
private byte[] ResponseAsync(byte[] buffer, string ip, int port)
{
byte[] buffer_ = new byte[10000]; // receiving buffer 10KB
TcpClient client = new TcpClient(ip, port);
NetworkStream stream = client.GetStream();
stream.Write(buffer, 0, buffer.Length);
//await write;
int i = 0;
while (stream.DataAvailable)
{
MessageBox.Show(i.ToString());
// write data to the appropriate point in buffer_ and update i
i += stream.Read(buffer_, i, 1024);
}
return buffer_.Take(i).ToArray();
}
Be aware though that if you receive more than 10,000 bytes this will throw an exception.
The chances are that you'll not read all the data from the stream as stream.DataAvailable will only say whether there's data available to be read not that the stream has finished.
Using a blocking, streaming .NET socket I'm connecting to a server. Whenever I'm reading small bits of data, all goes well and the data is received into my buffer:
using (var socket = new Socket(SocketType.Stream, ProtocolType.IP))
{
socket.Connect(IPAddress.Parse("127.0.0.1"), 5000);
byte[] buffer = new byte[BufferSize];
socket.Receive(buffer);
// Here buffer doesn't always contain all data the server sent me?
Console.WriteLine(Encoding.Default.GetString(buffer));
}
In some cases though, I'm not receiving everything the server sends me. Data seems to be chopped off. What can be the cause of this?
This is documented in the Receive() method, emphasis mine:
The Receive method reads data into the buffer parameter and returns the number of bytes successfully read. You can call Receive from both connection-oriented and connectionless sockets.
When ignoring the return value, you will not know what portion of your buffer actually contains relevant data. Depending on the protocol being used, you may or may not know the content length in advance. Some protocols provide this length, others close the connection when done, yet another can use a message boundary.
You'll have to hold the received data in another buffer, and return or output the entire message buffer when no more data is available or expected. This can be done like this:
int BufferSize = 1024;
using (var socket = new Socket(SocketType.Stream, ProtocolType.IP))
{
socket.Connect(IPAddress.Parse("127.0.0.1"), 5000);
byte[] buffer = new byte[BufferSize];
string message = "";
int bytesReceived;
do
{
bytesReceived = socket.Receive(buffer);
message += Encoding.ASCII.GetString(buffer, 0, bytesReceived);
} while (bytesReceived > 0);
Console.WriteLine(message);
}
The received bytes are ASCII characters (as defined in the made-up protocol), so each byte received indicates one character (you can't convert partially received multibyte unicode characters). The bytes are converted to a string and appended to the message variable. The code loops until the server closes the connection.
When the message size is known on beforehand, again, depending on the protocol being used, you can create a message buffer and copy the data there on each Receive():
// Received using another Receive() call
int messageSize = 1234;
int totalBytesReceived = 0;
byte[] messageBuffer = new byte[messageSize];
byte[] buffer = new byte[BufferSize];
int bytesReceived;
do
{
bytesReceived = socket.Receive(buffer);
// Copy the receive buffer into the message buffer, appending after
// previously received data (totalBytesReceived).
Buffer.BlockCopy(buffer, 0, messageBuffer, totalBytesReceived, bytesReceived);
totalBytesReceived += bytesReceived;
} while (bytesReceived > 0);
// This assumes the connection wasn't closed prematurely.
Console.WriteLine(Encoding.ASCII.GetString(messageBuffer));
This can in turn be put in a resuable method:
public byte[] ReceiveMessage(Socket socket, int messageSize)
{
byte[] messageBuffer = new byte[messageSize];
int bytesReceived = 0;
int totalBytesReceived = 0;
do
{
byte[] buffer = new byte[BufferSize];
// Receive at most the requested number of bytes, or the amount the
// buffer can hold, whichever is smaller.
int toReceive = Math.Min(messageSize - totalBytesReceived, BufferSize);
bytesReceived = socket.Receive(buffer, toReceive, SocketFlags.None);
// Copy the receive buffer into the message buffer, appending after
// previously received data (totalBytesReceived).
Buffer.BlockCopy(buffer, 0, messageBuffer, totalBytesReceived, bytesReceived);
totalBytesReceived += bytesReceived;
} while (bytesReceived > 0);
if (totalBytesReceived < messageSize)
{
throw new Exception("Server closed connection prematurely");
}
return messageBuffer;
}
There's the NetworkStream which wraps a socket, but it has the same reading issues as the socket itself. You will have to monitor the return value and keep calling Read() until you've received all bytes. Same goes for the TcpClient that has a GetStream() method, on whose return value you'll also have to continue reading until you've read all data.
I am trying to receive data at my server of any length through tcp connection. First my client sends length of data to server through stream.write then it send the actual data.
At Client I receive the length and loop until whole the data is received successfully.
The problem is: "I receive 0 size on the server no matters what the length of data is". I tried to figure out the issue but could not get where the problem is. Any kind of help/hint would be appreciated.
Server Side Code:
byte[] lengthOfData = new byte[2048];
byte[] buffer;
try
{
stream = client.GetStream();
eventLog1.WriteEntry("Size of 1st = "+stream.Read(lengthOfData,0,lengthOfData.Length));
int numBytesToRead = ByteArraySerializer.BytesArrayToInt(lengthOfData);
eventLog1.WriteEntry("number of bytes to read= "+numBytesToRead);
buffer = new byte[numBytesToRead+10];
int numBytesRead = 0;
do
{
int n = stream.Read(buffer, numBytesRead, 10);
numBytesRead += n;
numBytesToRead -= n;
eventLog1.WriteEntry("number of bytes read= " + numBytesRead);
} while (numBytesToRead > 0);
}
catch (Exception e) // Called automatically when Client Diposes or disconnects unexpectedly
{
eventLog1.WriteEntry("Connection Closes: "+e.ToString());
lock (connectedClients)
{
connectedClients.Remove(client);
}
client.Close();
break;
}
Client Side Code
byte[] command = ByteArraySerializer.Serialize<Command>(cmd);
byte[] sizeOfData = ByteArraySerializer.IntToBytesArray(command.Length);
stream.Write(sizeOfData, 0, sizeOfData.Length);
Console.WriteLine("Size of Data = "+command.Length);
stream.Write(command, 0, command.Length);
Change the following line at server
byte[] lengthOfData = new byte[2048];
to
byte[] lengthOfData = new byte[sizeof(int)];
The issue was that the read at server was supposed to read only 4 bytes integer whereas it was reading other data as well which was getting written after writing the length of data. We are supposed to read only 4 bytes if we want to get the length of data(integer).
I have a Client and Server code for files sending. For some reason I need to receive at client and send from Server...
Everything work perfectly, in some cases all files sent and received perfectly. In another cases after sending a few files programm crashes. Don't understand where the problem...
Errors:
client colsole
server console
client
// client code
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
class Client003
{
const string destFilePath = #"..\..\..\";
const int BufferSize = 1024;
public static void StartReceiving()
{
// Data buffer for sending data.
byte[] buffer;
// FileStream to read data
FileStream fileStream;
int fileNameLen = 0;
string fileName = "";
long fileLen = 0;
int NoOfPackets = 0;
int receivedBytes = 0;
int i, j;
// Connect to a remote device.
try
{
// Establish the remote endpoint for the socket.
// This example uses port 11000 on the local computer.
IPHostEntry ipHostInfo = Dns.Resolve(Dns.GetHostName());
IPAddress ipAddress = ipHostInfo.AddressList[0];
IPEndPoint remoteEP = new IPEndPoint(ipAddress, 11000);
// Create a TCP/IP socket.
Socket receiver = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
// Connect the socket to the remote endpoint. Catch any errors.
try
{
receiver.Connect(remoteEP);
buffer = new byte[4];
receiver.Receive(buffer, 4, 0);
int filesNumber = BitConverter.ToInt32(buffer, 0);
for (i = 0; i < filesNumber; i++)
{
buffer = new byte[4];
receiver.Receive(buffer, 4, 0);
fileNameLen = BitConverter.ToInt32(buffer, 0);
// --
buffer = new byte[fileNameLen];
receiver.Receive(buffer, fileNameLen, 0);
fileName = Encoding.UTF8.GetString(buffer);
// --
buffer = new byte[8];
receiver.Receive(buffer, 8, 0);
fileLen = BitConverter.ToInt64(buffer, 0);
// --
NoOfPackets = Convert.ToInt32(Math.Ceiling(
Convert.ToDouble(fileLen) / Convert.ToDouble(BufferSize) ));
fileStream = new FileStream(destFilePath + fileName, FileMode.OpenOrCreate, FileAccess.Write);
receivedBytes = 0;
// --
for (j = 0; j < NoOfPackets; j++)
{
if (fileLen > BufferSize)
{
buffer = new byte[BufferSize];
receivedBytes = receiver.Receive(buffer, BufferSize, 0);
fileStream.Write(buffer, 0, receivedBytes);
fileLen -= BufferSize;
}
else
{
buffer = new byte[fileLen];
receivedBytes = receiver.Receive(buffer, (int)fileLen, 0);
fileStream.Write(buffer, 0, receivedBytes);
}
}
fileStream.Close();
}
// Release the socket.
receiver.Shutdown(SocketShutdown.Both);
receiver.Close();
}
catch (ArgumentNullException ane)
{
Console.WriteLine("ArgumentNullException : {0}", ane.ToString());
}
catch (SocketException se)
{
Console.WriteLine("SocketException : {0}", se.ToString());
}
catch (Exception e)
{
Console.WriteLine("Unexpected exception : {0}", e.ToString());
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
public static int Main(String[] args)
{
StartReceiving();
return 0;
}
}
server
//server code
using System;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Collections.Generic;
class Server003
{
public static void StartListening()
{
// Data buffer for incoming data.
byte[] buffer;
byte[] fileNameByte;
byte[] fileNameLenByte;
byte[] fileLenByte;
// FileStream to write data
FileStream fileStream;
Int64 fileLen = 0;
int NoOfPackets = 0;
int readBytes = 0;
int i;
// Establish the local endpoint for the socket.
// Dns.GetHostName returns the name of the
// host running the application.
IPHostEntry ipHostInfo = Dns.Resolve(Dns.GetHostName());
IPAddress ipAddress = ipHostInfo.AddressList[0];
IPEndPoint localEndPoint = new IPEndPoint(ipAddress, 11000);
// Create a TCP/IP socket.
Socket listener = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
// Bind the socket to the local endpoint and
// listen for incoming connections.
try
{
listener.Bind(localEndPoint);
listener.Listen(10);
// Start listening for connections.
Console.WriteLine("Waiting for a connection...");
// Program is suspended while waiting for an incoming connection.
Socket handler = listener.Accept();
Int32 filesNumber = binFilesNames.Count;
byte[] filesNumberByte = BitConverter.GetBytes(filesNumber);
handler.Send(filesNumberByte);
// --
foreach (string binName in binFilesNames)
{
fileNameByte = Encoding.UTF8.GetBytes(binName);
fileNameLenByte = BitConverter.GetBytes(fileNameByte.Length);
handler.Send(fileNameLenByte);
handler.Send(fileNameByte);
// --
fileStream = new FileStream(sourceFilePath + binName, FileMode.Open, FileAccess.Read);
fileLen = fileStream.Length;
fileLenByte = BitConverter.GetBytes(fileLen);
handler.Send(fileLenByte);
// --
NoOfPackets = Convert.ToInt32(Math.Ceiling(
Convert.ToDouble(fileLen) / Convert.ToDouble(BufferSize)));
for (i = 0; i < NoOfPackets; i++)
{
if (fileLen > BufferSize)
{
buffer = new byte[BufferSize];
// reeding data from file and writing it to the bytes "buffer"
readBytes = fileStream.Read(buffer, 0, BufferSize);
// send bytes from "buffer"
handler.Send(buffer, readBytes, SocketFlags.None);
fileLen -= BufferSize;
}
else
{
buffer = new byte[fileLen];
// reeding data from file and writing it to the bytes "buffer"
readBytes = fileStream.Read(buffer, 0, (int)fileLen);
// send bytes from "buffer"
handler.Send(buffer, readBytes, SocketFlags.None);
}
}
fileStream.Close();
}
// Release the socket.
handler.Shutdown(SocketShutdown.Both);
handler.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
Console.WriteLine("\nPress ENTER to continue...");
Console.Read();
}
public static List<string> GetFiles()
{
var dir = new DirectoryInfo(sourceFilePath); // folder with files
var files = new List<string>(); // List with file names
foreach (FileInfo file in dir.GetFiles("T*260000.bin"))
{
files.Add(Path.GetFileName(file.FullName));
}
return files;
}
public static int Main(String[] args)
{
binFilesNames = GetFiles();
StartListening();
return 0;
}
const string sourceFilePath = #"..\..\..\Binaries\";
static List<string> binFilesNames;
const int BufferSize = 1024;
}
UPD:
I took into account the moments that pointed LB2. Here is the receive part and it works as it need:
while ((receivedBytes = receiver.Receive(buffer)) > 0) // receive bytes to "buffer"
{
var tmpBuff = buffer.Take(receivedBytes); // takes first receivedBytes elements
bufferList.AddRange(tmpBuff);
}
But I don't understand how sending work. When I send whole data at once - all ok, but when i trying to send partially it crashes:
This works and whole data sent:
handler.Send(buffer);
This one crashes:
int sentBytes = 0;
int sumSentBytes = 0;
do
{
// send bytes from "buffer"
sentBytes = handler.Send(buffer, sumSentBytes, BufferSize, SocketFlags.None);
sumSentBytes += sentBytes;
}
while (sentBytes > 0);
So what is the best way to construct sending of large amounts of data (in my case about 20Mb, but it depends)?
There are multiple bugs in the code to be able to pinpoint specifically where this particular came from. Here are a few things you should be aware and where code needs clean up:
Socket class is IDisposable and thus should be wrapped in using. (I don't know if this is full program, or just a snippet with a driver main(), but if you call StartReceiving enough times, it'll leak memory).
FileStream (that you have a in a for loop) is IDisposable and thus should be wrapped in using. (Call to .Close() may actually clean up enough, but still better to use using.)
Use of Socket.Receive() is incorrect. You cannot assume that you receive as many bytes as you requested. Receive() returns either 0 if connection is lost, or number of bytes (upto requested count) that are currently available in the receive buffer. So as you go through:
buffer = new byte[fileNameLen];
receiver.Receive(buffer, fileNameLen, 0);
fileName = Encoding.UTF8.GetString(buffer);
// --
buffer = new byte[8];
receiver.Receive(buffer, 8, 0);
fileLen = BitConverter.ToInt64(buffer, 0);
... it is quite possible that you only read part of fileName bytes, get partial fileName, and then remainder bytes for filename are actually (incorrectly) interpreted as fileLen.
You correctly use receivedBytes to copy received bytes to file stream, but then you incorrectly decrement fileLen by BufferSize rather than receivedBytes, thus corrupting your file by possibly writing only part of the stream, in this part of the code:
receivedBytes = receiver.Receive(buffer, BufferSize, 0);
fileStream.Write(buffer, 0, receivedBytes);
fileLen -= BufferSize;
You keep reallocating new byte[] in a loop for each call to .Receive which is unnecessary. You can keep reusing the same buffer.
For the server code, exception screenshot that you posted has ? for the message (likly encoding issues). Please trap and post the actual message.
These are just a few things that I spotted by casual review. Whether some of these are the culprits, or there is some other issue is hard to ascertain with these issues present.
I guess you get an exception:
ArgumentOutOfRangeException: size is greater than the length of buffer minus the value of the offset parameter.
You'll have to subtract the bytes already sent from the size parameter:
int bytesToSend = BufferSize - sumSentBytes;
sentBytes = handler.Send(buffer, sumSentBytes, bytesToSend, SocketFlags.None);
I have the following code:
private string Connect()
{
string responseData;
try
{
TcpClient client = new TcpClient(ServerIp, Port);
client.ReceiveBufferSize = Int32.MaxValue;
Byte[] data = Encoding.GetEncoding(1251).GetBytes(ReadyQuery);
NetworkStream stream = client.GetStream();
// send data
stream.Write(data, 0, data.Length);
// buffer
data = new Byte[65536];
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = Encoding.GetEncoding(1251).GetString(data, 0, bytes);
// close all
stream.Close();
client.Close();
return responseData;
}
I have problem with a big message. The receive message size is 22K chars. I get only part of message.
How can I receive big messages?
PS. In the debugger bytes equal 4096.
You call stream.Read in a loop until you read the entire message. If you know the message size in advance it's relatively easy:
int messageSize = 22000;
int readSoFar = 0;
byte [] msg = new byte[messageSize];
while(readSoFar < messageSize)
{
var read = stream.Read(msg, readSoFar, msg.Length - readSoFar);
readSoFar += read;
if(read==0)
break; // connection was broken
}
If the message size is part of the message (say, encoded in the first 4 bytes), you should read those first and then do as I suggested.