Develop Reference

XML Serialization over TCP Issues

Just testing a simple example program, for sending an XML Serialized object over TCP. I'm finding that unless I close the stream or TcpClient on the side that sends a serialized object, the receiver never receives it.
class Program
{
private static XmlSerializer positionSerializer = new XmlSerializer(typeof(GazePosition));
static void Main(string[] args)
{
new Thread(Server).Start();
new Thread(Client).Start();
Console.ReadLine();
}
static void Server()
{
TcpListener listener = new TcpListener(IPAddress.Loopback, 30000);
listener.Start();
TcpClient client = listener.AcceptTcpClient();
Stream stream = client.GetStream();
StreamReader reader = new StreamReader(stream);
var cmd = reader.ReadLine();
if (cmd == "GetPosition")
{
Console.WriteLine("Received GetPosition Command");
GazePosition pos = new GazePosition(-5, 5);
positionSerializer.Serialize(stream, pos);
}
//client.Close();
listener.Stop();
}
static void Client()
{
TcpClient client = new TcpClient();
client.Connect(IPAddress.Loopback, 30000);
Stream stream = client.GetStream();
StreamWriter writer = new StreamWriter(stream);
writer.WriteLine("GetPosition");
writer.Flush();
var pos = (GazePosition)positionSerializer.Deserialize(stream);
Console.WriteLine("Received: {0}", pos);
stream.Close();
client.Close();
}
}
[Serializable]
[XmlRoot("GazePosition")]
public class GazePosition
{
[XmlElement("X")]
public float X;
[XmlElement("Y")]
public float Y;
public GazePosition()
{
}
public GazePosition(float x, float y)
{
X = x;
Y = y;
}
public override string ToString()
{
return string.Format("{0},{1}", X, Y);
}
}
This prints out only:
Received GetPosition command
However, if I uncomment client.Close()
This prints out:
Received GetPosition command
Received -5, 5
I would like this example be able to scale to receiving multiple commands, but this is painful while I'm forced to close the stream each time I receive a command. If someone could enlighten me why this is happening, I would be very thankful!

Here what you are missing is that you are supposed to read from the TcpClient's stream on the client side. Please have a look what I simply did to get it to work. Please keep in mind that this is simply an example.
writer.WriteLine("GetPosition");
writer.Flush();
//Just like you did you should seperate the payload
//Either by size or new line or some special key you set
byte[] buffer = new byte[1024];
stream.Read(buffer, 0, buffer.Length);
using (var ms = new MemoryStream(buffer))
{
var gazePos = (GazePosition)positionSerializer.Deserialize(ms);
//DOSTUFF
}
//Added code finished
Here is some simple beta level library. You may want to have a look→
Simple Server
Edit: I further looked at NetworkStream's Dispose method. It has native calls to Windows API so I am not able to further into implementation details.However, as you can see here, during a graceful shutdown, which occurs when you dispose a network stream, sends a FD_CLOSE

Unity .net socket + protobuf issues

Currently I'm trying to develop a multiplayer game with Unity. I checked the Unity's built-in Network and NetworkManager but I need the server dedicated. Hence I developed a java server and implements the protolbuf which is provided by Google. It works with my cocos project but not with this Unity one.
Two problems, first, no matter what did I send to server, the server side has a InvalidProtolBufferException :
While parsing a protocol message, the input ended unexpectedly in the middle > of a field. This could mean either than the input has been truncated or that > embedded message misreported its own length.
second, the stream.read method always makes my unity not responding. Here is my client-side code:
public class SocketClient : MonoBehaviour {
public Text send;
public Text read;
const string ipAddress = "192.168.0.233";
const int port = 8080;
TcpClient socket;
NetworkStream stream;
BinaryWriter bw;
BinaryReader br;
// Use this for initialization
void Start () {
SetupSocket();
}
void SetupSocket() {
socket = new TcpClient(ipAddress, port);
stream = socket.GetStream();
bw = new BinaryWriter(socket.GetStream());
br = new BinaryReader(socket.GetStream());
}
// Update is called once per frame
void Update () {
ReadMessage();
}
public void SendMessage() {
//NetworkStream stream = new NetworkStream(socket.Client);
MessagesProto msg = new MessagesProto();
msg.id = int.Parse(send.text);
using (MemoryStream ms = new MemoryStream())
{
Serializer.Serialize<MessagesProto>(ms, msg);
ms.Position = 0;
byte[] data = ms.ToArray();
ms.Position = 0;
//foreach (byte d in data) {
// Debug.Log(d);
//}
stream.Write(data, 0 , data.Length);
}
}
public void ReadMessage() {
if (stream.CanRead) {
//byte[] receiveData = new byte[socket.ReceiveBufferSize];
byte[] receiveData = new byte[4];
socket.GetStream().Read(receiveData, 0, 4);
Debug.Log("Loading...");
using (MemoryStream ms = new MemoryStream()) {
ms.Write(receiveData, 0, receiveData.Length);
ms.Position = 0;
var msg = Serializer.Deserialize<MessagesProto>(ms);
read.text = msg.data + "";
}
}
}
}
I tried to set the stream.CanRead to stream.DataAvailable, no more crash but not reading anything either, these .Net sockets problems drive me crazy, anyone can help me with this please?

It is not responding because your socket code is synchronous(blocking). You can solve this by either using asynchronous version of the socket functions or use Thread to do all your receiving stuff. This has been answered many times and I will just link to the answers.
TCP Server in Unity
Just port the code to Client.
UDP.

Sending big files in socket programming c# [duplicate]

This question already has an answer here:
Why does my client socket not receive what my server socket sends?
(1 answer)
Closed 7 years ago.
I want to send big files with socket in c#. File can be transferred but when I want to open it, I see it is damaged. What's the problem?
I broke the file to 2KB in array in client code and send it. Then, in server code, I received it and put it in byte array and convert to file.
server code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Net;
using System.Net.Sockets;
namespace MyServer
{
class Program
{
static void Main(string[] args)
{
bool full = false;
byte[] data = new byte[2049];
byte[] bsize = new byte[2048];
List<byte> bytes = new List<byte>();
IPEndPoint ipep = new IPEndPoint(IPAddress.Any, 9050);
Socket newsock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
newsock.Bind(ipep);
newsock.Listen(100);
Console.WriteLine("Wating For Client ...");
Socket client = newsock.Accept();
IPEndPoint clientep = (IPEndPoint)client.RemoteEndPoint;
Console.WriteLine("Connected with {0} at port {1}",clientep.Address,clientep.Port);
Console.Write("Enter the name of file: ");
string Address = Console.ReadLine();
client.Receive(bsize);
int size = BitConverter.ToInt32(bsize, 0);
byte[] bytes2 = new byte[size];
int k = 0;
while (true)
{
client.Receive(data);
for(int i = k,j=0; i < k + 2048 && j<2048; i++,j++)
{
if (i == size)
{
full = true;
break;
}
bytes2[i] = data[j];
//bytes.Insert(i,data[j]);
}
k += 2048;
if (full == true)
break;
/*if (bytes.Count >= size)
break;*/
}
File.WriteAllBytes(#"G:\"+Address,bytes2 /*bytes.ToArray()*/);
Console.WriteLine("Disconnected from Client {0}",clientep.Address);
client.Close();
newsock.Close();
Console.ReadKey();
}
}
}
client code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Net;
using System.Net.Sockets;
namespace MyClient
{
class Program
{
static void Main(string[] args)
{
int reza = 0;
IPEndPoint ipep = new IPEndPoint(IPAddress.Parse("192.168.0.2"), 9050);
Socket server = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
try
{
server.Connect(ipep);
}
catch (SocketException e)
{
Console.WriteLine("Unable to connect to Server");
Console.WriteLine(e.ToString());
Console.ReadKey();
return;
}
Console.Write("Enter Address Of File:");
string Address = Console.ReadLine();
FileStream File = new FileStream(Address, FileMode.Open, FileAccess.Read);
byte[] bytes = System.IO.File.ReadAllBytes(#Address);
File.Close();
byte[] data = new byte[2048];
byte[] Size = BitConverter.GetBytes(bytes.Length);
server.Send(Size, Size.Length, SocketFlags.None);
for (int i = 0; i <= bytes.Length; i +=2048)
{
int k = i + 2048;
for(int j=i,d=0 ;j< k && j<bytes.Length; j++,d++)
{
data[d] = bytes[j];
reza++;
}
if (reza == 2048)
{
server.Send(data, data.Length, SocketFlags.None);
Console.Write("*");
reza = 0;
}
else
{
server.Send(data, reza, SocketFlags.None);
Console.WriteLine("*");
}
}
Console.WriteLine("Disconnecting from server ...");
server.Shutdown(SocketShutdown.Both);
server.Close();
Console.ReadKey();
}
}
}

The problem is that you ignore the return value from Receive(). There is no guarantee what-so-ever that your buffer will be completely filled.
The return value tells you how much data that the socket actually read from the network. Adjust your receiving code so it's taken into account. Same goes for the send. If the internal socket buffer gets full it will report less bytes sent than you have requested to send.
Other than that, I suggest you start to give your variables meaningful names. Your code is not very readable as is.
Here is a better receive routine:
var fileBuffer = new byte[size];
var receiveBuffer = new byte[2048];
var bytesLeftToReceive = size;
var fileOffset = 0;
while (bytesLeftToReceive > 0)
{
//receive
var bytesRead = client.Receive(receiveBuffer);
if (bytesRead == 0)
throw new InvalidOperationException("Remote endpoint disconnected");
//if the socket is used for other things after the file transfer
//we need to make sure that we do not copy that data
//to the file
int bytesToCopy = Math.Min(bytesRead, bytesLeftToReceive);
// copy data from our socket buffer to the file buffer.
Buffer.BlockCopy(receiveBuffer, 0, bytesLeftToReceive, fileBuffer, fileOffset);
//move forward in the file buffer
fileOffset += bytesToCopy;
//update our tracker.
bytesLeftToReceive -= bytesToCopy;
}
However, since you are transferring large files, isn't it better to write it directly to a file stream?
var stream = File.Create(#"C:\path\to\file.dat");
var receiveBuffer = new byte[2048];
var bytesLeftToReceive = size;
while (bytesLeftToReceive > 0)
{
//receive
var bytesRead = client.Receive(receiveBuffer);
if (bytesRead == 0)
throw new InvalidOperationException("Remote endpoint disconnected");
//if the socket is used for other things after the file transfer
//we need to make sure that we do not copy that data
//to the file
int bytesToCopy = Math.Min(bytesRead, bytesLeftToReceive);
// write to file
stream.Write(receiveBuffer, 0, bytesToCopy);
//update our tracker.
bytesLeftToReceive -= bytesToCopy;
}
On the send side I would do something like this:
// read directly from the file to reduce memory usage.
var fileName = #"....";
using (var file = File.OpenRead(fileName))
{
var sendBuffer = new byte[2048];
var fileSize = BitConverter.GetBytes((int)file.Length);
server.Send(fileSize, fileSize.Length, SocketFlags.None);
var bytesLeftToTransmit = fileSize;
while (bytesLeftToTransmit > 0)
{
var dataToSend = file.Read(sendBuffer, 0, sendBuffer.Length);
bytesLeftToTransmit -= dataToSend;
//loop until the socket have sent everything in the buffer.
var offset=0;
while (dataToSend > 0)
{
var bytesSent = socket.Send(sendBuffer, offset, dataToSend);
dataToSend -= bytesSent;
offset += bytesSent;
}
}
}
Finally you have a Socket.SendFile which have been optimized to send files. You can invoke it at any time for an open socket.

Sending multiple files Client & Server

I have a Client and Server code for files sending. For some reason I need to receive at client and send from Server...
Everything work perfectly, in some cases all files sent and received perfectly. In another cases after sending a few files programm crashes. Don't understand where the problem...
Errors:
client colsole
server console
client
// client code
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Text;
class Client003
{
const string destFilePath = #"..\..\..\";
const int BufferSize = 1024;
public static void StartReceiving()
{
// Data buffer for sending data.
byte[] buffer;
// FileStream to read data
FileStream fileStream;
int fileNameLen = 0;
string fileName = "";
long fileLen = 0;
int NoOfPackets = 0;
int receivedBytes = 0;
int i, j;
// Connect to a remote device.
try
{
// Establish the remote endpoint for the socket.
// This example uses port 11000 on the local computer.
IPHostEntry ipHostInfo = Dns.Resolve(Dns.GetHostName());
IPAddress ipAddress = ipHostInfo.AddressList[0];
IPEndPoint remoteEP = new IPEndPoint(ipAddress, 11000);
// Create a TCP/IP socket.
Socket receiver = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
// Connect the socket to the remote endpoint. Catch any errors.
try
{
receiver.Connect(remoteEP);
buffer = new byte[4];
receiver.Receive(buffer, 4, 0);
int filesNumber = BitConverter.ToInt32(buffer, 0);
for (i = 0; i < filesNumber; i++)
{
buffer = new byte[4];
receiver.Receive(buffer, 4, 0);
fileNameLen = BitConverter.ToInt32(buffer, 0);
// --
buffer = new byte[fileNameLen];
receiver.Receive(buffer, fileNameLen, 0);
fileName = Encoding.UTF8.GetString(buffer);
// --
buffer = new byte[8];
receiver.Receive(buffer, 8, 0);
fileLen = BitConverter.ToInt64(buffer, 0);
// --
NoOfPackets = Convert.ToInt32(Math.Ceiling(
Convert.ToDouble(fileLen) / Convert.ToDouble(BufferSize) ));
fileStream = new FileStream(destFilePath + fileName, FileMode.OpenOrCreate, FileAccess.Write);
receivedBytes = 0;
// --
for (j = 0; j < NoOfPackets; j++)
{
if (fileLen > BufferSize)
{
buffer = new byte[BufferSize];
receivedBytes = receiver.Receive(buffer, BufferSize, 0);
fileStream.Write(buffer, 0, receivedBytes);
fileLen -= BufferSize;
}
else
{
buffer = new byte[fileLen];
receivedBytes = receiver.Receive(buffer, (int)fileLen, 0);
fileStream.Write(buffer, 0, receivedBytes);
}
}
fileStream.Close();
}
// Release the socket.
receiver.Shutdown(SocketShutdown.Both);
receiver.Close();
}
catch (ArgumentNullException ane)
{
Console.WriteLine("ArgumentNullException : {0}", ane.ToString());
}
catch (SocketException se)
{
Console.WriteLine("SocketException : {0}", se.ToString());
}
catch (Exception e)
{
Console.WriteLine("Unexpected exception : {0}", e.ToString());
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
public static int Main(String[] args)
{
StartReceiving();
return 0;
}
}
server
//server code
using System;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Text;
using System.Collections.Generic;
class Server003
{
public static void StartListening()
{
// Data buffer for incoming data.
byte[] buffer;
byte[] fileNameByte;
byte[] fileNameLenByte;
byte[] fileLenByte;
// FileStream to write data
FileStream fileStream;
Int64 fileLen = 0;
int NoOfPackets = 0;
int readBytes = 0;
int i;
// Establish the local endpoint for the socket.
// Dns.GetHostName returns the name of the
// host running the application.
IPHostEntry ipHostInfo = Dns.Resolve(Dns.GetHostName());
IPAddress ipAddress = ipHostInfo.AddressList[0];
IPEndPoint localEndPoint = new IPEndPoint(ipAddress, 11000);
// Create a TCP/IP socket.
Socket listener = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
// Bind the socket to the local endpoint and
// listen for incoming connections.
try
{
listener.Bind(localEndPoint);
listener.Listen(10);
// Start listening for connections.
Console.WriteLine("Waiting for a connection...");
// Program is suspended while waiting for an incoming connection.
Socket handler = listener.Accept();
Int32 filesNumber = binFilesNames.Count;
byte[] filesNumberByte = BitConverter.GetBytes(filesNumber);
handler.Send(filesNumberByte);
// --
foreach (string binName in binFilesNames)
{
fileNameByte = Encoding.UTF8.GetBytes(binName);
fileNameLenByte = BitConverter.GetBytes(fileNameByte.Length);
handler.Send(fileNameLenByte);
handler.Send(fileNameByte);
// --
fileStream = new FileStream(sourceFilePath + binName, FileMode.Open, FileAccess.Read);
fileLen = fileStream.Length;
fileLenByte = BitConverter.GetBytes(fileLen);
handler.Send(fileLenByte);
// --
NoOfPackets = Convert.ToInt32(Math.Ceiling(
Convert.ToDouble(fileLen) / Convert.ToDouble(BufferSize)));
for (i = 0; i < NoOfPackets; i++)
{
if (fileLen > BufferSize)
{
buffer = new byte[BufferSize];
// reeding data from file and writing it to the bytes "buffer"
readBytes = fileStream.Read(buffer, 0, BufferSize);
// send bytes from "buffer"
handler.Send(buffer, readBytes, SocketFlags.None);
fileLen -= BufferSize;
}
else
{
buffer = new byte[fileLen];
// reeding data from file and writing it to the bytes "buffer"
readBytes = fileStream.Read(buffer, 0, (int)fileLen);
// send bytes from "buffer"
handler.Send(buffer, readBytes, SocketFlags.None);
}
}
fileStream.Close();
}
// Release the socket.
handler.Shutdown(SocketShutdown.Both);
handler.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
Console.WriteLine("\nPress ENTER to continue...");
Console.Read();
}
public static List<string> GetFiles()
{
var dir = new DirectoryInfo(sourceFilePath); // folder with files
var files = new List<string>(); // List with file names
foreach (FileInfo file in dir.GetFiles("T*260000.bin"))
{
files.Add(Path.GetFileName(file.FullName));
}
return files;
}
public static int Main(String[] args)
{
binFilesNames = GetFiles();
StartListening();
return 0;
}
const string sourceFilePath = #"..\..\..\Binaries\";
static List<string> binFilesNames;
const int BufferSize = 1024;
}
UPD:
I took into account the moments that pointed LB2. Here is the receive part and it works as it need:
while ((receivedBytes = receiver.Receive(buffer)) > 0) // receive bytes to "buffer"
{
var tmpBuff = buffer.Take(receivedBytes); // takes first receivedBytes elements
bufferList.AddRange(tmpBuff);
}
But I don't understand how sending work. When I send whole data at once - all ok, but when i trying to send partially it crashes:
This works and whole data sent:
handler.Send(buffer);
This one crashes:
int sentBytes = 0;
int sumSentBytes = 0;
do
{
// send bytes from "buffer"
sentBytes = handler.Send(buffer, sumSentBytes, BufferSize, SocketFlags.None);
sumSentBytes += sentBytes;
}
while (sentBytes > 0);
So what is the best way to construct sending of large amounts of data (in my case about 20Mb, but it depends)?

There are multiple bugs in the code to be able to pinpoint specifically where this particular came from. Here are a few things you should be aware and where code needs clean up:
Socket class is IDisposable and thus should be wrapped in using. (I don't know if this is full program, or just a snippet with a driver main(), but if you call StartReceiving enough times, it'll leak memory).
FileStream (that you have a in a for loop) is IDisposable and thus should be wrapped in using. (Call to .Close() may actually clean up enough, but still better to use using.)
Use of Socket.Receive() is incorrect. You cannot assume that you receive as many bytes as you requested. Receive() returns either 0 if connection is lost, or number of bytes (upto requested count) that are currently available in the receive buffer. So as you go through:
buffer = new byte[fileNameLen];
receiver.Receive(buffer, fileNameLen, 0);
fileName = Encoding.UTF8.GetString(buffer);
// --
buffer = new byte[8];
receiver.Receive(buffer, 8, 0);
fileLen = BitConverter.ToInt64(buffer, 0);
... it is quite possible that you only read part of fileName bytes, get partial fileName, and then remainder bytes for filename are actually (incorrectly) interpreted as fileLen.
You correctly use receivedBytes to copy received bytes to file stream, but then you incorrectly decrement fileLen by BufferSize rather than receivedBytes, thus corrupting your file by possibly writing only part of the stream, in this part of the code:
receivedBytes = receiver.Receive(buffer, BufferSize, 0);
fileStream.Write(buffer, 0, receivedBytes);
fileLen -= BufferSize;
You keep reallocating new byte[] in a loop for each call to .Receive which is unnecessary. You can keep reusing the same buffer.
For the server code, exception screenshot that you posted has ? for the message (likly encoding issues). Please trap and post the actual message.
These are just a few things that I spotted by casual review. Whether some of these are the culprits, or there is some other issue is hard to ascertain with these issues present.

I guess you get an exception:
ArgumentOutOfRangeException: size is greater than the length of buffer minus the value of the offset parameter.
You'll have to subtract the bytes already sent from the size parameter:
int bytesToSend = BufferSize - sumSentBytes;
sentBytes = handler.Send(buffer, sumSentBytes, bytesToSend, SocketFlags.None);

setting NetworkStream.ReceiveTimeout not triggering exception

This is a continuation of the this question. I am new to network programming, so I am just writing small sample stuff to gain understanding, but somewhat struggling with explaining results.
It seems setting NetworkStream.ReceiveTimeout is not working correctly when client that was supposed to be sending data simply closes before sending all the expected data.
Here is the sample code:
public static void Main(string[] args)
{
TcpListener listener = new TcpListener(IPAddress.Any, 10001);
listener.Start();
ThreadPool.QueueUserWorkItem(WriterThread);
using (TcpClient client = listener.AcceptTcpClient())
using (NetworkStream stream = client.GetStream())
{
client.ReceiveTimeout = (int)new TimeSpan(0, 0, 2).TotalMilliseconds;
stream.ReadTimeout = (int)new TimeSpan(0, 0, 2).TotalMilliseconds;
ReceiveMessage(stream, 1024);
}
listener.Stop();
Console.WriteLine("Done.");
Console.ReadKey(true);
}
private static void WriterThread(object state)
{
using (TcpClient client = new TcpClient())
{
client.Connect(new IPEndPoint(IPAddress.Loopback, 10001));
using (NetworkStream stream = client.GetStream())
{
byte[] bytes = Encoding.ASCII.GetBytes("obviously less than 1024 bytes");
stream.Write(bytes, 0, bytes.Length);
Thread.Sleep(10000); // comment out
}
}
}
private static byte[] ReceiveMessage(Stream stream, int length)
{
byte[] buffer = new byte[length];
int bufferFill = 0;
while (true)
{
bufferFill += stream.Read(buffer, bufferFill, buffer.Length - bufferFill);
if (buffer.Length == bufferFill)
return buffer;
Thread.Sleep(100);
}
}
This version works correctly triggering exception on the stream.Read() call. However If I comment out Thread.Sleep(10000), the client closes connection, but listener fails to recognize it. Main thread gets stuck inside the while(true) loop. The stream.Read() keeps returning zero, but no exception thrown.
Is this normal? If so how am I expected to handle abnormal client disconnections?

Yes, this sounds normal. There is no receive- or read timeout because the client has disconnected. This means that no more data is available for reading and the stream will return 0 immediately just as documented.
I would modify your ReceiveMessage method to something like the following:
private static byte[] ReceiveMessage(Stream stream, int length)
{
byte[] buffer = new byte[length];
int bufferFill = 0;
while (true)
{
int bytesRead = stream.Read(buffer, bufferFill, buffer.Length - bufferFill);
if (bytesRead == 0)
throw new Exception("No more data available.");
bufferFill += bytesRead;
if (buffer.Length == bufferFill)
return buffer;
Thread.Sleep(100);
}
}
Clearly if the stream.Read() call returns 0 before we have received all the expected bytes there must have been some form of disconnection or similar. Either way we will never get any more data from the stream.
Edit: The Stream class has no notion of a "message". The Read method blocks until more data becomes available if none is already in the buffer. It will however return 0 when no more data can be received, which in this case means the connection is closed.