Best Way to write large data over network stream of TcpClient - c#

We have a requirement to upload large firmware files to printers to upgrade the firmware of the device. The printer device is in the same network as my server, and the size of firmware that we are trying to upload is approximately to 200 - 500 MB. The approach that we have chosen is to load the firmware (.bin file) into Memory stream and write it in chunks over the network using TcpClient.
Based on the response from the network stream, we are displaying the status of firmware upgrade to the client. Following is the code snippet that we have used for firmware upgrade. I want to know if it is the best approach, as wrong one may harm the device.
EDIT:
class MyClass
{
int port = 9100;
string _deviceip;
byte[] m_ReadBuffer = null;
TcpClient _tcpclient;
NetworkStream m_NetworkStream;
static string CRLF = "\r\n";
public event EventHandler<DeviceStatus> onReceiveUpdate;
public async Task<bool> UploadFirmware(Stream _stream)
{
bool success = false;
try
{
_tcpclient = new TcpClient();
_tcpclient.Connect(_deviceip, port);
_stream.Seek(0, SeekOrigin.Begin);
m_NetworkStream = _tcpclient.GetStream();
byte[] buffer = new byte[1024];
m_ReadBuffer = new byte[1024];
int readcount = 0;
m_NetworkStream.BeginRead(m_ReadBuffer, 0, m_ReadBuffer.Length,
new AsyncCallback(EndReceive), null);
await Task.Run(() =>
{
while ((readcount = _stream.Read(buffer, 0, buffer.Length)) > 0)
{
m_NetworkStream.Write(buffer, 0, readcount);
m_NetworkStream.Flush();
}
});
success = true;
}
catch (Exception ex)
{
upgradeStatus = false;
}
return success;
}
private void EndReceive(IAsyncResult ar)
{
try
{
int nBytes;
nBytes = m_NetworkStream.EndRead(ar);
if (nBytes > 0)
{
string res = Encoding.UTF8.GetString(m_ReadBuffer, 0, nBytes);
DeviceStatus status = new DeviceStatus();
string[] readlines = res.Split(new string[] { CRLF },
StringSplitOptions.RemoveEmptyEntries);
foreach (string readline in readlines)
{
if (readline.StartsWith("CODE"))
{
//read readline string here
break;
}
}
}
if (m_NetworkStream.CanRead)
{
do
{
m_NetworkStream.BeginRead(m_ReadBuffer, 0, m_ReadBuffer.Length, new
AsyncCallback(EndReceive), null);
} while (m_NetworkStream.DataAvailable);
}
}
catch (ObjectDisposedException ods)
{
return;
}
catch (System.IO.IOException ex)
{
}
}
}
Any help will be really appreciated.

Your code is basically fine with a few issues:
m_NetworkStream.Flush(); AFAIK this does nothing. If it did something it would harm throughput. So delete that.
_stream.Seek(0, SeekOrigin.Begin); seeking is the callers concern, remove that. This is a layering violation.
Use bigger bigger buffers. Determine the right size experimentally. I usually start at 64KB for bulk transfers. This makes the IOs less chatty.
Turn on nagling which helps with bulk transfers because it saves you from spurious small packets.
You can replace the entire read-write-loop with Stream.Copy.
The way you report exceptions to the callers hides a lot of information. Just let the exception bubble out. Don't return a bool.
Use using for all resource to ensure they are cleaned up in the error case.
nBytes = m_NetworkStream.EndRead(ar); here, you assume that a single read will return all data that will be coming. You might receive just the first byte, though. Probably, you should use StreamReader.ReadLine in a loop until you know you are done.
catch (System.IO.IOException ex) { } What is that about? If firmware updates are such a critical thing suppressing errors appears very dangerous. How else can you find out about bugs?
I would convert the reading code to use await.

As the maximum length for a TcpPacket is 65535 (2^16-1), if you send any packet breaching this lenght it will be truncated. If I were you, I think the best way of sending large packets, is setting a Header of every packet and enumerating them. For example:
C->S ; [P1] <content>
and then the same structure, just plus 1 [P2] <content>
To do so, just use few substrings to truncate the data and sending them.
Cheers!

Related

C# NetworkStream detect all kind of disconnections and exceptions

I have this code to connect to streaming server. the server disconnect from time to time and I want to detect it and restart the connection when neede.
How can I detect in this code any kind of exceptions?
Because now I get disconnected and can't catch it.
this.ns = new NetworkStream(server);
while (true)
{
// Incoming message may be larger than the buffer size.
do
{
byte[] myReadBuffer = new byte[10240 * 5];
await this.ns.ReadAsync(myReadBuffer, 0, myReadBuffer.Length).ContinueWith((numberOfBytesRead) =>
{
string message = Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead.Result);
p.Invoke(message);
});
}
while (this.ns.DataAvailable);
}
Your variable numberOfBytesRead is actually the previous task that has finished from where you can check whether it is completed or failed.
if(numberOfBytesRead.IsFaulted)
{
var aggregatedEx = numberOfBytesRead.Exception;
//do something
}

Do TCP sockets automatically close after 64kB send?

I'm trying to send multiple files over TCP using C# TcpClient, for a file below 64kB it works great, but when I have more it throws an exeption that host-computer disconnected.
Here is my code:
Server side(sending).
1) SocketServer class
public abstract class SocketServer
{
private Socket serverSocket;
public SocketServer(IPEndPoint localEndPoint)
{
serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
serverSocket.Bind(localEndPoint);
serverSocket.Listen(0);
serverSocket.BeginAccept(BeginAcceptCallback, null);
}
private void BeginAcceptCallback(IAsyncResult ar)
{
Socket clientSocket = serverSocket.EndAccept(ar);
Console.WriteLine("Client connected");
ClientConnection clientConnection = new ClientConnection(this, clientSocket);
Thread clientThread = new Thread(new ThreadStart(clientConnection.Process));
clientThread.Start();
serverSocket.BeginAccept(BeginAcceptCallback, null);
}
internal abstract void OnReceiveMessage(ClientConnection client, byte header, byte[] data);
}
Here goes ClientConnection:
public class ClientConnection
{
private SocketServer server;
private Socket clientSocket;
private byte[] buffer;
private readonly int BUFFER_SIZE = 8192;
public ClientConnection(SocketServer server, Socket clientSocket)
{
this.server = server;
this.clientSocket = clientSocket;
buffer = new byte[BUFFER_SIZE];
}
public void Process()
{
clientSocket.BeginReceive(buffer, 0, buffer.Length, SocketFlags.Peek, BeginReceiveCallback, null);
}
private void BeginReceiveCallback(IAsyncResult ar)
{
int bytesReceived = clientSocket.EndReceive(ar);
if (bytesReceived >= 4)
{
clientSocket.Receive(buffer, 0, 4, SocketFlags.None);
// message size
int size = BitConverter.ToInt32(buffer, 0);
// read message
int read = clientSocket.Receive(buffer, 0, size, SocketFlags.None);
// if data still fragmented, wait for it
while (read < size)
{
read += clientSocket.Receive(buffer, read, size - read, SocketFlags.None);
}
ProcessReceivedData(size);
}
clientSocket.BeginReceive(buffer, 0, buffer.Length, SocketFlags.Peek, BeginReceiveCallback, null);
}
private void ProcessReceivedData(int size)
{
using (PacketReader pr = new PacketReader(buffer))
{
// message header = 1 byte
byte header = pr.ReadByte();
// next message data
byte[] data = pr.ReadBytes(size - 1);
server.OnReceiveMessage(this, header, data);
}
}
public void Send(byte[] data)
{
// first of all, send message length
clientSocket.Send(BitConverter.GetBytes(data.Length), 0, 4, SocketFlags.None);
// and then message
clientSocket.Send(data, 0, data.Length, SocketFlags.None);
}
}
Implementation of FileServer class:
public class FileServer : SocketServer
{
string BinaryPath;
public FileServer(IPEndPoint localEndPoint) : base(localEndPoint)
{
BinaryPath = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
}
internal override void OnReceiveMessage(ClientConnection client, byte hdr, byte[] data)
{
// Convert header byte to ENUM
Headers header = (Headers)hdr;
switch(header)
{
case Headers.Queue:
Queue(client, data);
break;
default:
Console.WriteLine("Wrong header received {0}", header);
break;
}
}
private void Queue(ClientConnection client, byte[] data)
{
// this message contains fileName
string fileName = Encoding.ASCII.GetString(data, 1, data.Length - 1);
// combine path with assembly location
fileName = Path.Combine(BinaryPath, fileName);
if (File.Exists(fileName))
{
FileInfo fileInfo = new FileInfo(fileName);
long fileLength = fileInfo.Length;
// pass the message that is now start a file transfer, contains:
// 1 byte = header
// 16 bytes = file length
using (PacketWriter pw = new PacketWriter())
{
pw.Write((byte)Headers.Start);
pw.Write(fileLength);
client.Send(pw.GetBytes());
}
//
using (FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.Read))
{
int read = 0, offset = 0;
byte[] fileChunk = new byte[8191];
while (offset < fileLength)
{
// pass message with file chunks, conatins
// 1 byte = header
// 8195 bytes = file chunk
using (PacketWriter pw = new PacketWriter())
{
fs.Position = offset;
read = fs.Read(fileChunk, 0, fileChunk.Length);
pw.Write((byte)Headers.Chunk);
pw.Write(fileChunk, 0, read);
client.Send(pw.GetBytes());
}
offset += read;
}
}
}
}
}
And helper classes:
public class PacketWriter : BinaryWriter
{
private MemoryStream memoryStream;
public PacketWriter() : base()
{
memoryStream = new MemoryStream();
OutStream = memoryStream;
}
public byte[] GetBytes()
{
byte[] data = memoryStream.ToArray();
return data;
}
}
public class PacketReader : BinaryReader
{
public PacketReader(byte[] data) : base(new MemoryStream(data))
{
binaryFormatter = new BinaryFormatter();
}
}
Client have the almost same code. Except receive:
internal override void OnReceiveMessage(ServerConnection client, byte hdr, byte[] data)
{
Headers header = (Headers)hdr;
switch(header)
{
case Headers.Start:
Start(data); // save length of file and prepare for receiving it
break;
case Headers.Chunk:
Chunk(data);
break;
default:
Console.WriteLine("Wrong header received {0}", header);
break;
}
}
private void Chunk(byte[] data)
{
// Process reveived data, write it into a file
}
Do TCP sockets automatically close after 64kB send?
No, TCP sockets do not automatically close after 64 KB of sent data, nor after any amount of sent data. They remain open until either end closes the connection or a network error of some sort happens.
Unfortunately, your question is extremely vague, providing practically no context. But I will point out one potentially serious bug in your client code: you are not restricting the data read by the client to that which you expect for the current file. If the server is sending more than one file, it is entirely possible and likely that on reaching the end of the data for the current file, a read containing that final sequence of bytes will also contain the initial sequence of bytes for the next file.
Thus, the expression totalBytesRead == fileLenght will fail to ever be true and your code will attempt to read all of the data the server sends as if it's all part of the first file.
One easy way to check for this scenario is to look at the size of the file that was written by the client. It will be much larger than expected, if the above is the issue.
You can fix the code by only ever receiving at most the number of bytes you actually expect to be remaining:
while ((bytesRead = binaryReader.Read(
buffer, 0, Math.Min(fileLenght - totalBytesRead, buffer.Length))) > 0)
{
fs.Write(buffer, 0, bytesRead);
totalBytesRead += bytesRead;
if (totalBytesRead == fileLenght)
{
break;
}
}
While you're at it, you might fix the spelling of the variable named fileLenght. I know Intellisense makes it easy to type the variable name whether it's spelled right or not, but if you one day have to grep through the source code looking for code that involves itself with "length" values, you'll miss that variable.
EDIT:
Having pointed out the problem in the code you posted with your question originally, I now find myself looking at completely different code, after your recent edit to the question. The example is still not a good, minimal, complete code example and so advice will still have to be of a limited nature. But I can see at least two major problems in your receive handling:
You are mixing Receive() and BeginReceive(). This is probably not too terrible in general; what makes it really bad in your case is that in the completion handler for the initial asynchronous receive, you are blocking the thread while you use the synchronous Receive() to (attempt to) receive the rest of the data, and then to make matters worse, in the FileServer case you continue to block that thread while you transmit the file data to the client.I/O completion handlers must not block the thread for any significant amount of time; they must retrieve the data for the completed operation, and return as quickly as is feasible. There's a lot of wiggle room, but not enough to justify tying up the IOCP thread for the entire file transfer.
While the first issue is bad, it's probably not bad enough to actually cause the connection to reset. In a low-volume scenario, and especially if you're just dealing with one client at a time at the moment, I doubt blocking the IOCP thread that's being used to call your I/O completion callback will cause any easily observable problems.What's much worse is that you allocate the buffer array once, with a length of 8192, and then proceed to attempt to stuff the entire counted-byte stream of bytes being sent. On the assumption that the client will be sending short messages, this is probably fine on the server side. But on the client side, the value of size can easily be greater than the 8192 bytes allocated in the buffer array.I find it very likely that your BeginReceiveCallback() method is throwing an ArgumentOutOfRangeException after receiving the first 8192 bytes, due to the fact that you are passing the Receive() method a size value that exceeds the length of the buffer array. I can't prove that, because the code example is incomplete. You'll have to follow up on that theory yourself.
As long as I'm reviewing the code you posted, some other irregularities stood out as I scanned through it:
It's a bit odd to be using a buffer length of 8191 when transmitting the file.
Your BeginReceiveCallback() method discards whatever bytes were sent initially. You call EndReceive(), which returns the count of bytes received, but the code that follows assumes that the size value in the stream is still waiting to be read, when it very easily could have been sent in the first block of bytes. Indeed, your comparison bytesReceived >= 4 seems to be intended to refrain from processing received data until you have that length value. Which is wrong for another reason:
If your header is 1 byte followed by a 4-byte count-of-bytes value, then you really need bytesReceived > 4. But worse, TCP is allowed to drip out a single byte at a time if it really wants to (it won't, but if you write your code correctly, it won't matter). Since you only process received data if you get at least 4 bytes, and since if you don't get 4 bytes, you just ignore whatever was sent so far, you could theoretically wind up ignoring all of the data that was sent.
That you have the risk that any data could be ignored at all is terrible. But #2 above guarantees it, and #3 just adds insult to injury.
When receiving the size value, you are reading a 32-bit integer, but when sending the size value (i.e. for the file itself), you are writing a 64-bit integer. You'll get the right number on the receiving end, because BinaryWriter uses little-endian and that means that the first four bytes of any 64-bit integer less than 2^31 are identical to the four bytes of the 32-bit integer representation of the same number. But you'll still wind up with four extra zero bytes after that; if treated as file data as they likely would, that would corrupt your file.

NetworkStream Receive, how to processing data without using 100% CPU?

I have a small game server I'm making that will have dozens of connections sending player data constantly. While I've finally accomplished some basics and now have data sending/receiving, I now face a problem of flooding the server and the client with too much data. I've tried to throttle it back but even then I am hitting 90-100% cpu simply because of receiving and processing the data received running up the CPU.
The method below is a bare version of receiving data from the server. The server sends a List of data to be received by the player, then it goes through that list. I've thought perhaps instead just using a dictionary with a key based on type rather than for looping but I don't think that will significantly improve it, the problem is that it is processing data non-stop because player positions are constantly being updated, sent to the server, then send to other players.
The code below shows receive for the client, the server receive looks very similar. How might I begin to overcome this issue? Please be nice, I am still new to network programming.
private void Receive(System.Object client)
{
MemoryStream memStream = null;
TcpClient thisClient = (TcpClient)client;
List<System.Object> objects = new List<System.Object>();
while (thisClient.Connected && playerConnected == true)
{
try
{
do
{
//when receiving data, first comes length then comes the data
byte[] buffer = GetStreamByteBuffer(netStream, 4); //blocks while waiting for data
int msgLenth = BitConverter.ToInt32(buffer, 0);
if (msgLenth <= 0)
{
playerConnected = false;
thisClient.Close();
break;
}
if (msgLenth > 0)
{
buffer = GetStreamByteBuffer(netStream, msgLenth);
memStream = new MemoryStream(buffer);
}
} while (netStream.DataAvailable);
if (memStream != null)
{
BinaryFormatter formatter = new BinaryFormatter();
memStream.Position = 0;
objects = new List<System.Object>((List<System.Object>)formatter.Deserialize(memStream));
}
}
catch (Exception ex)
{
Console.WriteLine("Exception: " + ex.ToString());
if (thisClient.Connected == false)
{
playerConnected = false;
netStream.Close();
thisClient.Close();
break;
}
}
try
{
if (objects != null)
{
for (int i = 0; i < objects.Count; i++)
{
if(objects[i] != null)
{
if (objects[i].GetType() == typeof(GameObject))
{
GameObject p = (GameObject)objects[i];
GameObject item;
if (mapGameObjects.TryGetValue(p.objectID, out item))
{
mapGameObjects[p.objectID] = p;;
}
else
{
mapGameObjects.Add(p.objectID, p);
}
}
}
}
}
}
catch (Exception ex)
{
Console.WriteLine("Exception " + ex.ToString());
if (thisClient.Connected == false)
{
playerConnected = false;
netStream.Close();
break;
}
}
}
Console.WriteLine("Receive thread closed for client.");
}
public static byte[] GetStreamByteBuffer(NetworkStream stream, int n)
{
byte[] buffer = new byte[n];
int bytesRead = 0;
int chunk = 0;
while (bytesRead < n)
{
chunk = stream.Read(buffer, (int)bytesRead, buffer.Length - (int)bytesRead);
if (chunk == 0)
{
break;
}
bytesRead += chunk;
}
return buffer;
}
Based on the code shown, I can't say why the CPU utilization is high. The loop will wait for data, and the wait should not consume CPU. That said, it still polls the connection in checking the DataAvailable property, which is inefficient and can cause you to ignore received data (in the implementation shown...that's not an inherent problem with DataAvailable).
I'll go one further than the other answer and state that you should simply rewrite the code. Polling the socket is just no way to handle network I/O. This would be true in any scenario, but it is especially problematic if you are trying to write a game server, because you're going to use up a lot of your CPU bandwidth needlessly, taking it away from game logic.
The two biggest changes you should make here are:
Don't use the DataAvailable property. Ever. Instead, use one of the asynchronous APIs for dealing with network I/O. My favorite approach with the latest .NET is to wrap the Socket in a NetworkStream (or get the NetworkStream from a TcpClient as you do in your code) and then use the Stream.ReadAsync() along with async and await. But the older asynchronous APIs for Sockets work well also.
Separate your network I/O code from the game logic code. The Receive() method you show here has both the I/O and the actual processing of the data relative to the game state in the same method. This two pieces of functionality really belong in two separate classes. Keep both classes, and especially the interface between them, very simple and the code will be a lot easier to write and to maintain.
If you decide to ignore all of the above, you should at least be aware that your GetStreamByteBuffer() method has a bug in it: if you reach the end of the stream before reading as many bytes were requested, you still return a buffer as large as was requested, with no way for the caller to know the buffer is incomplete.
And finally, IMHO you should be more careful about how you shutdown and close the connection. Read about "graceful closure" for the TCP protocol. It's important that each end signal that they are done sending, and that each end receive the other end's signal, before either end actually closes the connection. This will allow the underlying networking protocol to release resources as efficiently and as quickly as possible. Note that TcpClient exposes the socket as the Client property, which you can use to call Shutdown().
Polling is rarely a good approach to communication, unless you're programming 16-bit microcontrollers (and even then, probably not the best solution).
What you need to do is to switch to a producer-consumer pattern, where your input port (a serial port, an input file, or a TCP socket) will act as a producer filling a FIFO buffer (a queue of bytes), and some other part of your program will be able to asynchronously consume the enqueued data.
In C#, there are several ways to do it: you can simply write a couple of methods using a ConcurrentQueue<byte>, or a BlockingCollection, or you can try a library like the TPL Dataflow Library which IMO doesn't add too much value over existing structures in .NET 4. Prior to .NET 4, you would simply use a Queue<byte>, a lock, and a AutoResetEvent to do the same job.
So the general idea is:
When your input port fires a "data received" event, enqueue all received data into the FIFO buffer and set a synchronization event to notify the consumer,
In your consumer thread, wait for the synchonization event. When the signal is received, check if there is enough data in the queue. If yes, process it, if not, continue waiting for the next signal.
For robustness, use an additional watchdog timer (or simply "time since last received data") to be able to fail on timeout.
You want to use the Task-based Asynchronous Pattern. Probably making liberal use of the async function modifier and the await keyword.
You'd be best replacing GetStreamByteBuffer with a direct call to ReadAsync.
For instance you could asynchronously read from a stream like this.
private static async Task<T> ReadAsync<T>(
Stream source,
CancellationToken token)
{
int requestLength;
{
var initialBuffer = new byte[sizeof(int)];
var readCount = await source.ReadAsync(
initialBuffer,
0,
sizeof(int),
token);
if (readCount != sizeof(int))
{
throw new InvalidOperationException(
"Not enough bytes in stream to read request length.");
}
requestLength = BitConvertor.ToInt32(initialBuffer, 0);
}
var requestBuffer = new byte[requestLength];
var bytesRead = await source.ReadAsync(
requestBuffer,
0,
requestLength,
token);
if (bytesRead != requestLength)
{
throw new InvalidDataException(
string.Format(
"Not enough bytes in stream to match request length." +
" Expected:{0}, Actual:{1}",
requestLength,
bytesRead));
}
var serializer = new BinaryFormatter();
using (var requestData = new MemoryStream(requestBuffer))
{
return (T)serializer.Deserialize(requestData);
}
}
Like your code this reads an int from the stream to get the length, then reads that number of bytes and uses the BinaryFormatter to deserialize the data to the specified generic type.
Using this generic function you can simplify your logic,
private Task Receive(
TcpClient thisClient,
CancellationToken token)
{
IList<object> objects;
while (thisClient.Connected && playerConnected == true)
{
try
{
objects = ReadAsync<List<object>>(netStream, token);
}
catch (Exception ex)
{
Console.WriteLine("Exception: " + ex.ToString());
if (thisClient.Connected == false)
{
playerConnected = false;
netStream.Close();
thisClient.Close();
break;
}
}
try
{
foreach (var p in objects.OfType<GameObject>())
{
if (p != null)
{
mapGameObjects[p.objectID] = p;
}
}
}
catch (Exception ex)
{
Console.WriteLine("Exception " + ex.ToString());
if (thisClient.Connected == false)
{
playerConnected = false;
netStream.Close();
break;
}
}
}
Console.WriteLine("Receive thread closed for client.");
}
You need to put a Thread.Sleep(10) in your while loop. This is also a very fragile way to receive tcp data because it assumes the other side has sent all data before you call this receive. If the other side has only sent half of the data this method fails. This can be countered by either sending fixed sized packages or sending the length of a package first.
Your player position update is similar to the framebuffer update in the VNC protocol where the client request a screen frame & server responds to it with the updated screen data. But there is one exception, VNC server doesn't blindly send the new screen it only sends the changes if there is one. So you need to change the logic from sending all the requested list of objects to only to the objects which are changed after the last sent. Also in addition to it, you should send entire object only once after that send only the changed properties, this will greatly reduce the size of data sent & processed both at clients & server.

C# udp socket not receiving entire message

I have the following method which sends an rcon command to a game server.
public string sendCommand(string command)
{
byte[] bufferTemp = Encoding.ASCII.GetBytes(command);
byte[] bufferSend = new byte[bufferTemp.Length + 4];
//big enough to receive response
byte[] bufferRec = new byte[65000];
//intial 4 characters as per standard
bufferSend[0] = byte.Parse("255");
bufferSend[1] = byte.Parse("255");
bufferSend[2] = byte.Parse("255");
bufferSend[3] = byte.Parse("255");
int j = 4;
for (int i = 0; i < bufferTemp.Length; i++)
{
bufferSend[j++] = bufferTemp[i];
}
//send rcon command and get response
try
{
this.server.Socket.Send(bufferSend, SocketFlags.None);
this.server.Socket.Receive(bufferRec);
}
catch (SocketException e)
{
MessageBox.Show(e.ToString(), "Error occured");
}
string response = Encoding.ASCII.GetString(bufferRec);
return response;
}
out of all the commands I can possibly send, 1 of them returns a lot more data than the others and it seems that the *buffer_rec* byte array only gets about 1/4 of the message, but that array has been declared big enough to contain all the data.
On the subsequent 3 requests, the rest of the data gets output, as if it was buffered in some way.
I don't know why this is taking place. If you do, could you please let me know how to remedy the problem?
Thank you
Crouz
Ok, it seems that after 6 hours of looking into this I have finally come up with a solution, so here is a small change made to the receiving of data, note the thread sleeping for 10ms, this seems to be needed to let the datagrams arrive in due time.
//send rcon command and get response
string response = "";
try
{
this.server.Socket.Send(bufferSend, SocketFlags.None);
do
{
int bytesReceived = this.server.Socket.Receive(bufferRec);
response += Encoding.ASCII.GetString(bufferRec, 0, bytesReceived);
System.Threading.Thread.Sleep(10);
} while (this.server.Socket.Available > 0);
}
catch (SocketException e)
{
MessageBox.Show(e.ToString(), "Error occured");
}
If you think there is a better or more elegant way of handling this, don't hesitate to slam my code, but do so by offering an alternative as I am always happy to learn new things. Regards, Crouz

c# NetworkStream BeginRead seemingly overwriting byte

For some program, I want to send data over from a python program to a c# program. I put all of my data into a list in python and send it over after converting to bytes (packing them as doubles, so I have 8 bytes per number I am sending over). Having some understanding of how sockets and TCP streams work, the very first number in the list is the amount of bytes that the rest of the list takes up. Hence, the first 8 bytes of my stream tell me how many bytes I need to read to get all other data.
However, when I call BeginRead and it calls the callback, it has read 8 bytes less than I asked it to read. For instance, those first 8 bytes tell me there is 116432 bytes to read, but when I call EndRead it returns 116424.
Okay, so what? There's eight bytes missing, this amounts to one double being lost. This is a problem in and of itself, but I even figured out where this double has gone.
In python, at a specific point (while it is still doubles) in my data, I see I am sending this: "...,1961.0, 0.0128, 2033.0, 0.0442, 2034.0,..." when I inspect that same point in c# (after converting my bytes back to doubles), I see this: "..,1961.0, 2033.0002, 0.0442,2034.0,...".
To me, it seems clear that somehow these 8 bytes got mashed together into one, fusing the two number (bit-wise maybe?) together somehow. I have also noticed that the index of where this occurs in the byte data is roughly at the 64k-65k mark. So I'm suspecting that with 64kbytes being the max packet size of TCP packets, the stream has some kind of hiccup there and overwrites one part of my buffer without clearing it, leading to some literal mix up? Does anybody have any idea how I could fix this problem or what mistake I made that is causing this to happen?
I will paste the two relevant functions here.
private void Listen(int port)
{
try
{
// Perform a blocking call to accept requests.
// You could also user server.AcceptSocket() here.
var client = _server.AcceptTcpClient();
// Get a stream object for reading and writing
var stream = client.GetStream();
var pLength = new byte[8];
// Loop to receive all the data sent by the client.
while(_running)
{
if(!stream.DataAvailable && stream.Read(pLength, 0, 8) <= 0)
continue;
var nOfBytes = (int) BitConverter.ToDouble(pLength, 0);
pLength = new byte[8];
if (nOfBytes <= 0)
{
continue;
}
var localBytes = new byte[nOfBytes];
var scriptData = new ScriptData(stream, localBytes);
stream.BeginRead(localBytes, 0, nOfBytes, new AsyncCallback(GotAllBytes), scriptData);
}
// Shutdown and end connection
client.Close();
}
catch (SocketException e)
{
_errorBool = true;
_errorString = "Port: " + port + "\n" + e.Message + e.StackTrace;
}
finally
{
// Stop listening for new clients.
_server.Stop();
}
}
private void GotAllBytes(IAsyncResult result)
{
var scriptData = (ScriptData)result.AsyncState;
if (OnlinePaused)
{
scriptData.Stream.EndRead(result);
return;
}
var bytesRead = scriptData.Stream.EndRead(result);
_rawDataQueue.Enqueue(scriptData.Buffer.ToList());
}
Thanks for reading, I hope you can help out.
Thanks to Jeroen Mostert in the comments of the original question, this problem has been resolved by not working async but instead implementing a BinaryReader.
private void Listen(int port)
{
try
{
// Perform a blocking call to accept requests.
// You could also user server.AcceptSocket() here.
var client = _server.AcceptTcpClient();
// Get a stream object for reading and writing
var stream = client.GetStream();
var pLength = new byte[8];
// Loop to receive all the data sent by the client.
while(_running)
{
if(!stream.DataAvailable && stream.Read(pLength, 0, 8) <= 0)
continue;
var nOfBytes = (int) BitConverter.ToDouble(pLength, 0);
pLength = new byte[8];
if (nOfBytes <= 0)
{
continue;
}
var localBytes = new byte[nOfBytes];
var reader = new BinaryReader(stream);
ProcessData(reader);
}
// Shutdown and end connection
client.Close();
}
catch (SocketException e)
{
_errorBool = true;
_errorString = "Port: " + port + "\n" + e.Message + e.StackTrace;
}
finally
{
// Stop listening for new clients.
_server.Stop();
}
}
Thank you for everybody who was trying to help.

Categories

Resources