Develop Reference

Awaiting data from Serial Port in C#

I have an application that receives data from a wireless radio using RS-232. These radios use an API for communicating with multiple clients. To use the radios I created a library for communicate with them that other software can utilize with minimal changes from a normal SerialPort connection. The library reads from a SerialPort object and inserts incoming data into different buffers depending on the radio it receives from. Each packet that is received contains a header indicating its length, source, etc.
I start by reading the header, which is fixed-length, from the port and parsing it. In the header, the length of the data is defined before the data payload itself, so once I know the length of the data, I then wait for that much data to be available, then read in that many bytes.
Example (the other elements from the header are omitted):
// Read header
byte[] header = new byte[RCV_HEADER_LENGTH];
this.Port.Read(header, 0, RCV_HEADER_LENGTH);
// Get length of data in packet
short dataLength = header[1];
byte[] payload = new byte[dataLength];
// Make sure all the payload of this packet is ready to read
while (this.Port.BytesToRead < dataLength) { }
this.Port.Read(payload, 0, dataLength);
Obviously the empty while port is bad. If for some reason the data never arrives the thread will lock. I haven't encountered this problem yet, but I'm looking for an elegant way to do this. My first thought is to add a short timer that starts just before the while-loop, and sets an abortRead flag when it elapses that would break the while loop, like this:
// Make sure all the payload of this packet is ready to read
abortRead = false;
readTimer.Start();
while (this.Port.BytesToRead < dataLength && !abortRead) {}
This code needs to handle a constant stream of incoming data as quickly as it can, so keeping overhead to a minimum is a concern, and am wondering if I am doing this properly.

You don't have to run this while loop, the method Read would either fill the buffer for you or would throw a TimeoutException if buffer wasn't filled within the SerialPort.ReadTimeout time (which you can adjust to your needs).
But some general remark - your while loop would cause intensive CPU work for nothing, in the few milliseconds it would take the data to arrive you would have thousends of this while loop iterations, you should've add some Thread.Sleep inside.

If you want to truly adress this problem, you need to run the code in the background. There are different options to do that; you can start a thread, you start a Task or you can use async await.
To fully cover all options, the answer would be endless. If you use threads or tasks with the default scheduler and your wait time is expected to be rather short, you can use SpinWait.SpinUntil instead of your while loop. This will perform better than your solution:
SpinWait.SpinUntil(() => this.Port.BytesToRead >= dataLength);
If you are free to use async await, I would recommend this solution, since you need only a few changes to your code. You can use Task.Delay and in the best case you pass a CancellationToken to be able to cancel your operation:
try {
while (this.Port.BytesToRead < dataLength) {
await Task.Delay(100, cancellationToken);
}
}
catch(OperationCancelledException) {
//Cancellation logic
}

I think I would do this asynchronously with the SerialPort DataReceived event.
// Class fields
private const int RCV_HEADER_LENGTH = 8;
private const int MAX_DATA_LENGTH = 255;
private SerialPort Port;
private byte[] PacketBuffer = new byte[RCV_HEADER_LENGTH + MAX_DATA_LENGTH];
private int Readi = 0;
private int DataLength = 0;
// In your constructor
this.Port.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandler);
private void DataReceivedHandler(object sender, SerialDataReceivedEventArgs e)
{
if (e.EventType != SerialData.Chars)
{
return;
}
// Read all available bytes.
int len = Port.BytesToRead;
byte[] data = new byte[len];
Port.Read(data, 0, len);
// Go through each byte.
for (int i = 0; i < len; i++)
{
// Add the next byte to the packet buffer.
PacketBuffer[Readi++] = data[i];
// Check if we've received the complete header.
if (Readi == RCV_HEADER_LENGTH)
{
DataLength = PacketBuffer[1];
}
// Check if we've received the complete data.
if (Readi == RCV_HEADER_LENGTH + DataLength)
{
// The packet is complete add it to the appropriate buffer.
Readi = 0;
}
}
}

Do TCP sockets automatically close after 64kB send?

I'm trying to send multiple files over TCP using C# TcpClient, for a file below 64kB it works great, but when I have more it throws an exeption that host-computer disconnected.
Here is my code:
Server side(sending).
1) SocketServer class
public abstract class SocketServer
{
private Socket serverSocket;
public SocketServer(IPEndPoint localEndPoint)
{
serverSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
serverSocket.Bind(localEndPoint);
serverSocket.Listen(0);
serverSocket.BeginAccept(BeginAcceptCallback, null);
}
private void BeginAcceptCallback(IAsyncResult ar)
{
Socket clientSocket = serverSocket.EndAccept(ar);
Console.WriteLine("Client connected");
ClientConnection clientConnection = new ClientConnection(this, clientSocket);
Thread clientThread = new Thread(new ThreadStart(clientConnection.Process));
clientThread.Start();
serverSocket.BeginAccept(BeginAcceptCallback, null);
}
internal abstract void OnReceiveMessage(ClientConnection client, byte header, byte[] data);
}
Here goes ClientConnection:
public class ClientConnection
{
private SocketServer server;
private Socket clientSocket;
private byte[] buffer;
private readonly int BUFFER_SIZE = 8192;
public ClientConnection(SocketServer server, Socket clientSocket)
{
this.server = server;
this.clientSocket = clientSocket;
buffer = new byte[BUFFER_SIZE];
}
public void Process()
{
clientSocket.BeginReceive(buffer, 0, buffer.Length, SocketFlags.Peek, BeginReceiveCallback, null);
}
private void BeginReceiveCallback(IAsyncResult ar)
{
int bytesReceived = clientSocket.EndReceive(ar);
if (bytesReceived >= 4)
{
clientSocket.Receive(buffer, 0, 4, SocketFlags.None);
// message size
int size = BitConverter.ToInt32(buffer, 0);
// read message
int read = clientSocket.Receive(buffer, 0, size, SocketFlags.None);
// if data still fragmented, wait for it
while (read < size)
{
read += clientSocket.Receive(buffer, read, size - read, SocketFlags.None);
}
ProcessReceivedData(size);
}
clientSocket.BeginReceive(buffer, 0, buffer.Length, SocketFlags.Peek, BeginReceiveCallback, null);
}
private void ProcessReceivedData(int size)
{
using (PacketReader pr = new PacketReader(buffer))
{
// message header = 1 byte
byte header = pr.ReadByte();
// next message data
byte[] data = pr.ReadBytes(size - 1);
server.OnReceiveMessage(this, header, data);
}
}
public void Send(byte[] data)
{
// first of all, send message length
clientSocket.Send(BitConverter.GetBytes(data.Length), 0, 4, SocketFlags.None);
// and then message
clientSocket.Send(data, 0, data.Length, SocketFlags.None);
}
}
Implementation of FileServer class:
public class FileServer : SocketServer
{
string BinaryPath;
public FileServer(IPEndPoint localEndPoint) : base(localEndPoint)
{
BinaryPath = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
}
internal override void OnReceiveMessage(ClientConnection client, byte hdr, byte[] data)
{
// Convert header byte to ENUM
Headers header = (Headers)hdr;
switch(header)
{
case Headers.Queue:
Queue(client, data);
break;
default:
Console.WriteLine("Wrong header received {0}", header);
break;
}
}
private void Queue(ClientConnection client, byte[] data)
{
// this message contains fileName
string fileName = Encoding.ASCII.GetString(data, 1, data.Length - 1);
// combine path with assembly location
fileName = Path.Combine(BinaryPath, fileName);
if (File.Exists(fileName))
{
FileInfo fileInfo = new FileInfo(fileName);
long fileLength = fileInfo.Length;
// pass the message that is now start a file transfer, contains:
// 1 byte = header
// 16 bytes = file length
using (PacketWriter pw = new PacketWriter())
{
pw.Write((byte)Headers.Start);
pw.Write(fileLength);
client.Send(pw.GetBytes());
}
//
using (FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.Read))
{
int read = 0, offset = 0;
byte[] fileChunk = new byte[8191];
while (offset < fileLength)
{
// pass message with file chunks, conatins
// 1 byte = header
// 8195 bytes = file chunk
using (PacketWriter pw = new PacketWriter())
{
fs.Position = offset;
read = fs.Read(fileChunk, 0, fileChunk.Length);
pw.Write((byte)Headers.Chunk);
pw.Write(fileChunk, 0, read);
client.Send(pw.GetBytes());
}
offset += read;
}
}
}
}
}
And helper classes:
public class PacketWriter : BinaryWriter
{
private MemoryStream memoryStream;
public PacketWriter() : base()
{
memoryStream = new MemoryStream();
OutStream = memoryStream;
}
public byte[] GetBytes()
{
byte[] data = memoryStream.ToArray();
return data;
}
}
public class PacketReader : BinaryReader
{
public PacketReader(byte[] data) : base(new MemoryStream(data))
{
binaryFormatter = new BinaryFormatter();
}
}
Client have the almost same code. Except receive:
internal override void OnReceiveMessage(ServerConnection client, byte hdr, byte[] data)
{
Headers header = (Headers)hdr;
switch(header)
{
case Headers.Start:
Start(data); // save length of file and prepare for receiving it
break;
case Headers.Chunk:
Chunk(data);
break;
default:
Console.WriteLine("Wrong header received {0}", header);
break;
}
}
private void Chunk(byte[] data)
{
// Process reveived data, write it into a file
}

Do TCP sockets automatically close after 64kB send?
No, TCP sockets do not automatically close after 64 KB of sent data, nor after any amount of sent data. They remain open until either end closes the connection or a network error of some sort happens.
Unfortunately, your question is extremely vague, providing practically no context. But I will point out one potentially serious bug in your client code: you are not restricting the data read by the client to that which you expect for the current file. If the server is sending more than one file, it is entirely possible and likely that on reaching the end of the data for the current file, a read containing that final sequence of bytes will also contain the initial sequence of bytes for the next file.
Thus, the expression totalBytesRead == fileLenght will fail to ever be true and your code will attempt to read all of the data the server sends as if it's all part of the first file.
One easy way to check for this scenario is to look at the size of the file that was written by the client. It will be much larger than expected, if the above is the issue.
You can fix the code by only ever receiving at most the number of bytes you actually expect to be remaining:
while ((bytesRead = binaryReader.Read(
buffer, 0, Math.Min(fileLenght - totalBytesRead, buffer.Length))) > 0)
{
fs.Write(buffer, 0, bytesRead);
totalBytesRead += bytesRead;
if (totalBytesRead == fileLenght)
{
break;
}
}
While you're at it, you might fix the spelling of the variable named fileLenght. I know Intellisense makes it easy to type the variable name whether it's spelled right or not, but if you one day have to grep through the source code looking for code that involves itself with "length" values, you'll miss that variable.
EDIT:
Having pointed out the problem in the code you posted with your question originally, I now find myself looking at completely different code, after your recent edit to the question. The example is still not a good, minimal, complete code example and so advice will still have to be of a limited nature. But I can see at least two major problems in your receive handling:
You are mixing Receive() and BeginReceive(). This is probably not too terrible in general; what makes it really bad in your case is that in the completion handler for the initial asynchronous receive, you are blocking the thread while you use the synchronous Receive() to (attempt to) receive the rest of the data, and then to make matters worse, in the FileServer case you continue to block that thread while you transmit the file data to the client.I/O completion handlers must not block the thread for any significant amount of time; they must retrieve the data for the completed operation, and return as quickly as is feasible. There's a lot of wiggle room, but not enough to justify tying up the IOCP thread for the entire file transfer.
While the first issue is bad, it's probably not bad enough to actually cause the connection to reset. In a low-volume scenario, and especially if you're just dealing with one client at a time at the moment, I doubt blocking the IOCP thread that's being used to call your I/O completion callback will cause any easily observable problems.What's much worse is that you allocate the buffer array once, with a length of 8192, and then proceed to attempt to stuff the entire counted-byte stream of bytes being sent. On the assumption that the client will be sending short messages, this is probably fine on the server side. But on the client side, the value of size can easily be greater than the 8192 bytes allocated in the buffer array.I find it very likely that your BeginReceiveCallback() method is throwing an ArgumentOutOfRangeException after receiving the first 8192 bytes, due to the fact that you are passing the Receive() method a size value that exceeds the length of the buffer array. I can't prove that, because the code example is incomplete. You'll have to follow up on that theory yourself.
As long as I'm reviewing the code you posted, some other irregularities stood out as I scanned through it:
It's a bit odd to be using a buffer length of 8191 when transmitting the file.
Your BeginReceiveCallback() method discards whatever bytes were sent initially. You call EndReceive(), which returns the count of bytes received, but the code that follows assumes that the size value in the stream is still waiting to be read, when it very easily could have been sent in the first block of bytes. Indeed, your comparison bytesReceived >= 4 seems to be intended to refrain from processing received data until you have that length value. Which is wrong for another reason:
If your header is 1 byte followed by a 4-byte count-of-bytes value, then you really need bytesReceived > 4. But worse, TCP is allowed to drip out a single byte at a time if it really wants to (it won't, but if you write your code correctly, it won't matter). Since you only process received data if you get at least 4 bytes, and since if you don't get 4 bytes, you just ignore whatever was sent so far, you could theoretically wind up ignoring all of the data that was sent.
That you have the risk that any data could be ignored at all is terrible. But #2 above guarantees it, and #3 just adds insult to injury.
When receiving the size value, you are reading a 32-bit integer, but when sending the size value (i.e. for the file itself), you are writing a 64-bit integer. You'll get the right number on the receiving end, because BinaryWriter uses little-endian and that means that the first four bytes of any 64-bit integer less than 2^31 are identical to the four bytes of the 32-bit integer representation of the same number. But you'll still wind up with four extra zero bytes after that; if treated as file data as they likely would, that would corrupt your file.

stream socket reader receive 0 byte and app crash (windows phone 8)

How do you handle a null data (0 byte) that's coming from the reader? My app keeps force closing when it receives null data. Here is my code:
int i = 0;
private async void ReceiveData()
{
if (socket != null)
{
uint s = await dataReader.LoadAsync(1);
string data = dataReader.ReadString(s);
Debug.WriteLine(s);
if (data=="X")
{
i++;
}
if (data == null)
{
// what should I put here?
}
txtMessage.Text = i.ToString();
}
}
I'm having trouble when my app receives null data and suddenly it is forced to close. Any idea what code to put when my app receives a null data and waits for the next incoming data?
In this case I want the dataReader to keep reading the incoming data even if the data is null (no data received) and wait until it gets the next incoming data. How would you do that?
Thank you

To repeat an action, use a while loop.
However, if you read zero bytes from a socket stream, then that means the socket has been closed and you should stop reading.

Socket tcp c# how to clear input buffer?

I'm writing an application for windows phone and I need to communicate with a server and transmit data. The SERVER is written in C++ and I cannot modify it. The CLIENT is what I have to write. The Server is designed such that the client connect to it and transmit data. The connection remains open for all the transmission. By writing my code in C# I am able to receive data from the server but after the first receive, the data that I read in the buffer are alway the same. So I need a way to flush the input buffer so I can receive the new data (data are sent continuously). I'm using the class defined in here:
http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh202858%28v=vs.105%29.aspx
thanks a lot !!
I used this code for Receiving in the SocketClient.cs :
public string Receive()
{
string response = "Operation Timeout";
// We are receiving over an established socket connection
if (_socket != null)
{
// Create SocketAsyncEventArgs context object
SocketAsyncEventArgs socketEventArg = new SocketAsyncEventArgs();
socketEventArg.RemoteEndPoint = _socket.RemoteEndPoint;
// Setup the buffer to receive the data
socketEventArg.SetBuffer(new Byte[MAX_BUFFER_SIZE], 0, MAX_BUFFER_SIZE);
// Inline event handler for the Completed event.
// Note: This even handler was implemented inline in order to make
// this method self-contained.
socketEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(delegate(object s, SocketAsyncEventArgs e)
{
if (e.SocketError == SocketError.Success)
{
// *********************************************
// THIS part of the code was added to receive
// a vector of 3 double
Double[] OdomD = new Double[3];
for (int i = 0; i < 3; i++)
{
OdomD[i] = BitConverter.ToDouble(e.Buffer, 8 * i);
}
// *********************************************
}
else
{
response = e.SocketError.ToString();
}
_clientDone.Set();
});
// Sets the state of the event to nonsignaled, causing threads to block
_clientDone.Reset();
// Make an asynchronous Receive request over the socket
_socket.ReceiveAsync(socketEventArg);
// Block the UI thread for a maximum of TIMEOUT_MILLISECONDS milliseconds.
// If no response comes back within this time then proceed
_clientDone.WaitOne(TIMEOUT_MILLISECONDS);
}
else
{
response = "Socket is not initialized";
}
return response;
}
The Connect() method is exactly the same reported in the link above. So when the application start, the Connect() method is called as follow:
SocketClient client = new SocketClient();
// Attempt to connect to server for receiving data
Log(String.Format("Connecting to server '{0}' over port {1} (data) ...", txtRemoteHost.Text, 4444), true);
result = client.Connect(txtRemoteHost.Text, 4444);
Log(result, false);
That is done just once at the beginning, then I need receive this array of 3 double that is updated every second. So I use:
Log("Requesting Receive ...", true);
result = client.Receive();
Log(result, false);
The problem is that also if I debug the code and stop the execution inside Receive(), I always read the same value, that is the first value sent by the server. What I'm expecting is that every time I call client.Receive(), I get the new value, but this is not appening.
I had a similar problem by writing the same client in Matlab environment. I solved the problem by using the function flushinput(t) before to read the input buffer. In this way I was able to read always the last data sent by the server. I'm lookin for a function similar to that one ..
The size of the input buffer is fixed equal to the data that I'm expecting to receive, in that case is 24 bytes ( 3* sizeof(double) ) ..
Thanks a lot for you time !!

oleksii is right, you should call client.Receive() in a loop. You can choose to start a thread that covers the receive section of your code. Also note that client.Receive() will keep trying to receive from the buffer, and it will get stuck if there is no data available.

The main question was **how to clear the input buffer? ** or am I wrong?=!
Nevertheless; since you don't have a fixed buffer denoted as seen from you posted code and receive it via the SocketAsyncEventArgs, you could clear it with:
Array.Clear(e.Buffer, 0, e.Buffer.Length);

C# asynchronous socket Begin/EndReceive questions

I've implemented a async function in order to read large socket data, mostly files will be transferred. Code is based on this question:
public class StateObject
{
public Socket workSocket = null;
public const int BUFFER_SIZE = 1024;
public byte[] buffer = new byte[BUFFER_SIZE];
public StringBuilder sb = new StringBuilder();
}
public static void Read_Callback(IAsyncResult ar)
{
StateObject so = (StateObject) ar.AsyncState;
Socket s = so.workSocket;
int read = s.EndReceive(ar);
if (read > 0)
{
so.sb.Append(Encoding.ASCII.GetString(so.buffer, 0, read));
if (read == StateObject.BUFFER_SIZE)
{
s.BeginReceive(so.buffer, 0, StateObject.BUFFER_SIZE, 0,
new AyncCallback(Async_Send_Receive.Read_Callback), so);
return;
}
}
if (so.sb.Length > 0)
{
//All of the data has been read, so displays it to the console
string strContent;
strContent = so.sb.ToString();
Console.WriteLine(String.Format("Read {0} byte from socket" +
"data = {1} ", strContent.Length, strContent));
}
s.Close();
}
After all of the data has been captured in the stringbuilder, I decrypt it and convert the base64 string to a file using another method. There will be multiple socket receiving\sending files so every workersocket has an ID.
Now here's the case; I want some status on the receiving file. For example a progress bar showing how much is received and the total amount to be received. The Read_Callback method uses a recursive call in order to get all the data from the socket, which prevents me from getting the total amount to be received. Does anyone know how to get the total amount the socket has to receive? Or is my only option to send the size fixed to the listener before transferring the data?
As mentioned before, my stateobject class contains an ID for identifying the socket which the data is for. Upon first connection, the program stores the connection and received info in an arraylist.
In order to get the amount of bytes already read, I have to report the status to one of the initiated worker classes stored in the arraylist. If I want to do this during the recursive call, how will I perform this without losing performance when receiving data?
As last, which buffer size should be used? should this be the MTU of tcp which is 1500 bytes in most cases?

I would suggest sending the total file size as the first step in your communication, before sending along the actual file. For example your StateObject class could have an additional property that holds the total size of the thing your transmitting, then in a different thread (the UI thread) you can create a progress bar and report based of the total transmitted vs the total size of the file as a percentage or whatever you like. Just track how much data has been transmitted as it occurs and update your object appropriately as it does so.

In addition to CG's answer, I would also skip the stringbuilder and use something like:
FileStream fs = File.OpenRead(filePath);
Byte[] bytes = new Byte[fs.Length];
fs.Read(bytes, 0, Convert.ToInt32(fs.Length));
fs.Close();