Develop Reference

Awaiting data from Serial Port in C#

I have an application that receives data from a wireless radio using RS-232. These radios use an API for communicating with multiple clients. To use the radios I created a library for communicate with them that other software can utilize with minimal changes from a normal SerialPort connection. The library reads from a SerialPort object and inserts incoming data into different buffers depending on the radio it receives from. Each packet that is received contains a header indicating its length, source, etc.
I start by reading the header, which is fixed-length, from the port and parsing it. In the header, the length of the data is defined before the data payload itself, so once I know the length of the data, I then wait for that much data to be available, then read in that many bytes.
Example (the other elements from the header are omitted):
// Read header
byte[] header = new byte[RCV_HEADER_LENGTH];
this.Port.Read(header, 0, RCV_HEADER_LENGTH);
// Get length of data in packet
short dataLength = header[1];
byte[] payload = new byte[dataLength];
// Make sure all the payload of this packet is ready to read
while (this.Port.BytesToRead < dataLength) { }
this.Port.Read(payload, 0, dataLength);
Obviously the empty while port is bad. If for some reason the data never arrives the thread will lock. I haven't encountered this problem yet, but I'm looking for an elegant way to do this. My first thought is to add a short timer that starts just before the while-loop, and sets an abortRead flag when it elapses that would break the while loop, like this:
// Make sure all the payload of this packet is ready to read
abortRead = false;
readTimer.Start();
while (this.Port.BytesToRead < dataLength && !abortRead) {}
This code needs to handle a constant stream of incoming data as quickly as it can, so keeping overhead to a minimum is a concern, and am wondering if I am doing this properly.

You don't have to run this while loop, the method Read would either fill the buffer for you or would throw a TimeoutException if buffer wasn't filled within the SerialPort.ReadTimeout time (which you can adjust to your needs).
But some general remark - your while loop would cause intensive CPU work for nothing, in the few milliseconds it would take the data to arrive you would have thousends of this while loop iterations, you should've add some Thread.Sleep inside.

If you want to truly adress this problem, you need to run the code in the background. There are different options to do that; you can start a thread, you start a Task or you can use async await.
To fully cover all options, the answer would be endless. If you use threads or tasks with the default scheduler and your wait time is expected to be rather short, you can use SpinWait.SpinUntil instead of your while loop. This will perform better than your solution:
SpinWait.SpinUntil(() => this.Port.BytesToRead >= dataLength);
If you are free to use async await, I would recommend this solution, since you need only a few changes to your code. You can use Task.Delay and in the best case you pass a CancellationToken to be able to cancel your operation:
try {
while (this.Port.BytesToRead < dataLength) {
await Task.Delay(100, cancellationToken);
}
}
catch(OperationCancelledException) {
//Cancellation logic
}

I think I would do this asynchronously with the SerialPort DataReceived event.
// Class fields
private const int RCV_HEADER_LENGTH = 8;
private const int MAX_DATA_LENGTH = 255;
private SerialPort Port;
private byte[] PacketBuffer = new byte[RCV_HEADER_LENGTH + MAX_DATA_LENGTH];
private int Readi = 0;
private int DataLength = 0;
// In your constructor
this.Port.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandler);
private void DataReceivedHandler(object sender, SerialDataReceivedEventArgs e)
{
if (e.EventType != SerialData.Chars)
{
return;
}
// Read all available bytes.
int len = Port.BytesToRead;
byte[] data = new byte[len];
Port.Read(data, 0, len);
// Go through each byte.
for (int i = 0; i < len; i++)
{
// Add the next byte to the packet buffer.
PacketBuffer[Readi++] = data[i];
// Check if we've received the complete header.
if (Readi == RCV_HEADER_LENGTH)
{
DataLength = PacketBuffer[1];
}
// Check if we've received the complete data.
if (Readi == RCV_HEADER_LENGTH + DataLength)
{
// The packet is complete add it to the appropriate buffer.
Readi = 0;
}
}
}

C# Serial Port Conditional & Partial Reading

I am stumped. Searched myself blue in the face - no go.
I am trying to establish serial comms with a device that sends 2 different blocks of data (one after the other) every 1 second continuously. The first block starts with "PID" and the second block ends with "H18".
I only need to read once every 5 seconds.
My problem is two fold:
I have no idea/control when the read starts and often starts mid - block.
I have no control over the start and end cycle to ensure I get a full two blocks as I need both.
Both blocks are about 200 characters long in total, has no /r at the beginning and has /r/n in between various items.
I have tried doing two subsequent reads but no success. Tried playing with StartsWith and EndsWith but they are not recognized? The code has been all over the show, but here is the base I am working from currently:
static void DataReceivedHandlerbat(object sender, SerialDataReceivedEventArgs e)
{
var batm = sender as SerialPort;
if ((batm != null) && (!_gotResponse))
{
while (stringb.Length < 200)
{
byte[] buffer = new byte[batm.BytesToRead];
int numRead = batm.Read(buffer, 0, buffer.Length);
stringb.Append(System.Text.Encoding.ASCII.GetString(buffer));
// if (stringb.S == 0)
//{
// _gotResponse = true;
// break;
//}
}
}
}
and
/// Obtain Battery string
SerialPort batm = new SerialPort();
batm.PortName = "com4";
batm.BaudRate = 19200;
batm.DataBits = 8;
batm.Parity = Parity.None;
batm.StopBits = StopBits.One;
batm.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandlerbat);
batm.Open();
//batm.ReadExisting();
int timeoutMsb;
timeoutMsb = 1000;
var startTimeb = DateTime.Now;
while (!_gotResponse && ((DateTime.Now - startTimeb).TotalMilliseconds < timeoutMsb))
{
Thread.Sleep(20);
}
batm.Close();
_gotResponse = false;
//Build Battery String
String bat = stringb.ToString();
Please help me - I am fairly new to C# and have struggled for 4 days with this?

Is this a GUI application or a background service? Either way, ditch DataReceived event and use ReadAsync, as I showed in my blog post. Then, buffer all incoming data into a List<byte> (this easily deals with messages which arrive split into two), and implement some synchronization logic.
Here's an outline of how synchronization works:
loop through the List<byte>, find the beginning of a message
determine whether the entire message has been received
copy the message payload and fire off an event so the business logic that acts on the message can be separate from the buffering/parsing logic
remove bytes from the list up to the end of the detected message
search the remainder of the buffer for more valid messages (repeat 1-4)
when the buffer List<byte> doesn't contain a complete message, call ReadAsync again

TCP Socket fast packet sending

I have a class of the logged in the clients:
public class LoggedIn_Class
{
public Socket socket;
public string name;
}
And I have a list:
public static List<Object> LoggedIn_List = new List<Object>();
When someone login to my server I add it to the list:
LoggedIn_Class newClient = new LoggedIn_Class();
newClient.socket = currentSocket;
newClient.name = currentName;
LoggedIn_List.Add(newClient);
Now I'm trying to send a packet to every client with this:
foreach (LoggedIn_Class s in variables.LoggedIn_List.ToArray())
{
s.socket.Send(Packet);
}
Now the problem is the foreach processing too fast the items and it can't be send. This works properly, but I don't think this is the best solution...
foreach (LoggedIn_Class s in variables.LoggedIn_List.ToArray())
{
s.socket.Send(Packet);
Thread.Sleep(50);
}
Someone have any idea about this? What can I do to wait the end of the send?

I'm using TCP sockets. The client just not receive the packets, it don't have any exception.
There are plenty of answers about this. You real problem is that you have not understood what stream based protocols means. With TCP you do not send messages, you send streams of bytes.
As a result, the receiver side can not assume that everything received in one socket.Receive corresponds to exactly the same bytes as the other side sent using socket.Send.
You should therefore use some sort of mechanism to detect when a complete message have been received. Typically you either prefix each of your messages with a length prefix or use a delimiter after each message.
The reason to why it works when you use a Thread.Sleep is that the messages are short enough to be sent one at a time and you manage to receive them before the next message is sent. Without the delay, the nagle algorithm probably comes into play and merge your messages.
A naive implementation:
public void Send(byte[] msg)
{
var header = BitConverter.GetBytes(msg.Length);
socket.Send(header, 0, header.Length);
socket.Send(msg, 0, msg.Length);
}
Receiver side (probably faulty, but should give you a hint):
var buffer = new byte[4];
int bytesReceived = socket.Receive(buffer, 0, 4, ....);
if (bytesReceived < 4)
{
bytesReceived = socket.Receive(buffer, bytesReceived, 4 - bytesReceived, ....);
//this can fail too, probably should use a loop like below until
//all four length bytes have been received.
}
var bytesToRead = BitConverter.GetInt32(buffer, 0);
var buffer = new byte[bytesToRead];
var offset = 0;
while (bytesToRead > 0)
{
var len = Math.Min(bytesToRead, buffer.Length);
var read = socket.Receive(buffer, offset, len, .....);
bytesToRead -= read;
offset += read;
}
//now you got everything.

Lost a lot of udp packets on localhost

I use this code for receiving scanlines:
StateObject stateobj = (StateObject)ar.AsyncState;
stateobj.workSocket.BeginReceive(new System.AsyncCallback(VideoReceive), stateobj);
UdpClient client = stateobj.workSocket;
IPEndPoint ipendp = new IPEndPoint(IPAddress.Any, 0);
byte[] data = client.EndReceive(ar, ref ipendp);
BinaryReader inputStream = new BinaryReader(new MemoryStream(data));
inputStream.BaseStream.Position = 0;
int currentPart = inputStream.ReadInt32();
if (currentPart == part)
{
int a = 0;
int colum = inputStream.ReadInt32();
for (; a < packets.GetLength(1); a++)
{
packets[colum, a, 2] = inputStream.ReadByte();
packets[colum, a, 1] = inputStream.ReadByte();
packets[colum, a, 0] = inputStream.ReadByte();
}
receiverCheck++;
}
else if (currentPart != part)
{
part++;
mask2.Data = packets;
pictureBox1.BeginInvoke(new MethodInvoker(() => { pictureBox1.Image = mask2.ToBitmap(); }));
int colum = inputStream.ReadInt32();
for (int a = 0; a < packets.GetLength(1); a++)
{
packets[colum, a, 2] = inputStream.ReadByte();
packets[colum, a, 1] = inputStream.ReadByte();
packets[colum, a, 0] = inputStream.ReadByte();
}
}
After all scanlines have been received the image displayed in pictureBox.
This should work, but have a lot lost packets even on localhost (only ~ 95 of 480), so I have striped image. I found a similar problem here.
Answer:
private void OnReceive(object sender, SocketAsyncEventArgs e)
{
TOP:
if (e != null)
{
int length = e.BytesTransferred;
if (length > 0)
{
FireBytesReceivedFrom(Datagram, length, (IPEndPoint)e.RemoteEndPoint);
}
e.Dispose(); // could possibly reuse the args?
}
Socket s = Socket;
if (s != null && RemoteEndPoint != null)
{
e = new SocketAsyncEventArgs();
try
{
e.RemoteEndPoint = RemoteEndPoint;
e.SetBuffer(Datagram, 0, Datagram.Length); // don't allocate a new buffer every time
e.Completed += OnReceive;
// this uses the fast IO completion port stuff made available in .NET 3.5; it's supposedly better than the socket selector or the old Begin/End methods
if (!s.ReceiveFromAsync(e)) // returns synchronously if data is already there
goto TOP; // using GOTO to avoid overflowing the stack
}
catch (ObjectDisposedException)
{
// this is expected after a disconnect
e.Dispose();
Logger.Info("UDP Client Receive was disconnected.");
}
catch (Exception ex)
{
Logger.Error("Unexpected UDP Client Receive disconnect.", ex);
}
}
}
Answer has method FireBytesReceivedFrom(), but I can't find it. How can I use this code? And does this code help?

UDP doesn't guarantee that all packets will be received, it that they will arrive in any particular order. So even if you get this "working" be aware that it could (will probably) fail at some point.
When you call BeginReceive, you are starting an sync read. When data arrives, your event handler will be called, and it is then that you need to call EndReceive. Currently you are calling EndReceive immediately, which is probably why things are going wrong.
Some other notes:
I'd suggest that you don't try to be clever and re-use the buffer, as that could result in you losing data by overwriting data while you are trying to read it. Start off simple and add optimizations like this after you have it working well.
Also, the goto could be causing havoc. You seem to be trying to use it to retry, but this code is running IN the data received event handler. Event handlers should handle the event in the most lightweight way possible and then return, not start looping... especially as the loop here could cause a re-entrant call to the same event handler.
With async comms, you should start a read and exit. When you eventually receive the data (your event handler is called), grab it and start a new async read. Anything more complex than that is likely to cause problems.
The Fire... method you are missing probably just raises (fires) an event to tell clients that the data has arrived. this is the place where you should be grabbing the received data and doing something with it.
If you are using an example to build this code then I suggest you look for a better example. (in any case I would always recommend trying to find 3 examples so you can compare the implementations, as you will usually learn a lot more about something this way)

Serial Port Polling and Data handling

I am trying to read from several serial ports from sensors through microcontrollers. Each serial port will receive more than 2000 measurements (each measurement is 7 bytes, all in hex). And they are firing at the same time. Right now I am polling from 4 serial ports. Also, I translate each measurement into String and append it to a Stringbuilder. When I finish receiving data, they will be ouput in to a file. The problem is the CPU consumption is very high, ranging from 80% to 100%.
I went though some articles and put Thread.Sleep(100) at the end. It reduces CPU time when there is no data coming. I also put Thread.Sleep at the end of each polling when the BytesToRead is smaller than 100. It only helps to a certain extent.
Can someone suggest a solution to poll from serial port and handle data that I get? Maybe appending every time I get something causes the problem?
//I use separate threads for all sensors
private void SensorThread(SerialPort mySerialPort, int bytesPerMeasurement, TextBox textBox, StringBuilder data)
{
textBox.BeginInvoke(new MethodInvoker(delegate() { textBox.Text = ""; }));
int bytesRead;
int t;
Byte[] dataIn;
while (mySerialPort.IsOpen)
{
try
{
if (mySerialPort.BytesToRead != 0)
{
//trying to read a fix number of bytes
bytesRead = 0;
t = 0;
dataIn = new Byte[bytesPerMeasurement];
t = mySerialPort.Read(dataIn, 0, bytesPerMeasurement);
bytesRead += t;
while (bytesRead != bytesPerMeasurement)
{
t = mySerialPort.Read(dataIn, bytesRead, bytesPerMeasurement - bytesRead);
bytesRead += t;
}
//convert them into hex string
StringBuilder s = new StringBuilder();
foreach (Byte b in dataIn) { s.Append(b.ToString("X") + ","); }
var line = s.ToString();
var lineString = string.Format("{0} ---- {2}",
line,
mySerialPort.BytesToRead);
data.Append(lineString + "\r\n");//append a measurement to a huge Stringbuilder...Need a solution for this.
////use delegate to change UI thread...
textBox.BeginInvoke(new MethodInvoker(delegate() { textBox.Text = line; }));
if (mySerialPort.BytesToRead <= 100) { Thread.Sleep(100); }
}
else{Thread.Sleep(100);}
}
catch (Exception ex)
{
//MessageBox.Show(ex.ToString());
}
}
}

this is not a good way to do it, it far better to work on the DataReceived event.
basically with serial ports there's a 3 stage process that works well.
Receiving the Data from the serial port
Waiting till you have a relevant chunk of data
Interpreting the data
so something like
class DataCollector
{
private readonly Action<List<byte>> _processMeasurement;
private readonly string _port;
private SerialPort _serialPort;
private const int SizeOfMeasurement = 4;
List<byte> Data = new List<byte>();
public DataCollector(string port, Action<List<byte>> processMeasurement)
{
_processMeasurement = processMeasurement;
_serialPort = new SerialPort(port);
_serialPort.DataReceived +=SerialPortDataReceived;
}
private void SerialPortDataReceived(object sender, SerialDataReceivedEventArgs e)
{
while(_serialPort.BytesToRead > 0)
{
var count = _serialPort.BytesToRead;
var bytes = new byte[count];
_serialPort.Read(bytes, 0, count);
AddBytes(bytes);
}
}
private void AddBytes(byte[] bytes)
{
Data.AddRange(bytes);
while(Data.Count > SizeOfMeasurement)
{
var measurementData = Data.GetRange(0, SizeOfMeasurement);
Data.RemoveRange(0, SizeOfMeasurement);
if (_processMeasurement != null) _processMeasurement(measurementData);
}
}
}
Note: Add Bytes keeps collecting data till you have enough to count as a measurement, or if you get a burst of data, splits it up into seperate measurements.... so you can get 1 byte one time, 2 the next, and 1 more the next, and it will then take that an turn it into a measurement. Most of the time if your micro sends it in a burst, it will come in as one, but sometimes it will get split into 2.
then somewhere you can do
var collector = new DataCollector("COM1", ProcessMeasurement);
and
private void ProcessMeasurement(List<byte> bytes)
{
// this will get called for every measurement, so then
// put stuff into a text box.... or do whatever
}

First of all consider reading Using Stopwatches and Timers in .NET. You can break down any performance issue with this and tell exactly which part of Your code is causing the problem.
Use SerialPort.DataReceived Event to trigger data receiving process.
Separate receiving process and data manipulation process. Store Your data first then process.
Do not edit UI from reading loop.

I guess what you should be doing is adding an event handler to process incoming data:
mySerialPort.DataReceived += new SerialDataReceivedEventHandler(mySerialPort_DataReceived);
This eliminates the need to run a separate thread for each serial port you listen to. Also, each DataReceived handler will be called precisely when there is data available and will consume only as much CPU time as is necessary to process the data, then yield to the application/OS.
If that doesn't solve the CPU usage problem, it means you're doing too much processing. But unless you've got some very fast serial ports I can't imagine the code you've got there will pose a problem.