Develop Reference

C# Serial Port Conditional & Partial Reading

I am stumped. Searched myself blue in the face - no go.
I am trying to establish serial comms with a device that sends 2 different blocks of data (one after the other) every 1 second continuously. The first block starts with "PID" and the second block ends with "H18".
I only need to read once every 5 seconds.
My problem is two fold:
I have no idea/control when the read starts and often starts mid - block.
I have no control over the start and end cycle to ensure I get a full two blocks as I need both.
Both blocks are about 200 characters long in total, has no /r at the beginning and has /r/n in between various items.
I have tried doing two subsequent reads but no success. Tried playing with StartsWith and EndsWith but they are not recognized? The code has been all over the show, but here is the base I am working from currently:
static void DataReceivedHandlerbat(object sender, SerialDataReceivedEventArgs e)
{
var batm = sender as SerialPort;
if ((batm != null) && (!_gotResponse))
{
while (stringb.Length < 200)
{
byte[] buffer = new byte[batm.BytesToRead];
int numRead = batm.Read(buffer, 0, buffer.Length);
stringb.Append(System.Text.Encoding.ASCII.GetString(buffer));
// if (stringb.S == 0)
//{
// _gotResponse = true;
// break;
//}
}
}
}
and
/// Obtain Battery string
SerialPort batm = new SerialPort();
batm.PortName = "com4";
batm.BaudRate = 19200;
batm.DataBits = 8;
batm.Parity = Parity.None;
batm.StopBits = StopBits.One;
batm.DataReceived += new SerialDataReceivedEventHandler(DataReceivedHandlerbat);
batm.Open();
//batm.ReadExisting();
int timeoutMsb;
timeoutMsb = 1000;
var startTimeb = DateTime.Now;
while (!_gotResponse && ((DateTime.Now - startTimeb).TotalMilliseconds < timeoutMsb))
{
Thread.Sleep(20);
}
batm.Close();
_gotResponse = false;
//Build Battery String
String bat = stringb.ToString();
Please help me - I am fairly new to C# and have struggled for 4 days with this?

Is this a GUI application or a background service? Either way, ditch DataReceived event and use ReadAsync, as I showed in my blog post. Then, buffer all incoming data into a List<byte> (this easily deals with messages which arrive split into two), and implement some synchronization logic.
Here's an outline of how synchronization works:
loop through the List<byte>, find the beginning of a message
determine whether the entire message has been received
copy the message payload and fire off an event so the business logic that acts on the message can be separate from the buffering/parsing logic
remove bytes from the list up to the end of the detected message
search the remainder of the buffer for more valid messages (repeat 1-4)
when the buffer List<byte> doesn't contain a complete message, call ReadAsync again

c# detecting disconnect from server with timer

I have a client type application that is receiving packets from remote server.
Now and then it so happens that for some reason server disconnects me.
Sometimes there are some problems on my end ISP drops internet etc.
I have been trying to catch those exceptions and goog-ling for an answer but at the end every one
points to "make a timer and check periodically for received packets."
Now i have a function that is receiving all incoming traffic.
This function is executed every time packet is received.
Now my idea is to create a function that will create timer with let say 50 seconds time out.
This function will reset timer to 0 each time packet is received and restart it.
If timer reach 50 seconds it will throw an error "disconnected!" and some logic will follow
how to reconnect.
Now main problem i have is ... i can not "pause" my main packet receiving function.
I have tried to make it in another thread but program keep recreating new threads , killing threads by ID is a bad practice and i haven't gone down that road ... yet.
Is this a how i should handle my problem or someone has a better idea?
Below is my packet receive function.
public void OnReceive()
{
try
{
recv_pack = locSec.TrIncom();
if (recv_pack != null)
{
foreach (Packet packet in recv_pack)
{
byte[] packet_bytes = packet.GetBytes();
PacketHandler.HandlePacket(packet, locSec);
//here i would check for packet receive with timer
//CheckDisconnect();
}
}
}
catch()
{}
}
So far i have come up with this:
public bool CheckDisconnect()
{
bool KeepGoing = true;
for(int i = 0 ; i <= 50 && KeepGoing; i++ )
{
Thead.Sleep(1000);
if(i == 50)
{
KeepGoing = false;
Console.WriteLine("Disconnected!");
// ... starting reconnect procedure
}
}
}

Not sure if i understand completely, but if those two functions are in the same thread, can't you just make a global variable that controls the OnReceive() function and set it to false in your CheckDisconnect() function?

Voice Chat, Increased Delay(Buffer) - Can it be solved?

Okay so i am making a Voice chat software.
I am using NAudio for it, an excellent library.
But i got a problem. The buffer can go up when something happens. I guess it´s from example, when the OS loads something and the Voice Chat application is put on "hold" for a sec. During that time, it adds the data in the buffer, making the current data get delayed.
And as the receiver is playing at the same pace all the time, it will always be delayed.
Now i have a "solution" for this, which is to clear the buffer when it reaches a certain length. Though this is not ideal at all, and is more of a trick than a solution.
Now to the code parts.
First i initialize the things i use.
private NAudio.Wave.WaveInEvent SendStream = new WaveInEvent();
private NAudio.Wave.AsioOut Aut;
private NAudio.Wave.WaveFormat waveformat = new WaveFormat(48000, 16, 2);
private WasapiLoopbackCapture Waloop = new WasapiLoopbackCapture();
private NAudio.Wave.BufferedWaveProvider waveProvider;
waveProvider = new NAudio.Wave.BufferedWaveProvider(waveformat);
waveProvider.DiscardOnBufferOverflow = true;
SendStream.WaveFormat = waveformat;
waveformat is used just so i don´t have to rewrite it all the time.
DiscardOnBufferOverflow is used so if i set a certain lenght on the buffer, for example 20ms. It will Discard anything above, else it will return an exception. I think however it doesn´t do anything if i don´t set a length, it´s probably infinite at default.
And not much else, SendStream is a WaveInEvent, meaning it will run on a BackgroundThread when i use DataAvailable. Waloop is pretty much the same except it´s a loopback.
waveprovider is used in the receiving part to play back the audio.
Waveformat is, well waveformat, it´s importat to set it out, and have all the same, at least in my application.
Here is the receiving part. As you can se, it puts the data in a byte array, then plays it. nothing weird.
byte[] byteData = udpClient.Receive(ref remoteEP);
waveProvider.AddSamples(byteData, 0, byteData.Length);
Here is the sending/recording part.
private void Sendv2()
{
try
{
if (connect == true)
{
if (AudioDevice == "Wasapi Loopback")
{
SendStream.StopRecording();
Waloop.StartRecording();
}
else
{
Waloop.StopRecording();
SendStream.StartRecording();
}
}
}
catch (Exception e)
{
MessageBox.Show(e.Message);
}
}
void Sending(object sender, NAudio.Wave.WaveInEventArgs e)
{
if (connect == true && MuteMic.Checked == false)
{
udpClient.Send(e.Buffer, e.BytesRecorded, otherPartyIP.Address.ToString(), 1500);
}
}
void SendWaloop(object sender, NAudio.Wave.WaveInEventArgs e)
{
byte[] newArray16Bit = new byte[e.BytesRecorded / 2];
short two;
float value;
for (int i = 0, j = 0; i < e.BytesRecorded; i += 4, j += 2)
{
value = (BitConverter.ToSingle(e.Buffer, i));
two = (short)(value * short.MaxValue);
newArray16Bit[j] = (byte)(two & 0xFF);
newArray16Bit[j + 1] = (byte)((two >> 8) & 0xFF);
}
if (connect == true && MuteMic.Checked == false)
{
udpClient.Send(newArray16Bit, newArray16Bit.Length, otherPartyIP.Address.ToString(), 1500);
}
}
Waloop is a Loopback, so it goes through another "channel", but it´s not really important here.
Very simple, When data is available (when it´s recording) and if the connect is true etc, it will just send the buffer.
So pretty much like the receiver part but other way around.
Now how i currently solve this is like this:
if (waveProvider.BufferedDuration.Milliseconds > 40)
{
waveProvider.ClearBuffer();
TimesBufferClear++;
}
So i am clearing the buffer if it´s above 40ms (this is in a Timer at 600ms interval).
(TimesBufferClear++; is just so i can keep track of the times it had been cleared)
Now sadly, i have no idea how to prevent the buffer to be increased, and setting it to a forced state (20ms etc) will just cause the playback to be worse and worse the higher up the buffer goes as it doesn´t really stop, it just ignores the part above i think.
Here is the creationg of the input devices. It is a bit different from ASIO and Wasapi in my implementation, but it pretty much works the same, only real difference is that i tell the UI that ASIO is on or off as you can see in the code, at the end i add the DataAvailable event´s to both SendStream (any input, Microphone etc) and Waloop (Loopback sound that´s being played).
private void CheckAsio()
{
if (NAudio.Wave.AsioOut.isSupported())
{
Aut = new NAudio.Wave.AsioOut();
ASIO.Text += "\nSupported: " + Aut.DriverName;
ASIO.ForeColor = System.Drawing.Color.Green;
Aut.Init(waveProvider);
Aut.Play();
SendStream.NumberOfBuffers = 2;
SendStream.BufferMilliseconds = 10;
}
else
{
AsioSettings.Enabled = false;
ASIO.Text += "\n Not Supported: Wasapi used";
ASIO.ForeColor = System.Drawing.Color.DarkGray;
Wasout = new WasapiOut(AudioClientShareMode.Shared, 0);
Wasout.Init(waveProvider);
Wasout.Play();
SendStream.NumberOfBuffers = 2;
SendStream.BufferMilliseconds = 9;
}
SendStream.DataAvailable += Sending;
Waloop.DataAvailable += SendWaloop;
}
I am not sure if this even can be solved. But as i don´t see other voice chat programs have it, i am guessing there must be something that can be done.

The way this appears to be handled in most applications is to send blocks of data a defined rate (in samples/sec), and drop blocks of data that exceed that rate. If the sender is resource-limited and is not able to maintain the rate, the stream will have audio gaps. This used to happen in audio calls over dial-up when the transmission rate was locked higher than the network connection could handle, or when the CODEC code was taking up too much time.
But from the sound of things the buffering and skips are symptoms, not causes. The root of the problem is that your process is getting shelved for other operations. You can address this by running at a higher process and/or thread priority. The higher your priority the less interruptions you'll have, which will reduce the likelihood of data queuing up to be processed.
In .NET you can raise your process and/or thread priority fairly simply. For process priority:
using System.Diagnostics;
...
Process.GetCurrentProcess().PriorityClass = PriorityClass.Highest;
Or for a thread:
using System.Threading;
...
Thread.CurrentThread.Priority = ThreadPriority.Highest;
This is not a complete solution, since the OS will still steal time-slices from your application under various circumstances, but in a multi-CPU/core system with plenty of memory you should get a fairly good shot at a stable recording environment.
Of course there are no fool-proof methods, and there's always that one slow computer that will mess you up, so you should allow the system to drop excess samples when necessary. Keep track of how much data you're sending out and when it starts to back up, drop anything over your maximum samples/sec. That way your server (or client) isn't going to be buffering increasing amounts of data and lagging further and further behind real-time.
One option there is to time-stamp each packet you send so that the client can choose when to start dropping data to catch up. Better to lose a few milliseconds of output here and there than to drift further and further out of sync.

Serial Port Polling and Data handling

I am trying to read from several serial ports from sensors through microcontrollers. Each serial port will receive more than 2000 measurements (each measurement is 7 bytes, all in hex). And they are firing at the same time. Right now I am polling from 4 serial ports. Also, I translate each measurement into String and append it to a Stringbuilder. When I finish receiving data, they will be ouput in to a file. The problem is the CPU consumption is very high, ranging from 80% to 100%.
I went though some articles and put Thread.Sleep(100) at the end. It reduces CPU time when there is no data coming. I also put Thread.Sleep at the end of each polling when the BytesToRead is smaller than 100. It only helps to a certain extent.
Can someone suggest a solution to poll from serial port and handle data that I get? Maybe appending every time I get something causes the problem?
//I use separate threads for all sensors
private void SensorThread(SerialPort mySerialPort, int bytesPerMeasurement, TextBox textBox, StringBuilder data)
{
textBox.BeginInvoke(new MethodInvoker(delegate() { textBox.Text = ""; }));
int bytesRead;
int t;
Byte[] dataIn;
while (mySerialPort.IsOpen)
{
try
{
if (mySerialPort.BytesToRead != 0)
{
//trying to read a fix number of bytes
bytesRead = 0;
t = 0;
dataIn = new Byte[bytesPerMeasurement];
t = mySerialPort.Read(dataIn, 0, bytesPerMeasurement);
bytesRead += t;
while (bytesRead != bytesPerMeasurement)
{
t = mySerialPort.Read(dataIn, bytesRead, bytesPerMeasurement - bytesRead);
bytesRead += t;
}
//convert them into hex string
StringBuilder s = new StringBuilder();
foreach (Byte b in dataIn) { s.Append(b.ToString("X") + ","); }
var line = s.ToString();
var lineString = string.Format("{0} ---- {2}",
line,
mySerialPort.BytesToRead);
data.Append(lineString + "\r\n");//append a measurement to a huge Stringbuilder...Need a solution for this.
////use delegate to change UI thread...
textBox.BeginInvoke(new MethodInvoker(delegate() { textBox.Text = line; }));
if (mySerialPort.BytesToRead <= 100) { Thread.Sleep(100); }
}
else{Thread.Sleep(100);}
}
catch (Exception ex)
{
//MessageBox.Show(ex.ToString());
}
}
}

this is not a good way to do it, it far better to work on the DataReceived event.
basically with serial ports there's a 3 stage process that works well.
Receiving the Data from the serial port
Waiting till you have a relevant chunk of data
Interpreting the data
so something like
class DataCollector
{
private readonly Action<List<byte>> _processMeasurement;
private readonly string _port;
private SerialPort _serialPort;
private const int SizeOfMeasurement = 4;
List<byte> Data = new List<byte>();
public DataCollector(string port, Action<List<byte>> processMeasurement)
{
_processMeasurement = processMeasurement;
_serialPort = new SerialPort(port);
_serialPort.DataReceived +=SerialPortDataReceived;
}
private void SerialPortDataReceived(object sender, SerialDataReceivedEventArgs e)
{
while(_serialPort.BytesToRead > 0)
{
var count = _serialPort.BytesToRead;
var bytes = new byte[count];
_serialPort.Read(bytes, 0, count);
AddBytes(bytes);
}
}
private void AddBytes(byte[] bytes)
{
Data.AddRange(bytes);
while(Data.Count > SizeOfMeasurement)
{
var measurementData = Data.GetRange(0, SizeOfMeasurement);
Data.RemoveRange(0, SizeOfMeasurement);
if (_processMeasurement != null) _processMeasurement(measurementData);
}
}
}
Note: Add Bytes keeps collecting data till you have enough to count as a measurement, or if you get a burst of data, splits it up into seperate measurements.... so you can get 1 byte one time, 2 the next, and 1 more the next, and it will then take that an turn it into a measurement. Most of the time if your micro sends it in a burst, it will come in as one, but sometimes it will get split into 2.
then somewhere you can do
var collector = new DataCollector("COM1", ProcessMeasurement);
and
private void ProcessMeasurement(List<byte> bytes)
{
// this will get called for every measurement, so then
// put stuff into a text box.... or do whatever
}

First of all consider reading Using Stopwatches and Timers in .NET. You can break down any performance issue with this and tell exactly which part of Your code is causing the problem.
Use SerialPort.DataReceived Event to trigger data receiving process.
Separate receiving process and data manipulation process. Store Your data first then process.
Do not edit UI from reading loop.

I guess what you should be doing is adding an event handler to process incoming data:
mySerialPort.DataReceived += new SerialDataReceivedEventHandler(mySerialPort_DataReceived);
This eliminates the need to run a separate thread for each serial port you listen to. Also, each DataReceived handler will be called precisely when there is data available and will consume only as much CPU time as is necessary to process the data, then yield to the application/OS.
If that doesn't solve the CPU usage problem, it means you're doing too much processing. But unless you've got some very fast serial ports I can't imagine the code you've got there will pose a problem.

Serial Port; Missing Packets after few hours

I am new to Visual C#. I have to receive a packet of 468 bytes every second from a embedded device serially. The header of the packet is 0xbf, 0x13, 0x97, 0x74. After check validating the packet header i am saving this packet , process it, and display it graphically.
The problem is that i start losing packets after few hours. (Other software was logging the same data for the whole week and is working well).
The code is here...
private void DataRec(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
{
rtTotBytes = comport.BytesToRead;
rtTotBytesRead = comport.Read(rtSerBuff, 0, rtTotBytes);
this.Invoke(new ComportDelegate(ComportDlgtCallback), rtSerBuff, rtTotBytesRead);
}
//Delegate
delegate void ComportDelegate(byte[] sBuff, int sByte);
//Callback Function to Delegate
private void ComportDlgtCallback(byte[] SerBuff, int TotBytes)
{
for (int k = 0; k < TotBytes; k++)
{
switch (rtState)
{
case 0:
if (SerBuff[k] == 0xbf) { rtState = 1; TempBuff[0] = 0xbf; }
else rtState = 0;
break;
case 1:
if (SerBuff[k] == 0x13) { rtState = 2; TempBuff[1] = 0x13; }
else rtState = 0;
break;
case 2:
if (SerBuff[k] == 0x97) { rtState = 3; TempBuff[2] = 0x97; }
else rtState = 0;
break;
case 3:
if (SerBuff[k] == 0x74) { rtState = 4; TempBuff[3] = 0x74; rtCnt = 4; }
else rtState = 0;
break;
case 4:
if (rtCnt == 467)
{
TempBuff[rtCnt] = SerBuff[k];
TempBuff.CopyTo(PlotBuff, 0);
ProcessPacket(PlotBuff);
rtState = 0; rtCnt = 0;
}
else
TempBuff[rtCnt++] = SerBuff[k];
break;
}
}
}
Another question: can the BytesToRead be zero if a DataReceivedEvent had occured? Do you have to check (BytesToRead>0) in DataRecievedEvent?

Serial port input data must be treated as a stream, and not series of packets. For example, when device sends 0xbf, 0x13, 0x97, 0x74 packet, DataRec function may be called once with the whole packet, or twice with 0xbf, 0x13 and 0x97, 0x74 packets, or 4 times with one byte, etc. The program must be flexible enough to handle input stream using some parser. Your current program doesn't do this, it can miss logical packets which are received in a several function calls. Another situation is possible, when several packets are received in one DataRec function call - your program is not ready also for such situation.
Edit.
Typical serial port input stream handling algorithm should look like this:
DataRec function adds received data to input queue and calls parser.
Input queue is some byte array, which contains the data already received, but not parsed yet. New data is added to the end, and parsed packets are removed from the beginning of this queue.
Parser reads the input queue, handles all recognized packets and removes them from the queue, leaving all unrecognized data for the next call.

I think a problem could be that you can't be sure that you receive a full package within the DataReceived event. It is possible that you just got the first half of the packet and half a second later the second half.
So you should implement another layer where you put the data into a buffer. The further proceeding depends on the data format.
If you receive additionally informations like an end mark or the length of the data you could check if the buffer already contains these informations. If yes advance this full package to your routine.
If you don't have this information you have to wait till you receive the next header and forward the data within your buffer till this new header.

Have you checked the memory usage of the program?
Maybe you have a small interop class, memory or something which is not properly freed, adds up after a few hours and make the program run sluggish, causing it to lose data.
I'd use process explorer to check how memory and cpu use change after a few hours. Maybe check for hdd activity, too.
If this does not lead to results, use a full blown profiler like ANTS and try to run the program under the profiler to check for problems.

As Alex Farber points out, there's no guarantee that when your DataReceived handler is invoked, all the bytes are there.
If your buffers are always a fixed size, and at a low rate, you can use the Read function directly, rather than relying on the DataReceived event. Conceptually:
packetSize = 468;
...initialization...
comport.ReadTimeout = 2000; //packets expected every 1000 milliseconds, so give it some slack
while (captureFlag) {
comport.Read(rtSerBuff, 0, packetSize);
...do stuff...
}
This can be put into its own worker thread if you want.
Another approach would be to use the ReadLine method. You mention that the packets have a known starting signature. Do they also have a known ending signature that is guaranteed to not be repeated in the packet? If so, you can set the NewLine property to this ending signature and use ReadLine. Again, you can put this in a worker thread,