I've got a thread to read and parse serial data.
The messages are in binary format and start with either the character 'F', 'S', 'Q' or 'M'.
There are no newlines and there is no special ending character (the characters above state that a message is finished and everything before it is ready to be parsed).
How do I continuously read and parse the data?
All that comes to my mind is having a 4096 byte long input buffer (byte array) and then follow this procedure:
Track the position in the buffer manually
append available data to it via SerialPort.Read(buffer, position, byteCount)
try to parse as many messages as possible from the buffer
copy the rest to a temporary buffer
reset the input buffer
copy the contents of the temporary buffer to the original buffer
set the position in the buffer
Can you think of faster / easier approaches?
A very simple way to get ahead is to stop trying to make it faster. There is no point, serial port data rates are very, very low and modern computers are very, very fast. Your Read() call only ever returns a single byte, rarely 2.
Note that this is hard to see, when you debug and single-step through the code then you'll artificially slow down your program a great deal. Allowing more bytes to be received and thus more of them getting returned by the Read() call. But this doesn't happen when the program runs at normal speed.
So use SerialPort.BaseStream.ReadByte() instead. Makes the code very simple.
After acquiring some experience with SerialPort C# component
At the beginning: Take a serial port exclusevily.
Then:
1st parallel Task: Continues read entire buffer content after a regular
interval and pushes the read chunk into a "Gathering Collection" of data.
2nd parallel Task: Analyzes the "Gathering Collection" for a completed "phrase", delegates the clonned "phrase" to a "Phrase Manager" and excludes the phrase from the "Gathering Collection"
You have a freedom about "Gathering Collection" implementation, but what was important to me is that:
read all, but not a sized content from the Serial port buffer
to avoid losses and save an order in messages build your own port dispatcher rather let anybody open and close your port at any time for reading/writing.
detect the port read frequency experimenally. The more frequent read-operation will let your code detect fatser a "phrase" and start the proder handlind. Too frequent reading without detecting a "phrase" can cost you additional resource usage.
Related
I am researching the possibility of using pipelines for processing binary messages coming from network.
The binary messages i will be processing come with an payload and it is desirable to keep the payload in its binary form.
The idea is to read out the whole message and create a slice of message and its payload, once the message is completely read it will be passed to a channel chain for processing, the processing will not be instant and might take some time or be executed later and the goal is not to have the pipe reader wait until the processing is complete, then once the message processing is complete i would need to release the processed buffer region to the pipe writer.
Now of course i could just create a new byte array and copy the data coming from pipe writer but that would beat the purpose of no-copy? So as i understand i would need some buffer synchronization between the pipeline and the channel?
I observed the available apis (AdvanceTo) of pipe reader where its possible to tell the pipe reader what was consumed and what was examined but cant get around how this could be synced outside of the pipe reading method.
So the question would be whether there are some techniques or examples on how this can be achieved.
The buffer obtained from TryRead/ReadAsync is only valid until you call AdvanceTo, with the expectation that as soon as you've done that: anything you reported as consumed is available to be recycled for use elsewhere (which could be parallel/concurrent readers). Strictly speaking: even the bits you haven't reported as consumed: you still shouldn't treat as valid once you've called AdvanceTo (although in reality, it is likely that they'll still be the same segments - just: that isn't the concern of the caller; to the caller, it is only valid between the read and the advance).
This means that you explicitly can't do:
while (...)
{
var result = await pipe.ReadAsync();
if (TryIdentifyFrameBoundary(out var frame)) {
BeginProcessingInBackground(frame); // <==== THIS IS A PROBLEM!
reader.AdvanceTo(frame.End, frame.End);
}
else if { // take nothing
reader.AdvanceTo(buffer.Start, buffer.End);
if (result.IsCompleted) break; // that's all folks
}
}
because the "in background" bit, when it fires, could now be reading someone else's data (due to it being reused already).
So: either you need to process the frame contents as part of the read loop, or you're going to have to make a copy of the data, most likely by using:
c#
var len = checked ((int)buffer.Length);
var oversized = ArrayPool<byte>.Shared.Rent(len);
buffer.CopyTo(oversized);
and pass oversized to your background processing, remembering to only look at the first len bytes of it. You could pass this as a ReadOnlyMemory<byte>, but you need to consider that you're also going to want to return it to the array-pool afterwards (probably in a finally block), and passing it as a memory makes it a little more awkward (but not impossible, thanks to MemoryMarshal.TryGetArray).
Note: in early versions of the pipelines API, there was an element of reference-counting, which did allow you to preserve buffers, but it had a few problems:
it complicated the API hugely
it led to leaked buffers
it was ambiguous and confusing what "preserved" meant; is the count until it gets reused? or released completely?
so that feature was dropped.
What is the behaviour of the NetworkStream.Write() method, if I send data via a TcpClient's NetworkStream, and the TcpClient.SendBufferSize is smaller than the data?
The MSDN documentation for SendBufferSize says
If the network buffer is smaller than the amount of data you provide
the Write method, several network send operations will be performed
for every call you make to the Write method. You can achieve greater
data throughput by ensuring that your network buffer is at least as
large as your application buffer.
So I know that the data will be sent in multiple operations, and the receiving TCP-Stack should reassemble it into one continuous stream transparently.
But what happens exactly in my program during this time?
If there is enough space in the SendBuffer, TcpClient.GetStream().Write() will not block at all, and return immediately and so will NetworkStream.Flush().
If I set the TcpClient.SendBufferSize to a value smaller than the data, will the Write() block until
either the first part of the data has been received and ACKed,
or the TcpClient.SendTimeout has expired?
Or does it work in some other way? Does it actually wait for a TCP ACK?
Are there any other drawbacks besides higher overhead to such a smaller buffer size? Are there problems with changing the SendBufferSize on the fly?
Example:
byte[] data = new byte[20] // 20 byte data
tcpClient.SendBufferSize= 10; // 10 byte buffer
tcpClient.SendTimeout = 1000; // 1s timeout
tcpClient.GetStream().Write(data,0,20);
// will this block until the first 10 bytes have been full transmitted?
// does it block until a TCP ACK has been received? or something else?
// will it throw if the first 10 bytes have not been received in 1 second?
tcpClient.GetStream().Flush(); // would this make any difference?
My goal here is mainly getting a better understanding of the network internals.
Also , I was wondering if this could be abused to react more quickly to a network failure. If data is sent only infrequently, and each data packet is small enough to be transmitted at once, and there are no receipt messages in a given message protocol, it could take a long time after a network error until the next Write() is called; so a long time until an exception is thrown.
If the SentBuffer is very small, would an error be noticed more quickly, unless it happened at end of the data?
Could I abuse this to measure the time it takes for a single packet to be transmitted and ACKed?
I am new to serial communication. I have read a fair few tutorials, and most of what I am trying to do is working, however I have a question regarding serial communication with C#. I have a micro controller that is constantly sending data through a serial line. The data ist in this format:
bxxxxixx.xx,xx.xx*
where the x's represent different numbers, + or - signs.
At certain times want to read this information from my C# program on my PC. The problem that I am having is that my messages seem to be split in random positions even though I am using
ReadTo("*");
I assumed this would read everything upto the * character.
How can I make sure that the message I recieved is complete?
Thank you for your help.
public string receiveCommandHC()
{
string messageHC = "";
if (serialHC.IsOpen)
{
serialHC.DiscardInBuffer();
messageHC = serialHC.ReadTo("*");
}
return messageHC;
}
I'm not sure why you're doing it, but you're discarding everything in the serial ports in-buffer just before reading, so if the computer has already received "bxxx" at that point, you throw it away and you'll only be reading "xixx.xx,xx.xx".
You'll nearly always find in serial comms that data messages (unless very small) are split. This is mostly down to the speed of communication and the point at which you retrieve data from the port.
Usually you'd set your code to run in a separate thread (to help prevent impacting the performance of the rest of your code) which raises an event when a complete message is received and also takes full messages in for transmission. Read and write functionality is dealt with by worker threads (serial comms traffic is slow).
You'll need a read and a write buffer. These should be suitabley large to hold data for several cycles.
Append data read from the input to the end of your read buffer. Have the read buffer read on cyclicly for complete messages, from the start of the buffer.
Depending on the protocol used there is usually a data start and maybe a data end indicator and somewhere a message size (this may be fixed, again depending on your protocol). I gather form your protocol that the message start character is 'b' and the message end character is '*'. Discard all data preceeding your message start character ('b'), as this is from an incomplete message.
When a complete message is found, strip it from the front of the buffer and raise an event to indicate its arrival.
A similar process is run for sending data, except that you may need to split the message, hence data to be sent is appended to the end of the buffer and data being sent is read from the start.
I hope that this helps you in understanding how to cope with serial comms.
As pointed out by Marc you're currently clearing your buffer in a way that will cause problems.
edit
As I said in my comment I don't recognise serialHC, but if dealing with raw data then look at using the SerialPort class. More information on how to use it and an example (which roughly uses the process that I described above) can be found here.
I'm going to take a guess that you're actually getting the ends of commands, i.e. instead of getting b01234.56.78.9 (omitting the final *), you're getting (say) .56.78.9. That is because you discarded the input buffer. Don't do that. You can't know the state at that point (just before a read), so discarding the buffer is simply wrong. Remove the serialHC.DiscardInBuffer(); line.
If the SerialPort.Write() is a blocking operation( or is it not?), what would be the need for the BytesToWrite() method. It would always evaluate to zero, cause the last Write operation either succeeded in writing all data or failed, in either case the bytes to be written would be come zero.
Perhaps, there is more to it then what I have described.
SerialPort.Write is a blocking operation, yes. However, there are two buffers to be considered: The serial device, and the SerialPort buffers.
If the SerialPort object is configured to buffer, the write only blocks if there isn't enough room in that buffer. It will block for as long as it takes the buffer to empty enough to fit the new data. Otherwise it fills the buffer and returns.
If the SerialPort object does not buffer, the Write operation blocks only for as long as it takes to transfer the data to the serial device. That device has its own buffer(*), so the block may take far less time than the time it will take to send the data.
SerialPort.BytesToWrite includes both data in the device's buffer and data in the SerialPort object's buffer.
(*) Older UARTs did not have buffers and newer ones do but can be configured to not buffer.
Serial ports are very slow I/O devices that date from the stone age of computing. At a common baudrate of 9600 bits per second, it can only transmit a bit less than thousand bytes per second. Compare to a hard disk, a burst speed of 60,000,000 bytes per second is possible. Or more comparable, a network card can transmit 125,000,000 bytes per second.
So the serial port driver keeps a buffer that stores the bytes you write and slowly empties it while they are written by the UART chip. That buffer allows the Write() call to quickly return. Given that it is so slow, you might want to be aware of how full that buffer is so that you can avoid blocking on the Write() call. WriteBufferSize - BytesToWrite tells you how much space is available in that buffer.
The SerialPort property BytesToWrite Gets the number of bytes of data in the send buffer.
On the other hand the SerialPort method Write(string text) Writes the specified string to the serial port.
You do know how a serial port works right? A serial sends a certain amount of bytes every second depending on the baud used.
I am using naudio to generate pulse width modulated audio signals for controlling a pair of servo's. Currently I am using the WaveProvider32 class that Mark Heath wrote (http://mark-dot-net.blogspot.com/2009/10/playback-of-sine-wave-in-naudio.html) which implements the IWaveProvider interface. The sample rate is 44100
The audio signal is basically a block N wide where the first part of the signal all the values are high, and for the remainder of the block the values are low. Since the Read operation asks for more samples than the width of the block I just repeat the signal until I fill up the buffer. The problem I have is that the length of the buffer is not a multiple of the width of my signal block, so part of the last block is cut off which screws with the servo and makes it twitch. I realize I could do some code fanciness to keep track of it and offset the beginning of the next read, but I would be easier if I could set the number of values that WaveProvider had to provide at once so that I could make it a multiple (or maybe the exact width) of the signal block size.
Is that possible?
The amount of data requested by the Read function is determined by the IWaveOut implementation you choose, and the latency and number of buffers it is operating at. You would need to create an intermediate IWaveProvider that ensures that Read methods to the underlying provider always ask for the right number. Have a look at the BlockAlignReductionStream which I created for a similar issue.