I want to plot all the data points, I get from the TCP server. But I could not figure out a way to plot all the data points. Instead currently I print the string to the text box. From the text box only the first line is printed.
This is a real time data plotting for an oscilloscope GUI.
How can I plot all the values.
I tested with a sine wave with a I2S mic, it gave a distorted signal when plotted with the following code.
int t;
private void Timer1_Tick(object sender, EventArgs e)
{
one = new Thread(test);
one.Start();
t++;
}
public void test()
{
byte[] bytes = new byte[client.ReceiveBufferSize];
var readCount = stream.Read(bytes, 0, bytes.Length);
string datastring = Encoding.UTF8.GetString(bytes);
txtdata.Invoke((MethodInvoker)(() => txtdata.Text = datastring.Substring(0, 100)));
txtno.Invoke((MethodInvoker)(() =>
txtno.Text = ("\nnumber of bytes read: " + readCount)
));
String ch1 = txtdata.Text; ;
String[] ch1y = ch1.Split(new char[] { '\n', '\r' }, StringSplitOptions.RemoveEmptyEntries);
for (int a = 1; a < ch1y.Length - 1; a++)
{
chart1.Invoke((MethodInvoker)(() =>
chart1.Series[0].Points.AddXY(t, Convert.ToDouble(ch1y[a]))
));
}
}
The issue here is not how fast you plot the data, but the fact that you are trying to plot real-time analogue values using a non-uniform asynchronous pattern.
It sounds like you are trying to approach this from a First Principals perspective but you are skipping a lot of background understanding that is necessary to pull this off.
The oscilloscope has many functions that allow you to focus in on the specific width or length of the analogue sample that you see on the display, this might represent a tiny sample of audio, the amplitude of the line in the display usually represents the voltage of the electrical pulse at a relative point in time, not a specific point in time.
If your sine wave represents a constant 2KHz, and you can see 2 peaks (and 2 troughs) on the screen then you are actually looking at the data captured over an interval of 1ms, to draw a wavy line like that in a chart requires many points, so your initial attempt to plot this as a single X,Y ordinate is going to need many, hundreds perhaps, of discrete points over 1 ms to render the same sort of line, but you would also have to be extra specially careful to ensure that the x interval represented the exact amount of time, or the points you plot will not be in the right place in respect to the time the value was sampled.
This frequency of processing is not something you want to try to achieve with C#, that would need to be done way down at the hardware level.
The ESP32 you are using to sample the analogue input will be capturing a specific packet of data at a specific interval, you may have seen phrases like 8-Bit 16Khz Mono to describe audio quality.
Depending on what processing is going on in the ESP32, the bytes that you receieve will usually represent the total sample, we then need to use the bitness to determine how to break the bytes into an array of values, it is not a single value.
So 8-Bit Mono means that every 8 bits will represent a single value, so in this case each byte is a separate value. Mono means that this is a 1 dimensional array of values, in Stereo or 2 channel interleaved the bytes represents 2 separate arrays of values, so every second byte actually goes into the second array...
So you don't simply convert the bytes to text using UTF8 Encoding, you need to use other libraries, like NAudio to decode this, you can try it manually though.
In simple c# terms, assuming 8-bit mono, you could plot the entire sample using the array index as X and the byte value at the x index as Y, however this still isn't going to quite match what you see in the oscilloscope.
Instead have a think about what you actually want to see on the screen, do some research into using Fast Fourier Transform calculations (FFT) to analyse the analog readings, there are even ways to do this directly on the ESP32 which may reduce the processing you do in the C# code.
I reduced the baud rate. The wave shows almost a pure sine wave at 2kHz wave after that. But only the 2kHz wave looks like that. instead of running a thread, the chart was plotted for each clock tick with Add command.
Related
i build an project in c# with winform GUI on the .net framework. The aim of the programm is to get data (image frames) from a frame grabber card and saving the data after doing some pre-processing on it.
The plugged camera is quite fast and has a pixel size of 640x480 with 16 Bit for each pixel and is working at 350 frames per second.
I added the library from the manufacturer that rises an event for every incoming frame and transfers the data as one-dimensional int16 array.
What i want to do is to average the bunch of frames for every second and saving the result in binary as an float array. Look here:
void Camera_OnRawImage(object sender, short[] data)
{
float[] frame = Array.ConvertAll(data, x => (float)x);
this.AverageProcess.AddImage(frame);
}
My problem is that there is a lot of data coming into my application. So the program is not able to process and save all the data before the next frames are pending to rise an event.
I already tried different approaches like:
- multiple threads to parallel calculate the arrays
- using marshalling methods for a better array access
- converting the arrays to MathNET arrays and using the INTEL MKL on MathNET library to speed up the process. look here:
internal void AddImage(float[] image)
{
currentFrameOfMeasurement++;
Vector<float> newImage = Vector<float>.Build.DenseOfArray(image);
newImage = newImage.Multiply(preAverageFactor);
var row = FrameMat.Column(currentFrameOfPeriod).Add(newImage);
FrameMat.SetColumn(currentFrameOfPeriod, row);
currentFrameOfPeriod++;
if (currentFrameOfPeriod >= this.Settings.FramesPerPeriod)
{
currentFrameOfPeriod = 0;
currentPeriodCount++;
}
}
But that is also a litte bit to slow. The RAM is increasing while grabbing the data. My methods need more than 10ms - wich is quite to slow.
To process and save the data must be possible. There is an application from the manufacturer wich is doing a lot of processing on the grabbed data. It seems like the program is written in delphi. Maybe they are using better library for doing all these operations.
My question ist now. Does anybody has an suggestion what i can do to speed up the process and calulation these arrays.
Thank you in advance
Use the Synchronized Queue (here) feature and queue in all the frames that come into your program and write a thread which can process the items in queue.
I have a program that reads data over serial from an ADC on a PSoC.
The numbers are sent in the format <uint16>, inclusive of the '<' and '>' symbols, transmitted in binary format 00111100 XXXXXXXX XXXXXXXX 00111110 where the 'X's make up the 16 bit unsigned int.
Occasionally the read won't work very well and the program uses the binary data for the '>' symbol as part of its number resulting in the glitch as shown in this screenshot of 2500 samples (ignore the drop between samples 800 to 1500, that was me playing with the ADC input):
You can clearly see that the glitch causes the data to sample roughly the same value each time it happens.
The data is sent ten times a second, so what I was planning on doing was to take ten samples, remove any glitches (where the value is far away from the other samples) and then average the remaining values to smooth out the curve a bit. The output can go anywhere from 0 to 50000+ so I can't just remove values below a certain number.
I'm uncertain how to remove the values that are a long way out of the range of the other values in the 10-sample group, because there may be instances where there are two samples that are affected by this glitch. Perhaps there's some other way of fixing this glitchy data instead of just working around it!
What is the best way of doing this? Here's my code so far (this is inside the DataReceivedEvent method):
SerialPort sp = (SerialPort)sender; //set up serial port
byte[] spBuffer = new byte[4];
int indata = 0;
sp.Read(spBuffer, 0, 4);
indata = BitConverter.ToUInt16(spBuffer, 1);
object[] o = { numSamples, nudDutyCycle.Value, freqMultiplied, nudDistance.Value, pulseWidth, indata };
lock (dt) //lock for multithread safety
{
dt.Rows.Add(o); //add data to datatable
}
I suspect your problem may be because you are reading less bytes from the serial port than you think you are.
For example, sp.Read(spBuffer, 0, 4); won't necessarily read 4 bytes. It could read 1, 2, 3 or 4 bytes (but never 0).
If you know you should be reading a certain number of bytes, try something like this:
public static void BlockingRead(SerialPort port, byte[] buffer, int offset, int count)
{
while (count > 0)
{
// SerialPort.Read() blocks until at least one byte has been read, or SerialPort.ReadTimeout milliseconds
// have elapsed. If a timeout occurs a TimeoutException will be thrown.
// Because SerialPort.Read() blocks until some data is available this is not a busy loop,
// and we do NOT need to issue any calls to Thread.Sleep().
int bytesRead = port.Read(buffer, offset, count);
offset += bytesRead;
count -= bytesRead;
}
}
If there's a timeout during the read, there should be a TimeoutException, so no need to put your own timeout in there.
Then change calls like this:
sp.Read(spBuffer, 0, 4);
To this:
BlockingRead(sp, spbuffer, 0, 4);
A common method in engineering is add a damping function. A damping function basically acts on the differential of a parameter, i.e. the difference between successive values. There are no hard and fast rules about how to choose a damping function and mostly they are tweaked to produce a reasonable result.
So in your case what that means is that you compare the latest value with the one previous to it. If it is greater than a certain amount, either default the latest value to the previous one or reduce the latest value by some fixed factor, say 10% or 1%. That way you don't lose information but also don't have sudden jumps and glitches.
First of all, I would strongly suggest to just fix the parsing issue, then you won't have to worry about glitch values.
However, if you still decide to go down the route of fixing data afterwards:
I see all the glitched data is around a certain value: ~16000. In fact, judging from the graph, I'd say it's almost identical every time. You could simply ignore the data which is in the glitched value range (you would have to do some testing to find the exact bounds), and use the last non-glitched value instead.
I'm working on the new WindowsPhone platform. I have a few intances of a SoundEffectInstance that I would like to combine into a new single Sound file (either SoundEffectInstance, SoundEffect or MediaElement, it does not matter.) I then want to save that file as an mp3 to the phone.
How do I do that? Normally, I would try to send all the files to a bytearray but I'm not sure if that is the correct method here, or how to convert the bytearray into an MP3 format sound.
So for example I have SoundEffectInstance soudBackground, playing from 0 - 5 seconds. I then have SoundEffectInstance chime playing from 3 - 4 seconds, and SoundEffectInstance foreground playing from 3.5 to 7 seconds. I want to combine all these into a single mp3 file that lasts 7 seconds long.
There are two task that you are trying to accomplish here:
Combine several sound files into a single sound file
Save the resulting file as an MP3.
As far as I have found thus far you will have a good bit of challenges with item 2. To date I have not found a pure .Net MP3 encoder. All the ones I find rely on P/Invokes to native code (Which of course won't work on the phone).
As for combining the files, you don't want to treat them as a SoundEffectInstance. That class is only meant for playing and it abstracts most of the details of the sound file away. Instead you will need to treat the sound files as arrays of ints. I'm going to assume that the sample rate on all three sound files is the exact same and that these are 16-bit recordings. I am also going to assume that these wave files are recorded in mono. I'm keeping the scenario simple for now. You can extend upon it with stereo and various sample rates after you've mastered this simpler scenario.
The first 48 bytes of the wave files is nothing but header. Skip past that (for now) and read the contents of the wave files into their own arrays. Once they are all read we can start mixing them together. Ignoring the time differences in which you want to start playing these sounds if we wanted to start producing a sample that is the combined result of all three we could do it by adding the values in the sound file array together and writing that out to an array to hold our result. But there's a problem. 16-bit numbers can only go up to 32,767 (and down to -32,768). If the combined value of all three sounds were to go beyond these limits you'll get really bad distortion. The easiest (though not necessarily the best) way to handle this is to consider the maximum number of simultaneous sounds that will play and scale the values down accordingly. From the 3.5 second to 4 second mark you will have all three sounds playing. So we will scale by dividing by three. Another way is to sum up the sound samples using a data type that can go beyond this range and then normalizing the values back to this range when you are done mixing them together.
Let's define some parameters.
int SamplesPerSecond = 22000;
int ResultRecordingLength = 7;
short[] Sound01;
short[] Sound02;
short[] Sound03;
int[] ResultantSoundBuffer;
short[] ProcessedResultSoundBuffer;
//Insert code to populate sound array's here.
// Sound01.Length will equal 5.0*SamplesPerSecond
// Sound02.Length will equal 1.0*SamplesPerSecond
// Sound03.Length will equal 3.5*SamplesPerSecond
ResultantSound = new int[ResultRecordingLength*SamplesPerSecond];
Once you've got your sound files read and the array prepared for receiving the resulting file you can start rendering. There's several ways we could go about this. Here is one:
void InitResultArray(int[] resultArray)
{
for(int i=0;i<resultArray.Length;++i)
{
resultArray[i]=0;
}
}
void RenderSound(short[] sourceSound, int[] resultArray, double timeOffset)
{
int startIndex = (int)(timeOffset*SamplesPerSecond);
int readIndex = 0;
for(int readIndex=0;((readIndex<sourceSound.Length)&&(readIndex+sourceSound<resultArray.Length;++readIndex)
{
resultArray[readIndex+startIndex] += (int)sourceSound[readIndex];
}
}
RangeAdjust(int[] resultArray)
{
int max = int.MinimumValue;
int min = int.MaximumValue;
for(int i=0;i<resultArray;++i)
{
max = Math.Max(max, resultArray[i]);
min = Math.Min(min, resultArray[i]);
}
//I want the range normalized to [-32,768..32,768]
//you may want to normalize differently.
double scale = 65536d/(double)(max-min);
double offset = 32767-(max*scale);
for(int i=0;i<resultArray.Length;++i)
{
resultArray[i]= (scale*resultArray[i])+offset;
}
}
You would call InitResultAttay to ensure the result array is filled with zeros (I believe it is by default, but I still prefer to explicitly set it to zero) and then call RenderSound() for each sound that you want in your result. After you've rendered your sounds call RangeAdjust to normalize the sound. All that's left is to write it to a file. You'll need to convert from ints back to shorts.
short[] writeBuffer = new short[ResultantSound.Length];
for(int i=0;i<writeBuffer.Length;++i)
writeBuffer[i]=(short)ResultantSound[i];
Now the mixed sound is all ready to write to the file. There is just one thing missing, you need to write the 48 byte wave header before writing the file. I've written code on how to do that here: http://www.codeproject.com/KB/windows-phone-7/WpVoiceMemo.aspx
in my previous question on finding a reference audio sample in a bigger audio sample, it was proposed, that I should use convolution.
Using DSPUtil, I was able to do this. I played a little with it and tried different combinations of audio samples, to see what the result was. To visualize the data, I just dumped the raw audio as numbers to Excel and created a chart using this numbers. A peak is visible, but I don't really know how this helps me. I have these problems:
I don't know, how to infer the starting position of the match in the original audio sample from the location of the peak.
I don't know, how I should apply this with a continuous stream of audio, so I can react, as soon as the reference audio sample occurs.
I don't understand, why picture 2 and picture 4 (see below) differ so much, although, both represent an audio sample convolved with itself...
Any help is highly appreciated.
The following pictures are the result of the analysis using Excel:
A longer audio sample with the reference audio (a beep) near the end:
The beep convolved with itself:
A longer audio sample without the beep convolved with the beep:
The longer audio sample of point 3 convolved with itself:
UPDATE and solution:
Thanks to the extensive help of Han, I was able to achieve my goal.
After I rolled my own slow implementation without FFT, I found alglib which provides a fast implementation.
There is one basic assumption to my problem: One of the audio samples is contained completely within the other.
So, the following code returns the offset in samples in the larger of the two audio samples and the normalized cross-correlation value at that offset. 1 means complete correlation, 0 means no correlation at all and -1 means complete negative correlation:
private void CalcCrossCorrelation(IEnumerable<double> data1,
IEnumerable<double> data2,
out int offset,
out double maximumNormalizedCrossCorrelation)
{
var data1Array = data1.ToArray();
var data2Array = data2.ToArray();
double[] result;
alglib.corrr1d(data1Array, data1Array.Length,
data2Array, data2Array.Length, out result);
var max = double.MinValue;
var index = 0;
var i = 0;
// Find the maximum cross correlation value and its index
foreach (var d in result)
{
if (d > max)
{
index = i;
max = d;
}
++i;
}
// if the index is bigger than the length of the first array, it has to be
// interpreted as a negative index
if (index >= data1Array.Length)
{
index *= -1;
}
var matchingData1 = data1;
var matchingData2 = data2;
var biggerSequenceCount = Math.Max(data1Array.Length, data2Array.Length);
var smallerSequenceCount = Math.Min(data1Array.Length, data2Array.Length);
offset = index;
if (index > 0)
matchingData1 = data1.Skip(offset).Take(smallerSequenceCount).ToList();
else if (index < 0)
{
offset = biggerSequenceCount + smallerSequenceCount + index;
matchingData2 = data2.Skip(offset).Take(smallerSequenceCount).ToList();
matchingData1 = data1.Take(smallerSequenceCount).ToList();
}
var mx = matchingData1.Average();
var my = matchingData2.Average();
var denom1 = Math.Sqrt(matchingData1.Sum(x => (x - mx) * (x - mx)));
var denom2 = Math.Sqrt(matchingData2.Sum(y => (y - my) * (y - my)));
maximumNormalizedCrossCorrelation = max / (denom1 * denom2);
}
BOUNTY:
No new answers required! I started the bounty to award it to Han for his continued effort with this question!
Instead of a convolution you should use a correlation. The size of the correlation peak tells you how much both signals are alike, the position of the peak their relative position in time, or the delay between both signals.
Here we go for the bounty :)
To find a particular reference signal in a larger audio fragment, you need to use a cross-correlation algorithm. The basic formulae can be found in this Wikipedia article.
Cross-correlation is a process by which 2 signals are compared. This is done by multiplying both signals and summing the results for all samples. Then one of the signals is shifted (usually by 1 sample), and the calculation is repeated. If you try to visualize this for very simple signals such as a single impulse (e.g. 1 sample has a certain value while the remaining samples are zero), or a pure sine wave, you will see that the result of the cross-correlation is indeed a measure for for how much both signals are alike and the delay between them. Another article that may provide more insight can be found here.
This article by Paul Bourke also contains source code for a straightforward time-domain implementation. Note that the article is written for a general signal. Audio has the special property that the long-time average is usualy 0. This means that the averages used in Paul Bourkes formula (mx and my) can be left out.
There are also fast implementations of the cross-correlation based on the FFT (see ALGLIB).
The (maximum) value of the correlation depends on the sample values in the audio signals. In Paul Bourke's algorithm however the maximum is scaled to 1.0. In cases where one of the signals is contained entirely within another signal, the maximum value will reach 1. In the more general case the maximum will be lower and a threshold value will have to be determined to decide whether the signals are sufficiently alike.
I've coded up the FFT for a dataset I'm working with. My intent is to create a waterfall plot of the result, but the problem I'm running into is if I change my input data size, then I get a different number of frequency bins. Currently I'm just making my input dataset twice the size of the number of pixels I need to map to. I'm trying to figure out a way to map the frequency bins of any data set size to a specific number of pixels. For example, mapping an array of 500 values to an array that is 1250 elements long. It would be nice to have the option to perform linear and non-linear interpolation on the data mapping. I also might need to go the other way, say to map the values to an array that is 300 elements long. I'm not a math major and am coming up with a blank on this one.
All you need to do is zero-pad your input to the number of pixels you want to display. This is done by simply appending zeros to your input to bring it up to the length you want in pixels. When the FFT is performed, it will be done at the length after zero-padding and give you the same number of output samples as you have pixels.
When you zero-pad an FFT input, the FFT essentially interpolates in the frequency domain automatically. Check this out. Note than this does not actually give you any more information in the frequency domain (new frequency content will not just appear by using this method) even though the number of frequency domain samples is increased. This is just oversampling the existing data.
I'm not sure the best way to go the other way (downsampling), but I assume one way to do so would be to just transform the first N samples that you need. You would be losing data with this, but it may not matter depending on the application.
A discrete Fourier transform (fast or not) will always give you the same size output array as input array. If you want to scale it up then you need to look at using sinc functions with a suitable window. See e.g. http://en.wikipedia.org/wiki/Lanczos_resampling