【This is not a duplicate. Similar questions are about scenarios where people have control over the source data. I do not.】
In Japan there's something called the "Emergency Warning Broadcasting System." It looks like this when activated: http://www.youtube.com/watch?v=9hjlYvp9Pxs
In the above video, at around 2:37, a FSK-modulated signal is sent. I want to parse this signal; i.e. given a WAV file that contains the signal, I want to end up with a StringBuilder that contains 0s and 1s to process them later. I have the spec for the binary data and all, but the problem is that I know nothing about audio programming. :(
This is just for a hobby project, but I became hooked. TV and radio makers can pick up this signal and have their appliances do stuff in reaction to it, so it can't be that hard, right? :(
Facts about the signal:
The mark tone is 1024Hz, and the stop tone is 640Hz
Each tone is 15.625ms long
2 second pause before signal begins and after it ends (probably for detection purposes)
What I did so far:
Write a simple RIFF parser that accepts 8bit mono WAV files and allows me to get samples from them. I've tested it and it works.
A loop that takes 15.625ms of samples and:
Uses RMS to look for two seconds of silence
Uses the Goertzel algorithm to decide if the signal is 1024Hz or 640Hz
The problems I have:
0s and 1s are swallowed during the loop depending on the test data.
Given the clarity of the signal (YouTube-to-MP3 rip), that shouldn't happen.
If I generate a repeating 01 sequence in Audacity 30 times, my program will pick up around 10 of the 01 pairs, instead of 30
Sometimes 0s and 1s are swapped (side effect of the above?)
If I tweak the code so it works with one test sound file, other test sound files stop working
My questions:
Can anyone give me a high level overview on how FSK decoding would be done properly in software?
Do I need to apply some sort of filter that limits the signal to 640Hz+1024Hz and mutes everything else?
What is the best approach to keep the timing right? Maybe I'm doing it wrong?
Any links to beginner's literature on this kind of audio processing? I'd really like to learn and get this working.
The code that reads samples is (simplified):
StringBuilder ews_bits = new StringBuilder();
double[] samples = new double[(int)(samplesPerMs * 16.625D)];
int index = 0, readTo = /* current offset + RIFF subChunk2Size */;
BinaryReader br = /* at start of PCM data */;
while (br.BaseStream.Position < readTo)
{
switch (bitsPerSample / 8)
{
case 1: // 8bit
samples[index++] = ((double)br.ReadByte() - 127.5D) / 256D;
break;
case 2: // 16bit
samples[index++] = (double)br.ReadInt16() / 32768D;
break;
}
if (index != samples.Length)
continue;
/****** The sample buffer is full and we must process it. ******/
if (AudioProcessor.IsSilence(ref samples))
{
silence_count++;
if (state == ParserState.Decoding && silence_count > 150)
{
// End of EWS broadcast reached.
EwsSignalParser.Parse(ews_bits.ToString());
/* ... reset state; go back looking for silence... */
}
goto Done;
}
/****** The signal was not silence. ******/
if (silence_count > 120 && state == ParserState.SearchingSilence)
state = ParserState.Decoding;
if (state == ParserState.Decoding)
{
AudioProcessor.Decode(ref samples, sampleRate, ref ews_bits);
bool continue_decoding = /* check first 20 bits for signature */;
if (continue_decoding) goto Done;
// If we get here, we were decoding a junk signal.
state = ParserState.SearchingSilence;
}
/* Not enough silence yet */
silence_count = 0;
Done:
index = 0;
}
The audio processor is just a class with:
public static void Decode(ref double[] samples, int sampleRate, ref StringBuilder bitHolder)
{
double freq_640 = GoertzelMagnitude(ref samples, 640, sampleRate);
double freq_1024 = GoertzelMagnitude(ref samples, 1024, sampleRate);
if (freq_640 > freq_1024)
bitHolder.Append("0");
else
bitHolder.Append("1");
}
public static bool IsSilence(ref double[] samples)
{
// power_RMS = sqrt(sum(x^2) / N)
double sum = 0;
for (int i = 0; i < samples.Length; i++)
sum += samples[i] * samples[i];
double power_RMS = Math.Sqrt(sum / samples.Length);
return power_RMS < 0.01;
}
/// <remarks>http://www.embedded.com/design/embedded/4024443/The-Goertzel-Algorithm</remarks>
private static double GoertzelMagnitude(ref double[] samples, double targetFrequency, int sampleRate)
{
double n = samples.Length;
int k = (int)(0.5D + ((double)n * targetFrequency) / (double)sampleRate);
double w = (2.0D * Math.PI / n) * k;
double cosine = Math.Cos(w);
double sine = Math.Sin(w);
double coeff = 2.0D * cosine;
double q0 = 0, q1 = 0, q2 = 0;
for (int i = 0; i < samples.Length; i++)
{
double sample = samples[i];
q0 = coeff * q1 - q2 + sample;
q2 = q1;
q1 = q0;
}
double magnitude = Math.Sqrt(q1 * q1 + q2 * q2 - q1 * q2 * coeff);
return magnitude;
}
Thanks for reading. I hope you can help me.
This is how I would do it (high level description)
Run your signal through a FFT
look for steady peaks at about 640Hz+1024Hz (I would say at least +/- 10Hz)
if the signal is steady for about 10 ms (with steady I mean about 95% of the samples are in the same range 640Hz+/-10Hz (or 1024Hz+/-10Hz) take it as a detection of the tone. Use this detection also to synchronize your timer that tells you when to expect the next tone.
I got it about 90% working now after rewriting the sample parsing loop and silence detection parts. There were two main problems in my implementation. The first was that the silence detector was overeager, so I changed it from processing every millisecond of samples to every half-millisecond of samples. That brought me exactly to the start of FSK data.
The next problem was that I then thought I could naively let the demodulator look at 15.625ms of samples as it works itself through the WAV file. It turns out that while this works great for the first 90 bits or so, eventually tones become a little longer or shorter than expected and the demodulator goes out of sync. The current code finds and corrects 13 bits with such a timing mismatch. Particularly vulnerable to this are spots where the signal changes from mark to space and vice versa.
Guess there's a reason the word "analog" contains "anal". It is. I really wish I knew more about signal theory and digital signal processing. :(
How I discovered all of this: I imported the MP3 and trimmed it down to the FSK part using Audacity. Then I had Audacity generate labels for every bit. After that I went through highlighting bits according toe the labels.
Related
I have a piece of code in my C# Windows Form Application that looks like below:
List<string> RESULT_LIST = new List<string>();
int[] arr = My_LIST.ToArray();
string s = "";
Stopwatch sw = new Stopwatch();
sw.Start();
for (int i = 0; i < arr.Length; i++)
{
int counter = i;
for (int j = 1; j <= arr.Length; j++)
{
counter++;
if (counter == arr.Length)
{
counter = 0;
}
s += arr[counter].ToString();
RESULT_LIST.Add(s);
}
s = "";
}
sw.Stop();
TimeSpan ts = sw.Elapsed;
string elapsedTime = String.Format("{0:00}", ts.TotalMilliseconds * 1000);
MessageBox.Show(elapsedTime);
I use this code to get any combination of the numbers of My list. I have behaved with My_LIST like a recursive one. The image below demonstrates my purpose very clearly:
All I need to do is:
Making a formula to calculate the approximate run time of these two
nested for loops to guess the run time for any length and help the
user know the approximate time that he/she must wait.
I have used a C# Stopwatch like this: Stopwatch sw = new Stopwatch(); to show the run time and below are the results(Note that in order to reduce the chance of error I've repeated the calculation three times for each length and the numbers show the time in nano seconds for the first, second and third attempt respectively.):
arr.Length = 400; 127838 - 107251 - 100898
arr.Length = 800; 751282 - 750574 - 739869
arr.Length = 1200; 2320517 - 2136107 - 2146099
arr.Length = 2000; 8502631 - 7554743 - 7635173
Note that there are only one-digit numbers in My_LIST to make the time
of adding numbers to the list approximately equal.
How can I find out the relation between arr.Length and run time?
First, let's suppose you have examined the algorithm and noticed that it appears to be quadratic in the array length. This suggests to us that the time taken to run should be a function of the form
t = A + B n + C n2
You've gathered some observations by running the code multiple times with different values for n and measuring t. That's a good approach.
The question now is: what are the best values for A, B and C such that they match your observations closely?
This problem can be solved in a variety of ways; I would suggest to you that the least-squares method of regression would be the place to start, and see if you get good results. There's a page on it here:
www.efunda.com/math/leastsquares/lstsqr2dcurve.cfm
UPDATE: I just looked at your algorithm again and realized it is cubic because you have a quadratic string concat in the inner loop. So this technique might not work so well. I suggest you use StringBuilder to make your algorithm quadratic.
Now, suppose you did not know ahead of time that the problem was quadratic. How would you determine the formula then? A good start would be to graph your points on log scale paper; if they roughly form a straight line then the slope of the line gives you a clue as to the power of the polynomial. If they don't form a straight line -- well, cross that bridge when you come to it.
Well you gonna do some math here.
Since the total number of runs is exactly n^2, not O(n^2) but exactly n^2 times.
Then what you could do is to keep a counter variable for the number of items processed and use math to find out an estimate
int numItemProcessed;
int timeElapsed;//read from stop watch
int totalItems = n * n;
int remainingEstimate = ((float) totalItems - numItemProcessed) / numItemProcessed) * timeElapsed
Don't assume the algorithm is necessarily N^2 in time complexity.
Take the averages of your numbers, and plot the best fit on a log-log plot, then measure the gradient. This will give you an idea as to the largest term in the polynomial. (see wikipedia log-log plot)
Once you have that, you can do a least-squares regression to work out the coefficients of the polynomial of the correct order. This will allow an estimate from the data, of the time taken for an unseen problem.
Note: As Eric Lippert said, it depends on what you want to measure - averaging may not be appropriate depending on your use case - the first run time might be more correct.
This method will work for any polynomial algorithm. It will also tell you if the algorithm is polynomial (non-polynomial running times will not give straight lines on the log-log plot).
List<int> NPower = new List<int>();
List<double> list = new List<double>();
try
{
for (int i = 1; i < dataGridView1.Rows.Count; i++)
{
for (int n = 0; n < i + 30; n++)
{
NPower.Add(Convert.ToInt32(dataGridView1.Rows[i + n].Cells[6].Value));
}
}
average = NPower.Average();
total = Math.Pow(average, 4);
NPower.Clear();
}
catch (Exception)
{
average = NPower.Average();
NP = Convert.ToInt32(Math.Pow(average, (1.0 / 3.0)));
label19.Text = "Normalised Power: " + NP.ToString();
NPower.Clear();
}
Hi so i'm trying to calculate the normalized power for a cycling polar cycle. I know that for the normalized power you need to:
1) starting at the 30 s mark, calculate a rolling 30 s average (of the preceeding time points, obviously).
2) raise all the values obtained in step #1 to the 4th power.
3) take the average of all of the values obtained in step #2.
4) take the 4th root of the value obtained in step #3.
I think i have done that but the normalized power comes up with 16 which isnt correct. Could anyone look at my code to see if they could figure out a solution. Thankyou, sorry for my code i'm still quite new to this so my code might be in the incorrect format.
I'm not sure that I understand your requirements or code completely, but a few things I noticed:
Since you're supposed to start taking the rolling average after 30 seconds, shouldn't i be initialized to 30 instead of 1?
Since it's a rolling average, shouldn't n be initialized to the value of i instead of 0?
Why is the final result calculated inside a catch block?
Shouldn't it be Math.Pow(average, (1.0 / 4.0)) since you want the fourth root, not the third?
I'm programming an C# emulator and decided to output the PCM using CScore.
When the sample size (for each channel) is one byte, the sound outputs correctly, but when I increase the sample size to 16 bits, the sound is very very noisy.
A related question to that problem is how those 2 bytes are interpreted (are they signed? high byte first?)
This is roughly what I'm doing:
First I generate the samples as such
public void GenerateSamples(int sampleCount)
{
while(sampleCount > 0)
{
--sampleCount;
for(int c = 0; c < _numChannels; ++c)
{
_buffer[_sampleIndex++] = _outputValue;
}
// The amount of ticks in a sample
_tickCounter -= APU.MinimumTickThreshold;
if(_tickCounter < 0)
{
_tickCounter = _tickThreshold;
_up = !_up;
// Replicating signed behaviour
_outputValue = (short)(_up ? 32767 : -32768);
}
}
}
This will generate a simple square wave with the frequency determined by the _tickThreshold. If the _buffer is a byte array, the sound is correct.
I want to output it with shorts because it will enable me to use signed samples and simply add multiple channels in order to mix them.
This is how I'm outputting the sound.
for(int i = 0; i < sampleCount; ++i)
{
for(int c = 0; c < _numChannels; ++c)
{
short sample = _channel.Buffer[_channelSampleIndex++];
// Outputting the samples the other way around doesn't output
// sound for me
_buffer[_sampleIndex++] = (byte)sample;
_buffer[_sampleIndex++] = (byte)(sample >> 8);
}
}
The WaveFormat I'm using is determined like this:
_waveFormat = new WaveFormat(_apu.SampleRate, // 44000
_apu.SampleSize * 8, // 16
_apu.NumChannels); // 2
I'm pretty sure there is something obvious I'm missing, but I've been debugging this for a while and don't seem to pinpoint where the problem is.
Thanks
Walk of shame here.
The problem was that I wasn't taking in account that now I needed to generate half the amount of samples (CScore asked for amount of bytes, not samples).
In my example, I had to divide the sampleCount variable by the sampleSize to generate the correct amount of sound.
The noise came because I wasn't synchronizing the extra samples with the next Read call from CScore (I'm generating sound on the fly, instead of pre-buffering it. This way I have no delay introduced because of extra samples).
I found out about the problem looking at this: SampleToPcm16.cs
I'm trying to create a heart beat monitor and need some false information to feed into my program to check it works.
One of the requirements of my program is to set off an alarm if the heart rate integer drops below a certain figure.
However the problem I have is, if the heart rate drops below the minimum heart rate integer before it sets off the alarm and then bounces straight back up again, that's not a test for the alarm, see pseudo code:
If heart_rate < 60:
Count = Count + 1
If heart _rate > 60:
Reset Count variable
If Count = 10:
Set off alarm
So I need to generate data where it simulates a heart rate dropping below 60 (patient dieing).
Would the quickest way to do this create a list or text file that the program reads from to simulate the situation or is there a funky way that can simulate random numbers that can set off the alarm and will stay around that integer once the alarm has been set off?
Sorry if this sounds super confusing, I'm a python programmer, having to program in C#.
Your problem seams to be a "data stream testing" one. Manually generating data streams is not something I would personally characterize as "quick" (or even "sane"...). You either get an existing data set or you generate it (but not manually, unless it's a relatively small number of data points: 20~30).
That being said, you need to test something, which means you need to know that when a target case (heart rate below 60), your system correctly catches the "exception". This, in turn, means that you need to know where it happens, i.e. the exact moment in time.
My advice is: automatically generate a data set, full of "normal" values. At different time point (i.e. position in the data stream) manually insert "exception" values (59), with different frequencies, to see if the system resets the counter (less than 10 occurrences) or if it raises an alarm (at least 10 occurrences).
You would then have the following initial data stream:
80 78 78 81 80 81 ... 75 76 80 // say 100 values ("data points")
// 0 1 2 3 4 5 97 98 99 -> indices ("time points")
If you insert a "false alarm" (80 78 59 59 78) at index 2, for example, you know that your system should reset the counter. You would do the same for the "alarm"; you'll know the index (time point) when the alarm should be raised.
Edit: the part about "manually" inserting the targeted values should be interpreted as "empirically selecting an index of an array (or a list) filled with random data, where the data should be inserted".
public static class HeartrateGenerator
{
static Random random = new Random();
public static IEnumerable<int> GenerateHeartrate(
int totalSequenceLength,
int dropsBelow60After,
int bouncesBackAfter = -1)
{
// NOTE: check input data
int i = 0;
// return values > 60
while (i < dropsBelow60After)
{
i++;
yield return 60 + random.Next() % 60;
}
if (bouncesBackAfter > 0)
// return values < 60
while (i < bouncesBackAfter)
{
i++;
yield return random.Next() % 60;
}
// return normal values again
while (i < totalSequenceLength)
{
i++;
yield return 60 + random.Next() % 60;
}
}
}
example usage:
HeartrateGenerator.GenerateHeartrate(100, 20, 28);
Try this: you might want to change the rand.Next(min,max) to use normal range of heartbeat or whatever range you want to see for a heartbeat value.
public class Program
{
public void Main(string[] args)
{
Random rand = new Random();
bool flag = true;
int count = 0;
while (flag)
{
if(count==10)
{
Console.WriteLine ("dead");
flag = false;
break;
}
Thread.Sleep(1000);
var heart_rate = rand.Next(50,70);
Console.WriteLine (heart_rate);
if (heart_rate < 60)
{
count++;
}
else if(heart_rate > 60)
{
count=0;
}
}
}
}
I'm trying to implement a discrete fourier transform, but it's not working. I'm probably have written a bug somewhere, but I haven't found it yet.
Based on the following formula:
This function does the first loop, looping over X0 - Xn-1...
public Complex[] Transform(Complex[] data, bool reverse)
{
var transformed = new Complex[data.Length];
for(var i = 0; i < data.Length; i++)
{
//I create a method to calculate a single value
transformed[i] = TransformSingle(i, data, reverse);
}
return transformed;
}
And the actual calculating, this is probably where the bug is.
private Complex TransformSingle(int k, Complex[] data, bool reverse)
{
var sign = reverse ? 1.0: -1.0;
var transformed = Complex.Zero;
var argument = sign*2.0*Math.PI*k/data.Length;
for(var i = 0; i < data.Length; i++)
{
transformed += data[i]*Complex.FromPolarCoordinates(1, argument*i);
}
return transformed;
}
Next the explaination of the rest of the code:
var sign = reverse ? 1.0: -1.0; The reversed DFT will not have -1 in the argument, while a regular DFT does have a -1 in the argument.
var argument = sign*2.0*Math.PI*k/data.Length; is the argument of the algorithm. This part:
then the last part
transformed += data[i]*Complex.FromPolarCoordinates(1, argument*i);
I think I carefully copied the algorithm, so I don't see where I made the mistake...
Additional information
As Adam Gritt has shown in his answer, there is a nice implementation of this algorithm by AForge.net. I can probably solve this problem in 30 seconds by just copying their code. However, I still don't know what I have done wrong in my implementation.
I'm really curious where my flaw is, and what I have interpreted wrong.
My days of doing complex mathematics are a ways behind me right now so I may be missing something myself. However, it appears to me that you are doing the following line:
transformed += data[i]*Complex.FromPolarCoordinates(1, argument*i);
when it should probably be more like:
transformed += data[i]*Math.Pow(Math.E, Complex.FromPolarCoordinates(1, argument*i));
Unless you have this wrapped up into the method FromPolarCoordinates()
UPDATE:
I found the following bit of code in the AForge.NET Framework library and it shows additional Cos/Sin operations being done that are not being handled in your code. This code can be found in full context in the Sources\Math\FourierTransform.cs: DFT method.
for ( int i = 0; i < n; i++ )
{
dst[i] = Complex.Zero;
arg = - (int) direction * 2.0 * System.Math.PI * (double) i / (double) n;
// sum source elements
for ( int j = 0; j < n; j++ )
{
cos = System.Math.Cos( j * arg );
sin = System.Math.Sin( j * arg );
dst[i].Re += ( data[j].Re * cos - data[j].Im * sin );
dst[i].Im += ( data[j].Re * sin + data[j].Im * cos );
}
}
It is using a custom Complex class (as it was pre-4.0). Most of the math is similar to what you have implemented but the inner iteration is doing additional mathematical operations on the Real and Imaginary portions.
FURTHER UPDATE:
After some implementation and testing I found that the code above and the code provided in the question produce the same results. I also found, based on the comments what the difference is between what is generated from this code and what is produced by WolframAlpha. The difference in the results is that it would appear that Wolfram is applying a normalization of 1/sqrt(N) to the results. In the Wolfram Link provided if each value is multiplied by Sqrt(2) then the values are the same as those generated by the above code (rounding errors aside). I tested this by passing 3, 4, and 5 values into Wolfram and found that my results were different by Sqrt(3), Sqrt(4) and Sqrt(5) respectfully. Based on the Discrete Fourier Transform information provided by wikipedia it does mention a normalization to make the transforms for DFT and IDFT unitary. This might be the avenue that you need to look down to either modify your code or understand what Wolfram may be doing.
Your code is actually almost right (you are missing an 1/N on the inverse transform). The thing is, the formula you used is typically used for computations because it's lighter, but on purely theorical environments (and in Wolfram), you would use a normalization by 1/sqrt(N) to make the transforms unitary.
i.e. your formulas would be:
Xk = 1/sqrt(N) * sum(x[n] * exp(-i*2*pi/N * k*n))
x[n] = 1/sqrt(N) * sum(Xk * exp(i*2*pi/N * k*n))
It's just a matter of convention in normalisation, only amplitudes change so your results weren't that bad (if you hadn't forgotten the 1/N in the reverse transform).
Cheers