Develop Reference

Read Stdin continously

I have a C# program which takes FFMPEG raw data and begins streaming it to a monitor connected to the network. The program works with still images so I assume I have something wrong with my stdin reading speed. I want the program to work like this:
Begin reading stdin ffmpeg raw
When the buffer is at a specified length (I have to form the frames myself from the raw data, this specified length is width * height * 3)
Send frame & meantime start reading the next part of the raw data
Repeat until shutdown / video end. My current code can stream with 30 FPS, but its not quite good. I could achieve 60 FPS when sending still images. The timers are needed because I have to send the frames at very specific timestamps.
int width = 128;
int height = 64;
static Stream standard_input = Console.OpenStandardInput();
static byte[] buffer_one = new byte[width*height*3];
public static void ReadStandardInput(){
int in_len;
while((in_len = standard_input.Read(buffer, 0, height*width*3)) > 0 ){
if (in_len == height*width*3){
if (frame_delay_clock.IsRunning == false){
frame_delay_clock.Start();
}
if (frame_delay_clock.ElapsedMilliseconds >= wait_between_frames){
SendFrame(buffer, 15);
frame_delay_clock.Reset();
}
else{
continue;
}
}
else{
continue;
}
}
System.Console.WriteLine("STDIN ended.");
standard_input.Close();
}

.NET SerialPort Write() adding characters and moving bytes

I am developing a firmware upgrade app in C# using VisualStudio 2015. The app uses a XAML-based form and C# code. The target of the firmware upgrade is a custom board running an STM32F417 processor (ARM Cortex M4). The C# code connects with the 417, causes the 417 to jump to its bootloader (factory installed) and uses the bootloader API to load the new firmware.
After every transfer of 256 bytes (max transfer allowed), a checksum of the data and the number of bytes is sent, and an ACK byte is received from the 417 to show that everything checked out for that transfer. The transfer would fail after a couple of blocks of 256 and to test the transfer, I set up the C# code to just transfer the .hex file without any of the other checks.
I set the code to transfer the updated firmware as a .hex file. line by line. It writes data out the a serial line and is captured as text in HyperTerminal.
I implemented it as a write of 256 bytes:
byte[] buffer256 = new byte[256];
// Read the file and process one line at a time
System.IO.StreamReader file = new System.IO.StreamReader(path);
while ((line = file.ReadLine()) != null)
{
line = line.Substring(1, line.Length - 1);
// Create byte array from string
var bytes = GetBytesFromByteString(line).ToArray();
// Assign values to variables from byte array
type = bytes[3];
// BLOCK WRITE TO MEMORY
if (type == 0) // data type
{
length = bytes[0];
if (byteCounter >= 255)
{
_serialPort.Write(buffer256, 0, 256);
Array.Clear(buffer256, 0, buffer256.Length);
byteCounter = 0;
}
for (int i = 0; i < length; i++)
{
buffer256[byteCounter++] = bytes[4 + i];
}
} // end BLOCK WRITE TO MEMORY
counter++;
} // end WHILE loop for loading hex file
file.Close();
And also as a write of single bytes:
byte[] buffer256 = new byte[256];
// Read the file and process one line at a time
System.IO.StreamReader file = new System.IO.StreamReader(path);
while ((line = file.ReadLine()) != null)
{
line = line.Substring(1, line.Length - 1);
// Create byte array from string
var bytes = GetBytesFromByteString(line).ToArray();
// Assign values to variables from byte array
type = bytes[3];
if (type == 0)
{
length = bytes[0];
for (int i = 0; i < length; i++)
{
_serialPort.Write(bytes, 4 + i, 1);
} // end FOR loop
}// end SINGLE BYTE WRITING
counter++;
} // end WHILE loop for loading hex file
file.Close();
In both cases, the hex file is changed by writing it using the SerialPort class. I tried various other methods, even WriteAsync in the BaseStream class on which the SerialPort class is built (overriding Stream methods).
Here is a sample (8 words) of the original .hex file (data only) compared to the received data:
Original:
00F8012B 002AF9D1 18467047 70B50646
2DF0FEFC 04680546 0A220021 304600F0
Received:
00F8012B 002AF9D1 18464670 4770B506
462DF0FE FC046805 460A2200 21304600
These are lines 49 and 50 out of 13391. The checksums for some of the blocks were also checked and were wrong. I'm pretty sure that is why the firmware upgrades are failing — the bootloader is calculating a checksum based on what it is receiving and fails when it compares it to the checksum the C# app is calculating and sending over.
My question is: if the SerialPort (and underlying Stream) is unreliable, what should I replace it with and how? Is it possible to just replace the SerialPort class, maybe with a C++ class? Or would it be better to rewrite the entire app in C++ or something else?

The SerialPort class itself isn't unreliable. I've used it successfully on the desktop and numerous CE devices. The problem could be the hardware on either end, one of the OSes, some fiddly configuration value, or some combination of any of those.
It could also be your code. I'm not sure you're doing anything wrong, but reading binary data into a string using ReadLine seems pretty weird to me. Have you verified bytes[] is correct in a debugger?
PS--If one of the hardware issues is related to timing, switching to C++ can be helpful. Interop is pretty painless for something like this, so I wouldn't bother rewriting the whole app, but I don't know your situation so couldn't say for sure.

C# - Microphone noise detection

I am using nAudio Library to capture microphone input. But I have run into a problem.
I am using the code(which I have modified slightly) from an nAudio sample app.
The codes generates a WAV file based on mic input and renders it as a wave. Here is the code for that.
private void RenderFile()
{
SampleAggregator.RaiseRestart();
using (WaveFileReader reader = new WaveFileReader(this.voiceRecorderState.ActiveFile))
{
this.samplesPerSecond = reader.WaveFormat.SampleRate;
SampleAggregator.NotificationCount = reader.WaveFormat.SampleRate/10;
//Sample rate is 44100
byte[] buffer = new byte[1024];
WaveBuffer waveBuffer = new WaveBuffer(buffer);
waveBuffer.ByteBufferCount = buffer.Length;
int bytesRead;
do
{
bytesRead = reader.Read(waveBuffer, 0, buffer.Length);
int samples = bytesRead / 2;
double sum = 0;
for (int sample = 0; sample < samples; sample++)
{
if (bytesRead > 0)
{
sampleAggregator.Add(waveBuffer.ShortBuffer[sample] / 32768f);
double sample1 = waveBuffer.ShortBuffer[sample] / 32768.0;
sum += (sample1 * sample1);
}
}
double rms = Math.Sqrt(sum / (SampleAggregator.NotificationCount));
var decibel = 20 * Math.Log10(rms);
System.Diagnostics.Debug.WriteLine(decibel.ToString() + " in dB");
} while (bytesRead > 0);
int totalSamples = (int)reader.Length / 2;
TotalWaveFormSamples = totalSamples / sampleAggregator.NotificationCount;
SelectAll();
}
audioPlayer.LoadFile(this.voiceRecorderState.ActiveFile);
}
Below is a little chunk from a result of a 2second WAV file with no sound but only mic noise.
-54.089102453893 in dB
-51.9171950072361 in dB
-53.3478098666891 in dB
-53.1845794096928 in dB
-53.8851764055102 in dB
-57.5541358628342 in dB
-54.0121140454216 in dB
-55.5204248291508 in dB
-54.9012326746571 in dB
-53.6831017096011 in dB
-52.8728852678309 in dB
-55.7021600863786 in dB
As we can see, the db level hovers around -55 when there is no input sound, only silence. if I record saying "Hello" in mic in a normal tone, the db value will goto -20 or so. I read somewhere that average human talk is around 20dB and -3dB to -6dB is the ZERO value range for mic.
Question: Am I calculating the dB value correctly? (i used a formula proposed here by someone else)...Why dB is always coming up in negative? Am i missing a crucial concept or a mechanism?
I searched nAudio documentation at codeplex and didn't find an answer. In my observation, the documentation there needs to be more explanatory then just a bunch of Q&A [no offense nAudio :)]

If I understood the formula correctly, the actual value you're calculating is dBm, and that's absolutely ok since dB is just a unit to measure amplification and can't be used for measuring signal strength/amplitude (i.e. you can say I amplified the signal by 3 db, but can't say my signal strength is 6 dB).
The negative values are there just because of the logarithmic conversion part of the formula (converting watts/miliWatts to db) and since the signals you're dealing with are relativly weak.
So in conclusion, looks, like you've done everything right.
Hope it helps.
EDIT: BTW, as you can see, there really is ~23-25dbm difference between silence and human speech

Processing 16bit sample audio

Right now i have an audio file (2 Channels, 44.1kHz Sample Rate, 16bit Sample size, WAV) I would like to pass it into this method but i am not sure of any way to convert the WAV file to a byte array.
/// <summary>
/// Process 16 bit sample
/// </summary>
/// <param name="wave"></param>
public void Process(ref byte[] wave)
{
_waveLeft = new double[wave.Length / 4];
_waveRight = new double[wave.Length / 4];
if (_isTest == false)
{
// Split out channels from sample
int h = 0;
for (int i = 0; i < wave.Length; i += 4)
{
_waveLeft[h] = (double)BitConverter.ToInt16(wave, i);
_waveRight[h] = (double)BitConverter.ToInt16(wave, i + 2);
h++;
}
}
else
{
// Generate artificial sample for testing
_signalGenerator = new SignalGenerator();
_signalGenerator.SetWaveform("Sine");
_signalGenerator.SetSamplingRate(44100);
_signalGenerator.SetSamples(16384);
_signalGenerator.SetFrequency(5000);
_signalGenerator.SetAmplitude(32768);
_waveLeft = _signalGenerator.GenerateSignal();
_waveRight = _signalGenerator.GenerateSignal();
}
// Generate frequency domain data in decibels
_fftLeft = FourierTransform.FFTDb(ref _waveLeft);
_fftRight = FourierTransform.FFTDb(ref _waveRight);
}
Edit Hi sorry for the confusion. I'm currently new to audio signalling so my explanation of what I might like to get is wrong. For this method to work correctly, i believe i need to pass in the byte array of the data chunk in the wav file only. The end result would be to apply fft on it as shown in the code and transform it to a spectrogram. Thanks.

you need:
using System.IO;
and this code to get the byte array
byte[] data = File.ReadAllBytes(PathToFile);
where PathToFile is the Location (as String) of the .wav file.
Edit:
Right now i have an audio file (2 Channels, 44.1kHz Sample Rate, 16bit Sample size, WAV) I would like to pass it into this method but i am not sure of any way to convert the WAV file to a byte array.
He asks for a function to get the byte array from the .wav file he didn't say anything about getting the specific part of the byte array that contains the data of the music.
So Downvoting a correct answer is..

WasapiCapture NAudio

We are using the NAudio Stack written in c# and trying to capture the audio in Exclusive mode with PCM 8kHZ and 16bits per sample.
In the following function:
private void InitializeCaptureDevice()
{
if (initialized)
return;
long requestedDuration = REFTIMES_PER_MILLISEC * 100;
if (!audioClient.IsFormatSupported(AudioClientShareMode.Shared, WaveFormat) &&
(!audioClient.IsFormatSupported(AudioClientShareMode.Exclusive, WaveFormat)))
{
throw new ArgumentException("Unsupported Wave Format");
}
var streamFlags = GetAudioClientStreamFlags();
audioClient.Initialize(AudioClientShareMode.Shared,
streamFlags,
requestedDuration,
requestedDuration,
this.waveFormat,
Guid.Empty);
int bufferFrameCount = audioClient.BufferSize;
this.bytesPerFrame = this.waveFormat.Channels * this.waveFormat.BitsPerSample / 8;
this.recordBuffer = new byte[bufferFrameCount * bytesPerFrame];
Debug.WriteLine(string.Format("record buffer size = {0}", this.recordBuffer.Length));
initialized = true;
}
We configured the WaveFormat before calls this function to (8000,1) and also a period of 100 ms.
We expected the system to allocate 1600 bytes for the buffer and interval of 100 ms as requested.
But we noticed following occured:
1. the system allocated audioClient.BufferSize to be 4800 and "this.recordBuffer" an array of 9600 bytes (which means a buffer for 600ms and not 100ms).
2. the thread is going to sleep and then getting 2400 samples (4800 bytes) and not as expected frames of 1600 bytes
Any idea what is going there?

You say you are capturing audio in exclusive mode, but in the example code you call the Initialize method with AudioClientMode.Shared. It strikes me as very unlikely that shared mode will let you work at 8kHz. Unlike the wave... APIs, WASAPI does no resampling for you of playback or capture, so the soundcard itself must be operating at the sample rate you specify.