Develop Reference

Send multiple tone generators to a mixer and play the output of the mixer

Using NAudio Library
I have seen the following link:
https://markheath.net/post/mixing-and-looping-with-naudio (don't fully understand the detail)
I am able to play generated tones one after the other.
What I don't understand is how to combine multiple tone generators/sources into a mixer and play the output of that mixer as shown below:
I'm missing some fundamental understanding of the process here I think, so any pointers or further detail would really help me move forward.
Creating the mixer:
WaveMixerStream32 mixer = new WaveMixerStream32();
mixer.AddInputStream(GenerateUpperSine(args[0]));
mixer.AddInputStream(GenerateLowerSine(args[1]));
Creating the Sine Generators:
private static WaveStream GenerateUpperSine(string frequency){
var outFormat = WaveFormat.CreateIeeeFloatWaveFormat(44100, 1);
const int durationInSecods = 5;
var UpperFreq = new SignalGenerator();
{
Gain = 0.2,
Frequency = Double.Parse(frequency),
Type = SignalGeneratorType.Sin
}
var sp = UpperFreq.ToWaveProvider16();
byte{} data = new Byte[outFormat.AverageBytesPerSecond * durationInSeconds];
var bytesRead = new sp.Read(data, 0, data.Length);
return new RawSourceWaveStream(new MemoryStream(data), outFormat);
}
The GenerateLowerSine function is an exact duplicate of the upper one, just with some variables changed.
Calling this application from the command line as follows:
c:\path\to\exe\my.exe 1000 750
I have some code which I can add to the generation code which plays the tones as expected, one after the other. This may reside within the function and is as follows:
var wo = new WaveOutEvent();
wo.Init(UpperFreq);
wo.Play();
This indicates the basic functionality is working, at least in part.
What I believe I am missing is either:
how to send the output of the tone generators to the mixer and then initiate the playback from the output of the mixer
or
possibly connect the output of the mixer to a WaveOut device and send
the audio/data from the tone generators to the inputs of the mixer
Perhaps it is something else though.
Any help would be greatly appreciated.

SharpDX XAudio2: 6 SourceVoice limit

I have been playing around with SharpDX.XAudio2 for a few days now, and while things have been largely positive (the odd software quirk here and there) the following problem has me completely stuck:
I am working in C# .NET using VS2015.
I am trying to play multiple sounds simultaneously.
To do this, I have made:
- Test.cs: Contains main method
- cSoundEngine.cs: Holds XAudio2, MasteringVoice, and sound management methods.
- VoiceChannel.cs: Holds a SourceVoice, and in future any sfx/ related data.
cSoundEngine:
List<VoiceChannel> sourceVoices;
XAudio2 engine;
MasteringVoice master;
public cSoundEngine()
{
engine = new XAudio2();
master = new MasteringVoice(engine);
sourceVoices = new List<VoiceChannel>();
}
public VoiceChannel AddAndPlaySFX(string filepath, double vol, float pan)
{
/**
* Set up and start SourceVoice
*/
NativeFileStream fileStream = new NativeFileStream(filepath, NativeFileMode.Open, NativeFileAccess.Read);
SoundStream soundStream = new SoundStream(fileStream);
SourceVoice source = new SourceVoice(engine, soundStream.Format);
AudioBuffer audioBuffer = new AudioBuffer()
{
Stream = soundStream.ToDataStream(),
AudioBytes = (int)soundStream.Length,
Flags = SharpDX.XAudio2.BufferFlags.EndOfStream
};
//Make voice wrapper
VoiceChannel voice = new VoiceChannel(source);
sourceVoices.Add(voice);
//Volume
source.SetVolume((float)vol);
//Play sound
source.SubmitSourceBuffer(audioBuffer, soundStream.DecodedPacketsInfo);
source.Start();
return voice;
}
Test.cs:
cSoundEngine engine = new cSoundEngine();
total = 6;
for (int i = 0; i < total; i++)
{
string filepath = System.IO.Directory.GetParent(System.IO.Directory.GetCurrentDirectory()).Parent.FullName + #"\Assets\Planet.wav";
VoiceChannel sfx = engine.AddAndPlaySFX(filepath, 0.1, 0);
}
Console.Read(); //Input anything to end play.
There is currently nothing worth showing in VoiceChannel.cs - it holds 'SourceVoice source' which is the one parameter sent in the constructor!
Everything is fine and well running with up to 5 sounds (total = 5). All you hear is the blissful drone of Planet.wav. Any higher than 5 however causes the console to freeze for ~5 seconds, then close (likely a c++ error which debugger can't handle). Sadly no error message for us to look at or anything.
From testing:
- Will not crash as long as you do not have more than 5 running sourcevoices.
- Changing sample rate does not seem to help.
- Setting inputChannels for master object to a different number makes no difference.
- MasteringVoice seems to say the max number of inputvoices is 64.
- Making each sfx play from a different wav file makes no difference.
- Setting the volume for sourcevoices and/or master makes no difference.
From the XAudio2 API Documentation I found this quote: 'XAudio2 removes the 6-channel limit on multichannel sounds, and supports multichannel audio on any multichannel-capable audio card. The card does not need to be hardware-accelerated.'. This is the closest I have come to finding something that mentions this problem.
I am not well experienced with programming sfx and a lot of this is very new to me, so feel free to call me an idiot where appropriate but please try and explain things in layman terms.
Please, if you have any ideas or answers they would be greatly appreciated!
-Josh

As Chuck has suggested, I have created a databank which holds the .wav data, and I just reference the single data store with each buffer. This has improved the sound limit up to 20 - however this has not fixed the problem as a whole, likely because I have not implemented this properly.
Implementation:
class SoundDataBank
{
/**
* Holds a single byte array for each sound
*/
Dictionary<eSFX, Byte[]> bank;
string curdir => Directory.GetParent(Directory.GetCurrentDirectory()).Parent.FullName;
public SoundDataBank()
{
bank = new Dictionary<eSFX, byte[]>();
bank.Add(eSFX.planet, NativeFile.ReadAllBytes(curdir + #"\Assets\Planet.wav"));
bank.Add(eSFX.base1, NativeFile.ReadAllBytes(curdir + #"\Assets\Base.wav"));
}
public Byte[] GetSoundData(eSFX sfx)
{
byte[] output = bank[sfx];
return output;
}
}
In SoundEngine we create a SoundBank object (initialised in SoundEngine constructor):
SoundDataBank soundBank;
public VoiceChannel AddAndPlaySFXFromStore(eSFX sfx, double vol)
{
/**
* sourcevoice will be automatically added to MasteringVoice and engine in the constructor.
*/
byte[] buffer = soundBank.GetSoundData(sfx);
MemoryStream memoryStream = new MemoryStream(buffer);
SoundStream soundStream = new SoundStream(memoryStream);
SourceVoice source = new SourceVoice(engine, soundStream.Format);
AudioBuffer audioBuffer = new AudioBuffer()
{
Stream = soundStream.ToDataStream(),
AudioBytes = (int)soundStream.Length,
Flags = SharpDX.XAudio2.BufferFlags.EndOfStream
};
//Make voice wrapper
VoiceChannel voice = new VoiceChannel(source, engine, MakeOutputMatrix());
//Volume
source.SetVolume((float)vol);
//Play sound
source.SubmitSourceBuffer(audioBuffer, soundStream.DecodedPacketsInfo);
source.Start();
sourceVoices.Add(voice);
return voice;
}
Following this implementation now lets me play up to 20 sound effects - but NOT because we are playing from the soundbank. Infact, even running the old method for sound effects now gets up to 20 sfx instances.
This has improved up to 20 because we have done NativeFile.ReadAllBytes(curdir + #"\Assets\Base.wav") in the constructor for the SoundBank.
I suspect NativeFile is holding a store of loaded file data, so you regardless of whether you run the original SoundEngine.AddAndPlaySFX() or SoundEngine.AddAndPlaySFXFromStore(), they are both running from memory?
Either way, this has quadrupled the limit from before, so this has been incredibly useful - but requires further work.

XAudio2 - Cracking output when using a dynamic buffer

To provide a little bit of context. I am trying to output live audio from a camera in my c# application. After doing some research it seems pretty obvious to do it in a c++ managed dll. I chose the XAudio2 api because it should be pretty easy to implement and use with dynamic audio content.
So the idea is to create the XAudio device in c++ with an empty buffer and push in the audio from the c# code side. The audio chunks are pushed every 50ms because I want to keep the latency as small as possible.
// SampleRate = 44100; Channels = 2; BitPerSample = 16;
var blockAlign = (Channels * BitsPerSample) / 8;
var avgBytesPerSecond = SampleRate * blockAlign;
var avgBytesPerMillisecond = avgBytesPerSecond / 1000;
var bufferSize = avgBytesPerMillisecond * Time;
_sampleBuffer = new byte[bufferSize];
Everytime the timer runs it gets the pointer of the audio buffer, reads the data from the audio, copies the data to the pointer and calls the PushAudio method.
I am also using a stopwatch to check how long the processing took and calculate the interval again for the timer to include the processing time.
private void PushAudioChunk(object sender, ElapsedEventArgs e)
{
unsafe
{
_pushAudioStopWatch.Reset();
_pushAudioStopWatch.Start();
var audioBufferPtr = Output.AudioCapturerBuffer();
FillBuffer(_sampleBuffer);
Marshal.Copy(_sampleBuffer, 0, (IntPtr)audioBufferPtr, _sampleBuffer.Length);
Output.PushAudio();
_pushTimer.Interval = Time - _pushAudioStopWatch.ElapsedMilliseconds;
_pushAudioStopWatch.Stop();
DIX.Log.WriteLine("Push audio took: {0}ms", _pushAudioStopWatch.ElapsedMilliseconds);
}
}
This is the implementation of the c++ part.
Regarding to the documentation on msdn I created a XAudio2 device and added the MasterVoice and SourceVoice. The buffer is empty at first because the c# part is responsible to push in the audio data.
namespace Audio
{
using namespace System;
template <class T> void SafeRelease(T **ppT)
{
if (*ppT)
{
(*ppT)->Release();
*ppT = NULL;
}
}
WAVEFORMATEXTENSIBLE wFormat;
XAUDIO2_BUFFER buffer = { 0 };
IXAudio2* pXAudio2 = NULL;
IXAudio2MasteringVoice* pMasterVoice = NULL;
IXAudio2SourceVoice* pSourceVoice = NULL;
WaveOut::WaveOut(int bufferSize)
{
audioBuffer = new Byte[bufferSize];
wFormat.Format.wFormatTag = WAVE_FORMAT_PCM;
wFormat.Format.nChannels = 2;
wFormat.Format.nSamplesPerSec = 44100;
wFormat.Format.wBitsPerSample = 16;
wFormat.Format.nBlockAlign = (wFormat.Format.nChannels * wFormat.Format.wBitsPerSample) / 8;
wFormat.Format.nAvgBytesPerSec = wFormat.Format.nSamplesPerSec * wFormat.Format.nBlockAlign;
wFormat.Format.cbSize = 0;
wFormat.SubFormat = KSDATAFORMAT_SUBTYPE_PCM;
HRESULT hr = XAudio2Create(&pXAudio2, 0, XAUDIO2_DEFAULT_PROCESSOR);
if (SUCCEEDED(hr))
{
hr = pXAudio2->CreateMasteringVoice(&pMasterVoice);
}
if (SUCCEEDED(hr))
{
hr = pXAudio2->CreateSourceVoice(&pSourceVoice, (WAVEFORMATEX*)&wFormat,
0, XAUDIO2_DEFAULT_FREQ_RATIO, NULL, NULL, NULL);
}
buffer.pAudioData = (BYTE*)audioBuffer;
buffer.AudioBytes = bufferSize;
buffer.Flags = 0;
if (SUCCEEDED(hr))
{
hr = pSourceVoice->Start(0);
}
}
WaveOut::~WaveOut()
{
}
WaveOut^ WaveOut::CreateWaveOut(int bufferSize)
{
return gcnew WaveOut(bufferSize);
}
uint8_t* WaveOut::AudioCapturerBuffer()
{
if (!audioBuffer)
{
throw gcnew Exception("Audio buffer is not initialized. Did you forget to set up the audio container?");
}
return (BYTE*)audioBuffer;
}
int WaveOut::PushAudio()
{
HRESULT hr = pSourceVoice->SubmitSourceBuffer(&buffer);
if (FAILED(hr))
{
return -1;
}
return 0;
}
}
The problem I am facing is that I always have some cracking in the output. I tried to increase the interval of the timer or increased the buffer size a bit. Everytime the same result.
What am I doing wrong?
Update:
I created 3 buffers the XAudio engine can go through. The cracking got away. The missing part now is to fill the buffers at the right time from the c# part to avoid buffers with the same data.
void Render(void* param)
{
std::vector<byte> audioBuffers[BUFFER_COUNT];
size_t currentBuffer = 0;
// Get the current state of the source voice
while (BackgroundThreadRunning && pSourceVoice)
{
if (pSourceVoice)
{
pSourceVoice->GetState(&state);
}
while (state.BuffersQueued < BUFFER_COUNT)
{
std::vector<byte> resultData;
resultData.resize(DATA_SIZE);
CopyMemory(&resultData[0], pAudioBuffer, DATA_SIZE);
// Retreive the next buffer to stream from MF Music Streamer
audioBuffers[currentBuffer] = resultData;
// Submit the new buffer
XAUDIO2_BUFFER buf = { 0 };
buf.AudioBytes = static_cast<UINT32>(audioBuffers[currentBuffer].size());
buf.pAudioData = &audioBuffers[currentBuffer][0];
pSourceVoice->SubmitSourceBuffer(&buf);
// Advance the buffer index
currentBuffer = ++currentBuffer % BUFFER_COUNT;
// Get the updated state
pSourceVoice->GetState(&state);
}
Sleep(30);
}
}

XAudio2 does not copy the source data buffer at the time you submit it via SubmitSourceBuffer. You must keep that data (which is in your application memory) valid, and the buffer allocated for the entire time that XAudio2 will need to read out of it to process the data. This is done for efficiency to avoid the need for an extra copy, but puts the multi-threaded burden of keeping the memory available until it's done playing on you. That also means you can't modify the playing buffer.
Your current code is just reusing the same buffer which is causing the popping as you change the data while it's play. You can solve this with having 2 or three buffers you rotate between. A XAudio2 Source Voice has status information you can use to determine when it's done playing a buffer, or you can register for explicit callbacks which tell you when the buffer is no longer being used.
See DirectX Tool Kit for Audio and classic XAudio2 samples for examples of using XAudio2.

BASS WASAPI BPMCounter

I want to analyse my default playback device and detect the beats. I've been using the BASS WASAPI to get the FFT data of the selected device with:
int ret = BassWasapi.BASS_WASAPI_GetData(_fft, (int)BASSData.BASS_DATA_FFT2048);
Now I was using the data to generate spectrum data and display this to the user. In addition I want to detect the Beats using the BPMCounter Class from BASS. However as far as I can tell the BPMCounter.ProcessAudio() function requires a stream (which I don't get with WASAPI) in order to work. Is there a ways I can use BPMCounter with WASAPI? Would be great if someone can point me to the right direction. Thanks
Edit:
Tried this to convert the data to a stream, but without success:
int ret = BassWasapi.BASS_WASAPI_GetData(_fft, (int)BASSData.BASS_DATA_FFT2048); //get channel fft data
var chan = Bass.BASS_StreamCreate(0, 44100, BASSFlag.BASS_DEFAULT, BASSStreamProc.STREAMPROC_PUSH);
Bass.BASS_ChannelPlay(chan, false);
Bass.BASS_StreamPutData(chan, _fft, _fft.Length);
bool beat = _count.ProcessAudio(chan, true);
Debug.Write(beat);
beat is always False, however I can see at the Spectrum that the capturing of the FFT Data is corrent.

I've just started playing with this lib a few hours ago and i am still going through the examples. So my answer maybe is not what you want. For my project i also want to transform WASAPI into a stream and use it for a displaying a spectrum. What i did was to create a StreamPush, right after BASS_WASAPI initialization.
To init your WASAPI use this call and this delegate:
private InitWasapi()
{
WASAPIPROC _process = new WASAPIPROC(Process); // Delegate
bool res = BassWasapi.BASS_WASAPI_Init(_YourDeviceNumber, 0, 0, BASSWASAPIInit.BASS_WASAPI_BUFFER, 1f, 0f, _process, IntPtr.Zero);
if (!res)
{
// Do error checking
}
// This is the part you are looking for (maybe!)
// Use these flags because Wasapi needs 32-bit sample data
var info = BassWasapi.BASS_WASAPI_GetInfo();
_stream = Bass.BASS_StreamCreatePush(info.freq, info.chans, BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT, IntPtr.Zero);
BassWasapi.BASS_WASAPI_Start();
}
private int Process(IntPtr buffer, int length, IntPtr user)
{
Bass.BASS_StreamPutData(_stream, buffer, length);
return length;
}
Please note: This works, but i am still experimenting. For example i am not getting the same spectrum output as when i create the stream from the music file itself. There are some (small) differences. Maybe it's because i am using a custom EQ in Winamp for playing the same .mp3. So if anyone knows more on this subject, i would like also to hear it!

UWP AudioGraph setup not working in all devices, HRESULT: 0x88960001 thrown on AddOutgoingConnection

I'm having trouble with a AudioGraph in a UWP app. My trouble is that when I'm adding adding an Outgoing Connection to my input device node I get a Exception thrown with HRESULT: 0x88960001.
Using trial and error I figured out that the problem is with the MediaEncoding profile I set on both the input and output node... but I have now 2 settings only work on one machine and the other only works on another machine... I needed the audio to be Single Channel, 16bit sample with a 16K sample in PCM.
The code I'm using is fairly simple and was based on samples that exist online, the thing is it works only on some microphones and not others... I needed it to be generic and have always the same output so that I can input that to my service endpoint.
I left the 2 AudioEncodingProperties settings there (one of them commented out)
var result = await AudioGraph.CreateAsync(
new AudioGraphSettings(AudioRenderCategory.Media));
if (result.Status == AudioGraphCreationStatus.Success)
{
this.graph = result.Graph;
var microphone = await DeviceInformation.CreateFromIdAsync(
MediaDevice.GetDefaultAudioCaptureId(AudioDeviceRole.Default));
// Low gives us 1 channel, 16-bits per sample, 16K sample rate.
var outProfile = MediaEncodingProfile.CreateWav(AudioEncodingQuality.Low);
//outProfile.Audio = AudioEncodingProperties.CreatePcm(16000, 1, 16);
outProfile.Audio = AudioEncodingProperties.CreatePcm(44100, 2, 32);
var inProfile = MediaEncodingProfile.CreateWav(AudioEncodingQuality.Low);
//inProfile.Audio = AudioEncodingProperties.CreatePcm(16000, 1, 16);
inProfile.Audio = AudioEncodingProperties.CreatePcm(44100, 2, 32);
var outputResult = await this.graph.CreateFileOutputNodeAsync(file,
outProfile);
if (outputResult.Status == AudioFileNodeCreationStatus.Success)
{
this.outputNode = outputResult.FileOutputNode;
var inputResult = await this.graph.CreateDeviceInputNodeAsync(
MediaCategory.Speech,
inProfile.Audio,
microphone);
if (inputResult.Status == AudioDeviceNodeCreationStatus.Success)
{
//EXEPTION IS THROWN HERE
inputResult.DeviceInputNode.AddOutgoingConnection(
this.outputNode);
this.graph.Start();
}
}
}
else
{
throw new Exception("Could not create AudioGraph");
}
Thank you guys for our help

Instead of constructing the inputResult with the inProfile.Audio object, try using the default encoding properties for the graph:
var enc = graph.EncodingProperties;
CreateAudioDeviceInputNodeResult deviceInputNodeResult = await graph.CreateDeviceInputNodeAsync(MediaCategory.Media, enc, this.outputNode);
I had a similar issue and this cleared things up, and allowed the audio to be input from any device without specifying the encoding profile.