I have some DSP effects coded in the ISampleProvider model. To apply one effect I do this and it works fine.
string filename = "C:\myaudio.mp3";
MediaFoundationReader mediaFileReader = new MediaFoundationReader(filename);
ISampleProvider sampProvider = mediaFileReader.ToSampleProvider();
ReverbSampleProvider reverbSamplr = new ReverbSampleProvider(sampProvider);
IWavePlayer waveOutDevice.Init(reverbSamplr);
waveOutDevice.Play();
How can I apply multiple effects to the same input file simultaneously?
For example, if i have a Reverb effect and Distortion effect providers, how can I chain them together to apply them at the same time to one input file?
Effects can be chained together by passing one as the "source" for the next. So if you wanted your audio to go first through a reverb, and then distortion, you might do something like this, passing the original audio into the Reverb effect, the output of the reverb into the distortion effect and then sending the distortion to the waveOut device.
var reverb = new ReverbSampleProvider(sampProvider);
var distortion = new DistortionSampleProvider(reverb);
waveOutDevice.Init(distortion);
(n.b. NAudio does not come with built in reverb/distortion effects - you must make these yourself or source them from elsewhere)
Mark's answer is correct, but that approach is a pain if you're copy and pasting things around in different orders, because you have to change the variables that you're passing through.
For example, if you start with:
var lpf = new LowPassEffectStream(input);
var reverb = new ReverbEffectStream(lpf);
var stereo = new StereoEffectStream(reverb);
var vol = new VolumeSampleProvider(stereo);
waveOutDevice.Init(vol);
And you want to swap reverb and stereo, a quick copy-paste leaves you with the input variables backwards:
var lpf = new LowPassEffectStream(input);
var stereo = new StereoEffectStream(reverb); // <--
var reverb = new ReverbEffectStream(lpf); // <--
var vol = new VolumeSampleProvider(stereo);
waveOutDevice.Init(vol);
It also makes it easy to fix a parameter but forget to fix another, e.g. fixing the stereo effect to have lpf as its input, but forgetting to fix the reverb effect. This often results in skipped effects in the chain leading to frustrated debugging when the effect appears not to work.
To make things easier and less error-prone when I'm stacking effects together and re-ordering them, I created the following helper class:
class EffectChain : ISampleProvider
{
public EffectChain(ISampleProvider source)
{
this._sourceStream = source;
}
private readonly ISampleProvider _sourceStream;
private readonly List<ISampleProvider> _chain = new List<ISampleProvider>();
public ISampleProvider Head
{
get
{
return _chain.LastOrDefault() ?? _sourceStream;
}
}
public WaveFormat WaveFormat
{
get
{
return Head.WaveFormat;
}
}
public void AddEffect(ISampleProvider effect)
{
_chain.Add(effect);
}
public int Read(float[] buffer, int offset, int count)
{
return Head.Read(buffer, offset, count);
}
}
You can use it like this:
var effectChain = new EffectChain(input);
var lpf = new LowPassEffectStream(effectChain.Head);
effectChain.AddEffect(lpf);
var stereo = new StereoEffectStream(effectChain.Head);
effectChain.AddEffect(stereo);
var reverb = new ReverbEffectStream(effectChain.Head);
effectChain.AddEffect(reverb);
var vol = new VolumeSampleProvider(effectChain.Head);
effectChain.AddEffect(vol);
waveOutDevice.Init(effectChain);
This allows you to quickly re-order effects in the chain, as each effect takes the effect chain's head as an input. If you don't add any effects it just acts as a pass-through. You could easily expand this class to have more methods for managing the contained effects if you wanted to, but as it stands it works quite cleanly.
Related
I need a way to write data with minimum allocations, as it possible when read from ReadOnlySequence.
A best way that I found is SpanWriter from side library.
But probably some one knows a better standard solution.
using System.IO.Pipelines
namespace WriteSample
{
public async Task Write()
{
var pipe = new Pipe();
var memory = pipe.Writer.GetMemory(2048);
var writer = new Writer???(memory.Span);
writer.Write((byte)0xFF);
writer.Write((long)12345);
writer.WriteBigEndian((long)12345);
await pipe.Writer.FlushAsync()
}
}
The way provided by the runtime is using BinaryPrimitives:
var span = memory.Span;
span[0] = 0xFF;
BinaryPrimitives.WriteInt64LittleEndian(span.Slice(1), 12345);
BinaryPrimitives.WriteInt64BigEndian(span.Slice(9), 12345);
This does have the disadvantage of meaning you need to keep track of where in the span you've written to so far, unlike e.g. BinaryWriter. I don't know of any types in the runtime to help with this, so you'll probably have to write your own, if you care.
Something like:
public ref struct SpanWriter
{
private Span<byte> span;
public SpanWriter(Span<byte> span) => this.span = span;
public void WriteInt64BigEndian(long value)
{
BinaryPrimitives.WriteInt64BigEndian(span, value);
span = span.Slice(8);
}
}
I'm using NAudio to convert & trim some audio files, and I'm trying to add a fade-out to the last few seconds of each file.
I have checked this question, this, and this, but all the answers are talking about playing the wav file with fade, while I need to actually write that fade to an output file.
So, is there any way to do this using NAudio? If not, I'm open to other suggestions.
Edit: This is what I've tried:
private void PerformFadeOut(string inputPath, string outputPath)
{
WaveFileReader waveSource = new WaveFileReader(inputPath);
ISampleProvider sampleSource = waveSource.ToSampleProvider();
OffsetSampleProvider fadeOutSource = new OffsetSampleProvider(sampleSource);
// Assume the length of the audio file is 122 seconds.
fadeOutSource.SkipOver = TimeSpan.FromSeconds(120); // Hard-coded values for brevity
// Two seconds fade
var fadeOut = new FadeInOutSampleProvider(fadeOutSource);
fadeOut.BeginFadeOut(2000);
Player = new WaveOut();
Player.Init(fadeOut);
Player.Play();
}
When I play the audio after applying the fade using Player.Play() -as shown in the method above-, it works perfectly as expected, and I can hear the fade. Now, I would like to export this result to an output WAV file.
I tried doing that by adding the following line:
WaveFileWriter.CreateWaveFile(outputPath, waveSource);
However, the output file doesn't have any fade applied to it. So, what am I missing here?
Okay, let's wrap everything up in case someone encounters the same issue in the future:
With the great help of #yms, I managed to write the fade to a file by using:
WaveFileWriter.CreateWaveFile(outputPath, new SampleToWaveProvider(fadeOut));
But that caused the wave writer to only write the last two seconds which makes sense, so I tried using the DelayFadeOutSampleProvider class instead of FadeInOutSampleProvider. With that I was able to write the whole file, but it ended up causing the fading to start in a wrong position (it's more obvious when saving though. Not when playing).
I generated a 10 seconds wav file with Audacity, and used the following method for testing:
private static void PerformFadeOut(string inputPath, string outputPath, bool playNoSave = false)
{
WaveFileReader waveSource = new WaveFileReader(inputPath);
ISampleProvider sampleSource = waveSource.ToSampleProvider();
// Two seconds fade
var fadeOut = new DelayFadeOutSampleProvider(sampleSource);
fadeOut.BeginFadeOut(8000, 2000);
if(playNoSave)
{
// Here the fade is played exactly where expected (#00:08)
var player = new WaveOut();
player.Init(fadeOut);
player.Play();
}
else
{
// But when saving, the fade is applied #00:04!
WaveFileWriter.CreateWaveFile(outputPath, new SampleToWaveProvider(fadeOut));
}
}
Here's the file, before & after writing the fade-out:
As shown above, the fade-out doesn't start at the right position.
After some investigation in the DelayFadeOutSampleProvider, I found a bug in the Read method, so I modified it like this:
public int Read(float[] buffer, int offset, int count)
{
int sourceSamplesRead = source.Read(buffer, offset, count);
lock (lockObject)
{
if (fadeOutDelaySamples > 0)
{
int oldFadeOutDelayPos = fadeOutDelayPosition;
fadeOutDelayPosition += sourceSamplesRead / WaveFormat.Channels;
if (fadeOutDelayPosition > fadeOutDelaySamples)
{
int normalSamples = (fadeOutDelaySamples - oldFadeOutDelayPos) * WaveFormat.Channels;
int fadeOutSamples = (fadeOutDelayPosition - fadeOutDelaySamples) * WaveFormat.Channels;
// apply the fade-out only to the samples after fadeOutDelayPosition
FadeOut(buffer, offset + normalSamples, fadeOutSamples);
fadeOutDelaySamples = 0;
fadeState = FadeState.FadingOut;
return sourceSamplesRead;
}
}
if (fadeState == FadeState.FadingIn)
{
FadeIn(buffer, offset, sourceSamplesRead);
}
else if (fadeState == FadeState.FadingOut)
{
FadeOut(buffer, offset, sourceSamplesRead);
}
else if (fadeState == FadeState.Silence)
{
ClearBuffer(buffer, offset, count);
}
}
return sourceSamplesRead;
}
And now everything works just fine.
Here's my fork of the whole class if someone is interested, and I already asked the author (#mark-heath) to update the original gist with this fix.
You original code was using the original waveSource as input, which is why you had no fade.
The following alternative uses the fadeOut object:
WaveFileWriter.CreateWaveFile16(outputPath, fadeOut);
The signature of CreateWaveFile16 would be:
public static void CreateWaveFile16(string filename, ISampleProvider sourceProvider)
This can be seen in the source code of the class WaveFileWriter.
Another alternative is to use the class SampleToWaveProvider and covert your fadeOut object into an IWaveProvider, and that allows you to use the regular CreateWaveFile method.
WaveFileWriter.CreateWaveFile(outputPath, new SampleToWaveProvider(fadeOut))
As you know, in all cases your output file will only contain the last k seconds corresponding to the fadeout, a different class is needed if you want the whole file with a fadeout.
I'm experimenting with OpenCL (through Cloo's C# interface). To do so, I'm experimenting with the customary matrix-multiplication-on-the-GPU. The problem is, during my speed tests, the application crashes. I'm trying to be efficient regarding the the re-allocation of various OpenCL objects, and I'm wondering if I'm botching something in doing so.
I'll put the code in this question, but for a bigger picture, you can get the code from github here: https://github.com/kwende/ClooMatrixMultiply
My main program does this:
Stopwatch gpuSw = new Stopwatch();
gpuSw.Start();
for (int c = 0; c < NumberOfIterations; c++)
{
float[] result = gpu.MultiplyMatrices(matrix1, matrix2, MatrixHeight, MatrixHeight, MatrixWidth);
}
gpuSw.Stop();
So I'm basically doing the call NumberOfIterations times, and timing the average execution time.
Within the MultiplyMatrices call, the first time through, I call Initialize to setup all the objects I'm going to reuse:
private void Initialize()
{
// get the intel integrated GPU
_integratedIntelGPUPlatform = ComputePlatform.Platforms.Where(n => n.Name.Contains("Intel")).First();
// create the compute context.
_context = new ComputeContext(
ComputeDeviceTypes.Gpu, // use the gpu
new ComputeContextPropertyList(_integratedIntelGPUPlatform), // use the intel openCL platform
null,
IntPtr.Zero);
// the command queue is the, well, queue of commands sent to the "device" (GPU)
_commandQueue = new ComputeCommandQueue(
_context, // the compute context
_context.Devices[0], // first device matching the context specifications
ComputeCommandQueueFlags.None); // no special flags
string kernelSource = null;
using (StreamReader sr = new StreamReader("kernel.cl"))
{
kernelSource = sr.ReadToEnd();
}
// create the "program"
_program = new ComputeProgram(_context, new string[] { kernelSource });
// compile.
_program.Build(null, null, null, IntPtr.Zero);
_kernel = _program.CreateKernel("ComputeMatrix");
}
I then enter the main body of my function (the part that will be executed NumberOfIterations times).
ComputeBuffer<float> matrix1Buffer = new ComputeBuffer<float>(_context,
ComputeMemoryFlags.ReadOnly| ComputeMemoryFlags.CopyHostPointer,
matrix1);
_kernel.SetMemoryArgument(0, matrix1Buffer);
ComputeBuffer<float> matrix2Buffer = new ComputeBuffer<float>(_context,
ComputeMemoryFlags.ReadOnly | ComputeMemoryFlags.CopyHostPointer,
matrix2);
_kernel.SetMemoryArgument(1, matrix2Buffer);
float[] ret = new float[matrix1Height * matrix2Width];
ComputeBuffer<float> retBuffer = new ComputeBuffer<float>(_context,
ComputeMemoryFlags.WriteOnly | ComputeMemoryFlags.CopyHostPointer,
ret);
_kernel.SetMemoryArgument(2, retBuffer);
_kernel.SetValueArgument<int>(3, matrix1WidthMatrix2Height);
_kernel.SetValueArgument<int>(4, matrix2Width);
_commandQueue.Execute(_kernel,
new long[] { 0 },
new long[] { matrix2Width ,matrix1Height },
null, null);
unsafe
{
fixed (float* retPtr = ret)
{
_commandQueue.Read(retBuffer,
false, 0,
ret.Length,
new IntPtr(retPtr),
null);
_commandQueue.Finish();
}
}
The third or fourth time through (it's somewhat random, which hints at memory access issues), the program crashes. Here is my kernel (I'm sure there are faster implementations, but right now my goal is just to get something working without blowing up):
kernel void ComputeMatrix(
global read_only float* matrix1,
global read_only float* matrix2,
global write_only float* output,
int matrix1WidthMatrix2Height,
int matrix2Width)
{
int x = get_global_id(0);
int y = get_global_id(1);
int i = y * matrix2Width + x;
float value = 0.0f;
// row y of matrix1 * column x of matrix2
for (int c = 0; c < matrix1WidthMatrix2Height; c++)
{
int m1Index = y * matrix1WidthMatrix2Height + c;
int m2Index = c * matrix2Width + x;
value += matrix1[m1Index] * matrix2[m2Index];
}
output[i] = value;
}
Ultimately the goal here is to better understand the zero-copy features of OpenCL (since I'm using Intel's integrated GPU). I have been having trouble getting it to work and so wanted to step back a bit to see if I understood even more basic things...apparently I don't as I can't get even this to work without blowing up.
The only other thing I can think of is it's how I'm pinning the pointer to send it to the .Read() function. But I don't know of an alternative.
Edit:
For what it's worth, I updated the last part of code (the read code) to this, and it still crashes:
_commandQueue.ReadFromBuffer(retBuffer, ref ret, false, null);
_commandQueue.Finish();
Edit #2
Solution found by huseyin tugrul buyukisik (see comment below).
Upon placing
matrix1Buffer.Dispose();
matrix2Buffer.Dispose();
retBuffer.Dispose();
At the end, it all worked fine.
OpenCl resources like buffers, kernels and commandqueues should be released after other resources that they are bound-to are released. Re-creating without releasing depletes avaliable slots quickly.
You have been re-creating arrays in a method of gpu and that was the scope of opencl buffers. When it finishes, GC cannot track opencl's unmanaged memory areas and that causes leaks, which makes crashes.
Many opencl implementations use C++ bindings which needs explicit release commands by C#, Java and other environments.
Also the set-argument part is not needed many times when repeated kernel executions use exact same buffer order as parameters of kernel.
I've tried using something along the lines of:
public async Task ReadParagraph(int id)
{
var mp = (App.Current as App).mediaPlayer;
var ns = (App.Current as App).nextStream;
readerPosition = id;
var tb = Items.OfType<TextBlock>();
if (readerPosition < tb.Count())
{
if (synth == null)
synth = new SpeechSynthesizer();
synth.Voice = SpeechSynthesizer.DefaultVoice;
var str = await synth.SynthesizeTextToStreamAsync(tb.ElementAt(readerPosition).Text);
var ms = MediaSource.CreateFromStream(str, str.ContentType);
mp.Source = ms;
mp.PlaybackSession.PlaybackRate = 1.3;
mp.Play();
}
}
And in the App.xml.cs file:
private async void MediaPlayer_MediaEnded(MediaPlayer sender, object args)
{
await currentStory.Commands.ReadParagraph(Reader.Current.readerPosition + 1);
}
In theory this works. The problem is, there's 3s+ pause between each synth streams. On WP8.1, with MediaElement it worked perfectly, but it seems like MediaPlayer, with its Background Audio Playback capability seems to be working on different terms.
Also, is there any other way to change MediaPlayer's default playback rate instead of changing ebvery file's separately? It doesn't seem to work 100% of the time the way I have it set up currently.
I've also been thinking about initializing the entire text as a List<SpeechSynthesisStream>, but this seems like it would take up a lot of memory with long texts.
What'd the best option to handle this?
I'm using TuesPechkin for my web application, which I'm testing locally on IIS with VS2013. The user clicks a button and the page's current HTML is saved to a PDF file, which is then emailed out. This process is going to be run regularly as the site's data changes.
When converter.Convert(document) first runs, it converts without problem. Every subsequent attempt, however, results in the process hanging and me needing to restart VS.
Below is some default code I've been using to test.
public void MakePDF()
{
var document = new HtmlToPdfDocument
{
GlobalSettings =
{
ProduceOutline = true,
DocumentTitle = "Pretty Websites",
PaperSize = PaperKind.A4, // Implicit conversion to PechkinPaperSize
Margins =
{
All = 1.375,
Unit = TuesPechkin.Unit.Centimeters
}
},
Objects = {
new ObjectSettings { HtmlText = "<h1>Pretty Websites</h1><p>This might take a bit to convert!</p>" }
}
};
IConverter converter =
new ThreadSafeConverter(
new PdfToolset(
new Win32EmbeddedDeployment(
new TempFolderDeployment())));
byte[] result = converter.Convert(document);
}
Can anyone point me in the right direction on this? Most of my troubleshooting so far has led to some discussions on threading and pooling, but no concrete code solutions for running TuesPechkin more than once.
Have you tried the ThreadSafeConverter? The StandardConverter is only suitable for small console apps.
IConverter converter =
new ThreadSafeConverter(
new RemotingToolset<PdfToolset>(
new Win32EmbeddedDeployment(
new TempFolderDeployment())));
byte[] result = converter.Convert(document);
Note that you should keep the converter somewhere static, or as a singleton instance (as mentioned here).
Since this app on IIS, could get singleton converter, and use RemotingToolset
var toolSet = new RemotingToolset<PdfToolset>(winAnyCpuEmbeddedDeployment);
// Then
using TuesPechkin.Wkhtmltox.AnyCPU;
...
var converter = PDFHelper.Factory.GetConverter();
var result = converter.Convert(This.Document);
Reference : https://github.com/tloy1966/TuesPechkin