I'm trying to resample the WasapiLoopbackCapture's output from my soundcards 44100Hz, 16bit, 2 channel waveformat to a 16000Hz, 16bit, 1 channel format for later use in a System.Net.Sockets.NetworkStream (I want to write the converted bytes to the network stream)
But I have no idea how to start even!
I'm really new to signal processing and I've tried searching for tutorials but I just can't wrap my head around how to do this.
Here's what I got so far:
void StartRecording()
{
capture = new WasapiLoopbackCapture(device); // device is an audiodevice picked from the user. speaker, headphones etc
capture.ShareMode = NAudio.CoreAudioApi.AudioClientShareMode.Shared;
capture.DataAvailable += capture_DataAvailable;
capture.RecordingStopped += capture_RecordingStopped;
capture.StartRecording();
}
void capture_DataAvailable(object sender, WaveInEventArgs e)
{
outputStream.Write(e.Buffer, 0, e.BytesRecorded); // here I want to output audio to the NetworkStream.
// But I have to resample it first, which brings me to my question.
}
What I basically want to know is how I can get a byte-array which has been resampled and ready to be shipped off to the other side of the networkstream!
Any suggestions are really appreciated! Thank you in advance.
NAudio includes several different resamplers - one that uses ACM (WaveFormatConversionStream), one that uses Media Foundation (MediaFoundationResampler) and one written entirely in managed code (WdlResamplingSampleProvider). I discuss each of these in this post.
For your case, you want to do "input driven" resampling, where you know how many input samples you have and just want to pass them into the resampler. This can be a bit trickier than output driven resampling in NAudio. I've written another post about how to do input driven resampling using AcmStream. A similar technique can be used with the Media Foundation resampler transform or the WDL Resampling Provider but I'm afraid I don't have a code sample of that available yet.
Related
How could I display/preview input video using Black Magic Design DecklinkAPI.dll? I could get video frame by frame but I do not know how could I display the frame in the Form/Window. I could implement IDeckLinkInputCallback:
void IDeckLinkInputCallback.VideoInputFrameArrived(IDeckLinkVideoInputFrame video,
IDeckLinkAudioInputPacket audio)
{
IntPtr pData;
video.GetBytes(out pData);
// What should I do to get the preview?
System.Runtime.InteropServices.Marshal.ReleaseComObject(video);
}
Another way I see is to implement IDeckLinkScreenPreviewCallback:
void IDeckLinkScreenPreviewCallback.DrawFrame(IDeckLinkVideoFrame theFrame)
{
// Constructor: m_ph = new CDeckLinkDX9ScreenPreviewHelper();
m_ph.SetFrame(theFrame);
// Should I use this method instead to get the preview?
System.Runtime.InteropServices.Marshal.ReleaseComObject(theFrame);
}
There more complete code samples but they are still missing the important bit of code:
blackmagic SDK in c#.
BMD Decklink SDK documentation could be found here.
Thanks.
If you are using DeckLink SDK to capture video, then it is your responsibility to convert frames into format, which can be consumed by presentation API (GDI, GDI+, DirectShow, Media Foundation etc) - certain effort is expected here since you typically capture in non-RGB pixel format, and possibly at non-standard stride.
Alternatively, you can use DeckLink DirectShow video capture source which captures video and makes it available as a feed compatible with DirectShow API. You can use standard components to preview and process video. You can build and control DirectShow graphs in C# through DirectShow.NET library.
I have played with the Naudio examples, and am able to amplify a WAV file that is opened in using the "WaveFileStream" function. (Code Example: AudioPlaybackPanel) This works fine:
I add a variable declaration, so I can access the channel later:
SampleChannel waveFromFile;
And in the existing function, I set it:
private ISampleProvider CreateInputStream(string fileName)
{
...
this.fileWaveStream = plugin.CreateWaveStream(fileName);
var waveChannel = new SampleChannel(this.fileWaveStream, true);
waveFromFile = waveChannel;
...
}
Then I add an AMPLIFY button, and this works just as I expect:
float ampFactor = 1.0f;
private void ampButton_Click(object sender, EventArgs e)
{
ampFactor += 2;
if (ampFactor >= 9.0f)
ampFactor = 1.0f;
waveFromFile.Volume = ampFactor;
}
But how can I do this when the input is not a WAV file, but instead is a microphone?
If I am looking at the NAudio examples, and try to add this code to the "RecordingPanel" demo, and it is being ignored -- meaning I put the value in the Volume, but there is no change.
Is it possible to amplify the audio coming in from the microphone? And, if so, can someone show a sample code snippet? I've looked online, but I can't seem to find it.
To take advantage of SampleChannel's ability to modify samples, you'd actually need to pull the audio through the SampleChannel. To do that, you could put the recorded audio into a BufferedWaveProvider and then put that into SampleChannel. Then you'd need to make sure you pulled enough audio back out of the SampleChannel and into the WaveFileWriter so it doesn't fill up. You may also want to use a SampleToWaveProvider16 if you wanted a 16 bit WAV file.
I have a task of generating video from a sequence of images in my app and while searching for that i found out that FFMPEG is able to do that.Can anyone provide me any tutorial or link which can guide me in right direction.I am a newbiew in this so please help appropriately guys.
Will appreciate any sort of help
I could not manage to get the above example to work. However I did find another library that works amazingly well once. Try via NuGet "accord.extensions.imaging.io", then I wrote the following little function:
private void makeAvi(string imageInputfolderName, string outVideoFileName, float fps = 12.0f, string imgSearchPattern = "*.png")
{ // reads all images in folder
VideoWriter w = new VideoWriter(outVideoFileName,
new Accord.Extensions.Size(480, 640), fps, true);
Accord.Extensions.Imaging.ImageDirectoryReader ir =
new ImageDirectoryReader(imageInputfolderName, imgSearchPattern);
while (ir.Position < ir.Length)
{
IImage i = ir.Read();
w.Write(i);
}
w.Close();
}
It reads all images from a folder and makes a video out of them.
If you want to make it nicer you could probably read the image dimensions instead of hard coding, but you got the point.
http://electron.mit.edu/~gsteele/ffmpeg/
http://www.codeproject.com/Articles/7388/A-Simple-C-Wrapper-for-the-AviFile-Library
http://ffmpeg.org/ffmpeg.html -> search for For creating a video from many images:
All the links are from this question on SO
Link related to FFMPEG in .net (From this question);
FFMpeg.NET
FFMpeg-Sharp
FFLib.NET
http://ivolo.mit.edu/post/Convert-Audio-Video-to-Any-Format-using-C.aspx
Other resources
Expression Encoder
VLC
I bit late but I have made a tutorial on how I solved my similar problem if you did not succeed yet: Image sequence to video stream?
Same question asked here.
The answer there points to here, which is not exactly what you're doing, but is easily configurable to do the job.
I know there are lots of question like this.
But I don't want to use the Windows media encoder 9 because it's a problem to get one, and then it is no longer supported.
I know that, one possibility is to capture lots of screenshots and create a video with ffmpeg but I don't want use third party executables.
Is there are a .net only solution?
the answer is the Microsoft Expression Encoder. It is according to my opinion the easiest way to record something on vista and windows 7
private void CaptureMoni()
{
try
{
Rectangle _screenRectangle = Screen.PrimaryScreen.Bounds;
_screenCaptureJob = new ScreenCaptureJob();
_screenCaptureJob.CaptureRectangle = _screenRectangle;
_screenCaptureJob.ShowFlashingBoundary = true;
_screenCaptureJob.ScreenCaptureVideoProfile.FrameRate = 20;
_screenCaptureJob.CaptureMouseCursor = true;
_screenCaptureJob.OutputScreenCaptureFileName = string.Format(#"C:\test.wmv");
if (File.Exists(_screenCaptureJob.OutputScreenCaptureFileName))
{
File.Delete(_screenCaptureJob.OutputScreenCaptureFileName);
}
_screenCaptureJob.Start();
}
catch(Exception e) { }
}
Edit Based on Comment Feedback:
A developer by the name baSSiLL has graciously shared a repository that has a screen recording c# library as well as a sample project in c# that shows how it can be used to capture the screen and mic.
Starting a screen capture using the sample code is as straight forward as:
recorder = new Recorder(_filePath,
KnownFourCCs.Codecs.X264, quality,
0, SupportedWaveFormat.WAVE_FORMAT_44S16, true, 160);
_filePath is the path of the file I'd like to save the video to.
You can pass in a variety of codecs including AVI, MotionJPEG, X264, etc. In the case of x264 I had to install the codec on my machine first but AVI works out of the box.
Quality only comes into play when using AVI or MotionJPEG. The x264 codec manages its own quality settings.
The 0 above is the audio device I'd like to use. The Default is zero.
It currently supports 2 wave formats. 44100 at 16bit either stereo or mono.
The true parameter indicates that I want the audio encoded into mp3 format. I believe this is required when choosing x264 as the uncompressed audio combined in a .mp4 file would not play back for me.
The 160 is the bitrate at which to encode the audio.
~~~~~
To stop the recording you just
recorder.Dispose();
recorder = null;
Everything is open source so you can edit the recorder class and change dimensions, frames per second, etc.
~~~~
To get up and running with this library you will need to either download or pull from the github / codeplex libraries below. You can also use NuGet:
Install-Package SharpAvi
Original Post:
Sharp AVI:
https://sharpavi.codeplex.com/
or
https://github.com/baSSiLL/SharpAvi
There is a sample project within that library that has a great screen recorder in it along with a menu for settings/etc.
I found Screna first from another answer on this StackoverFlow question but I ran into a couple issues involving getting Mp3 Lame encoder to work correctly. Screna is a wrapper for SharpAVI. I found by removing Screna and going off of SharpAvi's sample I had better luck.
My objective is this: to allow users of my .NET program to choose their own .wav files for sound effects. These effects may be played simultaneously. NAudio seemed like my best bet.
I decided to use WaveMixerStream32. One early challenge was that my users had .wav files of different formats, so to be able to mix them together with WaveMixerStream32, I needed to "normalize" them to a common format. I wasn't able to find a good example of this to follow so I suspect my problem is a result of my doing this part wrong.
My problem is that when some sounds are played, there are very noticeable "clicking" sounds at their end. I can reproduce this myself.
Also, my users have complained that sometimes, sounds aren't played at all, or are "scratchy" all the way through. I haven't been able to reproduce this in development but I have heard this for myself in our production environment.
I've played the user's wav files myself using Windows Media and VLC, so I know the files aren't corrupt. It must be a problem with how I'm using them with NAudio.
My NAudio version is v1.4.0.0.
Here's the code I used. To set up the mixer:
_mixer = new WaveMixerStream32 { AutoStop = false, };
_waveOutDevice = new WaveOut(WaveCallbackInfo.NewWindow())
{
DeviceNumber = -1,
DesiredLatency = 300,
NumberOfBuffers = 3,
};
_waveOutDevice.Init(_mixer);
_waveOutDevice.Play();
Surprisingly, if I set "NumberOfBuffers" to 2 here I found that sound quality was awful, with audible "ticks" occurring several times a second.
To initialize a sound file, I did this:
var sample = new AudioSample(fileName);
sample.Position = sample.Length; // To prevent the sample from playing right away
_mixer.AddInputStream(sample);
AudioSample is my class. Its constructor is responsible for the "normalization" of the wav file format. It looks like this:
private class AudioSample : WaveStream
{
private readonly WaveChannel32 _channelStream;
public AudioSample(string fileName)
{
MemoryStream memStream;
using (var fileStream = File.OpenRead(fileName))
{
memStream = new MemoryStream();
memStream.SetLength(fileStream.Length);
fileStream.Read(memStream.GetBuffer(), 0, (int)fileStream.Length);
}
WaveStream originalStream = new WaveFileReader(memStream);
var pcmStream = WaveFormatConversionStream.CreatePcmStream(originalStream);
var blockAlignReductionStream = new BlockAlignReductionStream(pcmStream);
var waveFormatConversionStream = new WaveFormatConversionStream(
new WaveFormat(44100, blockAlignReductionStream.WaveFormat.BitsPerSample, 2), blockAlignReductionStream);
var waveOffsetStream = new WaveOffsetStream(waveFormatConversionStream);
_channelStream = new WaveChannel32(waveOffsetStream);
}
Basically, the AudioSample delegates to its _channelStream object. To play an AudioSample, my code sets its "Position" to 0. This code that does this is marshalled onto the UI thread.
This almost works great. I can play multiple sounds simultaneously. Unfortunately the sound quality is bad as described above. Can anyone help me figure out why?
Some points in response to your question:
Yes, you have to have all inputs at the same sample rate before you feed them into a mixer. This is simply how digital audio works. The ACM sample rate conversion provided by WaveFormatConversion stream isn't brilliant (has no aliasing protection). What sample rates are your input files typically at?
You are passing every input through two WaveFormatConversionStreams. Only do this if it is absolutely necessary.
I'm surprised that you are getting bad sound with NumberOfBuffers=2, which is now the default in NAudio. Have you been pausing and resuming, because there was a bug where a buffer could get dropped (fixed in the latest and will be fixed for NAudio 1.4 final)
A click at the end of a file can just mean it doesn't end on a zero sample. You would have to add a fade out to eliminate this (a lot of media players do this automatically)
Whenever you are troubleshooting a bad sound issue, I always recommend using WaveFileWriter to convert your WaveStream into a WAV file (taking care not to produce a never ending file!), so you can listen to it in another media player. This allows you to quickly determine whether it is your audio processing that is causing the problem, or the playback itself.