I am using a MediaPlaybackList to essentially 'stream' audio data coming in via Bluetooth as a byte[] on a fixed time gather interval. According to the MS documentation, MediaPlaybackList is 'gapless' playback between audio samples. But in my case, I have a popping sound and gap when transitioning to the next audio sample.
byte[] audioContent = new byte[audioLength];
chatReader.ReadBytes(audioContent);
MediaPlaybackItem mediaPlaybackItem = new MediaPlaybackItem(MediaSource.CreateFromStream(new MemoryStream(audioContent).AsRandomAccessStream(), "audio/mpeg"));
playbackList.Items.Add(mediaPlaybackItem);
if (_mediaPlayerElement.MediaPlayer.PlaybackSession.PlaybackState != MediaPlaybackState.Playing)
{
_mediaPlayerElement.MediaPlayer.Play(); ;
}
How can I achieve truly 'gapless' streaming audio using a method similar to this?
Also, I have tried writing my stream to a file realtime as the data comes in just to see if the popping sound or the gap is there. It plays from the file that the bytes are appended to perfectly with no pop or gap.
using (var stream = await playbackFile.OpenStreamForWriteAsync())
{
stream.Seek(0, SeekOrigin.End);
await stream.WriteAsync(audioContent, 0, audioContent.Length);
}
The MediaPlayer and in particular the MediaPlayerList is not designed to be used with a "live" audio stream. You must finish writing the data to the byte stream before adding it to the list and starting the MediaPlayer. Using the MediaPlayer is not the correct solution for this particular scenario.
A better solution would be to use the Audio Graph. The Audio Graph allows you to add input sources from actual audio endpoints so you don't need to fill the byte buffer with the streaming audio. You can then use sub-mix nodes to mix between the audio endpoint streams with no clicks or pops.
Related
I am trying to record two audio input (webcam + microphone) and one video input (webcam) via MediaCapture from C#. Just 1 audio and 1 video input works like a charm but the class itself does not allow to specify two audio input device ids.
An example:
var captureDeviceSettings = new MediaCaptureInitializationSettings
{
VideoDeviceId = videoDeviceId, #Webcam video input
AudioDeviceId = audioDeviceId, #Webcam audio input
StreamingCaptureMode = StreamingCaptureMode.AudioAndVideo,
}
I thought about using audio graph and using a submix node but, I need to specify a device ID for the media capture, the submix node does not give that. Further on, using the output device of audio graph seems not like the solution, I do not want to play microphone to default output device. Tried that but sounds horrible. I thought about creating a virtual audio device but I don't know how.
Any suggestions on that?
MediaCapture won't be able to do this, WASAPI maybe possibile, but it is not trivial to do.
Best option may be to utilize https://learn.microsoft.com/en-us/windows/win32/coreaudio/loopback-recording.
You will still have to mux in video stream though if you get loopback recording to work.
I'm making a small application which sent a part of video file to client to play on <video> element.
This is the code I'm having:
[RoutePrefix("api/video")]
public class VideoApiController : ApiController
{
[Route("")]
[HttpGet]
public async Task<HttpResponseMessage> GetVideoAsync([FromUri] GetVideoViewModel model)
{
var path = System.Web.Hosting.HostingEnvironment.MapPath($"~/wwwroot/{model.FileName}");
if (path == null || !File.Exists(path))
return new HttpResponseMessage(HttpStatusCode.NotFound);
using (var fileStream = File.OpenRead(path))
{
var bytes = new byte[model.To];
fileStream.Read(bytes, 0, bytes.Length - 1);
var memoryStream = new MemoryStream(bytes);
memoryStream.Seek(0, SeekOrigin.Begin);
var httpResponseMessage = new HttpResponseMessage(HttpStatusCode.PartialContent);
httpResponseMessage.Content = new ByteRangeStreamContent(memoryStream, new RangeHeaderValue(model.From, model.To), "video/webm");
return httpResponseMessage;
}
}
}
Then, I tested my end-point with postman.
If I selected byte range from 0 to 100000(http://localhost:64186/api/video?fileName=elephants-dream.webm&from=0&to=100000), video could be displayed on result panel:
However, when I selected byte range from 100000 to 200000(http://localhost:64186/api/video?fileName=elephants-dream.webm&from=100000&to=200000) at the first tiem, video was blank:
As I understand, video/webm uses a codec, metadata is included at some first bytes of stream.
If I want to play a part of video without play it from the beginning. What should I do ?
Thank you.
This is not very easy to do.
The first part of the stream is not just some metadata, it is all the information needed to tell the players how and what we are playing and even the color information to use in the streaming.
What you can do is read the header of the video extract the keyframes (Called Cues) then seek through your stream and start streaming the bits out with a new header. Basically, you will be building a web streaming software from scratch that will only work for this very specific video format (codec)
But here is the info to get you started and how to write the header in C++
Or you use ffmpeg and seek keyframes to jump to a good spot and let hundreds of other developers scream in frustration instead.
Microsoft has a zombie project which lets you wrap ffmpeg into c#.
Here are some other options for ffmpeg.
And how you can seek keyframes.
The other solution is to install a video streaming software to handle this.
I'd like to be able to mix the microphone output with a mp3-File, and output that to a specific device.
I got playing the mp3-File to a specific device working:
Mp3FileReader reader = new Mp3FileReader("C:\\Users\\Victor\\Music\\Musik\\Attack.mp3");
var waveOut = new WaveOut();// or WaveOutEvent()
waveOut.DeviceNumber = deviceId; //deviceId, like 0 or 1
waveOut.Init(reader);
waveOut.Play();
So would I would like to be able to do is basically send the microphone output always to specific output and overlay that output to that specific device with the sound of a mp3-file when for example a button is pressed.
Now is what I'm trying to do possible with naudio and if so how would I go about it?
Thanks!
The basic strategy is to put the audio rececived from the microphone into a BufferedWaveProvider. Then turn that into an ISampleProvider with the ToSampleProvider extension method. Now you can pass that into a MixingSampleProvider. Then play from the MixingSampleProvider. Now at any time you can mix in other audio by adding an input into the MixingSampleProvider
I'm trying to reimplement an existing Matlab 8-band equalizer GUI I created for a project last week in C#. In Matlab, songs load as a dynamic array into memory, where they can be freely manipulated and playing is as easy as sound(array).
I found the NAudio library which conveniently already has Mp3 extractors, players, and both convolution and FFT defined. I was able to open the Mp3 and read all its data into an array (though I'm not positive I'm going about it correctly.) However, even after looking through a couple of examples, I'm struggling to figure out how to take the array and write it back into a stream in such a way as to play it properly (I don't need to write to file).
Following the examples I found, I read my mp3's like this:
private byte[] CreateInputStream(string fileName)
{
byte[] stream;
if (fileName.EndsWith(".mp3"))
{
WaveStream mp3Reader = new Mp3FileReader(fileName);
songFormat = mp3Reader.WaveFormat; // songFormat is a class field
long sizeOfStream = mp3Reader.Length;
stream = new byte[sizeOfStream];
mp3Reader.Read(stream, 0, (int) sizeOfStream);
}
else
{
throw new InvalidOperationException("Unsupported Exception");
}
return stream;
}
Now I have an array of bytes presumably containing raw audio data, which I intend to eventually covert to floats so as to run through the DSP module. Right now, however, I'm simply trying to see if I can play the array of bytes.
Stream outstream = new MemoryStream(stream);
WaveFileWriter wfr = new WaveFileWriter(outstream, songFormat);
// outputStream is an array of bytes and a class variable
wfr.Write(outputStream, 0, (int)outputStream.Length);
WaveFileReader wr = new WaveFileReader(outstream);
volumeStream = new WaveChannel32(wr);
waveOutDevice.Init(volumeStream);
waveOutDevice.Play();
Right now I'm getting errors thrown in WaveFileReader(outstream) which say that it can't read past the end of the stream. I suspect that's not the only thing I'm not doing correctly. Any insights?
Your code isn't working because you never close the WaveFileWriter so its headers aren't written correctly, and you also would need to rewind the MemoryStream.
However, there is no need for writing a WAV file if you want to play back an array of byes. Just use a RawSourceWaveStream and pass in your MemoryStream.
You may also find the AudioFileReader class more suitable to your needs as it will provide the samples as floating point directly, and allow you to modify the volume.
My objective is this: to allow users of my .NET program to choose their own .wav files for sound effects. These effects may be played simultaneously. NAudio seemed like my best bet.
I decided to use WaveMixerStream32. One early challenge was that my users had .wav files of different formats, so to be able to mix them together with WaveMixerStream32, I needed to "normalize" them to a common format. I wasn't able to find a good example of this to follow so I suspect my problem is a result of my doing this part wrong.
My problem is that when some sounds are played, there are very noticeable "clicking" sounds at their end. I can reproduce this myself.
Also, my users have complained that sometimes, sounds aren't played at all, or are "scratchy" all the way through. I haven't been able to reproduce this in development but I have heard this for myself in our production environment.
I've played the user's wav files myself using Windows Media and VLC, so I know the files aren't corrupt. It must be a problem with how I'm using them with NAudio.
My NAudio version is v1.4.0.0.
Here's the code I used. To set up the mixer:
_mixer = new WaveMixerStream32 { AutoStop = false, };
_waveOutDevice = new WaveOut(WaveCallbackInfo.NewWindow())
{
DeviceNumber = -1,
DesiredLatency = 300,
NumberOfBuffers = 3,
};
_waveOutDevice.Init(_mixer);
_waveOutDevice.Play();
Surprisingly, if I set "NumberOfBuffers" to 2 here I found that sound quality was awful, with audible "ticks" occurring several times a second.
To initialize a sound file, I did this:
var sample = new AudioSample(fileName);
sample.Position = sample.Length; // To prevent the sample from playing right away
_mixer.AddInputStream(sample);
AudioSample is my class. Its constructor is responsible for the "normalization" of the wav file format. It looks like this:
private class AudioSample : WaveStream
{
private readonly WaveChannel32 _channelStream;
public AudioSample(string fileName)
{
MemoryStream memStream;
using (var fileStream = File.OpenRead(fileName))
{
memStream = new MemoryStream();
memStream.SetLength(fileStream.Length);
fileStream.Read(memStream.GetBuffer(), 0, (int)fileStream.Length);
}
WaveStream originalStream = new WaveFileReader(memStream);
var pcmStream = WaveFormatConversionStream.CreatePcmStream(originalStream);
var blockAlignReductionStream = new BlockAlignReductionStream(pcmStream);
var waveFormatConversionStream = new WaveFormatConversionStream(
new WaveFormat(44100, blockAlignReductionStream.WaveFormat.BitsPerSample, 2), blockAlignReductionStream);
var waveOffsetStream = new WaveOffsetStream(waveFormatConversionStream);
_channelStream = new WaveChannel32(waveOffsetStream);
}
Basically, the AudioSample delegates to its _channelStream object. To play an AudioSample, my code sets its "Position" to 0. This code that does this is marshalled onto the UI thread.
This almost works great. I can play multiple sounds simultaneously. Unfortunately the sound quality is bad as described above. Can anyone help me figure out why?
Some points in response to your question:
Yes, you have to have all inputs at the same sample rate before you feed them into a mixer. This is simply how digital audio works. The ACM sample rate conversion provided by WaveFormatConversion stream isn't brilliant (has no aliasing protection). What sample rates are your input files typically at?
You are passing every input through two WaveFormatConversionStreams. Only do this if it is absolutely necessary.
I'm surprised that you are getting bad sound with NumberOfBuffers=2, which is now the default in NAudio. Have you been pausing and resuming, because there was a bug where a buffer could get dropped (fixed in the latest and will be fixed for NAudio 1.4 final)
A click at the end of a file can just mean it doesn't end on a zero sample. You would have to add a fade out to eliminate this (a lot of media players do this automatically)
Whenever you are troubleshooting a bad sound issue, I always recommend using WaveFileWriter to convert your WaveStream into a WAV file (taking care not to produce a never ending file!), so you can listen to it in another media player. This allows you to quickly determine whether it is your audio processing that is causing the problem, or the playback itself.