Develop Reference

Play real-time sound buffer using C# Media.SoundPlayer

I have developed a system in which a C# program receives sound buffers (byte arrays) from another subsystem. It is supposed to play the incoming buffers continuously. I searched in the web and I decided to use SoundPlayer. It works perfectly in the offline mode (play the buffers after receiving them all). However, I have a problem in the real-time mode.
In the real-time mode the program at first waits for a number of buffer arrays (for example 200) to receive and accumulate them. Then it adds a wav header and plays it. However, after that for each next 200 arrays it plays repeatedly the first buffer.
I have read following pages:
Play wav/mp3 from memory
https://social.msdn.microsoft.com/Forums/vstudio/en-US/8ac2847c-3e2f-458c-b8ff-533728e267e0/c-problems-with-mediasoundplayer?forum=netfxbcl
and according to their suggestions I implemented my code as follow:
public class MediaPlayer
{
System.Media.SoundPlayer soundPlayer;
public MediaPlayer(byte[] buffer)
{
byte[] headerPlusBuffer = AddWaveHeader(buffer, false, 1, 16, 8000, buffer.Length / 2); //add wav header to the **first** buffer
MemoryStream memoryStream = new MemoryStream(headerPlusBuffer, true);
soundPlayer = new System.Media.SoundPlayer(memoryStream);
}
public void Play()
{
soundPlayer.PlaySync();
}
public void Play(byte[] buffer)
{
soundPlayer.Stream.Seek(0, SeekOrigin.Begin);
soundPlayer.Stream.Write(buffer, 0, buffer.Length);
soundPlayer.PlaySync();
}
}
I use it like this:
MediaPlayer _mediaPlayer;
if (firstBuffer)
{
_mediaPlayer = new MediaPlayer(dataRaw);
_mediaPlayer.Play();
}
else
{
_mediaPlayer.Play(dataRaw);
}
Each time _mediaPlayer.Play(dataRaw) is called, the first buffer is played again and again; the dataRaw is updated though.
I appreciate your help.

NAudio WasapiLoopbackCapture intermittent sound

Part 1
I have some NAudio related code
private void InitAudioOut(DateTime dtNow)
{
_pathOut = string.Format(BaseDirectory + #"\({0:HH-mm-ss dd-MM-yyyy} OUT).wav", dtNow);
_waveOut = new WasapiLoopbackCapture();
_waveOut.DataAvailable += WaveOutDataAvailable;
_waveOut.RecordingStopped += WaveOutRecordStopped;
_waveOutFileStream = new WaveFileWriter(_pathOut, _waveOut.WaveFormat);
_waveOut.StartRecording();
}
With this initialization of the sound recording process I have the followind WaveOutDataAvailable method:
private void WaveOutDataAvailable(object sender, WaveInEventArgs e)
{
var buf= e.Buffer;
_waveOutFileStream.Write(buf, 0, buf.Length);
_waveOutFileStream.Flush();
}
The sound in the resulting file is intermittent and slow, like having "blank" sections between the sound chunks, any ideas are appreciated.
End of part 1
Part 2
There is another version of this code where i'm trying to convert the WAV stream to mp3 stream on the fly and then write it to file, it looks like this:
private void InitAudioIn(DateTime dtNow)
{
_pathIn = string.Format(BaseDirectory + #"\({0:HH-mm-ss dd-MM-yyyy} IN).mp3", dtNow);
_waveIn = new WaveInEvent();
_waveIn.WaveFormat = new WaveFormat(44100, 2);
_waveIn.DataAvailable += WaveInDataAvailable;
_waveIn.RecordingStopped += WaveInRecordStopped;
_waveInFileStream = File.Create(_pathIn);
_waveIn.StartRecording();
}
With the WaveInDataAvailable method as follows:
private void WaveInDataAvailable(object sender, WaveInEventArgs e)
{
var wavToMp3Buffer = ConvertWavToMp3(e.Buffer, _waveIn.WaveFormat);
_waveInFileStream.Write(wavToMp3Buffer, 0, wavToMp3Buffer.Length);
_waveInFileStream.Flush();
}
The ConvertWavToMp3 method:
public byte[] ConvertWavToMp3(byte[] wavContent, WaveFormat waveFormat)
{
using (var baseMemoryStream = new MemoryStream())
using (var wavToMp3Writer = new LameMP3FileWriter(baseMemoryStream, waveFormat, 64))
{
wavToMp3Writer.Write(wavContent, 0, wavContent.Length);
wavToMp3Writer.Flush();
return baseMemoryStream.ToArray();
}
}
If i don't try to convert it to MP3 and just write it as a WAV file that it's absolutely fine, but if i try the MP3 conversion through the ConvertWavToMp3 method then the sound gets slow and intermittent, what is wrong with this implementation?

First part, you are making an invalid assumption about the buffer length being the same as the number of valid bytes in the buffer. Try:
private void WaveOutDataAvailable(object sender, WaveInEventArgs e)
{
_waveOutFileStream.Write(e.Buffer, 0, e.BytesRecorded);
}
Let the output stream handle flushing automatically. Trying to force data to disk like that will either not work or in some cases can cause unexpected results like partial block writes that can interfere with your data. Flush at the end of the recording, not during.
As to the second part...
Your code is creating a file that is the concatenation of a series of MP3 files, one for each buffer passed to your WaveInDataAvailable method, and including all the blank space at the end of those buffers. Of course it's not going to play back properly.
If you want to write an MP3 then do it directly. Make your _waveInFileStream an instance of LameMP3FileWriter and let it handle the work itself. Not only is this going to produce a much more useful output but you save yourself a lot of inefficient messing around with setting up and tearing down the encoder for every data block you receive.

Storing a wav file in an array

I need a fast method to store all samples of a wav file in an array. I am currently working around this problem by playing the music and storing the values from the Sample Provider, but this is not very elegant.
From the NAudio Demo I have the Audioplayer Class with this Method:
private ISampleProvider CreateInputStream(string fileName)
{
if (fileName.EndsWith(".wav"))
{
fileStream = OpenWavStream(fileName);
}
throw new InvalidOperationException("Unsupported extension");
}
var inputStream = new SampleChannel(fileStream, true);
var sampleStream = new NotifyingSampleProvider(inputStream);
SampleRate = sampleStream.WaveFormat.SampleRate;
sampleStream.Sample += (s, e) => { aggregator.Add(e.Left); }; // at this point the aggregator gets the current sample value, while playing the wav file
return sampleStream;
}
I want to skip this progress of getting the sample values while playing the file, instead I want the values immediatly without waiting till the end of the file. Basically like the wavread command in matlab.

Use AudioFileReader to read the file. This will automatically convert to IEEE float samples. Then repeatedly call the Read method to read a block of samples into a float[] array.

How do I save files to hard disk in a separate thread?

I've a camera and I'm reading the images in real time into an array.
I'm applying some algorithm to the image and displaying it. Then I get the next image and display it as well. So I'm streaming images from the camera to the display. However I also want to save images to hard disk once I've displayed them. I tried using the main thread but everything slowed down too much.
I then tried using ThreadPool (see code below). This doesn't slow the display down but I've found the images aren't being saved properly. It looks like they are not in the expected order and after about 50 images have been saved the subsequent image data looks garbled. I'm guessing too many threads are being started.
Is there a better way to do this? I think I only need one thread to save the images. Maybe some kind of queue that saves each image sequentially. Just as long as its done in the background and doesn't slow down the display. If someone could post a code snippet that would be fantastic.
short[] image1 = new short[20000];
while(streaming)
{
ReadImageFromCamera(ref image1)
ImageData data;
data.fileName = imageNumber;
data.image = image1;
ThreadPool.QueueUserWorkItem(WriteImageToFile, data); // Send the writes to the queue
}
private void WriteImageToFile(object imageData) {
try {
ImageData data = (ImageData)imageData;
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bf = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
string fName = myDirectory + #"/" + Convert.ToString(data.fileName) + #".spe";
using (Stream myStream = new FileStream(fName, FileMode.Create)) {
bf.Serialize(myStream, data.image);
}
}
catch (Exception) { }
}

I think you should avoid starting a new thread for each particular image. Since you have got just a single hard drive and store all files into the single directory, you should use just one disk writer thread. Then I'd recommend using some concurrent queue to transfer jobs from camera thread to disk writer thread. I don't show "code snippet" because this is not a thing you can write in good quality in a few lines of code.
Also you definitely must somewhere put 'new short[20000]' for each image, otherwise it is overwritten by next image before you save it to disk.
Also, I would expect that it is sufficient to write files in the main thread, because Windows uses concurrent techniques (mainly disk cache) automatically when you write data to disk. Are you sure that your hardware is fast enough to write all those data in real time?

When dealing with threads, ordering is no longer in your control. The thread pool can choose to schedule the threads in any order it likes. If you need things to happen sequentially in a specific order, threading does not make much sense anyway.
Regarding the corrupted images, it looks like the short[] image1 instance is being passed around. It is unclear what happens inside ReadImageFromCamera, but since you pass a pre-initialized array into it, chances are that the method will use that array and simply copy data into it (even though the ref keyword indicates that it might create a brand new array instance and assign that instead). Then you pass that array instance to WriteImageToFile on a separate thread.
Meanwhile, in parallell, you get the next image. Now you have a scenario where ReadImageFromCamera might write data into the array at the same time as WriteImageToFile is storing the data on disk. There you have your corrupted image. This can be avoided by passing a new array instance to WriteImageToFile:
ReadImageFromCamera(ref image1)
ImageData data;
data.fileName = imageNumber;
data.image = (short[])image1.Clone(); // create a new array instance, so that
// the next call to ReadImageFromCamera
// will not corrupt the data
ThreadPool.QueueUserWorkItem(WriteImageToFile, data);
Still, as has been mentioned by Al Kepp, since you have only one hard drive, launching many threads might not be your best option here. You could look into having one long-running separate thread for storing data on disk, and putting the images into some sort of queue that the storage thread picks up data from and writes to disk. This comes with its own set of problems dealing with concurrency, limiting the size of the queue and what not.

You need to create a distinct buffer for the thread to read data from, otherwise main thread will overwrite it when you dump it to a file. The way you are doing it seems to copy only references (image1 in particular).
So:
ThreadPool.QueueUserWorkItem(WriteImageToFile, data);
instead of data you'll send in a deep copy of data. Since it seems you are already doing it - but in the worker thread - you just need to move the copy before sending it.
HTH

You have to check before thinking about threads if the speed of a normal disk will be sufficient for your task as you may create images faster than writing to the disk. If the image creation is faster than writing I would look at using a Memory disk, but then you need to calculate if the size is sufficient until you stop the camera, so that you can write to the normal disk overnight.
If you use .NET 4.0 I would suggest that you use Concurrent queue together with a normal thread (as the thread will run until the program finishes).

Quick and dirty way is starting single new thread and work with global class members - the new thread should be able to access them while the main thread will update them.
First of all, have these lines outside of any function:
private List<ImageData> arrGlobalData = new List<ImageData>();
private bool keepWritingImages = true;
Now change the code in the "main" thread to this:
short[] image1 = new short[20000];
ThreadPool.QueueUserWorkItem(WriteImageToFile, null);
while(streaming)
{
ReadImageFromCamera(ref image1)
ImageData data = new ImageData();
data.fileName = imageNumber;
data.image = image1;
arrGlobalData.Add(data);
}
keepWritingImages = false;
And finally have such function for the new thread:
private void WriteImageToFile(object imageData)
{
while (keepWritingImages)
{
if (arrGlobalData.Count > 0)
{
ImageData data = arrGlobalData[0];
try
{
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bf = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
string fName = myDirectory + #"/" + Convert.ToString(data.fileName) + #".spe";
using (Stream myStream = new FileStream(fName, FileMode.Create))
{
bf.Serialize(myStream, data.image);
}
}
catch
{
}
finally
{
arrGlobalData.Remove(data);
}
}
Thread.Sleep(10);
}
}

You can do following.
public class AsyncFileWriter
{
private readonly FileStream fs;
private readonly AsyncCallback callback;
public Action FinishedCallback;
private IAsyncResult result;
private class AsyncState
{
public FileStream Fs;
}
private void WriteCore(IAsyncResult ar)
{
if (result != null)
{
FileStream stream = ((AsyncState)ar.AsyncState).Fs;
stream.EndWrite(result);
if (this.FinishedCallback != null)
{
FinishedCallback();
}
}
}
public AsyncFileWriter(FileStream fs, Action finishNotification)
{
this.fs = fs;
callback = new AsyncCallback(WriteCore);
this.FinishedCallback = finishNotification;
}
public AsyncFileWriter(FileStream fs)
: this(fs, null)
{
}
public void Write(Byte[] data)
{
result = fs.BeginWrite(data, 0, data.Length, callback, new AsyncState() { Fs = fs });
}
}
Later you can consume it as.
static void Main(string[] args)
{
FileStream fs = File.Create("D:\\ror.txt");
ManualResetEvent evt = new ManualResetEvent(false);
AsyncFileWriter writer = new AsyncFileWriter(fs, () =>
{
Console.Write("Write Finished");
evt.Set();
}
);
byte[] bytes = File.ReadAllBytes("D:\\test.xml");//Getting some random bytes
writer.Write(bytes);
evt.WaitOne();
Console.Write("Write Done");
}

Will a large System.IO.MemoryStream result in my application's memory usage increasing dramatically?

I am building a library that allows a user to download files from a URL. One of the options I am considering is letting the user specify the expected MD5 checksum for the file; the library's GetFile(string url) function ensures that the checksum for the downloaded stream matches the one specified by the user.
Being aware that the NetworkStream returned by HttpWebResponse.GetResponseStream() is not seekable, I found a way to duplicate the Stream thanks to the answers to this question: How can I read an Http response stream twice in C#?. Before I went any farther though, I wanted to figure out what the memory implications of this duplication would be; unfortunately, multiple searches on Google and MSDN have came to naught.
The library imposes no restriction on the size of the file to be downloaded. My question is, if the user selects a 2GB file, is the MemoryStream implementation in .NET 2.0 smart enough to use the PageFile and RAM efficiently enough that the system doesn't start to crawl due to a VM crunch? Also, Jon Skeet's comment on another question gave me something to think about - he averred that even after disposing a MemoryStream, the memory is not 100% free'ed. How and when can I ensure that the memory is actually released? Will it be released based on the system's requirements (and necessity)?
Thanks,
Manoj

You're saving it to a file, right? Why not save it chunk by chunk, updating a hash as you go, and then just check the hash at the end? I don't think you need to read the response twice, nor buffer it. As another answer points out, that would fail when you got over 1GB anyway.
Don't forget that as well as the current size of the MemoryStream, any time it has to grow you'll end up with (temporarily) the new array plus the old array in memory at the same time. Of course that wouldn't be a problem if you knew the content length beforehand, but it would still be nicer to just write it to disk and hash as you go.

MemoryStream is backed by an array. Even if you have a 64 bit OS this isn't going to work for more than 1GB as the framework won't allocate a larger array.

Afaik the CLR managed heap will not allocate anything bigger than 2 GB and the MemoryStream is backed by a live, contigous, byte[]. Large Object Heap doesn't allocations handle over 2GB, not even on x64.
But to store an entire file in memory just to compute a hash seems pretty low tech. You can compute the hash as you receive the bytes, chunk by chunk. After each IO completion you can hash the received bytes, then submit the write to the file. At the end, you have the hash computed and the file uploaded, huraay.
BTW, If you seek code to manipulate files, steer clear of any sample that contains the words ReadToEnd...
class Program
{
private static AutoResetEvent done = new AutoResetEvent(false);
private static AsyncCallback _callbackReadStream;
private static AsyncCallback _callbackWriteFile;
static void Main(string[] args)
{
try
{
_callbackReadStream = new AsyncCallback(CallbackReadStream);
_callbackWriteFile = new AsyncCallback(CallbackWriteFile);
string url = "http://...";
WebRequest request = WebRequest.Create(url);
request.Method = "GET";
request.BeginGetResponse(new AsyncCallback(
CallbackGetResponse), request);
done.WaitOne();
}
catch (Exception e)
{
Console.Error.WriteLine(e.Message);
}
}
private class State
{
public Stream ReponseStream { get; set; }
public HashAlgorithm Hash { get; set; }
public Stream FileStream { get; set; }
private byte[] _buffer = new byte[16379];
public byte[] Buffer { get { return _buffer; } }
public int ReadBytes { get; set; }
public long FileLength {get;set;}
}
static void CallbackGetResponse(IAsyncResult ar)
{
try
{
WebRequest request = (WebRequest)ar.AsyncState;
WebResponse response = request.EndGetResponse(ar);
State s = new State();
s.ReponseStream = response.GetResponseStream();
s.FileStream = new FileStream("download.out"
, FileMode.Create
, FileAccess.Write
, FileShare.None);
s.Hash = HashAlgorithm.Create("MD5");
s.ReponseStream.BeginRead(
s.Buffer
, 0
, s.Buffer.Length
, _callbackReadStream
, s);
}
catch (Exception e)
{
Console.Error.WriteLine(e.Message);
done.Set();
}
}
private static void CallbackReadStream(IAsyncResult ar)
{
try
{
State s = (State)ar.AsyncState;
s.ReadBytes = s.ReponseStream.EndRead(ar);
s.Hash.ComputeHash(s.Buffer, 0, s.ReadBytes);
s.FileStream.BeginWrite(
s.Buffer
, 0
, s.ReadBytes
, _callbackWriteFile
, s);
}
catch (Exception e)
{
Console.Error.WriteLine(e.Message);
done.Set();
}
}
static private void CallbackWriteFile(IAsyncResult ar)
{
try
{
State s = (State)ar.AsyncState;
s.FileStream.EndWrite(ar);
s.FileLength += s.ReadBytes;
if (0 != s.ReadBytes)
{
s.ReponseStream.BeginRead(
s.Buffer
, 0
, s.Buffer.Length
, _callbackReadStream
, s);
}
else
{
Console.Out.Write("Downloaded {0} bytes. Hash(base64):{1}",
s.FileLength, Convert.ToBase64String(s.Hash.Hash));
done.Set();
}
}
catch (Exception e)
{
Console.Error.WriteLine(e.Message);
done.Set();
}
}
}

I'm pretty sure you'll get an OutOfMemoryException. Easy way to try is try to read a DVD ISO image or something into memory using a memory stream. If you can read the whole thing, then you should be fine. If you get an exception, well, there you go.