To provide a little bit of context. I am trying to output live audio from a camera in my c# application. After doing some research it seems pretty obvious to do it in a c++ managed dll. I chose the XAudio2 api because it should be pretty easy to implement and use with dynamic audio content.
So the idea is to create the XAudio device in c++ with an empty buffer and push in the audio from the c# code side. The audio chunks are pushed every 50ms because I want to keep the latency as small as possible.
// SampleRate = 44100; Channels = 2; BitPerSample = 16;
var blockAlign = (Channels * BitsPerSample) / 8;
var avgBytesPerSecond = SampleRate * blockAlign;
var avgBytesPerMillisecond = avgBytesPerSecond / 1000;
var bufferSize = avgBytesPerMillisecond * Time;
_sampleBuffer = new byte[bufferSize];
Everytime the timer runs it gets the pointer of the audio buffer, reads the data from the audio, copies the data to the pointer and calls the PushAudio method.
I am also using a stopwatch to check how long the processing took and calculate the interval again for the timer to include the processing time.
private void PushAudioChunk(object sender, ElapsedEventArgs e)
{
unsafe
{
_pushAudioStopWatch.Reset();
_pushAudioStopWatch.Start();
var audioBufferPtr = Output.AudioCapturerBuffer();
FillBuffer(_sampleBuffer);
Marshal.Copy(_sampleBuffer, 0, (IntPtr)audioBufferPtr, _sampleBuffer.Length);
Output.PushAudio();
_pushTimer.Interval = Time - _pushAudioStopWatch.ElapsedMilliseconds;
_pushAudioStopWatch.Stop();
DIX.Log.WriteLine("Push audio took: {0}ms", _pushAudioStopWatch.ElapsedMilliseconds);
}
}
This is the implementation of the c++ part.
Regarding to the documentation on msdn I created a XAudio2 device and added the MasterVoice and SourceVoice. The buffer is empty at first because the c# part is responsible to push in the audio data.
namespace Audio
{
using namespace System;
template <class T> void SafeRelease(T **ppT)
{
if (*ppT)
{
(*ppT)->Release();
*ppT = NULL;
}
}
WAVEFORMATEXTENSIBLE wFormat;
XAUDIO2_BUFFER buffer = { 0 };
IXAudio2* pXAudio2 = NULL;
IXAudio2MasteringVoice* pMasterVoice = NULL;
IXAudio2SourceVoice* pSourceVoice = NULL;
WaveOut::WaveOut(int bufferSize)
{
audioBuffer = new Byte[bufferSize];
wFormat.Format.wFormatTag = WAVE_FORMAT_PCM;
wFormat.Format.nChannels = 2;
wFormat.Format.nSamplesPerSec = 44100;
wFormat.Format.wBitsPerSample = 16;
wFormat.Format.nBlockAlign = (wFormat.Format.nChannels * wFormat.Format.wBitsPerSample) / 8;
wFormat.Format.nAvgBytesPerSec = wFormat.Format.nSamplesPerSec * wFormat.Format.nBlockAlign;
wFormat.Format.cbSize = 0;
wFormat.SubFormat = KSDATAFORMAT_SUBTYPE_PCM;
HRESULT hr = XAudio2Create(&pXAudio2, 0, XAUDIO2_DEFAULT_PROCESSOR);
if (SUCCEEDED(hr))
{
hr = pXAudio2->CreateMasteringVoice(&pMasterVoice);
}
if (SUCCEEDED(hr))
{
hr = pXAudio2->CreateSourceVoice(&pSourceVoice, (WAVEFORMATEX*)&wFormat,
0, XAUDIO2_DEFAULT_FREQ_RATIO, NULL, NULL, NULL);
}
buffer.pAudioData = (BYTE*)audioBuffer;
buffer.AudioBytes = bufferSize;
buffer.Flags = 0;
if (SUCCEEDED(hr))
{
hr = pSourceVoice->Start(0);
}
}
WaveOut::~WaveOut()
{
}
WaveOut^ WaveOut::CreateWaveOut(int bufferSize)
{
return gcnew WaveOut(bufferSize);
}
uint8_t* WaveOut::AudioCapturerBuffer()
{
if (!audioBuffer)
{
throw gcnew Exception("Audio buffer is not initialized. Did you forget to set up the audio container?");
}
return (BYTE*)audioBuffer;
}
int WaveOut::PushAudio()
{
HRESULT hr = pSourceVoice->SubmitSourceBuffer(&buffer);
if (FAILED(hr))
{
return -1;
}
return 0;
}
}
The problem I am facing is that I always have some cracking in the output. I tried to increase the interval of the timer or increased the buffer size a bit. Everytime the same result.
What am I doing wrong?
Update:
I created 3 buffers the XAudio engine can go through. The cracking got away. The missing part now is to fill the buffers at the right time from the c# part to avoid buffers with the same data.
void Render(void* param)
{
std::vector<byte> audioBuffers[BUFFER_COUNT];
size_t currentBuffer = 0;
// Get the current state of the source voice
while (BackgroundThreadRunning && pSourceVoice)
{
if (pSourceVoice)
{
pSourceVoice->GetState(&state);
}
while (state.BuffersQueued < BUFFER_COUNT)
{
std::vector<byte> resultData;
resultData.resize(DATA_SIZE);
CopyMemory(&resultData[0], pAudioBuffer, DATA_SIZE);
// Retreive the next buffer to stream from MF Music Streamer
audioBuffers[currentBuffer] = resultData;
// Submit the new buffer
XAUDIO2_BUFFER buf = { 0 };
buf.AudioBytes = static_cast<UINT32>(audioBuffers[currentBuffer].size());
buf.pAudioData = &audioBuffers[currentBuffer][0];
pSourceVoice->SubmitSourceBuffer(&buf);
// Advance the buffer index
currentBuffer = ++currentBuffer % BUFFER_COUNT;
// Get the updated state
pSourceVoice->GetState(&state);
}
Sleep(30);
}
}
XAudio2 does not copy the source data buffer at the time you submit it via SubmitSourceBuffer. You must keep that data (which is in your application memory) valid, and the buffer allocated for the entire time that XAudio2 will need to read out of it to process the data. This is done for efficiency to avoid the need for an extra copy, but puts the multi-threaded burden of keeping the memory available until it's done playing on you. That also means you can't modify the playing buffer.
Your current code is just reusing the same buffer which is causing the popping as you change the data while it's play. You can solve this with having 2 or three buffers you rotate between. A XAudio2 Source Voice has status information you can use to determine when it's done playing a buffer, or you can register for explicit callbacks which tell you when the buffer is no longer being used.
See DirectX Tool Kit for Audio and classic XAudio2 samples for examples of using XAudio2.
Related
I have been playing around with SharpDX.XAudio2 for a few days now, and while things have been largely positive (the odd software quirk here and there) the following problem has me completely stuck:
I am working in C# .NET using VS2015.
I am trying to play multiple sounds simultaneously.
To do this, I have made:
- Test.cs: Contains main method
- cSoundEngine.cs: Holds XAudio2, MasteringVoice, and sound management methods.
- VoiceChannel.cs: Holds a SourceVoice, and in future any sfx/ related data.
cSoundEngine:
List<VoiceChannel> sourceVoices;
XAudio2 engine;
MasteringVoice master;
public cSoundEngine()
{
engine = new XAudio2();
master = new MasteringVoice(engine);
sourceVoices = new List<VoiceChannel>();
}
public VoiceChannel AddAndPlaySFX(string filepath, double vol, float pan)
{
/**
* Set up and start SourceVoice
*/
NativeFileStream fileStream = new NativeFileStream(filepath, NativeFileMode.Open, NativeFileAccess.Read);
SoundStream soundStream = new SoundStream(fileStream);
SourceVoice source = new SourceVoice(engine, soundStream.Format);
AudioBuffer audioBuffer = new AudioBuffer()
{
Stream = soundStream.ToDataStream(),
AudioBytes = (int)soundStream.Length,
Flags = SharpDX.XAudio2.BufferFlags.EndOfStream
};
//Make voice wrapper
VoiceChannel voice = new VoiceChannel(source);
sourceVoices.Add(voice);
//Volume
source.SetVolume((float)vol);
//Play sound
source.SubmitSourceBuffer(audioBuffer, soundStream.DecodedPacketsInfo);
source.Start();
return voice;
}
Test.cs:
cSoundEngine engine = new cSoundEngine();
total = 6;
for (int i = 0; i < total; i++)
{
string filepath = System.IO.Directory.GetParent(System.IO.Directory.GetCurrentDirectory()).Parent.FullName + #"\Assets\Planet.wav";
VoiceChannel sfx = engine.AddAndPlaySFX(filepath, 0.1, 0);
}
Console.Read(); //Input anything to end play.
There is currently nothing worth showing in VoiceChannel.cs - it holds 'SourceVoice source' which is the one parameter sent in the constructor!
Everything is fine and well running with up to 5 sounds (total = 5). All you hear is the blissful drone of Planet.wav. Any higher than 5 however causes the console to freeze for ~5 seconds, then close (likely a c++ error which debugger can't handle). Sadly no error message for us to look at or anything.
From testing:
- Will not crash as long as you do not have more than 5 running sourcevoices.
- Changing sample rate does not seem to help.
- Setting inputChannels for master object to a different number makes no difference.
- MasteringVoice seems to say the max number of inputvoices is 64.
- Making each sfx play from a different wav file makes no difference.
- Setting the volume for sourcevoices and/or master makes no difference.
From the XAudio2 API Documentation I found this quote: 'XAudio2 removes the 6-channel limit on multichannel sounds, and supports multichannel audio on any multichannel-capable audio card. The card does not need to be hardware-accelerated.'. This is the closest I have come to finding something that mentions this problem.
I am not well experienced with programming sfx and a lot of this is very new to me, so feel free to call me an idiot where appropriate but please try and explain things in layman terms.
Please, if you have any ideas or answers they would be greatly appreciated!
-Josh
As Chuck has suggested, I have created a databank which holds the .wav data, and I just reference the single data store with each buffer. This has improved the sound limit up to 20 - however this has not fixed the problem as a whole, likely because I have not implemented this properly.
Implementation:
class SoundDataBank
{
/**
* Holds a single byte array for each sound
*/
Dictionary<eSFX, Byte[]> bank;
string curdir => Directory.GetParent(Directory.GetCurrentDirectory()).Parent.FullName;
public SoundDataBank()
{
bank = new Dictionary<eSFX, byte[]>();
bank.Add(eSFX.planet, NativeFile.ReadAllBytes(curdir + #"\Assets\Planet.wav"));
bank.Add(eSFX.base1, NativeFile.ReadAllBytes(curdir + #"\Assets\Base.wav"));
}
public Byte[] GetSoundData(eSFX sfx)
{
byte[] output = bank[sfx];
return output;
}
}
In SoundEngine we create a SoundBank object (initialised in SoundEngine constructor):
SoundDataBank soundBank;
public VoiceChannel AddAndPlaySFXFromStore(eSFX sfx, double vol)
{
/**
* sourcevoice will be automatically added to MasteringVoice and engine in the constructor.
*/
byte[] buffer = soundBank.GetSoundData(sfx);
MemoryStream memoryStream = new MemoryStream(buffer);
SoundStream soundStream = new SoundStream(memoryStream);
SourceVoice source = new SourceVoice(engine, soundStream.Format);
AudioBuffer audioBuffer = new AudioBuffer()
{
Stream = soundStream.ToDataStream(),
AudioBytes = (int)soundStream.Length,
Flags = SharpDX.XAudio2.BufferFlags.EndOfStream
};
//Make voice wrapper
VoiceChannel voice = new VoiceChannel(source, engine, MakeOutputMatrix());
//Volume
source.SetVolume((float)vol);
//Play sound
source.SubmitSourceBuffer(audioBuffer, soundStream.DecodedPacketsInfo);
source.Start();
sourceVoices.Add(voice);
return voice;
}
Following this implementation now lets me play up to 20 sound effects - but NOT because we are playing from the soundbank. Infact, even running the old method for sound effects now gets up to 20 sfx instances.
This has improved up to 20 because we have done NativeFile.ReadAllBytes(curdir + #"\Assets\Base.wav") in the constructor for the SoundBank.
I suspect NativeFile is holding a store of loaded file data, so you regardless of whether you run the original SoundEngine.AddAndPlaySFX() or SoundEngine.AddAndPlaySFXFromStore(), they are both running from memory?
Either way, this has quadrupled the limit from before, so this has been incredibly useful - but requires further work.
Why is Windows documentation so lacking ?? It seems impossible to find an example of how this method is supposed to work StartPreviewToCustomSinkAsync
What I am trying to do is get a preview image from a video source (via MediaCapture) but can't understand how this method works (especially what the second parameter, IMediaExtension, is supposed to be/do).
Any chance any of you can help me with this ?
If all you need is to get a preview frame every now and then, there is a sample on the Microsoft github page that is relevant, although they target Windows 10. You may be interested in migrating your project to get this functionality.
GetPreviewFrame: This sample will capture preview frames as opposed to full-blown photos. Once it has a preview frame, it can edit the pixels on it.
Here is the relevant part:
private async Task GetPreviewFrameAsSoftwareBitmapAsync()
{
// Get information about the preview
var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;
// Create the video frame to request a SoftwareBitmap preview frame
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);
// Capture the preview frame
using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
{
// Collect the resulting frame
SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;
// Add a simple green filter effect to the SoftwareBitmap
EditPixels(previewFrame);
}
}
private unsafe void EditPixels(SoftwareBitmap bitmap)
{
// Effect is hard-coded to operate on BGRA8 format only
if (bitmap.BitmapPixelFormat == BitmapPixelFormat.Bgra8)
{
// In BGRA8 format, each pixel is defined by 4 bytes
const int BYTES_PER_PIXEL = 4;
using (var buffer = bitmap.LockBuffer(BitmapBufferAccessMode.ReadWrite))
using (var reference = buffer.CreateReference())
{
// Get a pointer to the pixel buffer
byte* data;
uint capacity;
((IMemoryBufferByteAccess)reference).GetBuffer(out data, out capacity);
// Get information about the BitmapBuffer
var desc = buffer.GetPlaneDescription(0);
// Iterate over all pixels
for (uint row = 0; row < desc.Height; row++)
{
for (uint col = 0; col < desc.Width; col++)
{
// Index of the current pixel in the buffer (defined by the next 4 bytes, BGRA8)
var currPixel = desc.StartIndex + desc.Stride * row + BYTES_PER_PIXEL * col;
// Read the current pixel information into b,g,r channels (leave out alpha channel)
var b = data[currPixel + 0]; // Blue
var g = data[currPixel + 1]; // Green
var r = data[currPixel + 2]; // Red
// Boost the green channel, leave the other two untouched
data[currPixel + 0] = b;
data[currPixel + 1] = (byte)Math.Min(g + 80, 255);
data[currPixel + 2] = r;
}
}
}
}
}
And declare this outside your class:
[ComImport]
[Guid("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
unsafe interface IMemoryBufferByteAccess
{
void GetBuffer(out byte* buffer, out uint capacity);
}
And of course, your project will have to allow unsafe code for all of this to work.
Have a closer look at the sample to see how to get all the details. Or, to have a walkthrough, you can watch the camera session from a recent //build/ conference, which includes a little bit of a walkthrough through some camera samples.
Alternatively, if you're bound to 8.1, you can look into the Lumia Imaging SDK, which can notify you when there's a new preview frame available.
There are much of examples on GitHub. If you are developing for Windows Phone 8.1 - samples are here
According this example recording looking like this:
private StspMediaSinkProxy mediaSink;
mediaSink = new StspMediaSinkProxy();
MediaEncodingProfile encodingProfile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Qvga);
var mfExtension = await mediaSink.InitializeAsync(encodingProfile.Audio, encodingProfile.Video);
await mediaCapture.StartRecordToCustomSinkAsync(encodingProfile, mfExtension);
So, you can understand how to get IMediaExtension from MediaEncodingProfile from this example.
You haven't post any code, but making Preview should be similar to code I have provide
I start learning Netduino short time ago. At now, I want to use it with MS5803 30BAR sensor. This Components communicate with I2C Protocol. I learned this protocol a little bit but not enough.
I wrote introducing of code. When I came main code, I did not do anything. My code is below.
Can anybody help about this matter? I will be so pleased :)
public class Program
{
public static void Main()
{
// Configuration of MS5803 30BA
I2CDevice i2c = new I2CDevice(new I2CDevice.Configuration(0x76>>1, 400));
byte[] read = new byte[1];
I2CDevice.I2CTransaction[] i2cTx = new I2CDevice.I2CTransaction[1];
i2cTx[0] = I2CDevice.CreateReadTransaction(read);
// ???
}
}
It looks like you're missing the I2C.Execute call. Without knowing anything about the device you're communicating with this will at least start the transmission.
Try to add this line after you create the read transaction.
i2c.Execute(i2cTX[0],500);
byte[] returnByte = new byte[3];
var readX = new I2CDevice.I2CTransaction[] {I2CDevice.CreateReadTransaction(returnByte) };
int executed = 0;
I2CDevice i2c = new I2CDevice(new I2CDevice.Configuration(0x76, 400));
executed = i2c.Execute(readX, 400);
if (executed == 0)
{
//Debug.Print("Read FAIL!");
}
else
{
//Debug.Print("Read SUCCESS!");
}
//throw new Exception("I2C transaction failed");
//you will need to do some bit shifting with the readX array to get your values.
}
Here an excellent document on netMF i2c: https://www.ghielectronics.com/docs/12/i2c
The device data sheet:
http://www.amsys-sensor.eu/sheets/amsys.en.ms5803_30ba.pdf
I want to capture and process data using Bass.NET using the BASS_ChannelGetData method. The examples I've seen that use this play audio files through the Bass.NET library and then sample that, however I wish to sample the data my soundcard outputs, so that I can capture and process audio data from third party audio players, for example Spotify.
Bass.BASS_ChannelGetData(handle, buffer, (int)BASSData.BASS_DATA_FFT256);
How would I get a handle that would allow me to process this data?
Bass.BASS_RecordInit do return a handle but if you look closely at the documentation they do use it only for playing (actually starting) the record channel. Their code sample uses a callback to retrieve the audio samples.
Take a look at Bass.BASS_RecordStart Method documentation.
private RECORDPROC _myRecProc; // make it global, so that the GC can not remove it
private int _byteswritten = 0;
private byte[] _recbuffer; // local recording buffer
...
if ( Bass.BASS_RecordInit(-1) )
{
_myRecProc = new RECORDPROC(MyRecording);
int recHandle = Bass.BASS_RecordStart(44100, 2, BASSFlag.BASS_RECORD_PAUSE, _myRecProc, IntPtr.Zero);
...
// start recording
Bass.BASS_ChannelPlay(recHandle, false);
}
...
private bool MyRecording(int handle, IntPtr buffer, int length, IntPtr user)
{
bool cont = true;
if (length > 0 && buffer != IntPtr.Zero)
{
// increase the rec buffer as needed
if (_recbuffer == null || _recbuffer.Length < length)
_recbuffer = new byte[length];
// copy from managed to unmanaged memory
Marshal.Copy(buffer, _recbuffer, 0, length);
_byteswritten += length;
// write to file
...
// stop recording after a certain amout (just to demo)
if (_byteswritten > 800000)
cont = false; // stop recording
}
return cont;
}
Note that you should be able to use BASS_ChannelGetData inside that callback instead of Marshal.Copy.
Did you mean resample instead of sample ? If so then BassMix class will handle that job.
Ok I have been at this for 2 days and need help with this last part.
I have a Microsoft LifeCam Cinema camera and I use the .NET DirectShowLib to capture the video stream. Well actually I use WPFMediaKit, but I am in the source code of that dealing directly with the direct show library now.
What I have working is:
- View the video output of the camera
- Record the video output of the camera in ASF or AVI (the only 2 MediaType's supported with ICaptureGraphBuilder2)
The problem is: I can save it as a .avi. This works fine and at a resolution of 1280x720 but it saves the file in RAW output. Meaning it is about 50-60MB per second. Way too high.
Or I can switch it to .asf and it outputs a WMV, but when I do this the capture and the output both go to resolution 320x240.
In WPFMediaKit there is a function I changed because apparently with Microsoft LifeCam Cinema cameras a lot of people have this problem. So instead of creating or changing the AMMediaType you iterate through and then use that to call SetFormat.
///* Make the VIDEOINFOHEADER 'readable' */
var videoInfo = new VideoInfoHeader();
int iCount = 0, iSize = 0;
videoStreamConfig.GetNumberOfCapabilities(out iCount, out iSize);
IntPtr TaskMemPointer = Marshal.AllocCoTaskMem(iSize);
AMMediaType pmtConfig = null;
for (int iFormat = 0; iFormat < iCount; iFormat++)
{
IntPtr ptr = IntPtr.Zero;
videoStreamConfig.GetStreamCaps(iFormat, out pmtConfig, TaskMemPointer);
videoInfo = (VideoInfoHeader)Marshal.PtrToStructure(pmtConfig.formatPtr, typeof(VideoInfoHeader));
if (videoInfo.BmiHeader.Width == DesiredWidth && videoInfo.BmiHeader.Height == DesiredHeight)
{
///* Setup the VIDEOINFOHEADER with the parameters we want */
videoInfo.AvgTimePerFrame = DSHOW_ONE_SECOND_UNIT / FPS;
if (mediaSubType != Guid.Empty)
{
int fourCC = 0;
byte[] b = mediaSubType.ToByteArray();
fourCC = b[0];
fourCC |= b[1] << 8;
fourCC |= b[2] << 16;
fourCC |= b[3] << 24;
videoInfo.BmiHeader.Compression = fourCC;
// pmtConfig.subType = mediaSubType;
}
/* Copy the data back to unmanaged memory */
Marshal.StructureToPtr(videoInfo, pmtConfig.formatPtr, true);
hr = videoStreamConfig.SetFormat(pmtConfig);
break;
}
}
/* Free memory */
Marshal.FreeCoTaskMem(TaskMemPointer);
DsUtils.FreeAMMediaType(pmtConfig);
if (hr < 0)
return false;
return true;
When that was implemented I could finally view the captured video as 1280x720 as long as I set the SetOutputFilename to a MediaType.Avi.
If I set it to a MediaType.Asf it goes to 320x240 and the output is the same.
Or the AVI works and outputs in the correct format but does so in RAW video, hence a very large file size. I have attempted to add a compressor to the graph but with no luck, this is far out of my experience.
I am looking for 1 of 2 answers.
Recording the ASF at 1280x720
Adding a compressor to the graph so that the filesize of my outputted AVI is small.
I figured this out. So I am posting it here for any other poor soul who passes by wondering why it doesn't work.
Download the source of the WPFMediaKit, you are going to need to change some code.
Go to Folder DirectShow > MediaPlayers and open up VideoCapturePlayer.cs
Find the function SetVideoCaptureParameters and replace it with this:
/// <summary>
/// Sets the capture parameters for the video capture device
/// </summary>
private bool SetVideoCaptureParameters(ICaptureGraphBuilder2 capGraph, IBaseFilter captureFilter, Guid mediaSubType)
{
/* The stream config interface */
object streamConfig;
/* Get the stream's configuration interface */
int hr = capGraph.FindInterface(PinCategory.Capture,
MediaType.Video,
captureFilter,
typeof(IAMStreamConfig).GUID,
out streamConfig);
DsError.ThrowExceptionForHR(hr);
var videoStreamConfig = streamConfig as IAMStreamConfig;
/* If QueryInterface fails... */
if (videoStreamConfig == null)
{
throw new Exception("Failed to get IAMStreamConfig");
}
///* Make the VIDEOINFOHEADER 'readable' */
var videoInfo = new VideoInfoHeader();
int iCount = 0, iSize = 0;
videoStreamConfig.GetNumberOfCapabilities(out iCount, out iSize);
IntPtr TaskMemPointer = Marshal.AllocCoTaskMem(iSize);
AMMediaType pmtConfig = null;
for (int iFormat = 0; iFormat < iCount; iFormat++)
{
IntPtr ptr = IntPtr.Zero;
videoStreamConfig.GetStreamCaps(iFormat, out pmtConfig, TaskMemPointer);
videoInfo = (VideoInfoHeader)Marshal.PtrToStructure(pmtConfig.formatPtr, typeof(VideoInfoHeader));
if (videoInfo.BmiHeader.Width == DesiredWidth && videoInfo.BmiHeader.Height == DesiredHeight)
{
///* Setup the VIDEOINFOHEADER with the parameters we want */
videoInfo.AvgTimePerFrame = DSHOW_ONE_SECOND_UNIT / FPS;
if (mediaSubType != Guid.Empty)
{
int fourCC = 0;
byte[] b = mediaSubType.ToByteArray();
fourCC = b[0];
fourCC |= b[1] << 8;
fourCC |= b[2] << 16;
fourCC |= b[3] << 24;
videoInfo.BmiHeader.Compression = fourCC;
// pmtConfig.subType = mediaSubType;
}
/* Copy the data back to unmanaged memory */
Marshal.StructureToPtr(videoInfo, pmtConfig.formatPtr, true);
hr = videoStreamConfig.SetFormat(pmtConfig);
break;
}
}
/* Free memory */
Marshal.FreeCoTaskMem(TaskMemPointer);
DsUtils.FreeAMMediaType(pmtConfig);
if (hr < 0)
return false;
return true;
}
Now that will sort out your screen display at what ever desired resolution you want, provided that your camera supports it.
Next you will soon figure out that this new correct capture you have isnt applied when writing the video to disk.
Since the ICaptureBuilder2 method only supports Avi and Asf (which is wmv) you need to set your mediatype to one of them.
hr = graphBuilder.SetOutputFileName(MediaSubType.Asf, this.m_fileName, out mux, out sink);
You will find that line in the SetupGraph function.
Asf will only output in 320x240, yet the Avi will output in the desired resolution, but uncompressed (meaning 50-60MB per second for a 1280x720 video feed), which is too high.
So that leaves you with 2 options
Figure out how to add a encoder (compression filter) to the Avi output
Figure out how to change the WMV profile
I tried 1, with no success. Mainly due to the fact this is my first time working with DirectShow and only just grasp the meaning of graphs.
But I was successful with #2 and here is how I did it.
Special thanks to (http://www.codeproject.com/KB/audio-video/videosav.aspx) I pulled out the needed code from here.
Create a new class in the same folder called WMLib.cs and place the following in it
Download the demo project from http://www.codeproject.com/KB/audio-video/videosav.aspx and copy and paste the WMLib.cs into your project (change the namespace as necessary)
Create a function in the VideoCapturePlayer.cs class
/// <summary>
/// Configure profile from file to Asf file writer
/// </summary>
/// <param name="asfWriter"></param>
/// <param name="filename"></param>
/// <returns></returns>
public bool ConfigProfileFromFile(IBaseFilter asfWriter, string filename)
{
int hr;
//string profilePath = "test.prx";
// Set the profile to be used for conversion
if ((filename != null) && (File.Exists(filename)))
{
// Load the profile XML contents
string profileData;
using (StreamReader reader = new StreamReader(File.OpenRead(filename)))
{
profileData = reader.ReadToEnd();
}
// Create an appropriate IWMProfile from the data
// Open the profile manager
IWMProfileManager profileManager;
IWMProfile wmProfile = null;
hr = WMLib.WMCreateProfileManager(out profileManager);
if (hr >= 0)
{
// error message: The profile is invalid (0xC00D0BC6)
// E.g. no <prx> tags
hr = profileManager.LoadProfileByData(profileData, out wmProfile);
}
if (profileManager != null)
{
Marshal.ReleaseComObject(profileManager);
profileManager = null;
}
// Config only if there is a profile retrieved
if (hr >= 0)
{
// Set the profile on the writer
IConfigAsfWriter configWriter = (IConfigAsfWriter)asfWriter;
hr = configWriter.ConfigureFilterUsingProfile(wmProfile);
if (hr >= 0)
{
return true;
}
}
}
return false;
}
In the SetupGraph function find the SetOutputFileName and below it put
ConfigProfileFromFile(mux, "c:\wmv.prx");
Now create a file called wmv.prx on your c: drive and place the relevant information in it.
You can see a sample of a PRX file from the demo project here: http://www.codeproject.com/KB/audio-video/videosav.aspx (Pal90.prx)
And now enjoy your .wmv file outputted at the right size.
Yes I know the code I placed in was rather scrappy but I will leave it up to you to polish it up.
Lifecam is known for unobvious behavior with setting capture format (more specifically, falling back to other formats). See prevoius discussions which are likely to suggest you a solution:
Can't make IAMStreamConfig.SetFormat() to work with LifeCam Studio
IAMStreamConfig settings are ignored by stream - Microsoft LifeCam Cinema records only on 640x480
Can Microsoft LifeCam record at a frame rates lower than 30 fps?