I have been using the latest version of the WPFMediaKit. What I am trying to do is write a sample application that will use the Samplegrabber to capture the video frames of video files so I can have them as individual Bitmaps.
So far, I have had good luck with the following code when constructing and rendering my graph. However, when I use this code to play back a .wmv video file, when the samplegrabber is attached, it will play back jumpy or choppy. If I comment out the line where I add the samplegrabber filter, it works fine. Again, it works with the samplegrabber correctly with AVI/MPEG, etc.
protected virtual void OpenSource()
{
FrameCount = 0;
/* Make sure we clean up any remaining mess */
FreeResources();
if (m_sourceUri == null)
return;
string fileSource = m_sourceUri.OriginalString;
if (string.IsNullOrEmpty(fileSource))
return;
try
{
/* Creates the GraphBuilder COM object */
m_graph = new FilterGraphNoThread() as IGraphBuilder;
if (m_graph == null)
throw new Exception("Could not create a graph");
/* Add our prefered audio renderer */
InsertAudioRenderer(AudioRenderer);
var filterGraph = m_graph as IFilterGraph2;
if (filterGraph == null)
throw new Exception("Could not QueryInterface for the IFilterGraph2");
IBaseFilter renderer = CreateVideoMixingRenderer9(m_graph, 1);
IBaseFilter sourceFilter;
/* Have DirectShow find the correct source filter for the Uri */
var hr = filterGraph.AddSourceFilter(fileSource, fileSource, out sourceFilter);
DsError.ThrowExceptionForHR(hr);
/* We will want to enum all the pins on the source filter */
IEnumPins pinEnum;
hr = sourceFilter.EnumPins(out pinEnum);
DsError.ThrowExceptionForHR(hr);
IntPtr fetched = IntPtr.Zero;
IPin[] pins = { null };
/* Counter for how many pins successfully rendered */
int pinsRendered = 0;
m_sampleGrabber = (ISampleGrabber)new SampleGrabber();
SetupSampleGrabber(m_sampleGrabber);
hr = m_graph.AddFilter(m_sampleGrabber as IBaseFilter, "SampleGrabber");
DsError.ThrowExceptionForHR(hr);
/* Loop over each pin of the source filter */
while (pinEnum.Next(pins.Length, pins, fetched) == 0)
{
if (filterGraph.RenderEx(pins[0],
AMRenderExFlags.RenderToExistingRenderers,
IntPtr.Zero) >= 0)
pinsRendered++;
Marshal.ReleaseComObject(pins[0]);
}
Marshal.ReleaseComObject(pinEnum);
Marshal.ReleaseComObject(sourceFilter);
if (pinsRendered == 0)
throw new Exception("Could not render any streams from the source Uri");
/* Configure the graph in the base class */
SetupFilterGraph(m_graph);
HasVideo = true;
/* Sets the NaturalVideoWidth/Height */
//SetNativePixelSizes(renderer);
}
catch (Exception ex)
{
/* This exection will happen usually if the media does
* not exist or could not open due to not having the
* proper filters installed */
FreeResources();
/* Fire our failed event */
InvokeMediaFailed(new MediaFailedEventArgs(ex.Message, ex));
}
InvokeMediaOpened();
}
And:
private void SetupSampleGrabber(ISampleGrabber sampleGrabber)
{
FrameCount = 0;
var mediaType = new AMMediaType
{
majorType = MediaType.Video,
subType = MediaSubType.RGB24,
formatType = FormatType.VideoInfo
};
int hr = sampleGrabber.SetMediaType(mediaType);
DsUtils.FreeAMMediaType(mediaType);
DsError.ThrowExceptionForHR(hr);
hr = sampleGrabber.SetCallback(this, 0);
DsError.ThrowExceptionForHR(hr);
}
I have read a few things saying the the .wmv or .asf formats are asynchronous or something. I have attempted inserting a WMAsfReader to decode which works, but once it goes to the VMR9 it gives the same behavior. Also, I have gotten it to work correctly when I comment out the IBaseFilter renderer = CreateVideoMixingRenderer9(m_graph, 1); line and have filterGraph.Render(pins[0]); -- the only drawback is that now it renders in an Activemovie widow of its own instead of my control, however the samplegrabber functions correctly and without any skipping. So I am thinking the bug is in the VMR9 / samplegrabbing somewhere.
Any help? I am new to this.
Some decoders will use hardware acceleration using DXVA. This is implemented by negotiating a partly-decoded format, and passing this partly decoded data to the renderer to complete decoding and render. If you insert a sample grabber configured to RGB24 between the decoder and the renderer, you will disable hardware acceleration.
That, I'm sure, is the crux of the problem. The details are still a little vague, I'm afraid, such as why it works when you use the default VMR-7, but fails when you use VMR-9. I would guess that the decoder is trying to use dxva and failing, in the vmr-9 case, but has a reasonable software-only backup that works well in vmr-7.
I'm not familiar with the WPFMediaKit, but I would think the simplest solution is to replace the explicit vmr-9 creation with an explicit vmr-7 creation. That is, if the decoder works software-only with vmr-7, then use that and concentrate on fixing the window-reparenting issue.
It ended up the the code I have posted (which in itself was fairly shamelessly slightly modified by me from Jeremiah Morrill's WPFMediakit source code) was in fact adequate to render the .WMV files and be sample grabbed.
It seems the choppiness has something to do with running through the VS debugger, or VS2008 itself. After messing around with the XAML for a while in the visual editor, then running the app, I will have this choppy behavior introduced. Shutting down VS2008 seems to remedy it. :P
So not much of an answer, but at least a (annoying - restarting VS2008) fix when it crops up.
Related
I want to analyse my default playback device and detect the beats. I've been using the BASS WASAPI to get the FFT data of the selected device with:
int ret = BassWasapi.BASS_WASAPI_GetData(_fft, (int)BASSData.BASS_DATA_FFT2048);
Now I was using the data to generate spectrum data and display this to the user. In addition I want to detect the Beats using the BPMCounter Class from BASS. However as far as I can tell the BPMCounter.ProcessAudio() function requires a stream (which I don't get with WASAPI) in order to work. Is there a ways I can use BPMCounter with WASAPI? Would be great if someone can point me to the right direction. Thanks
Edit:
Tried this to convert the data to a stream, but without success:
int ret = BassWasapi.BASS_WASAPI_GetData(_fft, (int)BASSData.BASS_DATA_FFT2048); //get channel fft data
var chan = Bass.BASS_StreamCreate(0, 44100, BASSFlag.BASS_DEFAULT, BASSStreamProc.STREAMPROC_PUSH);
Bass.BASS_ChannelPlay(chan, false);
Bass.BASS_StreamPutData(chan, _fft, _fft.Length);
bool beat = _count.ProcessAudio(chan, true);
Debug.Write(beat);
beat is always False, however I can see at the Spectrum that the capturing of the FFT Data is corrent.
I've just started playing with this lib a few hours ago and i am still going through the examples. So my answer maybe is not what you want. For my project i also want to transform WASAPI into a stream and use it for a displaying a spectrum. What i did was to create a StreamPush, right after BASS_WASAPI initialization.
To init your WASAPI use this call and this delegate:
private InitWasapi()
{
WASAPIPROC _process = new WASAPIPROC(Process); // Delegate
bool res = BassWasapi.BASS_WASAPI_Init(_YourDeviceNumber, 0, 0, BASSWASAPIInit.BASS_WASAPI_BUFFER, 1f, 0f, _process, IntPtr.Zero);
if (!res)
{
// Do error checking
}
// This is the part you are looking for (maybe!)
// Use these flags because Wasapi needs 32-bit sample data
var info = BassWasapi.BASS_WASAPI_GetInfo();
_stream = Bass.BASS_StreamCreatePush(info.freq, info.chans, BASSFlag.BASS_STREAM_DECODE | BASSFlag.BASS_SAMPLE_FLOAT, IntPtr.Zero);
BassWasapi.BASS_WASAPI_Start();
}
private int Process(IntPtr buffer, int length, IntPtr user)
{
Bass.BASS_StreamPutData(_stream, buffer, length);
return length;
}
Please note: This works, but i am still experimenting. For example i am not getting the same spectrum output as when i create the stream from the music file itself. There are some (small) differences. Maybe it's because i am using a custom EQ in Winamp for playing the same .mp3. So if anyone knows more on this subject, i would like also to hear it!
Context
WPF UI with a user control that instanciate multiple COMS and use filters with directshow.net
Problem
The audio pins' names change depending on which video is playing. ( both are .avi files )
As you can see in the screenshots, the sound pins are not the same. (one is 'Stream 01', while the other one is '01 Microsoft wave form ..... ')
In my code, I use ConnectDirect and the method GetPin. To use GetPin, you need to give a pin name.
Graphs
Graph generated with exactly the same code, only change the video files.
Question
How do I connect the filters when the pins name change depending on which .avi file is running ? btw one avi file is 'home made' while the other is a microsoft avi sample file (12 seconds blue clock)
Relevant code
//sound filter linker
IBaseFilter pACMWrapper = (IBaseFilter)new ACMWrapper();
hr = m_FilterGraph.AddFilter(pACMWrapper, "ACM wrapper");
//add le default direct sound device
IBaseFilter pDefaultDirectSoundDevice = null;
try
{
pDefaultDirectSoundDevice = (IBaseFilter)new DSoundRender();
hr = m_FilterGraph.AddFilter(pDefaultDirectSoundDevice, "Default DirectSound Device");
IBaseFilter aviSplitter;
//find the avi splitter automatically added when I connect samp grabber to source filter.
m_FilterGraph.FindFilterByName("AVI Splitter", out aviSplitter);
System.Windows.MessageBox.Show(""); // graph screenshot is from here.
hr = m_FilterGraph.Connect(GetPin(aviSplitter, "Stream 01"), GetPin(pACMWrapper, "Input"));
DsError.ThrowExceptionForHR(hr);
//connect audio filters
hr = m_FilterGraph.ConnectDirect(GetPin(pACMWrapper, "Output"), GetPin(pDefaultDirectSoundDevice, "Audio Input pin (rendered)"), null);
DsError.ThrowExceptionForHR(hr);
}
catch (Exception)
{
pDefaultDirectSoundDevice = null;
//log error, play video without sound
//throw;
}
GetPin code
private IPin GetPin(IBaseFilter destinationFilter, string pinName)
{
IEnumPins pinEnum;
int hr = destinationFilter.EnumPins(out pinEnum);
DsError.ThrowExceptionForHR(hr);
IPin[] pins = new IPin[1];
IntPtr fetched = Marshal.AllocCoTaskMem(4);
while (pinEnum.Next(1, pins, fetched) == 0)
{
PinInfo pInfo;
pins[0].QueryPinInfo(out pInfo);
bool found = (pInfo.name == pinName);
DsUtils.FreePinInfo(pInfo);
if (found)
return pins[0];
}
return null;
}
You don't need to choose an output pin using a hardcoded name. Instead, and it is a more reliable way in fact, you need to enumerate pins - as your GetPin function already does - then enumerating media types on the given pin. It is OK to look just at the first media type (if any). If its major type is MEDIATYPE_Audio then it's your pin to take, regardless of its effective name.
Question: What is the best way to programmatically disconnect and reconnect displays programmatically?
The Goal: Kill the video output (black screen with no backlight) on a display and later turn it back on. Imagine unplugging the video cord from the monitor, then plugging it back in.
My Attempt:
// Get the monitor to disable
uint iDevNum = 0;
DISPLAY_DEVICE displayDevice = new DISPLAY_DEVICE();
displayDevice.cb = Marshal.SizeOf(displayDevice);
EnumDisplayDevices(null, iDevNum, ref displayDevice, 0))
DEVMODE devMode = new DEVMODE();
EnumDisplaySettings(displayDevice.DeviceName, 0, ref devMode);
//
// Do something here to disable this display device!
//
// Save the display settings
ChangeDisplaySettingsEx(displayDevice.DeviceName, ref devMode,
IntPtr.Zero, ChangeDisplaySettingsFlags.CDS_NONE, IntPtr.Zero);
I can interact with each display, but I can't figure out how to disconnect one.
It is similar to "Disconnect this display" in the Screen Resolution properties in Windows 7:
Notes:
Turning off video output on all displays won't work because I need the other monitors to stay on.
The desktop area on the "dead" display does NOT need to be usable when it is off. Also, it is fine if windows move around.
References:
SO: Enabling a Second Monitor
How to Turn Off a Monitor
1) Get MultiMonitorHelper from here:
https://github.com/ChrisEelmaa/MultiMonitorHelper/tree/master
2) Extend Win7Display to disconnect the display:
using MultiMonitorHelper.DisplayModels.Win7.Enum;
using MultiMonitorHelper.DisplayModels.Win7.Struct;
/// <summary>
/// Disconnect a display.
/// </summary>
public void DisconnectDisplay(int displayNumber)
{
// Get the necessary display information
int numPathArrayElements = -1;
int numModeInfoArrayElements = -1;
StatusCode error = CCDWrapper.GetDisplayConfigBufferSizes(
QueryDisplayFlags.OnlyActivePaths,
out numPathArrayElements,
out numModeInfoArrayElements);
DisplayConfigPathInfo[] pathInfoArray = new DisplayConfigPathInfo[numPathArrayElements];
DisplayConfigModeInfo[] modeInfoArray = new DisplayConfigModeInfo[numModeInfoArrayElements];
error = CCDWrapper.QueryDisplayConfig(
QueryDisplayFlags.OnlyActivePaths,
ref numPathArrayElements,
pathInfoArray,
ref numModeInfoArrayElements,
modeInfoArray,
IntPtr.Zero);
if (error != StatusCode.Success)
{
// QueryDisplayConfig failed
}
// Check the index
if (pathInfoArray[displayNumber].sourceInfo.modeInfoIdx < modeInfoArray.Length)
{
// Disable and reset the display configuration
pathInfoArray[displayNumber].flags = DisplayConfigFlags.Zero;
error = CCDWrapper.SetDisplayConfig(
pathInfoArray.Length,
pathInfoArray,
modeInfoArray.Length,
modeInfoArray,
(SdcFlags.Apply | SdcFlags.AllowChanges | SdcFlags.UseSuppliedDisplayConfig));
if (error != StatusCode.Success)
{
// SetDisplayConfig failed
}
}
}
3) Extend Win7Display to reconnect the display using an answer from this post:
using System.Diagnostics;
/// <summary>
/// Reconnect all displays.
/// </summary>
public void ReconnectDisplays()
{
DisplayChanger.Start();
}
private static Process DisplayChanger = new Process
{
StartInfo =
{
CreateNoWindow = true,
WindowStyle = ProcessWindowStyle.Hidden,
FileName = "DisplaySwitch.exe",
Arguments = "/extend"
}
};
4) Update the methods in IDisplay.
5) Implement the methods:
IDisplayModel displayModel = DisplayFactory.GetDisplayModel();
List<IDisplay> displayList = displayModel.GetActiveDisplays().ToList();
displayList[0].DisconnectDisplay(0);
displayList[0].ReconnectDisplays();
There's a github project that I STILL haven't got around, but it's a starting point. You need to use Win7 specific API in order to change settings. ChangeDisplaySettings won't work.
Have a look: https://github.com/ChrisEelmaa/MultiMonitorHelper
This is what you need to do:
update the IDisplay interface to support TurnOff() method,
and then call it:
var displayModel = DisplayFactory.GetDisplayModel();
var displayList = displayModel.GetActiveDisplays().ToList();
var firstDisplay = displayList[0].TurnOff();
How to implement TurnOff()? I WOULD imagine this is how(I might be wrong here now):
You need to break the connection between GPU & monitor through breaking the "paths". You can break path between source and target like this:
Call SetDisplayConfig() and pass inside specific paths and make sure you map out the DISPLAYCONFIG_PATH_ACTIVE from the DISPLAY_PATH_INFO structure flags integer.
http://msdn.microsoft.com/en-us/library/windows/hardware/ff553945(v=vs.85).aspx
Sorry for not being more helpful, but this is pretty hardcore stuff, it took me quite a while to even understand the basics of that API. It's a starting point :-),
take a look at the example how to rotate specific monitor in Win7: How do I set the monitor orientation in Windows 7?
In all honesty, just wrap the DisplaySwitch.exe for Win7, and pass /internal or /external(depending if you want to disable/enable first/second monitor), this might or might not work for >2 monitors.
I spend a lot of time trying to make DTVViewer sample of DirectShow work unfortunately with no success. The video format of DVBT network is H264 and I found that the IntelliConnect behavior of IFilterGraph prefers to use Mpeg2 Video format.
For those who want to see the code, here it is. If you do not know anything about DirectShow I shared my experience with this code. And the most probably problem is described in step 5 and 6 of the tutorial.
The code for helper function which connects filters:
public static void UnsafeConnectFilters(IFilterGraph2 graph, IBaseFilter source, IBaseFilter dest, Func<AMMediaType, bool> sourceMediaPredicate=null, Func<AMMediaType, bool> destMediaPredicate=null) {
foreach(IPin spin in IteratePinsByDirection(source, PinDirection.Output)) {
if(IsConnected(spin))
continue;
int fetched;
AMMediaType[] sourceTypes=GetMajorType(spin, out fetched);
if(fetched>0) {
Guid sourceType=sourceTypes[0].majorType;
try {
if(sourceMediaPredicate!=null&&!sourceMediaPredicate(sourceTypes[0]))
continue;
foreach(IPin pin in IteratePinsByDirection(dest, PinDirection.Input)) {
if(IsConnected(pin))
continue;
var types=GetMajorType(pin, out fetched);
try {
if(fetched>0) {
Guid destType=types[0].majorType;
if(destMediaPredicate!=null&&!destMediaPredicate(types[0]))
continue;
if(sourceType==destType) {
spin.Connect(pin, types[0]);
return;
}
}
else {
spin.Connect(pin, sourceTypes[0]);
return;
}
}
finally {
}
}
}
finally {
}
}
}
}
Does anyone know about:
How should I connect the h264 pin to ffdshow?
How should I recommend the graph to use h264 video decoding?
Tutorial and details
Create the graph
_graph = (IFilterGraph2)new FilterGraph();
We are using DVBT network
IBaseFilter networkProvider = (IBaseFilter) new DVBTNetworkProvider();
... which must be tuned to 602000KHz#8MHz ONID=1 TSID=1 SID=6
ITuner tuner = (ITuner) networkProvider;
IDVBTuningSpace tuningspace = (IDVBTuningSpace) new DVBTuningSpace();
tuningspace.put_UniqueName("DVBT TuningSpace");
tuningspace.put_FriendlyName("DVBT TuningSpace");
tuningspace.put__NetworkType(typeof (DVBTNetworkProvider).GUID);
tuningspace.put_SystemType(DVBSystemType.Terrestrial);
ITuneRequest request;
tuningspace.CreateTuneRequest(out request);
ILocator locator = (ILocator) new DVBTLocator();
locator.put_CarrierFrequency(602000);
((IDVBTLocator) locator).put_Bandwidth(8);
request.put_Locator(locator);
IDVBTuneRequest dvbrequest = (IDVBTuneRequest) request;
dvbrequest.put_TSID(1);
dvbrequest.put_ONID(1);
dvbrequest.put_SID(6);
_graph.AddFilter(networkProvider, "Network Provider");
Create a mpeg2 demux to get separate EPG/Vidoe/Audio/Text streams out of single TV stream
_mpeg2Demultiplexer = (IBaseFilter) new MPEG2Demultiplexer();
_graph.AddFilter(_mpeg2Demultiplexer, "MPEG-2 Demultiplexer");
Now we search local filters for BDA Source Filter which in my case is IT9135 BDA Fitler
DsDevice[] devicesOfCat =
DsDevice.GetDevicesOfCat(FilterCategory.BDASourceFiltersCategory);
IBaseFilter iteDeviceFilter;
_graph.AddSourceFilterForMoniker(
devicesOfCat[0].Mon, null, devicesOfCat[0].Name, out iteDeviceFilter);
Now connect filters: [DVBT Net. Provider]->[BDA Src Filter]->[MPEG2Demux]-> ...
UnsafeConnectFilters(_graph, networkProvider, iteDeviceFilter);
UnsafeConnectFilters(_graph, iteDeviceFilter, _mpeg2Demultiplexer);
Two filters must be connected to demux, to provide epg (program guide data). sorry I do not know what they specifically are doig :P. They are located under BDATransportInformationRenderersCategory category. We try to find them by name and connect them to demux
DsDevice[] dsDevices =
DsDevice.GetDevicesOfCat(FilterCategory.BDATransportInformationRenderersCategory);
foreach (DsDevice dsDevice in dsDevices)
{
IBaseFilter filter;
_graph.AddSourceFilterForMoniker(
dsDevice.Mon, null, dsDevice.Name, out filter);
if(dsDevice.Name == "BDA MPEG2 Transport Information Filter")
_bdaTIF = filter;
else if(dsDevice.Name == "MPEG-2 Sections and Tables")
{
_mpeg2SectionsAndTables = filter;
}
UnsafeConnectFilters(_graph, _mpeg2Demultiplexer, filter);
}
Now demux is connected to both MPEG-2 Sections and Tables and BDA MPEG2 Transport Information Filter.
Now create h264 video type and add the output an output pin to demux for this type
AMMediaType h264 = new AMMediaType();
h264.formatType = FormatType.VideoInfo2;
h264.subType = MediaSubType.H264;
h264.majorType = MediaType.Video;
IPin h264pin;
((IMpeg2Demultiplexer) _mpeg2Demultiplexer).CreateOutputPin(h264, "h264", out h264pin);
Below, I tried to search for ffdshow Video Decoder which is capable of processing H264 video and is located under DirectShow Filters category(as in GraphStudio).
DsDevice[] directshowfilters =
DsDevice.GetDevicesOfCat(FilterCategory.LegacyAmFilterCategory);
IBaseFilter ffdshow = null;
foreach (DsDevice directshowfilter in directshowfilters)
{
if(directshowfilter.Name == "ffdshow Video Decoder")
{
_graph.AddSourceFilterForMoniker(
directshowfilter.Mon, null, directshowfilter.Name,
out ffdshow);
break;
}
}
Create a video renderer for video output ...
_videoRenderer = new VideoRendererDefault();
_graph.AddFilter((IBaseFilter)_videoRenderer, "Video Renderer");
... and audio ...
DsDevice defaultDirectSound =
DsDevice.GetDevicesOfCat(FilterCategory.AudioRendererCategory)[0];
_graph.AddSourceFilterForMoniker(
defaultDirectSound.Mon, null, defaultDirectSound.Name,
out _audioRender);
Here I tried to connect h264 output pin of demux to ffdshow. This method call fails with AccessViolationException. I'm not sure how to connect these two together :(.
Commenting this line will result in a graph which starts running, although there is an disconnected ffdshowVideoDecoder filter in the graph, will not show anything. IntelliConnect connects Mpeg2 video output to a locally available video decoder and as I said it will not display anything.
// UnsafeConnectFilters(_graph, _mpeg2Demultiplexer, ffdshow, type => type.majorType == MediaType.Video && type.subType == MediaSubType.H264);
ConnectFilters is borrowed from DTVViewer sample of directshowlib
ConnectFilters();
I moved actual tuning here
tuner.put_TuningSpace(tuningspace);
tuner.put_TuneRequest(request);
start the graph and wish for some sound or video to be displayed
int hr = (_graph as IMediaControl).Run();
DsError.ThrowExceptionForHR(hr);
check that the graph is running ...
FilterState pfs;
hr = (_graph as IMediaControl).GetState(1000, out pfs);
DsError.ThrowExceptionForHR(hr);
and it says that the graph is running.
Did you check that your ffdshow is enabled for H264/AVC? Open the filter properties and In "Codecs" section, H264/AVC format should be enabled (you can also disable the Mpeg2 decoder just to make sure it won't prefer this format).
Another thing, you can try using another Mpeg2 demultiplexer. The default "MPEG-2 Demultiplexer" is not behaving the same on different environments. There are many other filters that can demux TS and if you can invest some money, I'd recommend using MainConcept or Elecard.
Ok I have been at this for 2 days and need help with this last part.
I have a Microsoft LifeCam Cinema camera and I use the .NET DirectShowLib to capture the video stream. Well actually I use WPFMediaKit, but I am in the source code of that dealing directly with the direct show library now.
What I have working is:
- View the video output of the camera
- Record the video output of the camera in ASF or AVI (the only 2 MediaType's supported with ICaptureGraphBuilder2)
The problem is: I can save it as a .avi. This works fine and at a resolution of 1280x720 but it saves the file in RAW output. Meaning it is about 50-60MB per second. Way too high.
Or I can switch it to .asf and it outputs a WMV, but when I do this the capture and the output both go to resolution 320x240.
In WPFMediaKit there is a function I changed because apparently with Microsoft LifeCam Cinema cameras a lot of people have this problem. So instead of creating or changing the AMMediaType you iterate through and then use that to call SetFormat.
///* Make the VIDEOINFOHEADER 'readable' */
var videoInfo = new VideoInfoHeader();
int iCount = 0, iSize = 0;
videoStreamConfig.GetNumberOfCapabilities(out iCount, out iSize);
IntPtr TaskMemPointer = Marshal.AllocCoTaskMem(iSize);
AMMediaType pmtConfig = null;
for (int iFormat = 0; iFormat < iCount; iFormat++)
{
IntPtr ptr = IntPtr.Zero;
videoStreamConfig.GetStreamCaps(iFormat, out pmtConfig, TaskMemPointer);
videoInfo = (VideoInfoHeader)Marshal.PtrToStructure(pmtConfig.formatPtr, typeof(VideoInfoHeader));
if (videoInfo.BmiHeader.Width == DesiredWidth && videoInfo.BmiHeader.Height == DesiredHeight)
{
///* Setup the VIDEOINFOHEADER with the parameters we want */
videoInfo.AvgTimePerFrame = DSHOW_ONE_SECOND_UNIT / FPS;
if (mediaSubType != Guid.Empty)
{
int fourCC = 0;
byte[] b = mediaSubType.ToByteArray();
fourCC = b[0];
fourCC |= b[1] << 8;
fourCC |= b[2] << 16;
fourCC |= b[3] << 24;
videoInfo.BmiHeader.Compression = fourCC;
// pmtConfig.subType = mediaSubType;
}
/* Copy the data back to unmanaged memory */
Marshal.StructureToPtr(videoInfo, pmtConfig.formatPtr, true);
hr = videoStreamConfig.SetFormat(pmtConfig);
break;
}
}
/* Free memory */
Marshal.FreeCoTaskMem(TaskMemPointer);
DsUtils.FreeAMMediaType(pmtConfig);
if (hr < 0)
return false;
return true;
When that was implemented I could finally view the captured video as 1280x720 as long as I set the SetOutputFilename to a MediaType.Avi.
If I set it to a MediaType.Asf it goes to 320x240 and the output is the same.
Or the AVI works and outputs in the correct format but does so in RAW video, hence a very large file size. I have attempted to add a compressor to the graph but with no luck, this is far out of my experience.
I am looking for 1 of 2 answers.
Recording the ASF at 1280x720
Adding a compressor to the graph so that the filesize of my outputted AVI is small.
I figured this out. So I am posting it here for any other poor soul who passes by wondering why it doesn't work.
Download the source of the WPFMediaKit, you are going to need to change some code.
Go to Folder DirectShow > MediaPlayers and open up VideoCapturePlayer.cs
Find the function SetVideoCaptureParameters and replace it with this:
/// <summary>
/// Sets the capture parameters for the video capture device
/// </summary>
private bool SetVideoCaptureParameters(ICaptureGraphBuilder2 capGraph, IBaseFilter captureFilter, Guid mediaSubType)
{
/* The stream config interface */
object streamConfig;
/* Get the stream's configuration interface */
int hr = capGraph.FindInterface(PinCategory.Capture,
MediaType.Video,
captureFilter,
typeof(IAMStreamConfig).GUID,
out streamConfig);
DsError.ThrowExceptionForHR(hr);
var videoStreamConfig = streamConfig as IAMStreamConfig;
/* If QueryInterface fails... */
if (videoStreamConfig == null)
{
throw new Exception("Failed to get IAMStreamConfig");
}
///* Make the VIDEOINFOHEADER 'readable' */
var videoInfo = new VideoInfoHeader();
int iCount = 0, iSize = 0;
videoStreamConfig.GetNumberOfCapabilities(out iCount, out iSize);
IntPtr TaskMemPointer = Marshal.AllocCoTaskMem(iSize);
AMMediaType pmtConfig = null;
for (int iFormat = 0; iFormat < iCount; iFormat++)
{
IntPtr ptr = IntPtr.Zero;
videoStreamConfig.GetStreamCaps(iFormat, out pmtConfig, TaskMemPointer);
videoInfo = (VideoInfoHeader)Marshal.PtrToStructure(pmtConfig.formatPtr, typeof(VideoInfoHeader));
if (videoInfo.BmiHeader.Width == DesiredWidth && videoInfo.BmiHeader.Height == DesiredHeight)
{
///* Setup the VIDEOINFOHEADER with the parameters we want */
videoInfo.AvgTimePerFrame = DSHOW_ONE_SECOND_UNIT / FPS;
if (mediaSubType != Guid.Empty)
{
int fourCC = 0;
byte[] b = mediaSubType.ToByteArray();
fourCC = b[0];
fourCC |= b[1] << 8;
fourCC |= b[2] << 16;
fourCC |= b[3] << 24;
videoInfo.BmiHeader.Compression = fourCC;
// pmtConfig.subType = mediaSubType;
}
/* Copy the data back to unmanaged memory */
Marshal.StructureToPtr(videoInfo, pmtConfig.formatPtr, true);
hr = videoStreamConfig.SetFormat(pmtConfig);
break;
}
}
/* Free memory */
Marshal.FreeCoTaskMem(TaskMemPointer);
DsUtils.FreeAMMediaType(pmtConfig);
if (hr < 0)
return false;
return true;
}
Now that will sort out your screen display at what ever desired resolution you want, provided that your camera supports it.
Next you will soon figure out that this new correct capture you have isnt applied when writing the video to disk.
Since the ICaptureBuilder2 method only supports Avi and Asf (which is wmv) you need to set your mediatype to one of them.
hr = graphBuilder.SetOutputFileName(MediaSubType.Asf, this.m_fileName, out mux, out sink);
You will find that line in the SetupGraph function.
Asf will only output in 320x240, yet the Avi will output in the desired resolution, but uncompressed (meaning 50-60MB per second for a 1280x720 video feed), which is too high.
So that leaves you with 2 options
Figure out how to add a encoder (compression filter) to the Avi output
Figure out how to change the WMV profile
I tried 1, with no success. Mainly due to the fact this is my first time working with DirectShow and only just grasp the meaning of graphs.
But I was successful with #2 and here is how I did it.
Special thanks to (http://www.codeproject.com/KB/audio-video/videosav.aspx) I pulled out the needed code from here.
Create a new class in the same folder called WMLib.cs and place the following in it
Download the demo project from http://www.codeproject.com/KB/audio-video/videosav.aspx and copy and paste the WMLib.cs into your project (change the namespace as necessary)
Create a function in the VideoCapturePlayer.cs class
/// <summary>
/// Configure profile from file to Asf file writer
/// </summary>
/// <param name="asfWriter"></param>
/// <param name="filename"></param>
/// <returns></returns>
public bool ConfigProfileFromFile(IBaseFilter asfWriter, string filename)
{
int hr;
//string profilePath = "test.prx";
// Set the profile to be used for conversion
if ((filename != null) && (File.Exists(filename)))
{
// Load the profile XML contents
string profileData;
using (StreamReader reader = new StreamReader(File.OpenRead(filename)))
{
profileData = reader.ReadToEnd();
}
// Create an appropriate IWMProfile from the data
// Open the profile manager
IWMProfileManager profileManager;
IWMProfile wmProfile = null;
hr = WMLib.WMCreateProfileManager(out profileManager);
if (hr >= 0)
{
// error message: The profile is invalid (0xC00D0BC6)
// E.g. no <prx> tags
hr = profileManager.LoadProfileByData(profileData, out wmProfile);
}
if (profileManager != null)
{
Marshal.ReleaseComObject(profileManager);
profileManager = null;
}
// Config only if there is a profile retrieved
if (hr >= 0)
{
// Set the profile on the writer
IConfigAsfWriter configWriter = (IConfigAsfWriter)asfWriter;
hr = configWriter.ConfigureFilterUsingProfile(wmProfile);
if (hr >= 0)
{
return true;
}
}
}
return false;
}
In the SetupGraph function find the SetOutputFileName and below it put
ConfigProfileFromFile(mux, "c:\wmv.prx");
Now create a file called wmv.prx on your c: drive and place the relevant information in it.
You can see a sample of a PRX file from the demo project here: http://www.codeproject.com/KB/audio-video/videosav.aspx (Pal90.prx)
And now enjoy your .wmv file outputted at the right size.
Yes I know the code I placed in was rather scrappy but I will leave it up to you to polish it up.
Lifecam is known for unobvious behavior with setting capture format (more specifically, falling back to other formats). See prevoius discussions which are likely to suggest you a solution:
Can't make IAMStreamConfig.SetFormat() to work with LifeCam Studio
IAMStreamConfig settings are ignored by stream - Microsoft LifeCam Cinema records only on 640x480
Can Microsoft LifeCam record at a frame rates lower than 30 fps?