Develop Reference

How to wait for a MediaPlayer to complete a Position change

I want to use an invisible MediaPlayer to generate thumbnails at specific time positions in a video
I've tried waiting on the player's dispatcher thread with _player.Dispatcher.BeginInvoke(DispatcherPriority.SystemIdle, new Action(() => { })).Wait();
but that didn't seem to make any difference. It works with a long enough wait on the thread, but I don't like using an arbitrary number.
private MediaPlayer _player;
private void Player_MediaOpened(object sender, EventArgs e)
{
DrawingVisual dv = new DrawingVisual();
TimeSpan duration = _player.NaturalDuration.TimeSpan;
TimeSpan time = new TimeSpan();
TimeSpan interval = new TimeSpan(0, 0, 1); // One Second
RenderTargetBitmap bmp = new RenderTargetBitmap(160, 90, 96, 96, PixelFormats.Pbgra32);
while (time < duration)
{
_player.Position = time;
Thread.Sleep(10); // This is the line I'm unhappy about!
Application.Current?.Dispatcher.BeginInvoke(DispatcherPriority.ApplicationIdle, new Action(() => { })).Wait(); // Wait for the UI to catch up so you see all the images as they are being rendered
using (DrawingContext dc = dv.RenderOpen())
{
dc.DrawVideo(_player, new Rect(0, 0, 160, 90));
}
bmp.Render(dv);
Image thumbImage = new Image()
{
Width = 160,
Height = 90
};
thumbImage.Source = bmp.GetAsFrozen() as ImageSource;
thumbnailsPanel.Children.Add(thumbImage);
time = time.Add(interval);
}
_player.Close();
}
With the Thead.sleep(10); it works correctly but I'm uncomfortable with an arbirary pause. I'd like to wait until the MediaPlayer has completed _player.Position = time;. If Thead.sleep(10); is reduced to, say, 1ms, then the images produced are incorrect - they show earlier frames instead of the actual ones.
Does anyone know how to wait until the MediaPlayer has finished this asynchronous operation?

WPF streaming Bitmap data into an Image control across threads

I have a WPF application, and am streaming a camera frame from an opencv DLL into an Image control inside a UserControl.
This works for a while, and then crashes, giving me:
The calling thread cannot access this object because a different thread owns it.
My code is as follows:
The class that calls the image: (running in a thread)
private void imageShow()
{
while (true)
{
if (status == 1)
{
IntPtr ptr = getFrame(); // The DLL function that returns the image.
imgHalfSize = new Bitmap(640, 360, 3 * 640, PixelFormat.Format24bppRgb, ptr);
CameraFrame = ToBitmapSource(imgHalfSize);
CameraFrame.Freeze();
Thread.Sleep(20);
}
}
//conversion from Bitmap to BitmapSource
public BitmapSource CameraFrame;
[DllImport("gdi32")]
private static extern int DeleteObject(IntPtr o);
public static BitmapSource ToBitmapSource(Bitmap source)
{
IntPtr ptr = source.GetHbitmap(); //obtain the Hbitmap
BitmapSource bs = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
ptr,
IntPtr.Zero,
Int32Rect.Empty,
System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions());
DeleteObject(ptr); //release the HBitmap
return bs;
}
//The userControl that displays the image:
//xaml
<Image Name="ImageCameraFrame"/>
cs:
public FormCameraViewFull(DllFunctions _Dense)
{
Dense = _Dense; // The class as above
InitializeComponent();
Task.Factory.StartNew(() =>
{
InvokeMethodExample();
});
}
//thread function:
private void InvokeMethodExample()
{
while (true)
{
ImageCameraFrame.Dispatcher.Invoke((Action)(() => ImageCameraFrame.Source = Dense.CameraFrame));
}
}
As above, this works great for some time, then crashes. I was under the impression that the line CameraFrame.Freeze(); should stop this behavior, but I am clearly doing something wrong. Do I need a mutex, or similar lock here?
Thank you.

Use a DispatcherTimer with (an optionally async) Tick handler:
var timer = new DispatcherTimer
{
Interval = TimeSpan.FromMilliseconds(50)
};
timer.Tick += async (s, e) =>
{
ImageCameraFrame.Source = await Task.Run(() =>
{
var data = Dense.getFrame();
var format = PixelFormats.Rgb24;
var width = 640;
var height = 360;
var stride = (width * format.BitsPerPixel + 7) / 8;
var bitmap = BitmapSource.Create(width, height, 96, 96,
format, null, data, stride * height, stride);
bitmap.Freeze();
return bitmap;
});
};
timer.Start();

Display BitmapSource in WPF not working

The following code takes a snapshot of part of the screen (at the mouse coordinates) and should display it in an Image control.
public partial class MainWindow : Window
{
Timer timer = new Timer(100);
public MainWindow()
{
InitializeComponent();
timer.Elapsed += Timer_Elapsed;
timer.Start();
}
[System.Runtime.InteropServices.DllImport("gdi32.dll")]
public static extern bool DeleteObject(IntPtr hObject);
private void Timer_Elapsed(object sender, ElapsedEventArgs e)
{
viewerImage.Source = GetSnapAtMouseCoords((int)((Grid)viewerImage.Parent).ActualWidth, (int)((Grid)viewerImage.Parent).ActualHeight, System.Windows.Forms.Cursor.Position);
}
private BitmapSource GetSnapAtMouseCoords(int width, int height, System.Drawing.Point mousePosition)
{
IntPtr handle = IntPtr.Zero;
try
{
using (var screenBmp = new Bitmap(width, height, System.Drawing.Imaging.PixelFormat.Format32bppArgb))
{
using (var bmpGraphics = Graphics.FromImage(screenBmp))
{
bmpGraphics.CopyFromScreen(mousePosition.X, mousePosition.Y, 0, 0, screenBmp.Size);
handle = screenBmp.GetHbitmap();
var bs = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
handle,
IntPtr.Zero,
Int32Rect.Empty,
BitmapSizeOptions.FromEmptyOptions());
return bs;
}
}
}
finally
{
DeleteObject(handle);
}
}
}
Everything works up to the point where I set the image source to the BitmapSource. Unfortunately, the image is never rendered on screen.
I think that maybe this is because I am creating the BitmapSource on the GUI thread... But I am not so sure.
Any suggestions or ideas welcome.

Actually it's because you are accessing your GUI on a different Thread. You can either wrap the initial call like this:
Dispatcher.BeginInvoke(new Action(() =>
{
viewerImage.Source = GetSnapAtMouseCoords(
(int)((Grid)viewerImage.Parent).ActualWidth,
(int)((Grid)viewerImage.Parent).ActualHeight,
System.Windows.Forms.Cursor.Position);
}));
Or do all the processing in a background thread an just return a Frozen (Thread safe) BitmapSource. You would hover need to pass (int)((Grid)viewerImage.Parent).ActualWidth differently, since that is owned by the UI thread too.
bs.Freeze();
Dispatcher.BeginInvoke(new Action(() =>
{
viewerImage.Source = bs;
}));

Updating the content of the TransformedBitmap's source doesn't update the rendered picture

I need to render live video feed from web camera. I'm trying to change the source of my Image object each time new frame arrived. I create WritableBitmap that is updated with new frame each time and TransformedBitmap that applies flip transformation on WritableBitmap as its source. If i create new TransformedBitmap for each frame it works and i see video feed, although the memory footprint is growing very fast, causing the GC to collect the memory very frequently. If i'm trying to reuse single TransformedBitmap I see no update on rendering, first frame is always displayed.
How can I reuse the same TransformedBitmap object without growth of the memory footprint?
Here is the sample code using two different files to emulate the web cam.
public partial class Window1 : Window
{
Thread m_thread;
BitmapImage source1;
BitmapImage source2;
byte[] data;
int stride;
WriteableBitmap bitmap;
TransformedBitmap tb;
public Window1()
{
InitializeComponent();
FileStream stream = new FileStream("1.png", FileMode.Open, FileAccess.Read);
source1 = new BitmapImage();
source1.BeginInit();
source1.StreamSource = stream;
source1.EndInit();
FileStream stream2 = new FileStream("2.png", FileMode.Open, FileAccess.Read);
source2 = new BitmapImage();
source2.BeginInit();
source2.StreamSource = stream2;
source2.EndInit();
stride = source1.PixelWidth * (source1.Format.BitsPerPixel / 8);
data = new byte[stride * source1.PixelHeight];
bitmap = new WriteableBitmap(source1.PixelWidth, source1.PixelHeight, source1.DpiX, source1.DpiY, source1.Format, source1.Palette);
WritePixels(source1, bitmap, data);
tb = new TransformedBitmap(bitmap, new ScaleTransform(-1, 1));
m_thread = new Thread(ThreadFunc);
m_thread.Start();
}
public void ThreadFunc()
{
int i = 0;
while (true)
{
Dispatcher.Invoke(new Action(() => {
BitmapSource source = (i % 2 == 0) ? source2 : source1;
WritePixels(source, bitmap, data);
image1.Source = tb;
}));
++i;
Thread.Sleep(100);
}
}
public void WritePixels(BitmapSource source, WriteableBitmap target, byte[] data)
{
int stride = source.PixelWidth * (source.Format.BitsPerPixel / 8);
source.CopyPixels(data, stride, 0);
target.WritePixels(new Int32Rect(0, 0, source.PixelWidth, source.PixelHeight), data, stride, 0);
}
}

I got it, the problem was caused by the fact that the TransformedBitmap was the same so probably Image object didn't recognize the change, if I use two different TransformedBitmap objects that have the same source, and then after writing new content I switch the TransformedBitmap it works, I have new content rendered.
So the code is:
public partial class MainWindow : Window
{
Thread m_thread;
BitmapImage source1;
BitmapImage source2;
byte[] data;
int stride;
WriteableBitmap bitmap1;
TransformedBitmap tb1;
TransformedBitmap tb2;
public MainWindow()
{
InitializeComponent();
FileStream stream = new FileStream("1.jpg", FileMode.Open, FileAccess.Read);
source1 = new BitmapImage();
source1.BeginInit();
source1.StreamSource = stream;
source1.EndInit();
FileStream stream2 = new FileStream("2.jpg", FileMode.Open, FileAccess.Read);
source2 = new BitmapImage();
source2.BeginInit();
source2.StreamSource = stream2;
source2.EndInit();
stride = source1.PixelWidth * (source1.Format.BitsPerPixel / 8);
data = new byte[stride * source1.PixelHeight];
bitmap1 = new WriteableBitmap(source1.PixelWidth, source1.PixelHeight, source1.DpiX, source1.DpiY, source1.Format, source1.Palette);
WritePixels(source1, bitmap1, data);
tb1 = new TransformedBitmap(bitmap1, new ScaleTransform(-1, -1));
tb2 = new TransformedBitmap(bitmap1, new ScaleTransform(-1, -1));
m_thread = new Thread(ThreadFunc);
m_thread.Start();
}
public void ThreadFunc()
{
int i = 0;
while (true)
{
Dispatcher.Invoke(new Action(() => {
BitmapSource source = (i % 2 == 0) ? source2 : source1;
WritePixels(source, bitmap1, data);
// this is the trick we have to set different TransformedBitmap as
// Image.Source
// so since we don't want to create new one each time we just
// switch between two
image1.Source = (i % 2 == 0) ? tb1 : tb2;
}));
++i;
Thread.Sleep(100);
}
}
public void WritePixels(BitmapSource source, WriteableBitmap target, byte[] data)
{
int stride = source.PixelWidth * (source.Format.BitsPerPixel / 8);
source.CopyPixels(data, stride, 0);
target.WritePixels(new Int32Rect(0, 0, source.PixelWidth, source.PixelHeight), data, stride, 0);
}
}

Why do Graphics operations in background thread block Graphics operations in main UI thread?

I've got a background thread that is creating grayscale thumbnails of images in a given folder. The problem I'm seeing is that the Graphics.DrawImage() call in the background thread seems to be somehow blocking Graphics operations on the main UI thread.
I may be misinterpreting what I'm seeing here, and won't have a chance to do any in-depth profiling until later tonight, though I don't expect to be able to find much.
I've tried to come up with as small a repro case as possible. If you replace the form in a default project with the form below (and have some images in a folder to test with), you'll notice that the animating label will stutter as it bounces back and forth across the window. Yet if you uncomment the #define at the top so that a child control animates rather than redrawing the window contents, it runs perfectly smoothly.
Can anyone see what I'm doing wrong here or help me figure out how to avoid this stutter during the update loop?
//#define USE_LABEL_CONTROL
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Threading;
using System.Windows.Forms;
using Timer = System.Windows.Forms.Timer;
namespace ThreadTest
{
public partial class Form1 : Form
{
private const string ImageFolder = "c:\\pics";
private const string ImageType = "*.jpg";
public Form1()
{
InitializeComponent();
}
protected override void OnLoad(EventArgs e)
{
this.Size = new Size(300, 300);
string[] ImageFiles = Directory.GetFiles(ImageFolder,
ImageType,
SearchOption.AllDirectories);
// kick off a thread to create grayscale thumbnails of all images
this.thumbnailThread = new Thread(this.thumbnailThreadFunc);
this.thumbnailThread.Priority = ThreadPriority.Lowest;
this.thumbnailThread.Start(ImageFiles);
// set a timer to start us off...
this.startTimer = new Timer();
this.startTimer.Interval = 500;
this.startTimer.Tick += this.startTimer_Tick;
this.startTimer.Start();
#if USE_LABEL_CONTROL
this.label.Location = this.labelRect.Location;
this.label.Size = this.labelRect.Size;
this.label.Text = "Loaded: 0";
this.label.BorderStyle = BorderStyle.FixedSingle;
this.Controls.Add(this.label);
#endif
base.OnLoad(e);
}
void startTimer_Tick(object sender, EventArgs e)
{
// kill the timer
this.startTimer.Stop();
// update ourself in a loop
while (this.IsHandleCreated)
{
int NextTick = Environment.TickCount + 50;
// update the label position
this.labelRect.Offset(this.currentLabelDirection, 0);
if (this.labelRect.Right == this.ClientRectangle.Right ||
this.labelRect.Left == 0)
{
this.currentLabelDirection = -this.currentLabelDirection;
}
// update the display
#if USE_LABEL_CONTROL
this.label.Text = "Loaded: " + this.thumbs.Count;
this.label.Location = this.labelRect.Location;
#else
using (Graphics Dest = this.CreateGraphics())
{
this.redrawControl(Dest, this.ClientRectangle);
}
#endif
Application.DoEvents();
Thread.Sleep(Math.Max(0, NextTick - Environment.TickCount));
}
}
private void thumbnailThreadFunc(object ThreadData)
{
string[] ImageFiles = (string[]) ThreadData;
foreach (string ImageFile in ImageFiles)
{
if (!this.IsHandleCreated)
{
return;
}
using (Image SrcImg = Image.FromFile(ImageFile))
{
Rectangle SrcRect = new Rectangle(Point.Empty, SrcImg.Size);
Rectangle DstRect = new Rectangle(Point.Empty, new Size(300, 200));
Bitmap DstImg = new Bitmap(DstRect.Width, DstRect.Height);
using (Graphics Dst = Graphics.FromImage(DstImg))
{
using (ImageAttributes Attrib = new ImageAttributes())
{
Attrib.SetColorMatrix(this.grayScaleMatrix);
Dst.DrawImage(SrcImg,
DstRect,
0, 0, SrcRect.Width, SrcRect.Height,
GraphicsUnit.Pixel,
Attrib);
}
}
lock (this.thumbs)
{
this.thumbs.Add(DstImg);
}
}
}
}
#if !USE_LABEL_CONTROL
private void redrawControl (Graphics Dest, Rectangle UpdateRect)
{
Bitmap OffscreenImg = new Bitmap(this.ClientRectangle.Width,
this.ClientRectangle.Height);
using (Graphics Offscreen = Graphics.FromImage(OffscreenImg))
{
Offscreen.FillRectangle(Brushes.White, this.ClientRectangle);
Offscreen.DrawRectangle(Pens.Black, this.labelRect);
Offscreen.DrawString("Loaded: " + this.thumbs.Count,
SystemFonts.MenuFont,
Brushes.Black,
this.labelRect);
}
Dest.DrawImageUnscaled(OffscreenImg, 0, 0);
OffscreenImg.Dispose();
}
protected override void OnPaintBackground(PaintEventArgs e)
{
return;
}
protected override void OnPaint(PaintEventArgs e)
{
this.redrawControl(e.Graphics, e.ClipRectangle);
}
#endif
private ColorMatrix grayScaleMatrix = new ColorMatrix(new float[][]
{
new float[] {.3f, .3f, .3f, 0, 0},
new float[] {.59f, .59f, .59f, 0, 0},
new float[] {.11f, .11f, .11f, 0, 0},
new float[] {0, 0, 0, 1, 0},
new float[] {0, 0, 0, 0, 1}
});
private Thread thumbnailThread;
private Timer startTimer;
private List<Bitmap> thumbs = new List<Bitmap>();
private Label label = new Label();
private int currentLabelDirection = 1;
private Rectangle labelRect = new Rectangle(0, 125, 75, 20);
}
}

It turns out that the answer is to use multiple processes to handle background GDI+ tasks. If you run the above code under the concurrency profiler in VS2010, you'll see the foreground thread blocking on a critical section secured by the DrawImage() call in the background thread.
This thread also discusses this issue and points out that since it's using a critical section, the locks will be per-process and the background tasks can be parallelized using multiple processes instead of threads:
Parallelizing GDI+ Image Resizing .net

Any time you have performance problems, you need to consider as many factors as possible. In your case, there's a rather large difference between the label and the redraw implementations.
The label version is redrawing only a small (or two small) sections of the screen.
The redraw version is doing much more work and redrawing a larger area.
This in itself could be the problem, depending upon the capabilities of the computer you're using.