Develop Reference

How to improve performance on wpf GUI main thread that is done in two windows

I have a wpf application, It has mainWindow that creates _otherWindow that displays on a secondary monitor. Both windows have elements that need to change along with time. (mainWindow updates an Image and also plots a graph, finally _otherWindow updates a shape position depending on some computations).
What is my problem? well, I am reading a video frame by frame within a Thread (however I would like to allow this with stream taken with a camera). And as I update GUI every frame in a certain time, Application is getting a heavy load and is getting slow...
I realized that commenting either mainWindow updating Image, or commenting _otherWindow updating shape position codes make the application run nice, but the issue is when they run together.
Here is a detailed description
First I compute some things inside _otherWindow and compute position of a shape.
Then I compute some stuff related to image and the update frame adding some stuff to bitmap
Then I update position of shape inside _otherWindow
Finally I plot results (the plot needs data gotten from mainWindow and _otherWindow)
For this, I use tasks and wait for them.
I have this:
private Thread _camera;
private void CaptureVideo()
{
_camera = new Thread(CaptureVideoCallback)
{
Priority = ThreadPriority.Highest
};
_camera.Start();
}
private VideoCapture _capture;
private void CaptureVideoCallback()
{
//some computing here read from a video file...
_capture = new VideoCapture("someVideo.mp4");
for (var i = 0; i < _capture.FrameCount; i++)
{
_capture.Read(_frame);
if (_frame.Empty()) return;
//*************task that does heavy computation in other class
var heavyTaskOutput1 = Task.Factory.StartNew(() =>
{
_otherWindow.Dispatcher.Invoke(() =>
{
ResultFromHeavyComputationMethod1 = _otherWindow.HeavyComputationMethod1();
});
}
);
////*************task that does heavy computation in current class
var heavyTaskOutput2 = Task.Factory.StartNew(() =>
{
ResultFromHeavyComputationMethod2 = HeavyComputationMethod2(ref _frame);
var bitmap = getBitmapFromHeavyComputationMethod2();
bitmap.Freeze();
//update GUI in main thread
Dispatcher.CurrentDispatcher.Invoke(() => ImageSource = bitmap);
});
////*************wait both task to complete
Task.WaitAll(heavyTaskOutput1, heavyTaskOutput2 );
//update _otherWindow GUI
var outputGui = Task.Factory.StartNew(() =>
{
_otherWindow.Dispatcher.Invoke(() =>
{
_otherWindow.UpdateGui();
});
}
);
outputGui.Wait();
////*************plot in a char using gotten results, UPDATE GUI
Task.Run(() =>
{
PlotHorizontal();
});
}
}
What would be a good way to speed this up?
I mean I know that GUI stuff need to be done on main thread, but this is slowing down things.
Edit
Have changed code as Clemens suggested:
//*************task that does heavy computation in other class
var heavyTaskOutput1 = Task.Run(() =>
{
ResultFromHeavyComputationMethod1 = _otherWindow.HeavyComputationMethod1();
}
);
////*************task that does heavy computation in current class
var heavyTaskOutput2 = Task.Run(() =>
{
ResultFromHeavyComputationMethod2 = HeavyComputationMethod2(ref _frame);
var bitmap = getBitmapFromHeavyComputationMethod2();
bitmap.Freeze();
//update GUI in main thread
Dispatcher.CurrentDispatcher.Invoke(() => ImageSource = bitmap);
});
////*************wait both task to complete
Task.WaitAll(heavyTaskOutput1, heavyTaskOutput2);
//update _otherWindow GUI
var outputGui = Task.Run(() =>
{
_otherWindow.Dispatcher.Invoke(() =>
{
_otherWindow.UpdateGui();
});
}
);
outputGui.Wait();

It's a bit hard to guess. Do you have Visual Studio? I think even the Community edition has some profiling capabilities (menu: Analyze/Performance Profiler...). That may point out some non-obvious bottlenecks.
My thoughts:
getBitmapFromHeavyComputationMethod2 appears to return a new bitmap every time through. I can't infer the actual type it's returning, but it likely involves a semi-large un-managed memory allocation and implements IDisposable. You might check on whether you're disposing that appropriately.
Rather than create a new bitmap for every frame, can you use a WriteableBitmap? Be sure to lock and unlock it if you do. Perhaps ping-pong (alternate) between two bitmaps if you need to.
It appears you may be serializing your "heavy computation" with your I/O read (first one, then the other). Perhaps launch the read as an async as well, and wait on it in your WaitAll so that the computation and I/O can happen concurrently. Something in this shape:
var readResult = _capture.Read(_frame);
for (...) {
// check read result
// ...
// launch heavy computation
readResult = Task.Run(() => _capture.Read(nextFrame);
Task.WaitAll(pupilOutput, outputTest, readResult);
_frame = nextFrame;
}
Note this would read N+1 times for N frames--maybe your Read method is okay with that.

Updating Image control in UI thread from a timer callback in WPF

I have an image control in a Xaml file as follows:
<Viewbox Grid.Row="1" Grid.Column="0" Grid.ColumnSpan="2" VerticalAlignment="Top" HorizontalAlignment="Center">
<Image Name="Content"/>
</Viewbox>
I'd like to update this image with a different image every 10 seconds.
I create a system.threading.Timer instance, initialize it with a callback, pass in the UI control as a state object and set the interval to 10 seconds as follows:
contentTimer = new Timer(OnContentTimerElapsed, Content , 0 , (long) CONTENT_DISPLAY_TIME);
The callback looks as follows:
private void OnContentTimerElapsed( object sender )
{
Image content = (Image)sender;
//update the next content to be displayed
nCurrentImage = (nCurrentImage % NUM_IMAGES) + 1;
//get the url of the image file, and create bitmap from the jpeg
var path = System.IO.Path.Combine(Environment.CurrentDirectory, "../../DisplayContent/Image_" + nCurrentImage.ToString() + ".jpg");
Uri ContentURI = new Uri(path);
var bitmap = new BitmapImage(ContentURI);
//update the image control, by launching this code in the UI thread
content.Dispatcher.BeginInvoke(new Action(() => { content.Source = bitmap; }));
}
I still keep getting the following exception:
An unhandled exception of type 'System.InvalidOperationException' occurred in WindowsBase.dll
Additional information: The calling thread cannot access this object because a different thread owns it.
I was able to get a solution by updating just the numCurrentImage variable, and then updating the Content.Source in the MainWindow class in callbacks running on the UI thread, something as follows (note, I'm getting frames at 30fps from a kinect):
int nCurrentImage;
Public MainWindow()
{
InitializeComponent();
nCurrentImage = 1;
System.Timers.Timer contentTimer = new System.Timers.Timer(OnContentTimerElapsed, CONTENT_DISPLAY_TIME);
contentTimer.Elapsed += OnContentTimerElapsed;
...
//Some kinect related initializations
...
kinect.multiSourceReader.MultiSourceFrameArrived += OnMultiSourceFrameArrived;
}
private void OnContentTimerElapsed( object sender )
{
//update the next content to be displayed
nCurrentImage = (nCurrentImage % NUM_IMAGES) + 1;
}
private void OnMultiSourceFrameArrived(object sender, MultiSourceFrameArrivedEventArgs e)
{
UpdateContent(nCurrentImage);
}
private void UpdateContent(int numImage)
{
var path = System.IO.Path.Combine(Environment.CurrentDirectory, "../../DisplayContent/Image_" + numImage.ToString() + ".jpg");
Uri ContentURI = new Uri(path);
var bitmap = new BitmapImage(ContentURI);
Content.Source = bitmap;
}
Even though that works, it just doesn't make good programming sense to update it that way, since half of the work is being done by one thread, and the rest by the UI thread.
Any Ideas what I'm doing wrong?

Even though that works, it just doesn't make good programming sense to update it that way, since half of the work is being done by one thread, and the rest by the UI thread.
Actually this is exactly what you want to be doing. You want to be doing your non-UI work in a non-UI thread, and doing your UI work in a UI thread.
That said, while this is fundamentally what you want to be doing, you don't need to do all of this yourself so explicitly. You can simply use a DispatcherTimerand it will fire the callback in the UI thread, rather than a thread pool thread. It is, more or less, doing much of what you're doing manually.

Update the XAML image element, I like to name them all with an X to remind me it's a XAML element.
<Image Name="XContent"/>
When timer fires,
...
bitmap.Freeze();
XContent.Dispatcher.Invoke(()=>{
XContent.Source = bitmap;
}

Locking and Unlocking WritableBitmap

I'm trying to write into a WritableBitmap and I want to do the data processing in a non-UI thread.
So I'm calling the Lock and Unlock methods from the UI dispatcher and the rest is done on a different thread:
IntPtr pBackBuffer = IntPtr.Zero;
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.WriteLine("{1}: Begin Image Update: {0}", DateTime.Now, this.GetHashCode());
_mappedBitmap.Lock();
pBackBuffer = _mappedBitmap.BackBuffer;
}));
// Long processing straight on pBackBuffer...
Application.Current.Dispatcher.Invoke(new Action(()=>
{
Debug.WriteLine("{1}: End Image Update: {0}", DateTime.Now, this.GetHashCode());
// the entire bitmap has changed
_mappedBitmap.AddDirtyRect(new Int32Rect(0, 0, _mappedBitmap.PixelWidth,
_mappedBitmap.PixelHeight));
// release the back buffer and make it available for display
_mappedBitmap.Unlock();
}));
This code can be called from any thread, since it specifically calls the UI dispatcher when needed.
This works when my control is not under great stress. But when I call this every 100ms almost immediately I get an InvalidOperationException from AddDirtyRect with the following message:
{"Cannot call this method while the image is unlocked."}
I don't understand how this can happen. My Debug Output logs show that Lock indeed has been called for this instance of my class.
UPDATE
My entire scenario: I'm writing a class which will allow diplaying floating-point matrices in a WPF Image control. The class FloatingPointImageSourceAdapter allows setting data using the API
void SetData(float[] data, int width, int height)
And it exposes a ImageSource which an Image control Souce property can be bound to.
Internally this is implemented using WritableBitmap. Whenever a user sets new data I need to process the pixels and rewrite them into the buffer. The data is planned to be set at a high frequency and this is why I went for writing directly into the BackBuffer instead of calling WritePixels. Moreover, since the remapping of the pixels can take a while and the images can be quite large, I want to do the processing on a separate thread.
I have decided to deal with high stress by dropping frames. So I have an AutoResetEvent which keeps track of when the user has requested to update the data. And I have a background task which does the actual work.
class FloatingPointImageSourceAdapter
{
private readonly AutoResetEvent _updateRequired = new AutoResetEvent(false);
public FloatingPointImageSourceAdapter()
{
// all sorts of initializations
Task.Factory.StartNew(UpdateImage, TaskCreationOptions.LongRunning);
}
public void SetData(float[] data, int width, int height)
{
// save the data
_updateRequired.Set();
}
private void UpdateImage()
{
while (true)
{
_updateRequired.WaitOne();
Debug.WriteLine("{1}: Update requested from thread {2}, {0}", DateTime.Now, this.GetHashCode(), Thread.CurrentThread.ManagedThreadId);
IntPtr pBackBuffer = IntPtr.Zero;
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.WriteLine("{1}: Begin Image Update: {0}", DateTime.Now, this.GetHashCode());
_mappedBitmap.Lock();
pBackBuffer = _mappedBitmap.BackBuffer;
}));
// The processing of the back buffer
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.WriteLine("{1}: End Image Update: {0}", DateTime.Now, this.GetHashCode());
// the entire bitmap has changed
_mappedBitmap.AddDirtyRect(new Int32Rect(0, 0, _mappedBitmap.PixelWidth,
_mappedBitmap.PixelHeight));
// release the back buffer and make it available for display
_mappedBitmap.Unlock();
}));
}
}
}
I have dropped a lot of code here for the sake of bravity.
My test creates a task which calls SetData within certain intervals:
private void Button_Click_StartStressTest(object sender, RoutedEventArgs e)
{
var sleepTime = SleepTime;
_cts = new CancellationTokenSource();
var ct = _cts.Token;
for (int i = 0; i < ThreadsNumber; ++i)
{
Task.Factory.StartNew(() =>
{
while (true)
{
if (ct.IsCancellationRequested)
{
break;
}
int width = RandomGenerator.Next(10, 1024);
int height = RandomGenerator.Next(10, 1024);
var r = new Random((int)DateTime.Now.TimeOfDay.TotalMilliseconds);
var data = Enumerable.Range(0, width * height).Select(x => (float)r.NextDouble()).ToArray();
this.BeginInvokeInDispatcherThread(() => FloatingPointImageSource.SetData(data, width, height));
Thread.Sleep(RandomGenerator.Next((int)(sleepTime * 0.9), (int)(sleepTime * 1.1)));
}
}, _cts.Token);
}
}
I run this test with ThreadsNumber=1 and with SleepTime=100 and it crashes with the aforementioned exception.
UPDATE 2
I tried checking that my commands indeed execute serially.
I added another private field
private int _lockCounter;
And I manipulate it in my while loop:
private void UpdateImage()
{
while (true)
{
_updateRequired.WaitOne();
Debug.Assert(_lockCounter == 0);
_lockCounter++;
IntPtr pBackBuffer = IntPtr.Zero;
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.Assert(_lockCounter == 1);
++_lockCounter;
_mappedBitmap.Lock();
pBackBuffer = _mappedBitmap.BackBuffer;
}));
Debug.Assert(pBackBuffer != IntPtr.Zero);
Debug.Assert(_lockCounter == 2);
++_lockCounter;
// Process back buffer
Debug.Assert(_lockCounter == 3);
++_lockCounter;
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.Assert(_lockCounter == 4);
++_lockCounter;
// the entire bitmap has changed
_mappedBitmap.AddDirtyRect(new Int32Rect(0, 0, _mappedBitmap.PixelWidth,
_mappedBitmap.PixelHeight));
// release the back buffer and make it available for display
_mappedBitmap.Unlock();
}));
Debug.Assert(_lockCounter == 5);
_lockCounter = 0;
}
}
I was hoping that if the message order was somehow messed up my Debug.Asserts would catch this.
But everything with the counters is fine. They are incremented correctly according to the serial logic, and still I get the exception from AddDirtyRect.

Application.Current.Dispatcher.Invoke will try to execute the method that is passed as delegate to on the UI thread itself, this will happen when UI thread is free. If you try to execute the instructions on this continuously it'll almost nothing like performing the operation on the UI thread. Always the instruction executed on Application.Current.Dispatcher.Invoke should be very minimal say it should be only one line like which shall change only value on the UI nothing more than that. So avoid complex operations that are performed as part Dispatcher, move it out of dispatcher and do the only the operations that updates the UI

So after some (very long) digging, it turned out the real bug was hidden in the code I left out for the sake of bravity :-)
My class allows changing the size of the image. When setting data, I check if the new size is the same as the old size and if it isn't I initialize a new WritableBitmap.
What happened was that the size of the image was changed (using a different thread) sometime in the middle of the while loop. And this caused different stages of the processing code to process different instances of _mappedBitmap (since _mappedBitmap pointed to different instances throughout the different stages). So when the instance was changed to a new one, it was created in an unlocked state, thus causing the (rightful) exception.

Thread makes application halt

I'm currently trying to create a FileViewer control, and after I've added Items (Filenames with Icons, size, etc) to my ListView (Icon - Filename - Extension - Size) I also check if the file is an image (png/jpg, etc) but this I do on a different Thread.
My expectation of a thread is that it runs beside the main application, but after I've added all my files I start this thread. It checks all files in the ListView and creates thumbnails for them. If done correctly, ListView icons should appear one after one as they're loaded - but they're not. They all appear at the same time.
...and I can't do anything while the Thread is active.
Why is this happening and what am I doing wrong? I've dealt with Threads before and it's always worked, I invoke the method with a Callback.
Flow of the Thread:
Format file key = "C:\image.png" = "C_image_png".
Check if thumbnail to image exists (by checking it's key), then use it
Else load thumbnail with Image.FromFile().GetThumbnailImage() and add image with Key to Listview's images
Finally change the ImageKey of the ListView item.
All done in a thread.
private void GetFiles()
{
// Load all files in directory
Thread t = new Thread(new ThreadStart(GetImageFiles));
t.Priority = ThreadPriority.Lowest;
t.Start();
}
delegate void GetImageFilesCallback();
private void GetImageFiles()
{
if (this.IsHandleCreated)
{
if (files.InvokeRequired)
{
GetImageFilesCallback callback = new GetImageFilesCallback(GetImageFiles);
this.Invoke(callback);
}
else
{
string extension = "";
string key = "";
foreach (string file in _files)
{
extension = FileManager.GetExtension(file);
key = (DirectoryCurrent + file).Replace(":", "").Replace("\\", "_").Replace(".", "_");
foreach (string knownimages in _knownImageTypes)
{
if (extension.ToLower() == knownimages)
{
foreach (ListViewItem item in files.Items)
{
if (item.Text == file)
{
if (files.SmallImageList != null)
{
if (files.SmallImageList.Images[key] == null)
{
files.SmallImageList.Images.Add(key, Image.FromFile(DirectoryCurrent + file).GetThumbnailImage(16, 16, null, IntPtr.Zero));
files.LargeImageList.Images.Add(key, Image.FromFile(DirectoryCurrent + file).GetThumbnailImage(32, 32, null, IntPtr.Zero));
}
files.Items[item.Index].ImageKey = key;
}
}
}
}
}
}
files.Refresh();
}
}
}

The method that your thread calls is invoking itself onto the main thread, and then doing all the work in that thread, thereby blocking your UI.
You should arrange your code so that the thread code does not touch the ListView, but just loads each image, then invokes a main-thread method, passing the bitmaps so that the main thread can assign them to the ListView.
Here's a sketch of what I mean:
// this is your thread method
// it touches no UI elements, just loads files and passes them to the main thread
private void LoadFiles(List<string> filenames) {
foreach (var file in filenames) {
var key = filename.Replace(...);
var largeBmp = Image.FromFile(...);
var smallBmp = Image.FromFile(...);
this.Invoke(new AddImagesDelegate(AddImages), key, largeBmp, smallBmp);
}
}
// this executes on the main (UI) thread
private void AddImages(string key, Bitmap large, Bitmap small) {
// add bitmaps to listview
files.SmallImageList.Images.Add(key, small);
files.LargeImageList.Images.Add(key, large);
}
private delegate AddImagesDelegate(string key, Bitmap large, Bitmap small);

Read the following: http://www.codeproject.com/Articles/10311/What-s-up-with-BeginInvoke. The important thing with Invoke and BeginInvoke is that they both operate on the Main thread. BeginInvoke just doesn't wait for the message to be processed before returning control. Eventually though, the work will happen on the Main thread and will block until it is complete.

using the objects which are created by other thread

I just want to change the window's background in another thread. there are two program, one is work right, and the other throw an InvalidOperationException.
The right code:
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
Thread t = new Thread(new ParameterizedThreadStart(threadTest));
t.Start(#"C:\Users\Public\Pictures\Sample Pictures\Chrysanthemum.jpg");
}
void threadTest(object obj)
{
string path = obj as string;
this.Dispatcher.Invoke(new Func<object>(() => this.Background = new
}
}
the Error Code:
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
Thread t = new Thread(new ParameterizedThreadStart(threadTest));
t.Start(#"C:\Users\Public\Pictures\Sample Pictures\Chrysanthemum.jpg");
}
void threadTest(object obj)
{
string path = obj as string;
//this.Dispatcher.Invoke(new Func<object>(() => this.Background = new ImageBrush(new BitmapImage(new Uri(path)))));
ImageBrush background = new ImageBrush(new BitmapImage(new Uri(path)));
this.Dispatcher.Invoke(new Func<object>(() => this.Background = background));
}
}
the different between these codes is that, the error code create the ImageBrush object in the child thread.
So my question is that: in the wpf program, is the thread can only use the objects creates by own thread?
thanks for any reply.

Yes, you are right. Only the UI thread can use objects created by it. So, you can use the Dispatcher to "enqueue" the UI operations on it's proper thread.
Answering your second question, sure, there's a way to "pass" objects to the UI Thread. If you see the BeginInvoke structure (of the Dispatcher) it's:
public DispatcherOperation BeginInvoke(
Delegate d,
params Object[] args
)
Where the args is the params object array, there's where you put the params.
Now, if you are using some Freezable object (for example some Image, Brush, Transform or Geometry) then you need to object.Freeze(); before send it to the UI Thread.

Yes, correct, It's not only about WPF, but in general, about Windows programming.
You can not update UI object from other thread different from its own.
The reason for this is simply because, the message pumping and especially delivery to destination control of OS must be guaranteed. This is naturally valid for communication using SendMesage, but for PostMessage too.

If You create an object on a separate thread, You might use it on gui thread if You freeze it first. See Freezable objects.