I'm trying to write into a WritableBitmap and I want to do the data processing in a non-UI thread.
So I'm calling the Lock and Unlock methods from the UI dispatcher and the rest is done on a different thread:
IntPtr pBackBuffer = IntPtr.Zero;
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.WriteLine("{1}: Begin Image Update: {0}", DateTime.Now, this.GetHashCode());
_mappedBitmap.Lock();
pBackBuffer = _mappedBitmap.BackBuffer;
}));
// Long processing straight on pBackBuffer...
Application.Current.Dispatcher.Invoke(new Action(()=>
{
Debug.WriteLine("{1}: End Image Update: {0}", DateTime.Now, this.GetHashCode());
// the entire bitmap has changed
_mappedBitmap.AddDirtyRect(new Int32Rect(0, 0, _mappedBitmap.PixelWidth,
_mappedBitmap.PixelHeight));
// release the back buffer and make it available for display
_mappedBitmap.Unlock();
}));
This code can be called from any thread, since it specifically calls the UI dispatcher when needed.
This works when my control is not under great stress. But when I call this every 100ms almost immediately I get an InvalidOperationException from AddDirtyRect with the following message:
{"Cannot call this method while the image is unlocked."}
I don't understand how this can happen. My Debug Output logs show that Lock indeed has been called for this instance of my class.
UPDATE
My entire scenario: I'm writing a class which will allow diplaying floating-point matrices in a WPF Image control. The class FloatingPointImageSourceAdapter allows setting data using the API
void SetData(float[] data, int width, int height)
And it exposes a ImageSource which an Image control Souce property can be bound to.
Internally this is implemented using WritableBitmap. Whenever a user sets new data I need to process the pixels and rewrite them into the buffer. The data is planned to be set at a high frequency and this is why I went for writing directly into the BackBuffer instead of calling WritePixels. Moreover, since the remapping of the pixels can take a while and the images can be quite large, I want to do the processing on a separate thread.
I have decided to deal with high stress by dropping frames. So I have an AutoResetEvent which keeps track of when the user has requested to update the data. And I have a background task which does the actual work.
class FloatingPointImageSourceAdapter
{
private readonly AutoResetEvent _updateRequired = new AutoResetEvent(false);
public FloatingPointImageSourceAdapter()
{
// all sorts of initializations
Task.Factory.StartNew(UpdateImage, TaskCreationOptions.LongRunning);
}
public void SetData(float[] data, int width, int height)
{
// save the data
_updateRequired.Set();
}
private void UpdateImage()
{
while (true)
{
_updateRequired.WaitOne();
Debug.WriteLine("{1}: Update requested from thread {2}, {0}", DateTime.Now, this.GetHashCode(), Thread.CurrentThread.ManagedThreadId);
IntPtr pBackBuffer = IntPtr.Zero;
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.WriteLine("{1}: Begin Image Update: {0}", DateTime.Now, this.GetHashCode());
_mappedBitmap.Lock();
pBackBuffer = _mappedBitmap.BackBuffer;
}));
// The processing of the back buffer
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.WriteLine("{1}: End Image Update: {0}", DateTime.Now, this.GetHashCode());
// the entire bitmap has changed
_mappedBitmap.AddDirtyRect(new Int32Rect(0, 0, _mappedBitmap.PixelWidth,
_mappedBitmap.PixelHeight));
// release the back buffer and make it available for display
_mappedBitmap.Unlock();
}));
}
}
}
I have dropped a lot of code here for the sake of bravity.
My test creates a task which calls SetData within certain intervals:
private void Button_Click_StartStressTest(object sender, RoutedEventArgs e)
{
var sleepTime = SleepTime;
_cts = new CancellationTokenSource();
var ct = _cts.Token;
for (int i = 0; i < ThreadsNumber; ++i)
{
Task.Factory.StartNew(() =>
{
while (true)
{
if (ct.IsCancellationRequested)
{
break;
}
int width = RandomGenerator.Next(10, 1024);
int height = RandomGenerator.Next(10, 1024);
var r = new Random((int)DateTime.Now.TimeOfDay.TotalMilliseconds);
var data = Enumerable.Range(0, width * height).Select(x => (float)r.NextDouble()).ToArray();
this.BeginInvokeInDispatcherThread(() => FloatingPointImageSource.SetData(data, width, height));
Thread.Sleep(RandomGenerator.Next((int)(sleepTime * 0.9), (int)(sleepTime * 1.1)));
}
}, _cts.Token);
}
}
I run this test with ThreadsNumber=1 and with SleepTime=100 and it crashes with the aforementioned exception.
UPDATE 2
I tried checking that my commands indeed execute serially.
I added another private field
private int _lockCounter;
And I manipulate it in my while loop:
private void UpdateImage()
{
while (true)
{
_updateRequired.WaitOne();
Debug.Assert(_lockCounter == 0);
_lockCounter++;
IntPtr pBackBuffer = IntPtr.Zero;
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.Assert(_lockCounter == 1);
++_lockCounter;
_mappedBitmap.Lock();
pBackBuffer = _mappedBitmap.BackBuffer;
}));
Debug.Assert(pBackBuffer != IntPtr.Zero);
Debug.Assert(_lockCounter == 2);
++_lockCounter;
// Process back buffer
Debug.Assert(_lockCounter == 3);
++_lockCounter;
Application.Current.Dispatcher.Invoke(new Action(() =>
{
Debug.Assert(_lockCounter == 4);
++_lockCounter;
// the entire bitmap has changed
_mappedBitmap.AddDirtyRect(new Int32Rect(0, 0, _mappedBitmap.PixelWidth,
_mappedBitmap.PixelHeight));
// release the back buffer and make it available for display
_mappedBitmap.Unlock();
}));
Debug.Assert(_lockCounter == 5);
_lockCounter = 0;
}
}
I was hoping that if the message order was somehow messed up my Debug.Asserts would catch this.
But everything with the counters is fine. They are incremented correctly according to the serial logic, and still I get the exception from AddDirtyRect.
Application.Current.Dispatcher.Invoke will try to execute the method that is passed as delegate to on the UI thread itself, this will happen when UI thread is free. If you try to execute the instructions on this continuously it'll almost nothing like performing the operation on the UI thread. Always the instruction executed on Application.Current.Dispatcher.Invoke should be very minimal say it should be only one line like which shall change only value on the UI nothing more than that. So avoid complex operations that are performed as part Dispatcher, move it out of dispatcher and do the only the operations that updates the UI
So after some (very long) digging, it turned out the real bug was hidden in the code I left out for the sake of bravity :-)
My class allows changing the size of the image. When setting data, I check if the new size is the same as the old size and if it isn't I initialize a new WritableBitmap.
What happened was that the size of the image was changed (using a different thread) sometime in the middle of the while loop. And this caused different stages of the processing code to process different instances of _mappedBitmap (since _mappedBitmap pointed to different instances throughout the different stages). So when the instance was changed to a new one, it was created in an unlocked state, thus causing the (rightful) exception.
Related
I am trying to graph data with Interval timing in my Windows Forms Application.
I need the graphing elements to be ran on a separate thread, but I have three different functions being called and cant simply use thread.start();
I tried running just the controls in it's own thread but it tells me that another thread owns the controls.
When I run the program as is, the chart lags horribly, meaning it needs to be in an independent thread
This all needs to be ran on a seperate thread because my form lags when updating this
public void GraphData()
{
var mapper = Mappers.Xy<MeasureModel>()
.X(model => model.DateTime.Ticks) //use DateTime.Ticks as X
.Y(model => model.Value); //use the value property as Y
Charting.For<MeasureModel>(mapper);
//the ChartValues property will store our values array
ChartValues = new ChartValues<MeasureModel>();
cartesianChart1.Series = new SeriesCollection
{
new LineSeries
{
Values = ChartValues,
PointGeometrySize = 1,
StrokeThickness = 2
}
};
cartesianChart1.AxisX.Add(new Axis
{
DisableAnimations = true,
LabelFormatter = value => new System.DateTime((long)value).ToString("hh:mm:ss"),
Separator = new Separator
{
Step = TimeSpan.FromSeconds(1).Ticks
}
});
var thread = new Thread(() =>
{
SetAxisLimits(System.DateTime.Now);
Timer = new System.Windows.Forms.Timer
{
Interval = 20
};
Timer.Tick += TimerOnTick;
Timer.Tick += timerdata_event;
Timer.Start();
});
thread.Start();
}
public ChartValues<MeasureModel> ChartValues { get; set; }
public System.Windows.Forms.Timer Timer { get; set; }
private void SetAxisLimits(System.DateTime now)
{
cartesianChart1.AxisX[0].MaxValue = now.Ticks + TimeSpan.FromMilliseconds(1000).Ticks; // force the axis to be 100ms ahead
cartesianChart1.AxisX[0].MinValue = now.Ticks - TimeSpan.FromSeconds(4).Ticks; // only care about the last 8 seconds
}
private void TimerOnTick(object sender, EventArgs eventArgs)
{
var now = DateTime.Now;
var queue = new Queue(tempStore);
var b = queue.Dequeue();
ChartValues.Add(new MeasureModel
{
DateTime = now,
Value = double.Parse(b.ToString())
});
SetAxisLimits(now);
if (ChartValues.Count > 100) ChartValues.RemoveAt(0);
}
Take a look at BackgroundWorker. It's a system component that allows a threaded procedure to do its work asynchronously and provides two callback events (ProgressChanged, RunWorkerCompleted) that are run on the UI thread.
there are two things ...
1. windows form use queue not threading, and it's build on STA (Single Thread Architect) event though when we look at it, it look like MTA...
So first thing, if you run in separate thread, you should synchronize with windows queue... it sometime even cause errors if you don't do that when you work with controls
calling: this (which this mean this form instance)
this.Invoke(new Action(delegate(){ /* your Synchronized code comes here */}))
or (Action)delegate() {} as i saw in internet
Second put calculation on background, perform something such as dual buffering... mean you don't neither apply, nor synchronize with your form, while you are performing calculation or drawings...
once you are done, you just apply does are drawn in the back....
So your end code will be like this:
{ //New thread function
// Calculates
// Draw arts
// Synchronize
this.Invoke((Action)delegate() {
//Perform Synchronizing Code
// Apply your drawn image and graphics to your image viewer, aka picture box
}
// End of thread, or if you need something extra to be performed afterward...
}
But if you do not have access to that component drawer.... try see how the performance look like, if you pass it parameters in the background, then add it to your form? if the processing of initializing the component is lower than changing graphic while running, if that looked good, go with that way, cause when you draw in some component, every change cause the form to redraw, and it comes with performance penalty.
I have a wpf application, It has mainWindow that creates _otherWindow that displays on a secondary monitor. Both windows have elements that need to change along with time. (mainWindow updates an Image and also plots a graph, finally _otherWindow updates a shape position depending on some computations).
What is my problem? well, I am reading a video frame by frame within a Thread (however I would like to allow this with stream taken with a camera). And as I update GUI every frame in a certain time, Application is getting a heavy load and is getting slow...
I realized that commenting either mainWindow updating Image, or commenting _otherWindow updating shape position codes make the application run nice, but the issue is when they run together.
Here is a detailed description
First I compute some things inside _otherWindow and compute position of a shape.
Then I compute some stuff related to image and the update frame adding some stuff to bitmap
Then I update position of shape inside _otherWindow
Finally I plot results (the plot needs data gotten from mainWindow and _otherWindow)
For this, I use tasks and wait for them.
I have this:
private Thread _camera;
private void CaptureVideo()
{
_camera = new Thread(CaptureVideoCallback)
{
Priority = ThreadPriority.Highest
};
_camera.Start();
}
private VideoCapture _capture;
private void CaptureVideoCallback()
{
//some computing here read from a video file...
_capture = new VideoCapture("someVideo.mp4");
for (var i = 0; i < _capture.FrameCount; i++)
{
_capture.Read(_frame);
if (_frame.Empty()) return;
//*************task that does heavy computation in other class
var heavyTaskOutput1 = Task.Factory.StartNew(() =>
{
_otherWindow.Dispatcher.Invoke(() =>
{
ResultFromHeavyComputationMethod1 = _otherWindow.HeavyComputationMethod1();
});
}
);
////*************task that does heavy computation in current class
var heavyTaskOutput2 = Task.Factory.StartNew(() =>
{
ResultFromHeavyComputationMethod2 = HeavyComputationMethod2(ref _frame);
var bitmap = getBitmapFromHeavyComputationMethod2();
bitmap.Freeze();
//update GUI in main thread
Dispatcher.CurrentDispatcher.Invoke(() => ImageSource = bitmap);
});
////*************wait both task to complete
Task.WaitAll(heavyTaskOutput1, heavyTaskOutput2 );
//update _otherWindow GUI
var outputGui = Task.Factory.StartNew(() =>
{
_otherWindow.Dispatcher.Invoke(() =>
{
_otherWindow.UpdateGui();
});
}
);
outputGui.Wait();
////*************plot in a char using gotten results, UPDATE GUI
Task.Run(() =>
{
PlotHorizontal();
});
}
}
What would be a good way to speed this up?
I mean I know that GUI stuff need to be done on main thread, but this is slowing down things.
Edit
Have changed code as Clemens suggested:
//*************task that does heavy computation in other class
var heavyTaskOutput1 = Task.Run(() =>
{
ResultFromHeavyComputationMethod1 = _otherWindow.HeavyComputationMethod1();
}
);
////*************task that does heavy computation in current class
var heavyTaskOutput2 = Task.Run(() =>
{
ResultFromHeavyComputationMethod2 = HeavyComputationMethod2(ref _frame);
var bitmap = getBitmapFromHeavyComputationMethod2();
bitmap.Freeze();
//update GUI in main thread
Dispatcher.CurrentDispatcher.Invoke(() => ImageSource = bitmap);
});
////*************wait both task to complete
Task.WaitAll(heavyTaskOutput1, heavyTaskOutput2);
//update _otherWindow GUI
var outputGui = Task.Run(() =>
{
_otherWindow.Dispatcher.Invoke(() =>
{
_otherWindow.UpdateGui();
});
}
);
outputGui.Wait();
It's a bit hard to guess. Do you have Visual Studio? I think even the Community edition has some profiling capabilities (menu: Analyze/Performance Profiler...). That may point out some non-obvious bottlenecks.
My thoughts:
getBitmapFromHeavyComputationMethod2 appears to return a new bitmap every time through. I can't infer the actual type it's returning, but it likely involves a semi-large un-managed memory allocation and implements IDisposable. You might check on whether you're disposing that appropriately.
Rather than create a new bitmap for every frame, can you use a WriteableBitmap? Be sure to lock and unlock it if you do. Perhaps ping-pong (alternate) between two bitmaps if you need to.
It appears you may be serializing your "heavy computation" with your I/O read (first one, then the other). Perhaps launch the read as an async as well, and wait on it in your WaitAll so that the computation and I/O can happen concurrently. Something in this shape:
var readResult = _capture.Read(_frame);
for (...) {
// check read result
// ...
// launch heavy computation
readResult = Task.Run(() => _capture.Read(nextFrame);
Task.WaitAll(pupilOutput, outputTest, readResult);
_frame = nextFrame;
}
Note this would read N+1 times for N frames--maybe your Read method is okay with that.
I'm trying to create an image slideshow effect with WPF.
The method to update the slideshow with a new image is called every few seconds in a Windows.Forms.Timer, and runs in it's own thread within a Task (as seen below).
private void LoadImage()
{
Task t = Task.Run(() =>
{
this.Dispatcher.BeginInvoke((Action)(() =>
{
TimeSpan delay = new TimeSpan(0, 0, 0, 0, 0);
Fader.ChangeSource(image, BitmapFromUri(new Uri(compPath + oComps[nCount].name)), delay, delay);
image.Visibility = System.Windows.Visibility.Visible;
mediaElement.Stop();
mediaElement.Close(); ;
mediaElement2.Stop();
mediaElement2.Close();
mediaElement.Visibility = System.Windows.Visibility.Collapsed;
mediaElement2.Visibility = System.Windows.Visibility.Collapsed;
imageLoop.Interval = oComps[nCount].duration;
nCount++;
imageLoop.Start();
}));
});
}
Simultaneously, there is a scrolling text banner across the bottom of the canvas in an overlay. This too is running in it's own thread, updating the UI through a Dispatcher.
Every few images, both the scrolling text and the slideshow will pause for a second or two, seemingly waiting for the image to load. This behaviour is unexpected as each element is in a seperate thread.
Could this be a conflict between the two Task threads updating the UI thread?
What could be causing this?
Your code to put work on another thread does not put the work on another thread. Your BeginInvoke is sending it back to the UI thread and all your work is being done there.
Do the heavy work before you do the BeginInvoke call so the work actually happens on the background thread.
private void LoadImage()
{
Task t = Task.Run(() =>
{
//I assume BitmapFromUri is the slow step.
var bitmap = BitmapFromUri(new Uri(compPath + oComps[nCount].name);
//Now that we have our bitmap, now go to the main thread.
this.Dispatcher.BeginInvoke((Action)(() =>
{
TimeSpan delay = new TimeSpan(0, 0, 0, 0, 0);
//I assume Fader is a control and must be on the UI thread, if not then move that out of the BeginInvoke too.
Fader.ChangeSource(image, bitmap), delay, delay);
image.Visibility = System.Windows.Visibility.Visible;
mediaElement.Stop();
mediaElement.Close(); ;
mediaElement2.Stop();
mediaElement2.Close();
mediaElement.Visibility = System.Windows.Visibility.Collapsed;
mediaElement2.Visibility = System.Windows.Visibility.Collapsed;
imageLoop.Interval = oComps[nCount].duration;
nCount++;
imageLoop.Start();
}));
});
I suspect your banner is also not actually doing work on the other thread, you may want to look in to it.
A even better solultion if possible is rewrite BitmapFromUri to be async and not use threads at all.
private async Task LoadImageAsync()
{
TimeSpan delay = new TimeSpan(0, 0, 0, 0, 0);
var bitmap = await BitmapFromUriAsync(new Uri(compPath + oComps[nCount].name);
Fader.ChangeSource(image, bitmap), delay, delay);
image.Visibility = System.Windows.Visibility.Visible;
mediaElement.Stop();
mediaElement.Close(); ;
mediaElement2.Stop();
mediaElement2.Close();
mediaElement.Visibility = System.Windows.Visibility.Collapsed;
}
We have a video player written in WPF with a scroll bar. When the scroll bar is dragged right-left the CurrentFrameTime is updated and triggers UpdateFrames which, in turn, grabs the frame and shows it. That works fine.
But, sometimes grabbing the frame can take time (because of the disk for example) and though CurrentFrameTime value can be already changed, the UpdateFrames can be "stuck" and still be waiting for previous frame in ...GetAsync().Result.
What is decided to do is to move Dispatcher.BeginInvoke into ContinueWith block. Now, each time the CurrentFrameTime is changed, the previous operation will be canceled(we don't need to show the frame if frame time was already changed) and an up-to-date frame should be shown. But, for some reason because of this change the application became slower. When I drag the scroll it can take a few seconds before the image is updated.
What could happened that moving the code into ContinueWith has slowed down the video player?
MainApplication without ContinueWith
_threadUpdateUI = new Thread(new ThreadStart(UpdateFrames));
public long CurrentFrameTime
{
get{...}
set
{
...
_fetchFrame.Set();
}
}
void UpdateFrames()
{
while(run)
{
_fetchFrame.WaitOne();
var frame = Cache.Default.GetAsync(CurrentFrameTime)
.Result;
Dispatcher.BeginInvoke(new Action(() => ShowFrame(frame.Time, frame.Image)));
}
}
Cache
public Task<VideoFrame> GetAsync(long frameTime)
{
//this i used when cache is disabled
if (GrabSynchronously)
{
var tcs = new TaskCompletionSource<VideoFrame>();
//reading from file
var frame2 = FrameProvider.Instance.GetFrame(frameTime);
tcs.SetResult(frame2);
return tcs.Task;
}
...
}
MainApplication WITH ContinueWith
void ShowFrames()
{
while(run)
{
_fetchFrame.WaitOne();
_previousFrameCancellationToken.Cancel();
_previousFrameCancellationToken = new CancellationTokenSource();
Cache.Default.GetAsync(CurrentFrameTime).ContinueWith((task) =>
{
var frameTime = task.Result.Time;
var frameImage = task.Result.Image
Dispatcher.BeginInvoke(new Action(() => ShowFrame(frameTime, frameImage)));
}, _previousFrameCancellationToken.Token);
}
}
df
In your old way your UpdateFrames loop would block every .Result call. This made your loop self metering, only allowing one request "in flight" at a time even if _fetchFrame got .Set() called many times while it was waiting for .Result to finish.
In your new way every call to _fetchFrame.Set() triggers another task to start up and be "in flight" (assuming GrabSynchronously is false) even if it never gets used. This is flooding your system with requests and is causing your slowdown.
One possible solution is to put another semaphore of some type to limit the number of concurrent requests for frames you can handle.
Semaphore _frameIsProcessing = new Semaphore(5, 5); //Allows for up to 5 frames to be requested at once before it starts blocking requests.
private void ShowFrames()
{
while (run)
{
_fetchFrame.WaitOne();
_previousFrameCancellationToken.Cancel();
_previousFrameCancellationToken = new CancellationTokenSource();
_frameIsProcessing.WaitOne();
Cache.Default.GetAsync(CurrentFrameTime).ContinueWith((task) =>
{
_frameIsProcessing.Release();
if(_previousFrameCancellationToken.IsCancellationRequested)
return;
var frameTime = task.Result.Time;
var frameImage = task.Result.Image;
Dispatcher.BeginInvoke(new Action(() => ShowFrame(frameTime, frameImage)));
});
}
}
This is my exception
The calling thread cannot access this object because a different thread owns it.
my function get results from a calculation and I want to update an already opened window..
public override void UpdateResult(BaseMetricResults result)
{
var newResults = result as MetricUniformityResults;
if (newResults == null)
{
return;
}
DispatcherHelper.UIDispatcher.Invoke(() =>
{
TopToBottomGraph.CrossSectionPoints.Clear();
foreach (var point in newResults.TopToBottomGraph.CrossSectionPoints)
{
TopToBottomGraph.CrossSectionPoints.Add(point);
}
newResults.JetMap.Freeze(); //exception here
byte[] arr = new byte[(int) (newResults.JetMap.Width*newResults.JetMap.Height*3)];
newResults.JetMap.CopyPixels(arr, (int) (newResults.JetMap.Width*3), 0);
JetMap = BitmapSource.Create((int) newResults.JetMap.Width, (int) newResults.JetMap.Height, 96, 96,
PixelFormats.Rgb24, BitmapPalettes.WebPalette, arr,
(int) (newResults.JetMap.Width*3));
});
}
This is my last attempt I'm not sure if I have to freeze the bitmapsource or not...
Anyway newResults.JetMap is BitmapSource, and I have a property named JetMap which is the new BitmapSource, how can I update the old image with the new one?
You need to call Jetmap.Freeze(); right after you create it and not inside the dispatcher, once its its frozen you can set it inside the dispatcher and you wont get an exception
Your DispatcherHelper.UIDispatcher.Invoke method will execute on the UI thread. My best guess is that the newResults.JetMap bitmap was created on a different thread which is preventing you from modifying it. At the same time, you can't create the JetMap bitmap that you want to show on a thread other than the UI thread. So without more context, the best suggestion would be to ensure that the newResults.JetMap bitmap is also created in the main UI thread.