WPF realtime chart application architecture - c#

I have the following scenario in mind:
I want to send (via serial port) some commands to a device. This device does send me back a continuous stream of data (max. 12000 values per second).
To control some settings I need some buttons to send commands to the device to start/stop/change settings before and during data stream. Also I want to have a real time plot of this data. I will filter this data of course. Also at certain timestamps there will be a signal which indicates that I want to cut out a certain window of the received data.
This means I will have two charts. I made already some progress using WPF but now when I interact (zoom/pan) with the lower chart, the upper one freezes noticeable. This is because both have do be refreshed very often!
Work (data receiving/filtering) is done using threads but the update of the plot has to be done within the ui thread.
Any ideas how to solve this issue? Maybe using multiple processes?

You should use Reactive Extensions. It was built for this kind of thing.
http://msdn.microsoft.com/en-us/data/gg577609.aspx
Requesting a clear, picturesque explanation of Reactive Extensions (RX)?
On this second link, although the topic is javascript, much of what it says is about Reactive Extensions and cross-applies to Rx in C#.

I'm making a similar WPF application with real-time waveforms (about 500Hz). I have a background threads that receives real-time data, a separate threads to process them and prepare the data for drawing (I have a buffer with the "size" of the screen where I put the prepared values). In the UI thread I draw the waveforms to the RenderTargetBitmap which is in the end is rendered to the Canvas. This technique allows me have a lot of real-time waveforms on the screen and have zoom and pan working without any problems (about 40-50 fps).
Please let me know if you need some technical details, I can later share them with you.
I think you have some code in the UI thread that is not optimized well or can be moved to the background thread.
Btw, do you use any framework for charts?
Edit
philologon is right, you should use Rx for real-time data, it simplifies code A LOT. I also use them in my project.

Its a commercial product but there is a real-time WPF chart which can handle this use-case and then some. Please take a look at the Tutorial below:
http://www.scichart.com/synchronizing-chartmodifier-mouse-events-across-charts/
There is a live Silverlight demo of this behaviour here:
Sync Multichart Mouse Silverlight Demo
And this chart should be able to handle zooming while inputting values at high speed:
Realtime Performance Demo
Disclosure: I am the owner and tech-lead of SciChart

Related

Is that possible to get GUI refresh times at WPF application

I really need to calculate responsiveness of my UI. Some suggested to use WpfPerf.exe (performance profiling tools for WPF). It works great however it does not have logging so it shows only last 1 minute of FPS as image.
However i need statistical information. The gui runs laggy so i need to record when exactly gui is refreshed. When there is no lag it should be refreshed each 0.5 seconds.
So i wonder anyhow i can record the time whenever gui is refreshed ?
.net 4.5.2 c# wpf
To record the time the GUI is refreshed. You may refer to
Why is Frame Rate in WPF Irregular and Not Limited To Monitor Refresh?
It may be out of topic. I would like to share what I do to improve my application last week.
Use visual studio performance and diagnostic, if available, to find bottleneck.
use BackgroundWorker to prepare data, instead of doing it in UI thread.
use data virtulization for dispalying huge amount data, for example million rows. This article provides an excellent solution for displaying millon rows data smoothly.
Hope It will help.

Hardware Acceleration in MFC

I am currently developing a software that involves real-time rendering of performance metrics in graph and chart forms. I need to acquire data, process data and render an image as fast as possible. My backend is in C++, and I am at a point where I have to make a choice regarding the front end.
Given my backend is in C++, I was inclined to go with MFC. The other alternative is to go with WPF C# for frontend and interop with my C++ backend. I read recently that WPF provides for hardware acceleration, this should help me achieve a high frame rate. Does MFC hardware accelerate its graphics too? Does hardware acceleration even matter?
Given WPF's hardware acceleration, does that make WPF the most efficient alternative for graphics in my case?
WPF provides hardware acceleration using DirectX 9 I believe. However for line graphs the limiting factor is the amount of interface elements. We are creating a program in WPF that displays sEMG data real-time using Telerik charts. These can be configured to use direct2d acceleration under the hood which cranks up the performance a bit if you have a lot of datapoints. It's still jerky though because you cannot control the render thread of WPF.
It kind of depends on the amount of features you need in the graph. If the priority is to have silky smooth real-time display don't go with WPF.
Visual Studio 2010 added classes to MFC to support using Direct2D rendering from MFC programs.
To use Direct2D, you start by calling EnableD2DSupport() in your View's OnCreate (technically, I suppose it doesn't have to be in OnCreate, but that's the usual place). Then you'll receive AFX_WM_DRAW2D messages when the D2D display context needs updating, so you'll normally want to add a handler for that, and respond to it by rendering your content as needed.
Another possibility to consider would be to use an existing control to draw your graphs. There are quite a few around, including some that are free with quite liberal licensing. Just for example, CodeProject has a number of Charting controls, a few of which use D2D for their drawing, and quite a few more that don't.
Honestly, I'd be a little surprised at charts having information being updated fast enough for drawing speed to make a huge difference as a rule though. In most typical cases, the real limit will be the user's ability to comprehend what you're drawing. A user simply can't watch 100 different graphs each being updated at (say) 60 Hz, and have much hope of deriving much real meaning from most of them. In most cases, the real challenge isn't to draw more data faster, but to provide better ways for the user to focus on the few things they can follow at a time, and (for example) draw their attention to important changes when needed.

C# Win RT Metro image data storage and exchange with UI on another thread

I am writing Metro application for image sharing (kind of). I have separate project for downloading and handling communication to/from server (running on separate thread). I am now wondering what classes should I use for storing image data. I remeber good old days when there was Image class just for this purpose. But today I can seem to find anything like it. There are classes like BitmapSource and DrawingImage, but those are UI classes and can not/should not be used in non-UI threads. What would you suggest me for storing image data and easily transferin it back to UI thread, so I can use it as ImageSource (preferably).
EDIT: I would like to keep image data in memory, if possible (not save to galery/remove on exit)
EDIT: So far I am using IRandomAccessStream, but I dont like this solution, because you can read only once from it (then reset must be performed). Further more - i am not sure how it behaves when multiple BitmapImage(s) have it set as source..
See http://winrtxamltoolkit.codeplex.com/ - the WriteableBitmap may be the ticket. Haven't played with it, but the overview seems in the ball park.

Fastest way to Decode and Display many H264 Videos Simultaneously C#

As you might surmise from the question title, we are required to decode and display multiple (e.g., eight) H.264 encoded videos at the same time (and keep them all time synchronized, but that's another question for another time). The videos are usually at at 25 FPS with a resolution of 640x480.
I'm going to provide a bit of background before I get to the crux of the problem.
The feature needs to be baked into a fairly large C# 3.5 (WinForms) application. The videos will occupy rectangles in the application - the managed code needs to be able to specify where each video is drawn as well as it's size.
We get the H264 packets in C# and fire them into a native H264 decoder to get YUV12 image data.
An early attempt consisted of converting the YUV12 images to RGB24 and BitBlt'ing them to a HWND passed into the native code from C#. While functional, all BitBlt'ing had to happen on the UI thread which caused it to bog down when displaying more than a couple videos (on a 2.6 GHZ core 2 duo).
The current attempt spins up one-thread-per-cpu-core on startup and load balances the decoding/displaying of videos across these threads. The performance of this is mind-blasting (I find watching task manager much more interesting than the videos being displayed). UI-wise, it leaves a lot to be desired.
The millisecond we started drawing to an HWND created on the UI thread (e.g., a panel docked in a WinForms control) from a non-UI thread, we started getting all sorts of funky behavior due to the un-thread-safeness of WinForms. This led us to create the HWND's in native code and draw to those, with C# providing the rectangles they should be drawn to in screen coordinates.
Gah! CanOfWorms.Open().
Problem: When the C# application receives focus, it jumps to the front of the Z-Order and hides the video windows.
Solution: Place the video windows Always On Top.
Problem: When the user switches to another application, the video windows are still on top.
Solution: Detect activation and deactivation of the C# application and show/hide the video windows accordingly.
Problem: User says, "I want my videos playing on one monitor while I edit a Word document in the other!"
Solution: Tell user to shut up and that Word sucks anyways.
Problem: I get fired.
etc. etc.
I guess the crux of the problem is that we have HWND's created on a non-UI thread and we want to 'simulate' those being embedded in the C# application.
Any thoughts/suggestions? Am I completely out to lunch here?
I'm more than open to taking a completely different approach if one exists (This project required a lot of learning - winning the lottery would have a greater likelihood than me having picked the best approach at every step along the road).
Forget about BitBlt-ing and do this:
for each window you want your video to be played, create one DirectShow graph and attach the renderer of the graph to that window
before renderer in the graph put the samplegrabber filter. It will allow you to have callback in which you'll be able to fill the buffer
instead of blitting, decode to the buffer provided in samplegrabber.
In addition, I guess that you'll be able to put raw YUV12 into the buffer, as VMRenderer is able to display them directly.
Use DirectShowNet library.
EDIT:
And yes, BTW, if the videos are on the same 'canvas', you can use same technique with renderer and create only one large window, then shift decoded video rectangles 'by hand' and put them into the framebuffers buffer.
YET ANOTHER EDIT:
BitBlts are ALWAYS serialized, i.e. they can't run in parallel.
The millisecond we started drawing to an HWND created on the UI thread (e.g., a panel docked in a WinForms control) from a non-UI thread, we started getting all sorts of funky behavior due to the un-thread-safeness of WinForms. This led us to create the HWND's in native code and draw to those, with C# providing the rectangles they should be drawn to in screen coordinates.
What kind of funky behavior?
If you mean flickering or drawing delay, have you tried to lock() the panel or any other class for thread/drawing synchronisation?
Again: Whats the exact problem when you send the data to the decoder, receive a image, convert it and then draw it with an OnPaint handler. (Setup a different thread that loops at 25fps, call panel1.Invalidate() then)
I guess the crux of the problem is that we have HWND's created on a non-UI thread and we want to 'simulate' those being embedded in the C# application.
Don't do that. Try to draw the received data in your c# application.
In general, I wouldn't reccomend mixing native code and c#. Having the h264 decoder in native code is the only exception here.
Use your threads to decode the video packets (as you already do) then have one thread that loops and calls Invalidate(as said above). Then have an OnPaint handler for each panel you are displaying a video in. In this handler get the most recent video picture and draw it (e.Graphics).
I hope this helps, but also need more information about the problem...
I like the DirectShow answer posted earlier, but I wanted to include an additional option that might be easier for you to implement, based on this excerpt from your question:
While functional, all BitBlt'ing had to happen on the UI thread which caused it to bog down when displaying more than a couple videos
My idea is to start from that code, and use the Async CTP for Visual Studio 2010 that is currently available and includes a go-live license. From there it should be a relatively simple to modify this existing code to be more responsive: just add await and async keywords in a few places and the rest of the code should be largely unchanged.

How can I edit individual pixels in a window?

I want to create a simple video renderer to play around, and do stuff like creating what would be a mobile OS just for fun. My father told me that in the very first computers, you would edit a specific memory address and the screen would update. I would like to simulate this inside a window in Windows. Is there any way I can do this with C#?
This used to be done because you could get direct access to the video buffer. This is typically not available with today's systems, as the video memory is managed by the video driver and OS. Further, there really isn't a 1:1 mapping of video memory buffer and what is displayed anymore. With so much memory available, it became possible to have multiple buffers and switch between them. The currently displayed buffer is called the "front buffer" and other, non-displayed buffers are called "back buffers" (for more, see https://en.wikipedia.org/wiki/Multiple_buffering). We typically write to back buffers and then have the video system update the front buffer for us. This provides smooth updates, as the video driver synchronizes the update with the scan rate of the monitor.
To write to back buffers using C#, my favorite technique is to use the WPF WritableBitmap. I've also used the System.Drawing.Bitmap to update the screen by writing pixels to it via LockBits.
It's a full featured topic that's outside the scope (it won't fit, not that i won't ramble about it for hours :-) of this answer..but this should get you started with drawing in C#
http://www.geekpedia.com/tutorial50_Drawing-with-Csharp.html
Things have come a bit from the old days of direct memory manipulation..although everything is still tied to pixels.
Edit: Oh, and if you run into flickering problems and get stuck, drop me a line and i'll send you a DoubleBuffered panel to paint with.

Categories

Resources