I'm building a line chart control in Silverlight using PolyLineSegment and points. It works just as expected, but the application freezes for a long time when there's too much data that needs to be visualized (too many points). I can't move my code on a separate thread for an obvious reason - it deals with UI elements directly, so when I try to call them from a separate thread it results in exception (even if UI elements are not yet rendered).
Is there any way to create UI elements dynamically on a background thread and then pass them to the UI thread to be rendered? And if not, what would be the possible solution? I'm thinking of creating an Bitmap image instead of actual controls, but there won't be much interactivity in this case.
It sounds like you need to get a faster way of rendering your points. If you have 800k samples and only say, 800 pixels to display them in you're wasting 1000 points per pixel of calculations if you just load it into a PolyLineSegment.
I would revisit 'interpolating' the points (this is really coalescing for your large dataset). You want to make sure you capture the dynamic range of the function in each pixel correctly:
Figure out how many pixels wide the graph should be
Determine how many points per pixel in the X direction
For each chunk of points:
Build a histogram of the points
Draw a vertical line from max->min on your graph at the X where these points will map to. This captures the full range represented in the chunk.
If your points/pixel gets close to 1 you'll want to switch to the easy rendering to give better visual results as well.
For displaying a waveform (in your case PCM audio data) with "millions of points" you would be better off writing directly to a WritableBitmap. You then have only one render object.
You have already said there is not much processing in your calculations. Trying to use individual UIElements is way too big an overhead (IMHO). Point display is trivial to a bitmap and there are plenty of line drawing algorithms out there, optimised for speed, to do any line segments.
You can plot your points on a background thread and them update an image's ImageSource at the end of the processing to display it.
You certainly can do your compute work on background thread(s) and pass the finished results up to the UI tread with
Deployment.Current.Dispatcher.BeginInvoke
which is discussed here
Related
this is my first question, however I'm a long time lurker. I'll split up this into two parts, one part explaining what I'm doing and why I think this is the way to go, the second one being the actual question that I can't solve for myself.
What am I doing?
I'm currently developing a framework for rendering 2-dimensional features meant to be displayed in real-time. You can think of an application like Google Maps in your browser, however the framework is meant to render all kinds of geographical data (not just axis-aligned raster data, like those Google Tiles).
The framework is to be integrated into our (the company's) newest product which is a WPF application for the desktop and laptop.
Therefore I chose WPF for actually rendering geometry only; Visibility and Occlusion Culling are done by myself as well as input handling (mouse picking), moving the camera, etc..
Being a real-time application, it need to achieve at least 30 FPS. The framework performs adequate when rendering images: I can draw several thousand bitmaps per frame without a problem, however polyonal data turns out to be a major problem.
The actual question
I'm rendering my fair amount of polyline and polygon data using WPF, specifically using DrawingContext and StreamGeometry. My understanding so far is that this is the way to go for if I need performance. However I am not able to achieve the results that I expected from this.
This is how I fill the StreamGeometry with actual data:
using (StreamGeometryContext ctx = Geometry.Open())
{
foreach (var segment in segments)
{
var first = ToWpf(segment[0]);
ctx.BeginFigure(first, false, false);
// Skip the first point, obviously
List<Point> points = segment.Skip(1).Select(ToWpf).ToList();
ctx.PolyLineTo(points, true, false);
}
}
Geometry.Freeze();
And this is how I draw my geometry:
_dc.PushTransform(_mercatorToView);
_dc.DrawGeometry(null, _pen, polyline);
_dc.Pop();
As a test, I loaded ESRI shapes from OpenStreetMap into my application to test its performance, however I'm not satisfied at all:
My test data consists of ~3500 line segments with a total of ~20k lines.
Mapping each segment to its own StreamGeometry performed extremely bad, but I kinda expected that already: Rendering takes about 14 seconds.
I've then tried packing more segments into the same StreamGeometry, using multiple figures:
80 StreamGeometry, Rendering takes about 50ms.
However I can't get any better results than this. Increasing the amount of lines to around 100k makes my application nearly unusable: Rendering takes more than 100ms.
What else can I do besides freezing both the geometry as well the pen when rendering vector data?
I'm at the point where I'd rather make use of DirectX myself than to rely on WPF for me do to it because something seems to be going terribly wrong.
Edit
To further clarify what I am doing: The application visualizes geographic data in real-time, very much like an application like Google Maps in the browser: However it is supposed to visualize much, much more data. As you may know, Google Maps allows both zooming and panning, which requires > 25 FPS for it to appear as a fluent animation; anything less does not feel fluent.
*
Sorry but I shouldn't upload a video of this before the actual product is released. You may however envision something like Google Maps, however with tons of vector data (polygons and polylines).
*
There are two solutions, one of which is very often stated:
Cache heavy drawings in a bitmap
The implementation seems kinda easy, however I see some problems with this approach: In order to properly implement panning, I need to avoid drawing the heavy stuff each frame, and therefore I am left with the choice of either not updating the cached bitmap while panning the camera, or creating a bitmap which covers an even bigger region than the viewport, so that I only need to update the cached bitmap every so often.
The second "problem" is related to zooming. However it's more of a visual artifact than a real problem: Since the cached bitmap can't properly be updated at 30 FPS, I need to avoid that when zooming as well. I may very well scale the bitmap while zooming, only creating a new bitmap when the zoom ends, however the width of the polylines would not have a constant thickness, although they should.
This approach does seem to be used by MapInfo, however I can't say I'm too fond of it. It does seem to be the easiest to implement though.
Split geometry up into different drawing visuals
This approach seems to deal with the problem differently. I'm not sure if this approach works at all: It depends on whether or not I correctly understood how WPF is supposed to work in this area.
Instead of using one DrawingVisual for all stuff that needs to be drawn, I should use several, so that not every one needs to be RenderOpened(). I could simply change parameters, for example the matrix in the sample above, in order to reflect both camera panning and moving.
However I see some problems with this approach as well: Panning the camera will inevitably bring new geometry into the viewport, hence I would need to perform something similar than in the first approach, actually render stuff which is currently not visible, but may become visible due to the camera shifting; Drawing everything is out of the question as it may take ridiculous amounts of times for a rather small amount of data.
Problem related to both approaches
One big problem which neither of these approach can solve is that even if the overall frame-rate is stable, occasional hickups, either when updating the cached bitmaps (okay, this doesn't apply if the cached bitmap is only updated when the camera is no longer panned) or calling RenderOpen to draw the visible chunk of geometry, seem to be inevitable.
My thoughts so far
Since these are the only two solutions I ever see to this problem (I've done my fair share of googling for more than a year), I guess the only solution so far is to accept frame-rate hickups on even the most powerful GPUs (which should be able to rasterize hundreds of millions of primitives per second), a delayed updating of the viewport (in the case where bitmaps are only updated when the viewport is no longer moved) or to not use WPF at all and resort to DirectX directly.
I'm very glad for the help, however I can't say I'm impressed by WPFs rendering performance so far.
To improve 2D WPF rendering performance you could have a look at the RenderTargetBitmap (for WPF >= 3.5) or the BitmapCache class (for WPF >= 4).
Those classes are used for Cached Composition
From MSDN:
By using the new BitmapCache and BitmapCacheBrush classes, you can cache a complex part of the visual tree as a bitmap and greatly improve rendering time. The bitmap remains responsive to user input, such as mouse clicks, and you can paint it onto other elements just like any brush.
I would like to know if is there some good web solutions to show charts for "huge data sets", I've tried amcharts and Highcharts Stock (jquery solutions) without success.
At the beginning they were working, but at the moment the "chrome" is telling me that the javascript memory is full and the page crashes.
I've times where I need to show more than 20 lines, each one with more than 100.000 points, so in the end I can have gigantic jquery arrays that sure will crash the internet browser.
At the moment I am open to change to some flash, silverlight or other solution (not java applet because I am using C#).
What do you guys recommend?
UPDATE #1
For example: one purpose of this application is to see ECG channels.
The person will carry a device with several "sensors" (lets define 10 or 12, more or less), the device will save the data each second (or sometimes even in shorter intervals). And there can be cases that the person will use this system for 3 days).
Minimum data:
60 seconds * 60 minutes * 24 hours * 3 days = 259.200 points per line.
8 lines or more => 2.073.600 total points
Usability:
Well, in this health area normally the "readings" will be similar, no highs or lows enough to be recognized in a 3 days data. So for this example the best would be to load the data just when it is needed > the pan/zoom slide is showing just 1hour and when it moves to other, then AJAX get the rest of the DATA. Sure this is the way to go. BUT this is not the only case in my system.
I've other type or devices where the "highs and lows" are HUGE and the user would like to see ALL data in just one "chart" without zoom in. So, in this situations just from a simple look it is easy to see that something happened on the readings, then the user can make zoom in and since the data is already on memory no need to make more AJAX calls and refresh the chart.
Smart way to go: process the data in a way to do "reduce" the number of points when we are looking at a bigger "scale". Sure, this is the wise way to go, but once again, there are times when the result of some processing math will "fake" and hide the real readings and in the end there are some "behaviors" that will not show up on the chart.
So, for now I really need to find a way to display all of this points.
Note: I really appreciate all the feedback of you guys.
I think I'm with Neil here...there must be some way that this data can be processed before display...I mean, how can this amount of data even be displayed in a window? You say a line has 100000 points...if each of those points was unique in the X,Y plane, 100000 points would completely fill a 300x300 display window. 20 lines like this would completely saturate a normal 1024x1280 display.
Presumably, that can't be what you are looking for, so I'm assuming there must be a lot of cases where the points overlap. Preprocessing the data, to eliminate duplicate data points would help reduce the data size considerably.
It's hard to know exactly how this answer fits, or to give more precise instructions without further details, but if you have questions or clarifications, edit your question and I'll modify my answer (or delete it, if I've misinterpreted.)
Response to Edit 1:
I think that the way to approach the thinking for this is to recognize that for any given view, you can only show as many data points as you have horizontal resolution, so you can limit your data download to that.
From what I'm hearing (and I grant that I have very few details) this problem can be reduced to:
Figuring out how many points to get (based on horizontal resolution)
Calculating those points based on the data, horizontal scroll, zoom, and any heuristics.
Dynamically downloading that data
That sounds not too bad, and your original problem (of too much data crashing the system) disappears. That leaves you with secondary problem of how to calculate the height of the downloaded data.
I've other type or devices where the "highs and lows" are HUGE and the
user would like to see ALL data in just one "chart" without zoom in.
So, in this situations just from a simple look it is easy to see that
something happened on the readings...
There are a number of potential difficulties that I can see here...
If the timescale for these events is too short, they won't be visible on a naively drawn graph. If you have 100000 points in a particular line graph and your default viewing area is 1000 pixels wide with no zooming, you're only seeing 1 out of 100 datapoints. If some spike lasts for 10 of the datapoints, for example, unless you do something special, there's a good chance it won't be visible on the graph (so the user won't know to "zoom in" for more resolution). And how do you determine the height at which to plot the point? The actual datapoint at a specific spot? An average of the 100 data points that pixel covers? A rolling average? If don't average, you could miss spikes entirely. If you do average, you could lower the amplitude of the spikes or troughs if they are of short duration.
This, I think (and, again, I'm doing a lot of guesswork) sounds like the real challenge. Trying to find some way to display the graph which will definitely not be able to show all of the data at one time, but may be able to have some way to highlight points of interest dynamically (calculating, noting, and marking peaks and troughs with notations on the graph...things like that.)
Try out the Zoom Line chart from the FusionCharts stables.
I've myself created charts with 27,000 datapoints; and beyond the initial loading times, the chart worked smoothly.
Here is a blog post about the zoom line chart - http://blog.fusioncharts.com/2011/10/stuck-between-massive-historical-data-and-daily-intricacies-zoom-line-chart-to-the-rescue/
As a bonus, you can render the chart in pure JavaScript or Flash.
And it also works well with server-side languages. Check out their docs for more reading material - http://docs.fusioncharts.com/
We're currently creating a simple application for image manipulation in Silverlight, and we've hit a bit of a snag. We want users to be able to select an area of an image (either by drawing a freehand line around their chosen area or by creating a polygon around it), and then be able to apply effects to the pixels within that selection.
Creating a selection of images is easy enough, but we want a really fast algorithm for deciding which pixels should be manipulated (ie. something to detect which pixels are within the user's selection).
We've thought of three possibilities so far, but we're sure that there must be a really efficient and quick way of doing this that's better than these.
1. Pixel by pixel.
We just go through every pixel in an image and check whether it's within the user selection. Obviously this is far too slow!
2. Using a Line Crossing Algorithim.
The type of thing seen here.
3. Flood Fill.
Select the pixels along the path of the selection and then perform a flood fill within that selection. This might work fine.
This must a problem that's commonly solved, so we're guessing there's a ton more solutions that we've not even thought of.
What would you recommend?
Flood fill algorithm is a good choice.
Take a look at this implementation:
Queue-Linear Flood Fill: A Fast Flood Fill Algorithm
You should be able to use your polygon to create a clipping path. The mini-language for describing polygons for Silverlight is quiet well documented.
Alter the pixels on a copy of your image (all pixels is usually easy to modify than some pixels), then use the clipping path to render only the desired area of the changes back to the original image (probably using an extra buffer bitmap for the result).
Hope this helps. Just throwing the ideas out and see if any stick :)
I'm having a problem where drawing a grid using LineList and another (larger) grid overlapping it will make them flicker due to z-fighting. Using DepthBias will reduce that kind of problem when polygons and lines overlap but it apparently doesn't work when drawing lines in two separate DrawIndexedPrimitives calls.
Currently I "fixed" it by adding to the position of the second grid a small vector pointing towards the camera to simulate the DepthBias but the problem still happens when the camera is far from the grids.
Is there a better way to work around this problem?
From what I've heard you should take a look at your clip-planes. Example thread: xna.com
Edit: Dunno, about grids though, but you could always try! :)
Unfortunately, this is the natural behavior due to the limited precision of 32bit floating point numbers (as used by the depth buffer). You can either translate one set of lines minimally (as You do now) and try to chose Your clipping planes as close to each other as possible (as Rob mentioned), or:
Disable the depth buffer by setting device.RenderState.CompareFunction = CompareFunction.Allways, not by actually disabling the buffer!
Draw all Your lines.
Enable the depth buffer again by reversing the changes in step 1.
Draw all Your other geometry.
I have written a chart that displays financial data. Performance was good while I was drawing less than 10.000 points displayed as a connected line using PathGeometry together with PathFigure and LineSegments. But now I need to display up to 100.000 points at the same time (without scrolling) and it's already very slow with 50.000 points. I was thinking of StreamGeometry, but I am not sure since it's basically the same as a PathGeometry stroring the information as byte stream. Does any one have an idea to make this much more performant or maybe someone has even done something similar already?
EDIT: These data points do not change once drawn so if there is potential optimizing it, please let me know (line segments are frozen right now).
EDIT: I tried StreamGeometry. Creating the graphic took even longer for some reason, but this is not the issue. Drawing on the chart after drawing all the points is still as slow as the previous method. I think it's just too many data points for WPF to deal with.
EDIT: I've experimented a bit and I noticed that performance improved a bit by converting the coordinates which were previously in double to int to prevent WPF anti-aliasing sub-pixel lines.
EDIT: Thanks for all the responses suggesting to reduce the number of line segments. I have reduced them to at most twice the horizontal resolution for stepped lines and at most the horizontal resolution for simple lines and the performance is pretty good now.
I'd consider downsampling the number of points you are trying to render. You may have 50,000 points of data but you're unlikely to be able to fit them all on the screen; even if you charted every single point in one display you'd need 100,000 pixels of horizontal resolution to draw them all! Even in D3D that's a lot to draw.
Since you are more likely to have something like 2,048 pixels, you may as well reduce the points you are graphing and draw an approximate curve that fits onto the screen and has only a couple thousand verts. If for example the user graphs a time frame including 10000 points, then downsample those 10000 points to 1000 before graphing. There are numerous techniques you could try, from simple averaging to median-neighbor to Gaussian convolution to (my suggestion) bicubic interpolation. Drawing any number of points greater than 1/2 the screen resolution will simply be a waste.
As the user zooms in on a part of a graph, you can resample to get higher resolutions and more accurate curve fitting.
When you start dealing with hundreds of thousands of distinct vertices and vectors in your geometry, you should probably consider migrating your graphics code to use a graphics framework instead of depending on WPF (which, while built on top of Direct3D and therefore capable of remarkably efficient vector graphics rendering, has a lot of extra overhead going on that hampers its efficiency). It's possible to host both Direct3D and OpenGL graphics rendering windows within WPF -- I'd suggest moving that direction instead of continuing to work solely within WPF.
(EDIT: changed "DirectX" in original answer to "Direct3D")
Just ran into this question, but as I mentioned in this thread, the most performant approach might be to program against WPF's Visual layer.
Everything Visual in WPF eventually goes against this layer ... and so it is the most lightweight approach of them all.
See this and this for more info. Chapter 14 of Matthew MacDonald's Pro WPF in C# 2008 book also has a good section on it.
As another reference ... see Chapter 2 of Pavan Podila's book WPF Control Development Unleashed. On page 13, he discusses how DrawingVisuals would be an excellent choice for a charting component.
Finally, I just noticed that Charles Petzold wrote an MSDN Magazine article where the best overall (performant anyway) solution (to a scatter plot) was a DrawingVisual approach.
Another idea would be to use the Image control with the Source property set to a DrawingImage that you've dynamically created.
According to Pavan Podila in WPF Control Development Unleashed, this approach can be very helpful when you have thousands and thousands of visuals that don't need any interactivity. Check out page 25 of his book for more info.
This is an old thread, but I thought it was worth mentioning that you could attain interactivity with the above method by using the MouseUp() event. You know the size of the image's viewport, the resolution of the image, and the mouse's position. For example, you could maintain the collection actualScreenPoints through a timer attached to your UserControl_SizeChanged event:
double xworth = viewport.ActualWidth / (XEnd - XStart);
double xworth = viewport.ActualHeight / (YEnd - YStart);
List<Point> actualScreenPoints = new List<Point>();
for (var i = 0; i < points.Count; i++)
{
double posX = points[i].X * xworth;
double posY = points[i].Y * yworth;
actualScreenPoints.Add(posX, posY);
}
And then when your MouseUp() event fires, check if any of the points in the collection are within +-2px. There's your MouseUp on a given point.
I don't know how well it scales, but I've had some success using ZedGraph in WPF (WinForms control inside a WindowsFormsPresenter). I'm surprised no one mentioned it yet. It's worth taking a look at, even if you're not planning on using it for your current project.
ZedGraph
Good luck!
I believe the only method that might be faster while remaining in the WPF framework would be to override OnRender in a custom control. You can then render your geometry directly to the persisted scene, culling anything out of view. If the user can only see a small part of the data set at a time, culling could be enough on its own.
With this many data points, it's unlikely that the user can see full detail when the entire dataset is in view. So it might also be worthwhile to consider simplifying the dataset for full view and then showing a more detailed view if and when they zoom in.
Edit: Also, give StreamGeometry a shot. Its whole reason for existing is performance, and you never know until you try.
This is a very good question, and at it's heart begs the question "Can any user make practical use of, or business descisions from, a screen containing 100,000 discrete points?".
Following best practice in GUI design philosphy, the answer should be No, which would lead me to question whether there isn't a different way to meet the requirement for the application.
If there really is a bona-fide case for displaying 100,000 points on screen, with no scrolling, then using an off-screen buffer is the way to go. Composite your image to a bitmap, than whack that bitmap onto your Window / Page as needed. This way the heavy lifting is only done once, after which the hardware acceleration can be used every time the window needs to be drawn.
Hope this helps.
I haven't worked with WPF (disclaimer), but I suspect that your performance problem is because your code is trying to fit a smooth curved line through all of your data, and the time required increases geometrically (or worse) with the number of data points.
I don't know if this would be acceptable appearance-wise, but try graphing your data by connecting each point to the last with a straight line. This should make the time-to-graph proportional to the number of data points, and with as many points as you have the graph may end up looking exactly the same anyway.
Another idea would be to use the Image control with the Source property set to a DrawingImage that you've dynamically created.
According to Pavan Podila in WPF Control Development Unleashed, this approach can be very helpful when you have thousands and thousands of visuals that don't need any interactivity. Check out page 25 of his book for more info.