I need to speed up my image viewer, and wondering if I should be looking into creating my own DirectX control to do so.
My image viewer displays medical images. They can be pretty large. We're talking 55mb when it comes to mammography. The pixel data is 16bit greyscale stored in a ushort array. Without getting into the gory details, my current approach is loading the pixel data into an ImageSource, and using the WPF Image control.
I've never done anything with DirectX. Is it worth diving into it? Would it be any faster than the native WPF stuff? If so how significantly? Or, should I just forget about DirectX and look into areas where I can improve my current approach?
Before somebody says so, I know WPF utilize DirectX. I'm wondering If removing the WPF layer and writing the DirectX myself will improve performance.
I have some experience drawing multi-gigabyte satellite and chart imagery. Working with imagery around 55MB should probably work okay even without trying to optimize it too much. You haven't really given enough detail to recommend one alternative over the other, so I will give my opinion on the pros and cons.
Using 2D windows APIs will be the simplest to implement and should always be fast enough if you don't need to rotate and simply want to display an image and zoom and pan around. If you treat it as one large image the performance will not be as good when you zoom out if you are drawing with halftoning to give a nice smooth image. This is because it will effectively have to read all 55mb of image every time it draws.
To get around this performance issue you can make multiple bitmaps, effectively mip-mapping your image. As you zoom out you can pick the reduced resolution image closest to the resolution you are trying to draw . If you are not familiar with mip-mapping here is a Wikipedia link:
http://en.wikipedia.org/wiki/Mipmap
Implementing it with DirectX will be 10x as difficult. Different graphics hardware has different maximum texture sizes. Most likely you will need to break your image up in to multiple textures to draw and you will also have to keep track of render states, viewing matrices, etc.
However, if you do use DirectX, you can implement lots of real-time photo adjustments You can do real-time rotation by simply adjusting view matrices. You can do real-time contrast, brightness, gamma, and sharpness easily in a pixel shader.
There are two other API's I might suggest. If you are willing to limit yourself to Vista or later then Direct2D would be a little simpler than Direct3D. Also if you ever will need to implement it on a non-windows platform I would suggest using OpenGL instead. My current project is in Direct3D because a few years ago when we started it OpenGL was falling behind and I didn't forsee the popularity of Android devices. I now wish we had used OpenGL instead.
Try profiling to see where WPF is spending its time. Are you displaying the images at their native resolution? If not it might be worthwhile to do some preprocessing and create 1/2 resolution versions.
Related
I'm making a shape matching game for toddlers.
I drew the shapes using gimp on my mac.
The images all seemed stretched out on the android devices.
Is there some mathematical way i could figure out how much shorter I have to make the images on the mac computer so they look correctly on the android devices?
This is a tough question. I have made several Apps for iOS and have pretty much the same problem with graphics for buttons and backgrounds when going between iPhone and iPad, which have different aspect ratios (which I suspect is the problem you are having). Throw orientation into the mix as well.
I use Gimp, Inkscape, Spine, Cocos2d-x, and lots of other tools for reference (see here).
As far as I know, the general options for "different screen geometry" issues are (if there are others, let me know):
Different graphics for each geometry.
Scaling the graphics proportionally for the geometry (I suspect you have this).
Same graphics for all, scale proportionally based on one dimension and then place graphics on screen using percentages for fixed stuff.
Option 1 means extra work...not acceptable for a hobbyist or indie (like me) developer.
Option 2 usually looks like junk. Frustration and sadness usually ensue.
Option 3 is the option I use most often. I find that occasionally I have to add small changes (read: hacks) based on the device or geometry. In the case of iOS, it is not too bad, because there are only two geometries to deal with. In the case of Android, where there are lots of different tablet geometries, you could cut it down by working with ranges of geometries and picking a fixed orientation for your application.
At the end of the day, proportional scaling by a single dimension (height or width) has gotten me the most bang for the buck.
I usually make the graphics a little "bigger" than I will need so I only have to scale down. This consumes a bit more memory, but I re-use a lot, use sprite sheets, etc. There is a definite tradeoff with managing every bit of memory (which you have to do in a large AAA rated game) and what you can get away with in a small independent application. I am not encouraging sloppiness or memory mismanagement; I'm encouraging balancing the resources against the developmental needs of simplicity...making the same graphics for different situations burns a lot of daylight.
ONE MORE NOTE:
For applications where I use a framework for the graphics (i.e. games), I use a "viewport" to let me match up the physics with the graphics...that is to say, dynamically scale the size of the graphics based on how much of the total scene I want the user to see. This lets me put the graphic for the "body" on top of it and match up the size statically or dynamically as needed. The code is in C++ and is only loosely tied to the framework...and the concept is applicable to general situations for games (i.e. scaling the graphics as needed). You can find some description and code here, if you are interested.
Was this helpful?
You need to design your application so that it can function correctly regardless of device resolution as many mobile devices have many different aspect ratios and resolutions.
If your images are being stretched then you need to change the ui design so that they are enlarged and spaced out to make use of whatever space there is.
I would like to write a simple ray tracer using WPF. It is a learning project and thus I favour configurability over performance (otherwise I'd go for C++).
I still want relatively fast pixel drawing. A previous question on StackOverflow contains code to achieve this in WPF, by obtaining a GDI bitmap. From the relatively little I know about Windows programming,
GDI is slow
DirectX is fast
WPF uses DirectX underneath (not sure which parts of WPF though)
Is it possible to obtain pixel-level access using DirectX (not GDI) through the WPF Canvas (or similar)?
I will also consider suggestions for incorporating DirectX API calls within a WPF window (alongside other WPF controls) if that is possible.
Thanks in advance.
Interesting, but with raytracing, writing the pixels to the screen will (should) not be the slow part. You can use WriteableBitmap for the purpose, though. It's certainly quick enough for what you want.
http://msdn.microsoft.com/en-us/library/system.windows.media.imaging.writeablebitmap.aspx
(For info, I use it in this emu/IDE - http://0x10c-devkit.com/ - and it can refresh a low res display with great performance. There's the source to that on the github repository, the LEM1802 plugin.)
Ah, this bit: https://github.com/kierenj/0x10c-DevKit/blob/master/PluginAPI/NyaElektriska.LEM1802/GPU.cs - see UpdateDisplay.
Another solution is WriteableBitmapEx. It extends the builtin WriteableBitmap.
There is an open Source Project Called Direct Canvas wich is A hardware accelerated, 2D drawing API that supports vector graphics, multimedia files, extensible pixel shaders, blending modes and more!
http://directcanvas.codeplex.com/
Demo http://www.youtube.com/user/jdollah69#p/u
I'm creating a cad viewer which deals with very large image files and I am trying to optimise it for as high a framerate and low a memory footprint as possible.
It uses GDI+ for rendering onto a panel.
It's current flaw is with image rendering. Some of the files I'm using reference images which are particularly big (8000x8000 pixels). I've optimised the memory usage by only loading them when they become visible and disposing of them when they're not. This reduces the chance of the program running out of memory but prevents the images from being loaded and unloaded too often; however rendering the images themselves (context.DrawImage) still carries a very large overhead.
I'm now exploring ways of blitting the images into a smaller buffer of some sort, rendering this (generally much smaller) buffer, and then refreshing/rebuilding it when the zoom level changes significantly.
The problem is, I can't find any provision for this in GDI whatsoever. Can anyone suggest how I could achieve it?
I don't think GDI is designed for such high-speed updates of images. If you are trying to scroll the image, and tracking the mouse with each move, try to shift sections of the image and fill in the space opened up by the shift. Essentially reuse the tricks that programmers used when smoothly scrolling/panning graphics at a time when CPU's are slow and RAM is small.
If you're creating a new graphics application that needs a high framerate and are looking for suggestions, then I suggest abandoning GDI+ and using WPF. WPF uses hardware acceleration and supports retained-mode graphics; this has much better performance for less work than GDI+.
If there is some limitation that forbids WPF, please explain it in your question. This is relevant because such limitations can also impact GDI+ drawing.
GDI Binned in favour of Direct3D as 3D elements came into the equation anyway. Images turned into single thumbnails and larger tiles that are loaded in/out as required.
I faced a similar problem when developing my own GIS application. The best solution I found for this (even when using WPF) is to tile big images and display only the portions that are visible. This is being said, I would switch to WPF not only for the reasons given in the above answers but also for the good imaging support offered. See this link for more information
my current code is normal WPF with a custom image view. I need to pan and zoom a very high resolution picture but of course it needs a lot of CPU-Power to do this.
My question is: if I change the control from a image view to something like directX will this increase my zoom and panning expirience a lot or isnt there such a big difference?
(The graphic card we use is a Nvidia ion2 and the CPU is a intel atom with up to 2 Ghz)
2D acceleration is not as perfected as 3D is. See benchmarks here.
I believe using the picture as a texture and controlling the camera for pan and zoom should increase performance a lot.
From my knowledge WPF uses DirectX to render its content, so I wouldn't think that would give you a high performance boost.
If you're having performance issues I would think the answer is in looking through your caching algorythm.
WPF graphics isn't very powerful. We have made all graphics rendering in Direct3D9, and only displaying the 3D scene in D3DImage control.
When talking about large bitmap rendering, we've found the best way is to create a Direct3D texture. The creating of it reasonably fast, and rendering itself is very fast when the image dimensions is less than natively supported by the GPU (caps.MaxTextureWidth, caps.MaxTextureHeight). That typically is 8k x 8k or 16k x 16k. Talking about bitmaps of hundreds of MB, and should be sufficient for your use too.
To see the performance that can be obtained with it, you can download our Chart control, set a large bitmap to background image to a Geographic map. Then you see also how fast is it too zoom & pan etc. Beats WPF built-in image handling for sure :-)
(I'm one of LightningChart developers at Arction)
I'm trying to visualise a graph and allow people to play with it. I found the excellent Graph# library that can create an initial layout so that part is covered. Now I need to make a control that actually draws it and provides the necessary interactivity.
Graph# comes with a nice visualiser itself, however I don't like it because it is written in WPF (while my app is WinForms), and because I want to add some more interactivity options, which would require quite a remake of it anyway.
The graphs I'm drawing will routinely be pretty large, at about 100 vertices and the same amount of edges (the graphs will be trees 99% of time). That means that the resulting rendering can be up to 2000px by 2000px and even more. The users should be able to zoom in and out, scroll, highlight and drag vertices and edges, and get some popups with additional info when hovering the cursor above a vertex.
I'm worried that the standard System.Drawing might not be able to deliver a decent speed for this. I would like the dragging/zooming/scrolling operations to be smooth, and the popups should open with a little animation as well. Something like 20fps should be a necessity.
I know I can try to speed things up by pre-rendering a lot of the elements and keeping them as bitmaps in memory - but that would probably take up lots of RAM, and I'm still not sure if it would deliver the necessary performance.
What are your thoughts?
"Premature optimization is the root of all evil"
GDI+ can be great for your needs. Don't go and buy 3rd party libraries before you know you even need them.
I've done a thousand polygons on a 1000x800 pixel bitmap and redrawn it completely at over 100 frames per second, using just GDI+
That being said, if you have a lot of drawing to do, and your resolution is big.. Some of the 3rd party drawing libraries can go WAY beyond what managed GDI is capable of.
I recommend leaving Graph# alone, and just hosting it in your Windows Forms application.
The performance you will receive will be much better than trying to reimplement it in System.Drawing.
GDI+ will be plenty fast enough for what you're doing, especially if (as it sounds from your description) everything you're drawing is rectangles and vertical/horizontal lines. Polygons and non-linear shapes are a bit slower, but not much (the speed difference is partially dependent on the SmoothingMode of your Graphics object). Drawing cached Bitmaps with resizing is also quite fast, although it can slow down significantly if you use a high-quality InterpolationMode setting.
As a benchmark, I wrote a .Net Compact Framework GPS application for Windows Mobile that rendered about 10,000 lines on the screen in realtime. This only achieved a frame rate of a few frames per second, but the processing power on a Smartphone is, of course, way less than a modern PC.