My desktop application receives YUV-images (YUV4:2:0 YV12 to be exact) it need to display. I convert it to RGB and create BitmapSource out of it, which I then display. But this conversion isn't fast enough for my application, despite me using unsafe code with integer-only operations and right shifts instead of divisions (formulae taken here: https://msdn.microsoft.com/en-us/library/aa917087.aspx).
So: is there a way to display YUV-images without prior RGB-conversion? Some control I don't know of? Some hardware-accelerated converter maybe?
Related
This is a follow on to my previous question Speeding Up Image Handling
Apologies if I should have amended that question in some way rather than starting a new one.
I have tried all sorts of different things to speed up the drawing of an image on screen.
I thought that compressing the image until it is smaller would have an effect. However while this might save memory for the object I do not think it has any effect on how long it takes to draw. I tried converting the image to jpeg and using 100% compression. However while this creates a blocky image it does not affect the drawing time. I now think this is because the number of pixels that get rendered is not changed by this
I tried reducing the color palette to 256 colors. This makes the size smaller since it uses less bytes per pixel but does not seem to affect drawing on screen. I had thought that reducing the bytes per pixel GDI+ has to handle might save some time but it is not enough for me to see so far.
So am I wasting my time looking at compression and palette?
I assume that time taken will be affected by the number of pixels to be drawn (width x height) and I have resized the image to match the pixel size as displayed on screen. I think this is the one thing that does have an effect....
I have looked at how to stop autoscaling of the image - does my resize stop that or can the image still be autoscaled when it is rendered?
I am wondering if I can replace the DrawImage call with something using p/Invoke or other API call (which I admit I don't really understand).
You could use the Bitblt P/Invoke, which copies image data directly. It does require some initialisation to convert the image data to something the display device understands, but this copy operation is lightning fast. If you would like to know what exactly is happening in DrawImage btw, use a tool like ILSpy or Reflector.NET to inspect the method.
See BitBlt code not working for an example. For some information about Bitblt see http://www.codeproject.com/KB/GDI-plus/flicker_free.aspx.
I am working on a project where I need to play with two bitmaps. I am putting them in a grid one over the other with reduced opacity (to give a watermark effect).
I am rendering the grid to a bitmap using RenderTargetBitmap and saving the bitmap to a file.
Now my requirement is to load the rendered bitmap again and recover the original pictures separately. Is there any way to recover the original images again. I am not able to think any algorithm to implement this.
My aim is to give a watermarking effect and then recover the images individually.
No. The information is lost during "flattening" of the image.
You need to save both images and information about their properties (position, opacity) into single file. And restore it on load.
If your goal is to simulate watermarking and allow later 'dewatermarking', then assuming that you have your watermarking bitmap present at decoding time, you probably can do that. Sure you cannot use your initial approach - just simple merging of two layers is not reversible.
You need to use some reversible transformation, like rotating source image pixel RGB values vector, using watermark image pixel RGB values as parameters. While dewatermarking you just use negative values from watermark image.
Well, RGB vector is not ideal - you can go out of RGB space while rotating it. Probably you can find color space (or some other transformation in RGB space), better suited to your goal.
(English is not my first or even second language, thereby I apologize if you can't understand my idea - just ask over.)
Why don't you try to make it two layers of bitmap?
i wonder if you can use TIFF format, where you can store multiple images. that way on display you can choose to show with/without watermark.
the problem is as follows:
I have got a rendered bitmap in a byte array (together with the bitmap header). Now I want to manipulate the content of a bitmap. For that purpose I need to wrap this array with some high-level class, say BitmapImage . I want to avoid copying the array, or its data part etc. I want just to force .NET to interpret my byte array as the BitmapImage .
I want to change the content by rendering some stuff to RenderTargetBitmap , and then interpret its data bits as the bits of my image. For rendering I am to use standard DrawingContext, DrawingVisual pair. Thus, it would be great if I could WORK on the data part of my byte array and render my final image to this subarray. Then I did not need to perform any copying and the performance would be great.
How to speedup the calculation and get breathtaking results? :-)
Thank you in advance for the replies!
Cheers
I would use Writeablebitmap , manipulate it in the code and then show it in the Image. Depending on the context some caching could do a great job.
I'm working on a silverlight project where users get to create their own Collages.
The problem
When loading a bunch of images by using the BitmapImage class, Silverlight hogs up huge unreasonable amounts of RAM. 150 pictures where single ones fill up at most 4,5mb takes up about 1,6GB of RAM--thus ending up throwing memory exceptions.
I'm loading them through streams, since the user selects their own photos.
What I'm looking for
A class, method or some process to eliminate the huge amount of RAM being sucked up. Speed is an issue, so I don't want to be converting between images formats or anything like that. A fast resizing solution might work.
I've tried using a WriteableBitmap to render the images into, but I find this method forces me to reinvent the wheel when it comes to drag/drop and other things I want users to be able to do with the images.
What I would try is to take load each stream and resize it to a thumbnail (say, 640x480) before loading the next one. Then let the user work with the smaller images. Once you're ready to generate the PDF, reload the JPEGs from the original streams one at a time, disposing of each bitmap before loading the next.
I'm guessing you're doing something liek this:
Bitmap bitmap = new Bitmap (filename of jpeg);
and then doing:
OnPaint (...)
{
Graphics g = ....;
g.DrawImage (bitmap, ...);
}
This will be resizing the huge JPEG image to the size shown on screen every time you draw it. I'm guessing your JPEG is about 2500x2000 pixels in size. When you load a JPEG into a Bitmap, the bitmap loading code uncompresses the data and stores it either as RGB data in a format that will be easy to render (i.e. in the same pixel format as the display) or as a thing known as a Device Independant Bitmap (aka DIBitmap). These bitmaps require more RAM to store than a compressed JPEG.
Your current implementation is already doing format conversion and resizing, but doing it in an innefficent way, i.e. resizing a huge image down to screen size every time it's rendered.
Ideally, you want to load the image and scale it down. .Net has a system to do this:
Bitmap bitmap = new Bitmap (filename of JPEG);
Bitmap thumbnail = bitmap.GetThumbnailImage (width, height, ....);
bitmap.Dispose (); // this releases all the unmanged resources and makes the bitmap unusable - you may have been missing this step
bitmap = null; // let the GC know the object is no longer needed
where width and height are the size of the required thumbnail. Now, this might produce images that don't look as good as you might want them to (but it will use any embedded thumbnail data if it's present so it'll be faster), in which case, do a bitmap->bitmap resize.
When you create the PDF file, you'll need to reload the JPEG data, but from a user's point of view, that's OK. I'm sure the user won't mind waiting a short while to export the data to a PDF so long as you have some feedback to let the user know it's being worked on. You can also do this in a background thread and let the user work on another collage.
What might be happening to you is a little known fact about the garbage collection that got me as well. If an object is big enough ( I don't remember where the line is exactly ) Garbage Collection will decide that even though nothing currently in scope is linked to the object (in both yours and mine the objects are the images) it keeps the image in memory because it has decided that in case you ever want that image again it is cheaper to keep it around rather than delete it and reload it later.
This isn't a complete solution, but if you're going to be converting between bitmaps and JPEG's (and vice versa), you'll need to look into the FJCore image library. It's reasonably simple to use, and allows you to do things like resize JPEG images or move them to a different quality. If you're using Silverlight for client-side image processing, this library probably won't be sufficient, but it's certainly necessary.
You should also look into how you're presenting the images to the user. If you're doing collages with Silverlight, presumably you won't be able to use virtualizing controls, since the users will be manipulating all 150 images at once. But as other folks have said, you should also make sure you're not presenting bitmaps based on full-sized JPEG files either. A 1MB compressed JPEG is probably going to expand to a 10MB Bitmap, which is likely where a lot of your trouble is coming from. Make sure that you're basing the images you present to the user on much smaller (lower quality and resized) JPEG files.
The solution that finally worked for me was using WriteableBitmapEX to do the following:
Of course I only use thumbnails if the image isn't already small enough to store in memory.
The gotch'a I had was the fact that WriteableBitmap doesn't have a parameterless constructor, but initializing it with 0,0 as size and then loading the source sets these automatically. That didn't come naturally to me.
Thanks for the help everybody!
private WriteableBitmap getThumbnailFromBitmapStream(Stream bitmapStream, PhotoFrame photoFrame)
{
WriteableBitmap inputBitmap = new WriteableBitmap(0,0);
inputBitmap.SetSource(bitmapStream);
Size thumbnailSize = getThumbnailSizeFromWriteableBitmap(inputBitmap, photoFrame.size);
WriteableBitmap thumbnail = new WriteableBitmap(0,0);
thumbnail = inputBitmap.Resize((int)thumbnailSize.Width, (int)thumbnailSize.Height, WriteableBitmapExtensions.Interpolation.NearestNeighbor);
return thumbnail;
}
One additional variant to reduce ram using:
Dont load images, which ar invisible at this moment, and load them while user scrolling the page. This method uses by web developers to improve page load speed. For you its the way not to store hole amount of images in ram.
And I think the better way not to make thumbnails on run, but store them near the fullsize pictures and get only links for them. When it needed, you alway can get the link to fullsize picture and load it.
I've been fussing with this for the better part of the night, so maybe one of you can give me a hand.
I have found GDI+ DrawImage in C# to be far too slow for what I'm trying to render, and from the looks of forums, it's the same for other people as well.
I decided I would try using AlphaBlend or BitBlt from the Win32 API to get better performance. Anyway, I've gotten my images to display just fine except for one small detail—no matter what image format I use, I can't get the white background to disappear from my (transparent) graphics.
I've tried BMP and PNG formats so far, and verified that they get loaded as 32bppargb images in C#.
Here's the call I'm making:
// Draw the tile to the screen.
Win32GraphicsInterop.AlphaBlend(graphicsCanvas, destination.X, destination.Y, this.TileSize, this.TileSize,
this.imageGraphicsPointer, tile.UpperLeftCorner.X, tile.UpperLeftCorner.Y,
this.TileSize, this.TileSize,
new Win32GraphicsInterop.BLENDFUNCTION(Win32GraphicsInterop.AC_SRC_OVER, 0,
Convert.ToByte(opacity * 255),
Win32GraphicsInterop.AC_SRC_ALPHA));
For the record, AC_SRC_OVER is 0x00 and AC_SRC_ALPHA is 0x01 which is consistent with what MSDN says they ought to be.
Do any of you guys have a good solution to this problem or know a better (but still fast) way I can do this?
Graphics.DrawImage() speed is critically dependent on the pixel format. Format32bppPArgb is 10 times faster than any other one on any recent machine I've tried.
Also make sure you the image doesn't get resized, be sure to use a DrawImage() overload that sets the destination size equal to the bitmap size. Very important if the video adapter's DPI setting doesn't match the resolution of the bitmap.
Have you tried an opacity of just 255 rather than a calculated one?
This blog post describes what you're trying to do:-
http://blogs.msdn.com/andreww/archive/2007/10/10/preserving-the-alpha-channel-when-converting-images.aspx
Critical thing is that he carries out a conversion of the image to make the alpha channel compatible..
Ok. From a pure Win32 perspective:
In order for AlphaBlend to actually alpha blend... it needs the source device context to contain a selected HBITMAP representing an image with 32bpp bitmap with a pre-multiplied alpha channel.
To get a device bitmap with 32bpp you can either call one of the many functions that will create a screen compatible device bitmap and hope like hell the user has selected 32bpp as the desktop bitdepth. OR, ensure that the source bitmap is a DIBSection. Well, the library or framework that is creating it from the loaded image for you.
So, C# is loading your images with 32bpp argb, BUT, how are you converting that C# representation of the bitmap into a HBITMAP? You need to ensure that a DIB Section is being created, not a DDB (or device dependent bitmap), and that the DIB Section is 32bpp.