Is there a way to access the underlying memory of a BitmapImage object with c# pointers?
I know that there's a CopyPixels method but it makes a copy of the pixels array into a new array (duplicating memory requirements). If you open a large image (2gb) it allocates a lot of un-useful stuff. And if you want to operate some sort of elaboration, like CCLA, it takes a huge amount of memory.
I need only to read the pixel array.
Is it possible to address pixels directly like you can do in System.Drawing.Bitmap?
I wrote a fast bitmap access for System.Drawing.Bitmap, but as I'm using WPF, I need the same functionality for BitmapSource. Otherwise I have to duplicate the image loading (Bitmap for my old method and BitmapSource to show the image in WPF) taking a lot of memory and time.
Thank you
Lorenzo
A bitmap source does not necessarily have backing memory for the entire image. An example of when it does not would be an image file on disk which is lazy loaded.
The only access you have with WIC, and therefore WPF, is the CopyPixels method. Certain subclasses of BitmapSource will allow access to a buffer, but they are internally just allocating memory and calling CopyPixels themselves.
I would assume that whatever operation does not require access to the entire image at a time. If so, you can call CopyPixels to a smaller buffer, and window your access to the image. Most decoders, when a single pixel is requested, will buffer the entire stride, or in the case of JPEG, then entire block.
I am not sure what CCLA is, and cannot find a definition that seems to fit, but if it is some sort of transform on the source image, you can implement it as a BitmapSource. That way, you can compose a full chain which will
read the image from disk (TiffBitmapEncoder et al.)
scale or translate them (TransformedBitmap)
then window it to only the portion you need (CroppedBitmap)
use a format conversion (FormatConvertedBitmap)
pass it to your algorithm (CclaTransformBitmap)
and finally render it to a WritableBitmap, which gives you access to the buffer
With careful attention to the CacheOption used on the source image, as well as the order of transforms, you should be able to access an arbitrarily large image without significant memory impact.
If you already have a performant algorithm for GDI (System.Drawing), there is no sense re-implementing it. You can still display your final bitmap using Imaging.CreateBitmapSourceFromHBitmap or by using a WindowsFormsHost to host the control you previously built.
Related
In my game user can choose multiple images from gallery and load them into game. I have created a function that get images from filepath. In that function I declare local variable byte[]. so after read file do i need to dispose that byte[] to freed memory. here is my code :
if (File.Exists(filePath))
{
byte[] fileData = File.ReadAllBytes(filePath);
Texture2D tex = new Texture2D(2, 2);
tex.LoadImage(fileData);
}
Do I need to clear byte[] after texture is loaded from byte[] ???? User can choose any image file from gallery so there is no limit of filesize.
Short answer: No
byte[] is a managed resource. It will be fully cleaned up by the GC.
File related classes often involve unmanaged resources - usually the OS filehandles. But File.ReadAllBytes(string) looks like it follows my advise "create, use, dispose. All in the same piece of code, ideally using a using statement." So I expect not issues form it.
Networking and DB classes involve unamanged resources - usually network connections.
A lot of drawing related classes use unmanaged resources - primarily some unmanaged memory for performance.
But this is just a byte[]. As managed as any int[]. Note that you do not have a choice on the mater anyway as neither byte nor array implement IDisposeable. If you try to dispose if it, the compiler will wonder what you are talking about as there is no such function.
Why is Mat not enough in EmguCV?
Why can't Matrix<> load an image from a file itself?
For instance,
Mat img = new Mat(path);
is a valid operation. But,
Matrix<byte> img = new Matrix<byte>(path);
or,
Matrix<byte> img = Matrix<byte>.FromFile(path);
aren't valid operations.
Based on the information from the Emgu Wiki the fundamental difference between the two types is whether the underlying data array is managed or not.
Mat is a wrapper around the C++ cv::Mat class. Generally this class acts as a smart pointer which manages the memory allocated for the data array it owns (although it's able to just observe as well -- a good example of this capability is the ability to return a Mat header for a Matrix instance in C#). This means that OpenCV is able to (re)allocate the memory as necesssary. The trade-off is that in such cases it's more difficult to access the underlying data effectively in C#.
The Matrix class uses a managed array to hold the data. That means you can easily access the underlying data array in C#.
Honestly, the best person to tell you why it's not possible to load Matrix from an image file would be the author. My guess would be that it's intended to represent other things than images. Technically this could be added in the same way as the ability to load an image file was given to the Mat wrapper (the C++ equivalent has no such feature).
Background:
I'm building an application which will open a potentially large number of photos, generate a thumbnail to present to the user, then allow things like exif data viewing/clearing and minor post processing.
I want to allow the user to scroll through the images without pausing to load each one as it becomes visible, but I also don't want to keep dozens or hundreds of full size bitmap images in memory.
I had built a prototype of this task using System.Drawing using Image objects and their GenerateThumbnailImage method, but decided to move to WPF and use System.Windows.Media.ImageSource derived objects and the TransformedBitmap with a ScaledTransform to generate the thumbnail.
What I found, though, is that when I create a TransformedBitmap, it has a reference back to the source image, which is available and still present in memory. How do I release this source object?
Some relevant C# code:
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.IO;
using System;
...
private void LoadImage(){
//Called by my class internally to handle generating the thumbnail
//Intent is to keep only metadata and a thumbnail bitmap in memory
Stream handle = File.OpenRead(FileName);
BitmapDecoder source = BitmapDecoder.Create(handle,BitmapCreateOptions.None,BitmapCacheOption.OnLoad);
handle.Dispose();
//Determine a scaling ratio to force the larger of height or width to fit inside my desired thumbnail size (int)MaxDim.
ScaleRatio = Math.Min(MaxDim/Math.Max(source.Frames[0].PixelHeight,source.Frames[0].PixelWidth),1); //a public member of the class, Double
_ImageSource = new TransformedBitmap(source.Frames[0],new ScaleTransform(ScaleRatio,ScaleRatio)); //private member of the class, ImageSource
_ImageSource.Freeze();
_Exif = source.Frames[0].Metadata; //private member of the class, ImageMetadata
}
The problem here is that while I hoped that the (BitmapDecoder)source would be released, I can still access the object via _ImageSource.Source.
I have considered using CopyPixels or encoding the TransformedBitmap back into a byte[] stream to create a new, hopefully unattached bitmap, but both of those methods seem like unnecessary reprocessing if I can just abandon or dispose of the source or if there is some simple and fast way to create a shallow clone that I haven't discovered. My attempt at a shallow clone using BitmapFrame.Create(TransformedBitmap) doesn't free the memory either, but also doesn't leave me with an obvious reference.
Some testing watching memory consumption shows each image loaded costs about 30MB. An approximately 200x200#32bpp image should be about 160kB, not counting overhead.
The question again as a TL;DR: how do I release the reference to the source bitmap after a TransformedBitmap uses it?
I am an old delphi programmer, I am used to creating objects and using them entire time for efficient memory usage. But in c# (maybe all the tutorials I've ever seen), you are creating stuffs with new every time (thanks to garbage collector!!, let me do the coding)..
Anyway, I am trying to create a designing software which has lots of drawing.
My question is: do I have to create a graphics object, or use the protected override void OnPaint(PaintEventArgs e) e.Graphics every painting event.. because when I create a graphic object and then resize the control that I draw on, the graphic object that I created, has that clipping problem and only draws old rectangle region..
thanks
Caching objects makes sense when the object is expensive to create, cheap to store and relatively simple to keep updated. A Graphics object is unique in that none of these conditions are true:
It is very cheap to create, takes well less than a microsecond.
It is very expensive to store, the underlying device context is stored in the desktop heap of a session. The number of objects that can be stored is small, no more than 65535. All programs that run in the session share that heap.
It is very hard to keep updated, things happen behind your back that invalidates the device context. Like the user or your program changing the window size, invalidating the Graphics.ClipBounds property. You are wasting the opportunity to use the correct Graphics object, the one passed to you in a Paint event handler. Particularly a bug factory when you use double-buffering.
Caching a Graphics object is a bug.
If you want to draw on the surface always use the Graphics object from the Paint event!
If you want to draw into a Bitmap you create a Graphics object and use it as long as you want.
For the Paint event to work you need to collect all drawing in a List of graphic actions; so you will want to make a nice class to store all parameters needed.
In your case you may want to consider a mixed approach: Old graphic actions draw into a bitmap, which is the e.g. BackgroundImage or Image of your control
Current/ongoing drawing are done on the surface. This amounts to using the bitmap as a cache, so you don't have to redraw lots of actions on every little change etc
This is closely related to your undo/redo implementation. You could set a limit and draw those before into a Btimap and those after onto the surface..
PS: You also should rethink your GC attitude. It is simple, efficient and a blessing to have around. (And, yes, I have done my share of TP&Delphi, way back when they were affordable..) - Yes, we do the coding, but GC is not about coding but about house keeping. Boring at best.. (And you can always design to avoid it, but not with a Graphics object in a windows system.)
A general rule for every class that implements IDisposable is to Dispose() it, as soon as possible. Make sure you know about the using(...){} statement.
For drawing in WinForms (GDI+) the best practice is indeed to use the Graphics object from PaintEventArgs. And because you didn't create that one, do not Dispose() it. Don't stash it either.
I have to completely disagree with other more experienced members here who say it's no big deal or in fact better to recreate the Graphics object over and over.
The HDC is a pointer to a HDC__ struct, which is a struct with one member, "int unused". It's an absolute waste and stupidity to create another instance/object every time drawing needs to be done. The HDC is NOT large, it's either 4 or 8 bytes, and the struct it points to is in nearly all cases 4 bytes. Furthermore, on the point that one person made, it doesn't help that the graphics object be made with the "static" keyword at the beginning of the WndProc() before the switch, because the only way to give the Graphics object a device context or handle to paint on is by calling its constructor, so "static" does nothing to save you from creating it over and over again.
On top of that Microsoft recommends that you create a HDC pointer and assign it to the same value PAINTSTRUCT already has, every, single WM_PAINT message it sends.
I'm sorry but the WinAPI in my opinion is very bad. Just as an example, I spent all day researching how to make a child WS_EX_LAYERED window, to find out that in order to enable Win 8 features one has to add code in XML with the OS's ID number to the manifest. Just ridiculous.
I have some code that does
MemoryStream ms = new MemoryStream();
...
return Image.FromStream(ms);
It fails in very eclectic ways since the Image object does not hold a ref to the stream, so it can get disposed if the GC kicks in which results in GDI+ errors.
How do I work around this (without saving the stream to disk, or altering my method sigs) ?
This seems highly unlikely to me - it would cause a problem for almost any use of Image.FromStream.
It seems more likely to me that something's disposing of your MemoryStream, which it shouldn't.
Could you provide a short but complete program which demonstrates the problem? Forcing garbage collection should make it relatively easy to reproduce - you could even create your own class deriving from MemoryStream with a finalizer to show whether or not it really is being collected (well, finalized at least).
There isn't a way to do it without changing your code somewhat. The Remarks section for the documentation for the static FromStream method on the Image class states:
You must keep the stream open for the
lifetime of the Image.
That being said, you have to make sure that while the Image is accessing the Stream, the stream is open. It would also appear (looking through Reflector) that the FromImage method doesn't actually cause the Image instance to hold onto a reference to the Stream the image was loaded from.
That being said, you to somehow link the image and the MemoryStream (or Stream) together so that it doesn't get GCed. If don't really retain "ownership" of the image (it is passed around), then I recommend that you create a data structure which will hold the reference to the Image and to the Stream and pass the two around in tandem.