Return Bitmap with C++/CLI Wrapper - c#

I try to write a CLI wrapper for OpenCV, which returns a bitmap from a given OpenCV matrix. I use a function with a fix image inside the wrapper to test it:
Bitmap^ GetImage()
{
Mat frame = imread("Image.png", 0);
return gcnew Bitmap(frame.cols, frame.rows, 4 * frame.rows, System::Drawing::Imaging::PixelFormat::Format24bppRgb, IntPtr(frame.data));
}
My C# code contains the following code to store the image back:
Bitmap Test = Wrapper.GetImage();
Test.Save(#"C:\temp\Bla.bmp");
After executing the code, I´ve got this exception:
How can I fix it? What is the reason for this exception?

This won't work. Because the 'frame' variable goes out of scope when the function returns. Thus, the pointer is dead, and you have a GDI object with a pointer to garbage data.
https://msdn.microsoft.com/en-us/library/zy1a2d14(v=vs.110).aspx?cs-save-lang=1&cs-lang=cpp#code-snippet-1
The caller is responsible for allocating and freeing the block of memory specified by the scan0 parameter. However, the memory should not be released until the related Bitmap is released.
I am not sure what this Mat object is, but you get get the one dimensional row of bytes and do a Marshal::Copy() to the pre-allocated byte array (array^).
I would either return this array created from the Mat object and create the bitmap in C# like this:
https://stackoverflow.com/a/21555447/887584
Or you can of course do this in the C++ code if you want to keep separation of concerns, doing the same thing in C++:
auto stream = gcnew MemoryStream(bytes);
auto bitmap = gcnew Bitmap(stream);
delete stream;
return bitmap;

Related

Do I need to clear Byte[]?

In my game user can choose multiple images from gallery and load them into game. I have created a function that get images from filepath. In that function I declare local variable byte[]. so after read file do i need to dispose that byte[] to freed memory. here is my code :
if (File.Exists(filePath))
{
byte[] fileData = File.ReadAllBytes(filePath);
Texture2D tex = new Texture2D(2, 2);
tex.LoadImage(fileData);
}
Do I need to clear byte[] after texture is loaded from byte[] ???? User can choose any image file from gallery so there is no limit of filesize.
Short answer: No
byte[] is a managed resource. It will be fully cleaned up by the GC.
File related classes often involve unmanaged resources - usually the OS filehandles. But File.ReadAllBytes(string) looks like it follows my advise "create, use, dispose. All in the same piece of code, ideally using a using statement." So I expect not issues form it.
Networking and DB classes involve unamanged resources - usually network connections.
A lot of drawing related classes use unmanaged resources - primarily some unmanaged memory for performance.
But this is just a byte[]. As managed as any int[]. Note that you do not have a choice on the mater anyway as neither byte nor array implement IDisposeable. If you try to dispose if it, the compiler will wonder what you are talking about as there is no such function.

Should I dispose both the Bitmap and its clone?

I'm trying to crop a Bitmap. I found this code which seems to be the fastest solution, since it reuses the bitmap data instead of creating a copy (like the copy constructor does). That's fine since I don't need the original at all, and I'd even modify the original if that was possible. The whole point is to work around a bug in another library:
private static Bitmap GetBitmap(ChromiumWebBrowser browser, DefaultRenderHandler renderHandler)
{
Bitmap oldBitmap = null;
try
{
var bitmap = renderHandler.BitmapBuffer.CreateBitmap();
// Taking care of https://bitbucket.org/chromiumembedded/cef/issues/2864/osr-resize
if (bitmap.Size != browser.Size)
{
oldBitmap = bitmap;
var newWidth = Math.Min(bitmap.Size.Width, browser.Size.Width);
var newHeight = Math.Min(bitmap.Size.Height, browser.Size.Height);
var newX = (bitmap.Size.Width - newWidth) / 2;
var newY = (bitmap.Size.Height - newHeight) / 2;
return bitmap.Clone(new Rectangle(newX, newY, newWidth, newHeight), bitmap.PixelFormat);
}
else
return bitmap;
}
finally
{
oldBitmap?.Dispose();
Monitor.Exit(renderHandler.BitmapLock);
}
}
So here renderHandler.BitmapBuffer.CreateBitmap() creates a new bitmap that must be disposed. Originally, when there was no bug in the library, I just returned bitmap, which the caller disposed. But now I need all that cropping logic because CreateBitmap will sometimes return a bitmap with white edges around that need to be cropped off. The problem is that, I'm not sure if I need to dispose both the original bitmap and the clone. Here I'm disposing the original whenever I clone it (and the caller disposes the clone), but I'm afraid that it may sometimes break the clone. According to this explanation, Clone doesn't actually create a clone but reuses the original data, so perhaps disposing the original will sometimes release the memory used by the clone and throw an error when the caller tries to use it.
On the other hand, if I don't dispose the original, I'm not sure if disposing the clone will also dispose everything that needs to be disposed off in the original. Haven't found anything in the docs, and looking at the source code I see it calls some native methods which I have no idea what they do.

What is the fundamental difference between Matrix<TDepth> and Mat in EmguCV?

Why is Mat not enough in EmguCV?
Why can't Matrix<> load an image from a file itself?
For instance,
Mat img = new Mat(path);
is a valid operation. But,
Matrix<byte> img = new Matrix<byte>(path);
or,
Matrix<byte> img = Matrix<byte>.FromFile(path);
aren't valid operations.
Based on the information from the Emgu Wiki the fundamental difference between the two types is whether the underlying data array is managed or not.
Mat is a wrapper around the C++ cv::Mat class. Generally this class acts as a smart pointer which manages the memory allocated for the data array it owns (although it's able to just observe as well -- a good example of this capability is the ability to return a Mat header for a Matrix instance in C#). This means that OpenCV is able to (re)allocate the memory as necesssary. The trade-off is that in such cases it's more difficult to access the underlying data effectively in C#.
The Matrix class uses a managed array to hold the data. That means you can easily access the underlying data array in C#.
Honestly, the best person to tell you why it's not possible to load Matrix from an image file would be the author. My guess would be that it's intended to represent other things than images. Technically this could be added in the same way as the ability to load an image file was given to the Mat wrapper (the C++ equivalent has no such feature).

Wrong RGB values in GetPixel() from png image

This is the problem:
I save a Bitmap in .png with colors say ARGB(50,210,102,70) with dimension 1 x 1 pixel.
I retrieve the same image again and use the GetPixel(0,0) method, what I get is ARGB(50,209,102,70).
There is a slight variation in the retrieved value, the RGB values slightly differ but the A value remains same.
However when i use 255 for A value, the correct RGB values are returned.
So,.. Using a value less than 255 for A results in the problem mentioned above.
Here is the code which saves the Bitmap.
Bitmap bmpPut = new Bitmap(1, 1); //Also tried with 'PixelFormat.Format32bppArgb'
bmpPut.SetPixel(0, 0, Color.FromArgb(254, 220, 210, 70));
bmpPut.Save("1.png"); //Also tried with using 'ImageFormat.Png'
Here is the code which gets the pixel color
Bitmap bit = new Bitmap(Image.FromFile("1.png"));
MessageBox.Show("R:" + bit.GetPixel(0, 0).R.ToString() +
"| G: " + bit.GetPixel(0, 0).G.ToString() +
"| B: " + bit.GetPixel(0, 0).B.ToString() +
"| A: " + bit.GetPixel(0, 0).A.ToString());
What i get is ARGB(254,219,209,70)
P.S.: I read a few similar questions, they were'nt addressing this exact issue and I din't find a solution yet.
mammago has found a workaround, namely using the class constructor to construct a Bitmap object directly from a file, rather than constructing a Bitmap object indirectly via the Image object returned by Image.FromFile().
The purpose of this answer is to explain why that works, and in particular, what the actual difference is between the two approaches that causes different per-pixel color values to be obtained.
One proposal for the difference was color management. However, this appears to be a non-starter, as neither invocation is asking for color-management (ICM) support.
You can, however, tell a lot by inspecting the source code for the .NET BCL. In a comment, mammago posted links to the code for the Image and Bitmap class implementations, but wasn't able to discern the relevant differences.
Let's start with the Bitmap class constructor that creates a Bitmap object directly from a file, since that's the simplest:
public Bitmap(String filename) {
IntSecurity.DemandReadFileIO(filename);
// GDI+ will read this file multiple times. Get the fully qualified path
// so if our app changes default directory we won't get an error
filename = Path.GetFullPath(filename);
IntPtr bitmap = IntPtr.Zero;
int status = SafeNativeMethods.Gdip.GdipCreateBitmapFromFile(filename, out bitmap);
if (status != SafeNativeMethods.Gdip.Ok)
throw SafeNativeMethods.Gdip.StatusException(status);
status = SafeNativeMethods.Gdip.GdipImageForceValidation(new HandleRef(null, bitmap));
if (status != SafeNativeMethods.Gdip.Ok) {
SafeNativeMethods.Gdip.GdipDisposeImage(new HandleRef(null, bitmap));
throw SafeNativeMethods.Gdip.StatusException(status);
}
SetNativeImage(bitmap);
EnsureSave(this, filename, null);
}
Lots of stuff going on there, but most of it is not relevant. The first bits of code simply obtain and validate the path. After that is the important bit: a call to the native GDI+ function, GdipCreateBitmapFromFile, one of the many Bitmap-related functions provided by the GDI+ flat API. It does exactly what you would think, it creates a Bitmap object from a path to an image file without using color matching (ICM). This is the function that does the heavy lifting. The .NET wrapper then checks for errors and validates the resulting object again. If validation fails, it cleans up and throws an exception. If validation succeeds, it saves the handle in a member variable (the call to SetNativeImage), and then calls a function (EnsureSave) that does nothing unless the image if a GIF. Since this one isn't, we'll ignore that completely.
Okay, so conceptually, this is just a big, expensive wrapper around GdipCreateBitmapFromFile that performs a bunch of redundant validation.
What about Image.FromFile()? Well, the overload you're actually calling is just a stub that forwards to the other overload, passing false to indicate that color matching (ICM) is not desired. The code for the interesting overload is as follows:
public static Image FromFile(String filename,
bool useEmbeddedColorManagement) {
if (!File.Exists(filename)) {
IntSecurity.DemandReadFileIO(filename);
throw new FileNotFoundException(filename);
}
// GDI+ will read this file multiple times. Get the fully qualified path
// so if our app changes default directory we won't get an error
filename = Path.GetFullPath(filename);
IntPtr image = IntPtr.Zero;
int status;
if (useEmbeddedColorManagement) {
status = SafeNativeMethods.Gdip.GdipLoadImageFromFileICM(filename, out image);
}
else {
status = SafeNativeMethods.Gdip.GdipLoadImageFromFile(filename, out image);
}
if (status != SafeNativeMethods.Gdip.Ok)
throw SafeNativeMethods.Gdip.StatusException(status);
status = SafeNativeMethods.Gdip.GdipImageForceValidation(new HandleRef(null, image));
if (status != SafeNativeMethods.Gdip.Ok) {
SafeNativeMethods.Gdip.GdipDisposeImage(new HandleRef(null, image));
throw SafeNativeMethods.Gdip.StatusException(status);
}
Image img = CreateImageObject(image);
EnsureSave(img, filename, null);
return img;
}
This looks very much the same. It validates the file name in a slightly different way, but that isn't failing here, so we can ignore these differences. If embedded color management was not requested, it delegates to another native GDI+ flat API function to do the heavy lifting: GdipLoadImageFromFile.
Others have speculated that the difference may be a result of these two different native functions. It's a good theory, but I disassembled these functions, and though they have distinct implementations, there are no salient differences that would account for the behavior observed here. GdipCreateBitmapFromFile will perform validation, attempt to load a metafile if possible, and then call down to the constructor for an internal GpBitmap class to do the actual loading. GdipLoadImageFromFile is implemented similarly, except that it arrives at the GpBitmap class constructor indirectly via the internal GpImage::LoadImage function. Furthermore, I was unable to reproduce the behavior you described by calling these native functions directly in C++, so that eliminates them as candidates for an explanation.
Interestingly, I was also unable to reproduce the behavior you describe by casting the result of Image.FromFile to a Bitmap, e.g.:
Bitmap bit = (Bitmap)(Image.FromFile("1.png"));
Although not a good idea to rely on it, you can see that this is actually legal if you go back to the source code for Image.FromFile. It calls the internal CreateImageObject function, which delegates either to Bitmap.FromGDIplus to Metafile.FromGDIplus according to the actual type of the image being loaded. The Bitmap.FromGDIplus function just constructs a Bitmap object, calls the SetNativeImage function we have already seen to set its underlying handle, and returns that Bitmap object. Therefore, when you load a bitmap image from a file, Image.FromFile actually returns a Bitmap object. And this Bitmap object behaves identically to one created using the Bitmap class constructor.
The key to reproducing the behavior is to create a new Bitmap object based on the result of Image.FromFile, which is what exactly your original code did:
Bitmap bit = new Bitmap(Image.FromFile("1.png"));
This will call the Bitmap class constructor that takes an Image object, which delegates internally to one that takes explicit dimensions:
public Bitmap(Image original, int width, int height) : this(width, height) {
Graphics g = null;
try {
g = Graphics.FromImage(this);
g.Clear(Color.Transparent);
g.DrawImage(original, 0, 0, width, height);
}
finally {
if (g != null) {
g.Dispose();
}
}
}
And here is where we finally find an explanation for the behavior you describe in the question! You can see that it creates a temporary Graphics object from the specified Image object, fills the Graphics object with a transparent color, and finally draws a copy of the specified Image into that Graphics context. At this point, it is not the same image you're working with, but a copy of that image. This is where color matching can kick in, as well as a variety of other things that potentially affect the image.
In fact, aside from the unexpected behavior described in the question, the code you had written hid a bug: it fails to dispose the temporary Image object created by Image.FromFile!
Mystery solved. Apologies for the long, indirect answer, but hopefully it has taught you something about debugging! Do continue to use the solution recommended by mammago, as it is both simple and correct.
Replacing
Bitmap bit = new Bitmap(Image.FromFile("1.png"));
with
Bitmap bit = new Bitmap("1.png");
Should do the trick.
It seems like Image.FromFile() isn't as precise as the Bitmap constructor.

Image.RotateFlip leaking memory :/

Although I have been programming for about 11 years(mostly VB6, last 6 months C#), it's THE first time to actually ask a question :) I have found all my answers from teh interwebz but this issue i can't solve myself. Your site is among the most helpful places i have got the best answers from!
I will show the code i'm using (an extract of what's relevant). Problem is that when using RotateFlip method then the memory is increasing rapidly to ~200M and then get's collected by GC after some time. The main method calling it iterates about 30 times per second so the performance is of utmost importance here. I have tried using graphics matrix transform but this sometimes fails and shows non-flipped image. The application itself is based on using a webcam, hiding the preview, taking the callback picture and showing it in picturebox. It then overlays a rectangle on if from another class. That's the reason to use callback and not preview window.
Capture.cs class:
internal Bitmap LiveImage;
int ISampleGrabberCB.BufferCB(double bufferSize, IntPtr pBuffer, int bufferLen)
{
LiveImage = new Bitmap(_width, _height, _stride, PixelFormat.Format24bppRgb, pBuffer);
if (ExpImg) // local bool, used rarely when the picture saving is triggered
{
LiveImage.RotateFlip(RotateFlipType.RotateNoneFlipY);
var a = LiveImage.Clone(new Rectangle(Currect.Left, Currect.Top, Currect.Width, Currect.Height),
LiveImage.PixelFormat);
using (a)
a.Save("ocr.bmp", ImageFormat.Bmp);
}
else // dmnit, rotateflip leaks like h*ll but matrix transform doesn't sometimes flip :S
{
LiveImage.RotateFlip(RotateFlipType.RotateNoneFlipY);
/*using (var g = Graphics.FromImage(LiveImage))
{
g.Transform = _mtx;
g.DrawImage(LiveImage, 0, 0);
}*/
}
GC.Collect(); // gotta use it with rotateflip, otherwise it gets crazy big, like ~200M :O
return 0;
}
}
In main form i have an event that's updating the picture in the picturebox:
private void SetPic()
{
pctCamera.Image = _cam.LiveImage;
_cam.PicIsFree = false;
}
Because i need to get the image to main form which is in another class then i figured the most logical is the exposed Bitmap which is updated on every callback frame.
The reason i don't want to use matrix transform is because it's slower and sometimes with this speed it fails to flip the image and the frequency of such behavior is quite different with different PC's with different hardware capabilities and CPU speeds, also the fastest framerate 30fps with a 1.2GHz CPU shows this very frequently.
So, can you help me to figure it out? I'm not actually using it in current version, i'm using the commented-out matrix transform because i feel bad for using GC.Collect :(
Thank You!!!
pctCamera.Image = _cam.LiveImage;
Heavy memory usage like you observe is a sure sign that you missed an opportunity to call Dispose() somewhere, letting the unmanaged resources (memory mostly) used by a bitmap get released early instead of letting the garbage collector do it. The quoted statement is one such case, you are not disposing the old image referenced by the picture box. Fix:
if (pctCamera.Image != null) pctCamera.Image.Dispose();
pctCamera.Image = _cam.LiveImage;
You can rewrite your code like this:
internal Bitmap LiveImage;
int ISampleGrabberCB.BufferCB(double bufferSize, IntPtr pBuffer, int bufferLen)
{
using (LiveImage = new Bitmap(_width, _height, _stride, PixelFormat.Format24bppRgb, pBuffer))
{
LiveImage.RotateFlip(RotateFlipType.RotateNoneFlipY);
if (ExpImg) // local bool, used rarely when the picture saving is triggered
{
var a = LiveImage.Clone(new Rectangle(Currect.Left, Currect.Top, Currect.Width, Currect.Height),
LiveImage.PixelFormat);
using (a)
a.Save("ocr.bmp", ImageFormat.Bmp);
}
}
return 0;
}
Bitmap is an Image class, and implements the IDispose. As you create Bitmap each time, I suggest to use using statement for automatically freeing the resources.
GC.Collect is there for this situation. Collecting the data is the ONLY way to free it and when creating HUGE bitmaps its the way to go. Does a GC.Collect really slow things down?
Other then that you should keep the number of bitmap copies as low as possible.

Categories

Resources