I have a problem in a Windows Forms application with Bitmap.Save failing when I save to a MemoryStream. The problem only seems to occur intermittently on one machine (so far) and the bad news it is at a customer site. I can't debug on the machine, but I got a stack trace that narrowed the problem down to a single line of code.
Here's a condensed version of my code:
byte[] ConvertPixelsToBytes()
{
// imagine a picture class that creates a 24 bbp image, and has
// a method to get an unmanaged pixel buffer.
// imagine it also has methods for getting the width,
// height, pitch
// I suppose this line could return a bad address, but
// I would have expected that the Bitmap constructor would have
// failed if it was
System.IntPtr pPixels = picture.GetPixelData();
System.Drawing.Bitmap bmp = new System.Drawing.Bitmap(
picture.width(),
picture.height(),
picture.pitch(),
System.Drawing.Imaging.PixelFormat.Format24bppRgb,
pPixels );
// This line doesn't actually free the memory, but it could be freed in a
// background thread
// (2)
picture.releasePixelData(pPixels);
System.IO.MemoryStream memStream = new System.IO.MemoryStream();
try
{
// I don't see how this line could fail, but it does
// (3)
bmp.Save(memStream, System.Drawing.Imaging.ImageFormat.Bmp);
return memStream.ToArray();
}
catch(System.Runtime.InteropServices.ExternalException e)
{
// e.Message is the very helpful " A generic error occurred in GDI+."
}
finally
{
memStream.Dispose();
}
return new byte[0];
}
Any idea what might be going on? I'm pretty sure my pixel buffer is right, it always works on our dev/test machines and at other customer sites.
My thoughts on possible reasons for failure are
a. The bitmap constructor doesn't copy the pixel data, but keeps a reference to it, and the Save fails because the memory is released. I don't find the MSDN docs clear on this point, but I assume that the Bitmap copies the pixel data rather than assume it is locked.
b. The pixel data is invalid, and causes the Save method to fail. I doubt this since my pixel data is 24 Bits per pixel, so as far as I know it should not be invalid.
c. There's a problem with the .NET framework.
I would appreciate any thoughts on other possible failure reasons so I can add extra checks and logging information to my app so I can send something out into the field.
The MSDN docs for that Bitmap constructor leave no doubt whatsoever:
Remarks
The caller is responsible for allocating and freeing the block of memory specified by the scan0 parameter, however, the memory should not be released until the related Bitmap is released.
Have you tried moving
picture.releasePixelData(pPixels);
to
finally
{
memStream.Dispose();
picture.releasePixelData(pPixels);
}
it definitely sounds like a threading issue (especially since you state that the releasePixelData could happen on a background thread). Threading issues are always the ones that only happen on one machine, and it is always on the clients machine (probably due to the fact they only have 256Megs of memory or something ridiculous and garbage collector is kicking in early, or the machine has quad core and your developer machine is dual core or something).
Related
I'm using PrivateFontCollection to install uploaded fonts on my web server. The code below works, but on second upload of a font the PrivateFontCollection references the first font uploaded. So something isn't being disposed of correctly. Anyone spot what I'm doing wrong?
var fontName = string.Empty;
using (var ms = new MemoryStream(fontBytes))
{
// used to store our font and make it available in our app
using (var pfc = new PrivateFontCollection())
{
//create memory pointer
IntPtr data = Marshal.AllocCoTaskMem((int)ms.Length);
try
{
//copy the bytes to the unsafe memory block
Marshal.Copy(fontBytes, 0, data, (int)ms.Length);
// We HAVE to do this to register the font to the system (Weird .NET bug !)
uint cFonts = 0;
AddFontMemResourceEx(data, (uint)fontBytes.Length, IntPtr.Zero, ref cFonts);
//pass the font to the font collection
pfc.AddMemoryFont(data, (int)ms.Length);
var fontWithMime = "data:application/x-font-truetype;charset=utf-8;base64," + cleanFontData;
fontName = pfc.Families[0].Name;
//db work here
}
finally
{
ms.Close();
Marshal.FreeCoTaskMem(data);
}
}
}
PrivateFontCollection is a very flawed class and you have to be extremely careful using it. A very gross bug in your existing code is the Marshal.FreeCoTaskMem() call. It is up to you to ensure that you do not call this function until after your code stops using any Font object you created from the family. Failure to do so causes random glyph corruption, you only get an AccessViolationException if you are lucky. Underlying problem is that the font will continue to use the memory you allocated with AllocCoTaskMem(), it is completely unaware that the memory is no longer valid. Corruption occurs when the memory is re-used.
Furthermore, while the class has an AddMemoryFont() method, it does not have a corresponding RemoveMemoryFont() method. The only way to clean up is by calling PrivateFontCollection.Dispose(). This deletes all of the fonts in the collection. With the same stipulation as in the previous paragraph, you can only call Dispose() when you are sure that you no longer use any Font object. Calling it too early does not cause an exception.
Very awkward behavior, only truly safe way to use the PFC is to keep it around for the life of the app. Pretty painful in a web app of course.
You can assume that the added font is the last one in the FontFamily[] array. Not the first one as you have it implemented now.
According to AddFontMemResourceEx function:
To remove the fonts that were installed, call RemoveFontMemResourceEx.
However, when the process goes away, the system will unload the fonts
even if the process did not call RemoveFontMemResource.
However, I don't see you doing that. That might be the reason.
I have a problem where loading or cloning an png file to an .Net (3.5) Bitmap class causes an OutOfMemoryException within the GDI framework.
This code on run inside an ThreadPool.QueueUserWorkItem, at any time the code creates up to 20 calls to the image loading method that has this code.
// The GDI may throw an OutOfMemoryException on either load or clone of an bitmap
using (var bmp = new Bitmap(filename))
{
var clone = bmp.Clone(new Rectangle(new Point(0, 0), bmp.Size), PixelFormat.Format32bppPArgb);
//post clone bitmap to main thread
}
The solution is to put and try catch around the using statement
try {
using (var bmp = new Bitmap(filename))
{ ... }
}
catch (OutOfMemoryException) { // Swallow the exception}
I have tested and shown this defect on Win7 32&64bit machines and Win8 in Normal and safe modes. This defect only shows on 5 out of 8 machines where the tests where carried out.
Machines are high end Dell and Asus workstations and laptops with different video cards.
Due to the exception coming from the GDI software stack, I don't believe this defect is related to hardware.
The exception may be thrown less than 0.1% of the time, so rare and random. The files are in good order and are not the cause of the problem.
I am looking for an explanation as to why the GDI throws this exception, so I can explain it to my software team and our customer.
Solved: I assumed, incorrectly, that Visual Studio 2012 defaulted to building 64-bit apps when making new projects. Under Solution Properties, Build tab, there is a checkbox marked, "Prefer 32-bit". When I unchecked this checkbox and rebuilt my solution, I was able to load and process over 200 bitmaps at a time. Thanks to Mike Trusov for so politely pointing out the solution to me.
Original question: I've run into a small problem. I have more than 150 grayscale bitmaps that I want to load into RAM on an 8 GB system, but I can't seem to get past 50 or so without throwing an exception. I've tried loading the bitmaps into an array, which is a lost cause, of course, because .NET has a 2GB limit. But it failed long before 2GB worth of bitmaps were loaded. So I tried loading them into a List<List<Bitmap>>, but even that fails at about the same place. At the moment of the exception, Task Manager says I have 3939 MB of Available RAM waiting to be filled. Very strange.
I don't need these bitmaps to be contiguous in RAM. They can be a scattered bunch of 16 MB allocations for all I care. I just want to be able to fill available RAM with a bunch of bitmaps. That's all.
The exception has at various times been an OutOfMemory exception or an ArgumentException, depending on how much available RAM I had when I started the program. In either case, the stack trace dies inside System.Drawing.Bitmap..ctor(String filename). There is nothing wrong with the specific file being loaded at the time of the exception. In fact, when I have it load a different (or even overlapping) set of bitmaps, the error occurs at the same iteration.
Does anyone have a clue they can lend me on how to do this? Am I running into the .NET 2GB limit in some strange way?
To respond to a few questions and comments: I'm using Visual Studio 2012, .NET 4.5, 64-bit Windows 7, on a computer with 8 GB of RAM. Yes, I need all those bitmaps in RAM at the same time for a variety of reasons (performance, image processing reasons, etc). I have pondered using gcAllowVeryLargeObjects, but I don't need or want all my bitmaps in a long chunk of contiguous memory. I would much rather each Bitmap used its own separate memory allocation. Besides, if I had a machine with 64 GB of RAM, it would be absurd to be limited to even 150 Bitmaps of that size. Why won't these bitmaps load without throwing an OutOfMemoryException?
To me, it seems that .NET is trying to keep all Bitmaps in a single 2 GB region. If there was a way to get each Bitmap to (saying more than I know here) have its own separate address space, that might solve the problem. To invoke the language of the long ago days of MS-DOS, I want to allocate and access far memory using a long pointer, not have all my data stuck in a single near segment.
Here is the array code:
List<String> imageFiles; // List of .bmp filenames.
Bitmap[] bmps = new Bitmap[100]; // Stores/accesses the Bitmaps.
private void goButton_Click(object sender, EventArgs e)
{
int i;
// Load the bitmaps
if (bmps[0] == null)
{
// Load the list of bitmap files.
imageFiles = Directory.EnumerateFiles(#"C:\Temp", "*.bmp", SearchOption.TopDirectoryOnly).ToList();
// Read bitmap files
for (i = 0; i < bmps.Length; ++i)
{
bmps[i] = new Bitmap(imageFiles[i]); // <-- Exception occurs here when i == 52 or so.
}
}
}
Here is the List> code:
List<String> imageFiles; // List of .bmp filenames.
List<List<Bitmap>> bmps = new List<List<Bitmap>>(100); // Stores/accesses the Bitmaps.
private void goButton_Click(object sender, EventArgs e)
{
int i;
// Load the bitmaps
if (bmps.Count == 0)
{
// Load the list of bitmap files.
imageFiles = Directory.EnumerateFiles(#"C:\Temp", "*.bmp", SearchOption.TopDirectoryOnly).ToList();
// Read bitmap files
for (i = 0; i < 100; ++i)
{
// Load the bitmap into temporary Bitmap b.
Bitmap b = new Bitmap(imageFiles[i]); // <-- Exception occurs here when i == 52 or so.
// Create and add a List<Bitmap> that will receive the clone of Bitmap b.
bmps.Add(new List<Bitmap>(1));
// Clone Bitmap b and add that cloned Bitmap to the Bitmap of List<Bitmap>.
bmps[i].Add((Bitmap)b.Clone());
// Dispose Bitmap b.
b.Dispose();
}
}
}
There should be no issues loading more than 2gb of bitmaps into memory in a 64bit app (check project settings - might have to create a new configuration for Any CPU based on x86) running on 64bit OS (I'm guessing you are). Also a simple list should work:
var imageFiles = Directory.EnumerateFiles(#"C:\Temp", "*.bmp", SearchOption.TopDirectoryOnly).ToList();
var lst = new List<Bitmap>();
foreach (var imageFile in imageFiles)
{
lst.Add(new Bitmap(imageFile));
}
Do they ALL have to be loaded at the same time? Could you load say, 20 of them, then while you are processing or displaying those, you have a background thread prep the next 20.
I am getting an Out Of Memory Exception when I attempt to new a Bitmap and return it through a "get" accessor. The Bitmap is of size 640x480 with depth Int32.
My suspicion is that the C# Garbage Collector is unable to delete these old Bitmaps because they are being returned in the accessor to my variable. I have well over 2GB available so I wouldn't imagine that this "small" image is taking up too much memory. Unfortunately I HAVE to new the Bitmap due to thread locking issues (Trouble with locking an image between threads) The code is as follows:
public Bitmap LiveFrame { get { return GetFrame(500); } }
.....
private Bitmap GetFrame(int timeout)
{
Bitmap ret = null;
//CLEyeCameraGetFrame places image data into this._PrivateBitmap
bool success = CLEyeCameraGetFrame(_Camera, _PtrBmpPixels, timeout);
if(success)
ret = new Bitmap(this._PrivateBitmap);
return ret;
}
Note on unmanaged code:
CLEyeCameraGetFrame is in an unmanaged DLL. I allocate _PtrBmpPixels using Marshal.AllocHGlobal earlier in the code and don't touch it until freeing when closing the app. _PtrBmpPixels was used in the creation of the Bitmap _PrivateBitmap through its constructor that accepts the IntPtr argument "scan0". Thus, whenever _PtrBmpPixels is updated via CLEyeCameraGetFrame, _PrivateBitmap is also updated.
I attempted to fix this by Disposing the PcitureBox Bitmap before reusing it, but that broke the PictureBox display. I have two threads updating two different PictureBox/ImageBoxes:
lock (_CameraLocker)
{
if (_VideoPlaying)
{
try{
if (pbLiveFeed.Image != null)
pbLiveFeed.Image.Dispose();
pbLiveFeed.Image = _Camera.LiveFrame;
pbLiveFeed.Invalidate();
}catch (Exception ex) { }
}
....
lock (_CameraLocker)
{
try{
if (ibProcessed.Image != null)
ibProcessed.Image.Dispose();
procImage = new Image<Bgra, Int32>(_Camera.LiveFrame);
procImage.Draw(new Rectangle(10, 20, 20, 15), new Bgra(1.0, 1.0, 1.0, 1.0), 5);
ibProcessed.Image = procImage;
}catch (Exception ex) { }
}
Could Garbage collection be the cause of this? Is it not safe to return a newed object from a get accessor?
The only real clue in your question is in what is missing. You never said "and I make sure to dispose the bitmap after using it". Which is something you can overlook easily in .NET programming but will come to byte you when you use bitmaps.
The Bitmap class is a very small wrapper class around an unmanaged resource created by GDI+. The actual bitmap pixels are stored in unmanaged memory. Which is why the class has a Dispose() method, that releases the unmanaged memory allocation. The garbage collector can do little to help you get this memory released automatically, a Bitmap object is too small to induce a garbage collection often enough to keep up with the rate at which your program consumes unmanaged memory. An oom-kaboom is the outcome.
You'll need to look at the rest of your code and see how the LiveFrame property is getting used. And make sure that the returned bitmap is getting disposed. If, say, you assign it to a PictureBox.Image property then you have to dispose the old image before assigning it.
I have a 3rd party dll which generates a Bitmap and send back its reference. If I generate a System.Windows.Media.Imaging.BitmapSource out of it immediately then all goes well. But if I save the reference and later on (after a few seconds and many function calls) I try to generate the Bitmapsource, I get
System.AccessViolationException was
unhandled by user code
Message=Attempted to read or write
protected memory. This is often an
indication that other memory is
corrupt. Source="System.Drawing"
when doing :
System.Windows.Media.Imaging.BitmapSource bitmapSource = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
bmp.GetHbitmap(),
IntPtr.Zero,
Int32Rect.Empty,
System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions());
Any clues on whats going wrong here ? Any pointers will be useful. Thanks.
I think this indicates that the handle (reference to a resource managed by the operating system, rather than .Net) returned by bmp.GetHBitmap is no longer valid - possibly a Dispose has been called somewhere or something like that (not necessarily by your code though).
I'd recommend using another way of persisting the bitmap data that does not rely on handles - possibly stream out the binary data of the bitmap itself immediately, and then throw a reference to that around.
I had a big problem with Bitmaps and access violations as well. What I believe to be happening is that certain bitmap constructors leave file handles open when they should not. Thus, the program you are running detects that the files are in use, when they shouldn't be.
I eventually figured out a solution in that I make a copy of the original bitmap and then dispose the original. Here is my code, which preserves the resolution of the original Bitmap:
Bitmap Temp = new Bitmap(inFullPathName);
Bitmap Copy = new Bitmap(Temp.Width, Temp.Height);
Copy.SetResolution(Temp.HorizontalResolution, Temp.VerticalResolution);
using (Graphics g = Graphics.FromImage(Copy))
{
g.DrawImageUnscaled(Temp, 0, 0);
}
Temp.Dispose();
return Copy;
Obviously, for the first line, yours would be Bitmap Temp = MyThirdPartyDLL.GetBitmap(); or something. If you don't care about the resolution it can be simplified to:
Bitmap Temp = MyThirdPartyDLL.GetBitmap();
Bitmap Copy = new Bitmap(Temp, Temp.Width, Temp.Height);
Temp.Dispose();
return Copy;
After making this change, I was able to do all kinds of File I/O, etc, perfectly fine, hope you can do the same.