C# Memory Leak on on different machines - c#

Background info
I've developed a desktop application with Windows Form (C#) for scanning, previewing and saving images.
The app behaviour while scanning is the following:
Scan n images
Get a Bitmap for each image and store it in a temporary file
Show resized thumbnails as a preview
Image memory management: Compressible images
For managing the memory usage, I've created a CompressibleImage class which encapsulates a Bitmap file and reads/writes images file on a FileStream. When an image is not longer required by the application, it is written into a file stream. When the app requires an image (ie. user double-click on a thumbnail), a Bitmap file is created from the stream. Here are the main CompressibleImage's methods:
/// Gets the uncompressed image. If the image is compressed, it will be uncompressed
public Image GetDecompressedImage()
{
if (decompressedImage == null)
{
// Read Bitmap from file stream
stream.Seek(0, SeekOrigin.Begin);
decompressedImage = new Bitmap(stream);
}
return decompressedImage;
}
/// Clears the uncompressed image, leaving the compressed one in memory.
public void ClearDecompressedImage()
{
// If Bitmap file exists, write it to file and dispose it
if (decompressedImage != null)
{
if (stream == null)
{
stream = new FileStream(FileStreamPath, FileMode.Create);
}
decompressedImage.Save(stream, format);
// The Dispose() call does not solve the issue
// decompressedImage.Dispose();
decompressedImage = null;
}
}
/// <summary>
/// Class destructor. It disposes the decompressed image (if this exists),
/// closes the stream and delete the temporary file associated.
/// </summary>
~CompressibleImage()
{
if (decompressedImage != null)
{
decompressedImage.Dispose();
}
if(stream != null)
{
stream.Close();
File.Delete(stream.Name);
stream.Dispose();
}
}
Application level
The application uses CompressibleImage to create image files mainly during the scan method and the saving process.
The scan method works fine and it basically:
Acquire Bitmap from scanner
Create a CompressibleImage from the scanned Bitmap
Write the Bitmap to the file stream
The save method works fine on my machine and its behaviour is as follows:
1. For each CompressibleImage decompress (read & build) Bitmap from stream
2. Save image
3. Compress image
Here is the save method:
private void saveImage_button_Click(object sender, EventArgs e)
{
if (Directory.Exists(OutputPath) == false && File.Exists(OutputPath) == false)
{
Directory.CreateDirectory(OutputPath);
}
ListView.CheckedListViewItemCollection checkedItems = listView1.CheckedItems;
if(checkedItems.Count > 0)
{
for (int i = 0; i < checkedItems.Count; ++i)
{
int index = checkedItems[i].Index;
Bitmap image = (Bitmap)compressibleImageList.ElementAt(index).GetDecompressedImage();
try
{
image.Save(OutputPath + index.ToString() +
Module.PNG_FORMAT, ImageFormat.Png);
compressibleImageList.ElementAt(index).ClearDecompressedImage();
progressForm.Increment();
image = null;
}
catch (Exception ex) {
...
}
}
}
}
Issue Description
In my machine the application works fine. There is no memory leak and the scan and save methods do their job smoothly and with a reasonable memory usage (scanning 100 papersheets with less than < 140MB pick).
The problem is, when I tried to test the application on other machines, the Garbage collector doesn't free up the memory, causing a MemoryException during the execution of both methods and when the amount of image is reasonably high (>40). The exception is thrown inside the CompressibleImage.GetDecompressedImage() method when I attempt to decompress (read) an image:
decompressedImage = new Bitmap(stream);
Although I known that the GC clean the memory randomly, in this case it seems like it doesn't even run and in fact the memory is released only when I close the application.
It is possible such different behaviour on similar machines?
System's info
Here are some info about the testing environments. Both machines have:
Processor: Intel i7 2.30GHz
RAM: 8GB
Type: 64-bit
OS: Windows 7 Pro SP 1

Not quite sure about you MemoryException, please provide full stacktrace.
However, I can see you are doing an obvious mistake in your destructor.
You SHOULD NOT refer your managed resources in the Destructor. The reason is being that, GC and Finalizer are using heuristic algorithms to trigger them and you can never predict the execution order of finalizers for your managed objects.
That's why you should use 'disposing' flag in your dispose method and avoid touching managed objects when the execution is coming from the finalizer.
Below example show the general best practice to implement IDisposable interface. Reference : https://msdn.microsoft.com/en-us/library/system.idisposable.dispose(v=vs.110).aspx
public class DisposeExample
{
// A base class that implements IDisposable.
// By implementing IDisposable, you are announcing that
// instances of this type allocate scarce resources.
public class MyResource: IDisposable
{
// Pointer to an external unmanaged resource.
private IntPtr handle;
// Other managed resource this class uses.
private Component component = new Component();
// Track whether Dispose has been called.
private bool disposed = false;
// The class constructor.
public MyResource(IntPtr handle)
{
this.handle = handle;
}
// Implement IDisposable.
// Do not make this method virtual.
// A derived class should not be able to override this method.
public void Dispose()
{
Dispose(true);
// This object will be cleaned up by the Dispose method.
// Therefore, you should call GC.SupressFinalize to
// take this object off the finalization queue
// and prevent finalization code for this object
// from executing a second time.
GC.SuppressFinalize(this);
}
// Dispose(bool disposing) executes in two distinct scenarios.
// If disposing equals true, the method has been called directly
// or indirectly by a user's code. Managed and unmanaged resources
// can be disposed.
// If disposing equals false, the method has been called by the
// runtime from inside the finalizer and you should not reference
// other objects. Only unmanaged resources can be disposed.
protected virtual void Dispose(bool disposing)
{
// Check to see if Dispose has already been called.
if(!this.disposed)
{
// If disposing equals true, dispose all managed
// and unmanaged resources.
if(disposing)
{
// Dispose managed resources.
component.Dispose();
}
// Call the appropriate methods to clean up
// unmanaged resources here.
// If disposing is false,
// only the following code is executed.
CloseHandle(handle);
handle = IntPtr.Zero;
// Note disposing has been done.
disposed = true;
}
}
// Use interop to call the method necessary
// to clean up the unmanaged resource.
[System.Runtime.InteropServices.DllImport("Kernel32")]
private extern static Boolean CloseHandle(IntPtr handle);
// Use C# destructor syntax for finalization code.
// This destructor will run only if the Dispose method
// does not get called.
// It gives your base class the opportunity to finalize.
// Do not provide destructors in types derived from this class.
~MyResource()
{
// Do not re-create Dispose clean-up code here.
// Calling Dispose(false) is optimal in terms of
// readability and maintainability.
Dispose(false);
}
}
public static void Main()
{
// Insert code here to create
// and use the MyResource object.
}
}

When opening files or streams using a class that incorporates the IDisposable interface one should usually make use of using. This will ensure that after the using statement the Dispose method is called. If implemented correctly, this will ensure that the unmanaged resources are released.
MSDN Article on 'using' statement

Related

How can I structure a try-catch-finally block to handle errors inside finally?

I've got a problem with making calls to a third-party C++ dll which I've wrapped in a class using DllImport to access its functions.
The dll demands that before use a session is opened, which returns an integer handle used to refer to that session when performing operations. When finished, one must close the session using the same handle. So I did something like this:
public void DoWork(string input)
{
int apiHandle = DllWrapper.StartSession();
try
{
// do work using the apiHandle
}
catch(ApplicationException ex)
{
// log the error
}
finally
{
DllWrapper.CloseSession(apiHandle);
}
}
The problem I have is that CloseSession() sometimes causes the Dll in question to throw an error when running threaded:
System.AggregateException: One or more errors occurred. --->
System.AccessViolationException: Attempted to read or write protected
memory. This is often an indication that other memory is corrupt.
I'm not sure there's much I can do about stopping this error, since it seems to be arising from using the Dll in a threaded manner - it is supposed to be thread safe. But since my CloseSession() function does nothing except call that Dll's close function, there's not much wiggle room for me to "fix" anything.
The end result, however, is that the session doesn't close properly. So when the process tries again, which it's supposed to do, it encounters an open session and just keeps throwing new errors. That session absolutely has to be closed.
I'm at a loss as to how to design an error handling statement that's more robust any will ensure the session always closes?
I would change the wrapper to include disposal of the external resource and to also wrap the handle. I.e. instead of representing a session by a handle, you would represent it by a wrapper object.
Additionally, wrapping the calls to the DLL in lock-statements (as #Serge suggests), could prevent the multithreading issues completely. Note that the lock object is static, so that all DllWrappers are using the same lock object.
public class DllWrapper : IDisposable
{
private static object _lockObject = new object();
private int _apiHandle;
private bool _isOpen;
public void StartSession()
{
lock (_lockObject) {
_apiHandle = ...; // TODO: open the session
}
_isOpen = true;
}
public void CloseSession()
{
const int MaxTries = 10;
for (int i = 0; _isOpen && i < MaxTries; i++) {
try {
lock (_lockObject) {
// TODO: close the session
}
_isOpen = false;
} catch {
}
}
}
public void Dispose()
{
CloseSession();
}
}
Note that the methods are instance methods, now.
Now you can ensure the closing of the session with a using statement:
using (var session = new DllWrapper()) {
try {
session.StartSession();
// TODO: work with the session
} catch(ApplicationException ex) {
// TODO: log the error
// This is for exceptions not related to closing the session. If such exceptions
// cannot occur, you can drop the try-catch completely.
}
} // Closes the session automatically by calling `Dispose()`.
You can improve naming by calling this class Session and the methods Open and Close. The user of this class does not need to know that it is a wrapper. This is just an implementation detail. Also, the naming of the methods is now symmetrical and there is no need to repeat the name Session.
By encapsulating all the session related stuff, including error handling, recovery from error situations and disposal of resources, you can considerably diminish the mess in your code. The Session class is now a high-level abstraction. The old DllWrapper was somewhere at mid distance between low-level and high-level.

C# Read/Write file from multiple applications

I have scenario where we are maintaining Rates file(.xml) which is access by 3 different application running on 3 different servers. All 3 application uses RatesMaintenance.dll which has below 4 methods Load, Write, Read and Close.
All 3 applications writes into file continuously and hence I have added Monitor.Enter and Monitor.Exit mechanism assuming that these 3 operation from 3 different application will not collide. But at this moment, in some case, I am getting error - "Could not open the rates file"
As per my understanding, this means, some reason 3 application tries to access at same. Could anyone please suggest how to handle such scenario?
Monitor.Enter(RatesFileLock);
try
{
//Open Rates file
LoadRatesFile(false);
//Write Rates into file
WriteRatesToFile();
//Close Rates file
CloseRatesFile();
}
finally
{
Monitor.Exit(RatesFileLock);
}
Method signature of Load-
LoadRatesFile(bool isReadOnly)
For opening file-
new FileStream(RatesFilePath,
isReadOnly ? FileMode.Open : FileMode.OpenOrCreate,
isReadOnly ? FileAccess.Read : FileAccess.ReadWrite,
isReadOnly ? FileShare.ReadWrite : FileShare.None);
.... remaining Rates reading logic code here
For reading Rates from file-
Rates = LoadRatesFile(true);
For Writing Rates into file-
if (_RatesFileStream != null && _RatesInfo != null && _RatesFileSerializer != null)
{
_RatesFileStream.SetLength(0);
_RatesFileSerializer.Serialize(_RatesFileStream, _RatesInfo);
}
In closing file method-
_RatesFileStream.Close();
_RatesFileStream = null;
I hope, I try to explain my scenario in details. Please let me know in case anyone more details.
While the other answers are correct and that you won't be able to get a perfect solution with files that are accessed concurrently by multiple processes, adding a retry mechanism may make it reliable enough for your use case.
Before I show one way to do that, I've got two minor suggestions - C#'s "using" blocks are really useful for dealing with resources such as files and locks that you really want to be sure to dispose of after use. In your code, the monitor is always exited because you use try..finally (though this would still be clearer with an outer "lock" block) but you don't close the file if the WriteRatesToFile method fails.
So, firstly, I'd suggest changing your code to something like the following -
private static object _ratesFileLock = new object();
public void UpdateRates()
{
lock (_ratesFileLock)
{
using (var stream = GetRatesFileStream())
{
var rates = LoadRatesFile(stream);
// Apply any other update logic here
WriteRatesToFile(rates, stream);
}
}
}
private Stream GetRatesFileStream()
{
return File.Open("rates.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite);
}
private IEnumerable<Rate> LoadRatesFile(Stream stream)
{
// Apply any other logic here
return RatesSerialiser.Deserialise(stream);
}
private void WriteRatesToFile(IEnumerable<Rate> rates, Stream stream)
{
RatesSerialiser.Serialise(rates, stream);
}
This tries to opens the file stream once and then reuses it between load and write actions - and reliably dispose of it, even if an error is encountered inside in the using block (same applies to the "lock" block, which is simpler than Monitor.Enter/Exit and try..finally).
This could quite simply be extended to include a retry mechanism so that if the file is locked by another process then we wait a short time and then try again -
private static object _ratesFileLock = new object();
public void UpdateRates()
{
Attempt(TryToUpdateRates, maximumNumberOfAttempts: 50, timeToWaitBetweenRetriesInMs: 100);
}
private void TryToUpdateRates()
{
lock (_ratesFileLock)
{
using (var stream = GetRatesFileStream())
{
var rates = LoadRatesFile(stream);
// Apply any other update logic here
WriteRatesToFile(rates, stream);
}
}
}
private Stream GetRatesFileStream()
{
return File.Open("rates.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite);
}
private IEnumerable<Rate> LoadRatesFile(Stream stream)
{
// Apply any other logic here
return RatesSerialiser.Deserialise(stream);
}
private void WriteRatesToFile(IEnumerable<Rate> rates, Stream stream)
{
RatesSerialiser.Serialise(rates, stream);
}
private static void Attempt(Action work, int maximumNumberOfAttempts, int timeToWaitBetweenRetriesInMs)
{
var numberOfFailedAttempts = 0;
while (true)
{
try
{
work();
return;
}
catch
{
numberOfFailedAttempts++;
if (numberOfFailedAttempts >= maximumNumberOfAttempts)
throw;
Thread.Sleep(timeToWaitBetweenRetriesInMs);
}
}
}
What you're trying to do is difficult bordering on impossible. I won't say that it's impossible because there is always a way, but it's better not to try to make something work in a way it wasn't intended to.
And even if you get it to work and you could ensure that applications on multiple servers don't overstep each other, someone could write some other process that locks the same file because it doesn't know about the system in place for gaining access to that file and playing well with the other apps.
You could check to see if the file is in use before opening it, but there's no guarantee that another server won't open it in between when you checked and when you tried to open it.
The ideal answer is not to try to use a file as database accessed by multiple applications concurrently. That's exactly what databases are for. They can handle multiple concurrent requests to read and write records. Sometimes we use files for logs or other data. But if you've got applications going on three servers then you really need a database.

Out of memory exceptions when using MemoryStream in cache

We are dealing with a lot of files which need to be opened and close for data reads mostly.
Is it a good idea or not to cache the memorystream of each file in a temp hashtable or some other object?
We have noticed when opening files over 100MB we are running into out of memory exceptions.
We are using a wpf app.
We could successfully open the files 1 or 2 time sometimes 3 to 4 times but after that we are running into out of memory exceptions.
If you are currently caching these files, then you would expect to run out of memory quite quickly.
If you aren't caching them yet, don't, because you'll just make it worse. Perhaps you have a memory leak? Are you disposing of the memorystream once you've used it?
The best way to deal with large files is to stream data in and out (using FileStreams), so that you don't have to have the whole file in memory at once...
One issue with the MemoryStream is the internal buffer doubles in size each time the capacity is forced to increase. Even if your MemoryStream is 100MB and your file is 101MB, as soon as you try to write that last 1MB to the MemoryStream the internal buffer on MemoryStream is doubled to 200MB. You may reduce this if you give the Memory Buffer a starting capacity equal to that of you files. But this will still allow the files to use all of the memory and stop any new allocations after the some of the files are loaded. If create a cache object that is help inside of a WeakReference object you would be able to allow the garbage collector to toss a few of your cached files as needed. But don't forget you will need to add code to recreate the lost cache on demand.
public class CacheStore<TKey, TCache>
{
private static object _lockStore = new object();
private static CacheStore<TKey, TCache> _store;
private static object _lockCache = new object();
private static Dictionary<TKey, TCache> _cache =
new Dictionary<TKey, TCache>();
public TCache this[TKey index]
{
get
{
lock (_lockCache)
{
if (_cache.ContainsKey(index))
return _cache[index];
return default(TCache);
}
}
set
{
lock (_lockCache)
{
if (_cache.ContainsKey(index))
_cache.Remove(index);
_cache.Add(index, value);
}
}
}
public static CacheStore<TKey, TCache> Instance
{
get
{
lock (_lockStore)
{
if (_store == null)
_store = new CacheStore<TKey, TCache>();
return _store;
}
}
}
}
public class FileCache
{
private WeakReference _cache;
public FileCache(string fileLocation)
{
if (!File.Exists(fileLocation))
throw new FileNotFoundException("fileLocation", fileLocation);
this.FileLocation = fileLocation;
}
private MemoryStream GetStream()
{
if (!File.Exists(this.FileLocation))
throw new FileNotFoundException("fileLocation", FileLocation);
return new MemoryStream(File.ReadAllBytes(this.FileLocation));
}
public string FileLocation { get; private set; }
public MemoryStream Data
{
get
{
if (_cache == null)
_cache = new WeakReference(GetStream(), false);
var ret = _cache.Target as MemoryStream;
if (ret == null)
{
Recreated++;
ret = GetStream();
_cache.Target = ret;
}
return ret;
}
}
public int Recreated { get; private set; }
}
class Program
{
static void Main(string[] args)
{
var cache = CacheStore<string, FileCache>.Instance;
var fileName = #"c:\boot.ini";
cache[fileName] = new FileCache(fileName);
var ret = cache[fileName].Data.ToArray();
Console.WriteLine("Recreated {0}", cache[fileName].Recreated);
Console.WriteLine(Encoding.ASCII.GetString(ret));
GC.Collect();
var ret2 = cache[fileName].Data.ToArray();
Console.WriteLine("Recreated {0}", cache[fileName].Recreated);
Console.WriteLine(Encoding.ASCII.GetString(ret2));
GC.Collect();
var ret3 = cache[fileName].Data.ToArray();
Console.WriteLine("Recreated {0}", cache[fileName].Recreated);
Console.WriteLine(Encoding.ASCII.GetString(ret3));
Console.Read();
}
}
It's very dificutl say "yes" or "no", if is file content caching right in the common case and/or with so little informations. However - finited resources are real state of world, and you (as developer) must count with it. If you want cache something, you should use some mechanism for auto unloading data. In .NET framework you can use a WeakReference class, which unloads the target object (byte array and memory stream are objects too).
If you have the HW in you control, and you can use 64bit and have funds for very big RAM, you can cache big files.
However, you should be humble to resources (cpu,ram) and use the "cheap" way of implementation.
I think that the problem is that after you are done, the file is not disposed immediatly, it is waiting to the next GC cycle.
Streams are IDisposable, whice means you can and should use the using block. then the stream will dispose immidiatly when your are done dealing with it.
I don't think that caching such amount of data is a good solution, even if you don't get ever memroy overflow. Check out Memory Mapped files solution, which means that file lays on file system but speed of reading is almost equal to the in memory ones (there is an overhead for sure). Check out this link. MemoryMappedFiles
P.S. Ther are pretty good articles and examples on this topic arround in internet.
Good Luck.

Close application from captive IE session

My question is: "Can this be done better?" and if so, how? Any ideas?
We need to start a captive IE session from within an "invisible" C# .NET 3.5 application, and quit both the IE session and the "parent" application after processing a certain request.
I've been mucking around with this problem for the last week or so... and this morning I've finally reached what I think is a robust solution; but I'm a bit of a C# noob (though I've been a professional programmer for 10 years), so I'm seeking a second or third opinion; and any other options, critiques, suggestions, or comments... Especially: is SHDocVw still the preferred method of creating a "captive but not imbedded" Internet Explorer session?
As I see things, the tricky bit is disposing of the unmanaged InternetExplorerApplication COM object, so I've wrapped it in an IDisposable class called InternetExplorer
My basic approach is:
Application.Run MyApp, which is-a ApplicationContext, and is IQuitable.
I think an app is needed to keep the program open whilste we wait for the IE request?
I guess maybe a (non-daemon) listener-loop thread might also work?
MyApp's constructor creates a new InternetExporer object passing (IQuitable)this
InternetExporer's constructor starts a new IE session, and navigates it to a URL.
When a certain URL is requested InternetExporer calls-back the "parents" Quit method.
Background:
The real story is: I'm writing a plugin for MapInfo (A GIS Client). The plugin hijacks the "Start Extraction" HTTP request from IE to the server, modifies the URL slightly and sends a HTTPRequest in it's place. We parse the respose XML into MIF files [PDF 196K], which we then import and open in MapInfo. Then we quit the IE session, and close the "plugin" application.
SSCCE
using System;
using System.Windows.Forms;
// IE COM interface
// reference ~ C:\Windows\System32\SHDocVw.dll
using SHDocVw;
namespace QuitAppFromCaptiveIE
{
static class Program {
[STAThread]
static void Main() {
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Application.Run(new MyApp());
}
}
interface IQuitable {
void Quit();
}
public class MyApp : ApplicationContext, IQuitable {
private InternetExplorer ie = null;
public MyApp() {
// create a new Internet Explorer COM component - starts IE application.
this.ie = new InternetExplorer(this);
this.ie.Open("www.microsoft.com");
}
#region IQuitable Members
public void Quit() {
if (ie != null) {
ie.Dispose();
ie = null;
}
Application.Exit();
}
#endregion
}
class InternetExplorer : IDisposable, IQuitable
{
// allows us to end the parent application when IE is closed.
private IQuitable parent;
private bool _parentIsQuited = false;
private bool _ieIsQuited = false;
private SHDocVw.InternetExplorer ie; // Old (VB4 era) IE COM component
public InternetExplorer(IQuitable parent) {
// lock-onto the parent app to quit it when IE is closed.
this.parent = parent;
// create a new Internet Explorer COM component - starts IE application.
this.ie = new SHDocVw.InternetExplorerClass();
// hook-up our navigate-event interceptor
ie.BeforeNavigate2 += new DWebBrowserEvents2_BeforeNavigate2EventHandler(ie_BeforeNavigate2);
}
public void Open(string url) {
object o = null;
// make the captive IE session navigate to the given URL.
ie.Navigate(url, ref o, ref o, ref o, ref o);
// now make the ie window visible
ie.Visible = true;
}
// this callback event handler is invoked prior to the captive IE
// session navigating (opening) a URL. Navigate-TWO handles both
// external (normal) and internal (AJAX) requests.
// For example: You could create a history-log file of every page
// visited by each captive session.
// Being fired BEFORE the actual navigation allows you to hijack
// (ie intercept) requests to certain URLs; in this case a request
// to http://support.microsoft.com/ terminates the Browser session
// and this program!
void ie_BeforeNavigate2(object pDisp, ref object URL, ref object Flags, ref object TargetFrameName, ref object PostData, ref object Headers, ref bool Cancel) {
if (URL.Equals("http://support.microsoft.com/")) {
this.Quit();
}
}
#region IDisposable Members
public void Dispose() {
quitIE();
}
#endregion
private void quitIE() {
// close my unmanaged COM object
if (ie != null && !_ieIsQuited) {
_ieIsQuited = true;
ie.Quit();
ie = null;
}
}
#region IQuitable Members
public void Quit() {
// close my unmanaged COM object
quitIE();
// quit the parent app as well.
if (parent != null && !_parentIsQuited) {
_parentIsQuited = true;
parent.Quit();
parent = null;
}
}
#endregion
}
}
I'm reasonably sure that System.Windows.Forms.WebBrowser actually uses the IE Trident browser control internally. It shouldn't be necessary to do COM interop unless you are using C# 1.x.
The long and the short of it appears to be (I'm still NOT an expert, by any stretch of the imagination) that SHDocVw.dll is still the preferred method for launching a "captive" Internet Explorer session (as apposed to embedding a browser in your application).
The code I posted previously isn't the best solution, IMHO. In the final version:
IQuitable is history
Both MyApp and InternetExplorer classes implement IDisposable
Both Dispose methods just return if *_isDisposed* is true.
The following coded includes some pseudocode, for brevity:
private volatile bool _isDisposed = false;
/**
* _isDisposed stops the two "partners" in the conversation (us and
* Internet Explorer) from going into "infinite recursion", by calling
* each others Dispose methods within there Dispose methods.
*
* NOTE: I **think** that making _isDisposed volatile deals adequately
* with the inherent race condition, but I'm NOT certain! Comments
* welcome on this one.
*/
public void Dispose() {
if (!_isDisposed) {
_isDisposed = true;
try {
try { release my unmanaged resources } catch { log }
try {
IE calls MyApp.Dispose() here // and FALLOUT on failure
MyApp calls IE.Dispose(); here
} catch {
log
}
} finally {
base.Dispose(); // ALLWAYS dispose base, no matter what!
}
}
}
To quit the application from the IE class you just call it's local Dispose method, which calls MyApps Dispose, which calls IE's Dispose again but isDisposed is true so it just returns. Then we call Application.ExitThread() and fall out of MyApp's Dispose... and then we fall out of IE's Dispose method, and the event-system stops; and the application terminates nicely. Finally!
NOTE ON EDIT: I've just reused this approach with a Robo framework process, which is as a "captive process" of MyApp... the thing it's remote-controling. Convoluted, but it works... So I update my "self answer" with my latest learnings while I was here.

Finalizer launched while its object was still being used

Summary: C#/.NET is supposed to be garbage collected. C# has a destructor, used to clean resources. What happen when an object A is garbage collected the same line I try to clone one of its variable members? Apparently, on multiprocessors, sometimes, the garbage collector wins...
The problem
Today, on a training session on C#, the teacher showed us some code which contained a bug only when run on multiprocessors.
I'll summarize to say that sometimes, the compiler or the JIT screws up by calling the finalizer of a C# class object before returning from its called method.
The full code, given in Visual C++ 2005 documentation, will be posted as an "answer" to avoid making a very very large questions, but the essential are below:
The following class has a "Hash" property which will return a cloned copy of an internal array. At is construction, the first item of the array has a value of 2. In the destructor, its value is set to zero.
The point is: If you try to get the "Hash" property of "Example", you'll get a clean copy of the array, whose first item is still 2, as the object is being used (and as such, not being garbage collected/finalized):
public class Example
{
private int nValue;
public int N { get { return nValue; } }
// The Hash property is slower because it clones an array. When
// KeepAlive is not used, the finalizer sometimes runs before
// the Hash property value is read.
private byte[] hashValue;
public byte[] Hash { get { return (byte[])hashValue.Clone(); } }
public Example()
{
nValue = 2;
hashValue = new byte[20];
hashValue[0] = 2;
}
~Example()
{
nValue = 0;
if (hashValue != null)
{
Array.Clear(hashValue, 0, hashValue.Length);
}
}
}
But nothing is so simple...
The code using this class is wokring inside a thread, and of course, for the test, the app is heavily multithreaded:
public static void Main(string[] args)
{
Thread t = new Thread(new ThreadStart(ThreadProc));
t.Start();
t.Join();
}
private static void ThreadProc()
{
// running is a boolean which is always true until
// the user press ENTER
while (running) DoWork();
}
The DoWork static method is the code where the problem happens:
private static void DoWork()
{
Example ex = new Example();
byte[] res = ex.Hash; // [1]
// If the finalizer runs before the call to the Hash
// property completes, the hashValue array might be
// cleared before the property value is read. The
// following test detects that.
if (res[0] != 2)
{
// Oops... The finalizer of ex was launched before
// the Hash method/property completed
}
}
Once every 1,000,000 excutions of DoWork, apparently, the Garbage Collector does its magic, and tries to reclaim "ex", as it is not anymore referenced in the remaning code of the function, and this time, it is faster than the "Hash" get method. So what we have in the end is a clone of a zero-ed byte array, instead of having the right one (with the 1st item at 2).
My guess is that there is inlining of the code, which essentially replaces the line marked [1] in the DoWork function by something like:
// Supposed inlined processing
byte[] res2 = ex.Hash2;
// note that after this line, "ex" could be garbage collected,
// but not res2
byte[] res = (byte[])res2.Clone();
If we supposed Hash2 is a simple accessor coded like:
// Hash2 code:
public byte[] Hash2 { get { return (byte[])hashValue; } }
So, the question is: Is this supposed to work that way in C#/.NET, or could this be considered as a bug of either the compiler of the JIT?
edit
See Chris Brumme's and Chris Lyons' blogs for an explanation.
http://blogs.msdn.com/cbrumme/archive/2003/04/19/51365.aspx
http://blogs.msdn.com/clyon/archive/2004/09/21/232445.aspx
Everyone's answer was interesting, but I couldn't choose one better than the other. So I gave you all a +1...
Sorry
:-)
Edit 2
I was unable to reproduce the problem on Linux/Ubuntu/Mono, despite using the same code on the same conditions (multiple same executable running simultaneously, release mode, etc.)
It's simply a bug in your code: finalizers should not be accessing managed objects.
The only reason to implement a finalizer is to release unmanaged resources. And in this case, you should carefully implement the standard IDisposable pattern.
With this pattern, you implement a protected method "protected Dispose(bool disposing)". When this method is called from the finalizer, it cleans up unmanaged resources, but does not attempt to clean up managed resources.
In your example, you don't have any unmanaged resources, so should not be implementing a finalizer.
What you're seeing is perfectly natural.
You don't keep a reference to the object that owns the byte array, so that object (not the byte array) is actually free for the garbage collector to collect.
The garbage collector really can be that aggressive.
So if you call a method on your object, which returns a reference to an internal data structure, and the finalizer for your object mess up that data structure, you need to keep a live reference to the object as well.
The garbage collector sees that the ex variable isn't used in that method any more, so it can, and as you notice, will garbage collect it under the right circumstances (ie. timing and need).
The correct way to do this is to call GC.KeepAlive on ex, so add this line of code to the bottom of your method, and all should be well:
GC.KeepAlive(ex);
I learned about this aggressive behavior by reading the book Applied .NET Framework Programming by Jeffrey Richter.
this looks like a race condition between your work thread and the GC thread(s); to avoid it, i think there are two options:
(1) change your if statement to use ex.Hash[0] instead of res, so that ex cannot be GC'd prematurely, or
(2) lock ex for the duration of the call to Hash
that's a pretty spiffy example - was the teacher's point that there may be a bug in the JIT compiler that only manifests on multicore systems, or that this kind of coding can have subtle race conditions with garbage collection?
I think what you are seeing is reasonable behavior due to the fact that things are running on multiple threads. This is the reason for the GC.KeepAlive() method, which should be used in this case to tell the GC that the object is still being used and that it isn't a candidate for cleanup.
Looking at the DoWork function in your "full code" response, the problem is that immediately after this line of code:
byte[] res = ex.Hash;
the function no longer makes any references to the ex object, so it becomes eligible for garbage collection at that point. Adding the call to GC.KeepAlive would prevent this from happening.
Yes, this is an issue that has come up before.
Its even more fun in that you need to run release for this to happen and you end up stratching your head going 'huh, how can that be null?'.
Interesting comment from Chris Brumme's blog
http://blogs.msdn.com/cbrumme/archive/2003/04/19/51365.aspx
class C {<br>
IntPtr _handle;
Static void OperateOnHandle(IntPtr h) { ... }
void m() {
OperateOnHandle(_handle);
...
}
...
}
class Other {
void work() {
if (something) {
C aC = new C();
aC.m();
... // most guess here
} else {
...
}
}
}
So we can’t say how long ‘aC’ might live in the above code. The JIT might report the reference until Other.work() completes. It might inline Other.work() into some other method, and report aC even longer. Even if you add “aC = null;” after your usage of it, the JIT is free to consider this assignment to be dead code and eliminate it. Regardless of when the JIT stops reporting the reference, the GC might not get around to collecting it for some time.
It’s more interesting to worry about the earliest point that aC could be collected. If you are like most people, you’ll guess that the soonest aC becomes eligible for collection is at the closing brace of Other.work()’s “if” clause, where I’ve added the comment. In fact, braces don’t exist in the IL. They are a syntactic contract between you and your language compiler. Other.work() is free to stop reporting aC as soon as it has initiated the call to aC.m().
That's perfectly nornal for the finalizer to be called in your do work method as after the
ex.Hash call, the CLR knows that the ex instance won't be needed anymore...
Now, if you want to keep the instance alive do this:
private static void DoWork()
{
Example ex = new Example();
byte[] res = ex.Hash; // [1]
// If the finalizer runs before the call to the Hash
// property completes, the hashValue array might be
// cleared before the property value is read. The
// following test detects that.
if (res[0] != 2) // NOTE
{
// Oops... The finalizer of ex was launched before
// the Hash method/property completed
}
GC.KeepAlive(ex); // keep our instance alive in case we need it.. uh.. we don't
}
GC.KeepAlive does... nothing :) it's an empty not inlinable /jittable method whose only purpose is to trick the GC into thinking the object will be used after this.
WARNING: Your example is perfectly valid if the DoWork method were a managed C++ method... You DO have to manually keep the managed instances alive manually if you don't want the destructor to be called from within another thread. IE. you pass a reference to a managed object who is going to delete a blob of unmanaged memory when finalized, and the method is using this same blob. If you don't hold the instance alive, you're going to have a race condition between the GC and your method's thread.
And this will end up in tears. And managed heap corruption...
The Full Code
You'll find below the full code, copy/pasted from a Visual C++ 2008 .cs file. As I'm now on Linux, and without any Mono compiler or knowledge about its use, there's no way I can do tests now. Still, a couple of hours ago, I saw this code work and its bug:
using System;
using System.Threading;
public class Example
{
private int nValue;
public int N { get { return nValue; } }
// The Hash property is slower because it clones an array. When
// KeepAlive is not used, the finalizer sometimes runs before
// the Hash property value is read.
private byte[] hashValue;
public byte[] Hash { get { return (byte[])hashValue.Clone(); } }
public byte[] Hash2 { get { return (byte[])hashValue; } }
public int returnNothing() { return 25; }
public Example()
{
nValue = 2;
hashValue = new byte[20];
hashValue[0] = 2;
}
~Example()
{
nValue = 0;
if (hashValue != null)
{
Array.Clear(hashValue, 0, hashValue.Length);
}
}
}
public class Test
{
private static int totalCount = 0;
private static int finalizerFirstCount = 0;
// This variable controls the thread that runs the demo.
private static bool running = true;
// In order to demonstrate the finalizer running first, the
// DoWork method must create an Example object and invoke its
// Hash property. If there are no other calls to members of
// the Example object in DoWork, garbage collection reclaims
// the Example object aggressively. Sometimes this means that
// the finalizer runs before the call to the Hash property
// completes.
private static void DoWork()
{
totalCount++;
// Create an Example object and save the value of the
// Hash property. There are no more calls to members of
// the object in the DoWork method, so it is available
// for aggressive garbage collection.
Example ex = new Example();
// Normal processing
byte[] res = ex.Hash;
// Supposed inlined processing
//byte[] res2 = ex.Hash2;
//byte[] res = (byte[])res2.Clone();
// successful try to keep reference alive
//ex.returnNothing();
// Failed try to keep reference alive
//ex = null;
// If the finalizer runs before the call to the Hash
// property completes, the hashValue array might be
// cleared before the property value is read. The
// following test detects that.
if (res[0] != 2)
{
finalizerFirstCount++;
Console.WriteLine("The finalizer ran first at {0} iterations.", totalCount);
}
//GC.KeepAlive(ex);
}
public static void Main(string[] args)
{
Console.WriteLine("Test:");
// Create a thread to run the test.
Thread t = new Thread(new ThreadStart(ThreadProc));
t.Start();
// The thread runs until Enter is pressed.
Console.WriteLine("Press Enter to stop the program.");
Console.ReadLine();
running = false;
// Wait for the thread to end.
t.Join();
Console.WriteLine("{0} iterations total; the finalizer ran first {1} times.", totalCount, finalizerFirstCount);
}
private static void ThreadProc()
{
while (running) DoWork();
}
}
For those interested, I can send the zipped project through email.

Categories

Resources