I noticed that ManagementObject is IDisposable, but it's also returned from ManagementClass.GetInstances() and ManagementObjectSearcher.Get(), does this mean I need to dispose each object encountered?
Like so:
ManagementObject ret;
foreach(ManagementObject mo in searcher.Get()) {
if( IsWhatIWant(mo) ) ret = mo;
else mo.Dispose();
}
Further confounding this: there's a bug in ManagementBaseObject where it does not correctly implement IDisposable (See Using clause fails to call Dispose? ) so you need to call it yourself, or use a wrapper around it that does correctly call it.
This is irritating because I have so many ManagementObjectCollections around.
I've created a helper object to dispose all created management objects:
public class ManagementObjectDisposer : IDisposable
{
private List<IDisposable> disposables = new List<IDisposable>();
/// <summary>
/// Workaround to dispose ManagementBaseObject properly.
/// See http://stackoverflow.com/questions/11896282
/// </summary>
/// <param name="disposable"></param>
public static void DisposeOne(IDisposable disposable)
{
ManagementBaseObject mbo = disposable as ManagementBaseObject;
if (mbo != null)
mbo.Dispose();
else
disposable.Dispose();
}
public void Dispose()
{
Exception firstException = null;
foreach (IDisposable d in Enumerable.Reverse(disposables))
{
try
{
DisposeOne(d);
}
catch (Exception ex)
{
if (firstException == null)
firstException = ex;
else
cvtLogger.GetLogger(this).Error($"Swallowing exception when disposing: {d.GetType()}", ex);
}
}
disposables.Clear();
if (firstException != null)
throw firstException;
}
public T Add<T>(T disposable) where T : IDisposable
{
disposables.Add(disposable);
return disposable;
}
/// <summary>
/// Helper for ManagementObjectSearcher with adding all objects to the disposables.
/// </summary>
/// <param name="query">The query string.</param>
public IEnumerable<ManagementBaseObject> Search(string query)
{
ManagementObjectSearcher searcher = this.Add(new ManagementObjectSearcher(query));
return EnumerateCollection(searcher.Get());
}
/// <summary>
/// Helper for adding ManagementObjectCollection and enumerating it.
/// </summary>
public IEnumerable<ManagementBaseObject> EnumerateCollection(ManagementObjectCollection collection)
{
this.Add(collection);
ManagementObjectCollection.ManagementObjectEnumerator enumerator = this.Add(collection.GetEnumerator());
while (enumerator.MoveNext())
yield return this.Add(enumerator.Current);
}
}
Just use it like:
using (var moDisposer = new ManagementObjectDisposer())
{
foreach (var mobj = moDisposer.Search("SELECT * FROM Win32_Processor")
Console.WriteLine(mobj["DeviceID"]);
}
Note: the ManagementClass.GetInstances() is easy to add to the ManagementObjectDisposer, too.
This is irritating because I have so many ManagementObjectCollections around.
Which has nothing to do with calling Dispose(), that only releases the underlying unmanaged COM objects. ManagementObjectCollection is a managed class, instances of it are garbage collected. Which is automatic, you can only help by calling GC.Collect(). Your program is probably just creating a lot of System.Management objects, possibly because that's the only thing it ever does. The quoted bug was fixed in the current versions of .NET 3.5SP1 and .NET 4.5 that I have installed on my machine.
So if you don't have a patched version of .NET then you don't only see a lot of System.Management objects on the GC heap, your process will also consume a lot of unmanaged memory. If the garbage collector doesn't run frequently enough then that can cause the program to crash with OOM. You didn't mention that as a failure mode so it is not strongly indicated that you have a real problem.
The initial size of generation 0 of the GC heap is 2 megabytes, it can grow to 8+ megabytes. That is a lot of ManagementObjectCollections objects, it is a very small object that takes only 24 bytes in 32-bit mode. The actual collection is unmanaged. Use Perfmon.exe or your memory profiler to check that the garbage collector is running frequently enough. If it doesn't then keep an eye on your program's VM size. If that's ballooning then using a counter in your query loop and calling GC.Collect() when it is high enough is a viable workaround. Careful with the info you get out of a memory profiler, it can irritate for the wrong reasons.
Actually the code from:
http://referencesource.microsoft.com/#System.Management/managementobjectcollection.cs
and also the Symbols from the Microsoft Symbol Server
http://msdl.microsoft.com/download/symbols
Imply ManagementObjectCollection is IDisposable which implies it is for some reason using unmanaged resources or it is incorrectly using the IDisposable interface.
If you do a lot of WMI queries you should always dispose the return ManagementObjectCollection to avoid Quota violation exceptions.
System.Management.ManagementException: Quota violation
at System.Management.ManagementException.ThrowWithExtendedInfo(ManagementStatus errorCode)
at System.Management.ManagementObjectCollection.ManagementObjectEnumerator.MoveNext()
After two weeks of looking for a memory leak while working with WMI, the following solution helped:
using (ManagementObjectSearcher searcher = new ManagementObjectSearcher(scope, objectQuery, enumerationOptions))
using (ManagementObjectCollection collection = searcher.Get())
{
foreach (ManagementBaseObject obj in collection)
{
using (obj) // calls Component.Dispose()
{
try
{
// Your code
}
finally
{
obj.Dispose(); // calls ManagementBaseObject.Dispose()
}
}
}
}
The reason is in ManagementBaseObject.Dispose(), which is marked as new, so using doesn't call it.
So ManagementBaseObject.Dispose() was called by the finalizer (GC). With a huge amount of ManagementBaseObject (thousands per second) in a highly loaded system after a couple of days, the duration of the finalizers became more than an hour, which caused memory leak, OutOfMemoryException and application termination.
This issue is still present in .Net Framework 4.8.
Theoretically, you can remove using for ManagementBaseObject, because ManagementBaseObject.Dispose() calls the base Dispose(), but who knows what surprises are in other versions of .Net
Also, the Dispose method is marked as new in the inherited classes: ManagementObject and ManagementClass.
Related
I have a very large project with multiple pages, where each page has many IDisposable members.
I'm trying to figure out a way to dispose all the IDisposable members in a loop so I won't have to type x1.Dispose(); x2.Dispose; ... xn.Dispose on each class.
Is there a way to do this?
Thank you.
Sure, just make sure that you create a list to hold them, and a try finally block to protect yourself from leaking them.
// List for holding your disposable types
var connectionList = new List<IDisposable>();
try
{
// Instantiate your page states, this may be need to be done at a high level
// These additions are over simplified, as there will be nested calls
// building this list, in other words these will more than likely take place in methods
connectionList.Add(x1);
connectionList.Add(x2);
connectionList.Add(x3);
}
finally
{
foreach(IDisposable disposable in connectionList)
{
try
{
disposable.Dispose();
}
catch(Exception Ex)
{
// Log any error? This must be caught in order to prevent
// leaking the disposable resources in the rest of the list
}
}
}
However, this approach is not always ideal. The nature of the nested calls will get complicated and require the call to be so far up in the architecture of your program that you may want to consider just locally handling these resources.
Moreover, this approach critically fails in the scenario where these Disposable resources are intensive and need to be immediately released. While you can do this, i.e. track your Disposable elements and then do it all at once, it is best to try to get the object lifetime as short as possible for managed resources like this.
Whatever you do, ensure not to leak the Disposable resource. If these are connection threads, and they are inactive for some period of time, it may also be wise to simply look at their state and then re-use them in different places instead of letting them hang around.
Using reflection (not tested):
public static void DisposeAllMembersWithReflection(object target)
{
if (target == null) return;
// get all fields, you can change it to GetProperties() or GetMembers()
var fields = target.GetType().GetFields(System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.NonPublic);
// get all fields that implement IDisposable
var disposables = fields.Where(x => x.FieldType.GetInterfaces().Contains(typeof(IDisposable)));
foreach (var disposableField in disposables)
{
var value = (IDisposable)disposableField.GetValue(target);
if (value != null)
value.Dispose();
}
}
Create method which will dispose all your disposable objects:
public void DisposeAll()
{
x1.Dispose();
x2.Dispose();
x3.Dispose();
. . .
}
and call it wherever you need it.
Background info
I've developed a desktop application with Windows Form (C#) for scanning, previewing and saving images.
The app behaviour while scanning is the following:
Scan n images
Get a Bitmap for each image and store it in a temporary file
Show resized thumbnails as a preview
Image memory management: Compressible images
For managing the memory usage, I've created a CompressibleImage class which encapsulates a Bitmap file and reads/writes images file on a FileStream. When an image is not longer required by the application, it is written into a file stream. When the app requires an image (ie. user double-click on a thumbnail), a Bitmap file is created from the stream. Here are the main CompressibleImage's methods:
/// Gets the uncompressed image. If the image is compressed, it will be uncompressed
public Image GetDecompressedImage()
{
if (decompressedImage == null)
{
// Read Bitmap from file stream
stream.Seek(0, SeekOrigin.Begin);
decompressedImage = new Bitmap(stream);
}
return decompressedImage;
}
/// Clears the uncompressed image, leaving the compressed one in memory.
public void ClearDecompressedImage()
{
// If Bitmap file exists, write it to file and dispose it
if (decompressedImage != null)
{
if (stream == null)
{
stream = new FileStream(FileStreamPath, FileMode.Create);
}
decompressedImage.Save(stream, format);
// The Dispose() call does not solve the issue
// decompressedImage.Dispose();
decompressedImage = null;
}
}
/// <summary>
/// Class destructor. It disposes the decompressed image (if this exists),
/// closes the stream and delete the temporary file associated.
/// </summary>
~CompressibleImage()
{
if (decompressedImage != null)
{
decompressedImage.Dispose();
}
if(stream != null)
{
stream.Close();
File.Delete(stream.Name);
stream.Dispose();
}
}
Application level
The application uses CompressibleImage to create image files mainly during the scan method and the saving process.
The scan method works fine and it basically:
Acquire Bitmap from scanner
Create a CompressibleImage from the scanned Bitmap
Write the Bitmap to the file stream
The save method works fine on my machine and its behaviour is as follows:
1. For each CompressibleImage decompress (read & build) Bitmap from stream
2. Save image
3. Compress image
Here is the save method:
private void saveImage_button_Click(object sender, EventArgs e)
{
if (Directory.Exists(OutputPath) == false && File.Exists(OutputPath) == false)
{
Directory.CreateDirectory(OutputPath);
}
ListView.CheckedListViewItemCollection checkedItems = listView1.CheckedItems;
if(checkedItems.Count > 0)
{
for (int i = 0; i < checkedItems.Count; ++i)
{
int index = checkedItems[i].Index;
Bitmap image = (Bitmap)compressibleImageList.ElementAt(index).GetDecompressedImage();
try
{
image.Save(OutputPath + index.ToString() +
Module.PNG_FORMAT, ImageFormat.Png);
compressibleImageList.ElementAt(index).ClearDecompressedImage();
progressForm.Increment();
image = null;
}
catch (Exception ex) {
...
}
}
}
}
Issue Description
In my machine the application works fine. There is no memory leak and the scan and save methods do their job smoothly and with a reasonable memory usage (scanning 100 papersheets with less than < 140MB pick).
The problem is, when I tried to test the application on other machines, the Garbage collector doesn't free up the memory, causing a MemoryException during the execution of both methods and when the amount of image is reasonably high (>40). The exception is thrown inside the CompressibleImage.GetDecompressedImage() method when I attempt to decompress (read) an image:
decompressedImage = new Bitmap(stream);
Although I known that the GC clean the memory randomly, in this case it seems like it doesn't even run and in fact the memory is released only when I close the application.
It is possible such different behaviour on similar machines?
System's info
Here are some info about the testing environments. Both machines have:
Processor: Intel i7 2.30GHz
RAM: 8GB
Type: 64-bit
OS: Windows 7 Pro SP 1
Not quite sure about you MemoryException, please provide full stacktrace.
However, I can see you are doing an obvious mistake in your destructor.
You SHOULD NOT refer your managed resources in the Destructor. The reason is being that, GC and Finalizer are using heuristic algorithms to trigger them and you can never predict the execution order of finalizers for your managed objects.
That's why you should use 'disposing' flag in your dispose method and avoid touching managed objects when the execution is coming from the finalizer.
Below example show the general best practice to implement IDisposable interface. Reference : https://msdn.microsoft.com/en-us/library/system.idisposable.dispose(v=vs.110).aspx
public class DisposeExample
{
// A base class that implements IDisposable.
// By implementing IDisposable, you are announcing that
// instances of this type allocate scarce resources.
public class MyResource: IDisposable
{
// Pointer to an external unmanaged resource.
private IntPtr handle;
// Other managed resource this class uses.
private Component component = new Component();
// Track whether Dispose has been called.
private bool disposed = false;
// The class constructor.
public MyResource(IntPtr handle)
{
this.handle = handle;
}
// Implement IDisposable.
// Do not make this method virtual.
// A derived class should not be able to override this method.
public void Dispose()
{
Dispose(true);
// This object will be cleaned up by the Dispose method.
// Therefore, you should call GC.SupressFinalize to
// take this object off the finalization queue
// and prevent finalization code for this object
// from executing a second time.
GC.SuppressFinalize(this);
}
// Dispose(bool disposing) executes in two distinct scenarios.
// If disposing equals true, the method has been called directly
// or indirectly by a user's code. Managed and unmanaged resources
// can be disposed.
// If disposing equals false, the method has been called by the
// runtime from inside the finalizer and you should not reference
// other objects. Only unmanaged resources can be disposed.
protected virtual void Dispose(bool disposing)
{
// Check to see if Dispose has already been called.
if(!this.disposed)
{
// If disposing equals true, dispose all managed
// and unmanaged resources.
if(disposing)
{
// Dispose managed resources.
component.Dispose();
}
// Call the appropriate methods to clean up
// unmanaged resources here.
// If disposing is false,
// only the following code is executed.
CloseHandle(handle);
handle = IntPtr.Zero;
// Note disposing has been done.
disposed = true;
}
}
// Use interop to call the method necessary
// to clean up the unmanaged resource.
[System.Runtime.InteropServices.DllImport("Kernel32")]
private extern static Boolean CloseHandle(IntPtr handle);
// Use C# destructor syntax for finalization code.
// This destructor will run only if the Dispose method
// does not get called.
// It gives your base class the opportunity to finalize.
// Do not provide destructors in types derived from this class.
~MyResource()
{
// Do not re-create Dispose clean-up code here.
// Calling Dispose(false) is optimal in terms of
// readability and maintainability.
Dispose(false);
}
}
public static void Main()
{
// Insert code here to create
// and use the MyResource object.
}
}
When opening files or streams using a class that incorporates the IDisposable interface one should usually make use of using. This will ensure that after the using statement the Dispose method is called. If implemented correctly, this will ensure that the unmanaged resources are released.
MSDN Article on 'using' statement
I have a C# program that accesses a COM Interface to a piece of simulation software called Aspen Plus. I have a very strange memory leak.
When I need to get the result values out of the simulation, I run a series of calls like this, in some cases the variable returned might be null, so I insert a check for that. Then I use FinalReleaseComObject to clean up the COM references.
public override ValueType recvValueFromSim<ValueType>(string path) {
Happ.IHNode tree = this.Aspen.Tree;
dynamic node = tree.FindNode(path);
ValueType retVal = default(ValueType);
if (node != null && node.Value != null) {
retVal = node.Value;
}
Marshal.FinalReleaseComObject(node);
Marshal.FinalReleaseComObject(tree);
node = null;
return retVal;
}
Unfortunately, the above code leaks a lot. It leaks 2MB per simulation. At first I thought the Garbage Collector would eventually run and clean it up, but no dice. After running a couple of hundred simulations, I ran out of memory.
The bizarre thing is, the below code works fine and doesn't leak. I didn't like it, because using catch to check for null references seems like bad form, but this doesn't leak.
public override ValueType recvValueFromSim<ValueType>(string path) {
ValueType node;
try {
node = this.Aspen.Tree.FindNode(path).Value;
return node;
} catch {
return default(ValueType);
}
}
Why doesn't it leak? Does anybody know? The belies why I thought I knew about temporary references and releasing COM objects.
I have a C# COM DLL that calls WMI using the System.Management namespace. The DLL is being loaded into a C++ service. Every time I call the into the WMI classes I'm seeing a HUGE memory leak. After about an hour I am well over 1 GB of memory used.
If I take the same COM DLL and load it into PowerShell using Reflection.LoadFrom it does not leak memory. I have modified the DLL like so and it no longer leaks (still loading into the service with COM):
public class MyComObject
{
public void CallCom()
{
CallSomeWMIStuff();
}
}
To this. This no longer leaks!
public class MyComObject
{
public void CallCom()
{
//CallSomeWMIStuff();
}
}
Here's an example of some of the WMI code:
var scope = new ManagementScope( "root\\cimv2" );
scope.Connect();
using (var myservice = GetService("SomeService", scope))
{
//Some Stuff
}
...
ManagementObject GetService(string serviceName, MangementScope scope)
{
ManagementPath wmiPath = new ManagementPath( serviceName );
using (ManagementClass serviceClass = new ManagementClass( scope, wmiPath, null ))
{
using (ManagementObjectCollection services = serviceClass.GetInstances())
{
ManagementObject serviceObject = null;
// If this service class does not have an instance, create one.
if (services.Count == 0)
{
serviceObject = serviceClass.CreateInstance();
}
else
{
foreach (ManagementObject service in services)
{
serviceObject = service;
break;
}
}
return serviceObject;
}
}
}
EDIT: C++ Snippet:
NAMESPACE::ICSharpComPtr pCSharpCom = NULL;
HRESULT hr = pCSharpCom .CreateInstance(NAMESPACE::CLSID_CSharpCom);
if (FAILED(hr))
{
Log("Failed (hr=%08x)", hr);
return hr;
}
try
{
_bstr_t bstrData = pCSharpCom ->GetData();
strLine = (LPCTSTR)bstrData;
strMessage += strLine;
}
catch (_com_error& err)
{
_bstr_t desc = GetErrorMessage(err);
Log("Excepton %S", (const wchar_t*)desc);
return 0;
}
pCSharpCom ->Release();
Has anyone seen anything like this? We are seeing a similar issue with C++\CLI that's loading a different WMI related DLL directly.
Eventually, the WMI service will no longer be responsive and I will have to restart that service as well.
Edit:
This has to do with the apartment state of the COM object. Added a CoInitializeEx rather than a CoInitialize. I set the thread to MTA. At first it didn't look like this was working until I realized that first time the method was called we were seeing the thread state set to STA rather than MTA! Every subsequent call would be MTA. If I returned right away, before calling the System.Management classes when the thread was STA, I would no longer leak memory!
Any idea why the first one would be STA?
There is not a dispose in the RCW implementation so you are at the mercy of the GC to release the com objects you have created by default. However, you can try using Marshal.FinalReleaseComObject on the RCW instance once you are done with your COM objects. The will force the ref count to zero on the wrapped COM object and it should release. However, this also makes the RCW instance useless so be careful where you call it.
The problem had to do with the apartment state of the thread creating the COM object. There was one thread that created the COM object as MTA and another thread that was creating the COM object as STA. The STA thread was created first and then led to issues with the MTA threads. This caused the finalizer to block on GetToSTA.
I've been developing a C# application that uses DLL interop to an external database application.
This external app starts up at the same time along with my C# app and is available as long as my C# app is running.
Now the real question is related to managing the objects that I need to create to interact with the external application.
When I declare objects that are available from the referenced DLL's these objects have methods that operate with files (that are proprietary) and run some queries (like if did it by this external app GUI). These objects are destroyed "by me" using Marshal.ReleaseComObject(A_OBJECT) while others run in a diferent application domain, by using AppDomain.CreateDomain("A_DOMAIN"), do the operations and call an AppDomain.Unload("A_DOMAIN"), releasing the DLLs used for the operation...
These workarounds are made to ensure that this external application doesn't "block" files used in these operations, therefore allowing deletion or moving them from a folder.
e.g.
private static ClientClass objApp = new ClientClass();
public bool ImportDelimitedFile(
string fileToImport,
string outputFile,
string rdfFile)
{
GENERICIMPORTLib import = new GENERICIMPORTLibClass();
try
{
import.ImportDelimFile(fileToImport, outputFile, 0, "", rdfFile, 0);
return true;
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
return false;
}
finally
{
System.Runtime.InteropServices.Marshal.ReleaseComObject(import);
import = null;
}
}
public int DbNumRecs(string file)
{
if (!File.Exists(file))
{
return -1;
}
System.AppDomain newDomain = System.AppDomain.CreateDomain();
COMMONIDEACONTROLSLib db = new COMMONIDEACONTROLSLibClass();
try
{
db = objApp.OpenDatabase(file);
int count = (int)db.Count;
db.Close();
objApp.CloseDatabase(file);
return count;
}
catch (Exception ex)
{
return -1;
}
finally
{
System.AppDomain.Unload(newDomain);
GC.Collect();
GC.WaitForPendingFinalizers();
}
}
Both of these "solutions" were reached by trial and error, due to the fact I do not possess any kind of API manual. Are these solutions correct? Can you explain me the differences? Do I really need to work with both solutions or one should suffice?
Thanks!
Your use of AppDomains is wrong. Just because you create a new AppDomain before line X doesn't mean that line X is actually executing in that AppDomain.
You need to marshall a proxy class back across your AppDomain and use it in the current one.
public sealed class DatabaseProxy : MarshallByRefObject
{
public int NumberOfRecords()
{
COMMONIDEACONTROLSLib db = new COMMONIDEACONTROLSLibClass();
try
{
db = objApp.OpenDatabase(file);
int count = (int)db.Count;
db.Close();
objApp.CloseDatabase(file);
return count;
}
catch (Exception ex)
{
return -1;
}
}
}
and
public int NumberOfRecords()
{
System.AppDomain newDomain = null;
try
{
newDomain = System.AppDomain.CreateDomain();
var proxy = newDomain.CreateInstanceAndUnwrap(
typeof(DatabaseProxy).Assembly.FullName,
typeof(DatabaseProxy).FullName);
return proxy.NumberOfRecords();
}
finally
{
System.AppDomain.Unload(newDomain);
}
}
You can actually create an marshall back the COM object itself instead of instantiating it via your proxy. This code is completely written here and not tested, so may be buggy.
The first solution is the best one. Unmanaged COM uses a reference-counting scheme; IUnknown is the underlying reference-counting interface: http://msdn.microsoft.com/en-us/library/ms680509(VS.85).aspx. When the reference count reaches zero, it is freed.
When you create a COM object in .NET, a wrapper is created around the COM object. The wrapper maintains a pointer to the underlying IUnknown. When garbage collection occurs, the wrapper will call the underlying IUnknown::Release() function to free the COM object during finalization. As you noticed, the problem is that sometimes the COM object locks certain critical resources. By calling Marshal.ReleaseComObject, you force an immediate call to IUnknown::Release without needing to wait (or initiate) a general garbage collection. If no other references to the COM object are held, then it will immediately be freed. Of course, the .NET wrapper becomes invalid after this point.
The second solution apparently works because of the call to GC.Collect(). The solution is more clumsy, slower, and less reliable (the COM object might not necessarily be garbage collected: behavior is dependent on the specific .NET Framework version). The use of AppDomain contributes nothing as your code doesn't actually do anything apart from creating an empty domain and then unloading it. AppDomains are useful for isolating loaded .NET Framework assemblies. Because unmanaged COM code is involved, AppDomains won't really be useful (if you need isolation, use process isolation). The second function can probably be rewritten as:
public int DbNumRecs(string file) {
if (!File.Exists(file)) {
return -1;
}
// don't need to use AppDomain
COMMONIDEACONTROLSLib db = null; // don't need to initialize class here
try {
db = objApp.OpenDatabase(file);
return (int)db.Count;
} catch (Exception) } // don't need to declare unused ex variable
return -1;
} finally {
try {
if (db != null) {
db.Close();
Marshal.ReleaseComObject(db);
}
objApp.CloseDatabase(file); // is this line really needed?
} catch (Exception) {} // silently ignore exceptions when closing
}
}