I have a C# COM DLL that calls WMI using the System.Management namespace. The DLL is being loaded into a C++ service. Every time I call the into the WMI classes I'm seeing a HUGE memory leak. After about an hour I am well over 1 GB of memory used.
If I take the same COM DLL and load it into PowerShell using Reflection.LoadFrom it does not leak memory. I have modified the DLL like so and it no longer leaks (still loading into the service with COM):
public class MyComObject
{
public void CallCom()
{
CallSomeWMIStuff();
}
}
To this. This no longer leaks!
public class MyComObject
{
public void CallCom()
{
//CallSomeWMIStuff();
}
}
Here's an example of some of the WMI code:
var scope = new ManagementScope( "root\\cimv2" );
scope.Connect();
using (var myservice = GetService("SomeService", scope))
{
//Some Stuff
}
...
ManagementObject GetService(string serviceName, MangementScope scope)
{
ManagementPath wmiPath = new ManagementPath( serviceName );
using (ManagementClass serviceClass = new ManagementClass( scope, wmiPath, null ))
{
using (ManagementObjectCollection services = serviceClass.GetInstances())
{
ManagementObject serviceObject = null;
// If this service class does not have an instance, create one.
if (services.Count == 0)
{
serviceObject = serviceClass.CreateInstance();
}
else
{
foreach (ManagementObject service in services)
{
serviceObject = service;
break;
}
}
return serviceObject;
}
}
}
EDIT: C++ Snippet:
NAMESPACE::ICSharpComPtr pCSharpCom = NULL;
HRESULT hr = pCSharpCom .CreateInstance(NAMESPACE::CLSID_CSharpCom);
if (FAILED(hr))
{
Log("Failed (hr=%08x)", hr);
return hr;
}
try
{
_bstr_t bstrData = pCSharpCom ->GetData();
strLine = (LPCTSTR)bstrData;
strMessage += strLine;
}
catch (_com_error& err)
{
_bstr_t desc = GetErrorMessage(err);
Log("Excepton %S", (const wchar_t*)desc);
return 0;
}
pCSharpCom ->Release();
Has anyone seen anything like this? We are seeing a similar issue with C++\CLI that's loading a different WMI related DLL directly.
Eventually, the WMI service will no longer be responsive and I will have to restart that service as well.
Edit:
This has to do with the apartment state of the COM object. Added a CoInitializeEx rather than a CoInitialize. I set the thread to MTA. At first it didn't look like this was working until I realized that first time the method was called we were seeing the thread state set to STA rather than MTA! Every subsequent call would be MTA. If I returned right away, before calling the System.Management classes when the thread was STA, I would no longer leak memory!
Any idea why the first one would be STA?
There is not a dispose in the RCW implementation so you are at the mercy of the GC to release the com objects you have created by default. However, you can try using Marshal.FinalReleaseComObject on the RCW instance once you are done with your COM objects. The will force the ref count to zero on the wrapped COM object and it should release. However, this also makes the RCW instance useless so be careful where you call it.
The problem had to do with the apartment state of the thread creating the COM object. There was one thread that created the COM object as MTA and another thread that was creating the COM object as STA. The STA thread was created first and then led to issues with the MTA threads. This caused the finalizer to block on GetToSTA.
Related
I noticed that ManagementObject is IDisposable, but it's also returned from ManagementClass.GetInstances() and ManagementObjectSearcher.Get(), does this mean I need to dispose each object encountered?
Like so:
ManagementObject ret;
foreach(ManagementObject mo in searcher.Get()) {
if( IsWhatIWant(mo) ) ret = mo;
else mo.Dispose();
}
Further confounding this: there's a bug in ManagementBaseObject where it does not correctly implement IDisposable (See Using clause fails to call Dispose? ) so you need to call it yourself, or use a wrapper around it that does correctly call it.
This is irritating because I have so many ManagementObjectCollections around.
I've created a helper object to dispose all created management objects:
public class ManagementObjectDisposer : IDisposable
{
private List<IDisposable> disposables = new List<IDisposable>();
/// <summary>
/// Workaround to dispose ManagementBaseObject properly.
/// See http://stackoverflow.com/questions/11896282
/// </summary>
/// <param name="disposable"></param>
public static void DisposeOne(IDisposable disposable)
{
ManagementBaseObject mbo = disposable as ManagementBaseObject;
if (mbo != null)
mbo.Dispose();
else
disposable.Dispose();
}
public void Dispose()
{
Exception firstException = null;
foreach (IDisposable d in Enumerable.Reverse(disposables))
{
try
{
DisposeOne(d);
}
catch (Exception ex)
{
if (firstException == null)
firstException = ex;
else
cvtLogger.GetLogger(this).Error($"Swallowing exception when disposing: {d.GetType()}", ex);
}
}
disposables.Clear();
if (firstException != null)
throw firstException;
}
public T Add<T>(T disposable) where T : IDisposable
{
disposables.Add(disposable);
return disposable;
}
/// <summary>
/// Helper for ManagementObjectSearcher with adding all objects to the disposables.
/// </summary>
/// <param name="query">The query string.</param>
public IEnumerable<ManagementBaseObject> Search(string query)
{
ManagementObjectSearcher searcher = this.Add(new ManagementObjectSearcher(query));
return EnumerateCollection(searcher.Get());
}
/// <summary>
/// Helper for adding ManagementObjectCollection and enumerating it.
/// </summary>
public IEnumerable<ManagementBaseObject> EnumerateCollection(ManagementObjectCollection collection)
{
this.Add(collection);
ManagementObjectCollection.ManagementObjectEnumerator enumerator = this.Add(collection.GetEnumerator());
while (enumerator.MoveNext())
yield return this.Add(enumerator.Current);
}
}
Just use it like:
using (var moDisposer = new ManagementObjectDisposer())
{
foreach (var mobj = moDisposer.Search("SELECT * FROM Win32_Processor")
Console.WriteLine(mobj["DeviceID"]);
}
Note: the ManagementClass.GetInstances() is easy to add to the ManagementObjectDisposer, too.
This is irritating because I have so many ManagementObjectCollections around.
Which has nothing to do with calling Dispose(), that only releases the underlying unmanaged COM objects. ManagementObjectCollection is a managed class, instances of it are garbage collected. Which is automatic, you can only help by calling GC.Collect(). Your program is probably just creating a lot of System.Management objects, possibly because that's the only thing it ever does. The quoted bug was fixed in the current versions of .NET 3.5SP1 and .NET 4.5 that I have installed on my machine.
So if you don't have a patched version of .NET then you don't only see a lot of System.Management objects on the GC heap, your process will also consume a lot of unmanaged memory. If the garbage collector doesn't run frequently enough then that can cause the program to crash with OOM. You didn't mention that as a failure mode so it is not strongly indicated that you have a real problem.
The initial size of generation 0 of the GC heap is 2 megabytes, it can grow to 8+ megabytes. That is a lot of ManagementObjectCollections objects, it is a very small object that takes only 24 bytes in 32-bit mode. The actual collection is unmanaged. Use Perfmon.exe or your memory profiler to check that the garbage collector is running frequently enough. If it doesn't then keep an eye on your program's VM size. If that's ballooning then using a counter in your query loop and calling GC.Collect() when it is high enough is a viable workaround. Careful with the info you get out of a memory profiler, it can irritate for the wrong reasons.
Actually the code from:
http://referencesource.microsoft.com/#System.Management/managementobjectcollection.cs
and also the Symbols from the Microsoft Symbol Server
http://msdl.microsoft.com/download/symbols
Imply ManagementObjectCollection is IDisposable which implies it is for some reason using unmanaged resources or it is incorrectly using the IDisposable interface.
If you do a lot of WMI queries you should always dispose the return ManagementObjectCollection to avoid Quota violation exceptions.
System.Management.ManagementException: Quota violation
at System.Management.ManagementException.ThrowWithExtendedInfo(ManagementStatus errorCode)
at System.Management.ManagementObjectCollection.ManagementObjectEnumerator.MoveNext()
After two weeks of looking for a memory leak while working with WMI, the following solution helped:
using (ManagementObjectSearcher searcher = new ManagementObjectSearcher(scope, objectQuery, enumerationOptions))
using (ManagementObjectCollection collection = searcher.Get())
{
foreach (ManagementBaseObject obj in collection)
{
using (obj) // calls Component.Dispose()
{
try
{
// Your code
}
finally
{
obj.Dispose(); // calls ManagementBaseObject.Dispose()
}
}
}
}
The reason is in ManagementBaseObject.Dispose(), which is marked as new, so using doesn't call it.
So ManagementBaseObject.Dispose() was called by the finalizer (GC). With a huge amount of ManagementBaseObject (thousands per second) in a highly loaded system after a couple of days, the duration of the finalizers became more than an hour, which caused memory leak, OutOfMemoryException and application termination.
This issue is still present in .Net Framework 4.8.
Theoretically, you can remove using for ManagementBaseObject, because ManagementBaseObject.Dispose() calls the base Dispose(), but who knows what surprises are in other versions of .Net
Also, the Dispose method is marked as new in the inherited classes: ManagementObject and ManagementClass.
I am using the latest version of LuaInterface (http://code.google.com/p/luainterface/) in a C# application. I'm running into a problem where the Lua class is failing to clean up internal references in the ObjectTranslator 'objects' and 'objectsBackMap' dictionaries, resulting in always-growing memory usage.
The following code illustrates the problem:
public class Program
{
public static void Main()
{
Lua lua = new Lua();
string f = #"
return function(myClass, dotNetObject)
local f = function() dotNetObject:TestMethod() end
myClass:StoreFunction(f);
end";
var result = lua.DoString(f)[0] as LuaFunction;
MyClass testItem = new MyClass();
for (int i = 0; i < 50; i++)
{
DotNetObject o = new DotNetObject();
result.Call(testItem, o);
}
lua.DoString("collectgarbage()");
ObjectTranslator translator = (ObjectTranslator)typeof(Lua).GetField("translator", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance).GetValue(lua);
Console.WriteLine("Object count: " + translator.objects.Count);
//Prints out 51. 50 of these items are instances of 'DotNetObject', despite them being unreachable.
//Forcing a .NET garbage collection does not help. They are still being held onto by the object translator.
}
public class MyClass
{
public void StoreFunction(LuaFunction function)
{
//I would normally save off the function here to be executed later
}
}
public class DotNetObject
{
public void TestMethod()
{
}
}
}
The problem arises when the anonymous function (local f = ...) creates a closure involving a .NET object from the outer scope. As long as the Lua interpreter stays alive, 50 instances of the DotNetObject class I've created will never be garbage collected, even when forcing a GC in Lua.
Manually disposing of the LuaFunction (function.Dispose()) object in MyClass.StoreFunction solves the problem, but this is not desirable because in my real application I don't know when the Function will execute--or if it ever will. Forcing me to dispose of the LuaFunction changes the entire architecture of the application such that I'm basically doing manual memory management by disposing the object that contains the LuaFunction, and the object that contains that, all the way up the chain.
So, is this a bug in LuaInterface, or am I using the library incorrectly? Any advice is greatly appreciated, thanks!
I have a C# program that accesses a COM Interface to a piece of simulation software called Aspen Plus. I have a very strange memory leak.
When I need to get the result values out of the simulation, I run a series of calls like this, in some cases the variable returned might be null, so I insert a check for that. Then I use FinalReleaseComObject to clean up the COM references.
public override ValueType recvValueFromSim<ValueType>(string path) {
Happ.IHNode tree = this.Aspen.Tree;
dynamic node = tree.FindNode(path);
ValueType retVal = default(ValueType);
if (node != null && node.Value != null) {
retVal = node.Value;
}
Marshal.FinalReleaseComObject(node);
Marshal.FinalReleaseComObject(tree);
node = null;
return retVal;
}
Unfortunately, the above code leaks a lot. It leaks 2MB per simulation. At first I thought the Garbage Collector would eventually run and clean it up, but no dice. After running a couple of hundred simulations, I ran out of memory.
The bizarre thing is, the below code works fine and doesn't leak. I didn't like it, because using catch to check for null references seems like bad form, but this doesn't leak.
public override ValueType recvValueFromSim<ValueType>(string path) {
ValueType node;
try {
node = this.Aspen.Tree.FindNode(path).Value;
return node;
} catch {
return default(ValueType);
}
}
Why doesn't it leak? Does anybody know? The belies why I thought I knew about temporary references and releasing COM objects.
I've been developing a C# application that uses DLL interop to an external database application.
This external app starts up at the same time along with my C# app and is available as long as my C# app is running.
Now the real question is related to managing the objects that I need to create to interact with the external application.
When I declare objects that are available from the referenced DLL's these objects have methods that operate with files (that are proprietary) and run some queries (like if did it by this external app GUI). These objects are destroyed "by me" using Marshal.ReleaseComObject(A_OBJECT) while others run in a diferent application domain, by using AppDomain.CreateDomain("A_DOMAIN"), do the operations and call an AppDomain.Unload("A_DOMAIN"), releasing the DLLs used for the operation...
These workarounds are made to ensure that this external application doesn't "block" files used in these operations, therefore allowing deletion or moving them from a folder.
e.g.
private static ClientClass objApp = new ClientClass();
public bool ImportDelimitedFile(
string fileToImport,
string outputFile,
string rdfFile)
{
GENERICIMPORTLib import = new GENERICIMPORTLibClass();
try
{
import.ImportDelimFile(fileToImport, outputFile, 0, "", rdfFile, 0);
return true;
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
return false;
}
finally
{
System.Runtime.InteropServices.Marshal.ReleaseComObject(import);
import = null;
}
}
public int DbNumRecs(string file)
{
if (!File.Exists(file))
{
return -1;
}
System.AppDomain newDomain = System.AppDomain.CreateDomain();
COMMONIDEACONTROLSLib db = new COMMONIDEACONTROLSLibClass();
try
{
db = objApp.OpenDatabase(file);
int count = (int)db.Count;
db.Close();
objApp.CloseDatabase(file);
return count;
}
catch (Exception ex)
{
return -1;
}
finally
{
System.AppDomain.Unload(newDomain);
GC.Collect();
GC.WaitForPendingFinalizers();
}
}
Both of these "solutions" were reached by trial and error, due to the fact I do not possess any kind of API manual. Are these solutions correct? Can you explain me the differences? Do I really need to work with both solutions or one should suffice?
Thanks!
Your use of AppDomains is wrong. Just because you create a new AppDomain before line X doesn't mean that line X is actually executing in that AppDomain.
You need to marshall a proxy class back across your AppDomain and use it in the current one.
public sealed class DatabaseProxy : MarshallByRefObject
{
public int NumberOfRecords()
{
COMMONIDEACONTROLSLib db = new COMMONIDEACONTROLSLibClass();
try
{
db = objApp.OpenDatabase(file);
int count = (int)db.Count;
db.Close();
objApp.CloseDatabase(file);
return count;
}
catch (Exception ex)
{
return -1;
}
}
}
and
public int NumberOfRecords()
{
System.AppDomain newDomain = null;
try
{
newDomain = System.AppDomain.CreateDomain();
var proxy = newDomain.CreateInstanceAndUnwrap(
typeof(DatabaseProxy).Assembly.FullName,
typeof(DatabaseProxy).FullName);
return proxy.NumberOfRecords();
}
finally
{
System.AppDomain.Unload(newDomain);
}
}
You can actually create an marshall back the COM object itself instead of instantiating it via your proxy. This code is completely written here and not tested, so may be buggy.
The first solution is the best one. Unmanaged COM uses a reference-counting scheme; IUnknown is the underlying reference-counting interface: http://msdn.microsoft.com/en-us/library/ms680509(VS.85).aspx. When the reference count reaches zero, it is freed.
When you create a COM object in .NET, a wrapper is created around the COM object. The wrapper maintains a pointer to the underlying IUnknown. When garbage collection occurs, the wrapper will call the underlying IUnknown::Release() function to free the COM object during finalization. As you noticed, the problem is that sometimes the COM object locks certain critical resources. By calling Marshal.ReleaseComObject, you force an immediate call to IUnknown::Release without needing to wait (or initiate) a general garbage collection. If no other references to the COM object are held, then it will immediately be freed. Of course, the .NET wrapper becomes invalid after this point.
The second solution apparently works because of the call to GC.Collect(). The solution is more clumsy, slower, and less reliable (the COM object might not necessarily be garbage collected: behavior is dependent on the specific .NET Framework version). The use of AppDomain contributes nothing as your code doesn't actually do anything apart from creating an empty domain and then unloading it. AppDomains are useful for isolating loaded .NET Framework assemblies. Because unmanaged COM code is involved, AppDomains won't really be useful (if you need isolation, use process isolation). The second function can probably be rewritten as:
public int DbNumRecs(string file) {
if (!File.Exists(file)) {
return -1;
}
// don't need to use AppDomain
COMMONIDEACONTROLSLib db = null; // don't need to initialize class here
try {
db = objApp.OpenDatabase(file);
return (int)db.Count;
} catch (Exception) } // don't need to declare unused ex variable
return -1;
} finally {
try {
if (db != null) {
db.Close();
Marshal.ReleaseComObject(db);
}
objApp.CloseDatabase(file); // is this line really needed?
} catch (Exception) {} // silently ignore exceptions when closing
}
}
I have a class that instantiates a COM exe out of process. The class is
public class MyComObject:IDisposable
{
private bool disposed = false;
MyMath test;
public MyComObject()
{
test = new MyMath();
}
~MyComObject()
{
Dispose(false);
}
public double GetRandomID()
{
if (test != null)
return test.RandomID();
else
return -1;
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
private void Dispose(bool disposing)
{
if (test != null)
{
Marshal.ReleaseComObject(test);
test = null;
}
disposed = true;
}
}
and I call it as follows
static void Main(string[] args)
{
MyComObject test = new MyComObject();
MyComObject test2 = new MyComObject();
//Do stuff
test.Dispose();
test2.Dispose();
Console.ReadLine();
}
now, this cleans up my COM object when the program executes normally. However, if I close the program in the middle of its execution, the framework doesn't ever call the code that releases my unmanaged object. Which is fair enough. However, is there a way to force the program to clean itself up even though its been killed?
EDIT: it doesn't look promising from a hard kill from the taskmanager :(
Wrapping it in a try finally or using clause will get you most of the way there:
using (MyComObject test = new MyComObject())
using (MyComObject test2 = new MyComObject()) {
// do stuff
}
Console.ReadLine();
The specifics of how your app is being shutdown, though, will dictate any other measures you can do (like using CriticalFinalizerObject).
I think that a console app that gets closed (from the little x) is the same as a Ctrl-C - in which case you can subscribe to Console.CancelKeyPress for that.
Edit: You should also ReleaseComObject until it returns <= 0.
Well, one best practice is to use using statements:
using (MyComObject test = new MyComObject())
using (MyComObject test2 = new MyComObject())
{
//Do stuff
}
That essentially puts finally blocks in to call dispose automatically at the end of the scope. You should have using statements pretty much whenever you have an instance of IDisposable that you take responsibility for cleaning up. However, it doesn't fix the situation where your whole process is aborted abruptly. That's pretty rare though, and you may not want to worry about it. (It's pretty hard to get round it.) I'd have expected your finalizer to be called with your previous code though..