I want to test for object references held improperly and wrote a test that always failed. I simplified the test to the following behaviour:
[Test]
public void ScopesAreNotLeaking()
{
WeakReference weakRef;
Stub scope = null;
using (scope = new Stub())
{
weakRef = new WeakReference(scope);
}
scope = null;
GC.Collect();
GC.WaitForPendingFinalizers();
Assert.That(weakRef.Target, Is.Null);
}
This test however, which does the same without using, passes:
[Test]
public void ScopesAreNotLeaking()
{
WeakReference weakRef;
Stub scope = new Stub();
weakRef = new WeakReference(scope);
scope = null;
GC.Collect();
GC.WaitForPendingFinalizers();
Assert.That(weakRef.Target, Is.Null);
}
The used stub class is simple enough:
class Stub : IDisposable
{
public void Dispose() {}
}
Can someone please explain me that behaviour or - even better - has an idea how to make sure that the object gets garbage collected?
PS: Bear with me if a similar question was asked before. I only examined those questions that have using in the title.
using is not designed to force garbage collection but to ensure dispose is called. Dispose allows you to release non-garbage collected resources like file handles. Garbage collection happens when c# is good and ready.
I suspect there may be a local introduced by the using statement. Use ildasm to see if all the references in the function to the object are truly cleared before the call to GC.Collect. Also try to put the using bit in a separate function that returns the weak reference.
Your two test cases are not identical. At the end of a using statement, the resource's Dispose method is called, it is not set to null. Calling Dispose does not necessarily call the destructor.
The mark & sweep GC, like it is used in .NET, is not deterministic.
Even if called by GC.Collect() there is no guarantee that is really runs.
Additionally, the using-clause does not have anything to do with garbage collection. If just calls Dispose() on its target object.
Related
I have a test that I expected to pass but the behavior of the Garbage Collector is not as I presumed:
[Test]
public void WeakReferenceTest2()
{
var obj = new object();
var wRef = new WeakReference(obj);
wRef.IsAlive.Should().BeTrue(); //passes
GC.Collect();
wRef.IsAlive.Should().BeTrue(); //passes
obj = null;
GC.Collect();
wRef.IsAlive.Should().BeFalse(); //fails
}
In this example the obj object should be GC'd and therefore I would expect the WeakReference.IsAlive property to return false.
It seems that because the obj variable was declared in the same scope as the GC.Collect it is not being collected. If I move the obj declaration and initialization outside of the method the test passes.
Does anyone have any technical reference documentation or explanation for this behavior?
Hit the same issue as you - my test was passing everywhere, except for under NCrunch (could be any other instrumentation in your case). Hm. Debugging with SOS revealed additional roots held on a call stack of a test method. My guess is that they were a result of code instrumentation that disabled any compiler optimizations, including those that correctly compute object reachability.
The cure here is quite simple - don't ever hold strong references from a method that does GC and tests for aliveness. This can be easily achieved with a trivial helper method. The change below made your test case pass with NCrunch, where it was originally failing.
[TestMethod]
public void WeakReferenceTest2()
{
var wRef2 = CallInItsOwnScope(() =>
{
var obj = new object();
var wRef = new WeakReference(obj);
wRef.IsAlive.Should().BeTrue(); //passes
GC.Collect();
wRef.IsAlive.Should().BeTrue(); //passes
return wRef;
});
GC.Collect();
wRef2.IsAlive.Should().BeFalse(); //used to fail, now passes
}
private T CallInItsOwnScope<T>(Func<T> getter)
{
return getter();
}
There are a few potential issues I can see:
I am unaware of anything in the C# specification which requires that the lifetimes of local variables be limited. In a non-debug build, I think the compiler would be free to omit the last assignment to obj (setting it to null) since no code path would cause the value of obj will never be used after it, but I would expect that in a non-debug build the metadata would indicate that the variable is never used after the creation of the weak reference. In a debug build, the variable should exist throughout the function scope, but the obj = null; statement should actually clear it. Nonetheless, I'm not certain that the C# spec promises that the compiler won't omit the last statement and yet still keep the variable around.
If you are using a concurrent garbage collector, it would may be that GC.Collect() triggers the immediate start of a collection, but that the collection wouldn't actually be completed before GC.Collect() returns. In this scenario, it may not be necessary to wait for all finalizers to run, and thus GC.WaitForPendingFinalizers() may be overkill, but it would probably solve the problem.
When using the standard garbage collector, I would not expect the existence of a weak reference to an object to prolong the existence of the object in the way that a finalizer would, but when using a concurrent garbage collector, it's possible that abandoned objects to which a weak reference exists get moved to a queue of objects with weak references that need to be cleaned up, and that the processing of such cleanup happens on a separate thread that runs concurrently with everything else. In such case, a call to GC.WaitForPendingFinalizers() would be necessary to achieve the desired behavior.
Note that one should generally not expect that weak references will be invalidated with any particular degree of timeliness, nor should one expect that fetching Target after IsAlive reports true will yield a non-null reference. One should use IsAlive only in cases where one wouldn't care about the target if it's still alive, but would be interested in knowing that the reference has died. For example, if one has a collection of WeakReference objects, one may wish to periodically iterate through the list and remove WeakReference objects whose target has died. One should be prepared for the possibility that WeakReferences might remain in the collection longer than would be ideally necessary; the only consequence if they do so should be a slight waste of memory and CPU time.
As far as I know, calling Collect does not guarantee that all resources are released. You are merely making a suggestion to the garbage collector.
You could try to force it to block until all objects are released by doing this:
GC.Collect(2, GCCollectionMode.Forced, true);
I expect that this might not work absolutely 100% of the time. In general, I would avoid writing any code that depends on observing the garbage collector, it is not really designed to be used in this way.
Could it be that the .Should() extension method is somehow hanging on to a reference? Or perhaps some other aspect of the test framework is causing this issue.
(I'm posting this as an answer otherwise I can't easily post the code!)
I have tried the following code, and it works as expected (Visual Studio 2012, .Net 4 build, debug and release, 32 bit and 64 bit, running on Windows 7, quad core processor):
using System;
namespace Demo
{
internal class Program
{
private static void Main(string[] args)
{
var obj = new object();
var wRef = new WeakReference(obj);
GC.Collect();
obj = null;
GC.Collect();
Console.WriteLine(wRef.IsAlive); // Prints false.
Console.ReadKey();
}
}
}
What happens when you try this code?
This answer is not related to unit tests, but it might be helpful to somebody who's testing out weak references and wondering why they don't work as expected.
The issue is basically the JIT keeping the variables alive. This can be avoided by instantiating the WeakReference and the target object in a non-inlined method:
private static MyClass _myObject = new MyClass();
static void Main(string[] args)
{
WeakReference<object> wr = CreateWeakReference();
_myObject = null;
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
wr.TryGetTarget(out object targetObject);
Console.WriteLine(targetObject == null ? "NULL" : "It's alive!");
}
[MethodImpl(MethodImplOptions.NoInlining)]
private static WeakReference<object> CreateWeakReference()
{
_myObject = new MyClass();
return new WeakReference<object>(_myObject);
}
public class MyClass
{
}
Commenting out _myObject = null; will prevent garbage collection of that object.
I have a feeling that you need to call GC.WaitForPendingFinalizers() as I expect that week references are updated by the finalizers thread.
I had issues with the many years ago when writing a unit test and recall that WaitForPendingFinalizers() helped, so did making to calls to GC.Collect().
The software never leaked in real life, but writing a unit test to prove that the object was not kept alive was a lot harder then I hoped. (We had bugs in the past with our cache that did keep it alive.)
What is an alternate/better way I can safely dispose of the variables purge and factory in the following code?
public void Run( string XmlFragment ) {
XmlNode xmlNode = null;
try
{
xmlNode = Common.ConstructXmlNodeFromString(XmlFragment, "Params");
var list = DataList();
foreach (var item in list)
{
var factory = new PurgerFactory(item);
IPurger purge = factory.Purger;
purge.Purge();
purge = null;
factory = null;
}
Common.PurgeEmail(SuccessEmail());
}
catch (Exception ex)
{
string errorMessage = $"Purge Error: {ex.Message}. {Environment.NewLine} Stack Trace: {ex.StackTrace}";
Common.PurgeEmail(FailEmail(errorMessage));
}
}
As I think you know, C# has a garbage collector. So for normal objects that don't access an unmanaged resource, just letting the garbage collector clean them is fine.
If you want to deterministically close a managed resource the primary paradigm is inheriting from IDisposable and a using statement. This will call the Dispose function upon exiting the code block of the using statement.
If you otherwise want to clean stuff up, but you don't care when it happens you can use ~(MyType). This is called when the GC does a GC cycle, this is called the Finalizer. I personally haven't encountered a use-case for this vs IDisposable. But if you just want to make sure say a file, or otherwise is deleted when this email object is Garbage Collected, then it might be a good use case for you.
Your code doesn't indicate whether purge or factory implement IDisposable. If they do, you would call
purge.Dispose();
factory.Dispose();
or use using which ensures that Dispose is called even if the method throws an exception:
using(var factory = new PurgerFactory(item))
{
// do stuff
} // factory is disposed.
What we don't need to do is set variables to null. An object gets garbage collected sometime after there are no longer any references to it. It's true that setting factory = null removes that reference to the object, but the same thing happens anyway when the variable goes out of scope. So it's redundant.
There's really no need to try to force that to happen sooner. It's not like the object will get garbage collected the instant there are no more references to it, so we don't worry so much about that.
In this case, the variables are both declared within the foreach loop, so the references go out of scope immediately after each iteration of the loop. In other words, they already go out of scope right at the point when you're setting them to null.
That's part of what's awesome about .NET. Imagine if we had to set every reference variable to null. We'd need try/catch everywhere to make sure it happened. It would be as if every single object was disposable. What a nightmare. Instead it's handled for us in most normal scenarios so that we don't have to think about it or clutter our code with it.
How could be defined a code that store 'this' during class finalization? How the garbage collector should behave (if defined somewhere)?
In my mind the GC should finalize multiple times the class instance, and the following test application shall print "66", but the finalizer is executed only once, causing the application to print "6".
Few lines of code:
using System;
namespace Test
{
class Finalized
{
~Finalized()
{
Program.mFinalized = this;
}
public int X = 5;
}
class Program
{
public static Finalized mFinalized = null;
static void Main(string[] args)
{
Finalized asd = new Finalized();
asd.X = 6;
asd = null;
GC.Collect();
if (mFinalized != null)
Console.Write("{0}", mFinalized.X);
mFinalized = null;
GC.Collect();
if (mFinalized != null)
Console.Write("{0}", mFinalized.X);
}
}
}
What I'm trying to do is to understand how finalizers manage instance memory. In my application could be desiderable to re-use instance reference again for further processing.
It's clear that the finalizer doesn't "free" memory (at least in my test application). May the memory chunk be reused for other purposes? Or even freed? And if it isn't, that would be a memory leak or what?
Now, I'm confused more than before.
This is due to Resurrection. By storing the object in another variable during finalization (assigning this to a variable), you resurrect the obejct instance as far as the GC is concerned. You are allowed to resurrect your object in .NET, and you can actually cause the GC to finalize the object more than once, but you have to explicitly request it via GC.ReRegisterForFinalize .
For details, see Automatic Memory Management in the Microsoft .NET Framework.
GC.Collect does a sweep, special-casing any objects with a finalizer and not collecting them. Once these finalizer objects have finalized, GC then runs again over these objects. If they're no longer eligible for collection (by re-rooting, as you do), so be it. Normally the finalizer only runs once, but IIRC, you can request that it runs again.
Finalizer only gets called once. You're free to assign self to somewhere, and prevent the object being garbage collected. But once the object is available again for GC, it doesn't run the finalizer.
I'm interested in any good uses of resurrected objects.
The MSDN states "There are very few good uses of resurrection, and you really should avoid it if possible".
Also Bill Wagner in his Effective C# says "You cannot make this kind of construct work reliably. Dont try". But the book is 2 years old so maybe something changed?
I have this class:
class Foo : IDisposable
{
SomeBigResource resource;
void UsingResource()
{
using(Bar bar = new Bar(SomeBigResource)
bar.doStuff();
}
void Dispose()
{
resource.Dispose();
}
}
void Function()
{
using (Foo foo = new Foo(new SomeBigResource))
foo.UsingResource();
}
The Bar object has exactly the same Dispose() function.
Will my SomeBigResource be released or is the GC smart enough to release it later on when the second using is done?
If both the Dispose methods in Foo and Bar calls Dispose on the SomeBigResource object, the method will be called twice. If the method is implemented correctly it will release the resources the first time and do nothing the second time.
What you have is a confusion of responsibility, where both objects take responsibility for calling Dispose on the SomeBigResource object. This is something that you want to avoid, as one object can't know if the other object still needs the resource, so you want to put the responsibility in one place only.
You should either make the Foo object responsible for the life cycle of the resource, or handle it outside of the objects completely. The latter makes more sense, as that's where you create the SomeBigResource instance:
using (SomeBigResource resource = new SomeBigResource()) {
using (Foo foo = new Foo(resource)) {
foo.UsingResource();
}
}
Your resource will be disposed twice.
The GC is completely unaware of IDisposables and the using statement.
Each using statement translates into a try / finally block with a Dispose() call in the finally block.
This is a normal method call which will always execute.
Therefore, the inner using statement in UsingResource() will dispose the resource, and then the outer using statement in Function() will dispose it again.
All IDisposable implementations should be idempotent (calling it a second time should do no harm).
Therefore, (assuming that SomeBigResource implements IDisposable correctly), your code should work fine.
Note, though, that with your class, an empty using block will throw a NullReferenceException, which is very wrong.
You should add a null check in your Dispose method.
This depends on the implementation of Bar.Dispose(). Assuming it has a similar implementation as Foo, then, yes, the release will occur early.
Take a look at the Dispose pattern guidance and this SO question. There are more corner cases than you may think of immediately.
I assume this line should read:
using(Bar bar = new Bar(resource))
If that's the case, then when bar is disposed, it should dispose of resource.
So, yes. When bar is disposed in UsingResource, it will dispose resource, so that if you attempt to use it later, it should throw an ObjectDisposedException. (Although in your simple example, that shouldn't happen.)
I've got a C# class with a Dispose function via IDisposable. It's intended to be used inside a using block so the expensive resource it handles can be released right away.
The problem is that a bug occurred when an exception was thrown before Dispose was called, and the programmer neglected to use using or finally.
In C++, I never had to worry about this. The call to a class's destructor would be automatically inserted at the end of the object's scope. The only way to avoid that happening would be to use the new operator and hold the object behind a pointer, but that required extra work for the programmer isn't something they would do by accident, like forgetting to use using.
Is there any way to for a using block to be automatically used in C#?
Many thanks.
UPDATE:
I'd like to explain why I'm not accepting the finalizer answers. Those answers are technically correct in themselves, but they are not C++ style destructors.
Here's the bug I found, reduced to the essentials...
try
{
PleaseDisposeMe a = new PleaseDisposeMe();
throw new Exception();
a.Dispose();
}
catch (Exception ex)
{
Log(ex);
}
// This next call will throw a time-out exception unless the GC
// runs a.Dispose in time.
PleaseDisposeMe b = new PleaseDisposeMe();
Using FXCop is an excellent suggestion, but if that's my only answer, my question would have to become a plea to the C# people, or use C++. Twenty nested using statements anyone?
Where I work we use the following guidelines:
Each IDisposable class must have a finalizer
Whenever using an IDisposable object, it must be used inside a "using" block. The only exception is if the object is a member of another class, in which case the containing class must be IDisposable and must call the member's 'Dispose' method in its own implementation of 'Dispose'. This means 'Dispose' should never be called by the developer except for inside another 'Dispose' method, eliminating the bug described in the question.
The code in each Finalizer must begin with a warning/error log notifying us that the finalizer has been called. This way you have an extremely good chance of spotting such bugs as described above before releasing the code, plus it might be a hint for bugs occuring in your system.
To make our lives easier, we also have a SafeDispose method in our infrastructure, which calls the the Dispose method of its argument within a try-catch block (with error logging), just in case (although Dispose methods are not supposed to throw exceptions).
See also: Chris Lyon's suggestions regarding IDisposable
Edit:
#Quarrelsome: One thing you ought to do is call GC.SuppressFinalize inside 'Dispose', so that if the object was disposed, it wouldn't be "re-disposed".
It is also usually advisable to hold a flag indicating whether the object has already been disposed or not. The follwoing pattern is usually pretty good:
class MyDisposable: IDisposable {
public void Dispose() {
lock(this) {
if (disposed) {
return;
}
disposed = true;
}
GC.SuppressFinalize(this);
// Do actual disposing here ...
}
private bool disposed = false;
}
Of course, locking is not always necessary, but if you're not sure if your class would be used in a multi-threaded environment or not, it is advisable to keep it.
Unfortunately there isn't any way to do this directly in the code. If this is an issue in house, there are various code analysis solutions that could catch these sort of problems. Have you looked into FxCop? I think that this will catch these situations and in all cases where IDisposable objects might be left hanging. If it is a component that people are using outside of your organization and you can't require FxCop, then documentation is really your only recourse :).
Edit: In the case of finalizers, this doesn't really guarantee when the finalization will happen. So this may be a solution for you but it depends on the situation.
#Quarrelsome
If will get called when the object is moved out of scope and is tidied by the garbage collector.
This statement is misleading and how I read it incorrect: There is absolutely no guarantee when the finalizer will be called. You are absolutely correct that billpg should implement a finalizer; however it will not be called automaticly when the object goes out of scope like he wants. Evidence, the first bullet point under Finalize operations have the following limitations.
In fact Microsoft gave a grant to Chris Sells to create an implementation of .NET that used reference counting instead of garbage collection Link. As it turned out there was a considerable performance hit.
~ClassName()
{
}
EDIT (bold):
If will get called when the object is moved out of scope and is tidied by the garbage collector however this is not deterministic and is not guaranteed to happen at any particular time.
This is called a Finalizer. All objects with a finaliser get put on a special finalise queue by the garbage collector where the finalise method is invoked on them (so it's technically a performance hit to declare empty finalisers).
The "accepted" dispose pattern as per the Framework Guidelines is as follows with unmanaged resources:
public class DisposableFinalisableClass : IDisposable
{
~DisposableFinalisableClass()
{
Dispose(false);
}
public void Dispose()
{
Dispose(true);
}
protected virtual void Dispose(bool disposing)
{
if (disposing)
{
// tidy managed resources
}
// tidy unmanaged resources
}
}
So the above means that if someone calls Dispose the unmanaged resources are tidied. However in the case of someone forgetting to call Dispose or an exception preventing Dispose from being called the unmanaged resources will still be tidied away, only slightly later on when the GC gets its grubby mitts on it (which includes the application closing down or unexpectedly ending).
The best practice is to use a finaliser in your class and always use using blocks.
There isn't really a direct equivalent though, finalisers look like C destructors, but behave differently.
You're supposed to nest using blocks, that's why the C# code layout defaults to putting them on the same line...
using (SqlConnection con = new SqlConnection("DB con str") )
using (SqlCommand com = new SqlCommand( con, "sql query") )
{
//now code is indented one level
//technically we're nested twice
}
When you're not using using you can just do what it does under the hood anyway:
PleaseDisposeMe a;
try
{
a = new PleaseDisposeMe();
throw new Exception();
}
catch (Exception ex) { Log(ex); }
finally {
//this always executes, even with the exception
a.Dispose();
}
With managed code C# is very very good at looking after its own memory, even when stuff is poorly disposed. If you're dealing with unmanaged resources a lot it's not so strong.
This is no different from a programmer forgetting to use delete in C++, except that at least here the garbage collector will still eventually catch up with it.
And you never need to use IDisposable if the only resource you're worried about is memory. The framework will handle that on it's own. IDisposable is only for unmanaged resources like database connections, filestreams, sockets, and the like.
A better design is to make this class release the expensive resource on its own, before its disposed.
For example, If its a database connection, only connect when needed and release immediately, long before the actual class gets disposed.