I have a problem with memory leaks in my .NET Windows service application. So I've started to read articles about memory management in .NET. And i have found an interesting practice in one of Jeffrey Richter articles. This practice name is "object resurrection". It looks like situating code that initializes global or static variable to "this":
protected override void Finalize() {
Application.ObjHolder = this;
GC.ReRegisterForFinalize(this);
}
I understand that this is a bad practice, however I would like to know patterns that uses this practice. If you know any, please write here.
From the same article: "There are very few good uses of resurrection, and you really should avoid it if possible."
The best use I can think of is a "recycling" pattern. Consider a Factory that produces expensive, practically immutable objects; for instance, objects instantiated by parsing a data file, or by reflecting an assembly, or deeply copying a "master" object graph. The results are unlikely to change each time you perform this expensive process. It is in your best interest to avoid instantiation from scratch; however, for some design reasons, the system must be able to create many instances (no singletons), and your consumers cannot know about the Factory so that they can "return" the object themselves; they may have the object injected, or be given a factory method delegate from which they obtain a reference. When the dependent class goes out of scope, normally the instance would as well.
A possible answer is to override Finalize(), clean up any mutable state portion of the instance, and then as long as the Factory is in scope, reattach the instance to some member of the Factory. This allows the garbage-collection process to, in effect, "recycle" the valuable portion of these objects when they would otherwise go out of scope and be totally destroyed. The Factory can look and see if it has any recycled objects available in it's "bin", and if so, it can polish it up and hand it out. The factory would only have to instantiate a new copy of the object if the number of total objects in use by the process increased.
Other possible uses may include some highly specialized logger or audit implementation, where objects you wish to process after their death will attach themselves to a work queue managed by this process. After the process handles them, they can be totally destroyed.
In general, if you want dependents to THINK they're getting rid of an object, or to not have to bother, but you want to keep the instance, resurrection may be a good tool, but you'll have to watch it VERY carefully to avoid situations in which objects receiving resurrected references become "pack rats" and keep every instance that has ever been created in memory for the lifetime of the process.
Speculative: In a Pool situation, like the ConnectionPool.
You might use it to reclaim objects that were not properly disposed but to which the application code no longer holds a reference. You can't keep them in a List in the Pool because that would block GC collection.
A brother of mine worked on a high-performance simulation platform once. He related to me how that in the application, object construction was a demonstrable bottleneck to the application performance. It would seem the objects were large and required some significant processing to initialize.
They implemented an object repository to contain "retired" object instances. Before constructing a new object they would first check to see if one already existed in the repository.
The trade-off was increased memory consumption (as there might exist many unused objects at a time) for increased performance (as the total number of object constructions were reduced).
Note that the decision to implement this pattern was based on the bottlenecks they observed through profiling in their specific scenario. I would expect this to be an exceptional circumstance.
The only place I can think of using this, potentially, would be when you were trying to cleanup a resource, and the resource cleanup failed. If it was critical to retry the cleanup process, you could, technically, "ReRegister" the object to be finalized, which hopefully would succeed, the second time.
That being said, I'd avoid this altogether in practice.
For what I know .net calls finalizers in no specific order. If your class contains references to other objects they could have been finalized (and hence Disposed) when your finalizer is called. If you then decide to resurrect your object you will have references to finalized/disposed objects.
class A {
static Set<A> resurectedA = new Set<A>();
B b = new B();
~A() {
//will not die. keep a reference in resurectedA.
resurectedA.Add(this);
GC.ReRegisterForFinalize(this);
//at this point you may have a problem. By resurrecting this you are resurrecting b and b's Finalize may have already been called.
}
}
class B : IDisposable {
//regular IDisposable/Destructor pattern http://msdn.microsoft.com/en-us/library/b1yfkh5e(v=vs.110).aspx
}
Related
There are lots of examples of Arrays or Lists of IDisposable objects being returned from functions in .NET. For example, Process.GetProcesses().
If I call that method is it my responsibility to Dispose() of all the members of the array as I iterate through them?
Why should it be my responsibility since I never created the objects and the array that I was given is just pointers to the objects which were created outside of my code.
I always thought it was the creator's burden to Dispose().
So what is the proper rule here?
There is no general rule. It's going to depend on the situation, and how the method in question is designed, as to whether or not "you" are responsible for disposing of objects you have access to. This is where documentation is often important to help users of the type understand their responsibilities.
I always thought it was the creator's burden to Dispose()
This cannot be strictly true. It is sometimes the case that a disposable object will out-live the lifetime of the block of code creating it. While it simplest when the creator can dispose of the object, sometimes it's simply impossible for them to be able to. When returning a disposable object from a method is one situation where it's often not be possible for the code creating the disposable object to clean it up, as it's lifetime needs to be smaller than the lifetime of the disposable object.
With relatively few exceptions (most of which could be described as least-of-evils approaches to dealing with poorly-designed code that can't be changed), every IDisposable instance should at any given moment in time have exactly one well-defined owner. In cases where a method returns something of a type that implements IDisposable, the contract for the method will specify whether the method is relinquishing ownership (in which case the caller should ensure that the object gets disposed--either by disposing of the object itself or relinquishing ownership to someone else), or whether the method is merely returning a reference to an object which is owned by someone else.
In properly-written code, the question of whether or not an object should be disposed is rarely a judgment call. The owner of an object should ensure that it gets disposed; nobody else should dispose it. Occasionally it may be necessary to have a method accept a parameter indicating whether the method should transfer ownership of an IDisposable. For example, if code wants to create a sound, pass it to a "start playing sound" method, and never wants to deal with that sound again, it may be most convenient to have the code to play the sound accept take and dispose the sound when it's done; if code wants to be able to play a sound repeatedly, however, and will ensure that the sound object will stay alive as long as it's needed, it would be more convenient for the sound-playing code to not take ownership. Using separate methods may in some ways be cleaner, but using a parameter can aid encapsulation.
Generally, when code returns a list of objects that implement IDisposable, the purpose of the code is to identify objects without conveying any ownership interest in them. In the absence of an ownership interest, code receiving such a list should not call Dispose on it.
The GetProcesses method does not allocate any handles (or other non managed resources) within the Process instances returned.
Only if you call certain methods on the returned Process instances are handles created, in almost all cases these are released before returning (e.g. Process.Kill).
Therefore it is completely unnecessary in most situations to dispose every Process instance returned.
The rule is very simple: if you think that other programs will use your IDisposables, then do not destroy them. Otherwise, do it.
For example: GetProcesses() returns other processes being potentialy used by other programs, so you shouldn't dispose them.
From other side, files you've opened should be released for other processes in OS, so you should close and dispose wrapper streams above them (say, you should dispose steam returned by File.Open method).
Update:
From MSDN:
DO implement the Basic Dispose Pattern on types containing
instances of disposable types. See the Basic Dispose Pattern section
for details on the basic pattern. If a type is responsible for the
lifetime of other disposable objects, developers need a way to dispose
of them, too. Using the container’s Dispose method is a convenient way
to make this possible.
DO implement the Basic Dispose Pattern
and provide a finalizer on types holding resources that need to be
freed explicitly and that do not have finalizers.
For example, the pattern should be implemented on types storing unmanaged memory buffers. The Finalizable Types section discusses guidelines related to
implementing finalizers.
CONSIDER implementing the Basic Dispose Pattern on classes that themselves don’t hold unmanaged resources or disposable objects but are likely to have subtypes that do.
We have constructors and we can treat them as contracts to follow for object instantiation.
There's no other way to create an instance without providing exact set of parameters to constructor.
But how can we (and should we ever bother) enforce some pre-mortem activity?
We've got the finalizer but they are not recommended for general purpose finalization.
We also have IDisposable to implement. But if we work with a disposable object without using we have no guarantee that Dispose will be ever called.
Why is there now way to enforce some state of the object before it will be let go off?
Tiding up in finalizer is impossible because there's no guarantee that object graph is intact and referenced object of dying object are not already nulled by GC.
Sure, not calling for instance object's SaveState() by the client code gives some troubles to it, not to my object.
Nonetheless it is considered to be a good practice to require all needed dependencies to be injected in constructor (if no default value is available). Nobody readily says: "leave default constructor, create properties and throw exceptions if object is in invalid state."
Update:
As there are many votes for closing the question I'd say that some design patterns for this can also be an answer.
Whether you use DI or not you can just count how many times object was requested/created. But without explicit release call you do not know the moment when you should call dispose.
I simply do not understand how to implement disposals at right time.
Why is there no way to enforce some state of the object before it will be let go of?
Because the whole point of a garbage collector is to simulate a machine that has an infinite amount of memory. If memory is infinite then you never need to clean it up.
You're conflating a semantic requirement of your program -- that a particular side effect occur at a particular time -- with the mechanisms of simulating infinite storage. In an ideal world those two things should not have anything to do with each other. Unfortunately we do not live in an ideal world; the existence of finalizers is evidence of that.
If there are effects that you want to achieve at a particular time, then those effects are part of your program and you should write code that achieves them. If they are important then they should be visible in the code so that people reading the code can see and understand them.
Unfortunately, during the design of Java, it was anticipated that the garbage collector should be able to satisfy all cleanup requirements. That seems to have been a belief during early design stages of .NET as well.
Consequently, no distinction is made between:
an object reference which encapsulates exclusive ownership of its target;
a reference to an object which does not encapsulate ownership(its target is owned by someone else);
a reference whose owner knows that it will either encapsulate exclusive ownership or encapsulate none, and knows which case applies for the instance at hand;
or an object reference which encapsulates shared ownership.
If a language and framework were properly designed around such distinctions, it would seldom be necessary to write code where proper cleanup could not be statically verified (the first two cases, which probably apply 90%+ of the time, could easily be statically verified even with the .NET framework).
Unfortunately, because no such distinctions exist outside the very limited context of using statements, there's no way for a compiler or verifier to know, when a piece of code abandons a reference, whether anything else is expecting to clean up the object referred to thereby.
Consequently, there's no way to know in general whether the object should be disposed, and no generally-meaningful way to squawk if it should be but isn't.
Scenario
Lets say that I've decided I really need to call GC.Collect();
What do I need to do to ensure that an object is prepared to be garbage collected appropriately before I actually call this method?
Would assigning null to all properties of the object be enough? Or just assigning null to the object itself?
If you really need to know why.....
I have a distributed application in WCF that sends a DataContract over the wire every few seconds, containing 8 Dictionaries as DataMembers.
This is a lot of data and when it comes into the Client-side interface, a whole new DataContract object is created and the memory usage of the application is growing so big that I'm getting OutOfMemory Exceptions.
Thanks
EDIT
Thanks for all the comments and answers, it seems that everyone shares the same opinion.
What I cannot understand is how I can possibly dispose correctly because the connection is open constantly.
Once I've copied the data over from the incoming object, I don't need the object anymore, so would simply implementing IDisposable on that DataContract object be enough?
My original problem is here - Distributed OutOfMemory Exceptions
As long as nothing else can see the object it is already eligible for collection; nothing more is required. The key point here is to ensure that nothing else is watching it (or at least, nothing with a longer lifetime):
is it in a field somewhere?
is it in a variable in a method that is incomplete? (an infinite loop or iterator block, perhaps)
is it in a collection somewhere?
has it subscribed to some event?
it is captured in a closure (lambda / anon-method) that is still alive?
I genuinely doubt that GC.Collect() is the answer here; if it was eligible it would have already been collected. If it isn't elgible, calling GC.Collect() certainly won't help and quite possibly will make things worse (by tying up CPU when nothing useful can be collected).
You don't generally need to do anything.
If the object is no longer referenced then it's a candidate for collection. (And, conversely, if the object is still referenced then it's not a candidate for collection, however you "prepare" it.)
You need to clean up any unmanaged resources like database connections etc.
Typically by implementing IDisposable and call Dispose.
If you have a finalizer you should call GC.SuppressFinilize.
The rest is cleaned up by the garbage collector.
Edit:
And, oh, naturally you need to release all references to your object.
But, and here is this big but. Unless you have a very very special case you don't need to call GC.Collect. You probably forgets to release some resources or references, and GC.Collect won't help you with that. Make sure you call Dispose on everything Disposable (preferably with the using-pattern).
You should probably pick up a memory profiler like Ants memory profiler and look where all your memory has gone.
If you have no more direct reference to an object, and you're running out of memory, GC should do this automatically. Do make sure you call .Dispose() on your datacontext.
Calling GC.Collect will hardly ever prevent you from getting OutOfMemory exceptions, because .NET will call GC.Collect itself when it is unable to create a new object due to OOM. There is only one scenario where I can think of and that is when you have unreferenced objects that are registered in the finalizable queue. When these objects reference many other objects it can cause a OOM. The solution to this problem is actually not to call GC.Collect but to ensure that those objects are disposed correctly (and implement the dispose pattern correctly when you created those objects).
Using GC.Collect in general
Since you are trying to get rid of a very large collection, it's totally valid to use GC.Collect(). From the Microsoft docs:
... since your application knows more about its behavior than the runtime does, you could help matters by explicitly forcing some collections. For example, it might make sense for your application to force a full collection of all generations after the user saves his data file.
"Preparing" your objects
From the excellent Performance Considerations for Run-Time Technologies in the .NET Framework (from MSDN):
If you keep a pointer to a resource around, the GC has no way of knowing if you intend to use it in the future. What this means is that all of the rules you've used in native code for explicitly freeing objects still apply, but most of the time the GC will handle everything for you.
So, to ensure it's ready for GC, Make sure that you have no references to the objects you wish to collect (e.g. in collections, events, etc...). Setting the variable to null will mean it's ready for collection before the variable goes out of scope.
Also any object which implements IDisposable should have it's Dispose() method called to clean up unmanaged resources.
Before you use GC.Collect
Since it looks like your application is a server, using the Server GC may resolve your issue. It will likely run more often and be more performant in a multi-processor scenario.
The server GC is designed for maximum throughput, and scales with very high performance.
See the Choosing Which Garbage Collector to Use within Performance Considerations for Run-Time Technologies in the .NET Framework (from MSDN):
What's the purpose of implementing the IDisposable interface? I've seen some classes implementing it and I don't understand why.
If your class creates unmanaged resources, then you can implement IDisposable so that these resources will be cleaned up properly when the object is disposed of. You override Dispose and release them there.
When your classes makes use of some system resource, it's the class' responsibility to make sure the resource is freed too. By .Net design you're supposed to do that in the Dispose method of the class. The IDisposable interface marks that your class needs to free resource when it's no longer in use, and the Dispose method is made available so that users of your class can call it to free the consumed resources.
The IDisposable method is also essential if you want auto clean-up to work properly and want to use the using() statement.
As well as freeing unmanaged resources, objects can usefully perform some operation the moment they go out of scope. A useful example might be an timer object: such objects could print out the time elapsed since their construction in the Dispose() method. These objects could then be used to log the approximate time taken for some set of operations:
using(Timer tmr=new Timer("blah"))
{
// do whatever
}
This can be done manually, of course, but my feeling is that one should take advantage wherever possible of the compiler's ability to generate the right code automatically.
It all has to do with the garbage collection mechanism. Chris Sells describes garbage collection, finalizers, and the reason for the Dispose pattern (and the IDisposable interface) episode 10 of .NET Rocks! (starting about 34 minutes in).
Many objects manipulate other entities in ways that will cause problems if not cleaned up. These other entities may be almost anything, and they may be almost anywhere. As an example, a Socket object may ask another machine to open up a TCP connection. That other machine might not be capable of handling very many connections at once; indeed, it could be a web-equipped appliance that can only handle one connection at a time. If a program were to open a socket and simply forget about it, no other computer would be able to connect to the appliance unless or until the socket got closed (perhaps the appliance might close the socket itself after a few minutes of inactivity, but it would be useless until then).
If an object implements IDisposable, that means it has the knowledge and impetus required to perform necessary cleanup actions, and such actions need to be performed before such knowledge and impetus is lost. Calling IDisposable.Dispose will ensure that all such cleanup actions get carried out, whereupon the object may be safely abandoned.
Microsoft allows for objects to request protection from abandonment by registering a method called Finalize. If an object does so, the Finalize method will be called if the system detects that the object has been abandoned. Neither the object, nor any objects to which it holds direct or indirect references, will be erased from memory until the Finalize method has been given a chance to run. This provides something of a "backstop" in case an object is abandoned without being first Disposed. There are many traps, however, with objects that implement Finalize, since there's no guarantee as to when it will be called. Not only might an object be abandoned a long time before Finalize gets called, but if one isn't careful the system may actually call Finalize on an object while part of it is still in use. Dangerous stuff. It's far better to use Dispose properly.
Is there a catch or hidden problem in using a DisposableBase base class instead of recoding the Dispose pattern on every class?
Why aren't everyone using such a relevant class?
Edits:
I naturally only meant classes that implement IDisposable
I know it uses up the option for inheritance, but I'm willing to pay the price (at least when I can and it doesn't hurt me otherwise).
When I can seal the class, I do - but I have some cases where I want the base of an inheritance hierarchy to be Disposable.
You don't need to implement Dispose() on every class - just those with something that needs deterministic cleanup. Re a Disposable base-class, I'm not entirely sure it provides a whole lot - IDisposable isn't a complex interface. The main time it might be useful is if you are handling unmanaged resources and want a finalizer, but even then it isn't much code.
Personally, I wouldn't bother with such a base class. In particular, inheritance (in a single-inheritance world) gets restrictive very quickly. But more to the point, overriding a method isn't much different to simply providing a public Dispose() method.
Again: you only need a finalizer etc if you are handling unmanaged objects.
If I had a lot of these (unmanaged resouces), I might see whether I could get PostSharp to do the work for me. I don't know if one already exists, but it might be possible to create an aspect that handles (in particular) the finalizer etc. Who knows...
Well, it uses up your one option for inheritance to describe a single aspect of your class - that's not ideal, IMO. It would be interesting to try to do something with composition, where you have a reference to a DisposableHelper and the implementation of IDisposable just calls helper.Dispose, which has the rest of the boilerplate logic in - and can call back to your code via a callback delegate. Hmm. Subclasses could subscribe to a protected Disposing event to register "I need to do something"... it might be worth looking at some time.
Personally I don't find myself implementing IDisposable often enough to make it an issue - and when I do, I typically seal my classes anyway, so half of the stuff in the pattern becomes a non-issue.
As Marc Gravell said, you only need a finalizer if you are handling unmanaged objects. Introducing an unnecessary finalizer in a base class is a bad idea, as per the reasons in section 1.1.4 of the Dispose, Finalization, and Resource Management guidelines:
There is a real cost associated with
instances with finalizers, both from a
performance and code complexity
standpoint. ... Finalization increases the cost and duration of
your object’s lifetime as each
finalizable object must be placed on a
special finalizer registration queue
when allocated, essentially creating
an extra pointer-sized field to refer
to your object. Moreover, objects in
this queue get walked during GC,
processed, and eventually promoted to
yet another queue that the GC uses to
execute finalizers. Increasing the
number of finalizable objects directly
correlates to more objects being
promoted to higher generations, and an
increased amount of time spent by the
GC walking queues, moving pointers
around, and executing finalizers.
Also, by keeping your object’s state
around longer, you tend to use memory
for a longer period of time, which
leads to an increase in working set.
If you use SafeHandle (and related classes), it's unlikely that any classes that derive from DisposableBase would ever need to be finalized.