This stems from a recent couple of posts I've made on events and memory management in general. I'm making a new question as I don't think the software I'm using has anything to do with the overall problem and I'm trying to understand a little more about how to properly manage things. This is ASP.NET.
I've been trying to understand the needs for Dispose/Finalize over the past few days and believe that I've got to a stage where I'm pretty happy with when I should/shouldn't implement the Dispose/Finalize. 'If I have members that implement IDisposable, put explicit calls to their dispose in my dispose method' seems to be my understanding. So, now I'm thinking maybe my understanding of object lifetimes and what holds on to what is just wrong!
Rather than come up with some sample code that I think will illustrate my point, I'm going to describe as best I can actual code and see if someone can talk me through it.
So, I have a repository class, in it I have a DataContext that I create when the repository is created. I implement IDisposable, and when my calling object is done, I call Dispose on my repository and explicitly call DataContext.Dispose( ). Now, one of the methods of this class creates and returns a list of objects that's handed back to my front end.
Front End -> Controller -> Repository -> Controller -> Front End.
(Using Redgate Memory Profiler, I take a snapshot of my software when the page is first loaded). My front end creates a controller object on page load and then makes a request to the repository sending back a list of items. When the page is finished loading, I call Dispose on the controller which in turn calls dispose on the context. In my mind, that should mean that my connection is closed and that I have no instances of my controller class. If I then refresh the page, it jumps to two 'Live' instances of the controller class. If I look at the object retention graph, the objects I created in my call to the list are being held onto ultimately by what looks like Linq.
The controller/repository aside, if I create a list of objects somewhere, or I create an object and return it somewhere, am I safe to just assume that .NET will eventually come and clean things up for me or is there a best practice? The 'Live' instances suggest to me that these are still in memory and active objects, the fact that RMP apparently forces GC doesn't mean anything?
If the objects are no longer rooted, they will eventually get removed from memory altogether, but calling Dispose will not cause the object to be completely removed from memory right away. The Dispose merely allows you to clean up unmanaged resources, and resources that need to be freed up asap.
In other words: your connection should be closed, but there will still be a "disposed" DataContext instance in memory.
You might want to read this article about .NET garbage collection.
My answer to your your questions consists of 2 parts:
As an application developer, you would never need to perform GC yourself. That is up to .NET. In the Dispose() method, you would only be required to relinquish or free resources that you have acquired, but not required calling any method to do garbage collection explicitly. If you do, at best, when it's done is undeterministic.
When your controller returns a list to the front end, remember that the front end does not need to know nothing about what your server-side (backend) technology is. It could be C#, Java, PHP, you get the idea. The response the backend returns to the front end is just an HTTP response, and complies to the HTTP protocol. So once the response is framed and returned to the frontend, the controller method goes out of scope, the objects in that scope are up for garbage collected. You won't have to worry about memory leaks in that situation.
Related
I have a ASP.NET C# singleton (scoped to a session through HttpContext.Current.Session) which receives messages from code (warnings, errors, exceptions etc.). The messages are related to code problems, mainly used while debugging, not during production.
I wrote the custom destructor for this object so that its contents are written/appended as a file on a disk.
I wanted to ask two things related to this situation:
a] Is is a good idea to open a file and write to it during object destructor? The concurrency IO access is handled through static lock.
b] When is the destructor called for session scoped objects? Is it only when the session is expired on the server?
I also recommend using some existing logging package. If you do decide to do this yourself, and just to bare in mind for the future:
a) No it's not a good idea. You shouldn't access managed resources in the finalizer (destructor), so if you have some log strings in memory for example, it's bad practice to access them (or the list they are contained in) as they may have already been finalized themselves at this point.
I don't want to repeat the recommended pattern, so see https://stackoverflow.com/a/1943856/2586804
You'll see there is only one place you should access managed during Dispose and this is if it is called by user code and not be the GC. So this should help you come to the conclusion that to achieve this, you must call .Dispose() yourself (or by using a using) as when (and if) the GC does it, it cannot access the managed members that contain the log lines.
b) Don't know, but it doesn't matter as you cannot use finalizer for this purpose anyway.
The bottom line is you can't rely on GC to run code for you. It's bad practice because you don't know when it's going to happen, plus any reference to the object anywhere, now or in the future will prevent the object being collected and introduce a bug.
You also shouldn't get c# Finalizers/Destructors to run code because that's not what they are for, they are for freeing unmanaged resources so that the machine doesn't run out. And note, it's a rare occurrence to use them in C# because most peoples day-to-day work is all with managed objects.
Instead explicitly tell the object to write it's logs, a method called Flush would be a good name. Or just let it write one line at a time. This would be the usual behaviour.
To accomplish this implement IDisposable and then wrap instances of your logger in a using block which will guarantee to call the dispose method on your logger.
I'm using C# but probably the same in VB.NET. I C++ I would just set a break-point on an objects destructor to know when/if it was deleted/free'd. I understand that in winforms the base class calls SupressFinalize so that form destructors are never called, so I guess I can't do it that way. Is there another method to know if an object has been garbage collected? It seems like a catch-22 because if there was you would probably need a reference to check, but by holding that reference the garbage collected will not crush it.
I've read this What strategies and tools are useful for finding memory leaks in .NET?, and I understand there are tools and/or frameworks out there to handle this "big picture" and I'm sure in a few weeks I'll be trying out several of these methods. For now I just have a really strong feeling I might have a leak related to forms not being deleted, so just want to check this one thing (and I want to know just for the sake of knowing).
I know I can watch Dispose, but I'm pretty sure Dispose can be called but still end up with the form object still being around. To test that theory I created a known-issue where I registered for a callback event in my form, then closed the form without unregistering it. Sure enough, Dispose was called (and "disposing" was true), but later when the event was fired, it still hit my break-point inside the form that was supposedly disposed already.
There are really two issues here:
As for your original question, you can use a WeakReference to monitor an object's existence without affecting its lifetime.
Your underlying question suggests that you have a misunderstanding of what garbage collection is, and how it works. The point of garbage collection is that you should never care if an object has been collected or not. Instead, you should focus on the references to the object, and if they have been reassigned or made inaccessible from rooted references. Don't worry about the instance, worry about the references to it.
The entire concept of a managed language is that you don't need to care when an object is actually garbage collected. Lots and lots of time and effort go into the GC to make sure that it doesn't collect objects that it shouldn't, that it doesn't leave objects that it should collect (when it decides to do a pass over the generation that it's in) and that all of this is done reasonably efficiently. This means that if an object consumes a large amount of managed resources and also implements IDisposable (say, a DataTable or DataSet) is disposed it is still consuming a lat of memory, and disposing of it doesn't make it get garbage collected any quicker (although you should still dispose of it to ensure any managed resources go away).
The GC is built to work best when you leave it alone and let it do it's job rather than interfering with it by trying to, for example, manually cause collections to take place. This is occasionally useful for debugging purposes or learning about your program/the language, but it virtually never belongs in a production application.
Disposing has nothing to do with garbage collection or collecting an object. Disposing is the mechanism in place for dealing with a managed object that is holding onto an unmangaed resource (or another object that is holding onto an unmangaed resource). Disposing of the object is telling it to clear up that unmanaged resource, but it has nothing to do with the garbage collector freeing the managed resources of that object. The destructor is there so that you don't free the managed resources before freeing the unmanaged resources, but it's perfectly acceptable (and in fact should always happen) that the unmanaged resources are cleaned up (via dispose) before the managed resource is freed.
Now, it is still possible, given all of this, for a program to have a memory leak, but there are a few questions you really need to be asking yourself first. How large is the leak? Is it a one off, continuous over time, every time we do function X, etc? What would it take for the program to consume so much memory as to be disruptive (either run out of memory, cause other programs to run out of memory, etc) and is that likely to occur with common or legitimate uses of the program? Usually you don't need to start asking these types of questions until you start getting out of memory exceptions, or notice that you start running out of physical memory for a program that oughtn't to. If you notice these problems then you can start looking around for objects that implement IDisposable that aren't being disposed, to see if you're holding onto references to very large objects (or collections of objects) that you no longer need, etc.
Sorry for the wall of text.
I use a static variable for holding the count of objects. In constructor I increase this variable. This way I know how many instances of the object are created.
After using the objects, they are leaved dereferenced.
I'm doubtful if MEF is holding references to these objects so I force GC to do a clean up (Using GC.Collect() method). I expect on next object creation this variable start from zero but it resumes from last number. I put a logging mechanism in destructor for tracing, and objects are destroyed only after the application is closed.
Can I assume MEF has created other references to those objects ?
I use MEF and ExportFactory for creating my objects
Edit:
Maybe something with ExportLifetimeContext should be done ?
I force GC to do a clean up
If MEF still has references to the objects, then obviously this doesn't do anything. If the objects have become garbage, then the garbage collector will automatically collect them - asking it to do that explicitly is just a hint that might be ignored. Either way, this is not necessary.
I put a logging mechanism in destructor for tracing, and objects are destroyed only after the application is closed. Can I assume MEF has created other references to those objects ?
MEF will hold references to created objects so that it is able to repeatedly return the same reference when you ask for an export. To ask MEF to abandon these references, you should call CompositionContainer.Dispose. Obviously you cannot reuse the container any more after that, though you could create a new one.
MEF is also the owner of any IDisposable objects it creates. This means that when you dispose the container, it will call Dispose on any such objects before abandoning the reference.
It is preferable to rely on calls to Dispose to perform cleanup, rather than to use finalizers. There is no guarantee that finalizers are run at all.
edit:
I need to destroy the object after using it. But I don't want to destroy the container. I want MEF as a factory for creating new instances of asking part, and the caller should be capable of destroying the object when he doesn't need anymore. Can you help with this ?
This is what ExportFactory is for. (It was formerly called PartCreator). Unfortunately it is not yet available in .NET 4, unless you use Silverlight. You can use the preview releases from codeplex to already give it a try.
If you don't want to use the preview releases of MEF, you might be able to implement something like ExportFactory yourself by creating factory classes that wrap the container and by using its GetExport and ReleaseExport methods to acquire and release objects. Don't forget to set a PartCreationPolicy if you need to create multiple instances of the same part.
edit 2:
I somehow missed that you were already using ExportFactory all along. You simply need to call ExportLifeTimeContext.Dispose when you're done with the object!
The timely "destruction" (that is finalization) of objects in the CLR is not a good thing to rely on. If you're doing this for debugging purposes, there's no need. You can find out how many objects of some type are still in existence by following the instructions in my answer to this question:
Memory Leaks in C# WPF
If you are genuinely trying to make your software's behaviour depend on a count of the number of instances of a class that have not been reclaimed by the GC, then you need to rethink your design. There will be several better ways of achieving what you want.
For shared objects, MEF will keep references to them as long as the container is alive. You can have NonShared parts be disposed early in the right circumstances. If you are using ExportFactory, you need to dispose the ExportLifetimeContext returned by the CreateExport method to dispose any NonShared parts created as part of the request. If you called a GetExport method on the container, you can call the ReleaseExport method. If you got the exports some other way (ie SatisfyImports or something), there isn't a way to release them so you may want to refactor your application to use ExportFactory or GetExport.
MEF now has a desktop version which supports ExportFactory. Although the name has 'export' word, you should implement it where you are doing your 'imports'. A call to CreateExport() method for creating a new instance of your part will return an ExportLifetimeContext<T> an this object has a Dispose() method which could be used later for releasing exported object. This method would call your object Dispose() automatically and you don't need to call it manually.
This behavior is because the container is the owner of created objects by itself and even with a reference to these objects calling their Dispose() has no effect on them.
Scenario
Lets say that I've decided I really need to call GC.Collect();
What do I need to do to ensure that an object is prepared to be garbage collected appropriately before I actually call this method?
Would assigning null to all properties of the object be enough? Or just assigning null to the object itself?
If you really need to know why.....
I have a distributed application in WCF that sends a DataContract over the wire every few seconds, containing 8 Dictionaries as DataMembers.
This is a lot of data and when it comes into the Client-side interface, a whole new DataContract object is created and the memory usage of the application is growing so big that I'm getting OutOfMemory Exceptions.
Thanks
EDIT
Thanks for all the comments and answers, it seems that everyone shares the same opinion.
What I cannot understand is how I can possibly dispose correctly because the connection is open constantly.
Once I've copied the data over from the incoming object, I don't need the object anymore, so would simply implementing IDisposable on that DataContract object be enough?
My original problem is here - Distributed OutOfMemory Exceptions
As long as nothing else can see the object it is already eligible for collection; nothing more is required. The key point here is to ensure that nothing else is watching it (or at least, nothing with a longer lifetime):
is it in a field somewhere?
is it in a variable in a method that is incomplete? (an infinite loop or iterator block, perhaps)
is it in a collection somewhere?
has it subscribed to some event?
it is captured in a closure (lambda / anon-method) that is still alive?
I genuinely doubt that GC.Collect() is the answer here; if it was eligible it would have already been collected. If it isn't elgible, calling GC.Collect() certainly won't help and quite possibly will make things worse (by tying up CPU when nothing useful can be collected).
You don't generally need to do anything.
If the object is no longer referenced then it's a candidate for collection. (And, conversely, if the object is still referenced then it's not a candidate for collection, however you "prepare" it.)
You need to clean up any unmanaged resources like database connections etc.
Typically by implementing IDisposable and call Dispose.
If you have a finalizer you should call GC.SuppressFinilize.
The rest is cleaned up by the garbage collector.
Edit:
And, oh, naturally you need to release all references to your object.
But, and here is this big but. Unless you have a very very special case you don't need to call GC.Collect. You probably forgets to release some resources or references, and GC.Collect won't help you with that. Make sure you call Dispose on everything Disposable (preferably with the using-pattern).
You should probably pick up a memory profiler like Ants memory profiler and look where all your memory has gone.
If you have no more direct reference to an object, and you're running out of memory, GC should do this automatically. Do make sure you call .Dispose() on your datacontext.
Calling GC.Collect will hardly ever prevent you from getting OutOfMemory exceptions, because .NET will call GC.Collect itself when it is unable to create a new object due to OOM. There is only one scenario where I can think of and that is when you have unreferenced objects that are registered in the finalizable queue. When these objects reference many other objects it can cause a OOM. The solution to this problem is actually not to call GC.Collect but to ensure that those objects are disposed correctly (and implement the dispose pattern correctly when you created those objects).
Using GC.Collect in general
Since you are trying to get rid of a very large collection, it's totally valid to use GC.Collect(). From the Microsoft docs:
... since your application knows more about its behavior than the runtime does, you could help matters by explicitly forcing some collections. For example, it might make sense for your application to force a full collection of all generations after the user saves his data file.
"Preparing" your objects
From the excellent Performance Considerations for Run-Time Technologies in the .NET Framework (from MSDN):
If you keep a pointer to a resource around, the GC has no way of knowing if you intend to use it in the future. What this means is that all of the rules you've used in native code for explicitly freeing objects still apply, but most of the time the GC will handle everything for you.
So, to ensure it's ready for GC, Make sure that you have no references to the objects you wish to collect (e.g. in collections, events, etc...). Setting the variable to null will mean it's ready for collection before the variable goes out of scope.
Also any object which implements IDisposable should have it's Dispose() method called to clean up unmanaged resources.
Before you use GC.Collect
Since it looks like your application is a server, using the Server GC may resolve your issue. It will likely run more often and be more performant in a multi-processor scenario.
The server GC is designed for maximum throughput, and scales with very high performance.
See the Choosing Which Garbage Collector to Use within Performance Considerations for Run-Time Technologies in the .NET Framework (from MSDN):
What's the purpose of implementing the IDisposable interface? I've seen some classes implementing it and I don't understand why.
If your class creates unmanaged resources, then you can implement IDisposable so that these resources will be cleaned up properly when the object is disposed of. You override Dispose and release them there.
When your classes makes use of some system resource, it's the class' responsibility to make sure the resource is freed too. By .Net design you're supposed to do that in the Dispose method of the class. The IDisposable interface marks that your class needs to free resource when it's no longer in use, and the Dispose method is made available so that users of your class can call it to free the consumed resources.
The IDisposable method is also essential if you want auto clean-up to work properly and want to use the using() statement.
As well as freeing unmanaged resources, objects can usefully perform some operation the moment they go out of scope. A useful example might be an timer object: such objects could print out the time elapsed since their construction in the Dispose() method. These objects could then be used to log the approximate time taken for some set of operations:
using(Timer tmr=new Timer("blah"))
{
// do whatever
}
This can be done manually, of course, but my feeling is that one should take advantage wherever possible of the compiler's ability to generate the right code automatically.
It all has to do with the garbage collection mechanism. Chris Sells describes garbage collection, finalizers, and the reason for the Dispose pattern (and the IDisposable interface) episode 10 of .NET Rocks! (starting about 34 minutes in).
Many objects manipulate other entities in ways that will cause problems if not cleaned up. These other entities may be almost anything, and they may be almost anywhere. As an example, a Socket object may ask another machine to open up a TCP connection. That other machine might not be capable of handling very many connections at once; indeed, it could be a web-equipped appliance that can only handle one connection at a time. If a program were to open a socket and simply forget about it, no other computer would be able to connect to the appliance unless or until the socket got closed (perhaps the appliance might close the socket itself after a few minutes of inactivity, but it would be useless until then).
If an object implements IDisposable, that means it has the knowledge and impetus required to perform necessary cleanup actions, and such actions need to be performed before such knowledge and impetus is lost. Calling IDisposable.Dispose will ensure that all such cleanup actions get carried out, whereupon the object may be safely abandoned.
Microsoft allows for objects to request protection from abandonment by registering a method called Finalize. If an object does so, the Finalize method will be called if the system detects that the object has been abandoned. Neither the object, nor any objects to which it holds direct or indirect references, will be erased from memory until the Finalize method has been given a chance to run. This provides something of a "backstop" in case an object is abandoned without being first Disposed. There are many traps, however, with objects that implement Finalize, since there's no guarantee as to when it will be called. Not only might an object be abandoned a long time before Finalize gets called, but if one isn't careful the system may actually call Finalize on an object while part of it is still in use. Dangerous stuff. It's far better to use Dispose properly.