I have a large C# server application, I'm interested in learning how the GC class works, and in particular what actions should I take to determine the source of a possible memory leak.
Are there any books on the subject, or is it not really that ellaborate?
There are plenty of sources you can study.
I hope you don't miss basics:
CLR via C# 3rd Edition by Jeffrey Richter
I think before you go with details about GC, try to understand how IDisposable and resource management is handled:
Dispose, Finalization, and Resource Management. It pretty old but still awesome.
GC specific:
Garbage Collection / Fundamentals of Garbage Collection
Maoni's WebLog (Maoni Stephens is a software developer who spends her time implementing .NET's GC. In fact, she's been working on the GC since the early days of .NET.)
Video: Maoni Stephens and Andrew Pardoe: CLR 4 Garbage Collector - Inside Background GC
Video: Erik Meijer and Patrick Dussud - Inside Garbage Collection
Drill Into .NET Framework Internals to See How the CLR Creates Runtime Objects
Identify And Prevent Memory Leaks In Managed Code
Hope it helps to start.
Not a book, but our team has used the ANTS Memory Profiler with pretty good success for tracking down managed memory leaks. Their support section and included help walks you through the process of tracking down different types of memory issues. This doesn't include specifics on the GC class itself, just how to track down common mistakes (event handler deregistration, static variables, etc.).
Also not a book, but decent article.
Memory Leak Detection in .NET
There is an excellent article by Rico Mariani: Tracking down managed memory leaks (how to find a GC leak). I used this technique often and is easy and efficient. And getting yourself familiar with a true debugger like Windbg is a bonus side benefit!
There is also the SciTech .NET Memory Profiler, our team has been using that successfully.
To complement the answers above, there are more recent videos on Channel9 with Maoni Stephens (Principal developer for the GC on the CLR team at Microsoft) that walk you through basics of GC, what developers should look out for, how they should troubleshoot, and some of the tools you can use. I found the explanation of how the GC works and the concept of generations and roots really useful.
Here is the first part of a 3 episode series :
http://channel9.msdn.com/Shows/Defrag-Tools/Defrag-Tools-33-CLR-GC-Part-1
Related
I'm working on .Net project (windows service) and I found memory leak. Using ANTS Memory profiler, I've determined - root cause is in unmanaged memory scope. But I don't use any COM/COM+ or p/invoke - pure .Net (DataAcces - Fluent nHibernate, Automapper, WCF)..
Hypothetically, what can be reason? How can I find bottleneck.. I'm interested in any kind of experience, who've faced and solved such kind of isseus
A bit more details:
I know how objects are disposing (and do it right)
I (almost) don't use events
I'm using timers
I'm using multithreading (but it's rather parallelism)
My application is not stateless - I'm tracking objects during their life-cycle
About Memory Leaks
Generally Speaking Memory Leaks still can exist in .net. (Sort of, not all of these are text book memory leaks per se) As in things get stuck in generation 2 garbage collection or when not disposing of object that inherit from IDisposable or for that matter Streams, Events, SQL connections that stay open and mess with you connection pool.While I am not an expert in the matter I will point you to a couple of places that may help you on your quest.
Memory Management Fundementals
Detecting Memory Leaks in .NET
While you are at it Groking the way that garbage collection works helps in this sort of endevor a great deal. Start with something like The Fundementals of Garbage Collection for a general mental picture of just what the heck is going on with all that .NET magic behind the scenes.
On the COM Stuff
Inherent in .NET there are many things that are just simply com and or win32 wrappers calling PInvoke behind the scenes. If gettin down and dirty with insides is what you are wanting then I'd recommend you look at a Decompiler (like IL Spy) where you suspect the leaky code.
Check this out for a deeper discussion on all the wrapper related stuff.
Is .NET A Wrapper Around Win32?
Is there a way I can know who holds a reference to an object?
I see that my object is not finalized by the d'tor after I call GC.Collect.
There is no way to do this effectively in code. The best way to track down leaks of this sort is to use a tool like WinDbg. The SOS extension has several commands which are specifically designed to help track down memory leaks.
Rico did a thorough blog entry on the subject:
http://blogs.msdn.com/ricom/archive/2004/12/10/279612.aspx
In addition to Windbg, you could use a .net profiler. I've done it both ways -- the profile won't tell you anything that Windbg can't, but a good profiler is a little more friendly.
Windbg
In addition to the link from JaredPar, Tess Ferrandez has some great articles on using Windbg with .net. Here's a a memory leak article: http://blogs.msdn.com/tess/archive/2006/01/23/516139.aspx
Profiler:
I have had great luck with SciTech's profiler (I'm not affiliated in any way, great tool, though):
http://memprofiler.com/
They have some great how-to videos on their site as well.
You ought to call GC.WaitForPendingFinalizers after GC.Collect to allow the garbage collector to clear the freachable queue. Since the GC runs in a different thread it is possible that your application has moved on while the GC is trying to finalize any objects that require finalization. By calling GC.WaitForPendingFinalizers you are forcing your current thread to wait until the GC has finished clearing the freachable queue (the queue of items that require finalization) before resuming execution.
That being said however, I agree with Joel Coehoorn's comment - the GC works best when you leave it alone.
Does anyone know the major differences between the Java and .Net garbage collectors? A web search has not revealed much, and it was a question that came up in a test.
The difference is between the CLR (.Net) GC and the JVM GC rather than the languages themselves.
Both are subject to change and the specification of their behaviour loose to allow this to be changed without it affecting the correctness of programs.
There are some historical differences largely due to .Net being designed with lessons from the evolution of the java (and other gc based platforms). In the following do not assume that the .Net one was in some way superior because it included functionality from the beginning, it is simply the result of coming later.
A notable publicly visible difference is that the MS GC exposes its generational nature (via the GC api) this is likely to remain true for some time since this is an obvious approach to take based on the behaviour that most programs exhibit: Most allocations are extremely short lived.
Initial JVM's did not have generational garbage collectors though this feature was swiftly added.
The first generational collectors implemented by SunOracle and others tended to be Mark and Sweep. It was realized that a mark-sweep-compact approach would lead to much better memory locality justifying the additional copying overhead. The CLR runtime debuted with this behaviour.
A difference between SunOracle's and Microsoft's GC implementation 'ethos' is one of configurability.
Sun's provides a vast number of options (at the command line) to tweaks aspects of the GC or switch it between different modes. Many options are of the -X or -XX to indicate their lack of support across different versions or vendors. The CLR by contrast provides next to no configurability; your only real option is the use of the server or client collectors which optimise for throughput verses latency respectively.
Active research in GC strategies is ongoing in both companies (and in open source implementations) current approaches being used in the most recent GC implementations are per thread eden areas (improving locality and allowing the eden collection to potentially not cause a full pause) as well as pre-tenuring approaches, which try to avoid placing certain allocations into the eden generation.
This is just to add to ShuggyCoUk's excellent answer. The .NET GC also uses what is know as the large object heap (LOH). The CLR preallocates a bunch of objects on the LOH and all user allocated objects of at least 85000 bytes are allocated on the LOH as well. Furthermore, double[] of 1000 elements or more are allocated on the LOH as well due to some internal optimization.
The LOH is handled differently than the generational heaps in various ways:
It is only cleaned during a full collect and it is never compacted like the generational heaps.
Allocation from the LOH is done via a free list much like malloc is handled in the C runtime, whereas allocations from the generational heap is essentially done by just moving a pointer in generation 0.
I don't know if the JVM has something similar, but it is essential information on how memory is handled in .NET so hopefully, you find it useful.
If I recall correctly, the JVM doesn't release deallocated memory back to the operating system as the CLR does.
Java 5 introduced a lot of changes into its GC algorithms.
I'm not a C# maven, but these two articles suggest to me that both have evolved away from simple mark and sweep and towards newer generation models:
http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html
http://www.csharphelp.com/archives2/archive297.html
I found this:
In the J2SE platform version 1.4.2 there were four garbage collectors from which to choose but without an explicit choice by the user the serial garbage collector was always chosen. In version 5.0 the choice of the collector is based on the class of the machine on which the application is started.
here and this
Also just as the JVM manages the destruction of objects so also does the CLR via a Mark and Compact garbage collection algorithm
here
I hope this helps...
Can someone please explain me how garbage collection is working?
(I'm using C# and Java).
It's too complex topic to be covered in one simple answer.
Here is a list of recommended reading:
Wikipedia: Garbage collection
(computer science)
Garbage Collection: Automatic Memory
Management in the Microsoft .NET
Framework
Garbage Collection: Part 2:
Automatic Memory Management in the
Microsoft .NET Framework
Java theory and practice: A brief
history of garbage collection
The basic idea behind Garbage Collection is that you don't have to care about memory management. What the Garbage Collectod does is to periodically check upon objects references and find the one that are no more used (not referenced anymore) to reclaim their memory and compact the other.
Garbage Collector use various algorithms to perform their work and they differ for some details from a lenguage to another.
Wikipedia gives you a good starting point.
If you're looking for more in depth information about actual implementation of various Garbage Collectors (Java, .NET, ..) you can check here and here or search google for more.
Try the book Garbage Collection: Algorithms for Automatic Dynamic Memory Management. It wont have the more recent stuff in it, but it'll get you on your way.
Perfmon provides a number of counters for GC related performance...
Here's a nice webcast that discusses simple mark-and-sweep (non-generational) garbage collection, complete with nice animations to help clairy the concept.
I think you need to know that the Garbage Collector is a thread that runs on your program freeing the memory occupied by the objects whose references make them unreachable. You also need to know that the moment in which int GC runs can't be predicted, you may make a call to System.gc() to make a suggestion for the GC to run but not to make it run, it is the JVM who'll take that decision.
If you have:
Object objectReference = null;
the Object referenced by objectReference is GC bait. The subjects of "islands of isolation" and how the finalize() method works are interesting topics to read. I suggest a quick google search on both.
As far as I can tell, this is isn't possible, so I'm really just hoping for a left field undocumented allocation hook function.
I want a way to track allocations like in _CrtSetAllocHook, but for C#/.NET.
The only visibility to the garbage collector/allocation appears to be GC.CollectionCount.
Anyone have any other .NET memory mojo?
The CLR has a 'profiling API' that hooks into pretty much everything - it is what the commercial .NET memory profiling products use, I believe. Here is an MSDN link to the top level of the documentation: .NET Framework General Reference: About the Profiling API
See this MSDN magazine article for an introduction to the memory piece: Inspect and Optimize Your Program's Memory Usage with the .NET Profiler API
I would just use Red Gate's ANTS Profiler. It will tell you a lot about what's going on in memory without you having to learn the profiling API yourself.