I have a long-running application that consistently fails due to a memory leak.
I suspect my use of static properties may be the cause. Here's an example of what I have today:
public class StaticReferences
{
public static readonly object Fixed1 = new object();
}
public class ShortLived
{
public object Object1;
}
public class Doer // This class is instantiated once
{
public void DoStuff() // This method is called over and over again.
{
var shortLived = new ShortLived()
{
Object1 = StaticReferences.Fixed1
};
}
}
Will an instance of ShortLived with its reference to StaticReferences.Fixed1 (via the ShortLived.Object1 property) get properly garbage collected once it is out of scope?
No, just referencing global static properties won't create a memory leak. The example you posted is fine. shortLived will be cleaned up once its scope is over and the reference to Fixed1 will get cleaned up when your program exits. Your problem is very likely elsewhere but it's impossible to say from your simple example. Do you have any proof that you're looking at a memory leak?
I suggest you use a memory profiler or get a full memory dump and analyze it (WinDbg is free but there are other, easier to use but pay tools, too). Another tool you can try using is DebugDiag from Microsoft (also free) - get a dump and then run it through DebugDiag to get a memory report.
As #EricJ mentioned in his comment, the profiler in Visual Studio 2015 is also a great tool to analyze memory use and it's available in all editions, including the free Community Edition.
Related
I have the following simple program which I am trying to use with VS 2015's Diagnostic Tools related to Memory Usage.
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Begin");
Console.ReadLine();
Goo();
Console.WriteLine("End");
Console.ReadLine();
}
private static void Goo()
{
var list = new List<string>();
for (var i = 0; i < 1000; i++)
{
Foo(list);
}
}
private static void Foo(IEnumerable<string> strings)
{
foreach (var str in strings)
{
}
}
}
While profiling this application's project, I took couple of snapshots and was expecting to see 1000 boxed List<string>+Enumerator objects. For example, I get this kind of information in JetBrains's dotMemory product. But for some reason, I am unable to see this information in VS's tool...I am obviously missing something...can anyone point me in the right direction?
As you can see in the above snapshot, I get information about the mscorlib module only where as I do not see any information about my executing program. What am I missing?...some more info below:
I used Start Diagnostic Tools Without Debugging in visual studio
After taking and opening the snapshot, I even unselected the option Collapse small objects to see if this was hiding any info, but that also did not help.
Updated(responding to user answer):
I am using dotMemory version 4.4. Following is a snapshot of the data I get from it. NOTE: make sure to click the button Collect Allocations before you hit any key after seeing the Begin message
All objects created in Goo and Foo already collected when you get snapshot at "End" point. I profiled this code with dotMemory 10.0.1 and also do not see any objects created in Goo and Foo methods.
UPDATE: In dotMemory you are looking at "Memory Traffic" view. Memory traffic - are objects created and possible already collected to the point of time. dotMemory shows you a warning that it is not able to display collected objects. If you check "Start collecting allocation data immediatelly" checkbox in the profiling setup dialog, dotMemory will show you that these 1000 objects were allocated and already collected. In VS Diagnostic Tools you are looking at live objects graph. I'm not very familliar with this tool, but it seems that there is no information about memory traffic.
If you look at live objects graph in dotMemory ("All objects" view), you won't find these objects too.
The memory analysis tool works by iterating all of the GC roots and traversing the object graphs from there - similar to what the GC does. Once the method is out of scope, the local variable containing the enumerator is no longer a GC root. Any heap objects whose reference is stored in a local and is not referenced through another GC root are unreachable and essentially gone at that point.
I just delivered my first C# WebAPI application to the first customer. Under normal load, performance initially is even better than I expected. Initially.
Everything worked fine until, at some point, memory was up and garbage collection started running riot (as in "It collects objects that are not yet garbage"). At that point, there were multiple W3WP threads with some ten gigs of ram altogether, and single-digit gigs per worker. After a restart of the IIS everything was back to normal, but of course the memory usage is rising again.
Please correct me if I am wrong, but
Shouldn't C# have automatic garbage collection?
Shouldn't it be easy for GC to collect the garbage of a WebAPI application?
And please help me out:
How can I explicitly state what GC should collect, thus preventing memory leaks? Is someBigList = null; the way to go?
How can I detect where the memory leaks are?
EDIT: Let me clarify some things.
My .NET WebAPI application is mostly a bunch of
public class MyApiController:ApiController
{
[HttpGet]
public MyObjectClass[] MyApi(string someParam) {
List<MyObjectClass> list = new List<MyObjectClass>();
...
for/while/foreach {
MyObjectClass obj = new MyObjectClass();
obj.firstStringAttribute = xyz;
...
list.Add(obj);
}
return list.ToArray();
}
}
Under such conditions, GC should be easy: after "return", all local variables should be garbage. Yet with every single call the used memory increases.
I initially thought that C# WebAPI programs behave similar to (pre-compiled) PHP: IIS calls the program, it is executed, returns the value and is then completely disposed off.
But this is not the case. For instance, I found static variables to keep their data between runs, and now I disposed of all static variables.
Because I found static variables to be a problem for GC:
internal class Helper
{
private static List<string> someVar = new List<string>();
internal Helper() {
someVar=new List<string>();
}
internal void someFunc(string str) {
someVar.Add(str);
}
internal string[] someOtherFunc(string str) {
string[] s = someVar.ToArray();
someVar=new List<string>();
return s;
}
}
Here, under low-memory conditions, someVar threw a null pointer error, which in my opinion can only be caused by GC, since I did not find any code where someVar is actively nullified by me.
I think the memory increase slowed down since I actively set the biggest array variables in the most often used Controllers to null, but this is only a gut feeling and not even nearly a complete solution.
I will now do some profiling using the link you provided, and get back with some results.
Shouldn't C# have automatic garbage collection?
C# is a programming language for the .NET runtime, and .NET brings the automatic garbage collection to the table. So, yes, although technically C# isn't the piece that brings it.
Shouldn't it be easy for GC to collect the garbage of a WebAPI application?
Sure, it should be just as easy as for any other type of .NET application.
The common theme here is garbage. How does .NET determine that something is garbage? By verifying that there are no more live references to the object. To be honest I think it is far more likely that you have verified one of your assumptions wrongly, compared to there being a serious bug in the garbage collector in such a way that "It collects objects that are not yet garbage".
To find leaks, you need to figure out what objects are currently held in memory, make a determination whether that is correct or not, and if not, figure out what is holding them there. A memory profiler application would help with that, there are numerous available, such as the Red-Gate ANTS Memory Profiler.
For your other questions, how to make something eligible for garbage collection? By turning it into garbage (see definition above). Note that setting a local variable to null may not necessarily help or be needed. Setting a static variable to null, however, might. But the correct way to determine that is to use a profiler.
Here are some shot-in-the-dark type of tips you might look into:
Look at static classes, static fields, and static properties. Are you storing data there that is accumulating?
How about static events? Do you have this? Do you remember to unsubscribe the event when you no longer need it?
And by "static fields, properties, and events", I also mean normal instance fields, properties and events that are held in objects that directly or indirectly are stored in static fields or properties. Basically, anything that will keep the objects in memory.
Are you remembering to Dispose of all your IDisposable objects? If not, then the memory being used could be unmanaged. Typically, however, when the garbage collector collects the managed object, the finalizer of that object should clean up the unmanaged memory as well, however you might allocate memory that the GC algorithm isn't aware of, and thus thinks it isn't a big problem to wait with collection. See the GC.AddMemoryPressure method for more on this.
I have stumbled across a situation where garbage collection seems to be behaving differently between the same code running written as a Unit Test vs written in the Main method of a Console Application. I am wondering the reason behind this difference.
In this situation, a co-worker and I were in disagreement over the effects of registering an event handler on garbage collection. I thought that a demonstration would be better accepted than simply sending him a link to a highly rated SO answer. As such I wrote a simple demonstration as a unit test.
My unit test showed things worked as I said they should. However, my coworker wrote a console application that showed things working his way, which meant that GC was not occurring as I expected on the local objects in the Main method. I was able to reproduce the behavior he saw simply by moving the code from my test into the Main method of a Console Application project.
What I would like to know is why GC does not seem to collecting objects as expected when running in the Main method of a Console Application. By extracting methods so that the call to GC.Collect and the object going out of scope occurred in different methods, the expected behavior was restored.
These are the objects I used to define my test. There is simply an object with an event and an object providing a suitable method for an event handler. Both have finalizers setting a global variable so that you can tell when they have been collected.
private static string Log;
public const string EventedObjectDisposed = "EventedObject disposed";
public const string HandlingObjectDisposed = "HandlingObject disposed";
private class EventedObject
{
public event Action DoIt;
~EventedObject()
{
Log = EventedObjectDisposed;
}
protected virtual void OnDoIt()
{
Action handler = DoIt;
if (handler != null) handler();
}
}
private class HandlingObject
{
~HandlingObject()
{
Log = HandlingObjectDisposed;
}
public void Yeah()
{
}
}
This is my test (NUnit), which passes:
[Test]
public void TestReference()
{
{
HandlingObject subscriber = new HandlingObject();
{
{
EventedObject publisher = new EventedObject();
publisher.DoIt += subscriber.Yeah;
}
GC.Collect(GC.MaxGeneration);
GC.WaitForPendingFinalizers();
Thread.MemoryBarrier();
Assert.That(Log, Is.EqualTo(EventedObjectDisposed));
}
//Assertion needed for foo reference, else optimization causes it to already be collected.
Assert.IsNotNull(subscriber);
}
GC.Collect(GC.MaxGeneration);
GC.WaitForPendingFinalizers();
Thread.MemoryBarrier();
Assert.That(Log, Is.EqualTo(HandlingObjectDisposed));
}
I pasted the body above in to the Main method of a new console application, and converted the Assert calls to Trace.Assert invocations. Both equality asserts fail then fail. Code of resulting Main method is here if you want it.
I do recognize that when GC occurs should be treated as non-deterministic and that generally an application should not be concerning itself with when exactly it occurs.
In all cases the code was compiled in Release mode and targeting .NET 4.5.
Edit: Other things I tried
Making the test method static since NUnit supports that; test still worked.
I also tried extracting the whole Main method into an instance method on program and calling that. Both asserts still failed.
Attributing Main with [STAThread] or [MTAThread] in case this made a difference. Both asserts still failed.
Based on #Moo-Juice's suggestions:
I referenced NUnit to the Console app so that I could use the NUnit asserts, they failed.
I tried various changes to visibility to the both the test, test's class, Main method, and the class containing the Main method static. No change.
I tried making the Test class static and the class containing the Main method static. No change.
If the following code was extracted to a separate method, the test would be more likely to behave as you expected. Edit: Note that the wording of the C# language specification does not require this test to pass, even if you extract the code to a separate method.
{
EventedObject publisher = new EventedObject();
publisher.DoIt += subscriber.Yeah;
}
The specification allows but does not require that publisher be eligible for GC immediately at the end of this block, so you should not write code in such a way that you are assuming it can be collected here.
Edit: from ECMA-334 (C# language specification) §10.9 Automatic memory management (emphasis mine)
If no part of the object can be accessed by any possible continuation of execution, other than the running of finalizers, the object is considered no longer in use and it becomes eligible for finalization. [Note: Implementations might choose to analyze code to determine which references to an object can be used in the future. For instance, if a local variable that is in scope is the only existing reference to an object, but that local variable is never referred to in any possible continuation of execution from the current execution point in the procedure, an implementation might (but is not required to) treat the object as no longer in use. end note]
The problem isn't that it's a console application - the problem is that you're likely running it through Visual Studio - with a debugger attached! And/or you're compiling the console app as a Debug build.
Make sure you're compiling a Release build. Then go to Debug -> Start Without Debugging, or press Ctrl+F5, or run your console application from a command line. The Garbage Collector should now behave as expected.
This is also why Eric Lippert reminds you not to run any perf benchmarks in a debugger in C# Performance Benchmark Mistakes, Part One.
The jit compiler knows that a debugger is attached, and it deliberately de-optimizes the code it generates to make it easier to debug. The garbage collector knows that a debugger is attached; it works with the jit compiler to ensure that memory is cleaned up less aggressively, which can greatly affect performance in some scenarios.
Lots of the reminders in Eric's series of articles apply to your scenario. If you're interested in reading more, here are the links for parts two, three and four.
#GilShalit posted this comment a year ago:
"Well, we have come to distrust ODP
(.Net 2.0) after fighting a memory
leak (in code we supplied to a
customer) in GetOracleDecimal for over
a year... Good luck!" – GilShalit Aug
27 '09 at 12:44
How did you solve it?
We have a service that queries an Oracle database every few minutes that is not releasing memory; after some investigation using WinDbg I discovered that this type is piling up in the finalize queue: Oracle.DataAccess.Types.OpoDecCtx.
Here's the line that I think is the problem:
decimal volume = (decimal)OracleDecimal.SetPrecision(reader.GetOracleDecimal(5), 28);
I commented this out and the memory leak disappeared.
Any thoughts will be appreciated - thanks!
This is an old issue with ODP.NET (see here: Memory Problems with ODP.NET 10.1.0.4 ).
The OracleDecimal type holds a reference to an instance of an internal class named OpoDecCtx. OpoDecCtx implements IDisposable (as it's itself referencing unmanaged memory), but since OracleDecimal does not implement IDisposable, you'll have to wait for the garbage collector to run to free the underlying unmanaged memory. You can check all this using a tool such as .NET Reflector.
Although it's not technically a "physical" memory leak (memory will be eventually freed), it is actually a problem when you're dealing with a large amount of instances of the OracleDecimal type. I don't know why Oracle does not simply implement IDisposable, it's a simple thing to do...
Anyway, I suggest you do some hack job yourself, using reflection:
public static class OracleExtentions
{
public static void Dispose(this OracleDecimal od) // build an extension method
{
if (OracleDecimalOpoDecCtx == null)
{
// cache the data
// get the underlying internal field info
OracleDecimalOpoDecCtx = typeof(OracleDecimal).GetField("m_opoDecCtx", BindingFlags.Instance | BindingFlags.NonPublic);
}
IDisposable disposable = OracleDecimalOpoDecCtx.GetValue(od) as IDisposable;
if (disposable != null)
{
disposable.Dispose();
}
}
private static FieldInfo OracleDecimalOpoDecCtx;
}
And you would use it like this:
OracleDecimal od = reader.GetOracleDecimal(5);
decimal volume = (decimal)OracleDecimal.SetPrecision(od, 28);
od.Dispose();
I don't know if it is possible to change from ODP.NET to another provider but we solved this by ditching ODP a long time ago... and are using a 3rd-party (commercial) ADO.NET provider (not affiliated)... check this link out http://www.devart.com/dotconnect/oracle/docs/
I have an object, which lives forever. I am deleteing all references I can see, to it after using it, but it still not collected. Its life cycle is pretty sophisticated so I can't be sure that all references been cleared.
if ( container.Controls.Count > 0 )
{
var controls = new Control[ container.Controls.Count ];
container.Controls.CopyTo( controls, 0 );
foreach ( var control in controls )
{
container.Controls.Remove( control );
control.Dispose();
}
controls = null;
}
GC.Collect();
GC.Collect(1);
GC.Collect(2);
GC.Collect(3);
How can I find out what references does it still have? Why is it not collected?
Try using a memory profiler, (e.g. ants) it will tell you what is keeping the object alive. Trying to 2nd guess this type of problem is very hard.
Red-gate gives 14 days trial that should be more then enough time to tack down this problem and decide if a memory profiler provides you with long term value.
There are lots of other memory profilers on the market (e.g. .NET Memory Profiler) most of them have free trials, however I have found that the Red-Gate tools are easy to use, so tend try them first.
You'll have to use Windbg and Sosex extension.
The !DumpHeap and !GCRoot commands can help you to identify the instance, and all remaining references that keep it alive.
I solved a similar issue with the SOS extension (which apparently does no longer work with Visual Studio 2013, but works fine with older versions of Visual Studio).
I used following code to get the address of the object for which I wanted to track references:
public static string GetAddress(object o)
{
if (o == null)
{
return "00000000";
}
else
{
unsafe
{
System.TypedReference tr = __makeref(o);
System.IntPtr ptr = **(System.IntPtr**) (&tr);
return ptr.ToString ("X");
}
}
}
and then, in Visual Studio 2012 immediate window, while running in the debugger, type:
.load C:\Windows\Microsoft.NET\Framework\v4.0.30319\sos.dll
which will load the SOS.dll extension.
You can then use GetAddress(x) to get the hexadecimal address of the object (for instance 8AB0CD40), and then use:
!do 8AB0CD40
!GCRoot -all 8AB0CD40
to dump the object and find all references to the object.
Just keep in mind that if the GC runs, it might change the address of the object.
I've been using .NET Memory Profiler to do some serious memory profiling on one of our projects. It's a great tool to look into the memory management of your app. I don't get paid for this info :) but it just helped me alot.
The garbage collection in .NET is not a counting scheme (such as COM), but a mark-and-sweep implementation. Basically, the GC runs at "random" times when it feels the need to do so, and the collection of the objects is therefore not deterministic.
You can, however, manually trigger a collection (GC.Collect()), but you may have to wait for finalizers to run then (GC.WaitForPendingFinalizers()). Doing this in a production app, however, is discouraged, because it may affect the efficiency of the memory management (GC runs too often, or waits for finalizers to run). If the object still exists, it actually still has some live reference somewhere.
It is not collected because you haven't removed all references to it. The GC will only mark objects for collection if they have no roots in the application.
What means are you using to check up on the GC to see if it has collected your object?