What is the reason for pinned GC handles when working with unmanaged .net components? This happens from time to time without any code changed or something else. When investigating the issue, I see a lot of pinned GC-Handles
These handles seem to stick in the memory for the entire application lifetime. In this case, the library is GdPicture (14). Is there any way to investigate why those instances are not cleaned up? I'm using Dispose()/using everywhere and can't find any GC roots in the managed code.
Thanks a lot!
EDIT
Another behaviour that is strange is, that the task manager shows that the application uses about 6GB ram, when the memory profiler shows the usage of 400MB (red line is live bytes)
What is the reason for pinned GC handles when working with unmanaged .net components?
Pinning is needed when working with unmanaged code. It prevents objects from being moved during garbage collection so that the unmanaged code can have a pointer to it. The garbage collector will update all .NET references, but it will not update unmanaged pointer values.
Is there any way to investigate why those instances are not cleaned up?
No. The reason always is: there's a bug in the code. Either your code (assume that first) or in a 3rd party library (libraries are used often, chances are that leaks in the library have been found by someone else before).
I'm using Dispose()/using everywhere
Seems like you missed one or it's not using the disposable pattern.
Another behaviour that is strange is, that the task manager shows that the application uses about 6GB ram, when the memory profiler shows the usage of 400MB (red line is live bytes)
A .NET memory profiler may only show the .NET part of memory (400 MB) and omit the rest (5600 MB).
Task manager is not interested in .NET. It cares about physical RAM mostly, which is why Task Manager is not a good analytics tool in general. You don't want to analyze physical RAM, you want to analyze virtual memory.
To look for memory leaks, use Process Explorer and show the "Private Bytes" and "Virtual size" column. Process Explorer can also show you a graph over time per process.
How to proceed?
Forget about the unmanaged leak for a moment. Use a .NET profiler that has the capability of taking memory snapshots and allows you to see each individual object inside as well as a statistics.
Try to figure out the steps that it takes to create more leaks in a consistent way. Then
Take a snapshot
Repeat the leak procedure 10 times
Take a snapshot
Repeat the leak procedure another 10 times
Take a snaphot
Compare snapshot of step 1 and 3. Check for managed types that differ in multiples of 10. Compare snapshot of step 3 and 5. Check the same type again. It must be a multiple of 10. You can't leak 7 objects when you run a method 10 times.
Do a code review on the places where the affected types are used based on internal knowledge on the leak procedure (which methods are called) and the managed type. Make sure it's disposed or released properly.
Related
stackoverflow. I've written a application in C# (a raytracer). Problem is not with raytracer but with ray usage. Application keep consuming more ram over time.
Ram usage while not raytracing : start with 10mb goes up and up
Ram usege while raytracing : start 80mb then 120mb and up, if resolution is big
this number is up to 500mb.
I looked to VS Diagnostics tool and see application garbage collection is like 10 gc in 1 minute. This number is even bigger while application is raytracing. Raytracer returns a gigantic pixel array. I set pixel array to null after raytracing but application ram stucks in a range of 500-600 mb ram.
Are there any options to collect more garbage or free ram?
EDIT
This is a WinForms application, not WPF or something.
I cant be sure since there is no code, but it sounds like a memory leak. Verify that there are no static object referencing the problematic object. To verify, run GC.Collect() after you set the object to null since if its in generation 2 for example it might take a while for the garbage collector to check it and reclaim it's memory. if after GC.Collect you see that the memory is not reclaimed, you most likely have a memory leak somewhere. If you cant find the location of the rouge reference, you can use tools like this https://blogs.msdn.microsoft.com/visualstudioalm/2015/04/29/diagnosing-event-handler-leaks-with-the-memory-usage-tool-in-visual-studio-2015/
Good luck
Can i recommend you attempt to identify if any memory leaks are contributing to the progressive increase of memory usage of your application. There are plenty of free and paid tools that are available to achieve this task, along with endless advice on how to go about using them. The vs diagnostics tool has the ability to snapshot the memory usage of your application identifying where it is in use and what class types are present. Further information and example of use for the VS diagnostics tools can be found here : https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/Visual-Studio-2015-Diagnostic-Tools
Further Help : What strategies and tools are useful for finding memory leaks in .NET?
Similar searches for ".net identify memory leaks" on here and other platforms will also help.
Commonly memory leaks are caused by the incorrect disposal and the retention of circular references, so identify any locations across your code where circular references are present.
I have created an application which reads a file and fetches meta data of the file. When I launch the application the private working set is around 8MB (as viewed in Task Manager). When I scan the file, the memory shoots up to 150MB and stays there. If I add additional file using the same instance of the application the memory piles on. To understand this behavior, I used a memory profiler (Red gates) which showed me the following statistics :-
Out of the 150MB of private worker set memory
Unmanaged Memory :94MB
Others Resources (string,array etc) : 30MB
This puzzles me as I am not using any un-managed code nor any Pinvoke calls. I have also tried GC.Collect() without success.
Can someone please guide me as to how I can reduce the Unmanaged memory usage of my application and what could be the possible causes for the same.
Thanks in Advance
A recent update to ANTS shows you which .NET classes your application's unmanaged memory is used by. After enabling unmanaged memory profiling in the setup screen, navigate to the class list and sort by the new 'unmanaged size' column.
Though you may not have knowingly used unmanaged memory, many .NET Framework libraries do use native resources - for example the imaging libraries.
Currently i'm tracking down a memory leak issue in a Silverlight 4 application. I can watch the memory used by iexplorer.exe increase gradually using Task Manager.
However, I am looking for a way within the Silverlight code to tell how much memory the application is currently using within the IE Process. I have tried GC.GetTotalMemory but it does not represent the actual memory usage by iexplorer.
I have developer tools for tracking down the memory leak itself, so i'm not looking for that but a method within my Silverlight code to get total IE memory usage, or at least a closer representation of what is actually being used by Silverlight instead of what the Garbage collector thinks is being used.
GC allocated memory is good estimation of dynamic portion of memory used by SL application. Otherwise it is unclear what other types of memory usage you need.
If you need some other data consider using VaDump first ( http://msdn.microsoft.com/en-us/magazine/dd882521.aspx linked from Using VADump to track memory usage - OpenProcess Failed c0000034). When you figure out what portion/type of memory you are interested in it should be relatively easy to get this information (you may need to collect it with native calls, but likely all can be PInvoke'ed).
Update: I think all information you looking for is avaialble through performance counters - usage sample - http://msdn.microsoft.com/en-us/library/system.diagnostics.performancecounter.countername.aspx (but I'm don't think you can get it from SL).
Try the visual studio profiler
https://blogs.msdn.com/b/profiler/archive/2010/04/26/vs2010-silverlight-4-profiling.aspx
"For .NET memory profiling, use “VSPerfClrEnv /samplegc” or “VSPerfClrEnv /samplegclife” in your step 1 to get either allocation profiling or allocation and lifetime profiling. "
ANTS Profiler...
http://www.red-gate.com/products/dotnet-development/ants-memory-profiler/
I know it's a stupid question, but when i run my program which contains threading , i find that the memory(VM, and Memory used) by the application in the Task manager is increasing regarding that my threads are stopped at that moment, so i wonder if there's any way to know the source of this, or just know at which line the application is compiling now? .
i used the thread watch window but i didn't get any useful information ragarding that.
If you're sure your program is using excessive memory then getting your hands on a memory profiler would be a good first approach.
You can use the CLR Profiler application to get snapshots of your memory consumption. Then you'll be able to identify the source of your issue.
CLR Profiler is free and available here: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=be2d842b-fdce-4600-8d32-a3cf74fda5e1
Its worth noting that the memory profiler does not directly map to the memory(VM, and Memory used) or Working Set counter in Task Manager.
The working set of a program is a
collection of those pages in its
virtual address space that have been
recently referenced. It includes both
shared and private data. The shared
data includes pages that contain all
instructions your application
executes, including those in your DLLs
and the system DLLs. As the working
set size increases, memory demand
increases.
If memory serves the Memory Profiler will look at private bytes are which represent the actual memory that you are using.
Also See the section A comment on performance counters and how not to use taskmanager in this article from Tess Ferrandez
A memory intensive program that I wrote ran out of memory: threw an OutOfMemory exception. During attempts to reduce memory usage, I started calling GC.GetTotalMemory(true) (to write the total memory usage to debug file), which triggers a garbage collect.
For some reason, when calling this function I don't get an out of memory exception anymore. If I remove the calls again (keeping everything else the same), the exception gets thrown again. In my understanding, calls are automatically made to collect garbage when memory pressure increases, so I don't understand this behavior.
Can anyone explain why the out of memory exception is only thrown when there are no calls to GC.collect?
Update:
I'm using VS 2010, but I'm downtargeting the application to framework 3.5. I believe that defragmentation is indeed causing my problems.
I did some tests: When the exception is thrown, a call to GC.gettotalmemory tells me I am using ~800 * 10^6 bytes. However, task manager tells me the application is using 1700 mb. A rather large discrepancy. I'm now planning to allocate memory only once, and to never deallocate any large arrays but reusing them. Luckily, my program allows me to accomplish this without too much fuss.
I solved the problem by doing some smarter memory management. In particular by using a CustomList according to the suggestions on http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/
Is your app running at full CPU? I'm pretty sure automatic garbage collection only occurs when the application is idle. Otherwise, you have to run a manual cycle.
I'm fairly sure that running out of memory does not force a garbage collection. That probably sounds incredibly unintuitive to you but I think this was done for a good reason. It prevents the program from entering a death-spiral where it constantly tries to find more space and getting all objects firmly lodged into gen #2. From which it is very hard to recover again.
The true argument you pass to GetTotalMemory() forces a full garbage collection. I would guess that this happens to free up enough space in the Large Object Heap to satisfy the memory allocation. This will of course work only once. If your program just keeps running, gobbling up memory beyond the 1.5 gigabytes or so that it has already consumed then OOM is just around the corner again. This time without any way to recover. Surviving an OOM requires drastic measures.
You'll need a good memory profiler to find out what's really going on. Unmanaged C++ in your project is always a fertile source of memory leaks. The unmanaged kind, always hard to trouble-shoot.
GC.Collect is merely a "suggestion" to free unused memory - it does not guarantee its release.
[Edit]
It appears that, while once true when I was learning the JVM years ago, this may not be the case in .NET anymore. The MSDN Library says that GC.Collect "Forces an immediate garbage collection of all generations." Good stuff (for me, anyway) about this here.
If you have unmanaged resources that occupy a lot of memory, the garbage collector won't really recognize that memory pressure. If you clean up those resources in finalizers, then forcing a collection will result in those unmanaged resources being freed, while if you don't force a collection, the garbage collector might not realize that it needs to be collecting.
If you are performing large unmanaged allocations, you can use GC.AddMemoryPressure to tell the GC that, so it can take it into account when deciding whether to run a collection or not.
I was browsing around the Microsoft Connect site and I am seeing bug reports where people are making the same claim you are. The claim being that an OutOfMemoryException is occuring which can be resolved by periodically calling GC.Collect. I saw one report where the lead engineer from the garbage collector team responded back and said a bug was fixed in .NET 4.0 that should resolve a fragmentation issue with the large object heap. That is why I asked what version you were using.
It is certainly possible that you have stumbled upon a bug in the garbage collector. As with all GC related issues this could be very version dependent.
My advice would be to:
make sure you have the latest patches and service pack
refactor the code so that it is not as memory intensive
reuse LOH objects as much as possible instead of creating new ones
continue using GC.Collect at strategic points if necessary as a workaround