stackoverflow. I've written a application in C# (a raytracer). Problem is not with raytracer but with ray usage. Application keep consuming more ram over time.
Ram usage while not raytracing : start with 10mb goes up and up
Ram usege while raytracing : start 80mb then 120mb and up, if resolution is big
this number is up to 500mb.
I looked to VS Diagnostics tool and see application garbage collection is like 10 gc in 1 minute. This number is even bigger while application is raytracing. Raytracer returns a gigantic pixel array. I set pixel array to null after raytracing but application ram stucks in a range of 500-600 mb ram.
Are there any options to collect more garbage or free ram?
EDIT
This is a WinForms application, not WPF or something.
I cant be sure since there is no code, but it sounds like a memory leak. Verify that there are no static object referencing the problematic object. To verify, run GC.Collect() after you set the object to null since if its in generation 2 for example it might take a while for the garbage collector to check it and reclaim it's memory. if after GC.Collect you see that the memory is not reclaimed, you most likely have a memory leak somewhere. If you cant find the location of the rouge reference, you can use tools like this https://blogs.msdn.microsoft.com/visualstudioalm/2015/04/29/diagnosing-event-handler-leaks-with-the-memory-usage-tool-in-visual-studio-2015/
Good luck
Can i recommend you attempt to identify if any memory leaks are contributing to the progressive increase of memory usage of your application. There are plenty of free and paid tools that are available to achieve this task, along with endless advice on how to go about using them. The vs diagnostics tool has the ability to snapshot the memory usage of your application identifying where it is in use and what class types are present. Further information and example of use for the VS diagnostics tools can be found here : https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/Visual-Studio-2015-Diagnostic-Tools
Further Help : What strategies and tools are useful for finding memory leaks in .NET?
Similar searches for ".net identify memory leaks" on here and other platforms will also help.
Commonly memory leaks are caused by the incorrect disposal and the retention of circular references, so identify any locations across your code where circular references are present.
Related
What is the reason for pinned GC handles when working with unmanaged .net components? This happens from time to time without any code changed or something else. When investigating the issue, I see a lot of pinned GC-Handles
These handles seem to stick in the memory for the entire application lifetime. In this case, the library is GdPicture (14). Is there any way to investigate why those instances are not cleaned up? I'm using Dispose()/using everywhere and can't find any GC roots in the managed code.
Thanks a lot!
EDIT
Another behaviour that is strange is, that the task manager shows that the application uses about 6GB ram, when the memory profiler shows the usage of 400MB (red line is live bytes)
What is the reason for pinned GC handles when working with unmanaged .net components?
Pinning is needed when working with unmanaged code. It prevents objects from being moved during garbage collection so that the unmanaged code can have a pointer to it. The garbage collector will update all .NET references, but it will not update unmanaged pointer values.
Is there any way to investigate why those instances are not cleaned up?
No. The reason always is: there's a bug in the code. Either your code (assume that first) or in a 3rd party library (libraries are used often, chances are that leaks in the library have been found by someone else before).
I'm using Dispose()/using everywhere
Seems like you missed one or it's not using the disposable pattern.
Another behaviour that is strange is, that the task manager shows that the application uses about 6GB ram, when the memory profiler shows the usage of 400MB (red line is live bytes)
A .NET memory profiler may only show the .NET part of memory (400 MB) and omit the rest (5600 MB).
Task manager is not interested in .NET. It cares about physical RAM mostly, which is why Task Manager is not a good analytics tool in general. You don't want to analyze physical RAM, you want to analyze virtual memory.
To look for memory leaks, use Process Explorer and show the "Private Bytes" and "Virtual size" column. Process Explorer can also show you a graph over time per process.
How to proceed?
Forget about the unmanaged leak for a moment. Use a .NET profiler that has the capability of taking memory snapshots and allows you to see each individual object inside as well as a statistics.
Try to figure out the steps that it takes to create more leaks in a consistent way. Then
Take a snapshot
Repeat the leak procedure 10 times
Take a snapshot
Repeat the leak procedure another 10 times
Take a snaphot
Compare snapshot of step 1 and 3. Check for managed types that differ in multiples of 10. Compare snapshot of step 3 and 5. Check the same type again. It must be a multiple of 10. You can't leak 7 objects when you run a method 10 times.
Do a code review on the places where the affected types are used based on internal knowledge on the leak procedure (which methods are called) and the managed type. Make sure it's disposed or released properly.
Currently i'm tracking down a memory leak issue in a Silverlight 4 application. I can watch the memory used by iexplorer.exe increase gradually using Task Manager.
However, I am looking for a way within the Silverlight code to tell how much memory the application is currently using within the IE Process. I have tried GC.GetTotalMemory but it does not represent the actual memory usage by iexplorer.
I have developer tools for tracking down the memory leak itself, so i'm not looking for that but a method within my Silverlight code to get total IE memory usage, or at least a closer representation of what is actually being used by Silverlight instead of what the Garbage collector thinks is being used.
GC allocated memory is good estimation of dynamic portion of memory used by SL application. Otherwise it is unclear what other types of memory usage you need.
If you need some other data consider using VaDump first ( http://msdn.microsoft.com/en-us/magazine/dd882521.aspx linked from Using VADump to track memory usage - OpenProcess Failed c0000034). When you figure out what portion/type of memory you are interested in it should be relatively easy to get this information (you may need to collect it with native calls, but likely all can be PInvoke'ed).
Update: I think all information you looking for is avaialble through performance counters - usage sample - http://msdn.microsoft.com/en-us/library/system.diagnostics.performancecounter.countername.aspx (but I'm don't think you can get it from SL).
Try the visual studio profiler
https://blogs.msdn.com/b/profiler/archive/2010/04/26/vs2010-silverlight-4-profiling.aspx
"For .NET memory profiling, use “VSPerfClrEnv /samplegc” or “VSPerfClrEnv /samplegclife” in your step 1 to get either allocation profiling or allocation and lifetime profiling. "
ANTS Profiler...
http://www.red-gate.com/products/dotnet-development/ants-memory-profiler/
I know it's a stupid question, but when i run my program which contains threading , i find that the memory(VM, and Memory used) by the application in the Task manager is increasing regarding that my threads are stopped at that moment, so i wonder if there's any way to know the source of this, or just know at which line the application is compiling now? .
i used the thread watch window but i didn't get any useful information ragarding that.
If you're sure your program is using excessive memory then getting your hands on a memory profiler would be a good first approach.
You can use the CLR Profiler application to get snapshots of your memory consumption. Then you'll be able to identify the source of your issue.
CLR Profiler is free and available here: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=be2d842b-fdce-4600-8d32-a3cf74fda5e1
Its worth noting that the memory profiler does not directly map to the memory(VM, and Memory used) or Working Set counter in Task Manager.
The working set of a program is a
collection of those pages in its
virtual address space that have been
recently referenced. It includes both
shared and private data. The shared
data includes pages that contain all
instructions your application
executes, including those in your DLLs
and the system DLLs. As the working
set size increases, memory demand
increases.
If memory serves the Memory Profiler will look at private bytes are which represent the actual memory that you are using.
Also See the section A comment on performance counters and how not to use taskmanager in this article from Tess Ferrandez
We got "out of memory" issue on production servers. What API can we use to get live memory (physical and managed) usage of the ASP.NET application?
Thanks.
PS: we're forbidden to profile memory with tools.
Also an Out of memory doesn't necessarily mean what you think. heavy memory fragmentation due to excessive object creation can be reported as out of memory since the GC is not able to find a continuous block of memory large enough for next (or last attempted) allocation.
.Net performance counters can help you with a number of managed memory related stats.
This page details some of the ones available- http://msdn.microsoft.com/en-us/library/x2tyfybc.aspx
You can use normal windows memory counters to get an overview of non-managed memory- http://msdn.microsoft.com/en-us/library/aa965225(VS.85).aspx
Well, the first thing that I can suggest you is to take a memory dump when your w3wp.exe usage is high. You should takes these dumps and analyze it yourself, or get an expert to do it.
http://blogs.msdn.com/tess will tell you how if you are interested in doing it.
BUT... before you do any of the real exercise... two things that you absolutely must do.
Turn Debug = False in all your web.config files http://aspalliance.com/1341
Turn Trace = False in all your web.config files since the trace data is maintained in your memory and worsens memory shortage.
in addition to the performance counters you could use the profiling tools shipped with Visual Studio in order to monitor the memory usage. This step can help you on identifing memory issues before the code is released to production. Here's how to do it with VS2010: http://msdn.microsoft.com/en-us/library/dd264934.aspx
I know there is no simple answer to my question but I would appreciate ideas, guides or
some sort of things-to-look-at list
I have a net Windows service that is constantly throwing OutOfMemoryException.
The service has two builds for x86 and x64 Windows. However on x64 it consumes a lot more
memory. I have tried profiling it with various memory profilers. But I cannot get a clue what the problem is. The diagnosis - service consumes lot of VMSize and crashes app after 3 to 12 hours. The behaviuor is rather stochastic - there is no observable pattern for crash scenario.
Also I tried to look at performance counters (perfmon.exe). What I can see is that
heap size is growing and %GC time is on average 19%. Plus memory allocation is correlated with %CPU time.
My application has threads and locking objects, DB connections and WCF interface.
The general question that I am trying to solve:
Is it simply GC not been fast enough
to GC objects or some non-managed
(windows) objects are consuming
memory?
See first app in list
http://s45.radikal.ru/i109/1003/af/92a389d189e8.jpg http://s45.radikal.ru/i109/1003/af/92a389d189e8.jpg
The link to picture with performance counters view
http://s006.radikal.ru/i215/1003/0b/ddb3d6c80809.jpg
Is your issue that you don't know what is consuming a lot of memory? You can open up task manager when the process is using a lot of memory, right click your process and create a dump file which you can examine in windbg to find out exactly what's allocating memory.
Tess Ferrandez has a lot of excellent demos. She goes through the most useful stuff here...
Your problem is likely to be either a classic leak (objects that are still rooted when they shouldn't be) or Large Object Heap (LOH) fragmentation.
The best tool I have found for diagnosing this class of problem is the Son of Strike (SOS) extension to the Windows debugger. Download Microsoft's Debugging Tools for Windows to get the debuggers: CDB is the console debugger (which I prefer as it seems more responsive), WinDbg is the same thing wrapped as an MDI app. These tools are quite low-level and have a bit of learning curve but provide everything you need to know to find your problem.
In particular, run !DumpHeap -stat to see what types of objects are eating your memory. This command will also report at the bottom of the list if it notices any significant fragmentation. !EEHeap will list the heap segments — if there are a lot of LOH segments then I would suspect LOH fragmenation.
0:000> .loadby sos mscorwks
0:000> !EEHeap -gc
Number of GC Heaps: 1
generation 0 starts at 0x00f7a9b0
generation 1 starts at 0x00e79c3c
generation 2 starts at 0x00b21000
ephemeral segment allocation context: none
segment begin allocated size
00b20000 00b21000 010029bc 0x004e19bc(5118396)
Large object heap starts at 0x01b21000
segment begin allocated size
01b20000 01b21000 01b8ade0 0x00069de0(433632)
If there are many LOH segments then I would begin to suspect LOH fragmentation.
Before doing this, however, I would be interested to know:
Does the application use string.Intern()?
Does the application have transient objects that subscribe to events with long-lived objects?
(The reason I ask this is that 1. the .NET string intern tables are implemented in such a way that they can cause LOH fragmenation and 2. an event subscription provides an additional root for the subscribing object which is easy to forget.)
I have used .Net Memory Profiler it much better than clr profiler by microsoft. You have to learn about it a little bit. It can tell you which object are not disposing or have references. You can also sort object base on there type and memory. I used the trial ver which last 30 days during which i was able to solve problem in my application.
If your percentage time spent on GC is high then I would look at LOH Allocations perfmon counter. If there are frequent allocations in LOH this would cause the GC to work hard to collect, which is the reason for High percentage time spent on GC.
I did blog about the identifying high CPU in GC because of LOH where it shows how to get the exact call-stack which is allocating in LOH.
Hope this helps.