Is it possible to track allocation/deallocation? - c#

As far as I can tell, this is isn't possible, so I'm really just hoping for a left field undocumented allocation hook function.
I want a way to track allocations like in _CrtSetAllocHook, but for C#/.NET.
The only visibility to the garbage collector/allocation appears to be GC.CollectionCount.
Anyone have any other .NET memory mojo?

The CLR has a 'profiling API' that hooks into pretty much everything - it is what the commercial .NET memory profiling products use, I believe. Here is an MSDN link to the top level of the documentation: .NET Framework General Reference: About the Profiling API
See this MSDN magazine article for an introduction to the memory piece: Inspect and Optimize Your Program's Memory Usage with the .NET Profiler API

I would just use Red Gate's ANTS Profiler. It will tell you a lot about what's going on in memory without you having to learn the profiling API yourself.

Related

How does the EQATEC profiler compare to DotTrace?

I'm trying to understand why I can't get WPF to make my images display snappy, while startups like Facebook and Apple seem to have to trouble with making their software behave ;-)
Well, one obvious differece is that EQATEC is free, and that DotTrace isn't. However, given that usualy the free edition isn't sufficient and one has to move to the $$ ones, and given that DotTrace has a "personal license" for individual developers, this is less a differentiator than one could at first belive. And I also have Resharper...
So, leaving aside the "free" factor, is the EQATEC profiler comparable to the Jet Brains one?
EQATEC is not totally free. It just offers a free licence (note: a limited one).
I've used ANTS performance profiler, dotTrace, EQATEC, Slimtune and among memory profilers Scitech and ANTS memory profiler. Memory profilers are a different kind of beast. I found Scitech the best one, although ANTS memory profiler also offers some nice results. Among performance other profilers, ANTS and dotTrace are superior to all other known to me. Slimtune is very basic, but ... it works. EQATEC was a mixed experience.
The problem with EQATEC was that it failed to start (no matter what I tried, a crash occurred) with one of the applications I worked on, but it worked with another one although the limitations of the free version were somehow annoying.
Profilers have a steep learning curve and it takes some time to get used to one, learn to use it the right way and interpret the results correctly. So you need to put value of your time, value of the performance improvement in your application and value of the profiler into the equation and decide then. I went for ANTS.
If you rarely use it, the free/limited alternatives may do just fine. But if it should be your productivity tool, go for one of the best ones, there are trial versions available.
You can find some very good opinions here: What Are Some Good .NET Profilers?
But regarding to your profiling requirements: just pick one of those, trial or free and profile that piece of code, giving money away for such a minor profiling task is rediculous.

Is is possible to use Profiling API right from C#?

I just want to use .NET Profiling API (ICorProfilerCallback etc) but at the same time don't want to deal with C++. I've been looking around for a while and haven't found any example in C# but C# + C++ where the most interesting part is written using C++.
No, you cannot implement the CLR profiling APIs in managed code (C# or otherwise) since the profiling callbacks are called at very specific times when the managed environment is assumed to be in a certain state. Implementing your callbacks in managed code would violate a lot of assumptions.
David Broman, the developer of the CLR profiling APIs, has this to say:
You need to write your profiler in
C++. The profiler is called by the
runtime at very delicate points during
execution of the profiled application,
and it is often extremely unsafe to be
running managed code at those points.
David's blog is a great resource for dealing with the CLR profiling APIs.

C# Usage of GC and how to trace a Memory Leak

I have a large C# server application, I'm interested in learning how the GC class works, and in particular what actions should I take to determine the source of a possible memory leak.
Are there any books on the subject, or is it not really that ellaborate?
There are plenty of sources you can study.
I hope you don't miss basics:
CLR via C# 3rd Edition by Jeffrey Richter
I think before you go with details about GC, try to understand how IDisposable and resource management is handled:
Dispose, Finalization, and Resource Management. It pretty old but still awesome.
GC specific:
Garbage Collection / Fundamentals of Garbage Collection
Maoni's WebLog (Maoni Stephens is a software developer who spends her time implementing .NET's GC. In fact, she's been working on the GC since the early days of .NET.)
Video: Maoni Stephens and Andrew Pardoe: CLR 4 Garbage Collector - Inside Background GC
Video: Erik Meijer and Patrick Dussud - Inside Garbage Collection
Drill Into .NET Framework Internals to See How the CLR Creates Runtime Objects
Identify And Prevent Memory Leaks In Managed Code
Hope it helps to start.
Not a book, but our team has used the ANTS Memory Profiler with pretty good success for tracking down managed memory leaks. Their support section and included help walks you through the process of tracking down different types of memory issues. This doesn't include specifics on the GC class itself, just how to track down common mistakes (event handler deregistration, static variables, etc.).
Also not a book, but decent article.
Memory Leak Detection in .NET
There is an excellent article by Rico Mariani: Tracking down managed memory leaks (how to find a GC leak). I used this technique often and is easy and efficient. And getting yourself familiar with a true debugger like Windbg is a bonus side benefit!
There is also the SciTech .NET Memory Profiler, our team has been using that successfully.
To complement the answers above, there are more recent videos on Channel9 with Maoni Stephens (Principal developer for the GC on the CLR team at Microsoft) that walk you through basics of GC, what developers should look out for, how they should troubleshoot, and some of the tools you can use. I found the explanation of how the GC works and the concept of generations and roots really useful.
Here is the first part of a 3 episode series :
http://channel9.msdn.com/Shows/Defrag-Tools/Defrag-Tools-33-CLR-GC-Part-1

How to know which instruction taking the longest time to finish ? (to increase my program performance)

I need to know how can I get every instruction's duration so I can maintain the code to increase the performance of my program .
Use a profiler. If you have Visual Studio Team System, there's one included. Otherwise take a look at ANTS or dotTrace.
what you're looking for is a profiler I believe :)
see: Profiler and List of Performance Analysis Tools
You want an application profiler for that, it shows exactly what code takes how long.
You need to use a profiler to accomplish this. It exists several profilers, some are free.
My preference goes to Red Gate Ants.
I dont think you should go upto level of instructions to measure performance bottlenecks. Micro optimization can be harmful. You should go up to function profiling. If you are using VS2005 or 2008 you can use
Performance wizard
CLR profiler
to profile your functions.
Alternatly I personally recomend using Ants Profiler
Since Ants and dotTrace are very good but commercial tools (I wouldn't call them expensive - they're worth the money), I recently heard about the EQATEC Profiler, which is free of charge. Did not try it because of lack of time, but maybe you want to give it a try.
(no, I am not affiliated with them)
If you have an application running and you want to improve its performance using a profiler (.net or database) is a must .DotTrace and Ants are famous ones for good reasons.
If you are using SQL Server ,SQL server profiler is a great tool to trace and watch what's going on ,on the server side of your application.
If you want to decide what approach is better to use you can use ILDASM to disassemble your code to IL and see what's going on under the hood Although it's not a simple task but I think it worth it.
You might want to take a look at FxCop as well, might give you some more vauge hints as to what could be improved. (Oh, and it's free!)
I'm surprised no one have mentioned this yet, but if you want to know the cost of individual instructions, look them up here or here.
The cost of individual instructions varies between CPU's, but both AMD and Intel (and any other CPU maker) documents this.
The problem is that determining the cost of instructions is not straightforward. You have a lot of metrics to consider: There's the latency, whether it is pipelined (fully or partially), how big the instruction is (affects instruction cache) and so on. So this information is only really helpful if you're writing a single really performance-sensitive function where you're either writing assembly yourself, or closely reading the compiler-generated ASM to find and eliminate inefficiencies. And if you know a fair bit about how the CPU works.
But before you get to this point, you should use a profiler as everyone else have suggested. That helps you narrow down where the time is being spent, and what needs optimizing.

What Are Some Good .NET Profilers?

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
What profilers have you used when working with .net programs, and which would you particularly recommend?
I have used JetBrains dotTrace and Redgate ANTS extensively. They are fairly similar in features and price. They both offer useful performance profiling and quite basic memory profiling.
dotTrace integrates with Resharper, which is really convenient, as you can profile the performance of a unit test with one click from the IDE. However, dotTrace often seems to give spurious results (e.g. saying that a method took several years to run)
I prefer the way that ANTS presents the profiling results. It shows you the source code and to the left of each line tells you how long it took to run. dotTrace just has a tree view.
EQATEC profiler is quite basic and requires you to compile special instrumented versions of your assemblies which can then be run in the EQATEC profiler. It is, however, free.
Overall I prefer ANTS for performance profiling, although if you use Resharper then the integration of dotTrace is a killer feature and means it beats ANTS in usability.
The free Microsoft CLR Profiler (.Net framework 2.0 / .Net Framework 4.0) is all you need for .NET memory profiling.
2011 Update:
The Scitech memory profiler has quite a basic UI but lots of useful information, including some information on unmanaged memory which dotTrace and ANTS lack - you might find it useful if you are doing COM interop, but I have yet to find any profiler that makes COM memory issues easy to diagnose - you usually have to break out windbg.exe.
The ANTS profiler has come on in leaps and bounds in the last few years, and its memory profiler has some truly useful features which now pushed it ahead of dotTrace as a package in my estimation. I'm lucky enough to have licenses for both, but if you are going to buy one .Net profiler for both performance and memory, make it ANTS.
Others have covered performance profiling, but with regards to memory profiling
I'm currently evaluating both the Scitech .NET Memory Profiler 3.1 and ANTS Memory Profiler 5.1 (current versions as of September 2009). I tried the JetBrains one a year or two ago and it wasn't as good as ANTS (for memory profiling) so I haven't bothered this time. From reading the web sites it looks like it doesn't have the same memory profiling features as the other two.
Both ANTS and the Scitech memory profiler have features that the other doesn't, so which is best will depend upon your preferences. Generally speaking, the Scitech one provides more detailed information while the ANTS one is really incredible at identifying the leaking object. Overall, I prefer the ANTS one because it is so quick at identifying possible leaks.
Here are the main the pros and cons of each from my experience:
Common Features of ANTS and Scitech .NET Memory Profiler
Real-time analysis feature
Excellent how-to videos on their web sites
Easy to use
Reasonably performant (obviously slower than without the profiler attached, but not so much you become frustrated)
Show instances of leaking objects
Basically they both do the job pretty well
ANTS
One-click filters to find common leaks including: objects kept alive only by event handlers, objects that are disposed but still live and objects that are only being kept alive by a reference from a disposed object. This is probably the killer feature of ANTS - finding leaks is incredibly fast because of this. In my experience, the majority of leaks are caused by event handlers not being unhooked and ANTS just takes you straight to these objects. Awesome.
Object retention graph. While the same info is available in Scitech, it's much easier to interpret in ANTS.
Shows size with children in addition to size of the object itself (but only when an instance is selected unfortunately, not in the overall class list).
Better integration to Visual Studio (right-click on graph to jump to file)
Scitech .NET Memory Profiler
Shows stack trace when object was allocated. This is really useful for objects that are allocated in lots of different places. With ANTS it is difficult to determine exactly where the leaked object was created.
Shows count of disposable objects that were not disposed. While not indicative of a leak, it does identify opportunities to fix this problem and improve your application performance as a result of faster garbage collection.
More detailed filtering options (several columns can be filtered independently).
Presents info on total objects created (including those garbage collected). ANTS only shows 'live' object stats. This makes it easier to analyze and tune overall application performance (eg. identify where lots of objects being created unnecessarily that aren't necessarily leaking).
By way of summary, I think ANTS helps you find what's leaking faster while Scitech provides a bit more detail about your overall application memory performance and individual objects once you know what to look at (eg. stack trace on creation). If the stack trace and tracking of undisposed disposable objects was added to ANTS I wouldn't see the need to use anything else.
I recently discovered EQATEC Profiler http://www.eqatec.com/tools/profiler. It works with most .NET versions and on a bunch of platforms. It is easy to use and parts of it is free, even for commercial use.
[Full Disclosure]
While not yet as full-featured as some of the other .NET memory profilers listed here, there is a new entry on the market called JustTrace. It's made by Telerik and it's primary goal is to make tracing/profiling easier and faster to do for all types of apps (web/Silverlight/desktop).
If you've ever found profiling and optimization intimidating or slow with other tools, then JustTrace might be worth a look.
Don't forget nProf - a prefectly good, freeware profiler.
I have found dotTrace Profiler by JetBrains to be an excellent profiling tool for .NET and their ASP.NET mode is quality.
ANTS Profiler. I haven't used many, but I don't really have any complaints about ANTS. The visualization is really helpful.
AutomatedQA AQTime for timing and SciTech MemProfiler for memory.
If you're looking for something quick, easy, and free, http://code.google.com/p/slimtune/ seems to do the job fine.
I've been working with JetBrains dotTrace for WinForms and Console Apps (not tested on ASP.net yet), and it works quite well:
They recently also added a "Personal License" that is significantly cheaper than the corporate one. Still, if anyone else knows some cheaper or even free ones, I'd like to hear as well :-)
Don't forget the awesome scitech .net memory profiler
It's great for tracking down why your .net app is running out of memory.
I would add that dotTrace's ability to diff memory and performance trace sessions is absolutely invaluable (ANTS may also have a memory diff feature, but I didn't see a performance diff).
Being able to run a profiling session before and after a bug fix or enhancement, then compare the results is incredibly valuable, especially with a mammoth legacy .NET application (as in my case) where performance was never a priority and where finding bottlenecks could be VERY tedious. Doing a before-and-after diff allows you to see the change in call count for each method and the change in duration for each method.
This is helpful not only during code changes, but also if you have an application that uses a different database, say, for each client/customer. If one customer complains of slowness, you can run a profiling session using their database and compare the results with a "fast" database to determine which operations are contributing to the slowness. Of course there are many database-side performance tools, but sometimes I really helps to see the performance metrics from the application side (since that's closer to what the user's actually seeing).
Bottom line: dotTrace works great, and the diff is invaluable.
AQTime is reasonable, but has a bit of a learning curve and isn't as easy to use as the built in one in Team Suite
In the past, I’ve used the profiler that ships with Visual Studio Team System.
The current release of SharpDevelop (3.1.1) has a nice integrated profiler. It's quite fast, and integrates very well into the SharpDevelop IDE and its NUnit runner. Results are displayed in a flexible Tree/List style (use LINQ to create your own selection). Doubleclicking the displayed method jumps directly into the source code.
I've worked with RedGate's profiler in the past. Did the job for me.
Haven't tried it myself, but maybe dotTrace? Their ReSharper application is certainly a good one. Maybe dotTrace is too :)
I doubt that the profiler which comes with Visual Studio Team System is the best profiler, but I have found it to be good enough on many occasions. What specifically do you need beyond what VS offers?
EDIT: Unfortunately it is only available in VS Team System, but if you have access to that it is worth checking out.
The latest version of ANTS memory profiler (I think it's 5) simply rocks!!! I was haunting a leak using WinDbg and SOS since it proved to be the best way before, then I tried ANTS and I got it in minutes. Really a wonderful piece of software.
I would like to add yourkit java and .net profiler, I love it for Java, haven't tried .NET version though.
Unfortunate most of the profilers I tried failed when used with tail calls, most notably ANTS. I just end up writing my own. There is a simple implementation on CodeProject that you can use as a base.
Intel® VTune™ Performance Analyzer for quick sampling
I must bring an amazing tool to your notice which i have used sometime back. AVICode Interceptor Studio. In my previous company we used this wonderful tool to profile the webapplication (This is supposed to be the single largest web application in the world and the largest civilian IT project ever done). The performance team did wonders with the help of this magnificent tool. It is a pain to configure it, but that is a one time activity and i would say it is worth the time. Checkout this page for details.
Thanks,
James
For me SpeedTrace is the best tool on the market because it does not only help you to find bottlenecks inside your applications. It also helps you in troubleshooting scenarios to find out why your application was crashing, your setup did not install, your application hung up, your application performance is sometimes poor depending on the data input, e.g. to identify slow db transactions.
I've been testing Telerik's JustTrace recently and although it is well away from a finished product the guys are going in the right direction.
If Licensing is an issue you could try WINDBG for memory profiling
The NuMega True Time profiler lives on in DevPartner Studio by Micro Focus. It provides line and method level detail for .NET apps requiring only PDBs, no source needed (but it helps.) It can discriminate between algorithmically heavy routines versus those with long I/O waits using our proprietary per thread kernel mode timing driver. Version 10.5 ships with new 64-process support on February 4, 2011. Shameless plug: I work on the DevPartner product line. Follow up at http://www.DevPartner.com for news of the 10.5 launch.
Disclaimer: I am the Product Manager for DevPartner at Micro Focus.
I've found plenty of problems in a big C# app using this.
Usually the problem occurs during startup or shutdown as plugins are being loaded, and big data structures are being created, destroyed, serialized, or deserialized. Often they are created and initialized more than once, and change handlers get added multiple times, further compounding the problem.
In cases like this, the program can be so sluggish that only 2 samples are sufficient to pinpoint the guilty method / function / property call sites.
We selected YourKit Profiler for .NET in my company as it was the best value (price vs. feature). For a small company that wants to have flexible licensing (floating licenses) it was a perfect choice - ANTS was developer seat locket at the time.
Also, it provided us with the ability to attach to the running process which was not possible with dotTrace. Beware though that attaching is not the best option as everything .NET will slow down, but this was the only way to profile .NET applications started by other processes.
Feature wise, ANTS and dotTrace were better - but in the end YourKit was good enough.
If you're on ASP.NET MVC, you can try MVCMiniProfiler (http://benjii.me/2011/07/using-the-mvc-mini-profiler-with-entity-framework/)

Categories

Resources