Is it possible to enumerate all managed threads in C#? Visual Studio seems to be able to do this when you hit a break point while debugging. In the "Threads" window it shows a list of all running threads, including managed names. Does anyone know how it does this?
Debuggers are often able to do things "normal" code can't. You'd probably find that you can do this if you use the profiling/debugging API, but I don't believe you can do it from "normal" .NET code.
This sounds like a duplicate of "How to enumerate threads in .NET using the Name property?" - If so, the short answer is "keep track of your own threads yourself" - i.e. in a List<Thread> or similar.
Take a look at Managed Stack Explorer:
MSE works by quickly attaching to a
process when a stack trace is
requested and the detaching the second
the stack trace has been retrieved.
This way the interference in the
normal operation of the process is
minimized.
Unfortunately, this means it has to done by an external process.
A similar tool is StackDump, which uses MDbg to generate the dump.
You can enumerate threads by making an HTTP GET request to ProcInsp. ProcInsp allows you to get threads with stack traces of CLR processes (either in its UI or as a JSON message). The web API is available at /Process/%PID%/Threads.
Disclaimer: I am the developer of this tool. It is MIT-licensed and free for use.
Related
I am currently health checking an application which experiences UI stuttering during heavy usage.
Using Microsoft Concurrency Visualizer extension for Visual Studio 2015, showed that quite a lot of short-lived threads are created and stopped after ~100ms of execution.
Unfortunately, their displayed callstack is like clr.dll!0x98071 ntdll.dll!0x634fb and I am not quite sure how to extract useful information out of it.
I have no clue what is the purpose of those threads and which part of the code in the application is creating them.
How can I better identify where each one of them gets started?
In the code, I was able to grep a handful of Tasks, another of QueueUserWorkItems, several dozens of plain Thread instantiations, some System.Threading.Timer & System.Timers.Timer, no Reactive Extensions. I put breakpoints for all of them but it seems I am missing some...
I don't think those are from the threadpool because they would be displayed in synchronisation state in concurrency visualizer, instead they just end, and another one with another Id gets created later. But maybe I am misleading.
We also use a few third-party libs and a bunch of JuggerNET generated code, so maybe the origin is not even in the application itself.
I was finally able to find the culprit of those short-lived threads by looking closely at some of the cryptic callstacks, which included for instance:
mmdevapi.dll
wdmaud.drv
avrt.dll
audioses.dll
It led me to thinking that I should double check the sound alert system. It was indeed this one which spawned those threads.
Note:
I will not accept my answer however because I would like someone to share a better process or any kind of tips and tricks for diagnosing unwanted threads origin.
My application (base application is MFC interop with C++/CLI but it also contains a lot of C#, Windows Forms, WPF) has has a handle leak. Shortly after application start I can see the handle count in the task manager grow continuously (at a rate of 10 new handles per second). So I used handles.exe to see what kind of handles they are. I found out that the leaking handles are process handles. And they are process handles to the process of my application.
So I wonder what operations would typically create a handle to the process it runs in. Any idea? Have you ever seen something like this? What else could I do to track down the leak, considering that I can't use debug DLLs and that I can only use tools that can be xcopy deployed.
Update:
I was able to throw windbg and !handle, !htrace at it and found out that the process handles are all created using the following stack traces (ordered by frequency):
0x79f7570b: mscorwks!CorExitProcess+0x00022055
0x79f03edd: mscorwks!GetPrivateContextsPerfCounters+0x0000b6fe
0x79f04b87: mscorwks!GetPrivateContextsPerfCounters+0x0000c3a8
0x79f04b03: mscorwks!GetPrivateContextsPerfCounters+0x0000c324
0x79f919bf: mscorwks!CorExitProcess+0x0003e309
0x79f91b28: mscorwks!CorExitProcess+0x0003e472
0x792d6b4c: mscorlib_ni+0x00216b4c
0x1391a663: +0x1391a663
0x1391a0b1: +0x1391a0b1
0x7a9ea544: System_ni+0x005aa544
0x792a842f: mscorlib_ni+0x001e842f
or
0x7c8106f5: kernel32!CreateThread+0x0000001e
0x79f04bb2: mscorwks!GetPrivateContextsPerfCounters+0x0000c3d3
0x79f04b03: mscorwks!GetPrivateContextsPerfCounters+0x0000c324
0x79f919bf: mscorwks!CorExitProcess+0x0003e309
0x79f91b28: mscorwks!CorExitProcess+0x0003e472
0x792d6b4c: mscorlib_ni+0x00216b4c
0x1391a663: +0x1391a663
0x1391a0b1: +0x1391a0b1
0x7a9ea544: System_ni+0x005aa544
0x792a842f: mscorlib_ni+0x001e842f
or
0x08ec2eba: +0x08ec2eba
0x792b8277: mscorlib_ni+0x001f8277
0x792b8190: mscorlib_ni+0x001f8190
0x792b8040: mscorlib_ni+0x001f8040
0x792b7ff2: mscorlib_ni+0x001f7ff2
0x677e48f3: System_Runtime_Remoting_ni+0x000748f3
0x677e44be: System_Runtime_Remoting_ni+0x000744be
0x677e46ec: System_Runtime_Remoting_ni+0x000746ec
0x677e8408: System_Runtime_Remoting_ni+0x00078408
0x7926eb8d: mscorlib_ni+0x001aeb8d
Now what does that tell me?
The call stacks look wrong. Did you setup the symbol server correctly? .symfix should do the trick in Windbg. Afterwards you should get a better stacktrace.
It looks like part of the code that has this problem is managed so it would make sense to break on DuplicateHandle and OpenProcess and dump the managed call stack there. These two methods are the only ones which could produce a real process handle.
You can declare a breakpoint like this and execute commands when the breakpoint is hit.
In this case the managed stack is printed and then the execution does continue.
bp kernel32!OpenProcess "!ClrStack;g"
Had same issues with a webservice calling COM objects through interop.
I solved this by explicitely calling Marshal.ReleaseComObject against the interop objects I created. No issues after that moment for me.
Hope it helps.
So... are you doing performance counters explicitely (if so, try disabling them to narrow down on the source of the leaks).
Good afternoon,
I inherited some C# code from years ago. I have refactored it a bit to be asynchronous.
Evaluating the impact of my changes on the performance of the CPU, I used Process Explorer to watch, roughly, what my app was doing.
To my surprise, it appears to be doing what Process Explorer reports as I/O. In general, this is related to Disk I/O or Network I/O.
Based on what I can see of the code, I can't figure out an explicit call to either of those 2 I/O sources.
My question is: what is the best way to identify which section of code is causing I/O? We use dotTrace from JetBrains to profile our application, but, from what I can tell, it only handles CPU and Memory performance.
Thanks in advance for any pointers.
Regards,
Eric.
Process Monitor may be your answer. Refer to the following StackOverflow question for more information.
How can I profile file I/O?
Building on that answer, you may be able to search your solution for the filename of any commonly read or written files found with Process Monitor.
The stackshot method, also called random pausing, will find it, if it takes significant time.
If the I/O code is managed, you can load the symbols for the .net framework and set breakpoints in crucial functions (e.g. FileStream constructors etc.)
It involves some guess work but can be informative if you succeed.
In addition to Process Monitor, I find the Resource Monitor on Win7 (also available under 'Performance and Reliability' I think on Vista) very useful for diagnosing I/O-related slowdowns. Switch to the disk view and sort by Read/Write or Total (win7 only). Also keep an eye on the list of files that appear.
Other than that I don't know if I can reproduce it now that it's happened (I've been using this particular application for a week or two now without issue), assuming that I'm running my application in the VS debugger, how should I go about debugging a deadlock after it's happened? I thought I might be able to get at call stacks if I paused the program and hence see where the different threads were when it happened, but clicking pause just threw Visual Studio into a deadlock too till I killed my application.
Is there some way other than browsing through my source tree to find potential problems? Is there a way to get at the call stacks once the problem has occured to see where the problem is? Any other tools/tips/tricks that might help?
What you did was the correct way. If Visual Studio also deadlocks, that happens now and then. It's just bad luck, unless there's some other issue.
You don't have to run the application in the debugger in order to debug it. Run the application normally, and if the deadlock happens, you can attach VS later. Ctrl+Alt+P, select the process, choose debugger type and click attach. Using a different set of debugger types might reduce the risk of VS crashing (especially if you don't debug native code)
A deadlock involves 2 or more threads. You probably know the first one (probably your UI thread) since you noticed the deadlock in your application. Now you only need to find the other one. With knowledge of the architecture, it should be easy to find (e.g. what other threads use the same locks, interact with the UI etc)
If VS doesn't work at all, you can always use windbg. Download here: http://www.microsoft.com/whdc/devtools/debugging/default.mspx
I'd try different approaches in the following order:
First, inspect the code to look for thread-safety violations, making sure that your critical regions don't call other functions that will in turn try to lock a critical region.
Use whatever tool you can get your hands on to visualize thread activity, I use an in-house perl script that parses an OS log we made and graphs all the context switches and shows when a thread gets pre-empted.
If you can't find a good tool, do some logging to see the last threads that were running before the deadlock occurred. This will give you a clue as to where the issue might be caused, it helps if the locking mechanisms have unique names, like if an object has it's own thread, create a dedicated semaphore or mutex just to manage that thread.
I hope this helps. Good luck!
You can use different programs like Intel(R) Parallel Inspector:
http://software.intel.com/en-us/intel-parallel-inspector/
Such programs can show you places in your code with potential deadlocks. However you should pay for it, or use it only evaluation period. Don't know if there is any free tools like this.
Just like anywhere, there're no "Silver bullet" tools to catch all the deadlocks. It is all about the sequence in which different threads aquire resources so your job is to find out where the order was violated. Usually Visual Studio or other debugger will provide stack traces and you will be able to find out where the discrepancy is. DevPartner Studio does provide deadlock analysis but last time I've checked there were too many false positives. Some static analysis tools will find some potential deadlocks too.
Other than that it helps to get the architecture straight to enforce resource aquisition order. For example, layering helps to make sure upper level locks are taken before lower ones but beware of callbacks.
From C#, is it possible to detect the number of context switches that occurred while executing a block of code on a particular thread? Ideally, I'd like to know how many times and what CPU my thread code was scheduled on.
I know I can use tools like Event Tracing for Windows and the associated viewers, but this seemed a bit complicated to get the data I wanted.
Also, tools like Process Explorer make it too hard to tell how many switches occurred as a result of a specific block of code.
Background: I'm trying to test the actual performance of a low level lock primitive in .NET (as a result of some comments on a recent blog post I made.
It sounds like you may be looking for a programmatic solution, but if not, Microsoft's Process Explorer tool will tell you very easily the number of context switches for a particular thread.
Once in the tool, double-click your process, select the Threads tab, and select your thread.
The .NET tab has more specific .NET-related perf data.
I've never done this, but here are a few leads that might help:
The .NET profiler APIs might allow you to hook in? The ICorProfilerCallback interface has RuntimeThreadSuspended and RuntimeThreadResumed callbacks. But a comment on this blog post seems to indicate that they won't get you what you are looking for: "RuntimeThreadSuspended is issued when a thread is being suspended by the runtime, typically in preparation for doing a GC."
There is a "Context Switches/sec" perfmon counter that might help. I haven't looked at this counter specifically, but I'm guessing it operates on Win32 threads and not managed threads. You can use the profiling APIs to get the Win32 thread ID for any given managed thread ID.
Good luck! ;)
It looks like procexp might be using the kernel thread (KTHREAD) or executive thread (ETHREAD) data structures that have a ContextSwitches field on them. It might be possible to get this from managed code.