Develop Reference

Measuring network traffic in C#

I'd like my application to be able to monitor how much data (I don't need the actual packages, only amount of bytes) that goes in an out of the computer (on a single interface) at all time, and also over time. In other words; I want to be able to show a number displaying current up and down speed (for instance 5kB/s) as well as a graph over the last 5 minutes, and lastly I also want to be able to show a correct amount of data that has gone out/in of the interface in the last hour.
Is this doable in a simple manner? I don't mind using third-party assemblies, or having to write my own wrapper around some win32 api, however; said third-party assemblies must be able to be used with open-source applications and must not cost money.

In fact you may easily get all network traffic using performance counters, but I do not know any way to measure network traffic of your application than opening each packet and reading it. To measure all network traffic, refer to the following:network traffic in c#

Something like DUMeter?
Look here
.

You might want to look at performance counters.
On most of these network counters you can choose the network interface that is to be monitored. You can do the same in .Net code as well.
Here's a CodeProject article that contains a forms app that does basically what you're after: http://www.codeproject.com/Articles/6259/Monitoring-network-speed

Good way to do performance logging (C#)

I am trying to get some detailed performance information from an application my company is developing. Examples of information I am trying to get would be how long a network transaction takes, how much CPU/memory the application is using, how long it takes for a given method to complete, etc.
I have had some failed attempts at this in the past (like trying to measure small time periods by using DateTime.Now). On top of that I don't know much of anything about getting CPU and memory statistics. Are there any good .Net classes, libraries, or frameworks out there that would help me collect this sort of information and/or log it to a text file?

What you are looking for is Performance Counters. For .net you need this Performance Counter.

Performance counters are one way to go, and the System.Diagnostics.Stopwatch class are good foundational places to look for doing this.
With performance counters (beyond those provided) you will need to manage both the infrastructure of tracking the events, as well as reporting the data. The performance counter base classes supply the connection details for hooking up to the event log, but you will need to provide other reporting infrastructure if you need to report the data in another way (such as to a log file, or database).
The stopwatch class is a wrapper around the high performance timer, giving you microsecond or nanosecond resolution depending on the processor or the platform. If you do not need that high of resolution you can use System.DateTime..Now.Ticks to get the current tick count for the processor clock and do differential math with that, giving you millisecond or bettter precision for most operations.
When tracking CPU statistics be aware that multiple processors and multiple cores will complicate any accurate statistics in some cases.
One last caution with performance counters, be aware that not all performance counters are on all machines. For instance ASP.NET counters are not present on a machine which does not have IIS installed, etc.

for a modern opensource library to do performance metrics and monitoring consider using app-metrics.
github: https://github.com/AppMetrics/AppMetrics
website https://www.app-metrics.io

A platform independent open source performance recorder build on top of aspect injector can be found here: https://gitlab.com/hectorjsmith/csharp-performance-recorder. It can be added to any C# project. Instructions on how to use can be found in the README in Gitlab.
Benefits
It's aspect oriented, meaning you don't have to litter your code with DateTime.Now - you just annotate the appropriate methods
It takes care of pretty printing the results - you can do with the what you like e.g. printing them to a file
Notes
This performance recorder is focused on the timing of methods only. It doesn't cover CPU or memory usage.

For cpu/memory use performance counters. For more specific information about specific methods (or lines of code) and specific objects, use a profiler. red-gate makes a great profiler.
http://www.red-gate.com/products/ants_performance_profiler

What is the best way to debug performance problems?

I'm writing a plug-in for another program in C#.NET, and am having performance issues where commands take a lot longer then I would. The plug-in reacts to events in the host program, and also depends on utility methods of the the host program SDK. My plug-in has a lot of recursive functions because I'm doing a lot of reading and writing to a tree structure. Plus I have a lot of event subscriptions between my plugin and the host application, as well as event subscriptions between classes in my plug-in.
How can I figure out what is taking so long for a task to complete? I can't use regular breakpoint style debugging, because it's not that it doesn't work it's just that it's too slow. I have setup a static "LogWriter" class that I can reference from all my classes that will allow me to write out timestamped lines to a log file from my code. Is there another way? Does visual studio keep some kind of timestamped log that I could use instead? Is there someway to view the call stack after the application has closed?

You need to use profiler. Here link to good one: ANTS Performance Profiler.
Update: You can also write messages in control points using Debug.Write. Then you need to load DebugView application that displays all your debug string with precise time stamp. It is freeware and very good for quick debugging and profiling.

My Profiler List includes ANTS, dotTrace, and AQtime.
However, looking more closely at your question, it seems to me that you should do some unit testing at the same time you're doing profiling. Maybe start by doing a quick overall performance scan, just to see which areas need most attention. Then start writing some unit tests for those areas. You can then run the profiler while running those unit tests, so that you'll get consistent results.

In my experience, the best method is also the simplest. Get it running, and while it is being slow, hit the "pause" button in the IDE. Then make a record of the call stack. Repeat this several times. (Here's a more detailed example and explanation.)
What you are looking for is any statement that appears on more than one stack sample that isn't strictly necessary. The more samples it appears on, the more time it takes. The way to tell if the statement is necessary is to look up the stack, because that tells you why it is being done.
Anything that causes a significant amount of time to be consumed will be revealed by this method, and recursion does not bother it.
People seem to tackle problems like this in one of two ways:
Try to get good measurements before doing anything.
Just find something big that you can get rid of, rip it out, and repeat.
I prefer the latter, because it's fast, and because you don't have to know precisely how big a tumor is to know it's big enough to remove. What you do need to know is exactly where it is, and that's what this method tells you.

Sounds like you want a code 'profiler'. http://en.wikipedia.org/wiki/Code_profiler#Use_of_profilers
I'm unfamiliar with which profilers are the best for C#, but I came across this link after a quick google which has a list of free open-source offerings. I'm sure someone else will know which ones are worth considering :)
http://csharp-source.net/open-source/profilers

Despite the title of this topic I must argue that the "best" way is subjective, we can only suggest possible solutions.
I have had experience using Redgate ANTS Performance Profiler which will show you where the bottlenecks are in your application. It's definitely worth checking out.

Visual Studio Team System has a profiler baked in, its far from perfect, but for simple applications you can kind of get it to work.
Recently I have had the most success with EQATECs free profiler, or rolling my own tiny profiling class where needed.
Also, there have been quite a few questions about profilers in that past see: http://www.google.com.au/search?hl=en&q=site:stackoverflow.com+.net+profiler&btnG=Google+Search&meta=&aq=f&oq=

Don't ever forget Rico Mariani's advice on how to carry out a good perf investigation.

You can also use performance counter for asp.net applications.

What's the best tool to track a process's memory usage over a long period of time in Windows?

What is the best available tool to monitor the memory usage of my C#/.Net windows service over a long period of time. As far as I know, tools like perfmon can monitor the memory usage over a short period of time, but not graphically over a long period of time. I need trend data over days, not seconds.
To be clear, I want to monitor the memory usage at a fine level of detail over a long time, and have the graph show both the whole time frame and the level of detail. I need a small sampling interval, and a large graph.

Perfmon in my opinion is one of the best tools to do this but make sure you properly configure the sampling interval according to the time you wish to monitor.
For example if you want to monitor a process:
for 1 hour : I would use 1 second intervals (this will generate 60*60 samples)
for 1 day : I would use 30 second intervals (this will generate 2*60*24 samples)
for 1 week : I would use 1 minute intervals (this will generate 60*24*7 samples)
With these sampling intervals Perfmon should have no problem generating a nice graphical output of your counters.

Well I used perfmon, exported the results to a csv and used excel for statistics afterwards. That worked pretty well last time I needed to monitor a process

Playing around with Computer Management (assuming you're running Windows here) and it seems like you can make it monitor a process over time. Go to computer management -> performance logs and alerts and look at the counter/trace logs. Right click on counter logs and add a new log. Now click add object and select memory. Now click add counters and change the "Performance Object" to Process, and select your process.

As good as monitoring the memory is by itself, you're probably thinking of memory profiling to identify leaks or stale objects - http://memprofiler.com/ is a good choice here, but there are plenty of others.
If you want to do something very specific, don't be afraid to write your own WMI-based logger running on a timer - you could get this to email you process statistics, warn when it grows too fast or too high, send it as XML for charting, etc.

If you're familiar with Python, it's pretty easy to write a script for this.
Activestate Python (which is free) exposes the relevant parts of the Win32 API through the win32process module.
You can also check out all win32 related modules or use gotAPI to browse the Python standard libs.

I would recommend using the .Net Memory Validator tool from software verify.
This tool helped me to solve many different issues related to memory management in .Net application I have to work with.
I use more frequently the C++ version but they are quite similar and the fact that you can really see in real-time the type of the objects being allocated will be invaluable to you.

I've used ProcessMonitor if you need something more powerful than perfmon.

C# How can I determine where the slow parts of my code are?

I've not be coding long so I'm not familiar with which technique is quickest so I was wondering if there was a way to do this in VS or with a 3rd party tool?
Thanks

Profilers are great for measuring.
But your question was "How can I determine where the slow parts of my code are?".
That is a different problem. It is diagnosis, not measurement.
I know this is not a popular view, but it's true.
It is like a business that is trying to cut costs.
One approach (top down) is to measure the overall finances, then break it down by categories and departments, and try to guess what could be eliminated. That is measurement.
Another approach (bottom up) is to walk in at random into an office, pick someone at random, and ask them what they are doing at that moment and (importantly) why, in detail.
Do this more than once.
That is what Harry Truman did at the outbreak of WW2, in the US defense industry, and immediately uncovered massive fraud and waste, by visiting several sites. That is diagnosis.
In code you can do this in a very simple way: "Pause" it and ask it why it is spending that particular cycle. Usually the call stack tells you why, in detail.
Do this more than once.
This is sampling. Some profilers sample the call stack. But then for some reason they insist on summarizing time spent in each function, inclusive and exclusive. That is like summarizing by department in business, inclusive and exclusive.
It loses the information you need, which is the fine-grain detail that tells if the cycles are necessary.
To answer your question:
Just pause your program several times, and capture the call stack each time. If your code is very slow, the wasteful function calls will be on nearly every stack. They will point with precision to the "slow parts of your code".
ADDED: RedGate ANTS is getting there. It can give you cost-by-line, and it is quite spiffy. So if you're in .NET, and can spare 3 figures, and don't mind waiting around to install & learn it, it can tell you much of what your Pause key can tell you, and be much more pretty about it.

Profiling.
RedGate has a product.
JetBrains has a product.

I've used ANTS Profiler and I can join the others with recommendation.
The price is NEGLIGIBLE when you compare it with the amount of dev hours it will save you.
I you're developer for a living, and your company won't buy it for you, either change the company or buy it for yourself.

For profiling large complex UI applications then you often need a set of tools and approaches. I'll outline the approach and tools I used recently on a project to improve the performance of a .Net 2.0 UI application.
First of all I interviewed users and worked through the use cases myself to come up with a list of target use cases that highlighted the systems worse performing areas. I.e. I didn't want to spend n man days optimising a feature that was hardly ever used but very slow. I would want to spend time, however, optimising a feature that was a little bit sluggish but invoked a 1000 times a day, etc.
Once the candidate use cases were identified I instrumented my code with my own light weight logging class (I used some high performance timers and a custom logging solution because a needed sub-millisecond accuracy). You might, however, be able to get away with log4net and time stamps. The reason I instrumented code is that it is sometimes easier to read your own logs rather than the profiler's output. I needed both for a variety of reasons (e.g. measuring .Net user control layouts is not always straightforward using the profiler).
I then ran my instrumented code with the ANTS profiler and profiled the use case. By combining the ANTS profile and my own log files I was very quickly able to discover problems with our application.
We also profiled the server as well as the UI and were able to work out breakdowns for time spent in the UI, time spent on the wire, time spent on the server etc.
Also worth noting is that 1 run isn't enough, and the 1st run is usually worth throwing away. Let me explain: PC load, network traffic, JIT compilation status etc can all affect the time a particular operation will take. A simple strategy is to measure an operation n times (say 5), throw away the slowest and fastest run, the analyse the remianing profiles.

Eqatec profiler is a cute small profiler that is free and easy to use. It probably won't come anywhere near the "wow" factor of Ants profiler in terms of features but it still is very cool IMO and worth a look.

Use a profiler. ANTS costs money but is very nice.

i just set breakpoints, visual will tell you how many ms between breakpoint has passed. so you can find it manually.

ANTS Profiler is very good.

If you don't want to pay, the newer VS verions come with a profiler, but to be honest it doesn't seem very good. ATI/AMD make a free profiler... but its not very user friendly (to me, I couldn't get any useful info out of it).
The advice I would give is to time function calls yourself with code. If they are fast and you do not have a high-precision timer or the calls vary in slowness for a number of reasons (e.g. every x calls building some kind of cache), try running each one x10000 times or something, then dividing the result accordingly. This may not be perfect for some sections of code, but if you are unable to find a good, free, 3rd party solution, its pretty much what's left unless you want to pay.

Yet another option is Intel's VTune.