I'm trying to find out how much memory my own .Net server process is using (for monitoring and logging purposes).
I'm using:
Process.GetCurrentProcess().PrivateMemorySize64
However, the Process object has several different properties that let me read the memory space used:
Paged, NonPaged, PagedSystem, NonPagedSystem, Private, Virtual, WorkingSet
and then the "peaks": which i'm guessing just store the maximum values these last ones ever took.
Reading through the MSDN definition of each property hasn't proved too helpful for me. I have to admit my knowledge regarding how memory is managed (as far as paging and virtual goes) is very limited.
So my question is obviously "which one should I use?", and I know the answer is "it depends".
This process will basically hold a bunch of lists in memory of things that are going on, while other processes communicate with it and query it for stuff. I'm expecting the server where this will run on to require lots of RAM, and so i'm querying this data over time to be able to estimate RAM requirements when compared to the sizes of the lists it keeps inside.
So... Which one should I use and why?
If you want to know how much the GC uses try:
GC.GetTotalMemory(true)
If you want to know what your process uses from Windows (VM Size column in TaskManager) try:
Process.GetCurrentProcess().PrivateMemorySize64
If you want to know what your process has in RAM (as opposed to in the pagefile) (Mem Usage column in TaskManager) try:
Process.GetCurrentProcess().WorkingSet64
See here for more explanation on the different sorts of memory.
OK, I found through Google the same page that Lars mentioned, and I believe it's a great explanation for people that don't quite know how memory works (like me).
http://shsc.info/WindowsMemoryManagement
My short conclusion was:
Private Bytes = The Memory my process has requested to store data. Some of it may be paged to disk or not. This is the information I was looking for.
Virtual Bytes = The Private Bytes, plus the space shared with other processes for loaded DLLs, etc.
Working Set = The portion of ALL the memory of my process that has not been paged to disk. So the amount paged to disk should be (Virtual - Working Set).
Thanks all for your help!
If you want to use the "Memory (Private Working Set)" as shown in Windows Vista task manager, which is the equivalent of Process Explorer "WS Private Bytes", here is the code. Probably best to throw this infinite loop in a thread/background task for real-time stats.
using System.Threading;
using System.Diagnostics;
//namespace...class...method
Process thisProc = Process.GetCurrentProcess();
PerformanceCounter PC = new PerformanceCounter();
PC.CategoryName = "Process";
PC.CounterName = "Working Set - Private";
PC.InstanceName = thisProc.ProcessName;
while (true)
{
String privMemory = (PC.NextValue()/1000).ToString()+"KB (Private Bytes)";
//Do something with string privMemory
Thread.Sleep(1000);
}
To get the value that Task Manager gives, my hat's off to Mike Regan's solution above. However, one change: it is not: perfCounter.NextValue()/1000; but perfCounter.NextValue()/1024; (i.e. a real kilobyte). This gives the exact value you see in Task Manager.
Following is a full solution for displaying the 'memory usage' (Task manager's, as given) in a simple way in your WPF or WinForms app (in this case, simply in the title). Just call this method within the new Window constructor:
private void DisplayMemoryUsageInTitleAsync()
{
origWindowTitle = this.Title; // set WinForms or WPF Window Title to field
BackgroundWorker wrkr = new BackgroundWorker();
wrkr.WorkerReportsProgress = true;
wrkr.DoWork += (object sender, DoWorkEventArgs e) => {
Process currProcess = Process.GetCurrentProcess();
PerformanceCounter perfCntr = new PerformanceCounter();
perfCntr.CategoryName = "Process";
perfCntr.CounterName = "Working Set - Private";
perfCntr.InstanceName = currProcess.ProcessName;
while (true)
{
int value = (int)perfCntr.NextValue() / 1024;
string privateMemoryStr = value.ToString("n0") + "KB [Private Bytes]";
wrkr.ReportProgress(0, privateMemoryStr);
Thread.Sleep(1000);
}
};
wrkr.ProgressChanged += (object sender, ProgressChangedEventArgs e) => {
string val = e.UserState as string;
if (!string.IsNullOrEmpty(val))
this.Title = string.Format(#"{0} ({1})", origWindowTitle, val);
};
wrkr.RunWorkerAsync();
}`
Is this a fair description? I'd like to share this with my team so please let me know if it is incorrect (or incomplete):
There are several ways in C# to ask how much memory my process is using.
Allocated memory can be managed (by the CLR) or unmanaged.
Allocated memory can be virtual (stored on disk) or loaded (into RAM pages)
Allocated memory can be private (used only by the process) or shared (e.g. belonging to a DLL that other processes are referencing).
Given the above, here are some ways to measure memory usage in C#:
1) Process.VirtualMemorySize64(): returns all the memory used by a process - managed or unmanaged, virtual or loaded, private or shared.
2) Process.PrivateMemorySize64(): returns all the private memory used by a process - managed or unmanaged, virtual or loaded.
3) Process.WorkingSet64(): returns all the private, loaded memory used by a process - managed or unmanaged
4) GC.GetTotalMemory(): returns the amount of managed memory being watched by the garbage collector.
Working set isn't a good property to use. From what I gather, it includes everything the process can touch, even libraries shared by several processes, so you're seeing double-counted bytes in that counter. Private memory is a much better counter to look at.
I'd suggest to also monitor how often pagefaults happen. A pagefault happens when you try to access some data that have been moved from physical memory to swap file and system has to read page from disk before you can access this data.
Related
I've been using the methodology outlined by Shivprasad Koirala to check for memory leaks from code running inside a C# application (VoiceAttack). It basically involves using the Performance Monitor to track an application's private bytes as well as bytes in all heaps and compare these counters to assess if there is a leak and what type (managed/unmanaged). Ideally I need to test outside of Visual Studio, which is why I'm using this method.
The following portion of code generates the below memory profile (bear in mind the code has a little different format compared to Visual Studio because this is a function contained within the main C# application):
public void main()
{
string FilePath = null;
using (FileDialog myFileDialog = new OpenFileDialog())
{
myFileDialog.Title = "this is the title";
myFileDialog.FileName = "testFile.txt";
myFileDialog.Filter = "txt files (*.txt)|*.txt|All files (*.*)|*.*";
myFileDialog.FilterIndex = 1;
if (myFileDialog.ShowDialog() == DialogResult.OK)
{
FilePath = myFileDialog.FileName;
var extension = Path.GetExtension(FilePath);
var compareType = StringComparison.InvariantCultureIgnoreCase;
if (extension.Equals(".txt", compareType) == false)
{
FilePath = null;
VA.WriteToLog("Selected file is not a text file. Action canceled.");
}
else
VA.WriteToLog(FilePath);
}
else
VA.WriteToLog("No file selected. Action canceled.");
}
VA.WriteToLog("done");
}
You can see that after running this code the private bytes don't come back to the original count and the bytes in all heaps are roughly constant, which implies that there is a portion of unmanaged memory that was not released. Running this same inline function a few times consecutively doesn't cause further increases to the maximum observed private bytes or the unreleased memory. Once the main C# application (VoiceAttack) closes all the related memory (including the memory for the above code) is released. The bad news is that under normal circumstances the main application may be kept running indefinitely by the user, causing the allocated memory to remain unreleased.
For good measure I threw this same code into VS (with a pair of Thread.Sleep(5000) added before and after the using block for better graphical analysis) and built an executable to track with the Performance Monitor method, and the result is the same. There is an initial unmanaged memory jump for the OpenFileDialog and the allocated unmanaged memory never comes back down to the original value.
Does the memory and leak tracking methodology outlined above make sense? If YES, is there anything that can be done to properly release the unmanaged memory?
Does the memory and leak tracking methodology outlined above make sense?
No. You shouldn't expect unmanaged committed memory (Private Bytes) always be released. For instance processes have an unmanaged heap, which is managed to allow for subsequent allocations. And since Windows can page your committed memory, it isn't critical to minimize each processes committed memory.
If repeated calls don't increase memory use, you don't have a memory leak, you have delayed initialization. Some components aren't initialized until you use them, so their memory usage isn't being taken into account when you establish your baseline.
When I sum the WorkingSet64 property of a collection of processes generated from Process.GetProcesses(), the sum is less than the total physical memory usage indicated by GlobalMemoryStatusEx.
When I run an operation on a process (for example, to load a large file or inject code, etc), the physical memory of the system jumps up, but the WorkingSet64 sum does not track this.
Is there a way to acquire the actual, non-shared physical memory usage of a particular process?
You can get the process’s private working set.
stackoverflow answer:
“The private working set is the amount of memory used by a process that cannot be shared among other processes, while working set includes the memory shared by other processes.”
You can use a ProcessCounter to access this value.
Process thisProc = Process.GetCurrentProcess();
PerformanceCounter PC = new PerformanceCounter();
PC.CategoryName = "Process";
PC.CounterName = "Working Set - Private";
PC.InstanceName = thisProc.ProcessName;
int privateMemory = PC.Next()/1000; //this will be in KB.
I'm developing an application (.NET 4.0, C#) that:
1. Scans file system.
2. Opens and reads some files.
The app will work in background and should have low impact on the disk usage. It shouldn't bother users if they are doing their usual tasks and the disk usage is high. And vice versa, the app can go faster if nobody is using the disk.
The main issue is I don't know real amount and size of I/O operations because of using API (mapi32.dll) to read files. If I ask API to do something I don't know how many bytes it reads to handle my response.
So the question is how to monitor and manage the disk usage? Including file system scanning and files reading...
Check performance counters that are used by standard Performance Monitor tool? Or any other ways?
Using the System.Diagnostics.PerformanceCounter class, attach to the PhysicalDisk counter related to the drive that you are indexing.
Below is some code to illustrate, although its currently hard coded to the "C:" drive. You will want to change "C:" to whichever drive your process is scanning. (This is rough sample code only to illustrate the existence of performance counters - don't take it as providing accurate information - should always be used as a guide only. Change for your own purpose)
Observe the % Idle Time counter which indicates how often the drive is doing anything.
0% idle means the disk is busy, but does not necessarily mean that it is flat-out and cannot transfer more data.
Combine the % Idle Time with Current Disk Queue Length and this will tell you if the drive is getting so busy that it cannot service all the requests for data. As a general guideline, anything over 0 means the drive is probably flat-out busy and anything over 2 means the drive is completely saturated. These rules apply to both SSD and HDD fairly well.
Also, any value that you read is an instantaneous value at a point in time. You should do a running average over a few results, e.g. take a reading every 100ms and average 5 readings before using the information from the result to make a decision (i.e., waiting until the counters settle before making your next IO request).
internal DiskUsageMonitor(string driveName)
{
// Get a list of the counters and look for "C:"
var perfCategory = new PerformanceCounterCategory("PhysicalDisk");
string[] instanceNames = perfCategory.GetInstanceNames();
foreach (string name in instanceNames)
{
if (name.IndexOf("C:") > 0)
{
if (string.IsNullOrEmpty(driveName))
driveName = name;
}
}
_readBytesCounter = new PerformanceCounter("PhysicalDisk",
"Disk Read Bytes/sec",
driveName);
_writeBytesCounter = new PerformanceCounter("PhysicalDisk",
"Disk Write Bytes/sec",
driveName);
_diskQueueCounter = new PerformanceCounter("PhysicalDisk",
"Current Disk Queue Length",
driveName);
_idleCounter = new PerformanceCounter("PhysicalDisk",
"% Idle Time",
driveName);
InitTimer();
}
internal event DiskUsageResultHander DiskUsageResult;
private void InitTimer()
{
StopTimer();
_perfTimer = new Timer(_updateResolutionMillisecs);
_perfTimer.Elapsed += PerfTimerElapsed;
_perfTimer.Start();
}
private void PerfTimerElapsed(object sender, ElapsedEventArgs e)
{
float diskReads = _readBytesCounter.NextValue();
float diskWrites = _writeBytesCounter.NextValue();
float diskQueue = _diskQueueCounter.NextValue();
float idlePercent = _idleCounter.NextValue();
if (idlePercent > 100)
{
idlePercent = 100;
}
if (DiskUsageResult != null)
{
var stats = new DiskUsageStats
{
DriveName = _readBytesCounter.InstanceName,
DiskQueueLength = (int)diskQueue,
ReadBytesPerSec = (int)diskReads,
WriteBytesPerSec = (int)diskWrites,
DiskUsagePercent = 100 - (int)idlePercent
};
DiskUsageResult(stats);
}
}
A long term ago Microsoft Research published a paper on this (sorry I can’t remember the url).
From what I recall:
The program started off doing very few "work items".
They measured how long it took for each of their "work item".
After running for a bit, they could work out how fast an "work item" was with no load on the system.
From then on, if the "work item" were fast (e.g. no other programmers making requests), they made more requests, otherwise they backed-off
The basic ideal is:
“if they are slowing me down, then I
must be slowing them down, so do less
work if I am being slowed down”
Something to ponder: what if there are other processes which follow the same (or a similar) strategy? Which one would run during the "idle time"? Would the other processes get a chance to make use of the idle time at all?
Obviously this can't be done correctly unless there is some well-known OS mechanism for fairly dividing resources during idle time. In windows, this is done by calling SetPriorityClass.
This document about I/O prioritization in Vista seems to imply that IDLE_PRIORITY_CLASS will not really lower the priority of I/O requests (though it will reduce the scheduling priority for the process). Vista added new PROCESS_MODE_BACKGROUND_BEGIN and PROCESS_MODE_BACKGROUND_END values for that.
In C#, you can normally set the process priority with the Process.PriorityClass property. The new values for Vista are not available though, so you'll have to call the Windows API function directly. You can do that like this:
[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
public static extern bool SetPriorityClass(IntPtr handle, uint priorityClass);
const uint PROCESS_MODE_BACKGROUND_BEGIN = 0x00100000;
static void SetBackgroundMode()
{
if (!SetPriorityClass(new IntPtr(-1), PROCESS_MODE_BACKGROUND_BEGIN))
{
// handle error...
}
}
I did not test the code above. Don't forget that it can only work on Vista or better. You'll have to use Environment.OSVersion to check for earlier operating systems and implement a fall-back strategy.
See this question and this also for related queries. I would suggest for a simple solution just querying for the current disk & CPU usage % every so often, and only continue with the current task when they are under a defined threshold. Just make sure your work is easily broken into tasks, and that each task can be easily & efficiently start/stopped.
Check if the screensaver is running ? Good indication that the user is away from the keyboard
I am trying to get the memory usage of my application but for some reason i am getting different size than in task manager
I am using:
Task manager shows that my application occupies 45mb , while when i pulling it in my code i get 85mb
how can i get the same size as in task manager (without using wmi)
Presumably you're looking at the wrong column in "Task manager" or using the wrong property in Process class..
I guess you're looking for WorkingSet64 not PrivateMemorySize64. PrivateMemorySize64 is the amount of virtual memory allocated for the process, not the physical memory. For physical memory use WorkingSet64.
Also, you need to call process.Refresh() before accessing any of the dynamic properties in process class as it is heavily cached.
process.Refresh();
_data.MemoryUsed = (process.WorkingSet64).ConvertBytesToMegabytes().ToString(CultureInfo.InvariantCulture);
None of the above is working for me. I have therefore found this solution
How to get application memory usage as shown in Task Manager?
by Hans Passant which is doing a great job.
string prcName = Process.GetCurrentProcess().ProcessName;
var counter_Exec = new PerformanceCounter("Process", "Working Set - Private", prcName);
double dWsp_Exec = (double)counter_Exec.RawValue / 1024.0; <---that is the value in KB
Also, I sometimesfound a minor different (<10%) which could be related to a different update.
The only minor drawback is that the first instruction take a long time (5") for accumulating data so it might be handled in a thread.
Try once with the following code,may be it will help
PerformanceCounter PC = new PerformanceCounter();
PC.CategoryName = "ServerProcess";
PC.CounterName = "Working Set - Private";
PC.InstanceName = JSP[0].ProcessName; //Process
RAM_memorysize = PC.NextValue(); //float RAM_memorysize;
PC.Close();
PC.Dispose();
I'm developing an application (.NET 4.0, C#) that:
1. Scans file system.
2. Opens and reads some files.
The app will work in background and should have low impact on the disk usage. It shouldn't bother users if they are doing their usual tasks and the disk usage is high. And vice versa, the app can go faster if nobody is using the disk.
The main issue is I don't know real amount and size of I/O operations because of using API (mapi32.dll) to read files. If I ask API to do something I don't know how many bytes it reads to handle my response.
So the question is how to monitor and manage the disk usage? Including file system scanning and files reading...
Check performance counters that are used by standard Performance Monitor tool? Or any other ways?
Using the System.Diagnostics.PerformanceCounter class, attach to the PhysicalDisk counter related to the drive that you are indexing.
Below is some code to illustrate, although its currently hard coded to the "C:" drive. You will want to change "C:" to whichever drive your process is scanning. (This is rough sample code only to illustrate the existence of performance counters - don't take it as providing accurate information - should always be used as a guide only. Change for your own purpose)
Observe the % Idle Time counter which indicates how often the drive is doing anything.
0% idle means the disk is busy, but does not necessarily mean that it is flat-out and cannot transfer more data.
Combine the % Idle Time with Current Disk Queue Length and this will tell you if the drive is getting so busy that it cannot service all the requests for data. As a general guideline, anything over 0 means the drive is probably flat-out busy and anything over 2 means the drive is completely saturated. These rules apply to both SSD and HDD fairly well.
Also, any value that you read is an instantaneous value at a point in time. You should do a running average over a few results, e.g. take a reading every 100ms and average 5 readings before using the information from the result to make a decision (i.e., waiting until the counters settle before making your next IO request).
internal DiskUsageMonitor(string driveName)
{
// Get a list of the counters and look for "C:"
var perfCategory = new PerformanceCounterCategory("PhysicalDisk");
string[] instanceNames = perfCategory.GetInstanceNames();
foreach (string name in instanceNames)
{
if (name.IndexOf("C:") > 0)
{
if (string.IsNullOrEmpty(driveName))
driveName = name;
}
}
_readBytesCounter = new PerformanceCounter("PhysicalDisk",
"Disk Read Bytes/sec",
driveName);
_writeBytesCounter = new PerformanceCounter("PhysicalDisk",
"Disk Write Bytes/sec",
driveName);
_diskQueueCounter = new PerformanceCounter("PhysicalDisk",
"Current Disk Queue Length",
driveName);
_idleCounter = new PerformanceCounter("PhysicalDisk",
"% Idle Time",
driveName);
InitTimer();
}
internal event DiskUsageResultHander DiskUsageResult;
private void InitTimer()
{
StopTimer();
_perfTimer = new Timer(_updateResolutionMillisecs);
_perfTimer.Elapsed += PerfTimerElapsed;
_perfTimer.Start();
}
private void PerfTimerElapsed(object sender, ElapsedEventArgs e)
{
float diskReads = _readBytesCounter.NextValue();
float diskWrites = _writeBytesCounter.NextValue();
float diskQueue = _diskQueueCounter.NextValue();
float idlePercent = _idleCounter.NextValue();
if (idlePercent > 100)
{
idlePercent = 100;
}
if (DiskUsageResult != null)
{
var stats = new DiskUsageStats
{
DriveName = _readBytesCounter.InstanceName,
DiskQueueLength = (int)diskQueue,
ReadBytesPerSec = (int)diskReads,
WriteBytesPerSec = (int)diskWrites,
DiskUsagePercent = 100 - (int)idlePercent
};
DiskUsageResult(stats);
}
}
A long term ago Microsoft Research published a paper on this (sorry I can’t remember the url).
From what I recall:
The program started off doing very few "work items".
They measured how long it took for each of their "work item".
After running for a bit, they could work out how fast an "work item" was with no load on the system.
From then on, if the "work item" were fast (e.g. no other programmers making requests), they made more requests, otherwise they backed-off
The basic ideal is:
“if they are slowing me down, then I
must be slowing them down, so do less
work if I am being slowed down”
Something to ponder: what if there are other processes which follow the same (or a similar) strategy? Which one would run during the "idle time"? Would the other processes get a chance to make use of the idle time at all?
Obviously this can't be done correctly unless there is some well-known OS mechanism for fairly dividing resources during idle time. In windows, this is done by calling SetPriorityClass.
This document about I/O prioritization in Vista seems to imply that IDLE_PRIORITY_CLASS will not really lower the priority of I/O requests (though it will reduce the scheduling priority for the process). Vista added new PROCESS_MODE_BACKGROUND_BEGIN and PROCESS_MODE_BACKGROUND_END values for that.
In C#, you can normally set the process priority with the Process.PriorityClass property. The new values for Vista are not available though, so you'll have to call the Windows API function directly. You can do that like this:
[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
public static extern bool SetPriorityClass(IntPtr handle, uint priorityClass);
const uint PROCESS_MODE_BACKGROUND_BEGIN = 0x00100000;
static void SetBackgroundMode()
{
if (!SetPriorityClass(new IntPtr(-1), PROCESS_MODE_BACKGROUND_BEGIN))
{
// handle error...
}
}
I did not test the code above. Don't forget that it can only work on Vista or better. You'll have to use Environment.OSVersion to check for earlier operating systems and implement a fall-back strategy.
See this question and this also for related queries. I would suggest for a simple solution just querying for the current disk & CPU usage % every so often, and only continue with the current task when they are under a defined threshold. Just make sure your work is easily broken into tasks, and that each task can be easily & efficiently start/stopped.
Check if the screensaver is running ? Good indication that the user is away from the keyboard