I need to monitor handle usage on a Windows CE box.
Essentially I want to be able to see handle usage over time to tell if my applications / services are leaking handles (which I believe they are).
Any example code would be great.
While it is not exactly what you are looking for, I can recommend a little tool that we are using called CodeSnitch (http://www.entrek.com/codesnitch.html). It will instrument your code and keep track allocation and de-allocation of resources, including handles. I have used it to clean up several of our applications with great success. You can download a 2 week trial version to try it out.
Related
Is there a way for me to get the amount of memory and processor power needed for my application. I recently had a very unpleasant experience when one of my applications kept freezing the computers on which it was working. This is obviously related to the lack of hardware power, because it works perfectly on the stronger computers that I used for testing purposes, where the application worked perfectly. So my question is - is there a way to calculate the amount of hardware power needed to run the application smoothly?
Almost all of my applications are done in C#, so I would need a method that can work with that type of application.
Thanks
This is obviously related to the lack of hardware power
This entirely depends on what your application is doing. If you are solving problems in a "not so time efficient way", then you can optimize the code.
I would suggest that you analyze your code with a profiler.
This will tell you:
What parts of your code are taking up most RAM/CPU
How much RAM in total did your application need when it peeked
Information about CPU consumption
This is obviously related to the lack of hardware power, because it works perfectly on the
stronger computers that I used for testing purposes,
Whoever set up testing should be fired.
You have to have one set of computers that are similar to the ones the application will run in for testing. That was accepted practice 20 years ago - seems modern times do not care about that.
Seriously, you NEED to have a test set that is representative on your lowest accepted hardware level.
Otherwise - no, sorry, no magic button. Profilers do NOT necessarily help (debugging, profiler may use more memory). Try a profiler. Optimize code. But at the end... you need to have a decent testbed.
I'd argue that this should be checked during software installation. Later, if user was prompted for updating his/her hardware and dismissed the warning, you shouldn't care about that.
If you're using Windows Installer (MSI), you can play with a custom action and use System.Management classes to detect whatever you want.
I've got a for loop I want to parallelize with something like PLINQ's Parallel.ForEach().
The key here is that the C++ library i'm calling to do the computation is decidedly not thread safe, therefore, any plans to parallelize this need to do so across multiple processes.
I was thinking about using WCF to create a "distributor" process to which the "client" and multiple "calculators" could both connect and add/remove items to/from a queue and then the "calculator" sends the results directly back to the client which could update the gui as it receives them. This architecture would allow me to bring as many "calculators" online as I have processors and as I see it even bring them up across multiple computers creating a potential farm of processing power to which all the clients could share.
I'm just wondering if anyone has had any experience doing this and if there are existing application blocks or frameworks that I can use to build this for me. PLINQ does it within the process. is there like a DPLINQ (distributed) or something?
Also if that doesn't exist, does anybody want to give an opinion on my proposed architecture? Any obvious pitfalls? Does anyone think it will work!?!?!?
Sounds like you could be looking for Dryad. It's a Microsoft research project right now, but they do have an "academic release" available. My understanding is that they are also in the process of better productizing it (probably some kind of integration with Azure) for RTM sometime near the end of 2011. Mary Jo Foley covers more about this here.
A long time standard for controlling/dispatching distributed work is MPI. I've only ever used it from C++, but implementations from many languages exist. A quick google suggests that MPI.Net could be a good implementation for .Net!
I am going to be deploying a solution which includes a number of small long running processes which will live on a number of boxes. I wanted to develop a central dashboard for managing these processes and was looking for a good way to do so. I would want to get some counters from the processes and monitor things like memory usage and uptime as well as remotely restart them. In java I would use JMX and I was wondering if there was a similar technology in the .net space. So far I have come across
NetMX
WMI
WMI looks to really be more focused towards unmanaged code. NexMX seems to be ideal but not heavily used. Does anybody have some experiences doing something similar they could share? Any other technologies I should consider?
Never heard of NetMX, version 0.7 might have something to do with it. WMI is quite adroit at monitoring .NET apps as well as unmanaged apps. Fire up Perfmon.exe to see the .NET performance counters at work. Queryable with WMI, experiment with WMI Code Creator.
I know how antivirus detects viruses. I read few aticles:
How do antivirus programs detect viruses?
http://www.antivirusworld.com/articles/antivirus.php
http://www.agusblog.com/wordpress/what-is-a-virus-signature-are-they-still-used-3.htm
http://hooked-on-mnemonics.blogspot.com/2011/01/intro-to-creating-anti-virus-signatures.html
During this one month vacation I'm having. I want to learn & code a simple virus detection program:
So, there are 2-3 ways (from above articles):
Virus Dictionary : Searching for virus signatures
Detecting malicious behavior
I want to take the 2nd approach. I want to start off with simple things.
As a side note, recently I encountered a software named "ThreatFire" for this purpose. It does a pretty good job.
1st thing I don't understand is how can this program inter vent an execution of another between and prompt user about its action. Isnt it something like violation?
How does it scan's memory of other programs? A program is confined to only its virtual space right?
Is C# .NET correct for doing this kind of stuff?
Please post your ideas on how to go about it? Also mention some simple things that I could do.
This happens because the software in question likely has a special driver installed to allow it low level kernel access which allows it to intercept and deny various potentially malicious behavior.
By having the rights that many drivers do, this grants it the ability to scan another processes memory space.
No. C# needs a good chunk of the operating system already loaded. Drivers need to load first.
Learn about driver and kernel level programming. . . I've not done so, so I can't be of more help here.
I think system calls are the way to go, and a lot more doable than actually trying to scan multiple processes' memory spaces. While I'm not a low-level Windows guy, it seems like this can be accomplished using Windows API hooks- tie-ins to the low-level API that can modify system-wide response to a system call. These hooks can be installed as something like a kernel module, and intercept and potentially modify system calls. I found an article on CodeProject that offers more information.
In a machine learning course I took, a group decided to try something similar to what you're describing for a semester project. They used a list of recent system calls made by a program to determine whether or not the executing program was malicious, and the results were promising (think 95% recognition on new samples). In their project, they trained using SVMs on windowed call lists, and used that to determine a good window size. After that, you can collect system call lists from different malicious programs, and either train on the entire list, or find what you consider "malicious activity" and flag it. The cool thing about this approach (aside from the fact that it's based on ML) is that the window size is small, and that many trained eager classifiers (SVM, neural nets) execute quickly.
Anyway, it seems like it could be done without the ML if it's not your style. Let me know if you'd like more info about the group- I might be able to dig it up. Good luck!
Windows provides APIs to do that (generally the involve running at least some of your code in kernel). If you have sufficient privileges, you can also inject a .dll into other process. See http://en.wikipedia.org/wiki/DLL_injection.
When you have the powers described above, you can do that. You are either in kernel space and have access to everything, or inside the target process.
At least for the low-level in-kernel stuff you'd need something more low-level than C#, like C or C++. I'm not sure, but you might be able to do some of the rest things in a C# app.
The DLL injection sounds like the simplest starting point. You're still in user space, and don't have to learn how to live in the kernel world (it's completely different world, really).
Some loose ideas on topic in general:
you can interpose system calls issued by the traced process. It is generally assumed that a process cannot do anything "dangerous" without issuing a system call.
you can intercept its network traffic and see where it connects to, what does it send, what does it receive, which files does it touch, which system calls fail
you can scan its memory and simulate its execution in a sandbox (really hard)
with the system call interposition, you can simulate some responses to the system calls, but really just sandbox the process
you can scan the process memory and extract some general characteristics from it (connects to the network, modifies registry, hooks into Windows, enumerates processes, and so on) and see if it looks malicious
just put the entire thing in a sandbox and see what happens (a nice sandbox has been made for Google Chrome, and it's open source!)
What is the best available tool to monitor the memory usage of my C#/.Net windows service over a long period of time. As far as I know, tools like perfmon can monitor the memory usage over a short period of time, but not graphically over a long period of time. I need trend data over days, not seconds.
To be clear, I want to monitor the memory usage at a fine level of detail over a long time, and have the graph show both the whole time frame and the level of detail. I need a small sampling interval, and a large graph.
Perfmon in my opinion is one of the best tools to do this but make sure you properly configure the sampling interval according to the time you wish to monitor.
For example if you want to monitor a process:
for 1 hour : I would use 1 second intervals (this will generate 60*60 samples)
for 1 day : I would use 30 second intervals (this will generate 2*60*24 samples)
for 1 week : I would use 1 minute intervals (this will generate 60*24*7 samples)
With these sampling intervals Perfmon should have no problem generating a nice graphical output of your counters.
Well I used perfmon, exported the results to a csv and used excel for statistics afterwards. That worked pretty well last time I needed to monitor a process
Playing around with Computer Management (assuming you're running Windows here) and it seems like you can make it monitor a process over time. Go to computer management -> performance logs and alerts and look at the counter/trace logs. Right click on counter logs and add a new log. Now click add object and select memory. Now click add counters and change the "Performance Object" to Process, and select your process.
As good as monitoring the memory is by itself, you're probably thinking of memory profiling to identify leaks or stale objects - http://memprofiler.com/ is a good choice here, but there are plenty of others.
If you want to do something very specific, don't be afraid to write your own WMI-based logger running on a timer - you could get this to email you process statistics, warn when it grows too fast or too high, send it as XML for charting, etc.
If you're familiar with Python, it's pretty easy to write a script for this.
Activestate Python (which is free) exposes the relevant parts of the Win32 API through the win32process module.
You can also check out all win32 related modules or use gotAPI to browse the Python standard libs.
I would recommend using the .Net Memory Validator tool from software verify.
This tool helped me to solve many different issues related to memory management in .Net application I have to work with.
I use more frequently the C++ version but they are quite similar and the fact that you can really see in real-time the type of the objects being allocated will be invaluable to you.
I've used ProcessMonitor if you need something more powerful than perfmon.