Related
I have a client/server application written in C#/.NET 3.5 that I want to do a bit of performance testing on. I've been looking for a generic framework to help me but not had much luck. I would like something that can manage a set of clients and perform random actions for them based on some settings. I would also then like to record some data relating to this to help me work outsome rough thresholds for my system, e.g. I can support n users performing x actions per second.
I would write code specific to my application to perform tasks such as:
Login/logout a client.
Send messages to the server to perform various actions.
Record acknowledgements and other messages from the server.
Measure statistics specific to the system.
I'm hoping the framework will then be able to take a set of parameters to describe a testing scenario such as:
Number of clients logged in at a given time.
Perform a given number of actions per second for each client.
It would then run the scenario, manage and track all of the users and actions and collate all of the data. (This is the boring bit I'm trying to avoid coding myself...) Ideally it would have some general measurements built in, e.g. time between sending a message and receiving a response, but I could code them myself if not.
I don't want to do any profiling of my code; I can always attach a profiler whilst running these tests later on. Instead I want to make some rough conclusions about my system, i.e. how many users can I throw at it before it breaks. (If there is a better term for this than 'performance testing' please let me know... Stress testing maybe?)
I realise I'm not giving very many specifics about the system here. It strikes me as a fairly general situation - I'm sure there are lots of client/server systems out there that people need to do similar tests on. I've found lots of web based frameworks to do similar things but they seem to be pretty web ingrained and don't lend themselves easily to non-HTTP based systems.
Anyone know of anything that might help? My searching hasn't found anything yet. I should point out that I'm stuck with Visual Studio 2008 Professional for the foressable future so if 2010 can do this it's out of bounds for me. I guess it doesn't have to be a .NET framework provided I can still plugin my .NET code fairly easily.
EDIT To be clear my application isn't a website, it's a Windows Forms client application that connects via a custom protocol to a .NET service. I can write code to perform the relevant client actions, I just need a framework to put it in.
The keyword you are looking for is "Distributed testing".
Smart Bear have a product called TestComplete which supports distributed testing. I don't think it can run multiple instances of your client on a single machine though (maybe it can, but I guess it's not a good idea any way since it would impact the performance results).
They also have an open source project called LoadUI, it is built to integrate with SoapUI, however you might be able to hook it up to your own client-test tool. I have no idea how much effort that would cost.
These are the tools I know of, but there are many more distributed testing tools out there. While most are indeed for web-bases testing, they often are extensible enough to simply kick off a different (GUI-based) testing framework (my favorite is QAliber) which runs the tests on your client app.
To my knowledge, such a framework does not currently exist (and it is likely it won't exist because everyones scenarios are so different). Because "Performance" means different things to different people and different projects, you will usually have to roll your own. Thankfully, with the advent of .NET4 and the TPL this has become easier than ever before.
There are three aspects a Performance test needs to cover:
Define "Load"
Measure "Performance"
Evaluate the Results
As you can imagine, both of these are totally lax definitions. Load could be measured in Requests per Second, Logged In Users etc... Performance could be measured in Response Time, Memory Usage... you get the picture.
So your first step is to define a "unit load", that you can easily scale up. For example you could create a ClientUnitLoad class, that simulates a client and periodically performs actions against your service. The TPL makes it easy to scale your ClientUnitLoad (except you are bound by hardware constraints of course).
The next step is to measure Performance. How you measure heavily depends on the metric you want to collect: Simple Stopwatching is fine 90% of the cases but can only measure time. Building a custom tracing and metrics collection infrastructure is definitely worth it, as you will also need it in production to verify your system performs as expected in the real world.
The last step is to evaluate your Metrics. The largest issue with Performance Tests is that: a) they are slow, and b) unreliable. Theres no way around that. In general, you need large sample sizes (repeated test runs) and some sort of statistical analysis on your metrics to ignore outliers. The processed results should then be compared to a configurable performance baseline.
Depending on the desired level of sophistication, that performance baseline may have simple acceptance criteria (such as: Repsonse Time under 100ms) or advanced, specific criteria (such as: Repsonse Time does not decrease by more than 10% when the amount of logged in users is 10 vs. 1).
I've not tried it myself but this may be suitable for your needs.
https://browsermob.com/performance-testing
Can't really say if there is a testing framework for such a specific purpose, but maybe you could use a simple console application (since the main idea is to test server performance)? You could make multiple calls to the server in different threads and use, for example, Stopwatch class to measure response times.
Or, you could try overloading the service by just increasing the number of calls (likely the service will slow down before threadpool runs out of threads?).
Kindly check at the following link.This may help you.
http://msdn.microsoft.com/en-us/library/aa337591.aspx
I guess you will get the feature even in VS2008 but for ultimate edition.
http://blog.maartenballiauw.be/post/2008/02/code-performance-analysis-in-visual-studio-2008.aspx
Available in Developer and Team-based Visual Studio 2008. I think the feature has been improved in Visual Studio 2010 though.
I should clarify - this doesn't provide support for all the test cases you want to run. You would have to write that logic yourself in a separate project, set that as the startup project, then run the Performance Analysis.
I've done some performance tests a while ago in a client/server application using Microsoft Visual Studio Load Test
When you add unit tests to a load test, you exercise the performance
of non-Web based server components
You have to code your load generators as "Visual Studio Unit Tests", so that later you can add them into the Load Test project.
I am going to write a WMI Provider for getting battery information using C#.NET 4.0. Can you any one tell me how long it would take to write? . I need to support XP, Windows 7 and Server 2008.
I also need to implement the Interface to the battery to get the additional information which Win32_battery will not provide.
I am a inexperienced Programmer. But i have little idea about WMI Concepts
Thanks In advance..
Although WMI can seem a a little counter-intuitive for some programmers used to functionality we get in the .NET Framework, it is really quite simple. We had to do some WMI work for Hyper-V and we found a fairly generic sample we used to get started here:
http://blogs.technet.com/b/richard_macdonald/archive/2008/08/11/3103559.aspx
I think from the above you can see it won't take very long at all, it should be in the hours, if that. The Win32_Battery class is very simple, (here is an example). However, if you want to go deeper into WMI, I'd recommend you use the WMI CIM Studio from the WMI Administrative Tools
WMI Tools include: WMI CIM Studio: view and edit classes, properties, qualifiers, and instances in a CIM repository; run selected methods; generate and compile MOF files. WMI Object Browser: view objects, edit property values and qualifiers, and run methods.
It takes as long as it takes. An experienced programmer who deals with WMI providers regularly can probably do it quickly, especially if he has code from previous projects that he can reuse. An inexperienced programmer, or an experienced programmer whose experience lies in other areas, is going to take longer.
One if the first things I learned when I started with C# was the most important one. You can decompile any .NET assembly with Reflector or other tools. Many developers are not aware of this fact and most of them are shocked when I show them their source code.
Protection against decompilation is still a difficult task. I am still looking for a fast, easy and secure way to do it. I don't want to obfuscate my code so my method names will be a,b,c or so. Reflector or other tools should be unable to recognize my application as .NET assembly at all. I know about some tools already but they are very expensive. Is there any other way to protect my applications?
EDIT:
The reason for my question is not to prevent piracy. I only want to stop competitors from reading my code. I know they will and they already did. They even told me so.
Maybe I am a bit paranoid but business rivals reading my code doesn't make me feel good.
One thing to keep in mind is that you want to do this in a way that makes business sense. To do that, you need to define your goals. So, exactly what are your goals?
Preventing piracy? That goal is not achievable. Even native code can be decompiled or cracked; the multitude of warez available online (even for products like Windows and Photoshop) is proof a determined hacker can always gain access.
If you can't prevent piracy, then how about merely reducing it? This, too, is misguided. It only takes one person cracking your code for it to be available to everyone. You have to be lucky every time. The pirates only have to be lucky once.
I put it to you the goal should be to maximize profits. You appear to believe that stopping piracy is necessary to this endeavor. It is not. Profit is simply revenue minus costs. Stopping piracy increases costs. It takes effort, which means adding cost somewhere in the process, and so reduces that side of the equation. Protecting your product also fails to increase your revenue. I know you look at all those pirates and see all the money you could make if only they would pay your license fees instead, but the reality is this will never happen. There is some hyperbole here, but it generally holds that pirates who are unable to crack your security will either find a similar product they can crack or do without. They will never buy it instead, and therefore they do not represent lost sales.
Additionally, securing your product actually reduces revenue. There are two reasons for this. One is the small percentage of customers who have trouble with your activation or security, and therefore decide not to buy again or ask for their money back. The other is the small percentage of people who actually try a pirated version of software to make sure it works before buying. Limiting the pirated distribution of your product (if you are somehow able to succeed at this) prevents these people from ever trying your product, and so they will never buy it. Moreover, piracy can also help your product spread to a wider audience, thus reaching more people who will be willing to pay for it.
A better strategy is to assume that your product will be pirated, and think about ways to take advantage of the situation. A couple more links on the topic:
How do i prevent my code from being stolen?
Securing a .NET Application
At work here we use Dotfuscator from PreEmptive Solutions.
Although it's impossible to protect .NET assemblies 100% Dotfuscator makes it hard enough I think.
I comes with a lot of obfuscation techniques;
Cross Assembly Renaming
Renaming Schemes
Renaming Prefix
Enhanced Overload Induction
Incremental Obfuscation
HTML Renaming Report
Control Flow
String Encryption
And it turned out that they're not very expensive for small companies. They have a special pricing for small companies.
(No I'm not working for PreEmptive ;-))
There are freeware alternatives of course;
Host your service in any cloud service provider.
How to preventing decompilation of any C# application
Pretty much describes the entire situation.
At some point the code will have to be translated to VM bytecode, and the user can get at it then.
Machine code isn't that much different either. A good interactive disassembler/debugger like IDA Pro makes just about any native application transparent. The debugger is smart enough to use AI to identify common APIs, compiler optimizations, etc. it allows the user to meticuloulsy rebuild higher level constructs from the assembly generated from machine code.
And IDA Pro supports .Net to some extent too.
Honestly, after working on an reverse engineering ( for compatibility ) project for a few years, the main thing I got out of my experience is that I probably shouldn't worry too much about people stealing my code. If anyone wants it, it will never be very hard to get it no matter what scheme I implement.
No obsfuscator can protect your application, not even any one described here. See this link, it's an deobsfuscator which can deobsfuscate almost every obsfuscator out there.
https://github.com/0xd4d/de4dot
The best way which can help you (but remember that they are also not full prof) is to use mixed codes, code your important codes in unmanaged language and make a DLL like in C or C++ and then protect them either with Armageddon or Themida.
Themida is not for every cracker, it's one of the best protector in the market, it can also protect your .NET software.
I know you don't want to obfuscate, but maybe you should check out dotfuscator, it will take your compiled assemblies and obfuscate them for you. I think it can even encrypt them.
I've heard about some projects that directly compile IL into native code.
You can get some additional info from this post:
Is it possible to compile .NET IL code to machine code?
We use SmartAssembly for .NET protection of an enterprise level distributed application, and it has worked great for us.
If you want to fully protect your app from decompilation, look at Aladdin's Hasp. You can wrap your assemblies in an encrypted shell that can only be accessed by your application. Of course one wonders how they're able to do this but it works. I don't know however if they protect your app from runtime attachment/reflection which is what Crack.NET is able to do.
-- Edit
Also be careful of compiling to native code as a solution...there are decompilers for native code as well.
Do you API?
Instead of trying to protect your one ddl file in one of your products on all of your customers devices, why not create an API service for your precious product features? Let the actual product that is saved on a device consume that API to deliver the product as you want it.
I Think this way you are 100% sure that your code is not decompiled and you set your own limits in your API so that developers / hackers don't consume your API in a way you don't want it.
Sure is some more work, but in the end, you are in control.
If someone has to steal your code, it likely means your business model is not working. What do I mean by that? For example, I buy your product and then I ask for support. You're too busy or believe my request is not valid and a waste of your time. I decode your product in order to support my relative business. Your product becomes more valuable to me and I prioritize my time in a way to resolve the business model for leveraging your product. I recode and re-brand your product and then go out and make the money that you decided to leave on the table. There are reasons for protecting code, but most likely you are looking at the problem from the wrong perspective. Of course you are. You're the "coder", and I'm the business man. ;-) Cheers!
ps. I'm also a developer. i.e. "coder"
I know this is old but, Themida is the most advanced anti-cracking software I've ever used.
It's not free, though.
Besides the third party products listed here, there is another one: NetLib Encryptionizer. However it works in a different way than the obfuscators. Obfuscators modify the assembly itself with a deobfuscation "engine" built into it. Encryptionizer encrypts the DLLs (Managed or Unmanaged) at the file level. So it does not modify the DLL except to encrypt it. The "engine" in this case is a kernel mode driver that sits between your application and the operating system. (Disclaimer: I am from NetLib Security)
In a .NET C# program, is it easy to transition from FTP to SFTP? I'm trying to get a sense of how muh time it would take the contractor to make the transition. My personal experience is mostly with PHP, so I have no idea.
Basically, what I'm talking about, what steps would have to be made? Obviously, different commands, but would anything else in the code itself? Like do the commands require different formats, etc.?
Also, if anyone has a list of all the .NET/C# FTP and SFTP commands, that would be really helpful.
Clarification, as requested: The program is uploading extremely small files (20 KB) to a server. By format, I mean visually, because I was wondering about a find/replace job.
This is a pretty vague question. You haven't told us what the C# program is doing with FTP. Is it a server, is it a client, is it doing directory listings, is it uploading 100 GB files? What library is it using?
According to this forum post , there is no built-in support for SFTP in .NET, so you would have to use third-party libraries such as SharpSSH or Granados SSH.
I don't really know what you mean, "do the commands require different formats". Obviously, the code will use different:
Libraries
Types
Wire protocol.
It will obviously appear somewhat similar, thanks to the abstraction of the libraries. I suggest you provide more information, and a clearer question.
One thing that you'd need to consider is how well your current code is written. If your existing FTP implementation is horribly designed spaghetti code then converting it to SFTP may be next to impossible and take way longer than you'd like. Without knowing the current state of the code, it would be difficult for anyone to make a good estimation. And even if you do get an estimation from people on this site, I wouldn't recommend trusting it (even though the people on this site are great) since without all the information in front of them it will be next to impossible for anyone to come up with a reliable estimate.
Perhaps you should consider hiring a good consultant or business analyst to do a thorough estimate for you.
It really depends on what C# library your developer has used to implement FTP.
If, for example, they used edtFTPnet, a widely available open source library, then the upgrade path is trivial if you replace it with edtFTPnet/PRO. The PRO version has the identical API and just a few extra lines of code would be needed.
I've been down this road.
It depends, but keep in mind that SFTP, FTP-SSL and FTP are different.
If he's writing the SFTP libraries himself, a month or two, since it's a lot of work to make it perfect and compatible. But he should NOT do that.
In short, get him to use an external library to add SFTP functionality. This will make it pretty short. Maybe a week or two of full-time work, but it depends on how involved it is. There's open-source options.. But for $50-150 you can get a license to well-maintained code that's really easy to use. will save him days of work.
There's links above, but I'd look at:
Free:
http://www.enterprisedt.com/products/edtftpnet/overview.html
Commercial:
http://www.weonlydo.com/
First of all, apologies for the subjective sounding title. This is intended as a direct question.
At present I am working on a suite of tools:
A C# Windows Service, to primarily
maintain an Oracle database.
A C# Windows Service, (which will be
used on multiple node sites) to
process content of the database.
An ASP.NET web interface to
facilitate management of the overall
"system"
Currently the tho Windows Services have been developed as Console Applications (to ease debugging/development) and I am in the midst of converting these to Services. After testing for a couple days now with these services, I'm finding that I would like to increase the granularity of my logging. I'm finding that I miss Console.WriteLine() and I would like to provide an alternate log source like a flat-file for this type of output. This has lead me to think, "Should I be using a framework, or have I got enough?"
The reason I have mentioned the aspects I am developing is to provide insight to my situation. A "Core" DLL has been created, common across all components, abstracting the interaction layer between the applications and database. It is within this DLL that a class has been created which will attempt to "log to a table in the database" else on fail "log to local Event Log". This is it, that's the extent of logging.
Throughout the aforementioned tools, there are multiple instances of logging not dissimilar to:
Log.LogError("Code", e.Message + "\n" + e.StackTrace);
Although quite basic, this method does make use of reflection to Identify the source of the error.
My Question
Looking at my current logging solution it appears "sufficient" in terms of what it does and how it is integrated with all my solutions. However, I've been looking at logging frameworks (Notably log4net) and their features impress me. The ability to, if needed in the future, add another output format (such as an SMTP server) sounds kind of cool to me! :)
What I would like to know are the benefits of moving to a framework (like log4net)? The extent of how much I will have to adapt my code? Whether or not I am just looking at the greener grass on the other side? And finally, but probably most importantly, am I doing the right thing? Should I just add the ability to my Log class to "LogDebug" and be done with it? The last thing I would want to do is completely overhaul my suite, just for a "basic" feature, but if there are other benefits (to design, reliance, good practice? etc.) I'm interested.
Thanks,
Yes. Using an existing, proven logging framework (such as Log4net) is a good idea.
Log4Net is configurable at runtime (great for tracking down issues in production code).
As a commenter pointed out, it's also very simple to use.
Proper logging is especially beneficial when running code on multiple remote systems, as far as I recall, log4net will let you send your logs to a remote syslog server without much coding overhead (meaning you can view your logs from all machines in one centralized place) doing this will massively reduce the time it takes you to get information relating to a bug or problem with the system, and should also give you an indication of how prevalent the issue is.
As mentioned in other posts, log4net also allows for multiple appenders and multiple log levels, so determining where you want certain log information (i.e. in a database or in a local flat file, hey log4net even lets you spit logs out over telnet) to be stored is an absolute doddle.
As for implementing it, there are several good sites talking you through the setup. How you actually make use of the logging objects that log4net gives you is an architectural choice, but you could simply change the constructor of an object to take a log4net object and from within this object, just use the log4net object as you would Console.WriteLine.
I find the tutorial series here particularly useful, and it'll also go in to more depth than I can here about the benefits and the different ways of configuring log4net.
Yes, you definitely want to use a logging framework. A logging framework will allow you to:
Set the logging levels for the different logger instances.
Set the "appenders" or output for each of the different logger instances.
Perhaps, more importantly, if you use a logging framework, it is very easy to swap out one implementation of the logging framework for another (perhaps a null implementation that simply discards messages); whereas, if you write all your logging statements, directly, swapping out the implementation will be a nightmare.
I think you should use Log4net, simply because it's always better to reuse than to build your own thing. log4net has been used by a lot of developers and are pretty matured.
Think about your maintenance prospect; one or two months down the road, you might need to tweak your custom logging class a bit, to add some multithreading support etc. And when you are fixing the bugs arose from your logging class, you will miss Log4net.
Well one of the bigger benefits is not having to maintain the code yourself. Most of the time, logging frameworks have a lot more functionality than your own solution. Because they are so focused on logging, those frameworks usually are pretty complete in the both functionality and ways to implement it. And then there's reliability; there's nothing worse than a logging framework that's not logging anything because it's bugged. ;)
Take for example ELMAH for ASP.net applications. It also includes notifications, exports to various target formats, etc. Things that are pretty handy but you'll never build yourself unless you really need it.
How many changes to your code are needed obviously depends on both your code and the framework of choice. It's hard to say anything about that.
I am going to give a shout out to NLog (http://nlog-project.org/home) as it doesn't suffer from the 'Straight Java Port - then rewrite' syndrome of most oss .Net libs.
Some key benefits for us were the very fast Logger.IsFooEnabled (volatile read) and the overall performance of the system.
To each its own though, but I personally prefer NLog for my projects (and some of my clients too).
Cheers,
Florian
The advantage of using a good logging framework like Log4Net is that they have a small impact upon your code in terms of lines of code that you have to alter (in other words you only have to alter each existing logging line).
Also, if you are concerned about altering your code if you change frameworks, or if you feel you want to roll your own, then you could always create your own interface to a logging framework. Then you only ever have to change your code in one place after that.
I think sysadmins expect services to log to the application event log in windows.
Look up System.Diagnostics.EventLog, although log4net will write to that too..
The initial statement in the log4j website might help in some of your questions, the underlying principles are the same of log4net:
With log4j it is possible to enable
logging at runtime without modifying
the application binary. The log4j
package is designed so that these
statements can remain in shipped code
without incurring a heavy performance
cost. Logging behavior can be
controlled by editing a configuration
file, without touching the application
binary.
Using a logger hierarchy it is
possible to control which log
statements are output at arbitrarily
fine granularity but also great ease.
This helps reduce the volume of logged
output and minimize the cost of
logging.
In this case there's clearly no need to reinvent the wheel. Most Logging frameworks are somewhat straightforward, so the extend of changes will most likely depend on the size of your existing programs.
if you write your logger class properly it will be easily expendable to any of your needs. Any framework could impress you with many features but another framework is another variable in your debugging process as it can give you an error that does not exists or can make an error by itself in combination with your application. If you are ready to make beta testing for open source software project this is fine...
In your place i would write log class with ability to extend it features you find interesting to you project based on the list of features known frameworks have. I don't see any problem to log something to file and then send it over smpt, just one small function does the job.
Moreover, you can write your own class which will be pretty abstract and put your basic code in there, if you will ever need to use external framework for testing you class would be able to use it with minimal impact on code. Just take a look how there frameworks are implemented on the code level.
think of that you will need to learn how to properly use these frameworks when your only needs for now to log very small part of it...