Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
We've got a WPF app we'd like to run automated smoke tests on it. Simple tests like load document, save document, etc, etc. I was wondering if anyone could suggest existing frame works or applications that would help with that.
Thanks!
For testing frameworks both NUnit and MSTest are good. MSTest does have the advantage that it integrates very will with visual studio, so that makes things a bit easier. (There are nunit addins available for vs, but they are quite as fully integrated as mstest).
In terms of how you write the test, if you have written you app according to the MVVM pattern it's pretty simple to have your tests create and run your app using the view models and the commands without actually creating a view.
Even if you haven't used MVVM, hopefully you've still abstracted your logic layers away from your gui, so your tests can call into them without too much difficulty.
In terms of actual GUI testing, you can take a look at the MS UI Automation framework which should allow you to automate parts of your UI to run tests against it. There's a blog post here about how to get going with that, and an article here. There are also some commercial frameworks that overlay the UI automation stuff to make it a bit easier. One example is testautomationfx.
As a spoke test should be “end to end”, then I would look at automated UI testing tools like Test Complete, rather than unit testing tools – scripting the creation of a VM and then run yours installers is another good option.. You spoke test should include the installer for your app, as they tend not to be covered by unit tests, or used by your developers.
You are trying to avoid your testers spending time on “hopeless” builds – hence the need to include the installer.
Think of all the “daft” reasons that stops your test team being productive after they have spent time installing a new build – how many of these can you include in an automated system without having tests fail due to changes in the app.
A lot of people make the error of trying to cover too much in a spoke test – “deep testing” that covers all your logic should be in unit tests and/or “story tests” not your spoke test.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I have just been assigned a task to implement a unit testing strategy for a small development team using c sharp on visual studio 2012 professional edition (which doesn't include the powerful full version of MS testing module (test manager, test center, ability to generate unit tests automatically, etc)).
After searching from the web and read a lot of articles about unit testing in visual studio, I have come up with a few options such as "nUnit", "xUnit.net" and the integrated MS unitTest coming with Visual Studio 2012 professional.(Because its only professional version, I just fund the function is very limited).
Can anybody with experience in this field help me to select a right tool? If you have any other suggestions please specify.
Also, we need to pick up a "fake" framework for creating "stub" and "drivers" for module testing as well. Since we still only have the VS 2012 professional version, we don't have access to the "MS fakes". I know "Moq" could be a good solution.
I would love anybody could suggest any better option.
Thanks
I also recommend NUnit as a great starting point for your unit testing, because it is free, it has integration into Visual Studio (if you wish, but also can run standalone as well) and also can complement other testing frameworks (like MSTest) if you decide to use/buy later. In other words, you could have tests written using NUnit and MSTest coexisting.
Read A Step-by-Step Beginner Tutorial on Writing Unit Tests NUnit with C#.
As for mocking, I recommend the following:
Moq - free and regularly updated on GitHub
Rhino Mocks - free and regularly updated on website
TypeMock - not free, but extremely powerful mocking framework that allows you to mock concrete types (including sealed classes) which allows you to test third party software, such as SharePoint, etc.
I use nUnit And it's great.
It can be integrated with VS 2012 using the nUnit test adapter extension.
It can be used as a stand alone
jenkins CI server supports it (don't know about others...).
It is used very widely and you could find tutorials and example for practically everything you need.
You can also "Mock" objects, if this is what you mean by: ""fake" framework for creating "stub" and "drivers""
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Are there any open source frameworks based on Selenium WebDriver in C# that
Abstract all test and object logic away
Record actions straight into Selenium Webdriver C# code?
I am thinking of writing a framework (not sure about the recording feature), but maybe it's already been done!
•Abstract all test and object logic away -- must be done manually
Record actions straight into Selenium Webdriver C# code? -- the firefox plugin "selenium ide" can output test steps in C#. You can then use the "page object" design pattern to abstract away the object logic.
Here is the best example of using the "page object" pattern I have found. Example is C#
http://seleniumexamples.com/blog/examples/selenium2-in-net-framework/
C# frameworks for Selenium Webdriver:
https://github.com/ProtoTest/ProtoTest.Golem
https://github.com/ObjectivityBSS/Test.Automation
You can try Seleno:
http://teststack.github.com/TestStack.Seleno
It implements the Page Object pattern. It encourages a separation of concern between the logic of your page and your tests.
You can use the Selenium Firefox plugin to record actions, however in my experience it's only been useful for recording quick scripts to aid in development. The code it generates hasn't been of much use to myself.
Selenium IDE is a Firefox plugin that allows you to record your browser activity and export it to WebDriver C# code.
i've been looking for something similar for a while and came across this article on sauce labs which helped me quite a bit...
http://saucelabs.com/blog/index.php/2011/12/selenium-testing-framework-part-3-putting-it-all-together/
I have went down this route using the open source Selenide framework for java tests.
The fact that there is no de-facto standard selenium framework should be a sign that it isn't probably a good thing to. I would not recommend using a framework, I failed 2 times writing a testing suite because of it.
On the other hand writing a testing suite with very good OOP structure reflecting the webpage you are testing has worked wonders for me. The key is isolating the actual selenium webdriver code from the tests. You should very rarely use the reference to driver in tests, but instead initialize objects offering services (e.g. UI actions) representing the page elements you want to interact with and test. This is an ok example of this abstraction using the FindsBymethod in C#. The Page Object Model here is more or less what you should be going for.
It's alot of work to write a meta-application for UI testing instead of just copy and pasting recorded actions into code, but it does lead to more robust, reusable, and valuable tests.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
our organization is looking for a tool to help with performance testing on each release. We ship a whole bunch of new software and we want to ensure that performance on key functions has not slowed down since the last prod release. We have code in C# and Java. This can be anything from:
when i run this function it takes < 2 seconds
when i run this set of functions the total < 5 seconds
etc . .
Is it best to do this using our basic unit testing continuous integration (nunit, junit, team city) with hand written performance checks or are there specific tools that can help focus on on this area.
Any suggestions?
On my projects (which tend to use Spring), I use the AOP and the PerformanceMonitorInterceptor.
While you may not use Spring, it's definitely some good code to look at and can base a version of your own. I found AOP perfect for this situation because it does not clutter up the actual function calls. If you have a tiered application, then you can put these performance monitors at each level. (Typically for my webapps, I put it as my data access layer so I can monitor database query performance.)
You could try soap ui if your app is remotely accessible:
http://www.soapui.org/userguide/loadtest/index.html
It'll give you all the sort of stats your after:
http://www.soapui.org/userguide/loadtest/images/loadtest_editor.gif
What we use is python to write scripts + extensive logging to generate XML logs which can be then imported into spreadsheet.
I have been tinkering with a tool called 'Basher' that allows you to write "tasks" that are picked up by the system and subsequently run for a configurable amount of time (to allow the JVM to warm up for example) and then performs a run, recording task execution time, averages and the like.
The 1.0 version has been quietly around for a while and if you care to take a look, it is available at http://basher.sourceforge.net
The trunk version contains some improvements - there is maven integration, with configurations being specified in the pom.xml, the bare bones of a reporting framework, etc.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I'm involved in a project where we are using a Continuous Integration server and NUnit for unit testing and integration testing.
A customer asked us the other day if we were writting the tests before the code... Well, were are not doing it this way always. Specially when there are complex technology issues that we would like to test to understand the problem and the possible solution first.
I would like to know if we could still considered our development process as following Agile Development, say it to customers and don't lie.
I think you are mixing up things here.
Test-Driven-Development (TDD) does not necessarily mean you are using an agile approach. Surely, it is a best practice that many of us who do agile use, but TDD can also be used in a Waterfall process, replacing/supplementing the specification.
Continous-Integration on its own means having the code your team produces integrated on an at least daily basis. This does not only force every member of the team having to merge/checkin continously, but also assures you actually can do a release of every build.The unified build process forces you to overcome the "works on my machine syndrom". Because you could do a release everyday this supports an agile process, even though it is not absolutely necessary in the strict sense.
Using tests and integrating them into the build process is a way to enrich your buildprocess with automated Quality Assurance and deepen the level on which integration (integrity) is actually tested.
As long as you are developing in small iterations, focus on getting a working product rather than on getting extensive documentation, and the customer is continuosly involved in the project, it is agile development. Unit testing, TDD and integration testing are of course good and very advisable practices, but they do not decide whether your project is agile or not.
In the absence of Automated tests, CI only verifies that the code under source control is maintained in a compilable state between revisions and that the single-step build works properly. While this is useful, it isn't as useful as the automatic verification that the correctness of the code has been maintained between revisions.
With that said, I'd rather have some verification of code between check-ins than none. I'd rather have partial code coverage or an incomplete set of functional tests than nothing. Or worse.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'm looking to add a testing suite to my application, however I can't move to the newer testing frameworks for .NET 3.5.
Does anyone have a suggestion about good testing frameworks to use?
We use MbUnit and Rihno Mocks and they prove to work very well together. When doing TDD you will almost certainly need to do some form of dependency injection, while this can be done manually, its worth looking at an IoC container such as Castle Windsor.
It well worth looking at John Paul Bodhood's screen casts to get you started. JPB's Blog
NUnit and Rhino suit well and the auto-mocking container might be of interest.
If you're looking at BDD too then NBehave is probably a good choice. If however you just mean the style of BDD that relates to unit testing (xSpec) though you can get away with adding a framework (though things like specunit do add some synctactic sugar), but you might want to look at MSpec is also interesting.
Check out Rob Conery's screencast on BDD using MSpec. Very impressive http://blog.wekeroad.com/mvc-storefront/kona-3/
edit: I now use this approach: http://10printhello.com/the-one-bdd-framework-to-rule-them/
For a Mock Object library, I've found the BSD-licensed Rhino.Mocks to be rather pleasing.
I've had great success using NUnit as well.
I've also used NMock when the need arose for mock objects. As an added bonus, the factory for creating your mock objects is called the Mockery.
To facilitate the running of unit tests, I've used TestDriven.NET to run unit tests as I coded. Also, I've used Cruise Control .NET to watch SVN and check that every new commit builds and passes all unit tests.
This is probably a summary of what has already been said, but for TDD I personally use Rhino Mocks and MBUnit. Rhino Mocks is a mocking framework that is free and open source. The advantage of Rhino Mocks is we do not need to use magic strings in setting your expectations as you do in NMock.
I like MBUnit because MbUnit has the concept of RowTests which allow you to vary your inputs to your test method. MBUnit is also freely available.
You also want to make sure that whatever you choose for your unit testing framework is supported by your CI (Continuous Integration Server). Nunit is supported by default in Cruise Control.NET and you have to do a little extra work to get MBUnit to work in ccnet.
From an IDE standpoint you must have TestDriven.NET. TestDriven.NET allows you to right click and run tests in the IDE and it supports MBUnit and Nunit and others.
NBehave is the BDD library I have used. I have not used any others so I could not compare and contrast them with you, but NBehave is supported by Gallio from the MBUnit team, which means you can run your BDD tests just as you would your unit tests with TestDriven.NET.
I would also highly recommend Resharper. You will find your productivity increase significantly with this refactoring and guidance tool. It will assist you with changing your code as you are developing your tests.
Hope this helps
Using nUnit with TFS isn't too difficult. There's even a project on codeplex to implement this: NUnit for Team Build which even "publishes" the results to the warehouse.
I haven't tried it - but I would advise clients who have a large investment (or who have a strong preference for it over the MSTest tool) in nUnit who are interested in implementing TFS to continue with nUnit as opposed to trying convert all their existing tests.
NUnit is available at http://www.nunit.org
I would suggest this even when working on the MS stack - the support for non-MS frameworks is happening in the MVC previews which shows a definite movement in the right direction to allow us all to customise our stacks to fit.
I have to put a shout-out for Moq. It is a clean light mocking framework that guides you into the pit of success.
The testing tools built into TFS are okay. They will get the job done but can often be a little cumbersome to work with. The generated reports, code coverage and a few other portions are particularly bad. They make you go bald at 22 rather than 50.
If you are really loving the testing, consider trying some Continuous Integration. You will feel the pain from regression quickly and this pain potentially helps you get to the end goal faster.
Regardless of what you do, try out a few and see which one is the most natural, if you have time. Good luck and happy coding.
NUnit is always a favorite of mine. However if you are using TFS as your source control I suggest you stick with the Microsoft Stack.
I recommend the following:
TestDriven.NET - Unit Testing add on for VS that is fully integrated with all major unit testing frameworks including NUnit, MbUnit etc...
Typemock Isolator- A mocking framework for .Net Unit Testing
NUnit - An open source unit testing framework that is in C#.
For my project, I used NUnit and TestDriven.NET with great success. You can either create a separate library just to host your test code or put it in your executable or library. It all depends on whether you want your production code to be intertwined with your test code.
For Dependency Injection, I use NInject in my current project and it works great. If you use Constructor injection, you don't need to clutter your code with the [Inject] attribute.
I haven't used a mock library for my .NET 2.0 project but for another .NET 3.5 project I will use Moq.
Note that all these work with .NET 2.0 and higher. (except Moq)