So I've written a class and I have the code to test it, but where should I put that code? I could make a static method Test() for the class, but that doesn't need to be there during production and clutters up the class declaration. A bit of searching told me to put the test code in a separate project, but what exactly would the format of that project be? One static class with a method for each of the classes, so if my class was called Randomizer, the method would be called testRandomizer?
What are some best practices regarding organizing test code?
EDIT: I originally tagged the question with a variety of languages to which I thought it was relevant, but it seems like the overall answer to the question may be "use a testing framework", which is language specific. :D
Whether you are using a test framework (I highly recommend doing so) or not, the best place for the unit tests is in a separate assembly (C/C++/C#) or package (Java).
You will only have access to public and protected classes and methods, however unit testing usually only tests public APIs.
I recommend you add a separate test project/assembly/package for each existing project/assembly/package.
The format of the project depends on the test framework - for a .NET test project, use VSs built in test project template or NUnit in your version of VS doesn't support unit testing, for Java use JUnit, for C/C++ perhaps CppUnit (I haven't tried this one).
Test projects usually contain one static class init methods, one static class tear down method, one non-static init method for all tests, one non-static tear down method for all tests and one non-static method per test + any other methods you add.
The static methods let you copy dlls, set up the test environment and clear up the test enviroment, the non-static shared methods are for reducing duplicate code and the actual test methods for preparing the test-specific input, expected output and comparing them.
Where you put your test code depends on what you intend to do with the code. If it's a stand-alone class that, for example, you intend to make available to others for download and use, then the test code should be a project within the solution. The test code would, in addition to providing verification that the class was doing what you wanted it to do, provide an example for users of your class, so it should be well-documented and extremely clear.
If, on the other hand, your class is part of a library or DLL, and is intended to work only within the ecosystem of that library or DLL, then there should be a test program or framework that exercises the DLL as an entity. Code coverage tools will demonstrate that the test code is actually exercising the code. In my experience, these test programs are, like the single class program, built as a project within the solution that builds the DLL or library.
Note that in both of the above cases, the test project is not built as part of the standard build process. You have to build it specifically.
Finally, if your class is to be part of a larger project, your test code should become a part of whatever framework or process flow has been defined for your greater team. On my current project, for example, developer unit tests are maintained in a separate source control tree that has a structure parallel to that of the shipping code. Unit tests are required to pass code review by both the development and test team. During the build process (every other day right now), we build the shipping code, then the unit tests, then the QA test code set. Unit tests are run before the QA code and all must pass. This is pretty much a smoke test to make sure that we haven't broken the lowest level of functionality. Unit tests are required to generate a failure report and exit with a negative status code. Our processes are probably more formal than many, though.
In Java you should use Junit4, either by itself or (I think better) with an IDE. We have used three environments : Eclipse, NetBeans and Maven (with and without IDE). There can be some slight incompatibilities between these if not deployed systematically.
Generally all tests are in the same project but under a different directory/folder. Thus a class:
org.foo.Bar.java
would have a test
org.foo.BarTest.java
These are in the same package (org.foo) but would be organized in directories:
src/main/java/org/foo/Bar.java
and
src/test/java/org/foo/BarTest.java
These directories are universally recognised by Eclipse, NetBeans and Maven. Maven is the pickiest, whereas Eclipse does not always enforce strictness.
You should probably avoid calling other classes TestPlugh or XyzzyTest as some (old) tools will pick these up as containing tests even if they don't.
Even if you only have one test for your method (and most test authorities would expect more to exercise edge cases) you should arrange this type of structure.
EDIT Note that Maven is able to create distributions without tests even if they are in the same package. By default Maven also requires all tests to pass before the project can be deployed.
Most setups I have seen or use have a separate project that has the tests in them. This makes it a lot easier and cleaner to work with. As a separate project it's easy to deploy your code without having to worry about the tests being a part of the live system.
As testing progresses, I have seen separate projects for unit tests, integration tests and regression tests. One of the main ideas for this is to keep your unit tests running as fast as possible. Integration & regression tests tend to take longer due to the nature of their tests (connecting to databases, etc...)
I typically create a parallel package structure in a distinct source tree in the same project. That way your tests have access to public, protected and even package-private members of the class under test, which is often useful to have.
For example, I might have
myproject
src
main
com.acme.myapp.model
User
com.acme.myapp.web
RegisterController
test
com.acme.myapp.model
UserTest
com.acme.myapp.web
RegisterControllerTest
Maven does this, but the approach isn't particularly tied to Maven.
This would depend on the Testing Framework that you are using. JUnit, NUnit, some other? Each one will document some way to organize the test code. Also, if you are using continuous integration then that would also affect where and how you place your test. For example, this article discusses some options.
Create a new project in the same solution as your code.
If you're working with c# then Visual Studio will do this for you if you select Test > New Test... It has a wizard which will guide you through the process.
hmm. you want to test random number generator... may be it will be better to create strong mathematical proof of correctness of algorithm. Because otherwise, you must be sure that every sequence ever generated has a desired distribution
Create separate projects for unit-tests, integration-tests and functional-tests. Even if your "real" code has multiple projects, you can probably do with one project for each test-type, but it is important to distinguish between each type of test.
For the unit-tests, you should create a parallel namespace-hierarchy. So if you have crazy.juggler.drummer.Customer, you should unit-test it in crazy.juggler.drummer.CustomerTest. That way it is easy to see which classes are properly tested.
Functional- and integration-tests may be harder to place, but usually you can find a proper place. Tests of the database-layer probably belong somewhere like my.app.database.DatabaseIntegrationTest. Functional-tests might warrant their own namespace: my.app.functionaltests.CustomerCreationWorkflowTest.
But tip #1: be tough about separating the various kind of tests. Especially be sure to keep the collection of unit-tests separate from the integration-tests.
In the case of C# and Visual Studio 2010, you can create a test project from the templates which will be included in your project's solution. Then, you will be able to specify which tests to fire during the building of your project. All tests will live in a separate assembly.
Otherwise, you can use the NUnit Assembly, import it to your solution and start creating methods for all the object you need to test. For bigger projects, I prefer to locate these tests inside a separate assembly.
You can generate your own tests but I would strongly recommend using an existing framework.
Related
There are a lot of question on how to add Fakes Assembly but no one on what they are and what they are used for.
Fakes are used to mock or stub code for testing purposes. E.g., instead of connecting directly to a database in a unit test you create a fake of the class that accesses the database that "fakes" just enough behavior to be used instead of the real database. This isolates the unit test from the remaining system.
You can read more about fakes in Isolating Code Under Test with Microsoft Fakes.
Microsoft Fakes framework, helps you to test your code, when you want to use some external variables whose values change randomly (like stocks) and you are using that in your code (you want to test this code, without bothering about the external factors). Another case could be testing something which i not yet complete.
So in your code you have something which you want to test + Some external factors.
Fakes would help you by replacing external factors by your code.
We're using Microsoft's Unit Test program and we use the Unit Test Wizard to create one-to-one mapping for methods in each class from the business layer. The issue is the amount of work needed go through and determine if we are missing any tests after the initial tests were created.
Currently I have to run the wizard and look for tests that have a "1" appended to the default name [method][test]. Those with that name mean we have already have a test for that method. The ones without an append 1 mean those are methods that don't have a Unit Test that follow the default naming convention.
I'm wondering if there is away to map Unit Test to a method with attribute on the Method so it doesn't take as much work. And yes, I know if we were following TDD we would write the Unit Test first. We write the test in parallel to development (but sometimes in rush it is missed).
If you are using Visual Studio 2012 and have the appropriate version, it has proper code coverage analysis built in: "Run tests with code coverage".
Otherwise, you can use a diagnostic tool to run code coverage, such as NCover. You can do this from inside Visual Studio using TestDriven.net
I am currently working on a c# project which makes use of an SQLite Database. For the project I am required to do unit testing but was told that unit testing shouldn't involve external files, like database files for the testing and instead the test should emulate the database.
If I have a function that tests if a something exists in a database how could this sort of method be tested with a unit testing.
in general it makes life easier if external files are avoided and everything is done in code. There are no rules which says "shouldn't", and sometimes it just makes more sense to have the external dependency. But only after you have considered how not to have it, and realized what the tradeoffs are.
Secondly, what Bryan said is a good option and the one I've used before.
In an application that uses a database, there will be at least one component whose responsibility is to communicate with that database. The unit test for that component could involve a mocked database, but it is perfectly valid (and often desirable) to test the component using a real database. After all, the component is supposed to encapsulate and broker communication with that database -- the unit test should test that. There are numerous strategies to perform such unit tests conveniently -- see the list of related SO questions in the sidebar for examples.
The general prescription to avoid accessing databases in unit tests applies to non-database components. Since non-database components typically outnumber database-related components by a wide margin, the vast majority of unit tests should not involve a database. Indeed, if such non-database components required a database to be tested effectively, there is likely a design problem present -- probably improper separation of concerns.
Thus, the principle that unit tests should avoid databases is generally true, but it is not an absolute rule. It is just a (strong) guideline that aids in structuring complex systems. Following the rule too rigidly makes it very difficult to adequately test "boundary" components that encapsulate external systems -- places in which bugs find it very easy to hide! So, the question that one should really be asking oneself when a unit test demands a database is this: is the component under test legitimately accessing the database directly or should it instead collaborate with another that has that responsibility?
This same reasoning applies to the use of external files and other resources in unit tests as well.
With SQLite, you could use an in-memory database. You can stage your database by inserting data and then run your tests against it.
Once databases get involved it always blurs the line between unit testing and integration testing. Having said that, it is always a nice and very useful test to be able to put something in a database (Setup), Do your test and remove it at the end (Cleanup). This lets you test end to end one part of your application.
Personally I like to do this in an attribute driven fashion. By Specifying the Sql scripts to run for each test as an attribute like so ..
[PreSqlExecute("SetupTestUserDetails.sql")]
[PostSqlExecute("CleanupTestUserDetails.sql")]
public void UpdateUserNameTest()
{
The connectionstrings come from the app.config as usual and can even be a symbolic link to the app.config in your main project.
Unfortunately this isn't a standard feature with the MS test runner that ships with visual studio. If you are using Postsharp as your AOP framework, this is easy to do. If not, you can still get the same functionality for standard MS Test Runner, by using a feature of .Net called "Context Bound Objects". This lets you inject custom code into an object creation chain to do AOP like stuff as long as your objects inherit from ContextBoundObject.
I did a blog post with more details and a small, complete code sample here.
http://www.chaitanyaonline.net/2011/09/25/improving-integration-tests-in-net-by-using-attributes-to-execute-sql-scripts/
I think is really bad idea to have unit tests that depends on database information.
Also I think is a bad idea to use sqlite for unit tests.
You need to test objects protocol, so if you need something in your tests you should create them somewhere in the tests (usually at setUp).
Since is difficult to remove persistence, the popular way to do it is using SQLite, but always create what you need in unit tests.
check this link Unit Tests And Databases this will be more helpful I think
It's best to use a mocking framework, to mimic a database. For C# there is the Entity Framework. Even the use of sqlite is an outside dependency to your code.
I've written a class and want to test if it works well. For now I think the best way to do it is to create new console application referencing main project, then make new instance of my class and mess with it. This approach unlike others enables IntelliSense, using keywords (no full names for classes) and Debugging.
Anyone knows how to do it in more convenient way to do this without making new console app?
Using a console app to test your class is what I would call a "poor man's unit test."
You are on the right track in wanting to do this sort of testing and I (and most others on SO) would suggest using a unit testing framework of some sort to help you out. (Mocking may be important and useful to you as well.)
Here's the thing though. Regardless of what framework you use, or if you go with a console app to test your code, you do have to create a separate project, or a separate, significant chunk of code of some sort, to be able to execute tests properly and independently. That's just part of the process. It is an investment but don't let the extra work keep you from doing it. It will save a lot time, and your skin, a little while in the future. Maybe even next week.
Also, while you're looking up unit testing make sure to also study up on test-driven development (TDD.)
Unit testing is absolutely the way to go. Depending on what version of VS you are running, there may be unit testing functionality built in, or you may have to use an additional tool such as NUnit. Both options are good and will allow you to fully test your classes.
Bear in mind also, that a comprehensive suite of unit tests will make refactoring much easier in the long run as well. If you refactor and break your unit tests, you know you've made a boo-boo somewhere. :)
Unit testing is the way forward here> this is a good introductory article.
The basic concept of a unit test is that you isolate and invoke a specific portion of code and assert that the results are expected and within reason. For example lets say you have a simple method to return the square of a floating point number:
public float Square(float value)
{
return value * value;
}
A reasonable unit test would be that the method returns the correct value, handles negetive values etc:
Assert.AreEqual(25, Square(5));
Assert.AreEqual(100, Square(-10));
Unit tests are also a good way to see how your code handles edge cases:
Assert.Throws<OverflowException>(Square(float.MaxValue));
If you are using VS 2010, check out Pex and Moles...
http://research.microsoft.com/en-us/projects/pex/
The Console App approach is more of a test harness for your class, which is fine.
But you can also use a unit testing framework to test your class. Visual Studio has one built in or you can leverage something like NUnit.
Also, try the Object Test Bench in Visual Studio. It should allow you to create a new instance, modify and view properties, and call some methods. It usually only works with very simple apps, though.
If you use Visual Studio 2008 or higher you will be able to test your code using MSTest framework:
1.Open Test View window: Test/Windows/Test View;
2.Add new unit test project: right click on Solution in Solution Explorer/Add/New
Project/Test Project;
3.Remove all files apart from UnitTest.cs file in created test project;
4.Write your unit test in method under [TestMethod] attribute:
[TestClass]
public class UnitTest1
{
[TestMethod]
public void TestMethod1()
{
var ranges = new Ranges();
int sum = ranges.CountOccurrences(11);
Assert.AreEqual(128, sum);
}
}
5.Run your test from Test View window added in p.1
6.See test results in Test/Windows/Test Results window
I thought it will be a common question so I searched for a while but couldn't find it.
I am about to start a new project (C#, .net 3.5) and I was thinking about where I should I write the unit test code. I can create a unit test project and write all code there, or I can write the unit test code with the "class under test" itself.
What do you recommend and why? Things to consider before choosing an approach (caveats?)?
EDIT: About writing unit-test code with "code under test": Removing the test code from production assembly isn't difficult I guess. Thats what conditional compilation is for. Right?
Just throwing this point because answers are rejecting the second option just because production assemblies would be fatty.
Separate project, same solution. Use InternalsVisibleTo if you want to access internals from the test code.
Separating out test from production code:
makes it more obvious what's what
means you don't need dependencies on test frameworks in your production project
keeps your deployed code leaner
avoids including test data files in your deployment assembly
Keeping the code in the same solution:
keeps the test cycle quick
makes it easy to hop between production and test code
I always create a separate project in where I write my TestFixtures.
I do not want to litter my domain model (or whatever) with Test classes.
I do not want to distribute my tests to customers or users of my application, so therefore I put them in a separate project (which also leads to a separate assembly).
If you have the rare case that you want to test internal methods, you can use InternalsVisibleTo.
If you have the very rare case that you want to test private methods, you can use this technique, as explained by Davy Brion.
I prefer the first approach - separating to unit test to its own project.
placing the unit tests within the test subject will make it dirty. furthermore, you don't necessarily want to distribute your project with the unit tests which will make your dll's bigger and possibly expose things that you don't want to expose to the end user.
most of the open source projects that I saw had a different projects for unit tests.
You shoul place the unit tests in a seperate project.
You should also write them in a way, so that the SUT (System under Test) is not modified in a way to make unittests possible. I mean you should have no helper classes in you main project that exist "only" to support you tests.
Mixing test and production code is allways a bad plan, since you dont want to deliver all that extra code out to your clients. Keep the clear separation that another project offers.
I dont think the "keep the tests quick" argument is a really strong one. Make a clear cut... Testing code does not belong into a production enviroment IMHO...
Edit:
Comment on Edit above:
EDIT: About writing unit-test code with "code under test": Removing the test code from production assembly isn't difficult I guess. Thats what conditional compilation is for. Right?
Yes, it is "easy" to remove the code with a conditional compilation flag, but you wont have tested the final assembly you created, you only tested the assembly you created with the code inside it, then you recompile, creating a new,untested assembly and ship that one. Are you sure all your conditional flags are set 100% correct? I guess not, since you cant run the tests ;)