This is a very vague (and noob) question, but.....How would one approach testing a class library in C#? I am using nUnit for testing.
What I am trying to do is test database interaction. The input will be a serialized XML object, deserialize it to test it against the code, then the XML object will be re-serialized and outputted.
Hopefully, this gives some insight. I had thought about creating a test app that creates an instance of the library. Is there a different/better/more efficient approach I can take?
You have to create a separate project that uses NUnit's data annotations (TestFixture, Test, etc). Then you can build that project and load the created DLL into Nunit.
As far as the tests, just write them as you would normally (Arrange-Act-Assert is the more prevalent pattern)
Something like this
[Test]
public void MethodName_CallDatabase_ObjectDeserialized()
{
//Arrange
var db = new db();
//Act
var output = db.ExecuteCall();
//Assert
Assert.That(output, Is.EqualTo("123"));
}
As for the how, typically your solution will look like this:
Solution (Your Application)
+ YourApplication.Library
+ YourApplication.WebApp
+ YourApplication.Tests
The Tests project is a specific project you can add to your solution. In that, you can make Unit Test files which will use the NUnit DLL to mark them up and then run. That means marking those classes with TestFixture and specific methods in those classes as Test methods, and then they execute parts of your YourApplication.Library project with supporting Assert calls to verify the results of the library calls.
If you're really interested in being rigorous, there are a few parts to this.
Familiarize yourself with NUnit, MBUnit, or MSTest
Figure out where your library interacts with other libraries, frameworks, tools. Write some test cases that codify your assumptions about how those other things should behave (and how they should misbehave). Install and familiarize yourself with a mock/stub library, like Moq or Rhino.Mocks. These will tell you when your code is interacting with external code in a way that is not correct.
If you're feeling up to it, write some more tests (these straddle the line between integration and unit tests, IMO) that connect a real-but-simple database with the same or similar schema, like SQLite. Now your code is no longer talking with mocks that you've pre-programmed with your expectations, but it's also not reaching out to a full-on database.
Integration tests. Hook that thing up to a real [not-production] database.
Use one of the many existing unit test frameworks for this.
nUnit, xUnit or even MSTest (which comes with many editions of Visual Studio).
If you dont want to create a new test project, just add a new class in the existing class library project and add Nunit annotation TestFixture to it and have tests as methods inside the same. as follows
namespace APIWorkOut
{
[TestFixture]
public class TestClass
{
[Test]
public void getListOfUser()
{
API api = new API();
var listOFUsers = api.getUsers();
Console.Write(listOFUsers.Data.Count);
}
}
}
Related
I have tests written in XUnit using InlineData and MemberData attributes. I would like to run tests via code elsewhere in my project and have the attributes automatically fill in test data like they normally do when ran through the VS test runner.
If it weren't for the attributes I would just call the methods directly like any other normal method. The asserts are still checked and it functions fine. But if I call a method directly that has the attributes, the attributes are ignored and I must provide all the test data manually through code. Is there some sort of test runner class in XUnit that I can reuse to accomplish this? I've been trying to dig through their API to no avail.
Why I want to do this will take some explanation, but bear with me. I'm writing tests against specific interfaces rather than their concrete implementations (think standard collection interfaces for example). There's plenty there to test and I don't want to copy paste them for each concrete implementer (could be dozens). I write the tests once and then pass each concrete implementation of the interface as the first argument to the test, a subject to test on.
But this leaves a problem. XUnit sees the test and wants to run it, but it can't because there are no concrete implementations available at this layer, there's only the interface. So I want to write tests at the higher layer that just new up the concrete implementations, and then invoke the interface tests passing in the new subjects. I can easily do this for tests that only accept 1 argument, the subject, but for tests where I'm using InlineData or MemberData too I would like to reuse those test cases already provided and just add the subject as the first argument.
Available for reference is the GitHub issue How to programmatically run XUnit tests from the xUnit.net project.
The class AssemblyRunner is now part of Xunit.Runner.Utility.
From the linked issue, xUnit.net contributor Brad Wilson provided a sample runner in the samples.xunit project on GitHub. This program demonstrates the techniques described in the issue. Namely, the portion responsible for running the tests after they have been discovered is as follows:
using (var runner = AssemblyRunner.WithAppDomain(testAssembly))
{
runner.OnDiscoveryComplete = OnDiscoveryComplete;
runner.OnExecutionComplete = OnExecutionComplete;
runner.OnTestFailed = OnTestFailed;
runner.OnTestSkipped = OnTestSkipped;
Console.WriteLine("Discovering...");
runner.Start(typeName);
finished.WaitOne(); // A ManualResetEvent
finished.Dispose();
return result;
}
For a deeper dive, he describes a method using XunitFrontController and TestDiscoveryVisitor to find and run tests. This is what AssemblyRunner does for its implementation.
Nevermind, I figured it out. Taking a closer look at XUnit's attribute hierarchy I found that the DataAttributes (InlineData, MemberData, etc) have a GetData method you can call to retrieve the set of data they represent. With a little reflection I can easily find all the tests in my test class and call the test methods, invoking the data attribute's get data method if there are any present, and perform the tests via my own code that way. The GetData part would have been much harder if I had to role my own version of it. Thank you XUnit authors for not forcing me to do that.
Good afternoon/morning/evening folks,
I was wondering is it possible for me to "execute" a SpecFlow test via some sort of test harness (not NUnit)?
Previously my test harness I built ran MS Unit tests by calling methods from within the DLL that was created when I compiled the tests.
I'm assuming the same is possible in theory since a DLL is created, but im wondering how it will get all of the arguments etc.
So in short, is this possible if so is there a straight forward way to do this or am I barking up the wrong tree?
It's possible, but I'm not clear why you would want to.
Specflow is basically just a clever way of generating tests. Normally these are nUnit tests, but they can also be switched to use mstest. When you save your edits to the .feature file then VS runs a Custom Tool that converts your plaintext into a .feature.cs file that contains a code version of what you wrote with nUnit attributes applied to the methods.
Later, an nUnit runner (nUnit, resharper, gallio, teamcity etc) loads the dll and looks for all public methods marked with [Test] inside public classes marked with [TestFixture]. These methods get called.
There is nothing to stop you writing your own runner, however I'm not sure why you would do that. nUnit provides extensive reporting of the success of your test run in xml format, so its probably faster just to write something to parse that.
So I decided to invest some time on this and figured that using reflection was the way to do this task.
Here is some of my code:
TestRunner.TestDLLString = getDLL(project);
var TestDLL = Assembly.LoadFrom(TestDLLString);
Type myClassType = TestDLL.GetType("SeleniumDPS." + testname);
var instance = Activator.CreateInstance(myClassType);
MethodInfo myInitMethod = myClassType.GetMethod("Initialize");
try
{
myInitMethod.Invoke(instance, null);
}
catch (Exception ex)
{
//Error logging etc
}
I then repeat that for the "[TestMethod]" etc. I understand some people dislike reflection but in this instance the performance isnt critical so it works quite well for us.
So essentially what im doing is reading on the name of the test from an XML file then searching the DLL for that test method, then executing the Intitialize method, and later on executing the test method itself. After the test is run I then execute the cleanup method.
It might seem a bit hacky and NUnit might seem the logical choice for some, but as I mentioned earlier I needed a customizable approach. Thanks for all the suggestions though.
I've written a class and want to test if it works well. For now I think the best way to do it is to create new console application referencing main project, then make new instance of my class and mess with it. This approach unlike others enables IntelliSense, using keywords (no full names for classes) and Debugging.
Anyone knows how to do it in more convenient way to do this without making new console app?
Using a console app to test your class is what I would call a "poor man's unit test."
You are on the right track in wanting to do this sort of testing and I (and most others on SO) would suggest using a unit testing framework of some sort to help you out. (Mocking may be important and useful to you as well.)
Here's the thing though. Regardless of what framework you use, or if you go with a console app to test your code, you do have to create a separate project, or a separate, significant chunk of code of some sort, to be able to execute tests properly and independently. That's just part of the process. It is an investment but don't let the extra work keep you from doing it. It will save a lot time, and your skin, a little while in the future. Maybe even next week.
Also, while you're looking up unit testing make sure to also study up on test-driven development (TDD.)
Unit testing is absolutely the way to go. Depending on what version of VS you are running, there may be unit testing functionality built in, or you may have to use an additional tool such as NUnit. Both options are good and will allow you to fully test your classes.
Bear in mind also, that a comprehensive suite of unit tests will make refactoring much easier in the long run as well. If you refactor and break your unit tests, you know you've made a boo-boo somewhere. :)
Unit testing is the way forward here> this is a good introductory article.
The basic concept of a unit test is that you isolate and invoke a specific portion of code and assert that the results are expected and within reason. For example lets say you have a simple method to return the square of a floating point number:
public float Square(float value)
{
return value * value;
}
A reasonable unit test would be that the method returns the correct value, handles negetive values etc:
Assert.AreEqual(25, Square(5));
Assert.AreEqual(100, Square(-10));
Unit tests are also a good way to see how your code handles edge cases:
Assert.Throws<OverflowException>(Square(float.MaxValue));
If you are using VS 2010, check out Pex and Moles...
http://research.microsoft.com/en-us/projects/pex/
The Console App approach is more of a test harness for your class, which is fine.
But you can also use a unit testing framework to test your class. Visual Studio has one built in or you can leverage something like NUnit.
Also, try the Object Test Bench in Visual Studio. It should allow you to create a new instance, modify and view properties, and call some methods. It usually only works with very simple apps, though.
If you use Visual Studio 2008 or higher you will be able to test your code using MSTest framework:
1.Open Test View window: Test/Windows/Test View;
2.Add new unit test project: right click on Solution in Solution Explorer/Add/New
Project/Test Project;
3.Remove all files apart from UnitTest.cs file in created test project;
4.Write your unit test in method under [TestMethod] attribute:
[TestClass]
public class UnitTest1
{
[TestMethod]
public void TestMethod1()
{
var ranges = new Ranges();
int sum = ranges.CountOccurrences(11);
Assert.AreEqual(128, sum);
}
}
5.Run your test from Test View window added in p.1
6.See test results in Test/Windows/Test Results window
I've created a test suite in NUnit that references several distinct unit test fixtures in various assemblies.
I've pretty much used the example code from NUnit's docs:
namespace NUnit.Tests
{
using System;
using NUnit.Framework;
using System.Collections;
public class AllTests
{
[Suite]
public static IEnumerable Suite
{
get
{
ArrayList suite = new ArrayList();
suite.Add(new VisionMap.DotNet.Tests.ManagedInteropTest.DotNetUtilsTest());
return suite;
}
}
}
}
My goal is to add several tests to the list above so I can run them all in a batch.
But when I try to load the DLL in NUnit's GUI I get this:
What am I doing wrong?
I'm aware that the docs say the GUI won't run suites, but I've tried the console as well. Can somebody please tell me what Suites are good for and how I can use them to achieve my goal?
I'm using nunit 2.5.0.9122.
Edit
Well, no answers are forthcoming. I found an alternative solution in the end: Categories. I group test fixtures by giving them appropriate categories and then I can run a subset of them in batch, while still ignoring another subset.
Still, very odd that this Suite feature seems to be completely broken.
Suites aren't really needed for anything much at all these days. If you only wanted to use them to specify which tests do and don't get run this is much better achieved with Category attributes. This is what you ended up doing, and sounds like the best solution to your problem.
However, for others' and future reference, you can still use Suites in Nunit. You have to run them from the console, and only using the /fixture option. For example, to run the suite you specified above, you'd run (assuming your class was compiled into an assembly AllTests.dll):
nunit-console /fixture:AllTests.Suite AllTests.dll
You won't see any evidence of or way to run suites in the GUI - this is noted in the documentation. You can however run them from the console that is built into the GUI using commands like the above.
I use suites in some of my testing because I have some odd use cases that require me to sometimes need to pass an argument to my test methods. I do this by creating a suite such as the below. So there are some uses for them, just none needed in your case.
[Suite]
public static IEnumerable MySuite
{
get
{
var suite = new ArrayList{new TestClass1(arg1), TestClass2(arg2)};
return suite;
}
}
Is there any reason why you are returning "IEnumerable" instead of "TestSuite"?
[Suite]
public static TestSuite Suite
Update
Reading the small-print at the bottom of the page at NUnit site, it looks like Suite type tests will not show in in the Gui runner, so I guess that's the issue :)
Suites are currently not displayed in the Gui or run automatically by either runner when they are encountered. The historical purpose of the Suite mechanism was to provide a way of aggregating tests at the top level of each run. Hence, they are only supported when used with the /fixture option on the console or gui command line.
Update 2
I'm not sure what you are trying to achieve with the "Suite" feature, but if you are trying to find a way of configuring a set of test assemblies to be run together, I have used "NUnit Test Projects" to do this in the past (it's just an xml config file listing test dlls). This allows a fixed set of test assembly references to be configured and then loaded into the GUI or run by the console runner:
http://www.nunit.org/index.php?p=multiAssembly&r=2.5.5
So I've written a class and I have the code to test it, but where should I put that code? I could make a static method Test() for the class, but that doesn't need to be there during production and clutters up the class declaration. A bit of searching told me to put the test code in a separate project, but what exactly would the format of that project be? One static class with a method for each of the classes, so if my class was called Randomizer, the method would be called testRandomizer?
What are some best practices regarding organizing test code?
EDIT: I originally tagged the question with a variety of languages to which I thought it was relevant, but it seems like the overall answer to the question may be "use a testing framework", which is language specific. :D
Whether you are using a test framework (I highly recommend doing so) or not, the best place for the unit tests is in a separate assembly (C/C++/C#) or package (Java).
You will only have access to public and protected classes and methods, however unit testing usually only tests public APIs.
I recommend you add a separate test project/assembly/package for each existing project/assembly/package.
The format of the project depends on the test framework - for a .NET test project, use VSs built in test project template or NUnit in your version of VS doesn't support unit testing, for Java use JUnit, for C/C++ perhaps CppUnit (I haven't tried this one).
Test projects usually contain one static class init methods, one static class tear down method, one non-static init method for all tests, one non-static tear down method for all tests and one non-static method per test + any other methods you add.
The static methods let you copy dlls, set up the test environment and clear up the test enviroment, the non-static shared methods are for reducing duplicate code and the actual test methods for preparing the test-specific input, expected output and comparing them.
Where you put your test code depends on what you intend to do with the code. If it's a stand-alone class that, for example, you intend to make available to others for download and use, then the test code should be a project within the solution. The test code would, in addition to providing verification that the class was doing what you wanted it to do, provide an example for users of your class, so it should be well-documented and extremely clear.
If, on the other hand, your class is part of a library or DLL, and is intended to work only within the ecosystem of that library or DLL, then there should be a test program or framework that exercises the DLL as an entity. Code coverage tools will demonstrate that the test code is actually exercising the code. In my experience, these test programs are, like the single class program, built as a project within the solution that builds the DLL or library.
Note that in both of the above cases, the test project is not built as part of the standard build process. You have to build it specifically.
Finally, if your class is to be part of a larger project, your test code should become a part of whatever framework or process flow has been defined for your greater team. On my current project, for example, developer unit tests are maintained in a separate source control tree that has a structure parallel to that of the shipping code. Unit tests are required to pass code review by both the development and test team. During the build process (every other day right now), we build the shipping code, then the unit tests, then the QA test code set. Unit tests are run before the QA code and all must pass. This is pretty much a smoke test to make sure that we haven't broken the lowest level of functionality. Unit tests are required to generate a failure report and exit with a negative status code. Our processes are probably more formal than many, though.
In Java you should use Junit4, either by itself or (I think better) with an IDE. We have used three environments : Eclipse, NetBeans and Maven (with and without IDE). There can be some slight incompatibilities between these if not deployed systematically.
Generally all tests are in the same project but under a different directory/folder. Thus a class:
org.foo.Bar.java
would have a test
org.foo.BarTest.java
These are in the same package (org.foo) but would be organized in directories:
src/main/java/org/foo/Bar.java
and
src/test/java/org/foo/BarTest.java
These directories are universally recognised by Eclipse, NetBeans and Maven. Maven is the pickiest, whereas Eclipse does not always enforce strictness.
You should probably avoid calling other classes TestPlugh or XyzzyTest as some (old) tools will pick these up as containing tests even if they don't.
Even if you only have one test for your method (and most test authorities would expect more to exercise edge cases) you should arrange this type of structure.
EDIT Note that Maven is able to create distributions without tests even if they are in the same package. By default Maven also requires all tests to pass before the project can be deployed.
Most setups I have seen or use have a separate project that has the tests in them. This makes it a lot easier and cleaner to work with. As a separate project it's easy to deploy your code without having to worry about the tests being a part of the live system.
As testing progresses, I have seen separate projects for unit tests, integration tests and regression tests. One of the main ideas for this is to keep your unit tests running as fast as possible. Integration & regression tests tend to take longer due to the nature of their tests (connecting to databases, etc...)
I typically create a parallel package structure in a distinct source tree in the same project. That way your tests have access to public, protected and even package-private members of the class under test, which is often useful to have.
For example, I might have
myproject
src
main
com.acme.myapp.model
User
com.acme.myapp.web
RegisterController
test
com.acme.myapp.model
UserTest
com.acme.myapp.web
RegisterControllerTest
Maven does this, but the approach isn't particularly tied to Maven.
This would depend on the Testing Framework that you are using. JUnit, NUnit, some other? Each one will document some way to organize the test code. Also, if you are using continuous integration then that would also affect where and how you place your test. For example, this article discusses some options.
Create a new project in the same solution as your code.
If you're working with c# then Visual Studio will do this for you if you select Test > New Test... It has a wizard which will guide you through the process.
hmm. you want to test random number generator... may be it will be better to create strong mathematical proof of correctness of algorithm. Because otherwise, you must be sure that every sequence ever generated has a desired distribution
Create separate projects for unit-tests, integration-tests and functional-tests. Even if your "real" code has multiple projects, you can probably do with one project for each test-type, but it is important to distinguish between each type of test.
For the unit-tests, you should create a parallel namespace-hierarchy. So if you have crazy.juggler.drummer.Customer, you should unit-test it in crazy.juggler.drummer.CustomerTest. That way it is easy to see which classes are properly tested.
Functional- and integration-tests may be harder to place, but usually you can find a proper place. Tests of the database-layer probably belong somewhere like my.app.database.DatabaseIntegrationTest. Functional-tests might warrant their own namespace: my.app.functionaltests.CustomerCreationWorkflowTest.
But tip #1: be tough about separating the various kind of tests. Especially be sure to keep the collection of unit-tests separate from the integration-tests.
In the case of C# and Visual Studio 2010, you can create a test project from the templates which will be included in your project's solution. Then, you will be able to specify which tests to fire during the building of your project. All tests will live in a separate assembly.
Otherwise, you can use the NUnit Assembly, import it to your solution and start creating methods for all the object you need to test. For bigger projects, I prefer to locate these tests inside a separate assembly.
You can generate your own tests but I would strongly recommend using an existing framework.