I have written a class using TDD containing a method (method under test) which takes a simple value object as a parameter (range).
Code:
The method under test looks like this:
public List<string> In(IRange range)
{
var result = new List<string>();
for (int i = range.From; i <= range.To; i++)
{
//...
}
return result;
}
Furthermore I have a unit test to verify my method under test:
[TestMethod]
public void In_SimpleNumbers_ReturnsNumbersAsList()
{
var range = CreateRange(1, 2);
var expected = new List<string>() { "1", "2" };
var result = fizzbuzz.In(range);
CollectionAssert.AreEqual(expected, result);
}
private IRange CreateRange(int from, int to)
{
return new Fakes.StubIRange()
{
FromGet = () => { return from; },
ToGet = () => { return to; }
};
}
Question:
I have read Roy Osherove's book on unit testing ("The Art of Unit Testing"). In there he says
"external dependencies (filesystem, time, memory etc.) should be
replaced by stubs"
What does he mean by external dependency? Is my value object (range) also an external dependency which should be faked? Should I fake all dependencies a class have?
Can someone give me an advice
TL;DR
Do the simplest thing possible that solves your problem.
The longer I have been using TDD, the more I appreciate the value of being pragmatic. Writing super-duper isolated unit tests is not a value unto itself. The tests are there to help you write high quality code that is easy to understand and solves the right problem.
Adding an interface for the range class is a good idea if you have the need to be able to switch to another concrete range implementation without having to modify the code which depends on it.
However, if you do not have that need adding an interface serves no real purpose, but it does add some complexity, which actually takes you further from the goal of writing easy to understand code which solves the problem.
Be careful not to think to much about what might change in the future. YAGNI is a good principle to follow. If you have been doing TDD you won't have any problems refactoring the code if an actual need occurs in the future, since you have solid tests to rely on.
In general terms I would not consider a proper Value Object to be a dependency. If it is complex enough that you feel uncomfortable letting other code use it when under test it sounds like it is actually more like a service.
A unit test should run in isolation (completely in memory) without having to touch any external systems, such as file system, database, web service, mail service, system clock, or anything that is either slow, hard to setup, or undeterministic (such as the ever changing system time).
To be able to do this, you should abstract away those external dependencies which allows you to mock them in your tests.
A unit test however, goes one step further. In a unit test you often only want to test the logic of a single method or a single class. You're not interested to verify how multiple components integrate, but you just want to verify that the logic of that single class is correct, and that it communicates correctly with other components.
To be able to do this, you need to fake those other components (the class's dependencies). So in general you should indeed fake all dependencies (that contain behavior) a class has.
Related
I have 2 projects, one is the "Main" project and the other is the "Test" project.
The policy is that all methods in the Main project must have at least one accompanying test in the test project.
What I want is a new unit test in the Test project that verifies this remains the case. If there are any methods that do not have corresponding tests (and this includes method overloads) then I want this test to fail.
I can sort out appropriate messaging when the test fails.
My guess is that I can get every method (using reflection??) but I'm not sure how to then verify that there is a reference to each method in this Test project (and ignore references in projects)
You can use any existing software to measure code coverage but...
Don't do it!!! Seriously. The aim should be not to have 100% coverage but to have software that can easily evolve. From your test project you can invoke by reflection every single existing method and swallow all the exceptions. That will make your coverage around 100% but what good would it be?
Read about TDD. Start creating testable software that has meaningful tests that will save you when something goes wrong. It's not about coverage, it's about being safe.
This is an example of a meta-test that sounds good in principle, but once you get to the detail you should quickly realise is a bad idea. As has already been suggested, the correct approach is to encourage whoever owns the policy to amend it. As you’ve quoted the policy, it is sufficiently specific that people can satisfy the requirement without really achieving anything of value.
Consider:
public void TestMethod1Exists()
{
try
{
var classUnderTest = new ClassToTest();
classUnderTest.Method1();
}
catch (Exception)
{
}
}
The test contains a call to Method1 on the ClassToTest so the requirement of having a test for that method is satisfied, but nothing useful is being tested. As long as the method exists (which is must if the code compiled) the test will pass.
The intent of the policy is presumably to try to ensure that written code is being tested. Looking at some very basic code:
public string IsSet(bool flag)
{
if (flag)
{
return "YES";
}
return "NO";
}
As methods go, this is pretty simple (it could easily be changed to one line), but even so it contains two routes through the method. Having a test to ensure that this method is being called gives you a false sense of security. You would know it is being called but you would not know if all of the code paths are being tested.
An alternative that has been suggested is that you could just use code coverage tools. These can be useful and give a much better idea as to how well exercised your code is, but again they only give an indication of the coverage, not the quality of that coverage. So, let’s say I had some tests for the IsSet method above:
public void TestWhenTrue()
{
var classUnderTest = new ClassToTest();
Assert.IsString(classUnderTest.IsSet(true));
}
public void TestWhenFalse()
{
var classUnderTest = new ClassToTest();
Assert.IsString(classUnderTest.IsSet(false));
}
I’m passing sufficient parameters to exercise both code paths, so the coverage for the IsSet method should be 100%. But all I am testing is that the method returns a string. I’m not testing the value of the string, so the tests themselves don’t really add much (if any) value.
Code coverage is a useful metric, but only as part of a larger picture of code quality. Having peer reviews and sharing best practice around how to effectively test the code you are writing within your team, whilst it is less concretely measurable will have a more significant impact on the quality of your test code.
I have a class that deals with Account stuff. It provides methods to login, reset password and create new accounts so far.
I inject the dependencies through the constructor. I have tests that validates each dependency's reference, if the reference is null it throws an ArgumentNullException.
The Account class exposes each of these dependencies through read only properties, I then have tests that validates if the reference passed on the constructor is the same that the property returns. I do this to make sure the references are being held by the class. (I don't know if this is a good practice too.)
First question: Is this a good practice in TDD? I ask this because this class has 6 dependencies so far, and it gets very repetitive and also the tests get pretty long as I have to mock all the dependencies for each test. What I do is just a copy and paste every time and just change the dependency's reference being tested.
Second question: my account creation method does things like validating the model passed, inserting data in 3 different tables or a forth table if a certain set of values are present and sending an email. What should I test here? I have so far a test that checks if the model validation gets executed, if the Add method of each repository gets called, and in this case, I use the Moq's Callback method of the mocked repository to compare each property being added to the repository against the ones I passed by the model.
Something like:
userRepository
.Setup(r => r.Add(It.IsAny<User>()))
.Callback<User>(u =>
{
Assert.AreEqual(model.Email, u.Email);
Assert.IsNotNull(u.PasswordHash);
//...
})
.Verifiable();
As I said, these tests are getting longer, I think that it doesn't hurt to test anything I can, but I don't know if it's worth it as it it's taking time to write the tests.
The purpose of testing is to find bugs.
Are you really going to have a bug where the property exists but is not initialized to the value from the constructor?
public class NoNotReally {
private IMyDependency1 _myDependency;
public IMyDependency1 MyDependency {get {return _myDependency;}}
public NoNotReally(IMyDependency dependency) {
_myDependency = null; // instead of dependency. Really?
}
}
Also, since you're using TDD, you should write the tests before you write the code, and the code should exist only to make the tests pass. Instead of your unnecessary tests of the properties, write a test that demonstrates that your injected dependency is being used. In order or such a test to pass, the dependency will need to exist, it will need to be of the correct type, and it will need to be used in the particular scenario.
In my example, the dependency will come to exist because it's needed, not because some artificial unit test required it to be there.
You say writing these tests feels repetitive. I say you feel the major benefit of TDD. Which is in fact not writing software with less bugs and not writing better software, because TDD doesn't guarantee either (at least not inherently). TDD forces you to think about design decisions and make design decisions all. The. Time. (And reduce debugging time.) If you feel pain while doing TDD, it's usually because a design decision is coming back to bite you. Then it's time to switch to your refactoring hat and improve the design.
Now in this particular case it's just the design of your tests, but you have to make design decisions for those as well.
As for testing whether properties are set. If I understand you correctly, you exposed those properties just for the sake of testing? In that case I'd advise against that. Assume you have a class with a constructor parameter and have a test that asserts the construtor should throw on null arguments:
public class MyClass
{
public MyClass(MyDependency dependency)
{
if (dependency == null)
{
throw new ArgumentNullException("dependency");
}
}
}
[Test]
public void ConstructorShouldThrowOnNullArgument()
{
Assert.Catch<ArgumentNullException>(() => new MyClass(null));
}
(TestFixture class omitted)
Now when you start to write a test for an actual business method of the class under test, the parts will start to fit together.
[Test]
public void TestSomeBusinessFunctionality()
{
MyDependency mockedDependency;
// setup mock
// mock calls on mockedDependency
MyClass myClass = new MyClass(mockedDependency);
var result = myClass.DoSomethingOrOther();
// assertions on result
// if necessary assertion on calls on mockedDependency
}
At that point, you will have to assign the injected dependency from the constructor to a field so you can use it in the method later. And if you manage to get the test to pass without using the dependency... well, heck, obviously you didn't need it to begin with. Or, maybe, you'll only start to need it for the next test.
About the other point. When it becomes a hassle to test all the reponsibilities of a method or class, TDD is telling you that the method/class is doing to much and would maybe like to be split up into parts that are easy to test. E.g. one class for verification, one for mapping and one for executing the storage calls.
That can very well lead to over-engineering, though! So watch out for that and you'll develop a feeling for when to resist the urge for more indirection. ;)
To test if properties are mapped properly, I'd suggest to use stubs or self-made fake objects which have simple properties. That way you can simply compare the source and target properties and don't have to make lengthy setups like the one you posted.
Normally in unit tests (especially in TDD), you are not going to test every single statement in the class that you are testing. The main purpose of the TDD unit tests is to test the business logic of the class, not the initialization stuff.
In other words, you give scenarios (remember to include edge cases too) as input and check the results, which can either be the final values of the properties and/or the return values of the methods.
The reason you don't want to test every single possible code path in your classes is because should you ever decide to refactor your classes later on, you only need to make minimal changes to your TDD unit tests, as they are supposed to be agnostic to the actual implementation (as much as possible).
Note: There are other types of unit tests, such as code coverage tests, that are meant to test every single code path in your classes. However, I personally find these tests impractical, and certainly not encouraged in TDD.
I have various methods that I would like to unit test using Visual Studio's built in unit testing capability for C#. Things have been going pretty smoothly, but I've run into a scenario where I want to "stub out" a dependency on a particular function call.
Namely, I have some methods that are in the following format:
public class ClassWithMethodsIWantToUnitTest
{
public void myFcn(object someArg)
{
...
LoggerService.Instance.LogMessage("myMessage");
...
}
}
So basically, I want my unit test to simply verify that the call to "LogMessage" has occurred. I don't want to actually check a log file or anything. I want a way to see if the LoggerService line has been hit and executed.
LoggerService is a singleton, and if possible, I don't want to modify the code just for unit testing.
Based on the format of this problem, it seems to me that it should be possible to somehow take control of the code in my unit test. In other words, is there a way for me to make a fake version of LogMessage such that I can use it for my unit test? I don't want the "real" LogMessage function to be called if possible. I just want to test that the code hit the path that called the function.
Does this make any sense? Is this possible?
It certainly makes sense and is not an unknown problem.
Unfortunately you will probably need to change the code, so that it accepts dependency injection. That is, when testing you should be able to inject a specially crafted test object (a mock) instead of the real thing. In your case it probably means being able to set LoggerService.Instance to a special mock object.
The second thing you need is the fake logger instance that you will test against. You probably want a mock, which is a special object that can be set up to check behaviour of the caller. There are several mock frameworks around and I really recommend that you use one instead of trying to roll your own.
Anders' answer was definitely the "canonical" way of approaching the problem, but in .NET 4.5, there's a second option:
The Fakes framework.
It lets you add a "fakes" assembly to your unit test project that accepts a delegate to perform in place of the actual implementation of a method. Here's an example using File.ReadAllText
[TestMethod]
public void Foo()
{
using (ShimsContext.Create())
{
ShimFile.ReadAllTextString = path => "test 123";
var reverser = new TextReverser();
const string expected = "321 tset";
//Act
var actual = reverser.ReverseSomeTextFromAFile(#"C:\fakefile.txt");
//Assert
Assert.AreEqual(expected, actual);
}
}
What that test method is doing is temporarily (within the scope of the ShimsContext) replacing the implementation of File.ReadAllText with the lambda I provided. In this case, any time ReadAllText is called, it returns the string "test 123".
It's slower than regular DI, but if you're absolutely tied to a specific implementation of a singleton, it could be exactly what you need. Read more about it here.
What Anders said.
Several popular mocking frameworks are Moq and RhinoMocks
And you'd change your code so that the logger dependency was injected to your class:
public class ClassWithMethodsIWantToUnitTest
{
public ClassWithMethodsIWantToUnitTest(ILoggerService logger)
{}
public void myFcn(object someArg)
{
...
logger.LogMessage("myMessage");
...
}
}
Or something similar. A DI framework could inject the logger automatically into the class when it needs it. Esentially a DI framework automatically calls
new ClassWithMethodsIWantToUnitTest(LoggerService.Instance);
I'm making my first steps with unit testing and am unsure about two paradigms which seem to contradict themselves on unit tests, which is:
Every single unit test should be self-contained and not depend on others.
Don't repeat yourself.
To be more concrete, I've got an importer which I want to test. The Importer has a "Import" function, taking raw data (e.g. out of a CSV) and returning an object of a certain kind which also will be stored into a database through ORM (LinqToSQL in this case).
Now I want to test several things, e.g. that the returned object returned is not null, that it's mandatory fields are not null or empty and that it's attributes got the correct values. I wrote 3 unit tests for this. Should each test import and get the job or does this belong into a general setup-logic? On the other hand, believing this blog post, the latter would be a bad idea as far as my understanding goes. Also, wouldn't this violate the self-containment?
My class looks like this:
[TestFixture]
public class ImportJob
{
private TransactionScope scope;
private CsvImporter csvImporter;
private readonly string[] row = { "" };
public ImportJob()
{
CsvReader reader = new CsvReader(new StreamReader(
#"C:\SomePath\unit_test.csv", Encoding.Default),
false, ';');
reader.MissingFieldAction = MissingFieldAction.ReplaceByEmpty;
int fieldCount = reader.FieldCount;
row = new string[fieldCount];
reader.ReadNextRecord();
reader.CopyCurrentRecordTo(row);
}
[SetUp]
public void SetUp()
{
scope = new TransactionScope();
csvImporter = new CsvImporter();
}
[TearDown]
public void TearDown()
{
scope.Dispose();
}
[Test]
public void ImportJob_IsNotNull()
{
Job j = csvImporter.ImportJob(row);
Assert.IsNotNull(j);
}
[Test]
public void ImportJob_MandatoryFields_AreNotNull()
{
Job j = csvImporter.ImportJob(row);
Assert.IsNotNull(j.Customer);
Assert.IsNotNull(j.DateCreated);
Assert.IsNotNull(j.OrderNo);
}
[Test]
public void ImportJob_MandatoryFields_AreValid()
{
Job j = csvImporter.ImportJob(row);
Customer c = csvImporter.GetCustomer("01-01234567");
Assert.AreEqual(j.Customer, c);
Assert.That(j.DateCreated.Date == DateTime.Now.Date);
Assert.That(j.OrderNo == row[(int)Csv.RechNmrPruef]);
}
// etc. ...
}
As can be seen, I'm doing the line Job j = csvImporter.ImportJob(row);
in every unit test, as they should be self-contained. But this does violate the DRY principle and may possibly cause performance issues some day.
What's the best practice in this case?
Your test classes are no different from usual classes, and should be treated as such: all good practices (DRY, code reuse, etc.) should apply there as well.
That depends on how much of your scenario that's common to your test. In the blog post you refered to the main complaint was that the SetUp method did different setup for the three tests and that can't be considered best practise. In your case you've got the same setup for each test/scenario and then you should use a shared SetUp instead of duplicating the code in each test. If you later on find that there are more tests that does not share this setup or requires a different setup shared between a set of tests then refactor those test to a new test case class. You could also have shared setup methods that's not marked with [SetUp] but gets called in the beginning of each test that needs them:
[Test]
public void SomeTest()
{
setupSomeSharedState();
...
}
A way of finding the right mix could be to start off without a SetUp method and when you find that you're duplicating code for test setup then refactor to a shared method.
You could put the
Job j = csvImporter.ImportJob(row);
in your setup. That way you're not repeating code.
you actually should run that line of code for each and every test. Otherwise tests will start failing because of things that happened in other tests. This will become hard to maintain.
The performance problem isn't caused by DRY violations. You actually should setup everything for each and every test. These aren't unit tests, they're integration tests, you rely on external files to run the test. You could make ImportJob read from a stream instead of it directly opening a file. Then you could test with a memorystream.
Whether you move
Job j = csvImporter.ImportJob(row);
into the SetUp function or not, it will still be executed before every test is executed. If you have the exact same line at the top of each test, well then it is just logical that you move that line into the SetUp portion.
The blog entry that you posted complained about the setup of the test values being done in a function disconnected (possibly not on the same screen as) from the test itself -- but your case is different, in that the test data is being driven by an external text file, so that complaint doesn't match up with your specific use case either.
In one of my projects we agreed with team that we will not implement any initialization logic in unit tests constructors. We have Setup, TestFixtureSetup, SetupFixture (since version 2.4 of NUnit) attributes. They are enough for almost all cases when we need initialization. We force developers to use one of these attributes and to explicitly define whether we will run this initialization code before each test, before all tests in a fixture or before all tests in a namespace.
However I will disagree that unit tests should always confirm to all good practices supposed for a usual development. It is desirable, but it is not a rule. My point is that in real life customer doesn't pay for unit tests. Customer pays for the overall quality and functionality of the product. He is not interested to know whether you provide him a bug-free product by covering 100% of code by unit test/automated GUI tests or by employing 3 manual testers per one developer that will click on every piece of the screen after each build.
Unit tests don't add business value to the product, they allow you to save on development and testing efforts and force developers to write better code. So it is always up to you - will you spend additional time on UT refactoring to make unit tests perfect? Or will you spend the same amount of time to add new features for the customers of your product? Do not also forget that unit-tests should be as simple as possible. How to find a golden section?
I suppose this depends on the project, and PM or team lead need to plan and estimate quality of unit tests, their completeness and code coverage as if they estimate all other business features of your product. My opinion, that it is better to have copy-paste unit tests that cover 80% of production code then to have a very well designed and separated unit tests that cover only 20%.
So I have a class with a method as follows:
public class SomeClass
{
...
private SomeDependency m_dependency;
public int DoStuff()
{
int result = 0;
...
int someValue = m_dependency.GrabValue();
...
return result;
}
}
And I've decided that rather than to call m_dependency.GrabValue() each time, I really want to cache the value in memory (i.e. in this class) since we're going to get the same value each time anyway (the dependency goes off and grabs some data from a table that hardly ever changes).
I've run into problems however trying to describe this new behaviour in a unit test. I've tried the following (I'm using NUnit with RhinoMocks):
[Test]
public void CacheThatValue()
{
var depend = MockRepository.GeneraMock<SomeDependency>();
depend.Expect(d => d.GrabValue()).Repeat.Once().Return(1);
var sut = new SomeCLass(depend);
int result = sut.DoStuff();
result = sut.DoStuff();
depend.VerifyAllExpectations();
}
This however doesn't work; this test passes even without introducing any changes to the functionality. What am I doing wrong?
I see caching as orthogonal to Do(ing)Stuff. I would find a way to pull the caching logic outside of the method, either by changing SomeDependency or wrapping it somehow (I now have a cool idea for a caching class based around lambda expressions -- yum).
That way your tests for DoStuff don't need to change, you only need to make sure they work with the new wrapper. Then you can test the caching functionality of SomeDependency, or its wrapper, independently. With well-architected code putting a caching layer in place should be rather easy and neither your dependency nor your implementation should know the difference.
Unit tests shouldn't be testing implementation, they should test behavior. At the same time, the subject under test should have a narrowly-defined set of behavior.
To answer your question, you are using a Dynamic Mock and the default behavior is to allow any call that isn't configured. The additional calls are just returning "0". You need to set up an expectation that no more calls are made on the dependency:
depend.Expect(d => d.GrabValue()).Repeat.Once().Return(1);
depend.Expect(d => d.GrabValue()).Repeat.Never();
You may need to enter record/replay mode to get it to work properly.
This seems like a case for "tests drive the design". If caching is an implementation detail of SubDependency - and therefore can't be directly tested - then probably some of its functionality (specifically, its caching behavior) needs to be exposed - and since it's not natural to expose it within SubDependency, needs to be exposed in another class (let's call it "Cache"). In Cache, of course, the behavior is contractual - public, and thereby testable.
So the tests - and the smells - are telling us we need a new class. Test-Driven Design. Ain't it great?