So I have a class with a method as follows:
public class SomeClass
{
...
private SomeDependency m_dependency;
public int DoStuff()
{
int result = 0;
...
int someValue = m_dependency.GrabValue();
...
return result;
}
}
And I've decided that rather than to call m_dependency.GrabValue() each time, I really want to cache the value in memory (i.e. in this class) since we're going to get the same value each time anyway (the dependency goes off and grabs some data from a table that hardly ever changes).
I've run into problems however trying to describe this new behaviour in a unit test. I've tried the following (I'm using NUnit with RhinoMocks):
[Test]
public void CacheThatValue()
{
var depend = MockRepository.GeneraMock<SomeDependency>();
depend.Expect(d => d.GrabValue()).Repeat.Once().Return(1);
var sut = new SomeCLass(depend);
int result = sut.DoStuff();
result = sut.DoStuff();
depend.VerifyAllExpectations();
}
This however doesn't work; this test passes even without introducing any changes to the functionality. What am I doing wrong?
I see caching as orthogonal to Do(ing)Stuff. I would find a way to pull the caching logic outside of the method, either by changing SomeDependency or wrapping it somehow (I now have a cool idea for a caching class based around lambda expressions -- yum).
That way your tests for DoStuff don't need to change, you only need to make sure they work with the new wrapper. Then you can test the caching functionality of SomeDependency, or its wrapper, independently. With well-architected code putting a caching layer in place should be rather easy and neither your dependency nor your implementation should know the difference.
Unit tests shouldn't be testing implementation, they should test behavior. At the same time, the subject under test should have a narrowly-defined set of behavior.
To answer your question, you are using a Dynamic Mock and the default behavior is to allow any call that isn't configured. The additional calls are just returning "0". You need to set up an expectation that no more calls are made on the dependency:
depend.Expect(d => d.GrabValue()).Repeat.Once().Return(1);
depend.Expect(d => d.GrabValue()).Repeat.Never();
You may need to enter record/replay mode to get it to work properly.
This seems like a case for "tests drive the design". If caching is an implementation detail of SubDependency - and therefore can't be directly tested - then probably some of its functionality (specifically, its caching behavior) needs to be exposed - and since it's not natural to expose it within SubDependency, needs to be exposed in another class (let's call it "Cache"). In Cache, of course, the behavior is contractual - public, and thereby testable.
So the tests - and the smells - are telling us we need a new class. Test-Driven Design. Ain't it great?
Related
I have a class that deals with Account stuff. It provides methods to login, reset password and create new accounts so far.
I inject the dependencies through the constructor. I have tests that validates each dependency's reference, if the reference is null it throws an ArgumentNullException.
The Account class exposes each of these dependencies through read only properties, I then have tests that validates if the reference passed on the constructor is the same that the property returns. I do this to make sure the references are being held by the class. (I don't know if this is a good practice too.)
First question: Is this a good practice in TDD? I ask this because this class has 6 dependencies so far, and it gets very repetitive and also the tests get pretty long as I have to mock all the dependencies for each test. What I do is just a copy and paste every time and just change the dependency's reference being tested.
Second question: my account creation method does things like validating the model passed, inserting data in 3 different tables or a forth table if a certain set of values are present and sending an email. What should I test here? I have so far a test that checks if the model validation gets executed, if the Add method of each repository gets called, and in this case, I use the Moq's Callback method of the mocked repository to compare each property being added to the repository against the ones I passed by the model.
Something like:
userRepository
.Setup(r => r.Add(It.IsAny<User>()))
.Callback<User>(u =>
{
Assert.AreEqual(model.Email, u.Email);
Assert.IsNotNull(u.PasswordHash);
//...
})
.Verifiable();
As I said, these tests are getting longer, I think that it doesn't hurt to test anything I can, but I don't know if it's worth it as it it's taking time to write the tests.
The purpose of testing is to find bugs.
Are you really going to have a bug where the property exists but is not initialized to the value from the constructor?
public class NoNotReally {
private IMyDependency1 _myDependency;
public IMyDependency1 MyDependency {get {return _myDependency;}}
public NoNotReally(IMyDependency dependency) {
_myDependency = null; // instead of dependency. Really?
}
}
Also, since you're using TDD, you should write the tests before you write the code, and the code should exist only to make the tests pass. Instead of your unnecessary tests of the properties, write a test that demonstrates that your injected dependency is being used. In order or such a test to pass, the dependency will need to exist, it will need to be of the correct type, and it will need to be used in the particular scenario.
In my example, the dependency will come to exist because it's needed, not because some artificial unit test required it to be there.
You say writing these tests feels repetitive. I say you feel the major benefit of TDD. Which is in fact not writing software with less bugs and not writing better software, because TDD doesn't guarantee either (at least not inherently). TDD forces you to think about design decisions and make design decisions all. The. Time. (And reduce debugging time.) If you feel pain while doing TDD, it's usually because a design decision is coming back to bite you. Then it's time to switch to your refactoring hat and improve the design.
Now in this particular case it's just the design of your tests, but you have to make design decisions for those as well.
As for testing whether properties are set. If I understand you correctly, you exposed those properties just for the sake of testing? In that case I'd advise against that. Assume you have a class with a constructor parameter and have a test that asserts the construtor should throw on null arguments:
public class MyClass
{
public MyClass(MyDependency dependency)
{
if (dependency == null)
{
throw new ArgumentNullException("dependency");
}
}
}
[Test]
public void ConstructorShouldThrowOnNullArgument()
{
Assert.Catch<ArgumentNullException>(() => new MyClass(null));
}
(TestFixture class omitted)
Now when you start to write a test for an actual business method of the class under test, the parts will start to fit together.
[Test]
public void TestSomeBusinessFunctionality()
{
MyDependency mockedDependency;
// setup mock
// mock calls on mockedDependency
MyClass myClass = new MyClass(mockedDependency);
var result = myClass.DoSomethingOrOther();
// assertions on result
// if necessary assertion on calls on mockedDependency
}
At that point, you will have to assign the injected dependency from the constructor to a field so you can use it in the method later. And if you manage to get the test to pass without using the dependency... well, heck, obviously you didn't need it to begin with. Or, maybe, you'll only start to need it for the next test.
About the other point. When it becomes a hassle to test all the reponsibilities of a method or class, TDD is telling you that the method/class is doing to much and would maybe like to be split up into parts that are easy to test. E.g. one class for verification, one for mapping and one for executing the storage calls.
That can very well lead to over-engineering, though! So watch out for that and you'll develop a feeling for when to resist the urge for more indirection. ;)
To test if properties are mapped properly, I'd suggest to use stubs or self-made fake objects which have simple properties. That way you can simply compare the source and target properties and don't have to make lengthy setups like the one you posted.
Normally in unit tests (especially in TDD), you are not going to test every single statement in the class that you are testing. The main purpose of the TDD unit tests is to test the business logic of the class, not the initialization stuff.
In other words, you give scenarios (remember to include edge cases too) as input and check the results, which can either be the final values of the properties and/or the return values of the methods.
The reason you don't want to test every single possible code path in your classes is because should you ever decide to refactor your classes later on, you only need to make minimal changes to your TDD unit tests, as they are supposed to be agnostic to the actual implementation (as much as possible).
Note: There are other types of unit tests, such as code coverage tests, that are meant to test every single code path in your classes. However, I personally find these tests impractical, and certainly not encouraged in TDD.
I'm pretty new to Unit Testing and am exploring the Microsoft Fakes framework - primarily because it's free and it allows me to mock SharePoint objects easily with the Emulators package. I've seen various mentions on SO and elsewhere that Shims are evil and I more or less understand why. What I don't get is how to avoid them in one specific case - in other words, "how should I refactor my code to avoid having to use shims?"
For the code in question, I have a JobProcessor object that has properties and methods, some of which are private as they should only be called from the public Execute method. I want to test that when Execute is called and there is a Job available that its Process method is called as I need to do some extra logging.
Here's the relevant code:
//in system under test - JobProcessor.cs
private IJob CurrentJob { get; set; }
public void Execute()
{
GetJobToProcess(); //stores Job in CurrentJob property if found
if (ShouldProcessJob){
CurrentJob.ProcessJob();
}
}
I need to do some extra things if ProcessJob is called from a test, so I set up a Stub in my Test Method to do those extra things:
StubIJob fakeJob = new StubIJob(){
ProcessJob = () =>{
//do my extra things here
}
};
I'm testing the ProcessJob method itself elsewhere so I don't care that it doesn't do anything but my extra stuff here. As I understand things, I now need to set up a Shim to have the private method GetJobsToProcess from JobProcessor (my system under test) return my fake job so that my stubbed method is called:
processor = new JobProcessor();
ShimJobProcessor.AllInstances.GetJobToProcess = (#this) =>{
var privateProcessor = new PrivateObject(processor);
privateProcessor.SetProperty("CurrentJob", fakeJob); //force my test Job to be processed so the Stub is used
};
In this case, how should I avoid using the Shim? Does it matter?
Thanks.
This is a case where rather than using a shim or stub, I'd just make the method return a boolean to notify whether or not the inner call has happened.
The problem with using fakes there is that you're assuming that some method of some object is called, which the test should not know. Tests should be dumb, and only see the outside of the code. Tests, like any other code, should not care how a value was reached, just that it is correct.
However, your code has another issue as well. You're getting some unknown object and using it within the same scope. You should remove the call to GetJobToProccess from Execute.
It's the principle of Dependency Injection: a method should not spin up and hide it's dependencies; if it depends on an object, that object should be possible to change freely or be passed in. The exact implementation of the job should not matter to the execute method, and that, along with the naming, implies that you should not be getting that object and executing it in the same call.
Given the following simple service class, in the GetCategories() method, should you test the fact that the categoryRepository.Query() method was called, or should you be setting up a test that keeps a list of categories and returns those?
I guess what I am saying is would mocking the categoryRepository and verifying that it's Query method was called once cover this test case?
public class CategoryService : ValidatingServiceBase, ICategoryService
{
private readonly IRepository<Category> categoryRepository;
private readonly IRepository<SubCategory> subCategoryRepository;
private readonly IValidationService validationService;
public CategoryService(
IRepository<Category> categoryRepository,
IRepository<SubCategory> subCategoryRepository,
IValidationService validationService)
: base(validationService)
{
this.categoryRepository = categoryRepository;
this.subCategoryRepository = subCategoryRepository;
this.validationService = validationService;
}
public IEnumerable<Category> GetCategories()
{
return categoryRepository.Query().ToList();
}
}
Sample Test
[Fact]
public void GetCategories_Should_CallRepositoryQuery()
{
var categoryRepo = new Mock<IRepository<Category>>();
var service = new CategoryService(categoryRepo.Object, null, null);
service.GetCategories();
categoryRepo.Verify(x => x.Query(), Times.Once());
}
It doesn't matter. In both cases (mock + behavior verification vs stub + assertion) you achieve exactly the same result and require exactly the same level of details about inner workings of class. Stick to whichever one you think is more suited in given scenario.
Unit test you posted is an example of behavior verification. You don't assert any values but instead check whether some method was called. This is especially useful when method call has no visible results (think about logging) or doesn't return any value (obviously). It of course has drawbacks, especially when you do such verification for methods that do return value, and don't check it (as is your case - we'll get to it).
The stubbing and asserting approach uses the collaborators to generate value. It doesn't check whether methods were called (at least not directly, yet such test is performed when you setup stub and that setup works), but instead relies on correct flow of stubbed value.
Let's go with simple example. Say you test a method of your class, PizzaFactory.GetPizza which looks like this:
public Pizza GetPizza()
{
var dough = doughFactory.GetDough();
var cheese = ingredientsFactory.GetCheese();
var pizza = oven.Bake(dough, cheese);
return pizza;
}
With behavior verification you'd check whether doughFactory.GetDough was called, then ingredientsFactory.GetCheese and finally oven.Bake. Had such calls indeed been made, you'd assume pizza was created. You don't check that your factory returns pizza, but assume it happens if all process' steps were completed. You can already see that drawback I mentioned earlier - I can call all the correct methods but return something else, say:
var dough = doughFactory.GetDough();
var cheese = ingredientsFactory.GetCheese();
var pizza = oven.Bake(dough, cheese);
return garbageBin.FindPizza();
Not the pizza you ordered? Notice that all the correct calls to collaborators happened just as we assumed they would.
With stub + assert approach it all looks similar except instead of verification you have stubbing. You use values generated by earlier collaborators to stub later collaborators (if somehow you get wrong dough or cheese, oven will not return pizza we wanted). The final value is what your method returns and this is what we assert:
doughFactoryStub.Setup(df => dg.GetDough).Return("thick");
ingredientsFactoryStub.Setup(if => if.GetCheese()).Return("double");
var expectedPizza = new Pizza { Name = "Margherita" };
ovenStub.Setup(o => o.Bake("thick", "double")).Return(expectedPizza);
var actualPizza = pizzaFactory.GetPizza();
Assert.That(actualPizza, Is.EqualTo(expectedPizza));
If any part of the process fails (say doughFactory returns normal dough) then the final value will be different and test will fail.
And once again, in my opinion in your example it doesn't matter which approach you use. In any normal environment both methods will verify the same thing and require same level of knowledge about your implementation. To be extra safe you might want to prefer to use the stub + assert approach in case somebody plants you a garbage bin1. But if such thing happens, unit tests are your last problem.
1 Note however that it might not be intentional (especially when complex methods are considered).
Yes, that would be the way.
mockCategoryRepository.Setup(r => r.Query()).Returns(categories)
var actualCategories = new CategoryService(mockCategoryRepository, mock..).GetCategories();
CollectionAssert.AreEquivalent(categories, actualCategories.ToList());
It would look something similar with Moq and NUnit.
What you've presented is a white-box test - an approach also possible in unit testing, but recommended only for simple methods.
In the answer presented by Sruti the service is tested in a black-box sense. The knowledge about the inner method is used only to prepare the test, but you don't verify if the method was called one time, 10 times, or wasn't called at all. Personally, I verify method calls only to verify that some external API that must be stubbed is used correctly (example: sending e-mails). Usually it is sufficient not to care about how a method works, as long as it's producing correct results.
With black-box tests the code and the tests are easier to maintain. With white-box tests, most changes of some internal structure during refactoring of a class usually must be followed by changing test code. In black-box approach you have more freedom to rearrange everything, and still be sure that interface's external behaviour hasn't changed.
I have written a class using TDD containing a method (method under test) which takes a simple value object as a parameter (range).
Code:
The method under test looks like this:
public List<string> In(IRange range)
{
var result = new List<string>();
for (int i = range.From; i <= range.To; i++)
{
//...
}
return result;
}
Furthermore I have a unit test to verify my method under test:
[TestMethod]
public void In_SimpleNumbers_ReturnsNumbersAsList()
{
var range = CreateRange(1, 2);
var expected = new List<string>() { "1", "2" };
var result = fizzbuzz.In(range);
CollectionAssert.AreEqual(expected, result);
}
private IRange CreateRange(int from, int to)
{
return new Fakes.StubIRange()
{
FromGet = () => { return from; },
ToGet = () => { return to; }
};
}
Question:
I have read Roy Osherove's book on unit testing ("The Art of Unit Testing"). In there he says
"external dependencies (filesystem, time, memory etc.) should be
replaced by stubs"
What does he mean by external dependency? Is my value object (range) also an external dependency which should be faked? Should I fake all dependencies a class have?
Can someone give me an advice
TL;DR
Do the simplest thing possible that solves your problem.
The longer I have been using TDD, the more I appreciate the value of being pragmatic. Writing super-duper isolated unit tests is not a value unto itself. The tests are there to help you write high quality code that is easy to understand and solves the right problem.
Adding an interface for the range class is a good idea if you have the need to be able to switch to another concrete range implementation without having to modify the code which depends on it.
However, if you do not have that need adding an interface serves no real purpose, but it does add some complexity, which actually takes you further from the goal of writing easy to understand code which solves the problem.
Be careful not to think to much about what might change in the future. YAGNI is a good principle to follow. If you have been doing TDD you won't have any problems refactoring the code if an actual need occurs in the future, since you have solid tests to rely on.
In general terms I would not consider a proper Value Object to be a dependency. If it is complex enough that you feel uncomfortable letting other code use it when under test it sounds like it is actually more like a service.
A unit test should run in isolation (completely in memory) without having to touch any external systems, such as file system, database, web service, mail service, system clock, or anything that is either slow, hard to setup, or undeterministic (such as the ever changing system time).
To be able to do this, you should abstract away those external dependencies which allows you to mock them in your tests.
A unit test however, goes one step further. In a unit test you often only want to test the logic of a single method or a single class. You're not interested to verify how multiple components integrate, but you just want to verify that the logic of that single class is correct, and that it communicates correctly with other components.
To be able to do this, you need to fake those other components (the class's dependencies). So in general you should indeed fake all dependencies (that contain behavior) a class has.
In the past, I have only used Rhino Mocks, with the typical strict mock. I am now working with Moq on a project and I am wondering about the proper usage.
Let's assume that I have an object Foo with method Bar which calls a Bizz method on object Buzz.
In my test, I want to verify that Bizz is called, therefore I feel there are two possible options:
With a strict mock
var mockBuzz= new Mock<IBuzz>(MockBehavior.Strict);
mockBuzz.Setup(x => x.Bizz()); //test will fail if Bizz method not called
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.VerifyAll();
With a loose mock
var mockBuzz= new Mock<IBuzz>();
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.Verify(x => x.Bizz()) //test will fail if Bizz method not called
Is there a standard or normal way of doing this?
I used to use strict mocks when I first starting using mocks in unit tests. This didn't last very long. There are really 2 reasons why I stopped doing this:
The tests become brittle - With strict mocks you are asserting more than one thing, that the setup methods are called, AND that the other methods are not called. When you refactor the code the test often fails, even if what you are trying to test is still true.
The tests are harder to read - You need to have a setup for every method that is called on the mock, even if it's not really related to what you want to test. When someone reads this test it's difficult for them to tell what is important for the test and what is just a side effect of the implementation.
Because of these I would strongly recommend using loose mocks in your unit tests.
I have background in C++/non-.NET development and I've been more into .NET recently so I had certain expectations when I was using Moq for the first time. I was trying to understand WTF was going on with my test and why the code I was testing was throwing a random exception instead of the Mock library telling me which function the code was trying to call. So I discovered I needed to turn on the Strict behaviour, which was perplexing- and then I came across this question which I saw had no ticked answer yet.
The Loose mode, and the fact that it is the default is insane. What on earth is the point of a Mock library that does something completely unpredictable that you haven't explicitly listed it should do?
I completely disagree with the points listed in the other answers in support of Loose mode. There is no good reason to use it and I wouldn't ever want to, ever. When writing a unit test I want to be certain what is going on - if I know a function needs to return a null, I'll make it return that. I want my tests to be brittle (in the ways that matter) so that I can fix them and add to the suite of test code the setup lines which are the explicit information that is describing to me exactly what my software will do.
The question is - is there a standard and normal way of doing this?
Yes - from the point of view of programming in general, i.e. other languages and outside the .NET world, you should use Strict always. Goodness knows why it isn't the default in Moq.
I have a simple convention:
Use strict mocks when the system under test (SUT) is delegating the call to the underlying mocked layer without really modifying or applying any business logic to the arguments passed to itself.
Use loose mocks when the SUT applies business logic to the arguments passed to itself and passes on some derived/modified values to the mocked layer.
For eg:
Lets say we have database provider StudentDAL which has two methods:
Data access interface looks something like below:
public Student GetStudentById(int id);
public IList<Student> GetStudents(int ageFilter, int classId);
The implementation which consumes this DAL looks like below:
public Student FindStudent(int id)
{
//StudentDAL dependency injected
return StudentDAL.GetStudentById(id);
//Use strict mock to test this
}
public IList<Student> GetStudentsForClass(StudentListRequest studentListRequest)
{
//StudentDAL dependency injected
//age filter is derived from the request and then passed on to the underlying layer
int ageFilter = DateTime.Now.Year - studentListRequest.DateOfBirthFilter.Year;
return StudentDAL.GetStudents(ageFilter , studentListRequest.ClassId)
//Use loose mock and use verify api of MOQ to make sure that the age filter is correctly passed on.
}
Me personally, being new to mocking and Moq feel that starting off with Strict mode helps better understand of the innards and what's going on. "Loose" sometimes hides details and pass a test which a moq beginner may fail to see. Once you have your mocking skills down - Loose would probably be a lot more productive - like in this case saving a line with the "Setup" and just using "Verify" instead.