There are two testing scenarios I am unclear on. Both appear at first sight to create very brittle tests.
First, when should one unit test the interface (i.e. verify that an interface has a set signature)?
Second, when should one "test the sequence diagram" (a term I just made up) - meaning verifying calls are being made to the appropriate objects?
Testing the interface means that you should only test the members which are available on the public interface. In other words, don't test private stuff. Take the unit under test as a black-box. This makes the tests more maintainable because you can change the implementation details without breaking the tests. Tests will also express what your unit under test is good for, not how it is implemented.
Testing calls made on other objects is called "interaction test" (not to be confused with integration tests, where you don't mock the other objects). Interaction tests are needed when your unit under test calls a method on another object without depending on it. I try to explain it with an example.
Following method needs to be tested:
public decimal CalculateTax(Order order);
Lets assume that this method needs to call
TaxRules TaxRuleProvider.GetRules(Country country)
which returns some local rules. When it doesn't call it, it will not be able to return the correct result. It will miss vital information. You don't need to test if it had been called, just test the result.
Another method:
public void StoreThingy(Thingy toBeStored);
It will call
public void NotificationBroker.NotifyChanges(SomeChanges x);
StoreThingy doesn't depend on the notification. You can't decide on its interface if it sent notifications or not. You need to test this by an interaction test.
Typically the methods for interaction tests return void. In this category are all kinds of events and notifications and methods like Commit().
Related
I have a class that deals with Account stuff. It provides methods to login, reset password and create new accounts so far.
I inject the dependencies through the constructor. I have tests that validates each dependency's reference, if the reference is null it throws an ArgumentNullException.
The Account class exposes each of these dependencies through read only properties, I then have tests that validates if the reference passed on the constructor is the same that the property returns. I do this to make sure the references are being held by the class. (I don't know if this is a good practice too.)
First question: Is this a good practice in TDD? I ask this because this class has 6 dependencies so far, and it gets very repetitive and also the tests get pretty long as I have to mock all the dependencies for each test. What I do is just a copy and paste every time and just change the dependency's reference being tested.
Second question: my account creation method does things like validating the model passed, inserting data in 3 different tables or a forth table if a certain set of values are present and sending an email. What should I test here? I have so far a test that checks if the model validation gets executed, if the Add method of each repository gets called, and in this case, I use the Moq's Callback method of the mocked repository to compare each property being added to the repository against the ones I passed by the model.
Something like:
userRepository
.Setup(r => r.Add(It.IsAny<User>()))
.Callback<User>(u =>
{
Assert.AreEqual(model.Email, u.Email);
Assert.IsNotNull(u.PasswordHash);
//...
})
.Verifiable();
As I said, these tests are getting longer, I think that it doesn't hurt to test anything I can, but I don't know if it's worth it as it it's taking time to write the tests.
The purpose of testing is to find bugs.
Are you really going to have a bug where the property exists but is not initialized to the value from the constructor?
public class NoNotReally {
private IMyDependency1 _myDependency;
public IMyDependency1 MyDependency {get {return _myDependency;}}
public NoNotReally(IMyDependency dependency) {
_myDependency = null; // instead of dependency. Really?
}
}
Also, since you're using TDD, you should write the tests before you write the code, and the code should exist only to make the tests pass. Instead of your unnecessary tests of the properties, write a test that demonstrates that your injected dependency is being used. In order or such a test to pass, the dependency will need to exist, it will need to be of the correct type, and it will need to be used in the particular scenario.
In my example, the dependency will come to exist because it's needed, not because some artificial unit test required it to be there.
You say writing these tests feels repetitive. I say you feel the major benefit of TDD. Which is in fact not writing software with less bugs and not writing better software, because TDD doesn't guarantee either (at least not inherently). TDD forces you to think about design decisions and make design decisions all. The. Time. (And reduce debugging time.) If you feel pain while doing TDD, it's usually because a design decision is coming back to bite you. Then it's time to switch to your refactoring hat and improve the design.
Now in this particular case it's just the design of your tests, but you have to make design decisions for those as well.
As for testing whether properties are set. If I understand you correctly, you exposed those properties just for the sake of testing? In that case I'd advise against that. Assume you have a class with a constructor parameter and have a test that asserts the construtor should throw on null arguments:
public class MyClass
{
public MyClass(MyDependency dependency)
{
if (dependency == null)
{
throw new ArgumentNullException("dependency");
}
}
}
[Test]
public void ConstructorShouldThrowOnNullArgument()
{
Assert.Catch<ArgumentNullException>(() => new MyClass(null));
}
(TestFixture class omitted)
Now when you start to write a test for an actual business method of the class under test, the parts will start to fit together.
[Test]
public void TestSomeBusinessFunctionality()
{
MyDependency mockedDependency;
// setup mock
// mock calls on mockedDependency
MyClass myClass = new MyClass(mockedDependency);
var result = myClass.DoSomethingOrOther();
// assertions on result
// if necessary assertion on calls on mockedDependency
}
At that point, you will have to assign the injected dependency from the constructor to a field so you can use it in the method later. And if you manage to get the test to pass without using the dependency... well, heck, obviously you didn't need it to begin with. Or, maybe, you'll only start to need it for the next test.
About the other point. When it becomes a hassle to test all the reponsibilities of a method or class, TDD is telling you that the method/class is doing to much and would maybe like to be split up into parts that are easy to test. E.g. one class for verification, one for mapping and one for executing the storage calls.
That can very well lead to over-engineering, though! So watch out for that and you'll develop a feeling for when to resist the urge for more indirection. ;)
To test if properties are mapped properly, I'd suggest to use stubs or self-made fake objects which have simple properties. That way you can simply compare the source and target properties and don't have to make lengthy setups like the one you posted.
Normally in unit tests (especially in TDD), you are not going to test every single statement in the class that you are testing. The main purpose of the TDD unit tests is to test the business logic of the class, not the initialization stuff.
In other words, you give scenarios (remember to include edge cases too) as input and check the results, which can either be the final values of the properties and/or the return values of the methods.
The reason you don't want to test every single possible code path in your classes is because should you ever decide to refactor your classes later on, you only need to make minimal changes to your TDD unit tests, as they are supposed to be agnostic to the actual implementation (as much as possible).
Note: There are other types of unit tests, such as code coverage tests, that are meant to test every single code path in your classes. However, I personally find these tests impractical, and certainly not encouraged in TDD.
I am abstracting the history tracking portion of a class of mine so that it looks like this:
private readonly Stack<MyObject> _pastHistory = new Stack<MyObject>();
internal virtual Boolean IsAnyHistory { get { return _pastHistory.Any(); } }
internal virtual void AddObjectToHistory(MyObject myObject)
{
if (myObject == null) throw new ArgumentNullException("myObject");
_pastHistory.Push(myObject);
}
internal virtual MyObject RemoveLastObject()
{
if(!IsAnyHistory) throw new InvalidOperationException("There is no previous history.");
return _pastHistory.Pop();
}
My problem is that I would like to unit test that Remove will return the last Added object.
AddObjectToHistory
RemoveObjectToHistory -> returns what was put in via AddObjectToHistory
However, it isn't really a unit test if I have to call Add first? But, the only way that I can see to do this in a true unit test way is to pass in the Stack object in the constructor OR mock out IsAnyHistory...but mocking my SUT is odd also. So, my question is, from a dogmatic view is this a unit test? If not, how do I clean it up...is constructor injection my only way? It just seems like a stretch to have to pass in a simple object? Is it ok to push even this simple object out to be injected?
There are two approaches to those scenarios:
Interfere into design, like making _pastHistory internal/protected or injecting stack
Use other (possibly unit tested) methods to perform verification
As always, there is no golden rule, although I'd say you generally should avoid situations where unit tests force design changes (as those changes will most likely introduce ambiguity/unnecessary questions to code consumers).
Nonetheless, in the end it is you who has to weigh how much you want unit test code interfere into design (first case) or bend the perfect unit test definition (second case).
Usually, I find second case much more appealing - it doesn't clutter original class code and you'll most likely have Add already tested - it's safe to rely on it.
I think it's still a unit test, assuming MyObject is a simple object. I often construct input parameters to unit test methods.
I use Michael Feather's unit test criteria:
A test is not a unit test if:
It talks to the database
It communicates across the network
It touches the file system
It can't run at the same time as any of your other unit tests
You have to do special things to your environment (such as editing config files) to run it.
Tests that do these things aren't bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.
My 2 cents... how would the client know if remove worked or not ? How is a 'client' supposed to interact with this object? Are clients going to push in a stack to the history tracker? Treat the test as just another user/consumer/client of the test subject.. using exactly the same interaction as in real production.
I haven't heard of any rule stating that you're not allowed to call multiple methods on the object under test.
To simulate, stack is not empty. I'd just call Add - 99% case. I'd refrain from destroying the encapsulation of that object.. Treat objects like people (I think I read that in Object Thinking). Tell them to do stuff.. don't break-in and enter.
e.g. If you want someone to have some money in their wallet,
the simple way is to give them the money and let them internally put it into their wallet.
throw their wallet away and stuff in a wallet in their pocket.
I like Option1. Also see how it frees you from implementation details (which induce brittleness in tests). Let's say tomorrow the person decides to use an online wallet. The latter approach will break your tests - they will need to be updated for pushing in an online wallet now - even though the object behavior is not broken.
Another example I've seen is for testing Repository.GetX() where people break-in to the DB to inject records with SQL now in the unit test.. where it would have be considerably cleaner and easier to call Repository.AddX(x) first. Isolation is desired but not to the extent that it overrides pragmatism.
I hope I didn't come on too strong here.. it just pains me to see object APIs being 'contorted for testability' to the point where it no longer resembles the 'simplest thing that could work'.
I think you're trying to be a little overly specific with your definition of a unit test. You should be testing the public behavior of your class, not the minute implementation details.
From your code snippet, it looks like all you really need to care about is whether a) calling AddObjectToHistory causes IsAnyHistory to return true and b) RemoveLastObject eventually causes IsAnyHistory to return false.
As stated in the other answers I think your options can be broken down like so.
You take a dogmatic approach to your testing methodology and add constructor injection for the stack object so you can inject your own fake stack object and test your methods.
You write a separate test for add and remove, the remove test will use the add method but consider it a part of the test setup. As long as your add test passes, your remove should be too.
Here's my example:
[TestMethod]
public void NewAction_should_return_IndexAction()
{
NewViewModel viewModel = new NewViewModel()
{
Name = "José Inácio Santos Silva",
Email = "joseinacio#joseinacio.com",
Username = "joseinacio"
};
//IsUserRegistered is used to validate Username, Username is unique.
_mockAuthenticationService.Setup(x => x.IsUserRegistered(viewModel.Username )).Returns(false);
//IsUserRegistered is used to validate Email, Email is unique.
_mockUsuarioRepository.Setup(x => x.GetUserByEmail(viewModel.Email));
_mockDbContext.Setup(x => x.SaveChanges());
_mockUsuarioRepository.Setup(x => x.Add(It.IsAny<User>()));
_userController = new UserController(_mockUsuarioRepository.Object, _mockDbContext.Object, _mockAuthenticationService.Object);
ActionResult result = _userController.New(viewModel);
result.AssertActionRedirect().ToAction("Index");
_mockAuthenticationService.VerifyAll();
_mockUsuarioRepository.VerifyAll();
_mockDbContext.VerifyAll();
}
I have read some tutorials and they say that we should use only one mock per test.
But look at my test, it use 3 mocks, to check if my Action is working the right way I need to check these 3 mocks, do not agree?
How do I make this test in the correct way?
Each unit test should test only one thing.
In your unit test you are testing three mock objects. If the mockAuthenticationService fails, this will be reported and the unit test will stop there. Any errors with the other Mock objects are not reported and are effectively hidden.
In this situation you should create three unit tests, and in each one verify only one of the Mock objects. The rest are just used as stubs. (A stub is exactly the same as a Mock object, except you dont call VerifyAll on it at the end)
To avoid duplication and wasted effort, you should refactor that unit test so that most of the code is in a separate method. Each of the three unit tests calls this method and then verifies a single Mock.
You also have a test to ensure the correct redirect is called. This should also be in a separate test.
Quite simply:
[TestMethod]
public void NewAction_should_checkUserRegistered()
{
SetupTest();
_mockAuthenticationService.VerifyAll();
}
[TestMethod]
public void NewAction_should_GetUserByEmail()
{
SetupTest();
_mockUsuarioRepository.VerifyAll();
}
[TestMethod]
public void NewAction_should_SaveDBContext()
{
SetupTest();
_mockDbContext.VerifyAll();
}
[TestMethod]
public void NewAction_should_return_Redirects_Action()
{
var novoActionResult = SetupTest();
novoActionResult.AssertActionRedirect().ToAction("Index");
}
Short answer: "only one mock per test." is ambiguous. Use as many fakes as you need to isolate the code under test to a "unit" that is testing one condition.
It should be phrased: Only test one thing per test. If you are checking the state of more than one mock object you are probably testing more than one thing.
Long answer:
There is a lot to answer here to get the unit test written according to the best practices I have come across.
Common terminology from (The Art of Unit Testing), which I hope will come to be common:
Fake - an object that isolates the code under test from the rest of the application.
Stub - a simple fake object.
Mock - a fake object that stores what is passed to it, that you can inspect to verify the test.
Stubs and Mocks are both types of fake.
"only one mock per test." is wrong. You use as many fakes as you need to fully isolate the code under test from the rest of the application. If a method takes no parameters, there's nothing to fake. If a method takes a simple data type e.g. int, string, that doesn't have any complex behaviour, you don't need to fake it. If you have 2 repositories, context, a service object passed in, fake all of them, so no other production methods are being called.
You should have one condition per test as #Mongus Pong has said.
Test naming convention: MethodUnderTest_Condition_ExpectedBehaviour in this case you cannot do that as you have got more than one condition tested.
Test pattern: Arrange, Act, Assert. From your test, it seems as that is what you have done, but you have are arranging using private members. You should replace these with variables in each test, since the running order of tests is not always enforced, the state of these variables cannot be guaranteed, making your tests unreliable.
Buy a copy of "The Art of Unit Testing" http://artofunittesting.com/ it will answer a lot of more of your questions and is a great investment; one of the books that I'd grab if the office caught fire.
IMHO mocks and stubs are not that unique defined - every author uses them slightly different.
As I understand stubs "mock" behavior or "output" while you use mocks for example to check "input" into the mocked object/interface (like the Verify-Methods in MOQ).
If you see it this way then yes I too think you should only use one Mock because you should only test one thing - if you see it more like the stubs to inject testable interfaces then it's impossible to do.
If the VerifyAll is really needed here you indeed use 3 mocks, but I don't think they are nedded.
The best way to use Mock and stubs with Dev Magic Fake, so you can mock the UI and the DB for more information see Dev Magic Fake on codePlex
http://devmagicfake.codeplex.com/
Thanks
M.Radwan
In the past, I have only used Rhino Mocks, with the typical strict mock. I am now working with Moq on a project and I am wondering about the proper usage.
Let's assume that I have an object Foo with method Bar which calls a Bizz method on object Buzz.
In my test, I want to verify that Bizz is called, therefore I feel there are two possible options:
With a strict mock
var mockBuzz= new Mock<IBuzz>(MockBehavior.Strict);
mockBuzz.Setup(x => x.Bizz()); //test will fail if Bizz method not called
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.VerifyAll();
With a loose mock
var mockBuzz= new Mock<IBuzz>();
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.Verify(x => x.Bizz()) //test will fail if Bizz method not called
Is there a standard or normal way of doing this?
I used to use strict mocks when I first starting using mocks in unit tests. This didn't last very long. There are really 2 reasons why I stopped doing this:
The tests become brittle - With strict mocks you are asserting more than one thing, that the setup methods are called, AND that the other methods are not called. When you refactor the code the test often fails, even if what you are trying to test is still true.
The tests are harder to read - You need to have a setup for every method that is called on the mock, even if it's not really related to what you want to test. When someone reads this test it's difficult for them to tell what is important for the test and what is just a side effect of the implementation.
Because of these I would strongly recommend using loose mocks in your unit tests.
I have background in C++/non-.NET development and I've been more into .NET recently so I had certain expectations when I was using Moq for the first time. I was trying to understand WTF was going on with my test and why the code I was testing was throwing a random exception instead of the Mock library telling me which function the code was trying to call. So I discovered I needed to turn on the Strict behaviour, which was perplexing- and then I came across this question which I saw had no ticked answer yet.
The Loose mode, and the fact that it is the default is insane. What on earth is the point of a Mock library that does something completely unpredictable that you haven't explicitly listed it should do?
I completely disagree with the points listed in the other answers in support of Loose mode. There is no good reason to use it and I wouldn't ever want to, ever. When writing a unit test I want to be certain what is going on - if I know a function needs to return a null, I'll make it return that. I want my tests to be brittle (in the ways that matter) so that I can fix them and add to the suite of test code the setup lines which are the explicit information that is describing to me exactly what my software will do.
The question is - is there a standard and normal way of doing this?
Yes - from the point of view of programming in general, i.e. other languages and outside the .NET world, you should use Strict always. Goodness knows why it isn't the default in Moq.
I have a simple convention:
Use strict mocks when the system under test (SUT) is delegating the call to the underlying mocked layer without really modifying or applying any business logic to the arguments passed to itself.
Use loose mocks when the SUT applies business logic to the arguments passed to itself and passes on some derived/modified values to the mocked layer.
For eg:
Lets say we have database provider StudentDAL which has two methods:
Data access interface looks something like below:
public Student GetStudentById(int id);
public IList<Student> GetStudents(int ageFilter, int classId);
The implementation which consumes this DAL looks like below:
public Student FindStudent(int id)
{
//StudentDAL dependency injected
return StudentDAL.GetStudentById(id);
//Use strict mock to test this
}
public IList<Student> GetStudentsForClass(StudentListRequest studentListRequest)
{
//StudentDAL dependency injected
//age filter is derived from the request and then passed on to the underlying layer
int ageFilter = DateTime.Now.Year - studentListRequest.DateOfBirthFilter.Year;
return StudentDAL.GetStudents(ageFilter , studentListRequest.ClassId)
//Use loose mock and use verify api of MOQ to make sure that the age filter is correctly passed on.
}
Me personally, being new to mocking and Moq feel that starting off with Strict mode helps better understand of the innards and what's going on. "Loose" sometimes hides details and pass a test which a moq beginner may fail to see. Once you have your mocking skills down - Loose would probably be a lot more productive - like in this case saving a line with the "Setup" and just using "Verify" instead.
I'm using MS UnitTesting and trying to find my way around writing my first unit tests. It seems like all my unit tests start off with creating the same few objects...
[TestMethod]
CanCreateOrder()
{
<create an order>
...
}
[TestMethod]
CanSetOrderDeliveryAddress()
{
<create an order>
<create an address>
order.DeliveryAddress = address;
...
}
[TestMethod]
CanDispatchAnOrder()
{
<create an order>
<create an address>
order.DeliveryAddress = address;
order.Dispatch();
...
}
...etc
Is this normal, or have I got the wrong idea? I kind of thought every test was supposed to be independent, but how would it be possible to test Dispatch() without that implicitly relying on CreateOrder and SetDeliveryAddress already passing?
And the second part of the question, if the above approach looks ok, should I be using a factory or something to instantiate these objects in my test project? I'm not sure if a test project should only contain test classes/methods, or it's ok to add a bunch of helpers in there too.
You seem to be on the right track to me. In a lot of unit tests you will need to set up the state of the object you are testing in order to be able to test a particular part of behaviour.
It will help to group tests in classes when the bahaviour you are testing requires similar setup, and to use [TestInitialize] methods to reduce the duplication of that setup.
eg:
[TestClass]
public class WhenReadyToDispatch{
private Order order;
[TestInitialize]
public void Initialize
{
order = <create an order>
order.DeliveryAddress = <create an address>
}
[TestMethod]
CanChangeOrderDeliveryAddress()
{
order.DeliveryAddress = address;
}
[TestMethod]
CanDispatchAnOrder()
{
order.Dispatch();
}
}
It's fine to have helper classes in the test project - you should be aiming to make you test code well factored as your production code.
Your first question is to do with mocking and stubbing where you create a fake order and address created from the interface of each class. This is how you can then just be testing the Dispatch method.
You can then assert that the dispatch method did the right thing by checking what happened to your fake (stub/mock) objects.
In answer to the second part of your question;
It's a very good idea to have factory methods and even class hierarchies in order to make writing the tests easier. It's just as important to structure tests in a good way, as it is to structure production code.
I think part of your problem may be in the design of your order object. Trying to write a free standing test only to find that it relies on other functions generally suggest that they are not adequately decoupled. A couple of rules of thumb that may be appropriate here:
If Order.DeliveryAddress is just a simple getter/setter then don't worry about testing it. Thats like trying to prove that C# behaves as it should. There is little advantage to doing this. Conversely, having your dispatcher test rely on this property being in working order is not really a dependency.
However if Order.DeliveryAddress is performing logic, such as ensuring that the address is only modifiable for non-dispatched orders for example, then it is more complicated. You probably don't want to try to dispatch an entire order just to test that Order.DeliveryAddress is no longer modifiable afterwards.
Invoking Single Responsibility Principle (See 1 and 2) here would say that the Order class is now doing too much. It is both dispatching orders and enforcing object state integrity of the order data. In which case you probably want to split the dispatching functionality out into a DispatcherService that simply takes an order and dispatches it, setting an IsDispatched flag on the order in the process.
You can then test the DeliveryAddress behavior by just setting the IsDispatched property appropriately.
A third approach (which is sort of cheating but works well in situations where you are trying to get some testing over legacy objects) is to subclass Order to create a TestableOrder class that exposes to the test fixture the ability to tinker with the internal state of the class. In other words, it could expose a MarkAsDispatched() method that would set the classes internal IsDispatched flag and thus allow you to test that DeliveryAddress is only settable prior to being marked as dispatched.
Hope that helps.