In the past, I have only used Rhino Mocks, with the typical strict mock. I am now working with Moq on a project and I am wondering about the proper usage.
Let's assume that I have an object Foo with method Bar which calls a Bizz method on object Buzz.
In my test, I want to verify that Bizz is called, therefore I feel there are two possible options:
With a strict mock
var mockBuzz= new Mock<IBuzz>(MockBehavior.Strict);
mockBuzz.Setup(x => x.Bizz()); //test will fail if Bizz method not called
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.VerifyAll();
With a loose mock
var mockBuzz= new Mock<IBuzz>();
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.Verify(x => x.Bizz()) //test will fail if Bizz method not called
Is there a standard or normal way of doing this?
I used to use strict mocks when I first starting using mocks in unit tests. This didn't last very long. There are really 2 reasons why I stopped doing this:
The tests become brittle - With strict mocks you are asserting more than one thing, that the setup methods are called, AND that the other methods are not called. When you refactor the code the test often fails, even if what you are trying to test is still true.
The tests are harder to read - You need to have a setup for every method that is called on the mock, even if it's not really related to what you want to test. When someone reads this test it's difficult for them to tell what is important for the test and what is just a side effect of the implementation.
Because of these I would strongly recommend using loose mocks in your unit tests.
I have background in C++/non-.NET development and I've been more into .NET recently so I had certain expectations when I was using Moq for the first time. I was trying to understand WTF was going on with my test and why the code I was testing was throwing a random exception instead of the Mock library telling me which function the code was trying to call. So I discovered I needed to turn on the Strict behaviour, which was perplexing- and then I came across this question which I saw had no ticked answer yet.
The Loose mode, and the fact that it is the default is insane. What on earth is the point of a Mock library that does something completely unpredictable that you haven't explicitly listed it should do?
I completely disagree with the points listed in the other answers in support of Loose mode. There is no good reason to use it and I wouldn't ever want to, ever. When writing a unit test I want to be certain what is going on - if I know a function needs to return a null, I'll make it return that. I want my tests to be brittle (in the ways that matter) so that I can fix them and add to the suite of test code the setup lines which are the explicit information that is describing to me exactly what my software will do.
The question is - is there a standard and normal way of doing this?
Yes - from the point of view of programming in general, i.e. other languages and outside the .NET world, you should use Strict always. Goodness knows why it isn't the default in Moq.
I have a simple convention:
Use strict mocks when the system under test (SUT) is delegating the call to the underlying mocked layer without really modifying or applying any business logic to the arguments passed to itself.
Use loose mocks when the SUT applies business logic to the arguments passed to itself and passes on some derived/modified values to the mocked layer.
For eg:
Lets say we have database provider StudentDAL which has two methods:
Data access interface looks something like below:
public Student GetStudentById(int id);
public IList<Student> GetStudents(int ageFilter, int classId);
The implementation which consumes this DAL looks like below:
public Student FindStudent(int id)
{
//StudentDAL dependency injected
return StudentDAL.GetStudentById(id);
//Use strict mock to test this
}
public IList<Student> GetStudentsForClass(StudentListRequest studentListRequest)
{
//StudentDAL dependency injected
//age filter is derived from the request and then passed on to the underlying layer
int ageFilter = DateTime.Now.Year - studentListRequest.DateOfBirthFilter.Year;
return StudentDAL.GetStudents(ageFilter , studentListRequest.ClassId)
//Use loose mock and use verify api of MOQ to make sure that the age filter is correctly passed on.
}
Me personally, being new to mocking and Moq feel that starting off with Strict mode helps better understand of the innards and what's going on. "Loose" sometimes hides details and pass a test which a moq beginner may fail to see. Once you have your mocking skills down - Loose would probably be a lot more productive - like in this case saving a line with the "Setup" and just using "Verify" instead.
Related
I have a class that deals with Account stuff. It provides methods to login, reset password and create new accounts so far.
I inject the dependencies through the constructor. I have tests that validates each dependency's reference, if the reference is null it throws an ArgumentNullException.
The Account class exposes each of these dependencies through read only properties, I then have tests that validates if the reference passed on the constructor is the same that the property returns. I do this to make sure the references are being held by the class. (I don't know if this is a good practice too.)
First question: Is this a good practice in TDD? I ask this because this class has 6 dependencies so far, and it gets very repetitive and also the tests get pretty long as I have to mock all the dependencies for each test. What I do is just a copy and paste every time and just change the dependency's reference being tested.
Second question: my account creation method does things like validating the model passed, inserting data in 3 different tables or a forth table if a certain set of values are present and sending an email. What should I test here? I have so far a test that checks if the model validation gets executed, if the Add method of each repository gets called, and in this case, I use the Moq's Callback method of the mocked repository to compare each property being added to the repository against the ones I passed by the model.
Something like:
userRepository
.Setup(r => r.Add(It.IsAny<User>()))
.Callback<User>(u =>
{
Assert.AreEqual(model.Email, u.Email);
Assert.IsNotNull(u.PasswordHash);
//...
})
.Verifiable();
As I said, these tests are getting longer, I think that it doesn't hurt to test anything I can, but I don't know if it's worth it as it it's taking time to write the tests.
The purpose of testing is to find bugs.
Are you really going to have a bug where the property exists but is not initialized to the value from the constructor?
public class NoNotReally {
private IMyDependency1 _myDependency;
public IMyDependency1 MyDependency {get {return _myDependency;}}
public NoNotReally(IMyDependency dependency) {
_myDependency = null; // instead of dependency. Really?
}
}
Also, since you're using TDD, you should write the tests before you write the code, and the code should exist only to make the tests pass. Instead of your unnecessary tests of the properties, write a test that demonstrates that your injected dependency is being used. In order or such a test to pass, the dependency will need to exist, it will need to be of the correct type, and it will need to be used in the particular scenario.
In my example, the dependency will come to exist because it's needed, not because some artificial unit test required it to be there.
You say writing these tests feels repetitive. I say you feel the major benefit of TDD. Which is in fact not writing software with less bugs and not writing better software, because TDD doesn't guarantee either (at least not inherently). TDD forces you to think about design decisions and make design decisions all. The. Time. (And reduce debugging time.) If you feel pain while doing TDD, it's usually because a design decision is coming back to bite you. Then it's time to switch to your refactoring hat and improve the design.
Now in this particular case it's just the design of your tests, but you have to make design decisions for those as well.
As for testing whether properties are set. If I understand you correctly, you exposed those properties just for the sake of testing? In that case I'd advise against that. Assume you have a class with a constructor parameter and have a test that asserts the construtor should throw on null arguments:
public class MyClass
{
public MyClass(MyDependency dependency)
{
if (dependency == null)
{
throw new ArgumentNullException("dependency");
}
}
}
[Test]
public void ConstructorShouldThrowOnNullArgument()
{
Assert.Catch<ArgumentNullException>(() => new MyClass(null));
}
(TestFixture class omitted)
Now when you start to write a test for an actual business method of the class under test, the parts will start to fit together.
[Test]
public void TestSomeBusinessFunctionality()
{
MyDependency mockedDependency;
// setup mock
// mock calls on mockedDependency
MyClass myClass = new MyClass(mockedDependency);
var result = myClass.DoSomethingOrOther();
// assertions on result
// if necessary assertion on calls on mockedDependency
}
At that point, you will have to assign the injected dependency from the constructor to a field so you can use it in the method later. And if you manage to get the test to pass without using the dependency... well, heck, obviously you didn't need it to begin with. Or, maybe, you'll only start to need it for the next test.
About the other point. When it becomes a hassle to test all the reponsibilities of a method or class, TDD is telling you that the method/class is doing to much and would maybe like to be split up into parts that are easy to test. E.g. one class for verification, one for mapping and one for executing the storage calls.
That can very well lead to over-engineering, though! So watch out for that and you'll develop a feeling for when to resist the urge for more indirection. ;)
To test if properties are mapped properly, I'd suggest to use stubs or self-made fake objects which have simple properties. That way you can simply compare the source and target properties and don't have to make lengthy setups like the one you posted.
Normally in unit tests (especially in TDD), you are not going to test every single statement in the class that you are testing. The main purpose of the TDD unit tests is to test the business logic of the class, not the initialization stuff.
In other words, you give scenarios (remember to include edge cases too) as input and check the results, which can either be the final values of the properties and/or the return values of the methods.
The reason you don't want to test every single possible code path in your classes is because should you ever decide to refactor your classes later on, you only need to make minimal changes to your TDD unit tests, as they are supposed to be agnostic to the actual implementation (as much as possible).
Note: There are other types of unit tests, such as code coverage tests, that are meant to test every single code path in your classes. However, I personally find these tests impractical, and certainly not encouraged in TDD.
So I am playing around with mocking frameworks (Moq) for my unit tests, and was wondering when you should use a mocking framework?
What is the benefit/disadvantage between the following two tests:
public class Tests
{
[Fact]
public void TestWithMock()
{
// Arrange
var repo = new Mock<IRepository>();
var p = new Mock<Person>();
p.Setup(x => x.Id).Returns(1);
p.Setup(x => x.Name).Returns("Joe Blow");
p.Setup(x => x.AkaNames).Returns(new List<string> { "Joey", "Mugs" });
p.Setup(x => x.AkaNames.Remove(It.IsAny<string>()));
// Act
var service = new Service(repo.Object);
service.RemoveAkaName(p.Object, "Mugs");
// Assert
p.Verify(x => x.AkaNames.Remove("Mugs"), Times.Once());
}
[Fact]
public void TestWithoutMock()
{
// Arrange
var repo = new Mock<IRepository>();
var p = new Person { Id = 1, Name = "Joe Blow", AkaNames = new List<string> { "Joey", "Mugs" } };
// Act
var service = new Service(repo.Object);
service.RemoveAkaName(p, "Mugs");
// Assert
Assert.True(p.AkaNames.Count == 1);
Assert.True(p.AkaNames[0] == "Joey");
}
}
Use mock objects to truly create a unit test--a test where all dependencies are assumed to function correctly and all you want to know is if the SUT (system under test--a fancy way of saying the class you're testing) works.
The mock objects help to "guarantee" your dependencies function correctly because you create mock versions of those dependencies that produce results you configure. The question then becomes if the one class you're testing behaves as it should when everything else is "working."
Mock objects are particularly critical when you are testing an object with a slow dependency--like a database or a web service. If you were to really hit the database or make the real web service call, your test will take a lot more time to run. That's tolerable when it's only a few extra seconds, but when you have hundreds of tests running in a continuous integration server, that adds up really fast and cripples your automation.
This is what really makes mock objects important--reducing the build-test-deploy cycle time. Making sure your tests run fast is critical to efficient software development.
There are some rules I use writing unit-tests.
If my System Under Test (SUT) or Object Under Test has dependencies
then I mock them all.
If I test a method that returns result then I
check result only. If dependencies are passed as parameters of the method they should be mocked. (see 1)
If I test 'void' method then verifying mocks is the best option for testing.
There is an old one article by Martin Fowler Mocks Aren't Stubs.
In first test you use Mock and in the second one you use Stub.
Also I see some design issues that lead to your question.
If it is allowed to remove AkaName from AkaNames collection then it is OK to use stub and check state of the person. If you add specific method void RemoveAkaName(string name) into Person class then mocks should be used in order to verify its invocation. And logic of RemoveAkaName should be tested as part of Person class testing.
I would use stub for product and mock for repository for you code.
The mocking framework is used to remove the dependency, so the unit test will focus on the "Unit" to be tested. In your case, the person looks like a simple entity class, there is no need to use Mocking for it.
Mocking has many benefits especially in agile programming where many quick release cycles means that sytem strucutre and real data might be incomplete. In such cases you can mock a repository to mimic production code in order to continue work on ui, or services. This is usually complemented with an IoC mechanism like Ninject to simplify the switch to the real repositories. The two examples you give are equal and without any other context I would say it's a matter of choice between them. The fluent api in moq might be easier to read as its kind of self documenting. That's my opinion though ;)
Mock is used to test object that cannot function in isolation. suppose function A is dependent on function B, to perform unit testing on function A we even end up testing function B.By using mock you can simulate the functionality of function B and testing can be focussed only on function A.
A mocking framework is useful for simulating the integration points in the code under test. I would argue that your concrete example is not a good candidate for a mocking framework as you can already inject the dependency (Person) directly into the code. Using a mocking framework actually complicates it in this case.
A much better use case is if you have a repository making a call to a database. It is desirable from a unit test perspective to mock the db call and return predetermined data instead. The key advantages of this is removing dependencies on exiting data, but also performance since the db call will slow the test down.
When should you use mocks? Almost never.
Mocks turn your tests into white-box tests, which are very labor intensive to write and extremely labor intensive to maintain. This may not be an issue if you are in the medical industry, aerospace industry, or other high-criticality software business, but if you are writing regular commercial grade software, your tests should be black-box tests. In other words, you should be testing against the interface, not against the implementation.
What to use instead of mocks?
Use fakes. Martin Fowler has an article explaining the difference here: https://martinfowler.com/bliki/TestDouble.html but to give you an example, an in-memory database can be used as fake in place of a full-blown RDBMS. (Note how fakes are a lot less fake than mocks.)
I am abstracting the history tracking portion of a class of mine so that it looks like this:
private readonly Stack<MyObject> _pastHistory = new Stack<MyObject>();
internal virtual Boolean IsAnyHistory { get { return _pastHistory.Any(); } }
internal virtual void AddObjectToHistory(MyObject myObject)
{
if (myObject == null) throw new ArgumentNullException("myObject");
_pastHistory.Push(myObject);
}
internal virtual MyObject RemoveLastObject()
{
if(!IsAnyHistory) throw new InvalidOperationException("There is no previous history.");
return _pastHistory.Pop();
}
My problem is that I would like to unit test that Remove will return the last Added object.
AddObjectToHistory
RemoveObjectToHistory -> returns what was put in via AddObjectToHistory
However, it isn't really a unit test if I have to call Add first? But, the only way that I can see to do this in a true unit test way is to pass in the Stack object in the constructor OR mock out IsAnyHistory...but mocking my SUT is odd also. So, my question is, from a dogmatic view is this a unit test? If not, how do I clean it up...is constructor injection my only way? It just seems like a stretch to have to pass in a simple object? Is it ok to push even this simple object out to be injected?
There are two approaches to those scenarios:
Interfere into design, like making _pastHistory internal/protected or injecting stack
Use other (possibly unit tested) methods to perform verification
As always, there is no golden rule, although I'd say you generally should avoid situations where unit tests force design changes (as those changes will most likely introduce ambiguity/unnecessary questions to code consumers).
Nonetheless, in the end it is you who has to weigh how much you want unit test code interfere into design (first case) or bend the perfect unit test definition (second case).
Usually, I find second case much more appealing - it doesn't clutter original class code and you'll most likely have Add already tested - it's safe to rely on it.
I think it's still a unit test, assuming MyObject is a simple object. I often construct input parameters to unit test methods.
I use Michael Feather's unit test criteria:
A test is not a unit test if:
It talks to the database
It communicates across the network
It touches the file system
It can't run at the same time as any of your other unit tests
You have to do special things to your environment (such as editing config files) to run it.
Tests that do these things aren't bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.
My 2 cents... how would the client know if remove worked or not ? How is a 'client' supposed to interact with this object? Are clients going to push in a stack to the history tracker? Treat the test as just another user/consumer/client of the test subject.. using exactly the same interaction as in real production.
I haven't heard of any rule stating that you're not allowed to call multiple methods on the object under test.
To simulate, stack is not empty. I'd just call Add - 99% case. I'd refrain from destroying the encapsulation of that object.. Treat objects like people (I think I read that in Object Thinking). Tell them to do stuff.. don't break-in and enter.
e.g. If you want someone to have some money in their wallet,
the simple way is to give them the money and let them internally put it into their wallet.
throw their wallet away and stuff in a wallet in their pocket.
I like Option1. Also see how it frees you from implementation details (which induce brittleness in tests). Let's say tomorrow the person decides to use an online wallet. The latter approach will break your tests - they will need to be updated for pushing in an online wallet now - even though the object behavior is not broken.
Another example I've seen is for testing Repository.GetX() where people break-in to the DB to inject records with SQL now in the unit test.. where it would have be considerably cleaner and easier to call Repository.AddX(x) first. Isolation is desired but not to the extent that it overrides pragmatism.
I hope I didn't come on too strong here.. it just pains me to see object APIs being 'contorted for testability' to the point where it no longer resembles the 'simplest thing that could work'.
I think you're trying to be a little overly specific with your definition of a unit test. You should be testing the public behavior of your class, not the minute implementation details.
From your code snippet, it looks like all you really need to care about is whether a) calling AddObjectToHistory causes IsAnyHistory to return true and b) RemoveLastObject eventually causes IsAnyHistory to return false.
As stated in the other answers I think your options can be broken down like so.
You take a dogmatic approach to your testing methodology and add constructor injection for the stack object so you can inject your own fake stack object and test your methods.
You write a separate test for add and remove, the remove test will use the add method but consider it a part of the test setup. As long as your add test passes, your remove should be too.
I have been battling with writing some unit test for some of our code and I am struggling with this one:
We have a method that we overloaded and it is like this:
Public Client GetClient(int productID)
{
//Some sql that evaluate a client
if(!GetClient(clientRef,ClientTypeRef))
Return Client.UnknownClient;
//some other sql and codes
Return Client.CustomerClient;
}
The problem is how do I approach this, in my test I tried to add a mock to the GetClient(clientRef,ClientTypeRef) and returning an OK Client (anything other than Client.UnknownClient) to allow me to continue but I am getting a null reference? Is it possible to mock and test such methods, and how would I continue with this.
One of the reasons why unit testing has become so popular was that it was shown to encourage SOLID design principles.
Based on the little piece of code you've included, I think your difficulty may come from the fact that your GetClient(int productID) has too many responsibilities so you're struggling to test them separately.
The way I understand it, you are:
loading a client from the DB (by product Id?)
checking (with complex logic involving more queries to the db?) what kind of client it is
Rather than mocking GetClient(clientRef,ClientTypeRef) so that you can test every logical path under it, I would encourage you (if possible) to try and refactor your code, so that you separate the loading of the client from the checking of the client's type and possibly also some of the logic you group under //some other sql and codes.
That should make it easier to test every piece separately, but above all would make your code more maintainable.
What mocking frame work are you using?
I use moq, to mock this method you would do something like
_mockClientRepository.Setup(x => x.GetClient(It.IsAny<int>(),It.IsAny<int>())).Returns(Client.CustomerClient);
Basically this means that any integer passed to the method will always return Client.CustomerClient
Here's my example:
[TestMethod]
public void NewAction_should_return_IndexAction()
{
NewViewModel viewModel = new NewViewModel()
{
Name = "José Inácio Santos Silva",
Email = "joseinacio#joseinacio.com",
Username = "joseinacio"
};
//IsUserRegistered is used to validate Username, Username is unique.
_mockAuthenticationService.Setup(x => x.IsUserRegistered(viewModel.Username )).Returns(false);
//IsUserRegistered is used to validate Email, Email is unique.
_mockUsuarioRepository.Setup(x => x.GetUserByEmail(viewModel.Email));
_mockDbContext.Setup(x => x.SaveChanges());
_mockUsuarioRepository.Setup(x => x.Add(It.IsAny<User>()));
_userController = new UserController(_mockUsuarioRepository.Object, _mockDbContext.Object, _mockAuthenticationService.Object);
ActionResult result = _userController.New(viewModel);
result.AssertActionRedirect().ToAction("Index");
_mockAuthenticationService.VerifyAll();
_mockUsuarioRepository.VerifyAll();
_mockDbContext.VerifyAll();
}
I have read some tutorials and they say that we should use only one mock per test.
But look at my test, it use 3 mocks, to check if my Action is working the right way I need to check these 3 mocks, do not agree?
How do I make this test in the correct way?
Each unit test should test only one thing.
In your unit test you are testing three mock objects. If the mockAuthenticationService fails, this will be reported and the unit test will stop there. Any errors with the other Mock objects are not reported and are effectively hidden.
In this situation you should create three unit tests, and in each one verify only one of the Mock objects. The rest are just used as stubs. (A stub is exactly the same as a Mock object, except you dont call VerifyAll on it at the end)
To avoid duplication and wasted effort, you should refactor that unit test so that most of the code is in a separate method. Each of the three unit tests calls this method and then verifies a single Mock.
You also have a test to ensure the correct redirect is called. This should also be in a separate test.
Quite simply:
[TestMethod]
public void NewAction_should_checkUserRegistered()
{
SetupTest();
_mockAuthenticationService.VerifyAll();
}
[TestMethod]
public void NewAction_should_GetUserByEmail()
{
SetupTest();
_mockUsuarioRepository.VerifyAll();
}
[TestMethod]
public void NewAction_should_SaveDBContext()
{
SetupTest();
_mockDbContext.VerifyAll();
}
[TestMethod]
public void NewAction_should_return_Redirects_Action()
{
var novoActionResult = SetupTest();
novoActionResult.AssertActionRedirect().ToAction("Index");
}
Short answer: "only one mock per test." is ambiguous. Use as many fakes as you need to isolate the code under test to a "unit" that is testing one condition.
It should be phrased: Only test one thing per test. If you are checking the state of more than one mock object you are probably testing more than one thing.
Long answer:
There is a lot to answer here to get the unit test written according to the best practices I have come across.
Common terminology from (The Art of Unit Testing), which I hope will come to be common:
Fake - an object that isolates the code under test from the rest of the application.
Stub - a simple fake object.
Mock - a fake object that stores what is passed to it, that you can inspect to verify the test.
Stubs and Mocks are both types of fake.
"only one mock per test." is wrong. You use as many fakes as you need to fully isolate the code under test from the rest of the application. If a method takes no parameters, there's nothing to fake. If a method takes a simple data type e.g. int, string, that doesn't have any complex behaviour, you don't need to fake it. If you have 2 repositories, context, a service object passed in, fake all of them, so no other production methods are being called.
You should have one condition per test as #Mongus Pong has said.
Test naming convention: MethodUnderTest_Condition_ExpectedBehaviour in this case you cannot do that as you have got more than one condition tested.
Test pattern: Arrange, Act, Assert. From your test, it seems as that is what you have done, but you have are arranging using private members. You should replace these with variables in each test, since the running order of tests is not always enforced, the state of these variables cannot be guaranteed, making your tests unreliable.
Buy a copy of "The Art of Unit Testing" http://artofunittesting.com/ it will answer a lot of more of your questions and is a great investment; one of the books that I'd grab if the office caught fire.
IMHO mocks and stubs are not that unique defined - every author uses them slightly different.
As I understand stubs "mock" behavior or "output" while you use mocks for example to check "input" into the mocked object/interface (like the Verify-Methods in MOQ).
If you see it this way then yes I too think you should only use one Mock because you should only test one thing - if you see it more like the stubs to inject testable interfaces then it's impossible to do.
If the VerifyAll is really needed here you indeed use 3 mocks, but I don't think they are nedded.
The best way to use Mock and stubs with Dev Magic Fake, so you can mock the UI and the DB for more information see Dev Magic Fake on codePlex
http://devmagicfake.codeplex.com/
Thanks
M.Radwan