So I am playing around with mocking frameworks (Moq) for my unit tests, and was wondering when you should use a mocking framework?
What is the benefit/disadvantage between the following two tests:
public class Tests
{
[Fact]
public void TestWithMock()
{
// Arrange
var repo = new Mock<IRepository>();
var p = new Mock<Person>();
p.Setup(x => x.Id).Returns(1);
p.Setup(x => x.Name).Returns("Joe Blow");
p.Setup(x => x.AkaNames).Returns(new List<string> { "Joey", "Mugs" });
p.Setup(x => x.AkaNames.Remove(It.IsAny<string>()));
// Act
var service = new Service(repo.Object);
service.RemoveAkaName(p.Object, "Mugs");
// Assert
p.Verify(x => x.AkaNames.Remove("Mugs"), Times.Once());
}
[Fact]
public void TestWithoutMock()
{
// Arrange
var repo = new Mock<IRepository>();
var p = new Person { Id = 1, Name = "Joe Blow", AkaNames = new List<string> { "Joey", "Mugs" } };
// Act
var service = new Service(repo.Object);
service.RemoveAkaName(p, "Mugs");
// Assert
Assert.True(p.AkaNames.Count == 1);
Assert.True(p.AkaNames[0] == "Joey");
}
}
Use mock objects to truly create a unit test--a test where all dependencies are assumed to function correctly and all you want to know is if the SUT (system under test--a fancy way of saying the class you're testing) works.
The mock objects help to "guarantee" your dependencies function correctly because you create mock versions of those dependencies that produce results you configure. The question then becomes if the one class you're testing behaves as it should when everything else is "working."
Mock objects are particularly critical when you are testing an object with a slow dependency--like a database or a web service. If you were to really hit the database or make the real web service call, your test will take a lot more time to run. That's tolerable when it's only a few extra seconds, but when you have hundreds of tests running in a continuous integration server, that adds up really fast and cripples your automation.
This is what really makes mock objects important--reducing the build-test-deploy cycle time. Making sure your tests run fast is critical to efficient software development.
There are some rules I use writing unit-tests.
If my System Under Test (SUT) or Object Under Test has dependencies
then I mock them all.
If I test a method that returns result then I
check result only. If dependencies are passed as parameters of the method they should be mocked. (see 1)
If I test 'void' method then verifying mocks is the best option for testing.
There is an old one article by Martin Fowler Mocks Aren't Stubs.
In first test you use Mock and in the second one you use Stub.
Also I see some design issues that lead to your question.
If it is allowed to remove AkaName from AkaNames collection then it is OK to use stub and check state of the person. If you add specific method void RemoveAkaName(string name) into Person class then mocks should be used in order to verify its invocation. And logic of RemoveAkaName should be tested as part of Person class testing.
I would use stub for product and mock for repository for you code.
The mocking framework is used to remove the dependency, so the unit test will focus on the "Unit" to be tested. In your case, the person looks like a simple entity class, there is no need to use Mocking for it.
Mocking has many benefits especially in agile programming where many quick release cycles means that sytem strucutre and real data might be incomplete. In such cases you can mock a repository to mimic production code in order to continue work on ui, or services. This is usually complemented with an IoC mechanism like Ninject to simplify the switch to the real repositories. The two examples you give are equal and without any other context I would say it's a matter of choice between them. The fluent api in moq might be easier to read as its kind of self documenting. That's my opinion though ;)
Mock is used to test object that cannot function in isolation. suppose function A is dependent on function B, to perform unit testing on function A we even end up testing function B.By using mock you can simulate the functionality of function B and testing can be focussed only on function A.
A mocking framework is useful for simulating the integration points in the code under test. I would argue that your concrete example is not a good candidate for a mocking framework as you can already inject the dependency (Person) directly into the code. Using a mocking framework actually complicates it in this case.
A much better use case is if you have a repository making a call to a database. It is desirable from a unit test perspective to mock the db call and return predetermined data instead. The key advantages of this is removing dependencies on exiting data, but also performance since the db call will slow the test down.
When should you use mocks? Almost never.
Mocks turn your tests into white-box tests, which are very labor intensive to write and extremely labor intensive to maintain. This may not be an issue if you are in the medical industry, aerospace industry, or other high-criticality software business, but if you are writing regular commercial grade software, your tests should be black-box tests. In other words, you should be testing against the interface, not against the implementation.
What to use instead of mocks?
Use fakes. Martin Fowler has an article explaining the difference here: https://martinfowler.com/bliki/TestDouble.html but to give you an example, an in-memory database can be used as fake in place of a full-blown RDBMS. (Note how fakes are a lot less fake than mocks.)
Related
I have a class that deals with Account stuff. It provides methods to login, reset password and create new accounts so far.
I inject the dependencies through the constructor. I have tests that validates each dependency's reference, if the reference is null it throws an ArgumentNullException.
The Account class exposes each of these dependencies through read only properties, I then have tests that validates if the reference passed on the constructor is the same that the property returns. I do this to make sure the references are being held by the class. (I don't know if this is a good practice too.)
First question: Is this a good practice in TDD? I ask this because this class has 6 dependencies so far, and it gets very repetitive and also the tests get pretty long as I have to mock all the dependencies for each test. What I do is just a copy and paste every time and just change the dependency's reference being tested.
Second question: my account creation method does things like validating the model passed, inserting data in 3 different tables or a forth table if a certain set of values are present and sending an email. What should I test here? I have so far a test that checks if the model validation gets executed, if the Add method of each repository gets called, and in this case, I use the Moq's Callback method of the mocked repository to compare each property being added to the repository against the ones I passed by the model.
Something like:
userRepository
.Setup(r => r.Add(It.IsAny<User>()))
.Callback<User>(u =>
{
Assert.AreEqual(model.Email, u.Email);
Assert.IsNotNull(u.PasswordHash);
//...
})
.Verifiable();
As I said, these tests are getting longer, I think that it doesn't hurt to test anything I can, but I don't know if it's worth it as it it's taking time to write the tests.
The purpose of testing is to find bugs.
Are you really going to have a bug where the property exists but is not initialized to the value from the constructor?
public class NoNotReally {
private IMyDependency1 _myDependency;
public IMyDependency1 MyDependency {get {return _myDependency;}}
public NoNotReally(IMyDependency dependency) {
_myDependency = null; // instead of dependency. Really?
}
}
Also, since you're using TDD, you should write the tests before you write the code, and the code should exist only to make the tests pass. Instead of your unnecessary tests of the properties, write a test that demonstrates that your injected dependency is being used. In order or such a test to pass, the dependency will need to exist, it will need to be of the correct type, and it will need to be used in the particular scenario.
In my example, the dependency will come to exist because it's needed, not because some artificial unit test required it to be there.
You say writing these tests feels repetitive. I say you feel the major benefit of TDD. Which is in fact not writing software with less bugs and not writing better software, because TDD doesn't guarantee either (at least not inherently). TDD forces you to think about design decisions and make design decisions all. The. Time. (And reduce debugging time.) If you feel pain while doing TDD, it's usually because a design decision is coming back to bite you. Then it's time to switch to your refactoring hat and improve the design.
Now in this particular case it's just the design of your tests, but you have to make design decisions for those as well.
As for testing whether properties are set. If I understand you correctly, you exposed those properties just for the sake of testing? In that case I'd advise against that. Assume you have a class with a constructor parameter and have a test that asserts the construtor should throw on null arguments:
public class MyClass
{
public MyClass(MyDependency dependency)
{
if (dependency == null)
{
throw new ArgumentNullException("dependency");
}
}
}
[Test]
public void ConstructorShouldThrowOnNullArgument()
{
Assert.Catch<ArgumentNullException>(() => new MyClass(null));
}
(TestFixture class omitted)
Now when you start to write a test for an actual business method of the class under test, the parts will start to fit together.
[Test]
public void TestSomeBusinessFunctionality()
{
MyDependency mockedDependency;
// setup mock
// mock calls on mockedDependency
MyClass myClass = new MyClass(mockedDependency);
var result = myClass.DoSomethingOrOther();
// assertions on result
// if necessary assertion on calls on mockedDependency
}
At that point, you will have to assign the injected dependency from the constructor to a field so you can use it in the method later. And if you manage to get the test to pass without using the dependency... well, heck, obviously you didn't need it to begin with. Or, maybe, you'll only start to need it for the next test.
About the other point. When it becomes a hassle to test all the reponsibilities of a method or class, TDD is telling you that the method/class is doing to much and would maybe like to be split up into parts that are easy to test. E.g. one class for verification, one for mapping and one for executing the storage calls.
That can very well lead to over-engineering, though! So watch out for that and you'll develop a feeling for when to resist the urge for more indirection. ;)
To test if properties are mapped properly, I'd suggest to use stubs or self-made fake objects which have simple properties. That way you can simply compare the source and target properties and don't have to make lengthy setups like the one you posted.
Normally in unit tests (especially in TDD), you are not going to test every single statement in the class that you are testing. The main purpose of the TDD unit tests is to test the business logic of the class, not the initialization stuff.
In other words, you give scenarios (remember to include edge cases too) as input and check the results, which can either be the final values of the properties and/or the return values of the methods.
The reason you don't want to test every single possible code path in your classes is because should you ever decide to refactor your classes later on, you only need to make minimal changes to your TDD unit tests, as they are supposed to be agnostic to the actual implementation (as much as possible).
Note: There are other types of unit tests, such as code coverage tests, that are meant to test every single code path in your classes. However, I personally find these tests impractical, and certainly not encouraged in TDD.
Haven't really used much unit testing before, but read up on it a bit and got the idea that you really only should test 1 thing at a time. But how to do this in a nice way when for example saving and retrieving an object? I can't see that the save worked without using the "retrieve" function. And can't test the retrieve without saving something. At the moment I tried something like this... How can I assure that my test can know which one is not working?
[TestMethod]
public void TestSaveObject()
{
TestStorage storage = new TestStorage();
ObejctToSave s1 = new ObejctToSave {Name = "TEST1"};
ObejctToSave s2 = new ObejctToSave { Name = "TEST2" };
storage.SaveObject(s1);
storage.SaveObject(s2);
List<ObjectToSave> objects = storage.GetObjects();
Assert.AreEqual(2, objects.Count);
Assert.AreEqual("TEST1", objects[0].Name);
Assert.AreEqual("TEST2", objects[1].Name);
}
You noted yourself that unit-tests ought to test one thing at a time. And yet here you're testing two things - storage and retrieval.
If you want to test your service layer for handling persistence correctly, mock the persistence object (repository), and then call service methods to add object - verifying that appropriate methods on repository were called. The same for retrieval.
The main issue is whether:
you are implementing a persistence library. If yes, you should of course test peristence methods, using mock objects that will fake OS calls to file system operations.
you want to test your persistence methods (as your example suggests), but they are using 3rd party library. It doesn't make sense for unit-tests - this is the part when integration testing plays its role.
Briefly speaking - unit test tests a single unit - a "module" of your code separately from other modules. Other parts are being mocked for the purpose of verifying only the code of the unit being tested.
Integration test on the other hand tests a group of modules working together. Often integration tests are implemented to tests typical use cases of your whole system, sometimes they are used for regression testing of only a group of modules for example. There are many possibilities, but the point is that modules are being tested working together - hence integration.
I like the Idea to test one function by using it-s inverse-function
even if this means testing two things at a time and this is probably an integrationtest and not a unittest.
However there are some problems i experienced doing this.
sort order of lists false negative: In your example storage.GetObjects() might retrieve the ojects in the wrong order so your test might fail although every single object was handled correctly. The same applies for objects with sub-lists.
maintaining this test: if you add new properties to ObejctToSave and forget to update the corresponding test you get a false positive - ObejctToSave tells you success althoug the new property is not persisted properly.
For a workaround for these limitations in dotnet i did this
using nbuilder library that automatically assigns a reproducable value to each property so new properties are automatically tested
created a custom object-compare method compare<t>(t object1, t object2, params string[] propertyNamesToBeIgnoredInTest) that uses reflection to compare the properties and sorts sub-list before comparing them.
Interesting point. I think to do it in the most proper TDD way ( that wouldn't be necessary the way it should ) is to create a TestStorage mock, and assert the proper calls are satisfied on it. So you can create a separate test for the Save an Retrieve, both with the mock and proper expectations on it.
Here's my example:
[TestMethod]
public void NewAction_should_return_IndexAction()
{
NewViewModel viewModel = new NewViewModel()
{
Name = "José Inácio Santos Silva",
Email = "joseinacio#joseinacio.com",
Username = "joseinacio"
};
//IsUserRegistered is used to validate Username, Username is unique.
_mockAuthenticationService.Setup(x => x.IsUserRegistered(viewModel.Username )).Returns(false);
//IsUserRegistered is used to validate Email, Email is unique.
_mockUsuarioRepository.Setup(x => x.GetUserByEmail(viewModel.Email));
_mockDbContext.Setup(x => x.SaveChanges());
_mockUsuarioRepository.Setup(x => x.Add(It.IsAny<User>()));
_userController = new UserController(_mockUsuarioRepository.Object, _mockDbContext.Object, _mockAuthenticationService.Object);
ActionResult result = _userController.New(viewModel);
result.AssertActionRedirect().ToAction("Index");
_mockAuthenticationService.VerifyAll();
_mockUsuarioRepository.VerifyAll();
_mockDbContext.VerifyAll();
}
I have read some tutorials and they say that we should use only one mock per test.
But look at my test, it use 3 mocks, to check if my Action is working the right way I need to check these 3 mocks, do not agree?
How do I make this test in the correct way?
Each unit test should test only one thing.
In your unit test you are testing three mock objects. If the mockAuthenticationService fails, this will be reported and the unit test will stop there. Any errors with the other Mock objects are not reported and are effectively hidden.
In this situation you should create three unit tests, and in each one verify only one of the Mock objects. The rest are just used as stubs. (A stub is exactly the same as a Mock object, except you dont call VerifyAll on it at the end)
To avoid duplication and wasted effort, you should refactor that unit test so that most of the code is in a separate method. Each of the three unit tests calls this method and then verifies a single Mock.
You also have a test to ensure the correct redirect is called. This should also be in a separate test.
Quite simply:
[TestMethod]
public void NewAction_should_checkUserRegistered()
{
SetupTest();
_mockAuthenticationService.VerifyAll();
}
[TestMethod]
public void NewAction_should_GetUserByEmail()
{
SetupTest();
_mockUsuarioRepository.VerifyAll();
}
[TestMethod]
public void NewAction_should_SaveDBContext()
{
SetupTest();
_mockDbContext.VerifyAll();
}
[TestMethod]
public void NewAction_should_return_Redirects_Action()
{
var novoActionResult = SetupTest();
novoActionResult.AssertActionRedirect().ToAction("Index");
}
Short answer: "only one mock per test." is ambiguous. Use as many fakes as you need to isolate the code under test to a "unit" that is testing one condition.
It should be phrased: Only test one thing per test. If you are checking the state of more than one mock object you are probably testing more than one thing.
Long answer:
There is a lot to answer here to get the unit test written according to the best practices I have come across.
Common terminology from (The Art of Unit Testing), which I hope will come to be common:
Fake - an object that isolates the code under test from the rest of the application.
Stub - a simple fake object.
Mock - a fake object that stores what is passed to it, that you can inspect to verify the test.
Stubs and Mocks are both types of fake.
"only one mock per test." is wrong. You use as many fakes as you need to fully isolate the code under test from the rest of the application. If a method takes no parameters, there's nothing to fake. If a method takes a simple data type e.g. int, string, that doesn't have any complex behaviour, you don't need to fake it. If you have 2 repositories, context, a service object passed in, fake all of them, so no other production methods are being called.
You should have one condition per test as #Mongus Pong has said.
Test naming convention: MethodUnderTest_Condition_ExpectedBehaviour in this case you cannot do that as you have got more than one condition tested.
Test pattern: Arrange, Act, Assert. From your test, it seems as that is what you have done, but you have are arranging using private members. You should replace these with variables in each test, since the running order of tests is not always enforced, the state of these variables cannot be guaranteed, making your tests unreliable.
Buy a copy of "The Art of Unit Testing" http://artofunittesting.com/ it will answer a lot of more of your questions and is a great investment; one of the books that I'd grab if the office caught fire.
IMHO mocks and stubs are not that unique defined - every author uses them slightly different.
As I understand stubs "mock" behavior or "output" while you use mocks for example to check "input" into the mocked object/interface (like the Verify-Methods in MOQ).
If you see it this way then yes I too think you should only use one Mock because you should only test one thing - if you see it more like the stubs to inject testable interfaces then it's impossible to do.
If the VerifyAll is really needed here you indeed use 3 mocks, but I don't think they are nedded.
The best way to use Mock and stubs with Dev Magic Fake, so you can mock the UI and the DB for more information see Dev Magic Fake on codePlex
http://devmagicfake.codeplex.com/
Thanks
M.Radwan
I am working on a large codebase with basically no unit test coverage. We are about to start moving toward a more test-driven approach, so I thought I would try to write a unit test for a very simple function I added, basically
class ClassUnderTest {
public void SetNoMatchingImage() {
currentState.State = FSMState.NoMatchingImage;
... // Some more logic
NotifyViews();
}
public ViewStatus GetStatus() {
...
if (currentState.State == FSMState.NoMatchingImage)
return ViewStatus.EmptyScreen;
...
}
...
}
Ok, so test this, I would just like to do:
[Test]
public void TestSetNoMatchingImage() {
ClassUnderTest view = new ClassUnderTest(...);
view.SetNoMatchingImage();
Assert.AreEqual(ViewStatus.EmptyScreen, view.Status);
}
But my problem here is that the ClassUnderTest constructor takes 3 arguments to non-interfaces that cannot be null, so I cannot easily create a ClassUnderTest. I can try to either create instances of these classes or stub them, but the problem is the same for them: each of the contructors take arguments that has to be created. And the problem is the same for... and so on. The result is of course a very large overhead and a lot of code needed even for very simple tests.
Is there a good way of dealing with cases like this to make the test cases easier to write?
You'll have lots of situations like this when you start refactoring a project without tests if it wasn't design with Dependency Injection in mind and the mocking framework you use cannot mock concrete classes (such as NMock).
As Andriys just mentioned, Typemock (and moq too) can mock concrete classes as long as it got virtual members.
Personally, I would extract an interface from each of those three classes and inject the interfaces as part of some refactoring to make the class easy to test. I can't remember if VS has a refactor to extract an interface in 2 clicks, which wouldn't take too long.
I would recommend looking at Typemock Isolator framework. According to Art of Unit Testing book by Roy Osherove, it's your best bet when writing unit tests for legacy code:
...it’s the only one [framework] that allows you to create stubs and mocks of dependencies in production code without needing to refactor it at all, saving valuable time in bringing a component under test.
Cheers!
I'd second the recommendation for Typemock, and the solutions suggested in the other answers. In addition to what's already being said Michael Feathers has written a book dealing with the patterns that you're bumping up against called 'Working Effectively With Legacy Code' -
http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052/ref=sr_1_1?ie=UTF8&qid=1313835584&sr=8-1
There's a PDF extract here - http://www.objectmentor.com/resources/articles/WorkingEffectivelyWithLegacyCode.pdf
In the past, I have only used Rhino Mocks, with the typical strict mock. I am now working with Moq on a project and I am wondering about the proper usage.
Let's assume that I have an object Foo with method Bar which calls a Bizz method on object Buzz.
In my test, I want to verify that Bizz is called, therefore I feel there are two possible options:
With a strict mock
var mockBuzz= new Mock<IBuzz>(MockBehavior.Strict);
mockBuzz.Setup(x => x.Bizz()); //test will fail if Bizz method not called
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.VerifyAll();
With a loose mock
var mockBuzz= new Mock<IBuzz>();
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.Verify(x => x.Bizz()) //test will fail if Bizz method not called
Is there a standard or normal way of doing this?
I used to use strict mocks when I first starting using mocks in unit tests. This didn't last very long. There are really 2 reasons why I stopped doing this:
The tests become brittle - With strict mocks you are asserting more than one thing, that the setup methods are called, AND that the other methods are not called. When you refactor the code the test often fails, even if what you are trying to test is still true.
The tests are harder to read - You need to have a setup for every method that is called on the mock, even if it's not really related to what you want to test. When someone reads this test it's difficult for them to tell what is important for the test and what is just a side effect of the implementation.
Because of these I would strongly recommend using loose mocks in your unit tests.
I have background in C++/non-.NET development and I've been more into .NET recently so I had certain expectations when I was using Moq for the first time. I was trying to understand WTF was going on with my test and why the code I was testing was throwing a random exception instead of the Mock library telling me which function the code was trying to call. So I discovered I needed to turn on the Strict behaviour, which was perplexing- and then I came across this question which I saw had no ticked answer yet.
The Loose mode, and the fact that it is the default is insane. What on earth is the point of a Mock library that does something completely unpredictable that you haven't explicitly listed it should do?
I completely disagree with the points listed in the other answers in support of Loose mode. There is no good reason to use it and I wouldn't ever want to, ever. When writing a unit test I want to be certain what is going on - if I know a function needs to return a null, I'll make it return that. I want my tests to be brittle (in the ways that matter) so that I can fix them and add to the suite of test code the setup lines which are the explicit information that is describing to me exactly what my software will do.
The question is - is there a standard and normal way of doing this?
Yes - from the point of view of programming in general, i.e. other languages and outside the .NET world, you should use Strict always. Goodness knows why it isn't the default in Moq.
I have a simple convention:
Use strict mocks when the system under test (SUT) is delegating the call to the underlying mocked layer without really modifying or applying any business logic to the arguments passed to itself.
Use loose mocks when the SUT applies business logic to the arguments passed to itself and passes on some derived/modified values to the mocked layer.
For eg:
Lets say we have database provider StudentDAL which has two methods:
Data access interface looks something like below:
public Student GetStudentById(int id);
public IList<Student> GetStudents(int ageFilter, int classId);
The implementation which consumes this DAL looks like below:
public Student FindStudent(int id)
{
//StudentDAL dependency injected
return StudentDAL.GetStudentById(id);
//Use strict mock to test this
}
public IList<Student> GetStudentsForClass(StudentListRequest studentListRequest)
{
//StudentDAL dependency injected
//age filter is derived from the request and then passed on to the underlying layer
int ageFilter = DateTime.Now.Year - studentListRequest.DateOfBirthFilter.Year;
return StudentDAL.GetStudents(ageFilter , studentListRequest.ClassId)
//Use loose mock and use verify api of MOQ to make sure that the age filter is correctly passed on.
}
Me personally, being new to mocking and Moq feel that starting off with Strict mode helps better understand of the innards and what's going on. "Loose" sometimes hides details and pass a test which a moq beginner may fail to see. Once you have your mocking skills down - Loose would probably be a lot more productive - like in this case saving a line with the "Setup" and just using "Verify" instead.