I'm currently a Moq user and I'm researching other mocking frameworks.
When unit testing I frequently call _mock.VerifyNoOtherCalls() so I can be certain there are no unexpected interactions beyond the ones that I have already verified.
I've searched the FakeItEasy docs and cannot find the equivalent option in their framework. Can anyone suggest how I might do this?
Strict fakes
FakeItEasy supports strict fakes (similar to strict mocks in Moq):
var foo = A.Fake<IFoo>(x => x.Strict());
This will fail the moment an unexpected call is made.
Semi-strict fakes
It is also possible to configure all calls directly:
A.CallTo(fakeShop).Throws(new Exception());
and combine this with specifying different behaviors for successive calls, however in this case, there's no benefit to doing so over using a strict fake, as a strict fake will give better messages when unconfigured methods are called. So if you want to configure some methods to be called a limited number of times, you could
var fakeShop = A.Fake<IShop>(options => options.Strict());
A.CallTo(() => fakeShop.GetTopSellingCandy()).Returns(lollipop).Once();
A.CallTo(() => fakeShop.Address).Returns("123 Fake Street").Once();
fakeShop.GetTopSellingCandy() and fakeShop.Address can be called once, the second time it will fail.
Arbitrary checks
If you want to check if no calls are made at arbitrary points in the test:
A.CallTo(fakeShop).MustNotHaveHappened();
It might be better to filter out some of the methods that can be executed while debugging:
A.CallTo(a)
.Where(call => call.Method.Name != "ToString")
.MustNotHaveHappened();
You don't want a failing test because you hovered over the variable.
Related
So I am playing around with mocking frameworks (Moq) for my unit tests, and was wondering when you should use a mocking framework?
What is the benefit/disadvantage between the following two tests:
public class Tests
{
[Fact]
public void TestWithMock()
{
// Arrange
var repo = new Mock<IRepository>();
var p = new Mock<Person>();
p.Setup(x => x.Id).Returns(1);
p.Setup(x => x.Name).Returns("Joe Blow");
p.Setup(x => x.AkaNames).Returns(new List<string> { "Joey", "Mugs" });
p.Setup(x => x.AkaNames.Remove(It.IsAny<string>()));
// Act
var service = new Service(repo.Object);
service.RemoveAkaName(p.Object, "Mugs");
// Assert
p.Verify(x => x.AkaNames.Remove("Mugs"), Times.Once());
}
[Fact]
public void TestWithoutMock()
{
// Arrange
var repo = new Mock<IRepository>();
var p = new Person { Id = 1, Name = "Joe Blow", AkaNames = new List<string> { "Joey", "Mugs" } };
// Act
var service = new Service(repo.Object);
service.RemoveAkaName(p, "Mugs");
// Assert
Assert.True(p.AkaNames.Count == 1);
Assert.True(p.AkaNames[0] == "Joey");
}
}
Use mock objects to truly create a unit test--a test where all dependencies are assumed to function correctly and all you want to know is if the SUT (system under test--a fancy way of saying the class you're testing) works.
The mock objects help to "guarantee" your dependencies function correctly because you create mock versions of those dependencies that produce results you configure. The question then becomes if the one class you're testing behaves as it should when everything else is "working."
Mock objects are particularly critical when you are testing an object with a slow dependency--like a database or a web service. If you were to really hit the database or make the real web service call, your test will take a lot more time to run. That's tolerable when it's only a few extra seconds, but when you have hundreds of tests running in a continuous integration server, that adds up really fast and cripples your automation.
This is what really makes mock objects important--reducing the build-test-deploy cycle time. Making sure your tests run fast is critical to efficient software development.
There are some rules I use writing unit-tests.
If my System Under Test (SUT) or Object Under Test has dependencies
then I mock them all.
If I test a method that returns result then I
check result only. If dependencies are passed as parameters of the method they should be mocked. (see 1)
If I test 'void' method then verifying mocks is the best option for testing.
There is an old one article by Martin Fowler Mocks Aren't Stubs.
In first test you use Mock and in the second one you use Stub.
Also I see some design issues that lead to your question.
If it is allowed to remove AkaName from AkaNames collection then it is OK to use stub and check state of the person. If you add specific method void RemoveAkaName(string name) into Person class then mocks should be used in order to verify its invocation. And logic of RemoveAkaName should be tested as part of Person class testing.
I would use stub for product and mock for repository for you code.
The mocking framework is used to remove the dependency, so the unit test will focus on the "Unit" to be tested. In your case, the person looks like a simple entity class, there is no need to use Mocking for it.
Mocking has many benefits especially in agile programming where many quick release cycles means that sytem strucutre and real data might be incomplete. In such cases you can mock a repository to mimic production code in order to continue work on ui, or services. This is usually complemented with an IoC mechanism like Ninject to simplify the switch to the real repositories. The two examples you give are equal and without any other context I would say it's a matter of choice between them. The fluent api in moq might be easier to read as its kind of self documenting. That's my opinion though ;)
Mock is used to test object that cannot function in isolation. suppose function A is dependent on function B, to perform unit testing on function A we even end up testing function B.By using mock you can simulate the functionality of function B and testing can be focussed only on function A.
A mocking framework is useful for simulating the integration points in the code under test. I would argue that your concrete example is not a good candidate for a mocking framework as you can already inject the dependency (Person) directly into the code. Using a mocking framework actually complicates it in this case.
A much better use case is if you have a repository making a call to a database. It is desirable from a unit test perspective to mock the db call and return predetermined data instead. The key advantages of this is removing dependencies on exiting data, but also performance since the db call will slow the test down.
When should you use mocks? Almost never.
Mocks turn your tests into white-box tests, which are very labor intensive to write and extremely labor intensive to maintain. This may not be an issue if you are in the medical industry, aerospace industry, or other high-criticality software business, but if you are writing regular commercial grade software, your tests should be black-box tests. In other words, you should be testing against the interface, not against the implementation.
What to use instead of mocks?
Use fakes. Martin Fowler has an article explaining the difference here: https://martinfowler.com/bliki/TestDouble.html but to give you an example, an in-memory database can be used as fake in place of a full-blown RDBMS. (Note how fakes are a lot less fake than mocks.)
With Moq, it is possible to verify that a method is never called with certain arguments (that is, arguments satisfying certain predicates) using Times.Never.
But how to verify that, no mater how many times a method is called, it is always called with certain arguments?
The default appears to be Times.AtLeastOnce.
There is no Times.Always. Am I missing something obvious? Thanks!
Edit: I posted a suggestion to the Moq mailing list last week, but it doesn't look like it's been moderated yet. I'll post any updates here.
Edit: an example. Say I am testing a class which generates XML documents. I want to ensure that only valid documents are generated. In other words, test that the writer dependency is only ever given valid documents, with a valid sequence number, to write.
should_only_write_valid_xml_documents
Mock.Get(this.writer).Verify(
w => w.Write(
It.Is<XDocument>(doc => XsdValidator.IsValid(doc)),
It.Is<int>(n => n < 3)),
Times.Always);
And how many times is 'always?' Moq keeps track of all the times a certain method is called with certain arguments, and then uses that number for comparing to Times.Never, Times.AtLeastOnce, etc.
So, if a method is executed 4 times and you set it to 'Times.Always' what does that even mean?
Times.Never would check to make sure the number is zero.
Times.AtLeastOnce would check that the number is greater than or equal to one.
Times.Always would check that the number is.... ?
You could determine the number of times it SHOULD run programmatically, then do something like:
Times.Exactly(calculatedAmount)
But there is no way for Moq to know what 'always' means.
It sounds like you're wanting "Strict" mock behavior. If the method is called with anything other than the expected parameters, the test will fail.
This is available in Moq:
var mock = new Mock<IFoo>(MockBehavior.Strict);
(Example taken from the Moq QuickStart.)
Every invocation on the mock must now have a corresponding Setup.
Using strict mocks tends to lead to brittle tests. I would avoid this technique, or at least use it sparingly.
You can apply inversion of logic to verify ALWAYS.
E.g.:
Let's say you want to do the following verification:
mock.Verify(x => x.Method(It.Is<int>(i => i == 10)), Times.Always());
you can simply replace it with:
mock.Verify(x => x.Method(It.Is<int>(i => i != 10)), Times.Never());
My recommendation would be to create a 'fallback' matching for the method with It.IsAny conditions. You can then verify Times.Never on that matching, which should always have been superseded by the more specific match.
In the past, I have only used Rhino Mocks, with the typical strict mock. I am now working with Moq on a project and I am wondering about the proper usage.
Let's assume that I have an object Foo with method Bar which calls a Bizz method on object Buzz.
In my test, I want to verify that Bizz is called, therefore I feel there are two possible options:
With a strict mock
var mockBuzz= new Mock<IBuzz>(MockBehavior.Strict);
mockBuzz.Setup(x => x.Bizz()); //test will fail if Bizz method not called
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.VerifyAll();
With a loose mock
var mockBuzz= new Mock<IBuzz>();
foo.Buzz = mockBuzz
foo.Bar();
mockBuzz.Verify(x => x.Bizz()) //test will fail if Bizz method not called
Is there a standard or normal way of doing this?
I used to use strict mocks when I first starting using mocks in unit tests. This didn't last very long. There are really 2 reasons why I stopped doing this:
The tests become brittle - With strict mocks you are asserting more than one thing, that the setup methods are called, AND that the other methods are not called. When you refactor the code the test often fails, even if what you are trying to test is still true.
The tests are harder to read - You need to have a setup for every method that is called on the mock, even if it's not really related to what you want to test. When someone reads this test it's difficult for them to tell what is important for the test and what is just a side effect of the implementation.
Because of these I would strongly recommend using loose mocks in your unit tests.
I have background in C++/non-.NET development and I've been more into .NET recently so I had certain expectations when I was using Moq for the first time. I was trying to understand WTF was going on with my test and why the code I was testing was throwing a random exception instead of the Mock library telling me which function the code was trying to call. So I discovered I needed to turn on the Strict behaviour, which was perplexing- and then I came across this question which I saw had no ticked answer yet.
The Loose mode, and the fact that it is the default is insane. What on earth is the point of a Mock library that does something completely unpredictable that you haven't explicitly listed it should do?
I completely disagree with the points listed in the other answers in support of Loose mode. There is no good reason to use it and I wouldn't ever want to, ever. When writing a unit test I want to be certain what is going on - if I know a function needs to return a null, I'll make it return that. I want my tests to be brittle (in the ways that matter) so that I can fix them and add to the suite of test code the setup lines which are the explicit information that is describing to me exactly what my software will do.
The question is - is there a standard and normal way of doing this?
Yes - from the point of view of programming in general, i.e. other languages and outside the .NET world, you should use Strict always. Goodness knows why it isn't the default in Moq.
I have a simple convention:
Use strict mocks when the system under test (SUT) is delegating the call to the underlying mocked layer without really modifying or applying any business logic to the arguments passed to itself.
Use loose mocks when the SUT applies business logic to the arguments passed to itself and passes on some derived/modified values to the mocked layer.
For eg:
Lets say we have database provider StudentDAL which has two methods:
Data access interface looks something like below:
public Student GetStudentById(int id);
public IList<Student> GetStudents(int ageFilter, int classId);
The implementation which consumes this DAL looks like below:
public Student FindStudent(int id)
{
//StudentDAL dependency injected
return StudentDAL.GetStudentById(id);
//Use strict mock to test this
}
public IList<Student> GetStudentsForClass(StudentListRequest studentListRequest)
{
//StudentDAL dependency injected
//age filter is derived from the request and then passed on to the underlying layer
int ageFilter = DateTime.Now.Year - studentListRequest.DateOfBirthFilter.Year;
return StudentDAL.GetStudents(ageFilter , studentListRequest.ClassId)
//Use loose mock and use verify api of MOQ to make sure that the age filter is correctly passed on.
}
Me personally, being new to mocking and Moq feel that starting off with Strict mode helps better understand of the innards and what's going on. "Loose" sometimes hides details and pass a test which a moq beginner may fail to see. Once you have your mocking skills down - Loose would probably be a lot more productive - like in this case saving a line with the "Setup" and just using "Verify" instead.
I am doing ordered expectations in rhino mocks as described by ayende in this post. My code looks something like this:
using (_mocks.Ordered())
{
Expect.Call(() => _myMock.CallA());
Expect.Call(() => _myMock.CallB(40));
Expect.Call(() => _myMock.CallA());
}
_mocks.ReplayAll();
_myObjectUnderTest.DoStuff();
_mocks.VerifyAll();
After this I would like to add more expectations and test more of my object's methods. I want to do this because I have some basic test methods that do some setup on my test object, and I do not want to have to retest that functionality, so I just call the earlier test method. If I try and add more expectations after VerifyAll() I get an exception: "This action is invalid when the mock object is in verified state."
Part of my problem is that I don't really understand what all of the replay/verify stuff is doing, I just copied code. I am using strict mocks, so any setup code must have matching expectations or it fails. I do not want to duplicate the expectations for my setup code.
Is there a way to reset the mock to be ready to start over in some way?
Short answer: no. Now on to the long answer.
What the replay and verify stuff does is this. Before you call ReplayAll, you are telling Rhino what you expect the methods are going to do. You in a sense are recording calls to the mock objects you've created.
After you've recorded the method calls, you call ReplayAll telling Rhino you're now going to do 'something' that should execute the methods as recorded.
Last, you call VerifyAll asking Rhino to verify whether the methods were actually called as you've recorded them.
Now on to why you really should not want to re-use the mocks. With unit testing/TDD, your unit test should test as little as possible. This is to keep your unit tests simple and transparent. The whole idea of unit testing is that many many many small tests as a whole test the entire system.
For a very good presentation on TDD explaining this and other subjects, see http://www.infoq.com/presentations/integration-tests-scam.
P.S.: One small detail: As long as the methods don't return a result, you can write:
using (_mocks.Ordered())
{
Expect.Call(() => _myMock.CallA());
Expect.Call(() => _myMock.CallB(40));
Expect.Call(() => _myMock.CallA());
}
as:
using (_mocks.Ordered())
{
_myMock.CallA();
_myMock.CallB(40);
_myMock.CallA());
}
(at least with the latest version of Rhino)
Short answer: yes
Longer answer: Try the following
[Test]
public void RhinoMock_PerformMultipleChecks()
{
var myMock = MockRepository.GenerateDynamicMockWithRemoting<IComparable>();
// first round of checks
myMock.Expect(x => x.CompareTo("123")).Return(1);
myMock.Expect(x => x.CompareTo("-12")).Return(-1);
Assert.AreEqual(1, myMock.CompareTo("123"));
Assert.AreEqual(-1, myMock.CompareTo("-12"));
myMock.VerifyAllExpectations();
// reset
myMock.BackToRecord();
myMock.Replay();
// next round of checks
myMock.Expect(x => x.CompareTo(1.23)).Return(1);
myMock.Expect(x => x.CompareTo(-12)).Return(-1);
Assert.AreEqual(1, myMock.CompareTo(1.23));
Assert.AreEqual(-1, myMock.CompareTo(-12));
myMock.VerifyAllExpectations();
// reset
myMock.BackToRecord();
myMock.Replay();
// final round of checks
myMock.Expect(x => x.CompareTo(1.23m)).Return(1);
myMock.Expect(x => x.CompareTo(-12m)).Return(-111);
Assert.AreEqual(1, myMock.CompareTo(1.23m));
Assert.AreEqual(-111, myMock.CompareTo(-12m));
myMock.VerifyAllExpectations();
}
We have just released a re-written(for the 3rd time) module for our proprietary system. This module, which we call the Load Manager, is by far the most complicated of all the modules in our system to date. We are trying to get a comprehensive test suite because every time we make any kind of significant change to this module there is hell to pay for weeks in sorting out bugs and quirks. However, developing a test suite has proven to be quite difficult so we are looking for ideas.
The Load Manager's guts reside in a class called LoadManagerHandler, this is essentially all of the logic behind the module. This handler calls upon multiple controllers to do the CRUD methods in the database. These controllers are essentially the top layer of the DAL that sits on top and abstracts away our LLBLGen generated code.
So it is easy enough to mock these controllers, which we are doing using the Moq framework. However the problem comes in the complexity of the Load Manager and the issues that we receive aren't in dealing with the simple cases but the cases where there is a substantial amount of data contained within the handler.
To briefly explain the load manager contains a number of "unloaded" details, sometimes in the hundreds, that are then dropped into user created loads and reship pools. During the process of creating and populating these loads there is a multitude of deletes, changes, and additions that eventually cause issues to appear. However, because when you mock a method of an object the last mock wins, ie:
jobDetailControllerMock.Setup(mock => mock.GetById(1)).Returns(jobDetail1);
jobDetailControllerMock.Setup(mock => mock.GetById(2)).Returns(jobDetail2);
jobDetailControllerMock.Setup(mock => mock.GetById(3)).Returns(jobDetail3);
No matter what I send to jobDetailController.GetById(x) I will always get back jobDetail3. This makes testing almost impossible because we have to make sure that when changes are made all points are affected that should be affected.
So, I resolved to using the test database and just allowing the reads and writes to occur as normal. However, because you can't(read: should not) dictate the order of your tests, tests that are run earlier could cause tests that run later to fail.
TL/DR: I am essentially looking for testing strategies for data oriented code that is quite complex in nature.
As noted by Seb, you can indeed use a range matching:
controller.Setup(x => x.GetById(It.IsInRange<int>(1, 3, Range.Inclusive))))).Returns<int>(i => jobs[i]);
This code uses the argument passed to the method to calculate which value to return.
To get around the "last mock wins" with Moq, you could use the technique from this blog:
Moq Triqs - Successive Expectations
EDIT:
Actually you don't even need that. Based on your example, Moq will return different values based on the method argument.
public interface IController
{
string GetById(int id);
}
class Program
{
static void Main(string[] args)
{
var mockController = new Mock<IController>();
mockController.Setup(x => x.GetById(1)).Returns("one");
mockController.Setup(x => x.GetById(2)).Returns("two");
mockController.Setup(x => x.GetById(3)).Returns("three");
IController controller = mockController.Object;
Console.WriteLine(controller.GetById(1));
Console.WriteLine(controller.GetById(3));
Console.WriteLine(controller.GetById(2));
Console.WriteLine(controller.GetById(3));
Console.WriteLine(controller.GetById(99) == null);
}
}
Output is:
one
three
two
three
True
It sounds like LoaderManagerHandler does... quite a bit of work. "Manager" in a class name always somewhat worries me... from a TDD standpoint, it might be worth thinking about breaking the class up appropriately if possible.
How long is this class?
I've never used Moq, but it seems that it should be able to match a mock invocation by argument(s) supplied.
A quick look at the Quick Start documentation has the following excerpt:
//Matching Arguments
// any value
mock.Setup(foo => foo.Execute(It.IsAny<string>())).Returns(true);
// matching Func<int>, lazy evaluated
mock.Setup(foo => foo.Add(It.Is<int>(i => i % 2 == 0))).Returns(true);
// matching ranges
mock.Setup(foo => foo.Add(It.IsInRange<int>(0, 10, Range.Inclusive))).Returns(true);
I think you should be able to use the second example above.
A simple testing technique is to make sure everytime a bug is logged against a system, make sure a unit test is written covering that case. You can build up a pretty solid set of tests just from that technique. And even better you won't run into the same thing twice.
No matter what I send to jobDetailController.GetById(x) I will always get back jobDetail3
You should spend more time debugging your tests because what is happening is not how Moq behaves. There is a bug in your code or tests causing something to misbehave.
If you want to make repeated calls with the same inputs but different outputs you could also use a different mocking framework. RhinoMocks supports the record/playback idiom. You're right this is not always what you want with regards to enforcing call order. I do prefer Moq myself for its simplicity.