I want to test the order of execution of the methods of three dependencies within a class. That is, check to ensure a method is called on class a, then b and then c and are executed and in that order.
I was going to use the Moq.Sequence library but I have found that sequence testing is now supported out of the box. The only thing is that documentation is very light on the ground about the MoqSequence class, and I cant find a concrete example.
Also, my question is not about whether this is right or wrong specifically, but I believe testing the order of execution is a valid test (bit of a grey area here?)
There is bug when using MockSequence on same mock. It definitely will be fixed in later releases of Moq library (you can also fix it manually by changing Moq.MethodCall.Matches implementation).
If you want to use Moq only, then you can verify method call order via callbacks:
int callOrder = 0;
writerMock.Setup(x => x.Write(expectedType)).Callback(() => Assert.That(callOrder++, Is.EqualTo(0)));
writerMock.Setup(x => x.Write(expectedId)).Callback(() => Assert.That(callOrder++, Is.EqualTo(1)));
writerMock.Setup(x => x.Write(expectedSender)).Callback(() => Assert.That(callOrder++, Is.EqualTo(2)));
Related
I'm currently a Moq user and I'm researching other mocking frameworks.
When unit testing I frequently call _mock.VerifyNoOtherCalls() so I can be certain there are no unexpected interactions beyond the ones that I have already verified.
I've searched the FakeItEasy docs and cannot find the equivalent option in their framework. Can anyone suggest how I might do this?
Strict fakes
FakeItEasy supports strict fakes (similar to strict mocks in Moq):
var foo = A.Fake<IFoo>(x => x.Strict());
This will fail the moment an unexpected call is made.
Semi-strict fakes
It is also possible to configure all calls directly:
A.CallTo(fakeShop).Throws(new Exception());
and combine this with specifying different behaviors for successive calls, however in this case, there's no benefit to doing so over using a strict fake, as a strict fake will give better messages when unconfigured methods are called. So if you want to configure some methods to be called a limited number of times, you could
var fakeShop = A.Fake<IShop>(options => options.Strict());
A.CallTo(() => fakeShop.GetTopSellingCandy()).Returns(lollipop).Once();
A.CallTo(() => fakeShop.Address).Returns("123 Fake Street").Once();
fakeShop.GetTopSellingCandy() and fakeShop.Address can be called once, the second time it will fail.
Arbitrary checks
If you want to check if no calls are made at arbitrary points in the test:
A.CallTo(fakeShop).MustNotHaveHappened();
It might be better to filter out some of the methods that can be executed while debugging:
A.CallTo(a)
.Where(call => call.Method.Name != "ToString")
.MustNotHaveHappened();
You don't want a failing test because you hovered over the variable.
I have come across a very strange behavior in moq and I cannot understand whether it's a bug or I am doing something wrong. Here is the example:
List<CustomerDataTransaction> transactions0 = GetTransactionsSomehow();
List<CustomerDataTransaction> transactions1 = GetTransactionsSomehow();
var portfolioTransactions0 = new List<IPortfolioTransaction>();
var portfolioTransactions1 = new List<IPortfolioTransaction>();
m_TransactionMapperMock
.Setup(m => m.CreatePortfolioTransactions(transactions0))
.Returns(portfolioTransactions0);
m_TransactionMapperMock
.Setup(m => m.CreatePortfolioTransactions(transactions1))
.Returns(portfolioTransactions1);
I have checked that transaction0 is not equal to transactions1, so it's for sure different lists. But the mock returns portfolioTransactions1 twice when called with different parameters (transactions0 and transactions1). I tried to figure out what was wrong, but I did not find any reasons. Then I have added dummy elements to each of the lists and that has fixed it, mock started to return different values as planned. Is this a bug of Mock or I don't get something? As far as I understand the values inside the list should not affect it at all.
P.S. I don't know if it's important or not but the method accepts IEnumerable<CustomerDataTransaction>
Consider trying:
It.Is<IEnumerable<CustomerDataTransaction>>(t => t == transactions1))
From memory I thought Moq used reference equality
Expanding on NinjaNye's answer, using It.Is satisfies the test is because it ensures that you are looking at the exact instance of the object, not the equality operator.
With your testing framework, if you say Assert.NotEqual and give it two objects, it uses the equality operator and says one empty list equal to the other. Think about being able to compare objectA.Property1 to objectA.Property2. It makes sense that Assert.Equal would look at the values, not the instance of the value.
Xunit provides a method Assert.Same, which will provide the same functionality as Moq's It.Is<>, verifying it is the same instance of the object. I believe that most of the popular testing frameworks today provide similar behavior.
With Moq, it is possible to verify that a method is never called with certain arguments (that is, arguments satisfying certain predicates) using Times.Never.
But how to verify that, no mater how many times a method is called, it is always called with certain arguments?
The default appears to be Times.AtLeastOnce.
There is no Times.Always. Am I missing something obvious? Thanks!
Edit: I posted a suggestion to the Moq mailing list last week, but it doesn't look like it's been moderated yet. I'll post any updates here.
Edit: an example. Say I am testing a class which generates XML documents. I want to ensure that only valid documents are generated. In other words, test that the writer dependency is only ever given valid documents, with a valid sequence number, to write.
should_only_write_valid_xml_documents
Mock.Get(this.writer).Verify(
w => w.Write(
It.Is<XDocument>(doc => XsdValidator.IsValid(doc)),
It.Is<int>(n => n < 3)),
Times.Always);
And how many times is 'always?' Moq keeps track of all the times a certain method is called with certain arguments, and then uses that number for comparing to Times.Never, Times.AtLeastOnce, etc.
So, if a method is executed 4 times and you set it to 'Times.Always' what does that even mean?
Times.Never would check to make sure the number is zero.
Times.AtLeastOnce would check that the number is greater than or equal to one.
Times.Always would check that the number is.... ?
You could determine the number of times it SHOULD run programmatically, then do something like:
Times.Exactly(calculatedAmount)
But there is no way for Moq to know what 'always' means.
It sounds like you're wanting "Strict" mock behavior. If the method is called with anything other than the expected parameters, the test will fail.
This is available in Moq:
var mock = new Mock<IFoo>(MockBehavior.Strict);
(Example taken from the Moq QuickStart.)
Every invocation on the mock must now have a corresponding Setup.
Using strict mocks tends to lead to brittle tests. I would avoid this technique, or at least use it sparingly.
You can apply inversion of logic to verify ALWAYS.
E.g.:
Let's say you want to do the following verification:
mock.Verify(x => x.Method(It.Is<int>(i => i == 10)), Times.Always());
you can simply replace it with:
mock.Verify(x => x.Method(It.Is<int>(i => i != 10)), Times.Never());
My recommendation would be to create a 'fallback' matching for the method with It.IsAny conditions. You can then verify Times.Never on that matching, which should always have been superseded by the more specific match.
We have just released a re-written(for the 3rd time) module for our proprietary system. This module, which we call the Load Manager, is by far the most complicated of all the modules in our system to date. We are trying to get a comprehensive test suite because every time we make any kind of significant change to this module there is hell to pay for weeks in sorting out bugs and quirks. However, developing a test suite has proven to be quite difficult so we are looking for ideas.
The Load Manager's guts reside in a class called LoadManagerHandler, this is essentially all of the logic behind the module. This handler calls upon multiple controllers to do the CRUD methods in the database. These controllers are essentially the top layer of the DAL that sits on top and abstracts away our LLBLGen generated code.
So it is easy enough to mock these controllers, which we are doing using the Moq framework. However the problem comes in the complexity of the Load Manager and the issues that we receive aren't in dealing with the simple cases but the cases where there is a substantial amount of data contained within the handler.
To briefly explain the load manager contains a number of "unloaded" details, sometimes in the hundreds, that are then dropped into user created loads and reship pools. During the process of creating and populating these loads there is a multitude of deletes, changes, and additions that eventually cause issues to appear. However, because when you mock a method of an object the last mock wins, ie:
jobDetailControllerMock.Setup(mock => mock.GetById(1)).Returns(jobDetail1);
jobDetailControllerMock.Setup(mock => mock.GetById(2)).Returns(jobDetail2);
jobDetailControllerMock.Setup(mock => mock.GetById(3)).Returns(jobDetail3);
No matter what I send to jobDetailController.GetById(x) I will always get back jobDetail3. This makes testing almost impossible because we have to make sure that when changes are made all points are affected that should be affected.
So, I resolved to using the test database and just allowing the reads and writes to occur as normal. However, because you can't(read: should not) dictate the order of your tests, tests that are run earlier could cause tests that run later to fail.
TL/DR: I am essentially looking for testing strategies for data oriented code that is quite complex in nature.
As noted by Seb, you can indeed use a range matching:
controller.Setup(x => x.GetById(It.IsInRange<int>(1, 3, Range.Inclusive))))).Returns<int>(i => jobs[i]);
This code uses the argument passed to the method to calculate which value to return.
To get around the "last mock wins" with Moq, you could use the technique from this blog:
Moq Triqs - Successive Expectations
EDIT:
Actually you don't even need that. Based on your example, Moq will return different values based on the method argument.
public interface IController
{
string GetById(int id);
}
class Program
{
static void Main(string[] args)
{
var mockController = new Mock<IController>();
mockController.Setup(x => x.GetById(1)).Returns("one");
mockController.Setup(x => x.GetById(2)).Returns("two");
mockController.Setup(x => x.GetById(3)).Returns("three");
IController controller = mockController.Object;
Console.WriteLine(controller.GetById(1));
Console.WriteLine(controller.GetById(3));
Console.WriteLine(controller.GetById(2));
Console.WriteLine(controller.GetById(3));
Console.WriteLine(controller.GetById(99) == null);
}
}
Output is:
one
three
two
three
True
It sounds like LoaderManagerHandler does... quite a bit of work. "Manager" in a class name always somewhat worries me... from a TDD standpoint, it might be worth thinking about breaking the class up appropriately if possible.
How long is this class?
I've never used Moq, but it seems that it should be able to match a mock invocation by argument(s) supplied.
A quick look at the Quick Start documentation has the following excerpt:
//Matching Arguments
// any value
mock.Setup(foo => foo.Execute(It.IsAny<string>())).Returns(true);
// matching Func<int>, lazy evaluated
mock.Setup(foo => foo.Add(It.Is<int>(i => i % 2 == 0))).Returns(true);
// matching ranges
mock.Setup(foo => foo.Add(It.IsInRange<int>(0, 10, Range.Inclusive))).Returns(true);
I think you should be able to use the second example above.
A simple testing technique is to make sure everytime a bug is logged against a system, make sure a unit test is written covering that case. You can build up a pretty solid set of tests just from that technique. And even better you won't run into the same thing twice.
No matter what I send to jobDetailController.GetById(x) I will always get back jobDetail3
You should spend more time debugging your tests because what is happening is not how Moq behaves. There is a bug in your code or tests causing something to misbehave.
If you want to make repeated calls with the same inputs but different outputs you could also use a different mocking framework. RhinoMocks supports the record/playback idiom. You're right this is not always what you want with regards to enforcing call order. I do prefer Moq myself for its simplicity.
So I have a class with a method as follows:
public class SomeClass
{
...
private SomeDependency m_dependency;
public int DoStuff()
{
int result = 0;
...
int someValue = m_dependency.GrabValue();
...
return result;
}
}
And I've decided that rather than to call m_dependency.GrabValue() each time, I really want to cache the value in memory (i.e. in this class) since we're going to get the same value each time anyway (the dependency goes off and grabs some data from a table that hardly ever changes).
I've run into problems however trying to describe this new behaviour in a unit test. I've tried the following (I'm using NUnit with RhinoMocks):
[Test]
public void CacheThatValue()
{
var depend = MockRepository.GeneraMock<SomeDependency>();
depend.Expect(d => d.GrabValue()).Repeat.Once().Return(1);
var sut = new SomeCLass(depend);
int result = sut.DoStuff();
result = sut.DoStuff();
depend.VerifyAllExpectations();
}
This however doesn't work; this test passes even without introducing any changes to the functionality. What am I doing wrong?
I see caching as orthogonal to Do(ing)Stuff. I would find a way to pull the caching logic outside of the method, either by changing SomeDependency or wrapping it somehow (I now have a cool idea for a caching class based around lambda expressions -- yum).
That way your tests for DoStuff don't need to change, you only need to make sure they work with the new wrapper. Then you can test the caching functionality of SomeDependency, or its wrapper, independently. With well-architected code putting a caching layer in place should be rather easy and neither your dependency nor your implementation should know the difference.
Unit tests shouldn't be testing implementation, they should test behavior. At the same time, the subject under test should have a narrowly-defined set of behavior.
To answer your question, you are using a Dynamic Mock and the default behavior is to allow any call that isn't configured. The additional calls are just returning "0". You need to set up an expectation that no more calls are made on the dependency:
depend.Expect(d => d.GrabValue()).Repeat.Once().Return(1);
depend.Expect(d => d.GrabValue()).Repeat.Never();
You may need to enter record/replay mode to get it to work properly.
This seems like a case for "tests drive the design". If caching is an implementation detail of SubDependency - and therefore can't be directly tested - then probably some of its functionality (specifically, its caching behavior) needs to be exposed - and since it's not natural to expose it within SubDependency, needs to be exposed in another class (let's call it "Cache"). In Cache, of course, the behavior is contractual - public, and thereby testable.
So the tests - and the smells - are telling us we need a new class. Test-Driven Design. Ain't it great?