With Moq, it is possible to verify that a method is never called with certain arguments (that is, arguments satisfying certain predicates) using Times.Never.
But how to verify that, no mater how many times a method is called, it is always called with certain arguments?
The default appears to be Times.AtLeastOnce.
There is no Times.Always. Am I missing something obvious? Thanks!
Edit: I posted a suggestion to the Moq mailing list last week, but it doesn't look like it's been moderated yet. I'll post any updates here.
Edit: an example. Say I am testing a class which generates XML documents. I want to ensure that only valid documents are generated. In other words, test that the writer dependency is only ever given valid documents, with a valid sequence number, to write.
should_only_write_valid_xml_documents
Mock.Get(this.writer).Verify(
w => w.Write(
It.Is<XDocument>(doc => XsdValidator.IsValid(doc)),
It.Is<int>(n => n < 3)),
Times.Always);
And how many times is 'always?' Moq keeps track of all the times a certain method is called with certain arguments, and then uses that number for comparing to Times.Never, Times.AtLeastOnce, etc.
So, if a method is executed 4 times and you set it to 'Times.Always' what does that even mean?
Times.Never would check to make sure the number is zero.
Times.AtLeastOnce would check that the number is greater than or equal to one.
Times.Always would check that the number is.... ?
You could determine the number of times it SHOULD run programmatically, then do something like:
Times.Exactly(calculatedAmount)
But there is no way for Moq to know what 'always' means.
It sounds like you're wanting "Strict" mock behavior. If the method is called with anything other than the expected parameters, the test will fail.
This is available in Moq:
var mock = new Mock<IFoo>(MockBehavior.Strict);
(Example taken from the Moq QuickStart.)
Every invocation on the mock must now have a corresponding Setup.
Using strict mocks tends to lead to brittle tests. I would avoid this technique, or at least use it sparingly.
You can apply inversion of logic to verify ALWAYS.
E.g.:
Let's say you want to do the following verification:
mock.Verify(x => x.Method(It.Is<int>(i => i == 10)), Times.Always());
you can simply replace it with:
mock.Verify(x => x.Method(It.Is<int>(i => i != 10)), Times.Never());
My recommendation would be to create a 'fallback' matching for the method with It.IsAny conditions. You can then verify Times.Never on that matching, which should always have been superseded by the more specific match.
Related
I'm currently a Moq user and I'm researching other mocking frameworks.
When unit testing I frequently call _mock.VerifyNoOtherCalls() so I can be certain there are no unexpected interactions beyond the ones that I have already verified.
I've searched the FakeItEasy docs and cannot find the equivalent option in their framework. Can anyone suggest how I might do this?
Strict fakes
FakeItEasy supports strict fakes (similar to strict mocks in Moq):
var foo = A.Fake<IFoo>(x => x.Strict());
This will fail the moment an unexpected call is made.
Semi-strict fakes
It is also possible to configure all calls directly:
A.CallTo(fakeShop).Throws(new Exception());
and combine this with specifying different behaviors for successive calls, however in this case, there's no benefit to doing so over using a strict fake, as a strict fake will give better messages when unconfigured methods are called. So if you want to configure some methods to be called a limited number of times, you could
var fakeShop = A.Fake<IShop>(options => options.Strict());
A.CallTo(() => fakeShop.GetTopSellingCandy()).Returns(lollipop).Once();
A.CallTo(() => fakeShop.Address).Returns("123 Fake Street").Once();
fakeShop.GetTopSellingCandy() and fakeShop.Address can be called once, the second time it will fail.
Arbitrary checks
If you want to check if no calls are made at arbitrary points in the test:
A.CallTo(fakeShop).MustNotHaveHappened();
It might be better to filter out some of the methods that can be executed while debugging:
A.CallTo(a)
.Where(call => call.Method.Name != "ToString")
.MustNotHaveHappened();
You don't want a failing test because you hovered over the variable.
I am trying to use learn how to use FakeItEasy, and wanted to try using it with some data access code from an old project I have access to. While the basics of FIE seemed pretty easy, and I was able to get simple cases working, this one has me stumped.
The system used Entity Framework, and one of the data management classes handles Users, and I'm trying to figure out how to test just the basic GetUserByUserNumber function. I can use a fake IPersistenceManager<User> when instantiating the UserDataManager class, and then call the GetUserByUserNumber method, but the assertion to check that userPersistenceManager.ReadCustom was called always fails.
I've tried calling the Delete method on the fake userPersistenceManager, and the assertion for that works fine. I think it has something to do with the Linq Expression that the ReadCustom method takes as its first parameter. I just don't know how that should be handled. Any help with this would be appreciated!
This is the method in the UserDataManager that I'm trying to test:
public User GetUserByUserNumber(string userNumber, bool loadRelatedRecords = false)
{
if (string.IsNullOrWhiteSpace(userNumber))
{
throw MyAppExceptions.CreateMyAppFatalException(Constants.ExceptionKeys.Unexpected, new ArgumentNullException("userNumber"));
}
Logger.Write(string.Format("Executing GetUserByUserNumber with UserNumber {0}.", userNumber), LogCategory.General, TraceEventType.Verbose);
return _UserPersistenceManager.ReadCustom(mem => mem.UserNumber == userNumber, EntityConstants.EntityNames.UserDetail);
}
This is the IPersistenceManager method that I want to ensure is called:
TEntity ReadCustom(Expression<Func<TEntity, bool>> predicate, string includeEntityName);
This is my unit test:
[TestMethod]
public void GetUserByUserNumber_Calls_ReadCustom()
{
// Arrange
var userPersistenceManager = A.Fake<IPersistenceManager<User>>();
var dataManager = new UserDataManager(userPersistenceManager);
// Act
dataManager.GetUserByUserNumber("123456", false);
// Assert
A.CallTo(() => userPersistenceManager.ReadCustom(u => u.UserNumber == "123456", EntityConstants.EntityNames.UserDetail)).MustHaveHappened();
}
I think Tim Long's answer is essentially correct although my slant is not that this is a failing of mocking frameworks—it comes down to how easy it is (in general, not just when mocking) to determine whether two things are "the same".
The problem you have is that unless told otherwise, FakeItEasy uses .Equals to compare the arguments. Expressions don't compare well with .Equals, and so you'll get a mismatch.
One option is to explore Expression equality-checkers. There are a number of questions on StackOverflow about this already, such as How to check if two Expression<Func<T, bool>> are the same. If you can find a good way to determine the equality of the expressions, I think you could provide that method to FakeItEasy's argument matcher
(e.g. with A<Expression<Func<TEntity, bool>>.That.Matches(…)).
Alternatively, you can go Mr. Long's route and capture the argument and then interrogate it later. I suggested a similar approach just a bit ago when answering
How to fake an action<> with FakeItEasy.
In your case, you could capture the predicate and then verify its correctness by seeing how it reacts to various input objects - does it like ones with UserNumber "123456".
This seems to be an area of mocking frameworks that is very counter-intuitive and hard to use correctly. I generally shy away from doing argument matching and try to return or capture some sort of object that I can later make assertions against.
In your case, you are essentially comparing two expressions for equality. The 'look' different in the code even though they have the same syntax. I wonder, can you create those two expressions outside of the test context and see if they compare equal then?
I have come across a very strange behavior in moq and I cannot understand whether it's a bug or I am doing something wrong. Here is the example:
List<CustomerDataTransaction> transactions0 = GetTransactionsSomehow();
List<CustomerDataTransaction> transactions1 = GetTransactionsSomehow();
var portfolioTransactions0 = new List<IPortfolioTransaction>();
var portfolioTransactions1 = new List<IPortfolioTransaction>();
m_TransactionMapperMock
.Setup(m => m.CreatePortfolioTransactions(transactions0))
.Returns(portfolioTransactions0);
m_TransactionMapperMock
.Setup(m => m.CreatePortfolioTransactions(transactions1))
.Returns(portfolioTransactions1);
I have checked that transaction0 is not equal to transactions1, so it's for sure different lists. But the mock returns portfolioTransactions1 twice when called with different parameters (transactions0 and transactions1). I tried to figure out what was wrong, but I did not find any reasons. Then I have added dummy elements to each of the lists and that has fixed it, mock started to return different values as planned. Is this a bug of Mock or I don't get something? As far as I understand the values inside the list should not affect it at all.
P.S. I don't know if it's important or not but the method accepts IEnumerable<CustomerDataTransaction>
Consider trying:
It.Is<IEnumerable<CustomerDataTransaction>>(t => t == transactions1))
From memory I thought Moq used reference equality
Expanding on NinjaNye's answer, using It.Is satisfies the test is because it ensures that you are looking at the exact instance of the object, not the equality operator.
With your testing framework, if you say Assert.NotEqual and give it two objects, it uses the equality operator and says one empty list equal to the other. Think about being able to compare objectA.Property1 to objectA.Property2. It makes sense that Assert.Equal would look at the values, not the instance of the value.
Xunit provides a method Assert.Same, which will provide the same functionality as Moq's It.Is<>, verifying it is the same instance of the object. I believe that most of the popular testing frameworks today provide similar behavior.
I currently have the following method:
public void SetNewRandomValue() {
double newValue = numberGenerator.GenerateDouble(
genesValuesInterval.MinimumValue,
genesValuesInterval.MaximumValue
);
this.value = newValue;
}
What should be the guidelines for deciding how many tests (and which tests) to make to this method? I currently have done the following one (only after implementing the method -- that is, not test-first):
var interval = new Interval(-10, 10);
var numberGeneratorMock = new Mock<INumberGenerator>(MockBehavior.Strict);
var numberGenerator = numberGeneratorMock.Object;
double expectedValue = 5.0;
numberGeneratorMock.Setup(ng =>
ng.GenerateDouble(interval.MinimumValue, interval.MaximumValue))
.Returns(expectedValue);
var gene = new Gene(numberGenerator, 0, new Interval(-10, 10));
gene.SetNewRandomValue();
Assert.AreEqual<double>(expectedValue, gene.Value);
that basically just tests one situation. Regression-testingwise I'd say that I can't think of a way of messing up the code, turning it into mal functioning code and still have the test pass, that is, I think the method looks decently covered.
What are your opinions on this? How would you handle this little method?
Thanks
I would examine the code coverage with whatever testing tool you use, if a code coverage is available for your testing framework.
I personally like to work with either Microsoft Testing Tool or NUnit Testing Framework. I can then right-click my tests project and Test with NCover (while using NUnit), which will run the tests and tell me the percentage of code covered for each of my objects and tests.
I say that when you'll be done checking the code coverage which would result of at least a 98% code coverage, your code is likely to be well tested.
I'd recommend taking a look at Pex - it can really help generate the kind of unit tests you're looking for (i.e. figure out the different potential paths and results given a method and return value).
That test looks fine. The only thing you can actually assert about SetNewRandomValue is that the Value member is assigned afterward. You've mocked out the call to GenerateDouble and verified that Value contains the expected number, so you should be good.
You could also write a test to document (and verify) the expected behavior of Gene.SetNewRandomValue when NumberGenerator.GenerateDouble returns a value outside the specified interval.
You could definitely make a case for not unit testing this. IMHO, code inspection is a perfectly valid test methodology. You generally don't test things like property setters/getters, I think this method is simple enough to avoid unit testing for the same reason.
That said, if you really do want to test it, here's what I'd do: I'd test it with a couple values, not just once with 5. (SetNewRandomValue could be implemented as this.value = 5;, which should not pass.) I'd test it with a non-integer number, to confirm there's not a oddball cast to integer in there.
You could test that it's calling GenerateDouble with the proper parameters, though that's really testing an implementation detail. (SetNewRandomValue could be implemented as numberGenerator.GenerateDouble(0, interval.max - interval.min) + interval.min;, and that shouldn't fail the test.) You could use a real random number generator, and do SetNewRandomValue a few thousand times, and test that the values are evenly distributed in your expected range.
The method is doing three things:
Calling numberGenerator.GenerateDouble with genesValuesInterval.MinimumValue as the first parameter,
and with genesValuesInterval.MaximumValue as the second parameter,
and setting this.value to the result of that call.
Your test tests the third of these things, but not the first two. You could write two more tests that check the mock is called with the correct first and second parameters.
Edit (responding to comments below):
If the intended behaviour of this method is to set this.value to a random double within a previously specified range, then the above three tests are useful (assuming genesValuesInterval min and max are the previously specified range and that you have tests in place to assert that numberGenerator.GenerateDouble(min, max) returns a double within the specified range.
If the intended behaviour of this method is just to set this.value to a random double within (Double.MinValue, Double.MaxValue), then the first two tests are unnecessary as this is just an implementation detail.
If the inted
To answer how to test it, you should be able to describe what is the desired behavior.
Looking at your code, I assume that "Gene.SetNewRandomValue" is supposed to set self.Value to a number which falls within the Interval passed to the constructor.
I'm not super familiar with the Mock class, so I may be off base, but it appears that you are not testing that. What if your implementation had this typo?
double newValue = numberGenerator.GenerateDouble(
genesValuesInterval.MinimumValue,
genesValuesInterval.MinimumValue
);
Wouldn't your test still pass?
We have just released a re-written(for the 3rd time) module for our proprietary system. This module, which we call the Load Manager, is by far the most complicated of all the modules in our system to date. We are trying to get a comprehensive test suite because every time we make any kind of significant change to this module there is hell to pay for weeks in sorting out bugs and quirks. However, developing a test suite has proven to be quite difficult so we are looking for ideas.
The Load Manager's guts reside in a class called LoadManagerHandler, this is essentially all of the logic behind the module. This handler calls upon multiple controllers to do the CRUD methods in the database. These controllers are essentially the top layer of the DAL that sits on top and abstracts away our LLBLGen generated code.
So it is easy enough to mock these controllers, which we are doing using the Moq framework. However the problem comes in the complexity of the Load Manager and the issues that we receive aren't in dealing with the simple cases but the cases where there is a substantial amount of data contained within the handler.
To briefly explain the load manager contains a number of "unloaded" details, sometimes in the hundreds, that are then dropped into user created loads and reship pools. During the process of creating and populating these loads there is a multitude of deletes, changes, and additions that eventually cause issues to appear. However, because when you mock a method of an object the last mock wins, ie:
jobDetailControllerMock.Setup(mock => mock.GetById(1)).Returns(jobDetail1);
jobDetailControllerMock.Setup(mock => mock.GetById(2)).Returns(jobDetail2);
jobDetailControllerMock.Setup(mock => mock.GetById(3)).Returns(jobDetail3);
No matter what I send to jobDetailController.GetById(x) I will always get back jobDetail3. This makes testing almost impossible because we have to make sure that when changes are made all points are affected that should be affected.
So, I resolved to using the test database and just allowing the reads and writes to occur as normal. However, because you can't(read: should not) dictate the order of your tests, tests that are run earlier could cause tests that run later to fail.
TL/DR: I am essentially looking for testing strategies for data oriented code that is quite complex in nature.
As noted by Seb, you can indeed use a range matching:
controller.Setup(x => x.GetById(It.IsInRange<int>(1, 3, Range.Inclusive))))).Returns<int>(i => jobs[i]);
This code uses the argument passed to the method to calculate which value to return.
To get around the "last mock wins" with Moq, you could use the technique from this blog:
Moq Triqs - Successive Expectations
EDIT:
Actually you don't even need that. Based on your example, Moq will return different values based on the method argument.
public interface IController
{
string GetById(int id);
}
class Program
{
static void Main(string[] args)
{
var mockController = new Mock<IController>();
mockController.Setup(x => x.GetById(1)).Returns("one");
mockController.Setup(x => x.GetById(2)).Returns("two");
mockController.Setup(x => x.GetById(3)).Returns("three");
IController controller = mockController.Object;
Console.WriteLine(controller.GetById(1));
Console.WriteLine(controller.GetById(3));
Console.WriteLine(controller.GetById(2));
Console.WriteLine(controller.GetById(3));
Console.WriteLine(controller.GetById(99) == null);
}
}
Output is:
one
three
two
three
True
It sounds like LoaderManagerHandler does... quite a bit of work. "Manager" in a class name always somewhat worries me... from a TDD standpoint, it might be worth thinking about breaking the class up appropriately if possible.
How long is this class?
I've never used Moq, but it seems that it should be able to match a mock invocation by argument(s) supplied.
A quick look at the Quick Start documentation has the following excerpt:
//Matching Arguments
// any value
mock.Setup(foo => foo.Execute(It.IsAny<string>())).Returns(true);
// matching Func<int>, lazy evaluated
mock.Setup(foo => foo.Add(It.Is<int>(i => i % 2 == 0))).Returns(true);
// matching ranges
mock.Setup(foo => foo.Add(It.IsInRange<int>(0, 10, Range.Inclusive))).Returns(true);
I think you should be able to use the second example above.
A simple testing technique is to make sure everytime a bug is logged against a system, make sure a unit test is written covering that case. You can build up a pretty solid set of tests just from that technique. And even better you won't run into the same thing twice.
No matter what I send to jobDetailController.GetById(x) I will always get back jobDetail3
You should spend more time debugging your tests because what is happening is not how Moq behaves. There is a bug in your code or tests causing something to misbehave.
If you want to make repeated calls with the same inputs but different outputs you could also use a different mocking framework. RhinoMocks supports the record/playback idiom. You're right this is not always what you want with regards to enforcing call order. I do prefer Moq myself for its simplicity.