Develop Reference

NSubstitute ordered testing (Recived.InOrder) with return values for called methods results in CouldNotSetReturnDueToMissingInfoAboutLastCallException

I used to write my tests with RhinoMocks and have switched to NSubstitute.
Now I have a problem concerning ordered test.
Lets say I have three small classes like
public interface IProvider
{
int GetData();
}
public class Provider : IProvider
{
public int GetData()
{
return 3;
}
}
public interface ICalculator
{
int Calculate(int data);
}
public class Calculator : ICalculator
{
public int Calculate(int data)
{
if (data < 3)
{
return data;
}
return data * 2;
}
}
public class Operator
{
public void Operate(IProvider provider, ICalculator calculator)
{
int version = provider.GetData();
this.Result = calculator.Calculate(version);
}
public int Result
{
get;
private set;
}
}
When I write an ordered test using RhinoMocks, I can define the behaviour for the mocked classes like this:
[Test]
public void RhinoMockOrderedTest()
{
var mockRepository = new MockRepository();
var provider = mockRepository.DynamicMock<IProvider>();
var calculator = mockRepository.DynamicMock<ICalculator>();
using (mockRepository.Ordered())
{
provider.Expect(p => p.GetData()).Return(4);
calculator.Expect(c => c.Calculate(4)).Return(9);
}
mockRepository.ReplayAll();
var op = new Operator();
op.Operate(provider, calculator);
mockRepository.VerifyAll();
Assert.That(op.Result, Is.EqualTo(9));
}
Now I was trying to write an ordered test like the one above using NSubstitute, where I was also trying to check the call order and using defined return values:
[Test]
public void NSubstituteOrderedTest()
{
var provider = Substitute.For<IProvider>();
var calculator = Substitute.For<ICalculator>();
var op = new Operator();
op.Operate(provider, calculator);
Received.InOrder(() =>
{
provider.GetData().Returns(4);
calculator.Calculate(4).Returns(9);
});
Assert.That(op.Result, Is.EqualTo(9));
}
Unfortunalely this does not work. It seems to me, when I try to use .Returns for a methon inside of the Received.InOrder - Action, it will alwys fail like this:
NSubstitute.Exceptions.CouldNotSetReturnDueToMissingInfoAboutLastCallException
: Could not find information about the last call to return from.
Make sure you called Returns() after calling your substitute (for
example: mySub.SomeMethod().Returns(value)), and that you are not
configuring other substitutes within Returns() (for example, avoid
this: mySub.SomeMethod().Returns(ConfigOtherSub())).
If you substituted for a class rather than an interface, check that
the call to your substitute was on a virtual/abstract member. Return
values cannot be configured for non-virtual/non-abstract members.
Correct use: mySub.SomeMethod().Returns(returnValue);
Potentially problematic use:
mySub.SomeMethod().Returns(ConfigOtherSub()); Instead try: var
returnValue = ConfigOtherSub();
mySub.SomeMethod().Returns(returnValue);
How can I write this test using NSubstitute ?
Thanks,
Nico

NSubstitute works differently to Rhino Mocks here -- it only supports Arrange-Act-Assert (AAA) style tests. This means we need to stub out calls we're interested in (arrange), run the code we want to test (act), then assert the results are as expected (assert).
Received.InOrder is only for assertions, and works like NSubstitute's Received() method for each call. Returns arranges for a call to return a particular result. NSubstitute does not allow us to mix the two. We can't do sub.Received().Calculate().Returns(42), and it does not make sense to in AAA as there is little point in stubbing a return value after asserting we have already acted on the subject being tested and received all required calls.
Here is a passing version of the test from the question that separates the stubbing/arranging from the assertions:
[Test]
public void NSubstituteOrderedTest() {
// Arrange
var provider = Substitute.For<IProvider>();
var calculator = Substitute.For<ICalculator>();
provider.GetData().Returns(4);
calculator.Calculate(4).Returns(9);
// Act
var op = new Operator();
op.Operate(provider, calculator);
// Assert
Received.InOrder(() =>
{
provider.GetData();
calculator.Calculate(4);
});
Assert.That(op.Result, Is.EqualTo(9));
}
Aside: I know this is a simplified example, but I think it's worth noting that in many cases we can get away without testing the call ordering. For this simple case, we know GetData() is called first, as its value is passed to Calculate(), so the order is enforced via the data dependency. If the end result is correct, we know the call chain was correct. For more complicated cases we can use types for this (Connect() returns ConnectedDb, then Query(ConnectedDb db) ensures Connect() was called first.
Relying on knowledge of the implementation details of the code being tested (such as call ordering) can lead to brittle tests (i.e. they fail with small changes that should not affect the overall result), so it is best to avoid this where possible.
However, even with this disclaimer, it is sometimes useful to assert call order instead, so I hope this answer clears up this NSubstitute feature for you. :)

RhinoMocks stubs with simplified implementation, Try-method pattern

I want to mock a quite huge repository with RhinoMocks, mainly for the purpose of fully implementing a huge and frequently changing interface, just instead of using VisualStudio's "Implement Interface" default implementation (which requires all mocks to be updated for an interface change and causes a lot of junk code).
I currently use Stubs, but I haven't found out how to override mocked default methods, except defining every possible input value. This is especially bad when using the bool TryGet(key, out value) pattern and when I need a default behaviour besides, if the key is not found (here: return false/null, in other cases: throw exception).
Is there any way to implement method forwarding in RhinoMocks?
public interface IMyRepository
{
// would normally be implemented by database access
bool TryGetNameById(int id, out string name);
// more...
}
// within some other class:
public void SetupMockRepository()
{
IDictionary<int, string> namesByIds = new Dictionary<int, string>()
//create and fill with mock values
var mockRep = new MockRepository()
var repStub = mockRep.Stub<IMyRepository>()
// something like this, forward inner calls,
// without defining expected input values
var repStub.Stub(r => r.TryGetNameById(int id, out string name)
.OutRef((id) => (namesByIds.TryGetValue(id, out name) ? name : null))
.Return((id) => namesByIds.ContainsKey(id)));
}
EDIT:
I tried a delegate now, which looks better, but still has problems:
private delegate bool TryGet<TKey, TValue>(TKey key, out TValue value);
public void SetupMockRepository()
{
// code from above omitted
string outName;
_stub.Stub(r=>r.TryGetNameById(0, out outName))
.IgnoreArguments()
.Do(new TryGet<int,string>(namesByIds.TryGetValue))
}
This is accepted, but when I run it, I get an InvalidOperationException:
"Previous method 'TryGetValue(123)' requires a return value or an exception to throw"

You can always make your own fake objects. With a mix of that and partial mocks, you should gain a maximum of flexibility.
public interface IMyRepository
{
bool TryGetNameById(int id, out string name);
}
public class FakeRepository : IMyRepository
{
// public on purpose, this is for testing after all
public readonly Dictionary<int, string> nameByIds = new Dictionary<int, string>();
// important to make all methods/properties virtual that are part of the interface implementation
public virtual bool TryGetNameById(int id, out string name)
{
return this.nameByIds.TryGetValue(id, out name);
}
}
Your setup would become this then:
public void Setup()
{
var mock = MockRepository.GeneratePartialMock<FakeRepository>();
mock.nameByIds.Add(1, "test");
}
And if you need more sophisticated behavior, you can still stub method calls since all methods are virtual.

Here's my first, partial solution:
The mock object required a .Replay() after the delegate settings, because the methods were used before MockRep.ReplayAll() was called (before the test itself).
I currently use expectations and record/replay logic only in few cases (and do not fully comprehend what it is like), and it is disturbing to find the mock objects fail on seemingly correct settings. I just want a default implementation with a few overridden methods/properties, which does not cause build failures if somebody changes unrelated parts of the interface.
Another, new problem is that setting up a .Do() delegate like
mockObj.Stub(o => o.DoSth(x))
.IgnoreArguments()
.Do(new Action(x => this.DoSth(x)))
causes an expectation error on a completely different method, which has default stub implementation and not been changed.

Moq Equals Only Works With IEquatable

I'm using the Moq framework for unit tests, and I came across this interesting problem.
public interface Bar : IEquatable<Bar>
{
}
[TestClass]
public class TestClass
{
Mock<Bar> a;
Mock<Bar> b;
public TestClass()
{
a = new Mock<Bar>();
b = new Mock<Bar>();
a.Setup(bar => bar.Equals(b.Object)).Returns(true);
}
[TestMethod]
public void AssertEqualsTest()
{
Assert.AreEqual(a.Object, b.Object); //fails
}
[TestMethod]
public void AssertIsTrueTest()
{
Assert.IsTrue(a.Object.Equals(b.Object)); //passes
}
}
First Issue
So Assert.AreEqual just fails. I don't want to have to use the line from the second test every time I need to check equality even though most (if not all) of my classes inherit from IEquatable.
You might think it fails because Setup is only setting the IEquality.Equals() function (which Assert.AreEqual probably doesn't check), but if you add the line
a.Setup(x => x.Equals((object)b.Object)).Returns(true);
to the constructor, it still fails.
Second Issue
If you comment out the : IEquatable<Bar> from the interface declaration (so that a.Setup overwrites object.Equals), both tests fail.
My desired outcome is to be able to setup equals on a Mock object and call Assert.AreEqual.

First issue
Checked via dotPeek. Assert.AreEqual calls the static method object.Equals to compare the instances. object.Equals uses operator == first and since the mock instance does not implement that operator, this will default to comparing references. Clearly, a and b are different instances so the comparison returns false.
Second issue
I haven't looked at the internals of Moq but I assume this happens because the interface does not declare an Equals method. Confirmed with the following (which succeeds):
public interface IBar
{
}
public class Bar : IBar
{
public override bool Equals(object obj)
{
return false;
}
}
[TestClass]
public class Class1
{
[TestMethod]
public void TestMoq()
{
var a = new Mock<Bar>();
var b = new Mock<Bar>();
a.Setup(bar => bar.Equals(b.Object)).Returns(true);
Assert.IsTrue(a.Object.Equals(b.Object));
}
}
If I remove the Bar.Equals override, the test will also fail. Just a guess but since Moq uses Castle internally, this issue can be explained by this Q&A.
Anyway, I think what you're doing now with Assert.IsTrue(a.Object.Equals(b.Object)); and IEquatable is a sufficient workaround for this.
By the way, as JaredPar asked above, why are you comparing mocks?

I used..
mockLine2.CallBase = True
And Why?
We're testing an order service.
As order has two lines, service removes any line matching some criteria.
Lines are an IList, List (etc) uses .equals() to find the item you want, so to remove the line, you need an implementation of equals that passes:
Assert.IsTrue(mockLine2.Object.Equals(mockLine2.Object)
"Moq mock isn't detecting that a line equals itself, so the line collection on the order will fail to find the line.")
You could argue that as Order isn't the class under test we should mock the order and assert that order.Lines.Remove() is called with the right line, but we've found it on balance less painful to use the real domain objects (they're all fairly dumb) than mock them.

How can I avoid multiple asserts in this unit test?

This is my first attempt to do unit tests, so please be patient with me.
I'm still trying to unit test a library that converts lists of POCOs to ADO.Recordsets.
Right now, I'm trying to write a test that creates a List<Poco>, converts it into a Recordset (using the method I want to test) and then checks if they contain the same information (like, if Poco.Foo == RS.Foo, and so on...).
This is the POCO:
public class TestPoco
{
public string StringValue { get; set; }
public int Int32Value { get; set; }
public bool BoolValue { get; set; }
}
...and this is the test so far (I'm using xUnit.net):
[Fact]
public void TheTest()
{
var input = new List<TestPoco>();
input.Add(new TestPoco { BoolValue = true, Int32Value = 1, StringValue = "foo" });
var actual = input.ToRecordset();
Assert.Equal(actual.BoolValue, true);
Assert.Equal(actual.Int32Value, 1);
Assert.Equal(actual.StringValue, "foo");
}
What I don't like about this are the three asserts at the end, one per property of the POCO.
I've read lots of times that multiple asserts in one test are evil (and I understand the reasons why, and I agree).
The problem is, how can I get rid of them?
I have Roy Osherove's excellent book "The Art of Unit Testing" right in front of me, and he has an example which covers exactly this (for those who have the book: chapter 7.2.6, page 202/203):
In his example, the method under test returns an AnalyzedOutput object with several properties, and he wants to assert all the properties to check if each one contains the expected value.
The solution in this case:
Create another AnalyzedOutput instance, fill it with the expected values and assert if it's equal to the one returned by the method under test (and override Equals() to be able to do this).
But I think I can't do this in my case, because the method that I want to test returns an ADODB.Recordset.
And in order to create another Recordset with the expected values, I would first need to create it completely from scratch:
// this probably doesn't actually compile, the actual conversion method
// doesn't exist yet and this is just to show the idea
var expected = new ADODB.RecordsetClass();
expected.Fields.Append("BoolValue", ADODB.DataTypeEnum.adBoolean);
expected.Fields.Append("Int32Value", ADODB.DataTypeEnum.adInteger);
expected.Fields.Append("StringValue", ADODB.DataTypeEnum.adVarWChar);
expected.AddNew();
expected.BoolValue = true;
expected.Int32Value = 1;
expected.StringValue = "foo";
expected.Update();
I don't like this either, because this is basically a duplication of some of the code in the actual conversion method (the method under test), which is another thing to avoid in tests.
So...what can I do now?
Is this level of duplication still acceptable in this special situation, or is there a better way how to test this?

I'd argue that in the spirit of the thing, this is fine. The reason that multiple asserts are "evil", if I recall correctly, is that it implies that you are testing multiple things in one test. In this case, you are indeed doing that in that you are testing each field, presumably to make sure this works for several different types. Since that's all an object equality test would do anyway, I think you are in the clear.
If you really wanted to be militant about it, write one test per property (j/k!)

Multiple assertions per unit test are perfectly fine in my book, as long as the multiple assertions are all asserting the same test condition. In your case, they're testing that the conversion was successful, so the test passing is conditional on all of those assertions being true. As a result, it's perfectly fine!
I'd classify "one assertion per test" as a guideline, not a hard-and-fast rule. When you disregard it, consider why you're disregarding it.
That said, a way around it is to create a single test class that, on class setup, runs your test process. Then each test is just an assertion on a single property. For example:
public class ClassWithProperities
{
public string Foo { get; set; }
public int Bar { get; set; }
}
public static class Converter
{
public static ClassWithProperities Convert(string foo, int bar)
{
return new ClassWithProperities {Foo=foo, Bar=bar};
}
}
[TestClass]
public class PropertyTestsWhenFooIsTestAndBarIsOne
{
private static ClassWithProperities classWithProperties;
[ClassInitialize]
public static void ClassInit(TestContext testContext)
{
//Arrange
string foo = "test";
int bar = 1;
//Act
classWithProperties = Converter.Convert(foo, bar);
//Assert
}
[TestMethod]
public void AssertFooIsTest()
{
Assert.AreEqual("test", classWithProperties.Foo);
}
[TestMethod]
public void AssertBarIsOne()
{
Assert.AreEqual(1, classWithProperties.Bar);
}
}
[TestClass]
public class PropertyTestsWhenFooIsXyzAndBarIsTwoThousand
{
private static ClassWithProperities classWithProperties;
[ClassInitialize]
public static void ClassInit(TestContext testContext)
{
//Arrange
string foo = "Xyz";
int bar = 2000;
//Act
classWithProperties = Converter.Convert(foo, bar);
//Assert
}
[TestMethod]
public void AssertFooIsXyz()
{
Assert.AreEqual("Xyz", classWithProperties.Foo);
}
[TestMethod]
public void AssertBarIsTwoThousand()
{
Assert.AreEqual(2000, classWithProperties.Bar);
}
}

I agree with all the other comments that it is fine to do so, if you are logically testing one thing.
There is however a difference between have many assertions in a single unit test than having a separate unit test for each property. I call it 'Blocking Assertions' (Probably a better name out there). If you have many assertions in one test then you are only going to know about a failure in the first property that failed the assertion. If you have say 10 properties and 5 of them returned incorrect results then you will have to go through fixing the first one, re-run the test and notice another one failed, then fix that etc.
Depending on how you look at it this could be quite frustrating. On the flip side having 5 simple unit tests failing suddenly could also be off putting, but it might give you a clearer picture as to what have caused those to fail all at once and possibly direct you more quickly to a known fix (perhaps).
I would say if you need to test multiple properties keep the number down (possibly under 5) to avoid the blocking assertion issue getting out of control. If there are a ton of properties to test then perhaps it is a sign that your model is representing too much or perhaps you can look at grouping properties into multiple tests.

Those 3 asserts are valid. If you used a framework more like mspec, it would look like:
public class When_converting_a_TestPoco_to_Recordset
{
protected static List<TestPoco> inputs;
protected static Recordset actual;
Establish context = () => inputs = new List<TestPoco> { new TestPoco { /* set values */ } };
Because of = () => actual = input.ToRecordset ();
It should_have_copied_the_bool_value = () => actual.BoolValue.ShouldBeTrue ();
It should_have_copied_the_int_value = () => actual.Int32Value.ShouldBe (1);
It should_have_copied_the_String_value = () => actual.StringValue.ShouldBe ("foo");
}
I generally use mspec as a benchmark to see if my tests make sense. Your tests read just fine with mspec, and that gives me some semi-automated warm fuzzies that I'm testing the correct things.
For that matter, you've done a better job with multiple asserts. I hate seeing tests that look like:
Assert.That (actual.BoolValue == true && actual.Int32Value == 1 && actual.StringValue == "foo");
Because when that fails, the error message "expected True, got False" is completely worthless. Multiple asserts, and using the unit-testing framework as much as possible, will help you a great deal.

This should be something to check out http://rauchy.net/oapt/
Tool that generates a new test case for every assert.

How to assert a private method on concrete class is called (TypeMock/NMock/etc..)?

I am trying to write a unit test for the 'IsUnique' function in the class below that looks like this:
class Foo
{
public bool IsUnique(params...)
{
ValidateStuffExists(params);
return CheckUniqueness(params);
}
private void ValidateStuffExists(params)
{
//does some validation
}
private bool CheckUniqueness(params)
{
//does logic to determine if its unique per params
return result;
}
}
The only thing I want to test here is that ValidateStuffExists and CheckUniqueness is called and passed the arguments. That's all this function does so it's all I'm going to test (I'll bend the 'test public behavior only' psuedo-rule and test the private methods here, because its either have one big complicated method/tests or testing 2 private methods).
I am open to any mocking library. I use NMock and didn't think it was up for the Task - so I downloaded TypeMock as I've done reading and heard that this was the best and that it could mock out even concrete classes / non interface method calls...
I'm doing something like this in my test and it throws exception at the 'Isolate.WhenCalled' line:
CrmEntityUniqueValidator_Accessor target = new CrmEntityUniqueValidator_Accessor(); // TODO: Initialize to an appropriate value
DynamicEntity entity = null; // TODO: Initialize to an appropriate value
string[] propertyNames = null; // TODO: Initialize to an appropriate value
bool tru = true;
Isolate.WhenCalled(() => target.CheckUniqueness(entity, propertyNames, null, null)).WillReturn(tru);
target.ValidatePropertiesExist(entity, propertyNames);
Isolate.Verify.WasCalledWithArguments(() => target.ValidatePropertiesExist(entity, propertyNames));
Isolate.Verify.WasCalledWithArguments(() => target.CheckUniqueness(entity, propertyNames, null, null));
This throws an exception like "*** WhenCalled does not support using a method call as an argument."
Even though I'm able to do the same thing with a CLR class - I can mock out DateTime.Now doing this (code works):
DateTime endOfWorld = new DateTime(2012, 12, 23);
Isolate.WhenCalled(() => DateTime.Now).WillReturn(endOfWorld);
DateTime dt = DateTime.Now;
Assert.AreEqual(dt, endOfWorld);
Anyone have any advice here? Do I have to split these 2 methods into a seperate class and make an interface is the only way? or complicate my method/tests??? There must be something I'm missing here... Thanks much for any help in advance.
EDIT: I guess I'm trying to mock out the 2 private methods in the class for the one unit test. How could I do this without having to split out those 2 methods into a seperate class / interface?

try
Isolate.NonPublic.WhenCalled(object,"nonpublic method").IgnoreCall
or Isolate.Verify.NonPublic.WasCalled(object,"method"..

RhinoMocks has the AssertWasCalled method which may serve the purpose of guaranteeing that a method calls another method... but I'm not sure you can do that on a private function of the same class you are unit testing. I think you could only do that if the two methods you wanted to stub were in another dependency class that was injected.