I'm using the Moq framework for unit tests, and I came across this interesting problem.
public interface Bar : IEquatable<Bar>
{
}
[TestClass]
public class TestClass
{
Mock<Bar> a;
Mock<Bar> b;
public TestClass()
{
a = new Mock<Bar>();
b = new Mock<Bar>();
a.Setup(bar => bar.Equals(b.Object)).Returns(true);
}
[TestMethod]
public void AssertEqualsTest()
{
Assert.AreEqual(a.Object, b.Object); //fails
}
[TestMethod]
public void AssertIsTrueTest()
{
Assert.IsTrue(a.Object.Equals(b.Object)); //passes
}
}
First Issue
So Assert.AreEqual just fails. I don't want to have to use the line from the second test every time I need to check equality even though most (if not all) of my classes inherit from IEquatable.
You might think it fails because Setup is only setting the IEquality.Equals() function (which Assert.AreEqual probably doesn't check), but if you add the line
a.Setup(x => x.Equals((object)b.Object)).Returns(true);
to the constructor, it still fails.
Second Issue
If you comment out the : IEquatable<Bar> from the interface declaration (so that a.Setup overwrites object.Equals), both tests fail.
My desired outcome is to be able to setup equals on a Mock object and call Assert.AreEqual.
First issue
Checked via dotPeek. Assert.AreEqual calls the static method object.Equals to compare the instances. object.Equals uses operator == first and since the mock instance does not implement that operator, this will default to comparing references. Clearly, a and b are different instances so the comparison returns false.
Second issue
I haven't looked at the internals of Moq but I assume this happens because the interface does not declare an Equals method. Confirmed with the following (which succeeds):
public interface IBar
{
}
public class Bar : IBar
{
public override bool Equals(object obj)
{
return false;
}
}
[TestClass]
public class Class1
{
[TestMethod]
public void TestMoq()
{
var a = new Mock<Bar>();
var b = new Mock<Bar>();
a.Setup(bar => bar.Equals(b.Object)).Returns(true);
Assert.IsTrue(a.Object.Equals(b.Object));
}
}
If I remove the Bar.Equals override, the test will also fail. Just a guess but since Moq uses Castle internally, this issue can be explained by this Q&A.
Anyway, I think what you're doing now with Assert.IsTrue(a.Object.Equals(b.Object)); and IEquatable is a sufficient workaround for this.
By the way, as JaredPar asked above, why are you comparing mocks?
I used..
mockLine2.CallBase = True
And Why?
We're testing an order service.
As order has two lines, service removes any line matching some criteria.
Lines are an IList, List (etc) uses .equals() to find the item you want, so to remove the line, you need an implementation of equals that passes:
Assert.IsTrue(mockLine2.Object.Equals(mockLine2.Object)
"Moq mock isn't detecting that a line equals itself, so the line collection on the order will fail to find the line.")
You could argue that as Order isn't the class under test we should mock the order and assert that order.Lines.Remove() is called with the right line, but we've found it on balance less painful to use the real domain objects (they're all fairly dumb) than mock them.
Related
I used to write my tests with RhinoMocks and have switched to NSubstitute.
Now I have a problem concerning ordered test.
Lets say I have three small classes like
public interface IProvider
{
int GetData();
}
public class Provider : IProvider
{
public int GetData()
{
return 3;
}
}
public interface ICalculator
{
int Calculate(int data);
}
public class Calculator : ICalculator
{
public int Calculate(int data)
{
if (data < 3)
{
return data;
}
return data * 2;
}
}
public class Operator
{
public void Operate(IProvider provider, ICalculator calculator)
{
int version = provider.GetData();
this.Result = calculator.Calculate(version);
}
public int Result
{
get;
private set;
}
}
When I write an ordered test using RhinoMocks, I can define the behaviour for the mocked classes like this:
[Test]
public void RhinoMockOrderedTest()
{
var mockRepository = new MockRepository();
var provider = mockRepository.DynamicMock<IProvider>();
var calculator = mockRepository.DynamicMock<ICalculator>();
using (mockRepository.Ordered())
{
provider.Expect(p => p.GetData()).Return(4);
calculator.Expect(c => c.Calculate(4)).Return(9);
}
mockRepository.ReplayAll();
var op = new Operator();
op.Operate(provider, calculator);
mockRepository.VerifyAll();
Assert.That(op.Result, Is.EqualTo(9));
}
Now I was trying to write an ordered test like the one above using NSubstitute, where I was also trying to check the call order and using defined return values:
[Test]
public void NSubstituteOrderedTest()
{
var provider = Substitute.For<IProvider>();
var calculator = Substitute.For<ICalculator>();
var op = new Operator();
op.Operate(provider, calculator);
Received.InOrder(() =>
{
provider.GetData().Returns(4);
calculator.Calculate(4).Returns(9);
});
Assert.That(op.Result, Is.EqualTo(9));
}
Unfortunalely this does not work. It seems to me, when I try to use .Returns for a methon inside of the Received.InOrder - Action, it will alwys fail like this:
NSubstitute.Exceptions.CouldNotSetReturnDueToMissingInfoAboutLastCallException
: Could not find information about the last call to return from.
Make sure you called Returns() after calling your substitute (for
example: mySub.SomeMethod().Returns(value)), and that you are not
configuring other substitutes within Returns() (for example, avoid
this: mySub.SomeMethod().Returns(ConfigOtherSub())).
If you substituted for a class rather than an interface, check that
the call to your substitute was on a virtual/abstract member. Return
values cannot be configured for non-virtual/non-abstract members.
Correct use: mySub.SomeMethod().Returns(returnValue);
Potentially problematic use:
mySub.SomeMethod().Returns(ConfigOtherSub()); Instead try: var
returnValue = ConfigOtherSub();
mySub.SomeMethod().Returns(returnValue);
How can I write this test using NSubstitute ?
Thanks,
Nico
NSubstitute works differently to Rhino Mocks here -- it only supports Arrange-Act-Assert (AAA) style tests. This means we need to stub out calls we're interested in (arrange), run the code we want to test (act), then assert the results are as expected (assert).
Received.InOrder is only for assertions, and works like NSubstitute's Received() method for each call. Returns arranges for a call to return a particular result. NSubstitute does not allow us to mix the two. We can't do sub.Received().Calculate().Returns(42), and it does not make sense to in AAA as there is little point in stubbing a return value after asserting we have already acted on the subject being tested and received all required calls.
Here is a passing version of the test from the question that separates the stubbing/arranging from the assertions:
[Test]
public void NSubstituteOrderedTest() {
// Arrange
var provider = Substitute.For<IProvider>();
var calculator = Substitute.For<ICalculator>();
provider.GetData().Returns(4);
calculator.Calculate(4).Returns(9);
// Act
var op = new Operator();
op.Operate(provider, calculator);
// Assert
Received.InOrder(() =>
{
provider.GetData();
calculator.Calculate(4);
});
Assert.That(op.Result, Is.EqualTo(9));
}
Aside: I know this is a simplified example, but I think it's worth noting that in many cases we can get away without testing the call ordering. For this simple case, we know GetData() is called first, as its value is passed to Calculate(), so the order is enforced via the data dependency. If the end result is correct, we know the call chain was correct. For more complicated cases we can use types for this (Connect() returns ConnectedDb, then Query(ConnectedDb db) ensures Connect() was called first.
Relying on knowledge of the implementation details of the code being tested (such as call ordering) can lead to brittle tests (i.e. they fail with small changes that should not affect the overall result), so it is best to avoid this where possible.
However, even with this disclaimer, it is sometimes useful to assert call order instead, so I hope this answer clears up this NSubstitute feature for you. :)
I'm not very familiar with RhinoMocks and I'm trying to break into TDD on a side project of mine. I've been moving along okay until I hit a wall trying to get this behavior under test.
private int _result;
public int Result { get { return _result; } }
public void GetRandomValue()
{
Random n = new Random();
_result = n.Next(1,100);
}
Since this is an abstract class, how can I use RhinoMocks to verify that when I call the GetRandomValue method that the Result property is set? Or is this a necessary test? Or a better way to design this small bit of code?
The way I would go about testing it is
var randomizerClass = new RandomizerClass();
ranomizerClass.GetRandomValue();
var result = ranomizerClass.Result;
Assert.IsTrue(result > 0 && result < 101);
And the ranomizerClass is the concrete implementation of the abstract class. You don't need to separately test your abstract class since that will be done through the concrete implementation
Maybe you should extract the interface from the abstract class and simply implement the mock for it with corresponding methods?
Presumably when you instantiate the type, the _result field is set to it's default value of 0. So can't you just create an instance of the type, call the method, and then check the value of the Result property doesn't equal 0?
It seems odd that a method called GetRandomValue, doesn't actually return anything. Is there a reason why this method doesn't return the result? Also, I would instantiate your Random type outside the scope of the method.
How can you test that you're getting a random number? It's random, so you can't write a test that's actually asserting anything useful. You could make an IRandomNumberProvider interface and implement it on a RandomNumberProvider, then mock that for testing purposes, but you wouldn't be testing anything of value in my opinion. Even if you did like some other folks suggested and asserted that the number is between 1 and 100, you haven't tested anything useful in your code, you've just ensured that Microsoft's implementation of Next is working, which is a given.
Your GetRandomValue() method shouldn't set Result, though. It should return the result. Otherwise, you've introduced a side effect into your class, and that generally can cause weird, hard-to-track-down bugs.
In the cases where you need to test functionality in an abstract class, simply create a class that implements it in your test project.
E.g. given the following class
public abstract class MyClass
{
private int _result;
public int Result { get { return _result; } }
public void GetRandomValue()
{
Random n = new Random();
_result = n.Next(1,100);
}
}
Create a test class in your test project
public class StubMyClass : MyClass
{
}
and then test it as follows
[Test]
public void ShouldSetResultAfterCallingGetRandomValue()
{
var myClass = new StubMyClass();
myClass.GetRandomValue();
int expected = 1234;
Assert.That(myClass.Result, Is.EqualTo(expected));
}
This is my first attempt to do unit tests, so please be patient with me.
I'm still trying to unit test a library that converts lists of POCOs to ADO.Recordsets.
Right now, I'm trying to write a test that creates a List<Poco>, converts it into a Recordset (using the method I want to test) and then checks if they contain the same information (like, if Poco.Foo == RS.Foo, and so on...).
This is the POCO:
public class TestPoco
{
public string StringValue { get; set; }
public int Int32Value { get; set; }
public bool BoolValue { get; set; }
}
...and this is the test so far (I'm using xUnit.net):
[Fact]
public void TheTest()
{
var input = new List<TestPoco>();
input.Add(new TestPoco { BoolValue = true, Int32Value = 1, StringValue = "foo" });
var actual = input.ToRecordset();
Assert.Equal(actual.BoolValue, true);
Assert.Equal(actual.Int32Value, 1);
Assert.Equal(actual.StringValue, "foo");
}
What I don't like about this are the three asserts at the end, one per property of the POCO.
I've read lots of times that multiple asserts in one test are evil (and I understand the reasons why, and I agree).
The problem is, how can I get rid of them?
I have Roy Osherove's excellent book "The Art of Unit Testing" right in front of me, and he has an example which covers exactly this (for those who have the book: chapter 7.2.6, page 202/203):
In his example, the method under test returns an AnalyzedOutput object with several properties, and he wants to assert all the properties to check if each one contains the expected value.
The solution in this case:
Create another AnalyzedOutput instance, fill it with the expected values and assert if it's equal to the one returned by the method under test (and override Equals() to be able to do this).
But I think I can't do this in my case, because the method that I want to test returns an ADODB.Recordset.
And in order to create another Recordset with the expected values, I would first need to create it completely from scratch:
// this probably doesn't actually compile, the actual conversion method
// doesn't exist yet and this is just to show the idea
var expected = new ADODB.RecordsetClass();
expected.Fields.Append("BoolValue", ADODB.DataTypeEnum.adBoolean);
expected.Fields.Append("Int32Value", ADODB.DataTypeEnum.adInteger);
expected.Fields.Append("StringValue", ADODB.DataTypeEnum.adVarWChar);
expected.AddNew();
expected.BoolValue = true;
expected.Int32Value = 1;
expected.StringValue = "foo";
expected.Update();
I don't like this either, because this is basically a duplication of some of the code in the actual conversion method (the method under test), which is another thing to avoid in tests.
So...what can I do now?
Is this level of duplication still acceptable in this special situation, or is there a better way how to test this?
I'd argue that in the spirit of the thing, this is fine. The reason that multiple asserts are "evil", if I recall correctly, is that it implies that you are testing multiple things in one test. In this case, you are indeed doing that in that you are testing each field, presumably to make sure this works for several different types. Since that's all an object equality test would do anyway, I think you are in the clear.
If you really wanted to be militant about it, write one test per property (j/k!)
Multiple assertions per unit test are perfectly fine in my book, as long as the multiple assertions are all asserting the same test condition. In your case, they're testing that the conversion was successful, so the test passing is conditional on all of those assertions being true. As a result, it's perfectly fine!
I'd classify "one assertion per test" as a guideline, not a hard-and-fast rule. When you disregard it, consider why you're disregarding it.
That said, a way around it is to create a single test class that, on class setup, runs your test process. Then each test is just an assertion on a single property. For example:
public class ClassWithProperities
{
public string Foo { get; set; }
public int Bar { get; set; }
}
public static class Converter
{
public static ClassWithProperities Convert(string foo, int bar)
{
return new ClassWithProperities {Foo=foo, Bar=bar};
}
}
[TestClass]
public class PropertyTestsWhenFooIsTestAndBarIsOne
{
private static ClassWithProperities classWithProperties;
[ClassInitialize]
public static void ClassInit(TestContext testContext)
{
//Arrange
string foo = "test";
int bar = 1;
//Act
classWithProperties = Converter.Convert(foo, bar);
//Assert
}
[TestMethod]
public void AssertFooIsTest()
{
Assert.AreEqual("test", classWithProperties.Foo);
}
[TestMethod]
public void AssertBarIsOne()
{
Assert.AreEqual(1, classWithProperties.Bar);
}
}
[TestClass]
public class PropertyTestsWhenFooIsXyzAndBarIsTwoThousand
{
private static ClassWithProperities classWithProperties;
[ClassInitialize]
public static void ClassInit(TestContext testContext)
{
//Arrange
string foo = "Xyz";
int bar = 2000;
//Act
classWithProperties = Converter.Convert(foo, bar);
//Assert
}
[TestMethod]
public void AssertFooIsXyz()
{
Assert.AreEqual("Xyz", classWithProperties.Foo);
}
[TestMethod]
public void AssertBarIsTwoThousand()
{
Assert.AreEqual(2000, classWithProperties.Bar);
}
}
I agree with all the other comments that it is fine to do so, if you are logically testing one thing.
There is however a difference between have many assertions in a single unit test than having a separate unit test for each property. I call it 'Blocking Assertions' (Probably a better name out there). If you have many assertions in one test then you are only going to know about a failure in the first property that failed the assertion. If you have say 10 properties and 5 of them returned incorrect results then you will have to go through fixing the first one, re-run the test and notice another one failed, then fix that etc.
Depending on how you look at it this could be quite frustrating. On the flip side having 5 simple unit tests failing suddenly could also be off putting, but it might give you a clearer picture as to what have caused those to fail all at once and possibly direct you more quickly to a known fix (perhaps).
I would say if you need to test multiple properties keep the number down (possibly under 5) to avoid the blocking assertion issue getting out of control. If there are a ton of properties to test then perhaps it is a sign that your model is representing too much or perhaps you can look at grouping properties into multiple tests.
Those 3 asserts are valid. If you used a framework more like mspec, it would look like:
public class When_converting_a_TestPoco_to_Recordset
{
protected static List<TestPoco> inputs;
protected static Recordset actual;
Establish context = () => inputs = new List<TestPoco> { new TestPoco { /* set values */ } };
Because of = () => actual = input.ToRecordset ();
It should_have_copied_the_bool_value = () => actual.BoolValue.ShouldBeTrue ();
It should_have_copied_the_int_value = () => actual.Int32Value.ShouldBe (1);
It should_have_copied_the_String_value = () => actual.StringValue.ShouldBe ("foo");
}
I generally use mspec as a benchmark to see if my tests make sense. Your tests read just fine with mspec, and that gives me some semi-automated warm fuzzies that I'm testing the correct things.
For that matter, you've done a better job with multiple asserts. I hate seeing tests that look like:
Assert.That (actual.BoolValue == true && actual.Int32Value == 1 && actual.StringValue == "foo");
Because when that fails, the error message "expected True, got False" is completely worthless. Multiple asserts, and using the unit-testing framework as much as possible, will help you a great deal.
This should be something to check out http://rauchy.net/oapt/
Tool that generates a new test case for every assert.
I have the following code and am trying to unit test it:
public override IRenderable GetRenderable()
{
var val = SomeCalculationUsingClassMemberVariables();
return new EquationRenderable(val);
}
It seems like I want to use a factory here so I can separate the creation of the IRenderable from this class. The problem is that I have many of these classes that create different IRenderables that are constructed in different ways, so I would need to implement a new factory method for each one. What is the best way to solve this problem?
Good question.
First of all, feeling tempted to use AbstractFactories everywhere smells a bit as DI container is not used the "right" way or design could be improved.
But sometimes I've also come across this problem. I see following:
using AbstractFactory/Factory and Inject it. For C# you have to advantage that you can pass delegates, acting as interface for the creation of instance.
'new' is OK, simply test the output of 'new'.
Stub the call of 'new' inside extracted method (hacky!!)
Injection got mentioned already so I won't repeat. I am more into Java so please excuse some syntax errors.
Test the output of 'new'
I often use this, if the 'new' created instances are domain-objects and not services. Because it is returned directly in method I can test the direct output with my test.
Prod-Code:
...
public override IRenderable GetRenderable()
{
var val = SomeCalculationUsingClassMemberVariables();
return new EquationRenderable(val);
}
Test Case:
...
[Test]
public void test_new()
{
SUT sut = ...;
IRenderable r = sut.GetRenderable();
assertTrue(r instanceof EquationRenderable);
}
Stub the call of 'new' itself
Testing direct output from above is only possible if you somehow get it as return value. Things get more complicated if the "sideeffect" of your code are indirect outputs, which you can't sense directly by the return value. If so I often extract-method of the new-creation and then have it under control in my test. This is yucky and I more use it to go safe with my test and start more refactoring later (DI and factories). I sometimes do this in legacy code where services are created with 'new' directly and refactoring to DI is too risky without tests.
Prod-Code:
...
public override IRenderable GetRenderable()
{
var val = SomeCalculationUsingClassMemberVariables();
return createEquationRenderable();
}
public IRenderable createEquationRenderable()
{
return new EquationRenderable(val);
}
Test Case:
...
class Stubbed : SUT
{
boolean called = false;
public override EquationRenderable createEquationRenderable()
{
called=true;
return MyMock();
}
}
[Test]
public void test_new()
{
Stubbed sut = new Stubbed();
sut.GetRenderable();
assertTrue(sut.called);
// do further stuff on MyMock
}
I know, the example is overkill and a bit senseless, it is just for describing the idea. I am sure above could be shortcutted with mocking-frameworks for C#. Anyway testing the return-value direct output is more trivial and better approach here.
Maybe you have a more detailed example?
depending on the uniformity of your concrete IRenderable constructors you can use the following pattern for factory creating
public IRenderable CreateInstance<T>(object calculation) where T : IRenderable
{
Activator.CreateInstance<T>(new[] { calculation });
}
or if you have many different constructors you can use the params keyword to pass arbitrary amounts of arguments
public IRenderable CreateInstance<T>(params object[] args) where T : IRenderable
{
Activator.CreateInstance<T>(args);
}
To be able to do some kind of runtime check of the arguments you use this code before calling the Activator.CreateInstance
var types = args.Select(o => o.GetType()).ToArray();
var c = typeof(T).GetConstructor(types);
if (c == null)
{
throw new InvalidOperationException("No matched constructor")
}
A better way may be to simply unit test the code as is instead of refactoring it. Technically, this can be done by using a suitable mocking tool such as TypeMock Isolator or Microsoft Moles (there is a third one which I don't remember now).
I have this dataAccess mock object and I'm trying to verify that one of its methods is being invoked, and that the argument passed into this method fulfills certain constraints. As best I can tell, this method is indeed being invoked, and with the constraints fulfilled. This line of the test throws a MockException:
data.Verify(d => d.InsertInvoice(It.Is<Invoice>(i => i.TermPaymentAmount == 0m)), Times.Once());
However, removing the constraint and accepting any invoice passes the test:
data.Verify(d => d.InsertInvoice(It.IsAny<Invoice>()), Times.Once());
I've created a test windows form that instantiates this test class, runs its .Setup() method, and then calls the method which I am wishing to test. I insert a breakpoint on the line of code where the mock object is failing the test
data.InsertInvoice(invoice);
to actually hover over the invoice, and I can confirm that its .TermPaymentAmount decimal property is indeed zero at the time the method is invoked.
Out of desperation, I even added a call back to my dataAccess mock:
data.Setup(d => d.InsertInvoice(It.IsAny<Invoice>())).Callback((Invoice inv) => MessageBox.Show(inv.TermPaymentAmount.ToString("G17")));
And this gives me a message box showing 0. This is really baffling me, and no one else in my shop has been able to figure this out. Any help would be appreciated.
A barely related question, which I should probably ask independently, is whether it is preferable to use Mock.Verify() as I have here, or to use Mock.Expect(). Verifiable followed by Mock.VerifyAll() as I have seen other people doing? If the answer is situational, which situations would warrent the use of one over the other?
Just to rule this out (because I've got some strange behavior like you :
Try Times.Exactly(1) instead of Times.Once().
I've had some weird thing happening when using Times.Once()
Though, I would have guessed that Once() is using Exactly(1) internally...
I am using version 3.1.0.0 of Moq and am not experiencing this issue with the following test case.
internal class Program
{
private static void Main(string[] args)
{
var mock = new Mock<IData>();
mock.Setup(d => d.InsertInvoice(It.IsAny<Invoice>()));
var service = new MyService(mock.Object);
service.DoSomething(new Invoice());
mock.Verify(d => d.InsertInvoice(It.Is<Invoice>(i => i.TermPaymentAmount == 0m)), Times.Once());
Console.ReadLine();
}
}
internal class MyService
{
private readonly IData _data;
public MyService(IData data)
{
_data = data;
}
public void DoSomething(Invoice invoice)
{
_data.InsertInvoice(invoice);
}
}
public class Invoice
{
public decimal TermPaymentAmount { get; set; }
}
public interface IData
{
void InsertInvoice(Invoice invoice);
}
Could you supply maybe a bit more information on how you are using the mock?
My guess is that it may have something to do with the problems inherent with equivalency checking of floating point values. Is your Invoice.TermPaymentAmount property value the result of a calculation, perhaps?
For an idea of what I'm talking about, see this possibly related question:
Is it safe to check floating point values for equality to 0?