While refactoring unit tests in a project, I found some tests that should have been failing but were succeeding for some mysterious reason. After removing irrelevant code and moving everything into one method, the following minimal example still has the original behavior:
[Test]
public void TestThatShouldFail()
{
// Arrange
var mock = MockRepository.GenerateStub<ISomething>();
mock.Stub(wi => wi.SomeProperty).Return(MockRepository.GenerateStub<ISomeProperty>());
mock.SomeProperty.Stub(t => t.SomethingElse).Return(new SomethingElse());
...
// Act
_foo.Foo();
// Assert
mock.AssertWasCalled(wi => wi.SomeProperty.DoSomething());
}
The variable mock is never passed, exposed or exported in any way which is available to the code running in the Act part. Still, the test passes, which should mean that the DoSomething method was called on the SomeProperty of the mock variable, which is obviously wrong.
How can this happen?
The code above is already a result of some investigation, so it only contains the lines relevant to the problem and its solution.
The culprit is the last line of the Arrange part.
As it turns out, the Assert line doesn't really check whether the DoSomething method was called on SomeProperty, but whether SomeProperty was accessed on mock! And we actually did that in the last Arrange line, when stubbing a method on it.
I couldn't find any official documentation for it, so I can only assume that it considers only the first level of the expression in its argument, so one shouldn't pass expressions with multiple levels of member access (dots) to AssertWasCalled (or AssertWasNotCalled).
(The proper way to check it would be mock.SomeProperty.AssertWasCalled(wi => wi.DoSomething()), but it wasn't needed here.)
Related
What is the best way to unit test a method that doesn't return anything? Specifically in c#.
What I am really trying to test is a method that takes a log file and parses it for specific strings. The strings are then inserted into a database. Nothing that hasn't been done before but being VERY new to TDD I am wondering if it is possible to test this or is it something that doesn't really get tested.
If a method doesn't return anything, it's either one of the following
imperative - You're either asking the object to do something to itself.. e.g change state (without expecting any confirmation.. its assumed that it will be done)
informational - just notifying someone that something happened (without expecting action or response) respectively.
Imperative methods - you can verify if the task was actually performed. Verify if state change actually took place. e.g.
void DeductFromBalance( dAmount )
can be tested by verifying if the balance post this message is indeed less than the initial value by dAmount
Informational methods - are rare as a member of the public interface of the object... hence not normally unit-tested. However if you must, You can verify if the handling to be done on a notification takes place. e.g.
void OnAccountDebit( dAmount ) // emails account holder with info
can be tested by verifying if the email is being sent
Post more details about your actual method and people will be able to answer better.
Update: Your method is doing 2 things. I'd actually split it into two methods that can now be independently tested.
string[] ExamineLogFileForX( string sFileName );
void InsertStringsIntoDatabase( string[] );
String[] can be easily verified by providing the first method with a dummy file and expected strings. The second one is slightly tricky.. you can either use a Mock (google or search stackoverflow on mocking frameworks) to mimic the DB or hit the actual DB and verify if the strings were inserted in the right location. Check this thread for some good books... I'd recomment Pragmatic Unit Testing if you're in a crunch.
In the code it would be used like
InsertStringsIntoDatabase( ExamineLogFileForX( "c:\OMG.log" ) );
Test its side-effects. This includes:
Does it throw any exceptions? (If it should, check that it does. If it shouldn't, try some corner cases which might if you're not careful - null arguments being the most obvious thing.)
Does it play nicely with its parameters? (If they're mutable, does it mutate them when it shouldn't and vice versa?)
Does it have the right effect on the state of the object/type you're calling it on?
Of course, there's a limit to how much you can test. You generally can't test with every possible input, for example. Test pragmatically - enough to give you confidence that your code is designed appropriately and implemented correctly, and enough to act as supplemental documentation for what a caller might expect.
As always: test what the method is supposed to do!
Should it change global state (uuh, code smell!) somewhere?
Should it call into an interface?
Should it throw an exception when called with the wrong parameters?
Should it throw no exception when called with the right parameters?
Should it ...?
Try this:
[TestMethod]
public void TestSomething()
{
try
{
YourMethodCall();
Assert.IsTrue(true);
}
catch {
Assert.IsTrue(false);
}
}
Void return types / Subroutines are old news. I haven't made a Void return type (Unless I was being extremely lazy) in like 8 years (From the time of this answer, so just a bit before this question was asked).
Instead of a method like:
public void SendEmailToCustomer()
Make a method that follows Microsoft's int.TryParse() paradigm:
public bool TrySendEmailToCustomer()
Maybe there isn't any information your method needs to return for usage in the long-run, but returning the state of the method after it performs its job is a huge use to the caller.
Also, bool isn't the only state type. There are a number of times when a previously-made Subroutine could actually return three or more different states (Good, Normal, Bad, etc). In those cases, you'd just use
public StateEnum TrySendEmailToCustomer()
However, while the Try-Paradigm somewhat answers this question on how to test a void return, there are other considerations too. For example, during/after a "TDD" cycle, you would be "Refactoring" and notice you are doing two things with your method... thus breaking the "Single Responsibility Principle." So that should be taken care of first. Second, you might have idenetified a dependency... you're touching "Persistent" Data.
If you are doing the data access stuff in the method-in-question, you need to refactor into an n-tier'd or n-layer'd architecture. But we can assume that when you say "The strings are then inserted into a database", you actually mean you're calling a business logic layer or something. Ya, we'll assume that.
When your object is instantiated, you now understand that your object has dependencies. This is when you need to decide if you are going to do Dependency Injection on the Object, or on the Method. That means your Constructor or the method-in-question needs a new Parameter:
public <Constructor/MethodName> (IBusinessDataEtc otherLayerOrTierObject, string[] stuffToInsert)
Now that you can accept an interface of your business/data tier object, you can mock it out during Unit Tests and have no dependencies or fear of "Accidental" integration testing.
So in your live code, you pass in a REAL IBusinessDataEtc object. But in your Unit Testing, you pass in a MOCK IBusinessDataEtc object. In that Mock, you can include Non-Interface Properties like int XMethodWasCalledCount or something whose state(s) are updated when the interface methods are called.
So your Unit Test will go through your Method(s)-In-Question, perform whatever logic they have, and call one or two, or a selected set of methods in your IBusinessDataEtc object. When you do your Assertions at the end of your Unit Test you have a couple of things to test now.
The State of the "Subroutine" which is now a Try-Paradigm method.
The State of your Mock IBusinessDataEtc object.
For more information on Dependency Injection ideas on the Construction-level... as they pertain to Unit Testing... look into Builder design patterns. It adds one more interface and class for each current interface/class you have, but they are very tiny and provide HUGE functionality increases for better Unit-Testing.
You can even try it this way:
[TestMethod]
public void ReadFiles()
{
try
{
Read();
return; // indicates success
}
catch (Exception ex)
{
Assert.Fail(ex.Message);
}
}
it will have some effect on an object.... query for the result of the effect. If it has no visible effect its not worth unit testing!
Presumably the method does something, and doesn't simply return?
Assuming this is the case, then:
If it modifies the state of it's owner object, then you should test that the state changed correctly.
If it takes in some object as a parameter and modifies that object, then your should test the object is correctly modified.
If it throws exceptions is certain cases, test that those exceptions are correctly thrown.
If its behaviour varies based on the state of its own object, or some other object, preset the state and test the method has the correct Ithrough one of the three test methods above).
If youy let us know what the method does, I could be more specific.
Use Rhino Mocks to set what calls, actions and exceptions might be expected. Assuming you can mock or stub out parts of your method. Hard to know without knowing some specifics here about the method, or even context.
Depends on what it's doing. If it has parameters, pass in mocks that you could ask later on if they have been called with the right set of parameters.
What ever instance you are using to call the void method , You can just use ,Verfiy
For Example:
In My case its _Log is the instance and LogMessage is the method to be tested:
try
{
this._log.Verify(x => x.LogMessage(Logger.WillisLogLevel.Info, Logger.WillisLogger.Usage, "Created the Student with name as"), "Failure");
}
Catch
{
Assert.IsFalse(ex is Moq.MockException);
}
Is the Verify throws an exception due to failure of the method the test would Fail ?
Given the following simple service class, in the GetCategories() method, should you test the fact that the categoryRepository.Query() method was called, or should you be setting up a test that keeps a list of categories and returns those?
I guess what I am saying is would mocking the categoryRepository and verifying that it's Query method was called once cover this test case?
public class CategoryService : ValidatingServiceBase, ICategoryService
{
private readonly IRepository<Category> categoryRepository;
private readonly IRepository<SubCategory> subCategoryRepository;
private readonly IValidationService validationService;
public CategoryService(
IRepository<Category> categoryRepository,
IRepository<SubCategory> subCategoryRepository,
IValidationService validationService)
: base(validationService)
{
this.categoryRepository = categoryRepository;
this.subCategoryRepository = subCategoryRepository;
this.validationService = validationService;
}
public IEnumerable<Category> GetCategories()
{
return categoryRepository.Query().ToList();
}
}
Sample Test
[Fact]
public void GetCategories_Should_CallRepositoryQuery()
{
var categoryRepo = new Mock<IRepository<Category>>();
var service = new CategoryService(categoryRepo.Object, null, null);
service.GetCategories();
categoryRepo.Verify(x => x.Query(), Times.Once());
}
It doesn't matter. In both cases (mock + behavior verification vs stub + assertion) you achieve exactly the same result and require exactly the same level of details about inner workings of class. Stick to whichever one you think is more suited in given scenario.
Unit test you posted is an example of behavior verification. You don't assert any values but instead check whether some method was called. This is especially useful when method call has no visible results (think about logging) or doesn't return any value (obviously). It of course has drawbacks, especially when you do such verification for methods that do return value, and don't check it (as is your case - we'll get to it).
The stubbing and asserting approach uses the collaborators to generate value. It doesn't check whether methods were called (at least not directly, yet such test is performed when you setup stub and that setup works), but instead relies on correct flow of stubbed value.
Let's go with simple example. Say you test a method of your class, PizzaFactory.GetPizza which looks like this:
public Pizza GetPizza()
{
var dough = doughFactory.GetDough();
var cheese = ingredientsFactory.GetCheese();
var pizza = oven.Bake(dough, cheese);
return pizza;
}
With behavior verification you'd check whether doughFactory.GetDough was called, then ingredientsFactory.GetCheese and finally oven.Bake. Had such calls indeed been made, you'd assume pizza was created. You don't check that your factory returns pizza, but assume it happens if all process' steps were completed. You can already see that drawback I mentioned earlier - I can call all the correct methods but return something else, say:
var dough = doughFactory.GetDough();
var cheese = ingredientsFactory.GetCheese();
var pizza = oven.Bake(dough, cheese);
return garbageBin.FindPizza();
Not the pizza you ordered? Notice that all the correct calls to collaborators happened just as we assumed they would.
With stub + assert approach it all looks similar except instead of verification you have stubbing. You use values generated by earlier collaborators to stub later collaborators (if somehow you get wrong dough or cheese, oven will not return pizza we wanted). The final value is what your method returns and this is what we assert:
doughFactoryStub.Setup(df => dg.GetDough).Return("thick");
ingredientsFactoryStub.Setup(if => if.GetCheese()).Return("double");
var expectedPizza = new Pizza { Name = "Margherita" };
ovenStub.Setup(o => o.Bake("thick", "double")).Return(expectedPizza);
var actualPizza = pizzaFactory.GetPizza();
Assert.That(actualPizza, Is.EqualTo(expectedPizza));
If any part of the process fails (say doughFactory returns normal dough) then the final value will be different and test will fail.
And once again, in my opinion in your example it doesn't matter which approach you use. In any normal environment both methods will verify the same thing and require same level of knowledge about your implementation. To be extra safe you might want to prefer to use the stub + assert approach in case somebody plants you a garbage bin1. But if such thing happens, unit tests are your last problem.
1 Note however that it might not be intentional (especially when complex methods are considered).
Yes, that would be the way.
mockCategoryRepository.Setup(r => r.Query()).Returns(categories)
var actualCategories = new CategoryService(mockCategoryRepository, mock..).GetCategories();
CollectionAssert.AreEquivalent(categories, actualCategories.ToList());
It would look something similar with Moq and NUnit.
What you've presented is a white-box test - an approach also possible in unit testing, but recommended only for simple methods.
In the answer presented by Sruti the service is tested in a black-box sense. The knowledge about the inner method is used only to prepare the test, but you don't verify if the method was called one time, 10 times, or wasn't called at all. Personally, I verify method calls only to verify that some external API that must be stubbed is used correctly (example: sending e-mails). Usually it is sufficient not to care about how a method works, as long as it's producing correct results.
With black-box tests the code and the tests are easier to maintain. With white-box tests, most changes of some internal structure during refactoring of a class usually must be followed by changing test code. In black-box approach you have more freedom to rearrange everything, and still be sure that interface's external behaviour hasn't changed.
I have a class thats sole purpose is to run a method on other classes from an Interface.
Testing the interface of the classes is no problem, but the runner doesn't actually DO anything and (as it stands) the only parameter passed into the constructor is kept private.
In my case, the classes are importing text files into a database.
internal DataImporter
{
private List<IFileImporter> _importers;
public DataImporter(List<IFileImporter> importers){
_importers = importers;
public bool RunImporters()
{
//foreach importer, call its "Run" method - each one then does whatever it needs to do
//however, this need not call a specific "Run" method on IFileImporter
//I have another app that uses IFileImporter to check for presence of a file first
//then allow user to choose to import or not.
}
It seems to me there is nothing to test here? I can't test the value of _importers, and I don't want to make it public JUST for the sake of testing. DataImporter is specific to this instance, so creating an interface seems to add no benefit.
I've re-used IFileImporters elsewhere, but this is the only "bulk" importer, others are called manually from a winforms app, still others are not in this project at all.
So, do I need to test this...what CAN I test about this?
In a nutshell, yes. I can think of a number of tests right off the top of my head.
Your test ensures that all importers are called by mocking out IFileImporters you pass in the constructor. At a minimum it asserts that what you pass in the constructor are actually used by the method.
A test to ensure that if any importer that raises an exception, your class behaves as you expect.
A test should assert the behaviour if the list is empty. (default to True return?)
A test should also assert the behaviour you expect if one or more importers fail. (Are you and-ing or or-ing your importer results for your RunImporters result?)
Does the method run every importer regardless of if a previous fails, or returns false on the first failure?
There should also be a test on your constructor or Assertion for if its provided a null list.
You can assert that the methods have been called. You can refer to these questions to see how that's done:
How to assert if a method was called within another method in RhinoMocks?
Rhino Mocks - How to assert a mocked method was called n-times?
You can also assert that no exception was encountered within the DataImporterclass.
If you have absolute control over the interface used it might be helpful to have each one of them return a boolean indicating whether the specific operation was successful or not.
Your method returns bool and that's where you start, for example:
What happens when there is no importers? Example: Run_ReturnsFalse_WhenThereIsNoImporters
What happens when one of them breaks (or throws)? Example: Run_ReturnsFalse_WhenAtLeastOneImporterFails
What is indicating success (returning true, I suppose)? Example: Run_ReturnsTrue_WhenAllImportersSucceed
You setup your mocks to simulate each of those scenarios and test agains that. When there's trully no observable effects (like returned value), you'll do it by verifying that calls were made on mocked objects.
I have been trying to get past this problem for a couple of days now to no avail. I am trying to test the functionality of AnObject.AnAction (preemptive apologies, I have to obfuscate class/method names.) My goal is to keep MyTestObj.do_work(AnObject) from actually doing anything when it is called. It has code that I do want to be called in a unit test environment.
Here is my Unit test code:
Mock< MyTestObj > myTestObj_mock = new Mock< MyTestObj > ();
myTestObj_mock.Setup( e => e.do_work( It.IsAny< AnObject > () ) );
...
AnObject tester = new AnObject();
tester.anAction( myTestObj_mock.Object );
within the method AnObject.AnAction( MyTestObj mto ):
...
mto.do_work( this );
...
It is at this point when ran that I get this error:
System.InvalidCastException : Unable to cast object of type
'myNamespace.AnObject' to type 'System.Delegate'.
MyTestObj.do_work is a virtual method.
I have absolutely no idea why this error is coming up. I've stepped through the code as NUnit ran it and I am certain that is the point in which it crashes. It will not go any further into the code.
I'm really stumped. Does anybody have any idea what is going on?
Thanks in advance!
***Updates***
I tried to create an instance of AnObject within AnObject.AnAction and pass it instead of 'this':
...
AnObject test = new AnObject();
mto.do_work( test );
...
This did not have any effect.
As your question is written you do not need to setup the myTestObj_mock.do_work().
Setup is used to say, when method X is called return value Y. Or alternativly that when method X is called, exception Z is returned. Ask yourself is that really what I need to do?
Here you aren't returning anything from the method:
mto.do_work( this );
Also, here:
myTestObj_mock.Setup( e => e.do_work( It.IsAny< AnObject > () ) );
You aren't defining any Return for your setup.
You should be able to supply a mock myTestObj_mock without defining the setup in this case. Just remove the line with the Setup and run the test again.
A guess:
If you defined the Setup because you wanted your test to make sure AnObject.AnAction really calls the do_work method, what you need to do is define a Verify method rather than a Setup, likewise:
mock.Verify(foo => foo.Execute("ping"));
or in your case something like
myTestObj_mock.Verify(m => m.do_work(It.IsAny< AnObject > ()), Times.AtLeastOnce());
Another guess:
If you defined the mock because you pass a "this" reference and expect do_work to modify some parameters of "this" for the test to pass then you shouldn't be mocking here. Remember that mocks aren't really instances of your objects, so the code that modify the this reference will never get called.
I was finally able to get to a compiler and play with the code and here is what I found. It is basically what Giles said above for his first part of the question.
Here is the code hint for Setup:
Specifies a setup on the mocked type for a call to a value returning method.
And, here is the actual code for Setup
public ISetup<T> Setup(Expression<Action<T>> expression)
So, Setup is actually setting up the code as an Action (a delegate, basically). When the test runner is run, it hits this point and usually expects to feed the delegate to the Returns method
public IReturnsResult<TMock> Returns(TResult value)
So, this is never done, and instead of Mock running through the whole expected path and negotiating all of the code out, then it returns mistyped values.
Sooo, this is basically where Giles explanation is correct. If you are testing that the do_work method is being called, then you want to use Verify (which truly uses your fake object as a mock). However, if not, then you have no need to even set this up as it does nothing. If that is the case, then maybe that piece of code should not be in that specific code block, but you would be best to evaluate that.
And, finally, the Setup method should only be used if you are truly using your object as a stub, simply using it to inject logic into your code.
Hopefully, that makes sense and helps you with your problem.
Also, a Callback might be of use here depending on what you are doing. But, I am not sure. They have many examples in the Moq quickstart
UPDATE FOR MY ATTEMPT AT RECREATING using Version 4.0.10827...and had no issues
[Test]
public void teststuff()
{
Mock<MyTestObj> myTestObj_mock = new Mock<MyTestObj>();
myTestObj_mock.Setup(e => e.do_work(It.IsAny<AnObject>()));
AnObject tester = new AnObject();
tester.anAction(myTestObj_mock.Object);
}
...
public class MyTestObj
{
public virtual void do_work(AnObject o)
{
}
}
public class AnObject
{
public void anAction(MyTestObj obj)
{
obj.do_work(new AnObject());
}
}
I have the below structure in a test of mine, intended to test that a certain log is being called with the right complex argument object, even when it throws an exception which is then wrapped and generally manipulated further.
The logThing has a method:
void AddEntry(LogEntry);
So I am using When..Do to make it throw an exception,
public void UnitTest()
{
// Arrange
ILogThing logThing = Substitute.For<ILogThing>()
SystemUnderTest system = new SystemUnderTest(logThing);
List<LogEntry> actualEntries = new List<LogEntry>();
LogEntry expectedEntry = GetSomeTestData();
logThing.When(
lt => lt.AddEntry(Arg.Do<LogEntry>(r => actualEntries.Add(r)))).Do(
call => { throw new InvalidOperationException("testMessage"); });
// Act
try
{
system.DoSomethingWhichLogs(someArgs)
}
catch(WrappedException ex)
{
// Assert
Assert.AreEqual(1, actualEntries.Count);
Assert.AreEqual(actualEntries[0], expectedEntry);
}
}
However, the expected call to Arg.Do() never happens with this setup.
I have put a breakpoint in the catch block, and used Visual Studio's immediate window to call RecievedCalls<>() on the logThing, and it does have a record of a single call to logThing with the right arguments - it's just that Arg.Do appears to only execute after the When..Do block has finished. Clearly that means that since I am throwing in the When..Do, it never reaches it.
I really didn't expect NSubstitute to order the calls in this way, is that expected behaviour?
If so, is there anything I can do to test the incoming argument like this, or should I just put my argument checking into the main When..Do block (which makes it harder to read)?
The system under test does various things to the exception which include wrapping it together with the logEntry, so it -is- useful for me to have all of these checks in one test - I did think about splitting it into two separate tests, but realized that if I did that, I couldn't easily pin down where the incorrect wrapped output was coming from (it could either be the part that originally generates the logEntry, or the part wrapping it) wheras with this pattern I can check to make sure the logThing is receiving what I expect it to. Still, if there's a better way to do that, I'm certainly open to suggestions.
The exact order of When..Do and Arg.Do isn't defined, and I wouldn't recommend counting on it as I imagine it could change between versions depending on implementation. (If you have a good reason to define it in a particular order please post your suggestion to the usergroup.)
If you just want to check logThing received the expected LogEntry, you can check the argument after the fact using Arg.Is():
logThing.Received().AddEntry(Arg.Is(expectedEntry));
If you need more complex comparison logic, you can do the same thing but use a method to check the argument:
logThing.Received().AddEntry(Arg.Is<LogEntry>(x => CheckLogEntry(x)));
CheckLogEntry can do whatever checks you require, and you can keep the When..Do throwing as before.
If you need to use the When..Do approach, you can can keep your existing approach but move the capture of the argument into the Do call to ensure the order of calls that you expect:
logThing.When(
lt => lt.AddEntry(Arg.Any<LogEntry>())).Do(
call =>
{
actualEntries.Add(call.Arg<LogEntry>());
throw new InvalidOperationException("testMessage");
});
In terms of suggestions for other ways of testing this, I'm not entirely clear on what you are trying to do here, but if you are having trouble isolating where the wrapped output is coming from maybe you could move this responsibility to another dependency? This would let you substitute for this dependency and control exactly where this happened from the perspective of this test.