Related
What is the best way to unit test a method that doesn't return anything? Specifically in c#.
What I am really trying to test is a method that takes a log file and parses it for specific strings. The strings are then inserted into a database. Nothing that hasn't been done before but being VERY new to TDD I am wondering if it is possible to test this or is it something that doesn't really get tested.
If a method doesn't return anything, it's either one of the following
imperative - You're either asking the object to do something to itself.. e.g change state (without expecting any confirmation.. its assumed that it will be done)
informational - just notifying someone that something happened (without expecting action or response) respectively.
Imperative methods - you can verify if the task was actually performed. Verify if state change actually took place. e.g.
void DeductFromBalance( dAmount )
can be tested by verifying if the balance post this message is indeed less than the initial value by dAmount
Informational methods - are rare as a member of the public interface of the object... hence not normally unit-tested. However if you must, You can verify if the handling to be done on a notification takes place. e.g.
void OnAccountDebit( dAmount ) // emails account holder with info
can be tested by verifying if the email is being sent
Post more details about your actual method and people will be able to answer better.
Update: Your method is doing 2 things. I'd actually split it into two methods that can now be independently tested.
string[] ExamineLogFileForX( string sFileName );
void InsertStringsIntoDatabase( string[] );
String[] can be easily verified by providing the first method with a dummy file and expected strings. The second one is slightly tricky.. you can either use a Mock (google or search stackoverflow on mocking frameworks) to mimic the DB or hit the actual DB and verify if the strings were inserted in the right location. Check this thread for some good books... I'd recomment Pragmatic Unit Testing if you're in a crunch.
In the code it would be used like
InsertStringsIntoDatabase( ExamineLogFileForX( "c:\OMG.log" ) );
Test its side-effects. This includes:
Does it throw any exceptions? (If it should, check that it does. If it shouldn't, try some corner cases which might if you're not careful - null arguments being the most obvious thing.)
Does it play nicely with its parameters? (If they're mutable, does it mutate them when it shouldn't and vice versa?)
Does it have the right effect on the state of the object/type you're calling it on?
Of course, there's a limit to how much you can test. You generally can't test with every possible input, for example. Test pragmatically - enough to give you confidence that your code is designed appropriately and implemented correctly, and enough to act as supplemental documentation for what a caller might expect.
As always: test what the method is supposed to do!
Should it change global state (uuh, code smell!) somewhere?
Should it call into an interface?
Should it throw an exception when called with the wrong parameters?
Should it throw no exception when called with the right parameters?
Should it ...?
Try this:
[TestMethod]
public void TestSomething()
{
try
{
YourMethodCall();
Assert.IsTrue(true);
}
catch {
Assert.IsTrue(false);
}
}
Void return types / Subroutines are old news. I haven't made a Void return type (Unless I was being extremely lazy) in like 8 years (From the time of this answer, so just a bit before this question was asked).
Instead of a method like:
public void SendEmailToCustomer()
Make a method that follows Microsoft's int.TryParse() paradigm:
public bool TrySendEmailToCustomer()
Maybe there isn't any information your method needs to return for usage in the long-run, but returning the state of the method after it performs its job is a huge use to the caller.
Also, bool isn't the only state type. There are a number of times when a previously-made Subroutine could actually return three or more different states (Good, Normal, Bad, etc). In those cases, you'd just use
public StateEnum TrySendEmailToCustomer()
However, while the Try-Paradigm somewhat answers this question on how to test a void return, there are other considerations too. For example, during/after a "TDD" cycle, you would be "Refactoring" and notice you are doing two things with your method... thus breaking the "Single Responsibility Principle." So that should be taken care of first. Second, you might have idenetified a dependency... you're touching "Persistent" Data.
If you are doing the data access stuff in the method-in-question, you need to refactor into an n-tier'd or n-layer'd architecture. But we can assume that when you say "The strings are then inserted into a database", you actually mean you're calling a business logic layer or something. Ya, we'll assume that.
When your object is instantiated, you now understand that your object has dependencies. This is when you need to decide if you are going to do Dependency Injection on the Object, or on the Method. That means your Constructor or the method-in-question needs a new Parameter:
public <Constructor/MethodName> (IBusinessDataEtc otherLayerOrTierObject, string[] stuffToInsert)
Now that you can accept an interface of your business/data tier object, you can mock it out during Unit Tests and have no dependencies or fear of "Accidental" integration testing.
So in your live code, you pass in a REAL IBusinessDataEtc object. But in your Unit Testing, you pass in a MOCK IBusinessDataEtc object. In that Mock, you can include Non-Interface Properties like int XMethodWasCalledCount or something whose state(s) are updated when the interface methods are called.
So your Unit Test will go through your Method(s)-In-Question, perform whatever logic they have, and call one or two, or a selected set of methods in your IBusinessDataEtc object. When you do your Assertions at the end of your Unit Test you have a couple of things to test now.
The State of the "Subroutine" which is now a Try-Paradigm method.
The State of your Mock IBusinessDataEtc object.
For more information on Dependency Injection ideas on the Construction-level... as they pertain to Unit Testing... look into Builder design patterns. It adds one more interface and class for each current interface/class you have, but they are very tiny and provide HUGE functionality increases for better Unit-Testing.
You can even try it this way:
[TestMethod]
public void ReadFiles()
{
try
{
Read();
return; // indicates success
}
catch (Exception ex)
{
Assert.Fail(ex.Message);
}
}
it will have some effect on an object.... query for the result of the effect. If it has no visible effect its not worth unit testing!
Presumably the method does something, and doesn't simply return?
Assuming this is the case, then:
If it modifies the state of it's owner object, then you should test that the state changed correctly.
If it takes in some object as a parameter and modifies that object, then your should test the object is correctly modified.
If it throws exceptions is certain cases, test that those exceptions are correctly thrown.
If its behaviour varies based on the state of its own object, or some other object, preset the state and test the method has the correct Ithrough one of the three test methods above).
If youy let us know what the method does, I could be more specific.
Use Rhino Mocks to set what calls, actions and exceptions might be expected. Assuming you can mock or stub out parts of your method. Hard to know without knowing some specifics here about the method, or even context.
Depends on what it's doing. If it has parameters, pass in mocks that you could ask later on if they have been called with the right set of parameters.
What ever instance you are using to call the void method , You can just use ,Verfiy
For Example:
In My case its _Log is the instance and LogMessage is the method to be tested:
try
{
this._log.Verify(x => x.LogMessage(Logger.WillisLogLevel.Info, Logger.WillisLogger.Usage, "Created the Student with name as"), "Failure");
}
Catch
{
Assert.IsFalse(ex is Moq.MockException);
}
Is the Verify throws an exception due to failure of the method the test would Fail ?
Given the following simple service class, in the GetCategories() method, should you test the fact that the categoryRepository.Query() method was called, or should you be setting up a test that keeps a list of categories and returns those?
I guess what I am saying is would mocking the categoryRepository and verifying that it's Query method was called once cover this test case?
public class CategoryService : ValidatingServiceBase, ICategoryService
{
private readonly IRepository<Category> categoryRepository;
private readonly IRepository<SubCategory> subCategoryRepository;
private readonly IValidationService validationService;
public CategoryService(
IRepository<Category> categoryRepository,
IRepository<SubCategory> subCategoryRepository,
IValidationService validationService)
: base(validationService)
{
this.categoryRepository = categoryRepository;
this.subCategoryRepository = subCategoryRepository;
this.validationService = validationService;
}
public IEnumerable<Category> GetCategories()
{
return categoryRepository.Query().ToList();
}
}
Sample Test
[Fact]
public void GetCategories_Should_CallRepositoryQuery()
{
var categoryRepo = new Mock<IRepository<Category>>();
var service = new CategoryService(categoryRepo.Object, null, null);
service.GetCategories();
categoryRepo.Verify(x => x.Query(), Times.Once());
}
It doesn't matter. In both cases (mock + behavior verification vs stub + assertion) you achieve exactly the same result and require exactly the same level of details about inner workings of class. Stick to whichever one you think is more suited in given scenario.
Unit test you posted is an example of behavior verification. You don't assert any values but instead check whether some method was called. This is especially useful when method call has no visible results (think about logging) or doesn't return any value (obviously). It of course has drawbacks, especially when you do such verification for methods that do return value, and don't check it (as is your case - we'll get to it).
The stubbing and asserting approach uses the collaborators to generate value. It doesn't check whether methods were called (at least not directly, yet such test is performed when you setup stub and that setup works), but instead relies on correct flow of stubbed value.
Let's go with simple example. Say you test a method of your class, PizzaFactory.GetPizza which looks like this:
public Pizza GetPizza()
{
var dough = doughFactory.GetDough();
var cheese = ingredientsFactory.GetCheese();
var pizza = oven.Bake(dough, cheese);
return pizza;
}
With behavior verification you'd check whether doughFactory.GetDough was called, then ingredientsFactory.GetCheese and finally oven.Bake. Had such calls indeed been made, you'd assume pizza was created. You don't check that your factory returns pizza, but assume it happens if all process' steps were completed. You can already see that drawback I mentioned earlier - I can call all the correct methods but return something else, say:
var dough = doughFactory.GetDough();
var cheese = ingredientsFactory.GetCheese();
var pizza = oven.Bake(dough, cheese);
return garbageBin.FindPizza();
Not the pizza you ordered? Notice that all the correct calls to collaborators happened just as we assumed they would.
With stub + assert approach it all looks similar except instead of verification you have stubbing. You use values generated by earlier collaborators to stub later collaborators (if somehow you get wrong dough or cheese, oven will not return pizza we wanted). The final value is what your method returns and this is what we assert:
doughFactoryStub.Setup(df => dg.GetDough).Return("thick");
ingredientsFactoryStub.Setup(if => if.GetCheese()).Return("double");
var expectedPizza = new Pizza { Name = "Margherita" };
ovenStub.Setup(o => o.Bake("thick", "double")).Return(expectedPizza);
var actualPizza = pizzaFactory.GetPizza();
Assert.That(actualPizza, Is.EqualTo(expectedPizza));
If any part of the process fails (say doughFactory returns normal dough) then the final value will be different and test will fail.
And once again, in my opinion in your example it doesn't matter which approach you use. In any normal environment both methods will verify the same thing and require same level of knowledge about your implementation. To be extra safe you might want to prefer to use the stub + assert approach in case somebody plants you a garbage bin1. But if such thing happens, unit tests are your last problem.
1 Note however that it might not be intentional (especially when complex methods are considered).
Yes, that would be the way.
mockCategoryRepository.Setup(r => r.Query()).Returns(categories)
var actualCategories = new CategoryService(mockCategoryRepository, mock..).GetCategories();
CollectionAssert.AreEquivalent(categories, actualCategories.ToList());
It would look something similar with Moq and NUnit.
What you've presented is a white-box test - an approach also possible in unit testing, but recommended only for simple methods.
In the answer presented by Sruti the service is tested in a black-box sense. The knowledge about the inner method is used only to prepare the test, but you don't verify if the method was called one time, 10 times, or wasn't called at all. Personally, I verify method calls only to verify that some external API that must be stubbed is used correctly (example: sending e-mails). Usually it is sufficient not to care about how a method works, as long as it's producing correct results.
With black-box tests the code and the tests are easier to maintain. With white-box tests, most changes of some internal structure during refactoring of a class usually must be followed by changing test code. In black-box approach you have more freedom to rearrange everything, and still be sure that interface's external behaviour hasn't changed.
I have this class (it's more pseudocode)
public class Roles
{
private ProcessData processData;
Roles(ProcessData pd)
{
processData = pd;
}
public string[] GetLoginsThatCanCallAction(string actionName)
{
return GetPeopleThatCanCallActionFromDb(actionName)
.Union(processData.ProcessOwner)
.Union(GetFromDb(Role.Administrators);
// there may be many more Union(xyz) calls here
.ToArray();
}
// I can refactor those methods to be mockable
private GetPeopleThatCanCallActionFromDb(...)
private GetFromDb(...)
}
Now my question is. Would you write one test for each Union call in GetLoginsThaatCanRunAction method?
Or is is just enough that I write one test and assert that method returns logins returned from all methods called inside GetLoginsThatCanCallAction.
I can see reasons to do it both ways. But maybe someone will convince me to on or the other solution.
Edit:
I think I wasn't clear with my question: I wanted to ask if you would write this test
var pd = new ProcessData()
pd.ProcessOwner = "Owner";
var r = new Roles(processData)
SetupSoThatCallsForPeopleThatCanCallActionWillReturn("Joe");
SetupSoThatCallForAdministratorsWillReturn("Administrator");
var logins = r.GetLoginsThatCanCallAction("some action");
Assert.That(logins, Contains("Owner");
Assert.That(logins, Contains("Joe");
Assert.That(logins, Contains("Administrator");
or would you split it into 3 separate tests with one Assert in each one?
Interesting topic, Your problem is that you are trying to write a test case after you have some code developed. I would have 1 test for each Union call. The reason being do you want to test that you get a value returned from all methods OR do you want to test that under different assumptions EACH method will return a login ?
For me its more important knowing that each method will return a login based on different use cases than a generic test that will return me pass / fail.
I hope that makes sense.
I would write one test for GetLoginsThatCanCallAction mocking the external objects. From your example, that would possibly mean mocking the Union calls. The reason being, when I am writing this code, I am not concerned with the logic being used in Union. (I have had cases where I haven't even written them yet).
If the union calls behavior can change, (IE it throws an exception), I would have a test for each of those. However, I would have my test-suite generating those test cases for me rather than trying to write them all by hand.
You are concerned that the GetLoginsThatCanCallAction behaves correctly. You also want to have control over what the Union calls return as well.
That being said, you would also want to have an automated test that executes the entire process that GetLoginsThatCanCallAction gets used in so that you are verifying the connections between the classes that you are mocking in the unit test. Barring that not being possible actually executing the process yourself manually.
One other note, if the class is hard to test. That is a code smell that your design is not as modular as it could be.
I would also avoid mocking internal methods of a class, if you need to do that to test the function. It is a sign that your class has another class hiding inside. Your class is doing more than one thing the S in SOLID
You should exercise only public API of unit under test. Your unit has single public method GetLoginsThatCanCallAction. It does not matter whether this method call other methods, or implemented as one large method. That's an implementation details. What really matters is whether this method correctly communicates with dependency and returns expected result:
// Arrange
Mock<IProcessData> processData = new Mock<IProcessData>();
processData.Setup(d => d.ProcessOwner).Returns(new[] { "Bob" });
var expected = new []{ "Bob", "Joe" };
// Act
var actual = roles.GetLoginsThatCanCallAction("Drink");
// Assert
processData.VerifyGet(d => d.ProcessOwner); // verify communication
CollectionAssert.AreEquivalent(expected, actual); // verify result
So I have something along the lines of this
private List<ConcurrentQueue<int>> listOfQueues = new List<ConcurrentQueue<int>>()
public void InsertInt(int id, int value)
{
listOfQueues[id].Enqueue(value);
}
Is this something I shouldn't be unit testing?
If it is how do I test the InsertInt method without using any other methods?
is it ok to use the method to retrieve the data from the queue to test if the it was enter correctly?
Is this something I should be using mocks for?
You generally do not want to test private members. The point of unit testing is that from the outside, the class takes the specified inputs and (when requested) will give you the correct outputs. You don't care about the implementation of HOW it gives you those outputs, you just care that it gives you the correct outputs for external usage.
To show this, say you create a unit test verifies that your InsertInt() method inserts the int to listOfQueues. Then your requirements change, and you have to change your implementation strategy, and instead of using a List<ConcurrentQueue<int>> it becomes a Dictionary<string, ConcurrentQueue<int>>. This may not actually require a change to your inputs, and your outputs can still pass any output verification, but your InsertInt() unit test will fail because it's hard coded to the implementation.
The better idea is to do unit tests that makes sure that if you call InsertInt() with the correct input, that your output methods will still return the corrected output, as well as creating unit tests that calling InsertInt() with invalid parameters causes exceptions. You can then change everything about your internal implementation and be confident that your class is still working correctly. Unit testing the implementation adds extra over-head while providing very little benefit in testability.
Note that I am NOT saying that this method should not be unit tested, it's just that the unit tests need to be developed in a way that reflects how outside objects will interact with your class.
Yes, you should unit test it. You can use a private accessor to get the listOfQueues.
You must make sure with a unit test that the method behaves as expected with exceptions and that the item really is inserted.
Check out this article on how to unit test private methods http://msdn.microsoft.com/en-us/library/ms184807(v=vs.80).aspx
You should not be testing the behaviour of the queues - this is an implementation detail that you could change without changing the behaviour of the method. for example, you could replace the ConcurrentQueue with another data structure, perhaps a tree, without needing to update the unit tests.
What you are testing is that this method accepts the inputs and stores the value as you expect. Therefore you will need some way of interrogating the state of the system such as
public int GetInt(int id)
Then you test that the method inserts as you expect by retrieving them using the same id.
You should test that the public method returns the results you expect in every case you can think of, and leave it to the method to store the values as it sees fit. Therefore I would probably test the method like this, with different inputs :
[TestCase(1,2,3)] // whatever test cases make sense for you
[TestCase(4,5,6)]
[TestCase(7,8,9)]
[Test]
public void Test_InsertInt_OK( int testId, int testValue, int expectedValue)
{
InsertInt(testId, testValue);
Assert.AreEqual( GetInt(testId), expectedValue )
}
As this method is a public method, it needs to be unit tested.
A quick look at your method would reveal that passing -1 as the value of id would cause an ArgumentOutOfRangeException. This would have been realized during coding if the unit test case for this method had been designed (assuming that many such methods exist).
To check whether the insertion is successful, you can use the method pointed out by #Oskar Kjellin.
If you want to get dirty, then you can use Reflection, to check whether the value has been inserted or not.
// Sample with Queue instead of ConcurrentQueue
private void TestInsertInt(int id, int value)
{
myInstance.InsertInt(id, value);
FieldInfo field = myInstance.GetType().GetField("listOfQueues", BindingFlags.NonPublic | BindingFlags.Instance);
List<Queue<int>> listOfQueues = field.GetValue(myInstance) as List<Queue<int>>;
int lastInserted = listOfQueues[id].Last();
Assert.AreEqual(lastInserted, value);
}
Is this something I shouldn't be unit
testing?
If you are targeting 100% test coverage then yes
If it is how do I test the InsertInt
method without using any other
methods?
Several options:
Use a different accessor method as
you suggest.
Make the container internal and give the unit test assembly access to internals
Factor the container out into a separate class which you pass as a dependency to this class and then in your unit test pass a mock container instead
Use reflection to access the private members
Another hack is to use the PrivateObject.
Personally, I would either use DI(Dependency Injection) or make the private, internal for the sake of Unit Testing!
I just started using mock objects (using Java's mockito) in my tests recently. Needless to say, they simplified the set-up part of the tests, and along with Dependency Injection, I would argue it made the code even more robust.
However, I have found myself tripping in testing against implementation rather than specification. I ended up setting up expectations that I would argue that it's not part of the tests. In more technical terms, I will be testing the interaction between SUT (the class under test) and its collaborators, and such dependency isn't part of contract or the interface of the class!
Consider that you have the following:
When dealing with XML node, suppose that you have a method, attributeWithDefault() that returns the attribute value of the node if it's available, otherwise it would return a default value!
I would setup the test like the following:
Element e = mock(Element.class);
when(e.getAttribute("attribute")).thenReturn("what");
when(e.getAttribute("other")).thenReturn(null);
assertEquals(attributeWithDefault(e, "attribute", "default"), "what");
assertEquals(attributeWithDefault(e, "other", "default"), "default");
Well, here not only did I test that attributeWithDefault() adheres to the specification, but I also tested the implementation, as I required it to use Element.getAttribute(), instead of Element.getAttributeNode().getValue() or Element.getAttributes().getNamedItem().getNodeValue(), etc.
I assume that I am going about it in the wrong way, so any tips on how I can improve my usage of mocks and best practices will be appreciated.
EDIT:
What's wrong with the test
I made the assumption above that the test is a bad style, here is my rationale.
The specification doesn't specify which method gets called. A client of the library shouldn't care of how attribute is retrieved for example, as long as it is done rightly. The implementor should have free reign to access any of the alternative approaches, in any way he sees fit (with respect to performance, consistency, etc). It's the specification of Element that ensures that all these approaches return identical values.
It doesn't make sense to re-factor Element into a single method interface with getElement() (Go is quite nice about this actually). For ease of use, a client of the method should be able to just to use the standard Element in the standard library. Having interfaces and new classes is just plain silly, IMHO, as it makes the client code ugly, and it's not worth it.
Assuming the spec stays as is and the test stays as is, a new developer may decide to refactor the code to use a different approach of using the state, and cause the test to fail! Well, a test failing when the actual implementation adheres to the specification is valid.
Having a collaborator expose state in multiple format is quite common. A specification and the test shouldn't depend on which particular approach is taken; only the implementation should!
This is a common issue in mock testing, and the general mantra to get away from this is:
Only mock types you own.
Here if you want to mock collaboration with an XML parser (not necessarily needed, honestly, as a small test XML should work just fine in a unit context) then that XML parser should be behind an interface or class that you own that will deal with the messy details of which method on the third party API you need to call. The main point is that it has a method that gets an attribute from an element. Mock that method. This separates implementation from design. The real implementation would have a real unit test that actually tests you get a successful element from a real object.
Mocks can be a nice way of saving boilerplate setup code (acting essentially as Stubs), but that isn't their core purpose in terms of driving design. Mocks are testing behavior (as opposed to state) and are not Stubs.
I should add that when you use Mocks as stubs, they look like your code. Any stub has to make assumptions about how you are going to call it that are tied to your implementation. That is normal. Where it is a problem is if that is driving your design in bad ways.
When designing unit tests you will always effectively test your implementation, and not some abstract specification. Or one can argue that you will test the "technical specification", which is the business specification extended with technical details. There is nothing wrong with this. Instead of testing that:
My method will return a value if defined or a default.
you are testing:
My method will return a value if defined or a default provided that the xml Element supplied will return this attribute when I call getAttribute(name).
The only solution I can see for you here (and I have to admit I'm not familiar with the library you're using) is to create a mock element that has all of the functionality included, that is, also have the ability to set the value of getAttributeNote().getValue() and getAttributes().getNamedItem().getNodeValue().
But, assuming they're all equivalent, it's fine to just test one. It's when it varies that you need to test all cases.
I don't find anything wrong with your use of the mocks. What you are testing is the attributeWithDefault() method and it's implementation, not whether Element is correct or not. So you mocked Element in order to reduce the amount of setup required. The test ensures that the implementation of attributeWithDefault() fits the specification, naturally there needs to be some specific implementation that can be run for the test.
You're effectively testing your mock object here.
If you want to test the attributeWithDefault() method, you must assert that e.getAttribute() gets called with the expected argument and forget about the return value. This return value only verifies the setup of your mock object.
(I don't know how this is exactly done with Java's mockito, I'm a pure C# guy...)
It depends on whether getting the attribute via calling getAttribute() is part of the specification, or if it is an implementation detail that might change.
If Element is an interface, than stating that you should use 'getAttribute' to get the attribute is probably part of the interface. So your test is fine.
If Element is a concrete class, but attributeWithDefault should not be aware of how you can get the attribute, than maybe there is a interface waiting to appear here.
public interface AttributeProvider {
// Might return null
public String getAttribute(String name);
}
public class Element implements AttributeProvider {
public String getAttribute(String name) {
return getAttributeHolder().doSomethingReallyTricky().toString();
}
}
public class Whatever {
public String attributeWithDefault(AttributeProvider p, String name, String default) {
String res = p.getAtribute(name);
if (res == null) {
return default;
}
}
}
You would then test attributeWithDefault against a Mock AttributeProvider instead of an Element.
Of course in this situation it would probably be an overkill, and your test is probably just fine even with an implementation (You will have to test it somewhere anyway ;) ). However this kind of decoupling might be usefull if the logic ever goes any more complicated, either in getAttribute or in attributeWithDefualt.
Hoping this helps.
It seems to me that there are 3 things you want to verify with this method:
It gets the attribute from the right place (Element.getAttribute())
If the attribute is not null, it is returned
If the attribute is null, the string "default" is returned
You're currently verifying #2 and #3, but not #1. With mockito, you could verify #1 by adding
verify(e.getAttribute("attribute"));
verify(e.getAttribute("other"));
Which ensures that the methods are actually getting called on your mock. Admittedly this is a little clunky in mockito. In easymock, you'd do something like:
expect(e.getAttribute("attribute")).andReturn("what");
expect(e.getAttribute("default")).andReturn(null);
It has the same effect, but I think makes your test a bit easier to read.
If you are using dependency injection then the collaborators should be part of the contract. You need to be able to inject all collaborators in through the constructor or a public property.
Bottom line: if you have a collaborator that you newing up instead of injecting then you probably need to refactor the code. This is a change of mindset necessary for testing/mocking/injecting.
This is a late answer, but it takes a different viewpoint from the other ones.
Basically, the OP is right in thinking the test with mocking is bad, for the reasons he stated in the question. Those saying that mocks are ok have not provided good reasons for it, IMO.
Here is a complete version of the test, in two versions: one with mocking (the BAD one) and another without (the GOOD one). (I took the liberty of using a different mocking library, but that doesn't change the point.)
import javax.xml.parsers.*;
import org.w3c.dom.*;
import org.junit.*;
import static org.junit.Assert.*;
import mockit.*;
public final class XmlTest
{
// The code under test, embedded here for convenience.
public static final class XmlReader
{
public String attributeWithDefault(
Element xmlElement, String attributeName, String defaultValue
) {
String attributeValue = xmlElement.getAttribute(attributeName);
return attributeValue == null || attributeValue.isEmpty() ?
defaultValue : attributeValue;
}
}
#Tested XmlReader xmlReader;
// This test is bad because:
// 1) it depends on HOW the method under test is implemented
// (specifically, that it calls Element#getAttribute and not some other method
// such as Element#getAttributeNode) - it's therefore refactoring-UNSAFE;
// 2) it depends on the use of a mocking API, always a complex beast which takes
// time to master;
// 3) use of mocking can easily end up in mock behavior that is not real, as
// actually occurred here (specifically, the test records Element#getAttribute
// as returning null, which it would never return according to its API
// documentation - instead, an empty string would be returned).
#Test
public void readAttributeWithDefault_BAD_version(#Mocked final Element e) {
new Expectations() {{
e.getAttribute("attribute"); result = "what";
// This is a bug in the test (and in the CUT), since Element#getAttribute
// never returns null for real.
e.getAttribute("other"); result = null;
}};
String actualValue = xmlReader.attributeWithDefault(e, "attribute", "default");
String defaultValue = xmlReader.attributeWithDefault(e, "other", "default");
assertEquals(actualValue, "what");
assertEquals(defaultValue, "default");
}
// This test is better because:
// 1) it does not depend on how the method under test is implemented, being
// refactoring-SAFE;
// 2) it does not require mastery of a mocking API and its inevitable intricacies;
// 3) it depends only on reusable test code which is fully under the control of the
// developer(s).
#Test
public void readAttributeWithDefault_GOOD_version() {
Element e = getXmlElementWithAttribute("what");
String actualValue = xmlReader.attributeWithDefault(e, "attribute", "default");
String defaultValue = xmlReader.attributeWithDefault(e, "other", "default");
assertEquals(actualValue, "what");
assertEquals(defaultValue, "default");
}
// Creates a suitable XML document, or reads one from an XML file/string;
// either way, in practice this code would be reused in several tests.
Element getXmlElementWithAttribute(String attributeValue) {
DocumentBuilder dom;
try { dom = DocumentBuilderFactory.newInstance().newDocumentBuilder(); }
catch (ParserConfigurationException e) { throw new RuntimeException(e); }
Element e = dom.newDocument().createElement("tag");
e.setAttribute("attribute", attributeValue);
return e;
}
}