I have this class (it's more pseudocode)
public class Roles
{
private ProcessData processData;
Roles(ProcessData pd)
{
processData = pd;
}
public string[] GetLoginsThatCanCallAction(string actionName)
{
return GetPeopleThatCanCallActionFromDb(actionName)
.Union(processData.ProcessOwner)
.Union(GetFromDb(Role.Administrators);
// there may be many more Union(xyz) calls here
.ToArray();
}
// I can refactor those methods to be mockable
private GetPeopleThatCanCallActionFromDb(...)
private GetFromDb(...)
}
Now my question is. Would you write one test for each Union call in GetLoginsThaatCanRunAction method?
Or is is just enough that I write one test and assert that method returns logins returned from all methods called inside GetLoginsThatCanCallAction.
I can see reasons to do it both ways. But maybe someone will convince me to on or the other solution.
Edit:
I think I wasn't clear with my question: I wanted to ask if you would write this test
var pd = new ProcessData()
pd.ProcessOwner = "Owner";
var r = new Roles(processData)
SetupSoThatCallsForPeopleThatCanCallActionWillReturn("Joe");
SetupSoThatCallForAdministratorsWillReturn("Administrator");
var logins = r.GetLoginsThatCanCallAction("some action");
Assert.That(logins, Contains("Owner");
Assert.That(logins, Contains("Joe");
Assert.That(logins, Contains("Administrator");
or would you split it into 3 separate tests with one Assert in each one?
Interesting topic, Your problem is that you are trying to write a test case after you have some code developed. I would have 1 test for each Union call. The reason being do you want to test that you get a value returned from all methods OR do you want to test that under different assumptions EACH method will return a login ?
For me its more important knowing that each method will return a login based on different use cases than a generic test that will return me pass / fail.
I hope that makes sense.
I would write one test for GetLoginsThatCanCallAction mocking the external objects. From your example, that would possibly mean mocking the Union calls. The reason being, when I am writing this code, I am not concerned with the logic being used in Union. (I have had cases where I haven't even written them yet).
If the union calls behavior can change, (IE it throws an exception), I would have a test for each of those. However, I would have my test-suite generating those test cases for me rather than trying to write them all by hand.
You are concerned that the GetLoginsThatCanCallAction behaves correctly. You also want to have control over what the Union calls return as well.
That being said, you would also want to have an automated test that executes the entire process that GetLoginsThatCanCallAction gets used in so that you are verifying the connections between the classes that you are mocking in the unit test. Barring that not being possible actually executing the process yourself manually.
One other note, if the class is hard to test. That is a code smell that your design is not as modular as it could be.
I would also avoid mocking internal methods of a class, if you need to do that to test the function. It is a sign that your class has another class hiding inside. Your class is doing more than one thing the S in SOLID
You should exercise only public API of unit under test. Your unit has single public method GetLoginsThatCanCallAction. It does not matter whether this method call other methods, or implemented as one large method. That's an implementation details. What really matters is whether this method correctly communicates with dependency and returns expected result:
// Arrange
Mock<IProcessData> processData = new Mock<IProcessData>();
processData.Setup(d => d.ProcessOwner).Returns(new[] { "Bob" });
var expected = new []{ "Bob", "Joe" };
// Act
var actual = roles.GetLoginsThatCanCallAction("Drink");
// Assert
processData.VerifyGet(d => d.ProcessOwner); // verify communication
CollectionAssert.AreEquivalent(expected, actual); // verify result
Related
If a public method only calls external methods and contains no special logic within, should I add unit test on it?
The method below only calls external methods and use return values to construct a student object.
public student GetStudent(int ID){
var n = GetName(ID);
var a = GetAddress(ID);
return new student{
name = n;
address = a;
}
}
If I have a separate set of unit tests on GetName() and GetAddress() method already, do I still need to have unit tests on GetStudent() method?
Since tests already ensure GetName() and GetAddress() will return valid result(e.g. not null), GetStudent() should be fine accordingly.
I find it is rather difficult to decide whether I should do test on a method or not. May you also recommend some external resources (e.g. website, book, blog) on this regards?
Thank you.
you need to specify GetStudent behavior in test. for example what should happen in case of unreal id? exeption/null this should be described in test. you need to check final objct: can/cant name be null, can/cant address be null. it will be good documentation for GetStudent function
The answer is "primarily opinion-based" but in common, you can test this method and check:
Method does not return null
All fields of student class are set
You should mock GetName and GetAddress method and check that both of them are called with correct argument
So, this example is very simple, yes. But, maybe in future, you'll have some logic in this method and you need to ensure that everything is OK.
It's my philosophy that I always unit test methods that are exposed, no matter how trivial they are. Reason is if someone else is going to be working on this code, and they decide they want to change things, there is no Quality Assurance that their new code will break the system, unless you have Unit Tests. Unit Tests are there to make to make sure the code is of good quality and to make sure that other developers don't inherently break the system.
Given the following simple service class, in the GetCategories() method, should you test the fact that the categoryRepository.Query() method was called, or should you be setting up a test that keeps a list of categories and returns those?
I guess what I am saying is would mocking the categoryRepository and verifying that it's Query method was called once cover this test case?
public class CategoryService : ValidatingServiceBase, ICategoryService
{
private readonly IRepository<Category> categoryRepository;
private readonly IRepository<SubCategory> subCategoryRepository;
private readonly IValidationService validationService;
public CategoryService(
IRepository<Category> categoryRepository,
IRepository<SubCategory> subCategoryRepository,
IValidationService validationService)
: base(validationService)
{
this.categoryRepository = categoryRepository;
this.subCategoryRepository = subCategoryRepository;
this.validationService = validationService;
}
public IEnumerable<Category> GetCategories()
{
return categoryRepository.Query().ToList();
}
}
Sample Test
[Fact]
public void GetCategories_Should_CallRepositoryQuery()
{
var categoryRepo = new Mock<IRepository<Category>>();
var service = new CategoryService(categoryRepo.Object, null, null);
service.GetCategories();
categoryRepo.Verify(x => x.Query(), Times.Once());
}
It doesn't matter. In both cases (mock + behavior verification vs stub + assertion) you achieve exactly the same result and require exactly the same level of details about inner workings of class. Stick to whichever one you think is more suited in given scenario.
Unit test you posted is an example of behavior verification. You don't assert any values but instead check whether some method was called. This is especially useful when method call has no visible results (think about logging) or doesn't return any value (obviously). It of course has drawbacks, especially when you do such verification for methods that do return value, and don't check it (as is your case - we'll get to it).
The stubbing and asserting approach uses the collaborators to generate value. It doesn't check whether methods were called (at least not directly, yet such test is performed when you setup stub and that setup works), but instead relies on correct flow of stubbed value.
Let's go with simple example. Say you test a method of your class, PizzaFactory.GetPizza which looks like this:
public Pizza GetPizza()
{
var dough = doughFactory.GetDough();
var cheese = ingredientsFactory.GetCheese();
var pizza = oven.Bake(dough, cheese);
return pizza;
}
With behavior verification you'd check whether doughFactory.GetDough was called, then ingredientsFactory.GetCheese and finally oven.Bake. Had such calls indeed been made, you'd assume pizza was created. You don't check that your factory returns pizza, but assume it happens if all process' steps were completed. You can already see that drawback I mentioned earlier - I can call all the correct methods but return something else, say:
var dough = doughFactory.GetDough();
var cheese = ingredientsFactory.GetCheese();
var pizza = oven.Bake(dough, cheese);
return garbageBin.FindPizza();
Not the pizza you ordered? Notice that all the correct calls to collaborators happened just as we assumed they would.
With stub + assert approach it all looks similar except instead of verification you have stubbing. You use values generated by earlier collaborators to stub later collaborators (if somehow you get wrong dough or cheese, oven will not return pizza we wanted). The final value is what your method returns and this is what we assert:
doughFactoryStub.Setup(df => dg.GetDough).Return("thick");
ingredientsFactoryStub.Setup(if => if.GetCheese()).Return("double");
var expectedPizza = new Pizza { Name = "Margherita" };
ovenStub.Setup(o => o.Bake("thick", "double")).Return(expectedPizza);
var actualPizza = pizzaFactory.GetPizza();
Assert.That(actualPizza, Is.EqualTo(expectedPizza));
If any part of the process fails (say doughFactory returns normal dough) then the final value will be different and test will fail.
And once again, in my opinion in your example it doesn't matter which approach you use. In any normal environment both methods will verify the same thing and require same level of knowledge about your implementation. To be extra safe you might want to prefer to use the stub + assert approach in case somebody plants you a garbage bin1. But if such thing happens, unit tests are your last problem.
1 Note however that it might not be intentional (especially when complex methods are considered).
Yes, that would be the way.
mockCategoryRepository.Setup(r => r.Query()).Returns(categories)
var actualCategories = new CategoryService(mockCategoryRepository, mock..).GetCategories();
CollectionAssert.AreEquivalent(categories, actualCategories.ToList());
It would look something similar with Moq and NUnit.
What you've presented is a white-box test - an approach also possible in unit testing, but recommended only for simple methods.
In the answer presented by Sruti the service is tested in a black-box sense. The knowledge about the inner method is used only to prepare the test, but you don't verify if the method was called one time, 10 times, or wasn't called at all. Personally, I verify method calls only to verify that some external API that must be stubbed is used correctly (example: sending e-mails). Usually it is sufficient not to care about how a method works, as long as it's producing correct results.
With black-box tests the code and the tests are easier to maintain. With white-box tests, most changes of some internal structure during refactoring of a class usually must be followed by changing test code. In black-box approach you have more freedom to rearrange everything, and still be sure that interface's external behaviour hasn't changed.
I have written a class using TDD containing a method (method under test) which takes a simple value object as a parameter (range).
Code:
The method under test looks like this:
public List<string> In(IRange range)
{
var result = new List<string>();
for (int i = range.From; i <= range.To; i++)
{
//...
}
return result;
}
Furthermore I have a unit test to verify my method under test:
[TestMethod]
public void In_SimpleNumbers_ReturnsNumbersAsList()
{
var range = CreateRange(1, 2);
var expected = new List<string>() { "1", "2" };
var result = fizzbuzz.In(range);
CollectionAssert.AreEqual(expected, result);
}
private IRange CreateRange(int from, int to)
{
return new Fakes.StubIRange()
{
FromGet = () => { return from; },
ToGet = () => { return to; }
};
}
Question:
I have read Roy Osherove's book on unit testing ("The Art of Unit Testing"). In there he says
"external dependencies (filesystem, time, memory etc.) should be
replaced by stubs"
What does he mean by external dependency? Is my value object (range) also an external dependency which should be faked? Should I fake all dependencies a class have?
Can someone give me an advice
TL;DR
Do the simplest thing possible that solves your problem.
The longer I have been using TDD, the more I appreciate the value of being pragmatic. Writing super-duper isolated unit tests is not a value unto itself. The tests are there to help you write high quality code that is easy to understand and solves the right problem.
Adding an interface for the range class is a good idea if you have the need to be able to switch to another concrete range implementation without having to modify the code which depends on it.
However, if you do not have that need adding an interface serves no real purpose, but it does add some complexity, which actually takes you further from the goal of writing easy to understand code which solves the problem.
Be careful not to think to much about what might change in the future. YAGNI is a good principle to follow. If you have been doing TDD you won't have any problems refactoring the code if an actual need occurs in the future, since you have solid tests to rely on.
In general terms I would not consider a proper Value Object to be a dependency. If it is complex enough that you feel uncomfortable letting other code use it when under test it sounds like it is actually more like a service.
A unit test should run in isolation (completely in memory) without having to touch any external systems, such as file system, database, web service, mail service, system clock, or anything that is either slow, hard to setup, or undeterministic (such as the ever changing system time).
To be able to do this, you should abstract away those external dependencies which allows you to mock them in your tests.
A unit test however, goes one step further. In a unit test you often only want to test the logic of a single method or a single class. You're not interested to verify how multiple components integrate, but you just want to verify that the logic of that single class is correct, and that it communicates correctly with other components.
To be able to do this, you need to fake those other components (the class's dependencies). So in general you should indeed fake all dependencies (that contain behavior) a class has.
So I have something along the lines of this
private List<ConcurrentQueue<int>> listOfQueues = new List<ConcurrentQueue<int>>()
public void InsertInt(int id, int value)
{
listOfQueues[id].Enqueue(value);
}
Is this something I shouldn't be unit testing?
If it is how do I test the InsertInt method without using any other methods?
is it ok to use the method to retrieve the data from the queue to test if the it was enter correctly?
Is this something I should be using mocks for?
You generally do not want to test private members. The point of unit testing is that from the outside, the class takes the specified inputs and (when requested) will give you the correct outputs. You don't care about the implementation of HOW it gives you those outputs, you just care that it gives you the correct outputs for external usage.
To show this, say you create a unit test verifies that your InsertInt() method inserts the int to listOfQueues. Then your requirements change, and you have to change your implementation strategy, and instead of using a List<ConcurrentQueue<int>> it becomes a Dictionary<string, ConcurrentQueue<int>>. This may not actually require a change to your inputs, and your outputs can still pass any output verification, but your InsertInt() unit test will fail because it's hard coded to the implementation.
The better idea is to do unit tests that makes sure that if you call InsertInt() with the correct input, that your output methods will still return the corrected output, as well as creating unit tests that calling InsertInt() with invalid parameters causes exceptions. You can then change everything about your internal implementation and be confident that your class is still working correctly. Unit testing the implementation adds extra over-head while providing very little benefit in testability.
Note that I am NOT saying that this method should not be unit tested, it's just that the unit tests need to be developed in a way that reflects how outside objects will interact with your class.
Yes, you should unit test it. You can use a private accessor to get the listOfQueues.
You must make sure with a unit test that the method behaves as expected with exceptions and that the item really is inserted.
Check out this article on how to unit test private methods http://msdn.microsoft.com/en-us/library/ms184807(v=vs.80).aspx
You should not be testing the behaviour of the queues - this is an implementation detail that you could change without changing the behaviour of the method. for example, you could replace the ConcurrentQueue with another data structure, perhaps a tree, without needing to update the unit tests.
What you are testing is that this method accepts the inputs and stores the value as you expect. Therefore you will need some way of interrogating the state of the system such as
public int GetInt(int id)
Then you test that the method inserts as you expect by retrieving them using the same id.
You should test that the public method returns the results you expect in every case you can think of, and leave it to the method to store the values as it sees fit. Therefore I would probably test the method like this, with different inputs :
[TestCase(1,2,3)] // whatever test cases make sense for you
[TestCase(4,5,6)]
[TestCase(7,8,9)]
[Test]
public void Test_InsertInt_OK( int testId, int testValue, int expectedValue)
{
InsertInt(testId, testValue);
Assert.AreEqual( GetInt(testId), expectedValue )
}
As this method is a public method, it needs to be unit tested.
A quick look at your method would reveal that passing -1 as the value of id would cause an ArgumentOutOfRangeException. This would have been realized during coding if the unit test case for this method had been designed (assuming that many such methods exist).
To check whether the insertion is successful, you can use the method pointed out by #Oskar Kjellin.
If you want to get dirty, then you can use Reflection, to check whether the value has been inserted or not.
// Sample with Queue instead of ConcurrentQueue
private void TestInsertInt(int id, int value)
{
myInstance.InsertInt(id, value);
FieldInfo field = myInstance.GetType().GetField("listOfQueues", BindingFlags.NonPublic | BindingFlags.Instance);
List<Queue<int>> listOfQueues = field.GetValue(myInstance) as List<Queue<int>>;
int lastInserted = listOfQueues[id].Last();
Assert.AreEqual(lastInserted, value);
}
Is this something I shouldn't be unit
testing?
If you are targeting 100% test coverage then yes
If it is how do I test the InsertInt
method without using any other
methods?
Several options:
Use a different accessor method as
you suggest.
Make the container internal and give the unit test assembly access to internals
Factor the container out into a separate class which you pass as a dependency to this class and then in your unit test pass a mock container instead
Use reflection to access the private members
Another hack is to use the PrivateObject.
Personally, I would either use DI(Dependency Injection) or make the private, internal for the sake of Unit Testing!
I have a unit test called TestMakeAValidCall(). It tests my phone app making a valid call.
I am about to write another test called TestShowCallMessage() that needs to have a valid call made for the test. Is it bad form to just call TestMakeAValidCall() in that test?
For reference this is my TestMakeAValidCall() test.
[TestMethod]
public void TestMakeAValidCall()
{
//Arrange
phone.InCall = false;
phone.CurrentNumber = "";
// Stub the call to the database
data.Expect(x => x.GetWhiteListData()).
Return(FillTestObjects.GetSingleEntryWhiteList());
// Get some bogus data
string phoneNumber = FillTestObjects.GetSingleEntryWhiteList().
First().PhoneNumber;
// Stub th call to MakeCall() so that it looks as if a call was made.
phone.Expect(x => x.MakeCall(phoneNumber)).
WhenCalled(invocation =>
{
phone.CurrentNumber = phoneNumber;
phone.InCall = true;
});
//Act
// Select the phone number
deviceControlForm.SelectedNumber = phoneNumber;
// Press the call button to make a call.
deviceMediator.CallButtonPressed();
//Assert
Assert.IsTrue(phone.InCall);
Assert.IsTrue(phone.CurrentNumber == phoneNumber);
}
Refactor the setup to another method and call that method from both tests. Tests should not call other tests.
IMHO, you should do one of the following:
Create a method that returns a valid call, and use it separately for both tests (not one calling the other)
Mock the valid call for the ShowCallMessageTest
To offer a counter point:
I strongly believe that well designed unit test should depend on one another!
Of course, that makes sense only if the testing framework is aware of these dependencies such that it can stop running dependent test when a dependency fails. Even better, such a framework can pass the fixture from test to test, such that can build upon a growing and extending fixture instead of rebuilding it from scratch for each single test. Of course, caching is done to take care no side-effects are introduced when more than one test depends from the same example.
We implemented this idea in the JExample extension for JUnit. There is no C# port yet, though there are ports for Ruby and Smalltalk and ... the most recent release of PHPUnit picked up both our ideas: dependencies and fixture reuse.
PS: folks are also using it for Groovy.
I think its a bad idea. You want your unit test to test one thing and one thing only. Instead of creating a call through your other test, mock out a call and pass it in as an argument.
A unit test should test one unit/function of your code by definition. Having it call other unit tests makes it test more than one unit. I break it up in to individual tests.
Yes - unit tests should be separate and should aim to test only one thing (or at least a small number of closely-related things). As an aside, the calls to data.Expect and phone.Expect in your test method are creating expectations rather than stub calls, which can make your tests brittle if you refactor...
A unit vs. module....we also think tests should depend on reusable methods as well and should test at an api level testing integration of classes. Many just tests a single class but many bugs are at that integration between the class level. We also use verifydesign to guarantee the api does not depend on implementation. This allows you to refactor the whole component/module without touching a test(and we went through that once actually and it worked great). Of course, any architectural changes force you to refactor the tests but at least design changes in the module don't cause test refactor work(unless you change the behavior of the api of course implicitly like firing more events than you used to but that "would" be an api change anyways).
"Could someone ellaborate on how the refactoring would look like in this case? – Philip Bergström Nov 28 '15 at 15:33"
I am currently doing something like this and this is what i came up with:
Notice that ProcessorType and BuildProcessors both call TestLevels
the actual content besides that fact is unimportant
its using XUnit, and Shouldly NuGet package
private static void TestLevels(ArgProcessor incomingProcessor)
{
Action<ProcessorLevel, int> currentLevelIteration = null;
currentLevelIteration = (currentProcessor, currentLevel) =>
{
currentProcessor.CurrentLevel.ShouldBeEquivalentTo(currentLevel);
ProcessorLevel nextProcessor = currentProcessor.CurrentProcessor;
if (nextProcessor != null)
currentLevelIteration(nextProcessor, currentLevel + 1);
};
currentLevelIteration(incomingProcessor, 0);
}
[Theory]
[InlineData(typeof(Build), "Build")]
public void ProcessorType(Type ProcessorType, params string[] args)
{
ArgProcessor newCLI = new OriWeb_CLI.ArgProcessor(args);
IncomingArgumentsTests.TestLevels(newCLI);
newCLI.CurrentProcessor.ShouldBeOfType(ProcessorType);
}
[Theory]
[InlineData(typeof(Build.TypeScript), "TypeScript")]
[InlineData(typeof(Build.CSharp), "CSharp")]
public void BuildProcessors(Type ProcessorType, params string[] args)
{
List<string> newArgs = new List<string> {"Build"};
foreach(string arg in args) newArgs.Add(arg);
ArgProcessor newCLI = new OriWeb_CLI.ArgProcessor(newArgs.ToArray());
IncomingArgumentsTests.TestLevels(newCLI);
newCLI.CurrentProcessor.CurrentProcessor.ShouldBeOfType(ProcessorType);
}