Most of my test methods first try two or three trivial operations which should raise an exception, and then begin the real work. In Java, I would write it like this:
#Test
public void TestSomething() {
try {
testedClass.testedMethod(null);
fail();
catch (IllegalArgumentException ex) {
// OK
}
// and now let's get to the point ...
// ...
}
I wanted to stick with this habit in C#, but it seems there is no way to force a test method to fail. I've been looking a round for a while, but with no luck. Have I missed something?
PS: I know the correct way of testing these situations is this:
[TestMethod]
[ExpectedException(ArgumentNullException)]
public void TestSomethingWithNull() {
testedClass.TestedMethod(null);
}
[TestMethod]
public void TestSomething() {
// now the non-trivial stuff...
}
...but, I don't like this. When I have, let's say, 6 test methods in my test class, and each of those tests should start with covering three trivial, one-line situations which should raise an exception, using this approach turns my 6 tests into 18. In a bigger application, this really pollutes my Test Explorer and makes the results more difficult to scan through.
And let's say I want to test a method, whose responsibility is to validate each property of a given instance of some class, and raise a ValidationException if any value is incorrect. That could be easily handled by one TestValidation() test, but with this approach, turns it into:
TestValidationWithNull();
TestValidationWithInvalidProperty1();
TestValidationWithInvalidProperty2();
TestValidationWithNullProperty2();
Imagine you have 20 properties... :)
Of course, if this is the only way to do it, I'll bite.
You can use Assert.Fail() or throw NotImplementedException (if method which you are testing is not implemented yet).
But for testing if code throws exception I suggest you to use ExpectedException attribute (if you are stick to MSTest) - test will fail, if exception will not be thrown.
You need
Assert.Fail("Optional Message");
or you can just throw an exception from inside the method
You should also check out the TestCase attribute and TestCaseSource in NUnit. These might greatly simplify your code & testing when you want to pass different parameters to a test.
The reason why it's advisable to have separate test methods for all the "trivial" stuff is because an Assert.Fail() early in the test method may hide later problems. As you say though, lots of trivial test methods gets unwieldy all too quickly.
Remember though that Asset.Equals (for NUnit at least) can compare arrays. So you can change your tests to something like the following and simulationously test many aspects of the item under test, yet have visibility of all results:
public void TestSomething()
{
var result1 = false;
try
{
testedClass.testedMethod(null);
result1 = true;
}
catch (IllegalArgumentException ex) { }
var result2 = SomeDetailedTest();
var expected = new Object[] { false, 42 };
var actual = new Object[] { result1, result2 };
Assert.AreEqual(expected, actual);
}
Related
Using TDD first time in my life today. I am using nUnit.
I have one method, where I can insert multiple different inputs and check if result works.
I read that multiple asserts in one test is not a problem, and I really don't want to write new test for each input.
Example with multiple asserts:
[TestFixture]
public class TestClass
{
public Program test;
[SetUp]
public void Init()
{
test = new Program();
}
[Test]
public void Parse_SimpleValues_Calculated()
{
Assert.AreEqual(25, test.ParseCalculationString("5*5"));
Assert.AreEqual(125, test.ParseCalculationString("5*5*5"));
Assert.AreEqual(10, test.ParseCalculationString("5+5"));
Assert.AreEqual(15, test.ParseCalculationString("5+5+5"));
Assert.AreEqual(50, test.ParseCalculationString("5*5+5*5"));
Assert.AreEqual(3, test.ParseCalculationString("5-1*2"));
Assert.AreEqual(7, test.ParseCalculationString("7+1-1"));
}
}
But when something fails it is very hard to read which assert failed, I mean if you have them a lot, you have to go through all and find the right assert.
Is there any elegant way to show what input did we set if assert fails, instead of result and expected result?
Thank you.
I mean if you have them a lot, so you have to go through all.
No you don't - you just look at the stack trace. If you're running the tests within an IDE, I find that that's easily good enough to work out which line failed.
That said, there is a (significantly) better way - parameterized tests with TestCaseAttribute. So for example:
[Test]
[TestCase("5*5", 25)]
[TestCase("5*5*5", 125)]
[TestCase("5+5", 10)]
// etc
public void Parse_SimpleValues_Calculated(string input, int expectedOutput)
{
Assert.AreEqual(expectedOutput, test.ParseCalculationString(input));
}
Now your unit test runner will show you each test case separately, and you'll be able to see which one fails. Additionally, it will run all of the tests, even if an early one fails - so you don't end up fixing one only to find that the next one failed unexpectedly.
There's also TestCaseSourceAttribute for cases where you want to specify a collection of inputs separately - e.g. to use across multiple tests.
I have a test method like this
[TestMethod]
public void TestMethod1()
{
var smthing= new doSmthing();
smthing.doSomefunction("Test Service","Test Operation");
// do something next
var 2ndsmthing = do2ndSmthing();
2ndsmthing.do2ndSomeThing("Test","Method")
}
believe me these two functions or calls needs to be under same test method how can i prevent the test from stopping if something goes wrong while calling first method? i.e while calling doSmthing() I heard that its a bad idea to use Try...Catch blocks in test method. How can i solve this?
Any idea really appreciated.
If you expect and want to test for an exception, you can use the ExpectedException attribute on your test method.
If you want to "swallow" the exception thrown by the first two lines, you would have to try/catch it. As with any exception handling, you should catch the most specific exception you can.
In 15 years, several of which were spent on TDD projects, I have never personally seen a test that swallows an exception so that some other code can run.
You should split them in to two tests. [/soapbox]
I have this class (it's more pseudocode)
public class Roles
{
private ProcessData processData;
Roles(ProcessData pd)
{
processData = pd;
}
public string[] GetLoginsThatCanCallAction(string actionName)
{
return GetPeopleThatCanCallActionFromDb(actionName)
.Union(processData.ProcessOwner)
.Union(GetFromDb(Role.Administrators);
// there may be many more Union(xyz) calls here
.ToArray();
}
// I can refactor those methods to be mockable
private GetPeopleThatCanCallActionFromDb(...)
private GetFromDb(...)
}
Now my question is. Would you write one test for each Union call in GetLoginsThaatCanRunAction method?
Or is is just enough that I write one test and assert that method returns logins returned from all methods called inside GetLoginsThatCanCallAction.
I can see reasons to do it both ways. But maybe someone will convince me to on or the other solution.
Edit:
I think I wasn't clear with my question: I wanted to ask if you would write this test
var pd = new ProcessData()
pd.ProcessOwner = "Owner";
var r = new Roles(processData)
SetupSoThatCallsForPeopleThatCanCallActionWillReturn("Joe");
SetupSoThatCallForAdministratorsWillReturn("Administrator");
var logins = r.GetLoginsThatCanCallAction("some action");
Assert.That(logins, Contains("Owner");
Assert.That(logins, Contains("Joe");
Assert.That(logins, Contains("Administrator");
or would you split it into 3 separate tests with one Assert in each one?
Interesting topic, Your problem is that you are trying to write a test case after you have some code developed. I would have 1 test for each Union call. The reason being do you want to test that you get a value returned from all methods OR do you want to test that under different assumptions EACH method will return a login ?
For me its more important knowing that each method will return a login based on different use cases than a generic test that will return me pass / fail.
I hope that makes sense.
I would write one test for GetLoginsThatCanCallAction mocking the external objects. From your example, that would possibly mean mocking the Union calls. The reason being, when I am writing this code, I am not concerned with the logic being used in Union. (I have had cases where I haven't even written them yet).
If the union calls behavior can change, (IE it throws an exception), I would have a test for each of those. However, I would have my test-suite generating those test cases for me rather than trying to write them all by hand.
You are concerned that the GetLoginsThatCanCallAction behaves correctly. You also want to have control over what the Union calls return as well.
That being said, you would also want to have an automated test that executes the entire process that GetLoginsThatCanCallAction gets used in so that you are verifying the connections between the classes that you are mocking in the unit test. Barring that not being possible actually executing the process yourself manually.
One other note, if the class is hard to test. That is a code smell that your design is not as modular as it could be.
I would also avoid mocking internal methods of a class, if you need to do that to test the function. It is a sign that your class has another class hiding inside. Your class is doing more than one thing the S in SOLID
You should exercise only public API of unit under test. Your unit has single public method GetLoginsThatCanCallAction. It does not matter whether this method call other methods, or implemented as one large method. That's an implementation details. What really matters is whether this method correctly communicates with dependency and returns expected result:
// Arrange
Mock<IProcessData> processData = new Mock<IProcessData>();
processData.Setup(d => d.ProcessOwner).Returns(new[] { "Bob" });
var expected = new []{ "Bob", "Joe" };
// Act
var actual = roles.GetLoginsThatCanCallAction("Drink");
// Assert
processData.VerifyGet(d => d.ProcessOwner); // verify communication
CollectionAssert.AreEquivalent(expected, actual); // verify result
So, i'm new to unit testing, and even more so to test first development. Is it valid for me to have just a single assert.isTrue statement in my unit test where I pass in my method and a valid parameter, and compare it to the known good answer?
Method
public static string RemoveDash(string myNumber)
{
string cleanNumber = myNumber.Replace("-","");
return cleanNumber;
}
Test
[TestMethod()]
public void TestRemoveDash()
{
Assert.IsTrue(RemoveDash("50-00-0")=="50000");
}
That's pretty valid if it tests the functionality of your method, which it very much seems to be doing.
Might consider using Equals here instead, but it doesn't really matter. Also, I know this is a test example, but always make sure to test cases where the input is not what is expected as well as whatever other valid forms it can come in (this can be in the same test method or a different one depending on your preference)
Testers sometimes read our tests so I attempt to make them as readble as possible. I would prefer to use the following, rather than the single Assert:
[TestMethod()]
public void TestRemoveDash()
{
string expected = "50000";
string actual = RemoveDash("50-00-0");
Assert.AreEqual(expected,actual);
}
The only comment is to use Assert.AreEqual instead of Assert.IsTrue:
Assert.IsAreEqual("50000", RemoveDash("50-00-0"));
The reason for that is that if the test fail the error message you get is more descriptive of what was meant to happen and what actually did happen. A message that says "Expected value <50000> but was actually <50-00-0>" is a lot better than "Expected value to be true, but was false."
As a rule of thumb, whenever you find yourself wanting to use Assert.IsTrue, go through Assert methods and see if there is a better method to test your expectation (e.g. Assert.IsInstanceOfType, Assert.IsNotNull, etc).
This seems perfectly valid - however, why not include a few other tests in that method, along the same lines but testing that e.g. RemoveDash("-") == "" and RemoveDash("-5") == "5" etc?
I hope this doesn't come across as a stupid question but its something I have been wondering about. I wish to write unit test a method which contains some logic to check that certain values are not null.
public void MyMethod(string value1, string value2)
{
if(value1 != null)
{
//do something (throw exception)
}
if(value2 != null)
{
//do something (throw exception)
}
//rest of method
}
I want to test this by passing null values into the method. My question is should I create a unit test for each argument or can I create one unit test which checks what happens if I set value1 to null and then checks what happens if I set value2 to null.
i.e.
[TestMethod]
public void TestMyMethodShouldThrowExceptionIfValue1IsNull()
{
//test
}
[TestMethod]
public void TestMyMethodShouldThrowExceptionIfValue2IsNull()
{
//test
}
or
[TestMethod]
public void TestMyMethodWithNullValues()
{
//pass null for value1
//check
//pass null for value2
//check
}
Or does it make any difference? I think I read somewhere that you should limit yourself to one assert per unit test. Is this correct?
Thanks in advance
Zaps
You should write a unit test for each test case (assertion) to avoid Assertion Roulette.
The "ideal" unit test test one thing only in order to pinpoint errors exactly.
In practice, this is not nearly as important as most TDD proponents state, because tests don't fail frequently, and finding out which assert failed takes almost no time compared with the rest of the work involved in investigating and fixing the problem.
Doing extra work when writing the tests to save yourself work when it fails (which may never happen) is a form of YAGNI.
If having multiple methods is no extra work beyond typing more method declarations, you should do it, but if it leads to duplicated setup code, I see absolutely nothing wrong with testing several conditions in one test method.
If you are doing two tests in the same test method, your tests are not doing "unit-test".
For instance, what if the test for the first null value fails ?
If both tests are in the same test method, the second test will probably not be executed ; which means the test on the second null value depends on the test on the first null value.
On the other hand, if you have two separate test methods, you can test each case in perfect isolation.
Judging from the code of your MyMethod method, there is no link between the two conditions ; which means there shouldn't probably be any dependancy between the tests for those two conditions.
So : you should use two distinct tests.
To extrapolate to a non-technical thought.
Say you have a car, and you want to test the color.
Tests might be:
Car is red.
Car is blue.
Car is painted.
Now it might make sense to have "Painted" and "blue", but they really are different things. And if you test red and blue, it will always fail - or the failure would not make sense from an isolation standpoint.
Always test ONE thing at a time as you suggest, and many things comprise the test suite.