C# How to simplify unit testing string parameters using AutoFixture - c#

I'm trying to create a simple way to test string parameters in unit tests, for most string parameters, I want to check the behaviour when the parameter is Null, Empty or consists only of whitespaces.
In most of the cases, I'm checking the parameters with string.IsNullOrWhiteSpace(), and throw an exception if it has one of those three values.
Now for unit testing, it seems I have to write, for every string parameter, three unit tests. One for null values, one for empty values and one for Whitespaces.
Imagine a method with 3 or 4 string parameters, then I need to write 9 or 12 unit tests...
Can anyone come up with a simple way to test this? Maybe using AutoFixture?

To avoid duplicating the same test multiple times you can write a parameterized test.
If you're using xUnit, you would write a so called theory. A theory means you're proving a principle, that is that a certain function behaves as expected when given different samples of the same kind of input data. For example:
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData(" ")]
public void Should_throw_argument_exception_when_input_string_is_invalid(string input)
{
Assert.Throws<ArgumentException>(() => SystemUnderTest.SomeFunction(input));
}
The xUnit runner will run this test multiple times, each time assigning the input parameter to one of the values specified in the [InlineData] attribute.
If the function being tested has more than one parameter, you might not care what values are passed to the remaining ones, as long as at least one of them is a string that's either null, empty or contains only whitespace.
In this case, you can combine parameterized tests with AutoFixture. AutoFixture is designed to give you general-purpose test data that is good enough to use in most scenarios when you don't necessarily care about what the exact values are.
In order to use it with xUnit theories, you'll need to add the AutoFixture.Xunit NuGet package (or AutoFixture.Xunit2 depending on which version you're using) to your test project and use the InlineAutoData attribute:
[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_first_input_string_is_invalid(
string input1,
string input2,
string input3)
{
Assert.Throws<ArgumentException>(() =>
SystemUnderTest.SomeFunction(input1, input2, input3));
}
Only the input1 parameter will have a specific value, while the rest will be assigned random strings by AutoFixture.
One thing to note here is that the values passed through the [InlineAutoData] attribute are assigned to the test parameters based on their position. Since we need to test the same behavior for all three parameters individually, we'll have to write three tests:
[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_second_input_string_is_invalid(
string input2,
string input1,
string input3)
{
Assert.Throws<ArgumentException>(() =>
SystemUnderTest.SomeFunction(input1, input2, input3));
}
[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_third_input_string_is_invalid(
string input3,
string input1,
string input2)
{
Assert.Throws<ArgumentException>(() =>
SystemUnderTest.SomeFunction(input1, input2, input3));
}
A word on property-based testing
I can't help but think that these kinds of test scenarios are a perfect match for property-based testing. Without going into too much detail, property-based testing is about proving a specific behavior (or "property") of a function by running it multiple times with generated input.
In other words:
Property-based tests make statements about the output of your code
based on the input, and these statements are verified for many
different possible inputs.
In your case, you could write a single test to verify that the function throws an ArgumentException whenever at least one of the arguments is either null, an empty string or a string that only contains whitespace.
In .NET, you can use a library called FsCheck to write and execute property-based tests. While the API is primarily designed to be used from F#, it can also be used from C#.
However, in this particular scenario, I think you're better off sticking with regular parameterized tests and AutoFixture to achieve the same goal. In fact, writing these tests with FsCheck and C# would end up being more verbose and wouldn't really buy you much in terms of robustness.
Update: AutoFixture.Idioms
As #RubenBartelink pointed out in the comments, there is another option. AutoFixture encapsulates common assertions in a small library called AutoFixture.Idioms. You can use it to centralize the expectations on how the string parameters are validated by a method and use them in your tests.
While I do have my reservations on this approach, I'll add it here as another possible solution for the sake of completeness:
[Theory, AutoData]
public void Should_throw_argument_exception_when_the_input_strings_are_invalid(
ValidatesTheStringArguments assertion)
{
var sut = typeof(SystemUnderTest).GetMethod("SomeMethod");
assertion.Verify(sut);
}
public class ValidatesTheStringArguments : GuardClauseAssertion
{
public ValidatesTheStringArguments(ISpecimenBuilder builder)
: base(
builder,
new CompositeBehaviorExpectation(
new NullReferenceBehaviorExpectation(),
new EmptyStringBehaviorExpectation(),
new WhitespaceOnlyStringBehaviorExpectation()))
{
}
}
public class EmptyStringBehaviorExpectation : IBehaviorExpectation
{
public void Verify(IGuardClauseCommand command)
{
if (!command.RequestedType.IsClass
&& !command.RequestedType.IsInterface)
{
return;
}
try
{
command.Execute(string.Empty);
}
catch (ArgumentException)
{
return;
}
catch (Exception e)
{
throw command.CreateException("empty", e);
}
throw command.CreateException("empty");
}
}
public class WhitespaceOnlyStringBehaviorExpectation : IBehaviorExpectation
{
public void Verify(IGuardClauseCommand command)
{
if (!command.RequestedType.IsClass
&& !command.RequestedType.IsInterface)
{
return;
}
try
{
command.Execute(" ");
}
catch (ArgumentException)
{
return;
}
catch (Exception e)
{
throw command.CreateException("whitespace", e);
}
throw command.CreateException("whitespace");
}
}
Based on the expectations expressed in NullReferenceBehaviorExpectation, EmptyStringBehaviorExpectation, and WhitespaceOnlyStringBehaviorExpectation AutoFixture will automatically attempt to invoke the method named "SomeMethod" with null, empty strings and whitespace respectively.
If the method doesn't throw the correct exception – as specified in the catch block inside the expectation classes – then AutoFixture will itself throw an exception explaining what happend. Here's an example:
An attempt was made to assign the value whitespace to the parameter
"p1" of the method "SomeMethod", and no Guard Clause prevented this.
Are you missing a Guard Clause?
You can also use AutoFixture.Idioms without parameterized tests by simply instantiating the objects yourself:
[Fact]
public void Should_throw_argument_exception_when_the_input_strings_are_invalid()
{
var assertion = new ValidatesTheStringArguments(new Fixture());
var sut = typeof(SystemUnderTest).GetMethod("SomeMethod");
assertion.Verify(sut);
}

Related

Fluent assertions: Assert one OR another value

Using fluent assertions, I would like to assert that a given string contains either one of two strings:
actual.Should().Contain("oneWay").Or().Should().Contain("anotherWay");
// eiter value should pass the assertion.
// for example: "you may do it oneWay." should pass, but
// "you may do it thisWay." should not pass
Only if neither of the values is contained, the assertion should fail. This does NOT work (not even compile) as there is no Or() operator.
This is how I do it now:
bool isVariant1 = actual.Contains(#"oneWay");
bool isVariant2 = actual.Contains(#"anotherWay");
bool anyVariant = (isVariant1 || isVariant2);
anyVariant.Should().BeTrue("because blahblah. Actual content was: " + actual);
This is verbose, and the "because" argument must get created manually to have a meaningful output.
Is there a way to do this in a more readable manner? A solution should also apply to other fluent assertion types, like Be(), HaveCount() etc...
I am using FluentAssertions version 2.2.0.0 on .NET 3.5, if that matters.
I would make it an extension to the string assertions. Something like this:
public static void BeAnyOf(this StringAssertions assertions, string[] expectations, string because, params string[] args) {
Execute.Assertion
.ForCondition(expectations.Any(assertions.Subject.Contains))
.BecauseOf(because, args)
.FailWith("Expected {context:string} to be any of {0}{reason}", expectations);
}
You could even fork the repository and provide me with a Pull Request to make it part of the next version.
Should not this work?
actual.Should().BeOneOf("oneWay", "anotherWay");
Worked for me using v3.1.229.
You could make it a little more readable by writing a simple string extension:
public static class StringExt
{
public static bool ContainsAnyOf(this string self, params string[] strings)
{
return strings.Any(self.Contains);
}
}
Then you could do this:
actual.ContainsAnyOf("oneWay", "anotherWay").Should().BeTrue("because of this reason");
Unfortunately this doesn't help with the "reason" part of the message, but I think it's a little better.

Can I get a string representation of a condition passed as a bool into a function?

I know the title is probably really hard to understand, it was hard to think of a proper title, but here's the essence of what I want to do.
Basically I want to have a method like this:
void Validate(bool validation)
{
if (!validation)
{
throw new Exception();
}
}
And then I want to call it like:
try
{
Validate(1 > 2);
}
catch (Exception e)
{
// This is where I would output the error to the user
}
I want to get the 1 > 2 part as a string without defining it as one elsewhere, or evaluating a string to a bool, or using predicates, or using external methods. Ideally this would be done via reflection. I'll also take suggestions on a better way to do what I want to do. Assume that the bool could be anything: 1 > 2, "cheese" != "ham", objectA == objectB, etc.
You can't. Well, perhaps you happen to can (in Python, one could hack something like this together, I suppose, although it wouldn't be pretty, wouldn't work reliably and would require having the source code at hand), but generally:
You don't have string repesentations of code at runtime.
Arguments (expressions) are evaluated before the function is called.
The evaluation yields nothing but a lone bool that doesn't remember the slightest bit about where it came from.
Before you're looking for some nasty nasty hack to emulate this, check if it isn't easier to add a string literal during compilation.
The closest you are going to be able to get is to use lambda expressions, which would look something to the effect of:
void Validate(Expression<Func<bool>> validation)
{
if (!Lambda.Compile(validation)())
{
string message = "..." //parse lambda expression here.
//see: http://msdn.microsoft.com/en-us/library/bb397951.aspx
throw new Exception(message);
}
}
try
{
Validate(() => 1 > 2);
}
catch (Exception e)
{
Console.Write(e.Message)// This is where I would output the error to the user
}
Though, honestly, I'm not sure it's worth the trouble, and you wouldn't want to use it in a tight loop, due to the dynamic compilation of the lambda expression (though you could possibly cache the result of the compilation if necessary)

Rhino Mocks : How to match array arguments in an expectation?

Again at the Rhino Mocks Noob Wall
mockUI.Expect( x => x.Update( new Frame[] {Frame.MakeIncompleteFrame(1, 5)} ) );
This is the exact argument that I need to match. Via trace statements, I have verified that is the actual output as well i.e. the code behaves as intended but the test disagrees. RhinoMocks responds with
TestBowlingScorer.TestGamePresenter.TestStart:
Rhino.Mocks.Exceptions.ExpectationViolationException : IScoreObserver.Update([Frame# 1, Score = 0 Rolls [ 5, PENDING, ]]); Expected #1, Actual #0.
A Frame object contains few properties but doesn't override Equals() yet (overridden ToString() seen above). Update receives an array of Frames;
How do I setup this expectation? I see an Is.Matching constraint.. not sure how to use it or rather am concerned with the verbose nature of it.
I have a helper NUnit style custom Assert
public static void AssertFramesAreEqual(Frame[] expectedFrames, Frame[] actualFrames)
{
// loop over both collections
// compare attributes
}
#Gishu,
Yeah, that's it. I just also learned about the Arg<> static class which should allow you to do something like this:
mockUI.Expect( x => x.Update(Arg<Frame[]>
.Matches(fs=>HelperPredicates.CheckFrames ( expected, fs)) ));
There is also the Arg<>.List configuration starting point which I have not explored yet but might be even better for what you want
Verified works.. don't know if this is THE RhinoMocks way
var expectedFrames = new Frame[] { Frame.MakeIncompleteFrame(1, 5) };
mockUI.Expect( x => x.Update(null) )
.IgnoreArguments()
.Constraints( Is.Matching<Frame[]>( frames => HelperPredicates.CheckFramesMatch(expectedFrames, frames) ) );
The helper predicate is just a function that returns a boolean value - True on an exact match else false.
public static bool CheckFramesMatch(Frame[] expectedFrames, Frame[] actualFrames)
{
// return false if array lengths differ
// loop over corresponding elements
// return false if any attribute differs
// return true
}

What's the best alternative to an out of control switch statement?

I have inherited a project that has some huge switch statement blocks, with some containing up to 20 cases. What is a good way to rewrite these?
Why would you want to rewrite them in a different structure? If you really have 20 cases that have to be handled individually, a switch/case is the way to go. A big chain of if/then logic would be horrible to maintain.
Polymorphism is another option if you are using an object-oriented language. Each subclass would implement it's own functionality in the method.
Polymorphism. But it may not be a trivial refactoring.
Some examples and refs:
Refactoring (Googe books)
Switch Statement code smell and Polymorphism
Refactoring switch-statements
As others have pointed out, it depends on the switch statement. However, in the past I have refactored switch statements by proceeding in the following way. Suppose we have a switch statement like this, with a lot of repeating code
switch(x){
case 1:
makeitso("foo");
globalLog += "foo";
case 2:
makeitso("bar");
globalLog += "bar";
case 3:
makeitso("baz");
globalLog += "baz";
...
default:
throw("input error");
}
the first thing to do is to recognize the parts that are common (in the real world, this will probably be a little more substantial)
makeitso([some string]);
globalLog += [some string];
and turn that into a function
function transformInput(somestring) {
makeitso(somestring);
globalLog += somestring;
}
then for the parts that change in each case, use a hash or an array;
var transformvalues = ["foo", "bar", "baz"];
from here we can do this:
var tvals = ["foo", "bar", "baz" ... ];
function transformInput(somestring) {
makeitso(somestring);
globalLog += somestring;
}
var tval = tvals[x];
if(tval!==undefined) {
transformInput(tval);
} else {
throw ("invalid input");
}
And with tvals factored out of the switch statement, it could even be provided externally to expand the number of cases you can handle. Or you could build it dynamically. In the real world, the switch statement will often have special cases, however. I leave that as an excercise for the reader.
Three suggestions (echoing some already given):
Maybe a switch isn't as bad as you think. It can be ugly if the case blocks are large. Shorten them by extracting the logic into methods.
In an OO language, polymorphism might be the answer, as many have pointed out.
In a functional language, like Javascript, write a function that returns the function you need to run for whatever input. That might use a switch statement itself, or it might use a lookup table.
You could always use a lookup table.
There's nothing wrong with having 20 cases in a switch statement. You can tidy the code by refactoring and, at the very least, move the case processing into methods/functions.
Depending on what the switch statement is evaluating, you may want to refactor it using the Strategy Pattern. Take a look at this post for an example of replacing a switch on enum values with separate classes to handle each function.
It also may be that using the switch with 20 cases may actually be the best course of action for it. As long as it's readable and each result clearly conveys what the action is there's no need to really refactor it.
In general, I think you should refactor only when you need to, such as when you want to add more features, but the current design isn't up for the task. You should then refactor without adding the new functionality, and only then add the new feature.
In other circumstances, don't bother with refactoring. Do it when you need to, otherwise there are probably more important things to do.
If you really really need to, then the Visitor Design Pattern is a common switch-case replacement, though you should note it does have drawbacks. (i.e., check out www.objectmentor.com/resources/articles/acv.pdf)
It depends what the switch statement is doing.
If it's matching characters or strings, say in a parser, and you don't have the same set of patterns repeated everywhere in the code, then a switch statement might be ok.
If it's matching (say) an integer against a list of allowed values, you can create a base class and a set of derived classes for each value. Then, whatever generates the integer data in the first place can create an instance of the derived class with all of the switch statement "answers" instead.
A third option is to create a data structure that maps patterns to actions (i.e., functions or objects with virtual methods). You can look up the switch value in this data strucutre, and execute the appropriate action.
if it's working without major bugs (by not major I mean they don't make you pull your hair out) why bother refactor it? Don't refactor everything.
If you want, you can change it to polymorphism, but this will be a shotgun surgery, you'll probably have to refactor a whole lot more than just this switch block.
Visit https://github.com/Pedram-Ahmadpour/Switch-Case
Client side
Create an instance of your Condition, then pass a condition to Condition object by Switch() function.
int sense = 2;
ConditionSense conditionSense = new ConditionSense();
conditionSense.Switch(sense);
Server side
Create a condition action. This interface defines how to execute a Condition.
public interface IAction
{
void Do();
}
Create list of cases you want. These classes must implement condition action and ICase; Keep them light.
public class CaseCry : IAction, ICase<int?>
{
public int? Key { get { return 2; } }
public void Do()
{
Sense.Cry cry = new Sense.Cry();
cry.Act();
}
}
ICase just holds a Key that it is used by Switch() function to navigate the cases.
public interface ICase<TCase>
{
TCase Key { get; }
}
Create a Condition class that it Inherites SwitchCase generic abstract class.
Add all cases witch you want to Cases property.
Define a Switch() function and navigate Cases property to find matches cases, then execute them as a condition action.
public class ConditionSense : SwitchCase<int?>
{
public ConditionSense()
{
Cases = new List<ICase<int?>>
{
new CaseSmile(),
new CaseCry()
};
DefaultCases = new List<ICase<int?>> {
new CaseNoSense()
};
}
public void Switch(int? key)
{
IEnumerable<IAction> matches = Cases.Where(p => p.Key.Equals(key))
.Select(p => p as IAction);
if (matches.Count() > 0)
foreach (IAction match in matches)
match.Do();
else
foreach (IAction defaultCase in DefaultCases)
defaultCase.Do();
}
}
Smile, Cry..., can be huge, don't worry about size of them; condition action and ICase keep them lazy load.
public class Sense
{
public class Smile
{
public void Act()
{
Console.WriteLine("I'm smiling :-)");
}
}
public class Cry
{
public void Act()
{
Console.WriteLine("I'm crying :-(");
}
}
public class NoSense
{
public void Act()
{
Console.WriteLine("I've no sense :-|");
}
}
}

NUnit: Running multiple assertions in a single test

I have been asked to write a testing application that needs to test a new stored procedure on multiple rows in a database, in essence I want to do something like this:
[Test]
public void TestSelect()
{
foreach(id in ids)
{
DataTable old = Database.call("old_stored_proc",id);
DataTable new_ = Database.call("new_stored_proc",id);
Assert.AreEqual(old.Rows[0]["column"],ne_.Rows[0]["column"]);
}
}
When I run this test, if 1 row doesn't match the other, the entire test fails; instead I would like to count how many times the assertion was passed and how many times it has failed. Is there a way to do this with NUnit?
I realize that NUnit might be overkill and this is a simple task without it...I just wanted to learn it. ;)
Seems like you are just Asserting the wrong thing. If you want to check all the values and then assert that there are no errors (or show the number of errors) then try this:
[Test]
public void TestSelect()
{
int errors = 0;
foreach(id in ids)
{
DataTable old = Database.call("old_stored_proc",id);
DataTable new_ = Database.call("new_stored_proc",id);
if (old.Rows[0]["column"] != new_.Rows[0]["column"])
{
errors++;
}
}
Assert.AreEqual(0, errors, "There were " + errors + " errors.");
}
1) If the id's are constant and not looked up at test run time, create a separate unit test fixture for each id. That way you will know which id's are actually failing. See here for a write up on the problems with data driven tests:
http://googletesting.blogspot.com/2008/09/tott-data-driven-traps.html
2) If you need to dynamically look up the id's making it impossible to create a fixture for each id, use akmad's suggestion with one change. Keep a list of id's where the values are not equal and add the list to the error message. It will be extremely difficult to diagnose a failing test that only states the number of errors, as you won't know what id's cause the errors.
3) I don't know how difficult it would be to do in NUnit, but in PyUnit, when we need to run tests on dynamically generated data, we dynamically create tests fixtures and attach them to the TestCase class so that we have a failed test for each piece of data that does not pass. Though I imagine this would be much more difficult without python's dynamic abilities.
I know that the question is specifically about NUnit, but interestingly enough, Gallio/MbUnit has a feature which allows to run and catch several assertions at once.
[Test]
public void MultipleTest()
{
Assert.Multiple(() =>
{
Assert.IsTrue(blabla);
Assert.AreEqual(pik, pok);
// etc.
}
}
The Assert.Multiple is catching all the failing assertions and is going to report them at the end of the test.
I would count the number of rows which do not match and then would write an assertion which will compare this number with 0 and would return the number of non matching strings in the message.
you could also use Assert.Greater for this.
P.S. In principal you should try to do one assertion per unit test. That's the gist of it.
Well you could declare a counter and then assert the value of the counter to determine pass/fail
Also, you could do the bulk of the work in the test setup, and then just create multiple tests.
I'm not clear as to why you need all the assert stmts in the same test.
Based on the objective you laid out, the entire test should fail if one row doesn't match another. Counting the number of times an assertion passes or fails gives you less information than a comparison of the outcome you expected with the outcome you actually got.
I recently had the same issue. I combined the idea of counting errors with Yann Trevin's mention of Assert.Multiple into an extension method for IEnumberable that lets me do things like:
[Test]
public void TestEvenNumbers()
{
int[] numbers = new int[] { 2, 4, 12, 22, 13, 42 };
numbers.AssertAll((num) => Assert.That((num % 2) == 0, "{0} is an odd number", num));
}
Which results in the NUnit output:
TestEvenNumbers:
5 of 6 tests passed; 0 inconclusive
FAILED: 13: 13 is an odd number
Expected: True
But was: False
Expected: 6
But was: 5
And the solution to the OP's problem would be:
[Test]
public void TestSelect()
{
ids.AssertAll(CheckStoredProcedures);
}
private void CheckStoredProcedures(Id id)
{
DataTable old = Database.call("old_stored_proc",id);
DataTable new_ = Database.call("new_stored_proc",id);
Assert.AreEqual(old.Rows[0]["column"], new_.Rows[0]["column"]);
}
Here is the extension method (note that I used "All" instead of "Multiple" for consistency with Linq terminology):
using System;
using System.Text;
using System.Collections.Generic;
using NUnit.Framework;
public static class NUnitExtensions
{
public static void AssertAll<T>(this IEnumerable<T> objects, Action<T> test)
{
int total = 0;
int passed = 0;
int failed = 0;
int inconclusive = 0;
var sb = new StringBuilder();
foreach (var obj in objects)
{
total++;
try
{
test(obj);
passed++;
}
catch (InconclusiveException assertion)
{
inconclusive++;
string message = string.Format("INCONCLUSIVE: {0}: {1}", obj.ToString(), assertion.Message);
Console.WriteLine(message);
sb.AppendLine(message);
}
catch (AssertionException assertion)
{
failed++;
string message = string.Format("FAILED: {0}: {1}", obj.ToString(), assertion.Message);
Console.WriteLine(message);
sb.AppendLine(message);
}
}
if (passed != total)
{
string details = sb.ToString();
string message = string.Format("{0} of {1} tests passed; {2} inconclusive\n{3}", passed, total, inconclusive, details);
if (failed == 0)
{
Assert.Inconclusive(message);
}
else
{
Assert.AreEqual(total, passed, message);
}
}
}
}
You can use [TestCase()] attribute if a simple hard coded list of IDs.
[Test]
[TestCase(1234)]
[TestCase(5678)]
[TestCase(7654)]
public void TestSelect(int id)
{
DataTable old = Database.call("old_stored_proc", id);
DataTable new_ = Database.call("new_stored_proc", id);
Assert.AreEqual(old.Rows[0]["column"], new_.Rows[0]["column"]);
}
This will generate three separate tests for each ID and whatever nunit test runner you use will display pass/fail counts.
If need to generate a dynamic list of IDs then recommend using [TestCaseSource()] attribute.

Categories

Resources