Develop Reference

Nunit: TestCaseSource does not expect to generate all test cases?

Hi everyone I have a problem with generate test cases for TestCaseSource. I wrote this code for tests:
using System;
using System.Collections.Generic;
using System.Linq;
using NUnit.Framework;
namespace HeapSort.Tests
{
[TestFixture]
public class Tests
{
[Test, TestCaseSource(typeof(TestsGenerator),"TestCases")]
public void IsEqualCollections(int[] received, int[] expected)
{
CollectionAssert.AreEqual(received, expected);
}
}
public class TestsGenerator
{
public static IEnumerable<TestCaseData> TestCases
{
get
{
for (var i = 0; i < 25; i++)
{
int[] t1 = GenerateCollection(i), t2 = t1.ToArray();
HeapSort.Sort(t1);
Array.Sort(t2);
yield return new TestCaseData(t1, t2);
}
}
}
private static int[] GenerateCollection(int seed)
{
var rnd = new Random(seed+DateTime.Now.Millisecond);
int size = rnd.Next(100, 10000);
int[] array = new int[size];
for (var i = 0; i < size; i++)
array[i] = rnd.Next(-100, 100);
return array;
// return Enumerable
// .Repeat(100, rnd.Next(100, 10000))
// .Select(i => rnd.Next(-100, 100))
// .ToArray();
}
}
}
Where is the problem? Rather than get 25 tests, I get from 1 to 8. And often at the starting point of testing it shows that the tests are 7/8 and in the end there is only one test case.
How can I solve this problem?
UPD1: What's interesting is when I run tests through the console I handle all 25 tests how do I achieve the same results through the GUI!?
P.S. sorry for my bad english.
Perhaps I should mention that I'm working under Ubuntu in Rider

DateTime.Now is normally not very accurate. Your loop is generating many identical tests because they are all starting from the same seed. Why are you using a seed rather than simply letting Random work on its own?
Different runners will handle identical test in different ways. If you indicate what runner you are using to execute your tests, I can edit this answer with more information. However, in general, you most certainly don't want to generate a bunch of tests with the same data. They don't do anything for you!

The reason is TestExplorer looks at the name of test case and groups ones with equal names. So you don't see all test cases results. As reason is TestExplorer you don't have the same problem running test using console. Look at this issue for details. According to the issue there are two solutions:
create class for your test case and override ToString() method
making it return unique values for each test case.
use TestCaseData class and its method SetName to set unique name for each test.

C# How to simplify unit testing string parameters using AutoFixture

I'm trying to create a simple way to test string parameters in unit tests, for most string parameters, I want to check the behaviour when the parameter is Null, Empty or consists only of whitespaces.
In most of the cases, I'm checking the parameters with string.IsNullOrWhiteSpace(), and throw an exception if it has one of those three values.
Now for unit testing, it seems I have to write, for every string parameter, three unit tests. One for null values, one for empty values and one for Whitespaces.
Imagine a method with 3 or 4 string parameters, then I need to write 9 or 12 unit tests...
Can anyone come up with a simple way to test this? Maybe using AutoFixture?

To avoid duplicating the same test multiple times you can write a parameterized test.
If you're using xUnit, you would write a so called theory. A theory means you're proving a principle, that is that a certain function behaves as expected when given different samples of the same kind of input data. For example:
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData(" ")]
public void Should_throw_argument_exception_when_input_string_is_invalid(string input)
{
Assert.Throws<ArgumentException>(() => SystemUnderTest.SomeFunction(input));
}
The xUnit runner will run this test multiple times, each time assigning the input parameter to one of the values specified in the [InlineData] attribute.
If the function being tested has more than one parameter, you might not care what values are passed to the remaining ones, as long as at least one of them is a string that's either null, empty or contains only whitespace.
In this case, you can combine parameterized tests with AutoFixture. AutoFixture is designed to give you general-purpose test data that is good enough to use in most scenarios when you don't necessarily care about what the exact values are.
In order to use it with xUnit theories, you'll need to add the AutoFixture.Xunit NuGet package (or AutoFixture.Xunit2 depending on which version you're using) to your test project and use the InlineAutoData attribute:
[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_first_input_string_is_invalid(
string input1,
string input2,
string input3)
{
Assert.Throws<ArgumentException>(() =>
SystemUnderTest.SomeFunction(input1, input2, input3));
}
Only the input1 parameter will have a specific value, while the rest will be assigned random strings by AutoFixture.
One thing to note here is that the values passed through the [InlineAutoData] attribute are assigned to the test parameters based on their position. Since we need to test the same behavior for all three parameters individually, we'll have to write three tests:
[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_second_input_string_is_invalid(
string input2,
string input1,
string input3)
{
Assert.Throws<ArgumentException>(() =>
SystemUnderTest.SomeFunction(input1, input2, input3));
}
[Theory]
[InlineAutoData(null)]
[InlineAutoData("")]
[InlineAutoData(" ")]
public void Should_throw_argument_exception_when_the_third_input_string_is_invalid(
string input3,
string input1,
string input2)
{
Assert.Throws<ArgumentException>(() =>
SystemUnderTest.SomeFunction(input1, input2, input3));
}
A word on property-based testing
I can't help but think that these kinds of test scenarios are a perfect match for property-based testing. Without going into too much detail, property-based testing is about proving a specific behavior (or "property") of a function by running it multiple times with generated input.
In other words:
Property-based tests make statements about the output of your code
based on the input, and these statements are verified for many
different possible inputs.
In your case, you could write a single test to verify that the function throws an ArgumentException whenever at least one of the arguments is either null, an empty string or a string that only contains whitespace.
In .NET, you can use a library called FsCheck to write and execute property-based tests. While the API is primarily designed to be used from F#, it can also be used from C#.
However, in this particular scenario, I think you're better off sticking with regular parameterized tests and AutoFixture to achieve the same goal. In fact, writing these tests with FsCheck and C# would end up being more verbose and wouldn't really buy you much in terms of robustness.
Update: AutoFixture.Idioms
As #RubenBartelink pointed out in the comments, there is another option. AutoFixture encapsulates common assertions in a small library called AutoFixture.Idioms. You can use it to centralize the expectations on how the string parameters are validated by a method and use them in your tests.
While I do have my reservations on this approach, I'll add it here as another possible solution for the sake of completeness:
[Theory, AutoData]
public void Should_throw_argument_exception_when_the_input_strings_are_invalid(
ValidatesTheStringArguments assertion)
{
var sut = typeof(SystemUnderTest).GetMethod("SomeMethod");
assertion.Verify(sut);
}
public class ValidatesTheStringArguments : GuardClauseAssertion
{
public ValidatesTheStringArguments(ISpecimenBuilder builder)
: base(
builder,
new CompositeBehaviorExpectation(
new NullReferenceBehaviorExpectation(),
new EmptyStringBehaviorExpectation(),
new WhitespaceOnlyStringBehaviorExpectation()))
{
}
}
public class EmptyStringBehaviorExpectation : IBehaviorExpectation
{
public void Verify(IGuardClauseCommand command)
{
if (!command.RequestedType.IsClass
&& !command.RequestedType.IsInterface)
{
return;
}
try
{
command.Execute(string.Empty);
}
catch (ArgumentException)
{
return;
}
catch (Exception e)
{
throw command.CreateException("empty", e);
}
throw command.CreateException("empty");
}
}
public class WhitespaceOnlyStringBehaviorExpectation : IBehaviorExpectation
{
public void Verify(IGuardClauseCommand command)
{
if (!command.RequestedType.IsClass
&& !command.RequestedType.IsInterface)
{
return;
}
try
{
command.Execute(" ");
}
catch (ArgumentException)
{
return;
}
catch (Exception e)
{
throw command.CreateException("whitespace", e);
}
throw command.CreateException("whitespace");
}
}
Based on the expectations expressed in NullReferenceBehaviorExpectation, EmptyStringBehaviorExpectation, and WhitespaceOnlyStringBehaviorExpectation AutoFixture will automatically attempt to invoke the method named "SomeMethod" with null, empty strings and whitespace respectively.
If the method doesn't throw the correct exception – as specified in the catch block inside the expectation classes – then AutoFixture will itself throw an exception explaining what happend. Here's an example:
An attempt was made to assign the value whitespace to the parameter
"p1" of the method "SomeMethod", and no Guard Clause prevented this.
Are you missing a Guard Clause?
You can also use AutoFixture.Idioms without parameterized tests by simply instantiating the objects yourself:
[Fact]
public void Should_throw_argument_exception_when_the_input_strings_are_invalid()
{
var assertion = new ValidatesTheStringArguments(new Fixture());
var sut = typeof(SystemUnderTest).GetMethod("SomeMethod");
assertion.Verify(sut);
}

Assert that value is equal to any of a collection of expected values

Does NUnit provide a constraint to find whether the actual value is the element of a given enumerable or array, in other words, that it is equal to any of multiple expected values? Something like:
Assert.That(actual, Is.EqualToAnyOf(new[] { 1, 2, 3 }))
That is, to point out, the actual is a single value. I expect the value to be either 1, 2, or 3. The assertion
Assert.That(actual, Contains.Element(expected))
checks logically the same, but it is the opposite intention: Here we have a collection of actual values and expect one value to be in it.
Furthermore, I found these but they all don't fit:
Assert.That(actual, Is.EqualTo(expected)) // only allows one value
Assert.That(actual, Is.InRange(start, end)) // only works for consecutive numbers
Assert.That(actual, Is.SubsetOf(expected)) // only works if actual is an enumerable
Assert.That(expected.Contains(actual)) // meaningless "expected: true but was: false" message

CollectionAssert should be what you need if I am not overlooking something. It is as simple as:
CollectionAssert.Contains(IEnumerable expected, object actual);
However, there seems to be several ways to achieve your goal, such as:
[Test]
public void CollectionContains()
{
var expected = new List<int> { 0, 1, 2, 3, 5 };
var actual = 5;
CollectionAssert.Contains(expected, actual);
Assert.That(expected, Contains.Item(actual));
}
Above assertions should basically assert the same and could be used interchangeably.
Edit:
Question was modified, stating that Assert.That(expected, Contains.Item(actual)); is not valid even though it logically tests the same thing.

There is a way to do this built in to NUnit, using the Or constraint:
Assert.That(actual, Is.EqualTo(1).Or.EqualTo(2).Or.EqualTo(3))
If your list is more dynamic, you can build your list of Ors like this:
var expected = new[] { 1, 2, 3 };
var constraints = Is.EqualTo(expected[0]);
for(var i = 1; i < expected.Length; i++)
constraints = constraints.Or.EqualTo(expected[i]);
Assert.That(actual, constraints);
That latter answer doesn't read as well in the fluid syntax, but does achieve the dynamic building of or constraints. You could probably wrap that in a custom constraint as patrick-quirk demonstrated in order to achieve a more readbale fluid syntax, but that's up to you.

I know this is an old question, bu I have a maybe better (and native) suggestion.
With NUnit 3.x (I'm on 3.10.1) you can use Is.AnyOf:
Assert.That(
actualValue,
Is.AnyOf(expectedValue1, expectedValue2, expectedValue3),
"My error message");

The only way I could see to accomplish this is by creating your own constraint. It's pretty straightforward to do though.
The constraint class itself:
public class OneOfValuesConstraint : EqualConstraint
{
readonly ICollection expected;
NUnitEqualityComparer comparer = new NUnitEqualityComparer();
public OneOfValuesConstraint(ICollection expected)
: base(expected)
{
this.expected = expected;
}
public override bool Matches(object actual)
{
// set the base class value so it appears in the error message
this.actual = actual;
Tolerance tolerance = Tolerance.Empty;
// Loop through the expected values and return true on first match
foreach (object value in expected)
if (comparer.AreEqual(value, actual, ref tolerance))
return true;
// No matches, return false
return false;
}
// Overridden for a cleaner error message (contributed by #chiccodoro)
public override void WriteMessageTo(MessageWriter writer)
{
writer.DisplayDifferences(this);
}
public override void WriteDescriptionTo(MessageWriter writer)
{
writer.Write("either of ");
writer.WriteExpectedValue(this.expected);
}
}
And to make it fluent, create a static method to wrap it (contributed by #chicodorro):
public static class IsEqual
{
public static OneOfValuesConstraint ToAny(ICollection expected)
{
return new OneOfValuesConstraint(expected);
}
}
Then to use it:
int[] expectedValues = new[] { 0, 1, 2 };
Assert.That(6, IsEqual.ToAny(expectedValues));
Fails with the message:
Expected: either of < 0, 1, 2 >
But was: 6

Unit testing in C# how to?

I am new in the world of unit testing in C#.
I have a piece of code in my Main.cs
public static string Generate(int length)
{
char[] chars = "$%##!*abcdefghijklmnopqrstuvwxyz1234567890?;:ABCDEFGHIJKLMNOPQRSTUVWXYZ^&".ToCharArray();
string password = string.Empty;
Random random = new Random();
for (int i = 0; i < length; i++)
{
int x = random.Next(1, chars.Length);
if (!password.Contains(chars.GetValue(x).ToString()))
password += chars.GetValue(x);
else
i--;
}
return password;
}
now i don't know how to test this code with unit tests can someone give me a example?
EDIT:
I have make a test code
[TestMethod]
[Timeout(1000)]
public void RenderingPasswordShouldHaveMaximumSize()
{
var amountOfCharacters = Int32.MaxValue;
var generator = new PasswordGenerator();
var target = generator.Generate(amountOfCharacters);
// TODO
Assert.Fail("This method should throw an exception if you try to create a password with too many characters");
}
But it gives me the following error: Message: Test 'RenderingPasswordShouldHaveMaximumSize' exceeded execution timeout period can someone help me with this needs to have a max size of 74!

the idea of unit testing is to put something in a small method of you and check if the result is ok.
Visual Studio there is a project template for that. and there are also other tools like NUnit oderXUnit for tests in C#
There is a great pluralsight course:
and a webcast of Uncle Bob in which he demonstrates test driven development http://cleancoders.com/codecast/clean-code-episode-6-part-1/show
See also "Verifying Code by using Unit Tests" at msdn

A simple example using NUnit. Here I'm testing that when I pass 0 as an argument, nothing is generated (perhaps you should throw an Exception?)
[TextFixture]
public class Tests {
[Test]
public void Test_Length0_ReturnsNothing() {
string result = Generate(0);
Assert.IsTrue(string.IsNullOrEmpty(result));
}
}
You can write then similar tests (eg. to make sure it include the characters you want, etc).

There are much nuances in unit testing. I would recommend you to read book about unit testing "The Art of Unit Testing: With Examples in .Net".
There are described a lot of techniques and approach in unit testing. You can also find many examples here.

var amountOfCharacters = Int32.MaxValue;
var generator = new PasswordGenerator();
var target = generator.Generate(amountOfCharacters);
You're specifying the number of characters the password should contain as 2,147,483,647 characters...
chars.Length
There's only 74 possible values in your array.
No doubt it times out because the loop takes longer to iterate 2.2 billion times trying to find the last few values in your array.
for (int i = 0; i < length; i++)
Your logic changes too since you're not specifying the length of the password, but the number of iterations you want to do.

Loop inside a unit test

Can we have a loop inside a unit test?
My method returns an IEnumerable<IEnumerable>, I would like to unit test this logic where the IEnumerable<IEnumerable> is created. Basically I wanna test if the count of elements in the IEnumerable are as expected.
I cannot figure out an alternate way to test the inner IEnumerable without having a looping statement. Please let me know if this is a good practice.

There is no technical reason you can't do it. You can have multiple Assert statements in a unit test. Having an Assert statement in a loop is simply a shorthand way of having multiple Assert statements in a test.
However, some people think there should only be a single Assert statement in a unit test.
I personally don't agree - I think a test should test a single thing - and in order to do that sometimes you may need more than one Assert statement.
If your method returns an IEnumerable of Product's, and each Product contains an IEnumerable of Color's, then I think the following test is is fine:
[Test]
public void All_products_should_have_some_colors()
{
var products = GetProducts();
foreach (var product in products)
{
Assert.IsNotEmpty(product.Colors);
}
}
However, you do need to be aware that if the IEnumerable contains 0 elements, the loop will never execute any of the Assert statements and your unit test will "pass" - whereas you probably would have intended it to fail.
To remedy that situation, you could have a separate test making sure that there are more than 0 elements in the IEnumerable (i.e. that GetProducts does actually return some Product's):
Assert.IsNotEmpty(products);

One reason to avoid writing a loop in a test would be to keep the test concise and readable at a glance. Since you have tagged the question with NUnit and you say you just want to test that the element count is as expected, consider making your Asserts using the NUnit Constraints.
For example,
IEnumerable<IEnumerable<char>> someStrings = new[] { "abc", "cat", "bit", "hat" };
Assert.That(someStrings, Has.All.With.Length.EqualTo(3).And.All.Contains("a"));
fails with the message:
Expected: all items property Length equal to 3 and all items String containing "a"
But was: < "abc", "cat", "bit", "hat" >
but passes if you change "bit" to "bat".
The book xUnit Test Patterns: Refactoring Test Code
By Gerard Meszaros
has many great answers to questions such as yours.

Yes you can have loops in unit test, but with caution. As mentioned by Alex York, loops are acceptable if you test one thing; i.e. one expectation.
If you use loops, then I recommend that you must do two things:
As mentioned above, test for a non-empty iteration set. Iterating over an empty set is a false positive. False positive results are bane of all automated testing because nobody double checks a green result.
Include a test description that describes the current iteration. At a minimum include the iteration index.
Here is an example from my testing of the Greater Than property of an object.
[Test]
public void TestCompare_XtoY_GreaterThan()
{
int numObjects = mOrderedList.Count;
for (int i = 1; i < numObjects; ++i)
{
for (int j = 0; j < i; ++j)
{
string testDescription = string.Format("{0} is greater than {1} which implies\n {2}\n is greater than\n {3}"
, i, j
, mOrderedList[i], mOrderedList[j]
);
Assert.IsTrue(0 < mOrderedList[i].CompareTo(mOrderedList[j]), testDescription);
Assert.IsTrue(0 < mOrderedList[i].Compare(mOrderedList[i], mOrderedList[j]), testDescription);
Assert.IsTrue(0 < mOrderedList[j].Compare(mOrderedList[i], mOrderedList[j]), testDescription);
Assert.Greater(mOrderedList[i], mOrderedList[j], testDescription);
}
}
}
I test that my ordered list is non-empty in the test setup using:
[SetUp]
public void GeneralTestSetup()
{
// verify the iterated sources are not empty
string testDescription = string.Format("The ordered list of objects must have at least 3 elements, but instead has only {0}.", mOrderedList.Count);
Assert.IsTrue(2 < mOrderedList.Count, testDescription);
}
I have multiple Asserts even within my loop, but all of the asserts are testing the single expectation:
if i > j then mOrderedList[i] > mOrderedList[j]
The test description at the iteration level is so you get Failures such as:
10 is greater than 9 which implies
TestActionMethodInfo: [Actions.File, Version=1.0.446.0, File, VerifyReadOnly]
is greater than
TestActionMethodInfo: [Actions.File, Version=1.0.446.0, File, Write]
Expected: True
But was: False
instead of just:
Expected: True
But was: False
The question/debate about my code above:
Is am I testing one thing?
I am asserting on 4 different comparison methods within the object which could be argued as testing 4 things not one. The counter is greater than is greater than is greater than and that all of the methods which make that evaluation should be consistent.