We are using OpenCover for our solution test coverage and I noticed that
public async Task Build(ProcessorContext context)
{
context.ResultBetSlip.Bets.Clear();
// Here OpenCover tells me that I need to cover two branches
// so I assume I need to verify that GetAvailablePlugins can be empty and
// no Build method was called.
// How do I do that?
foreach (var plugin in _pluginFactory.GetAvailablePlugins())
{
await plugin.Build(context);
}
}
Now my test would look like that
[Test]
public async Task Build_ShouldntEnterForeachWhenThereAreNoPluginsRegistered()
{
// Arrange
var pluginFactoryMock = new Mock<IBetSlipProcessorServicePluginFactory>();
var sut = new BetSlipProcessorService(pluginFactoryMock.Object);
pluginFactoryMock
.Setup(pf => pf.GetAvailablePlugins())
.Returns(new List<IBetSlipProcessorServicePlugin>());
// Act
await sut.Build(AutoFixtureSimplified.Create<ProcessorContext>());
// Assert
???
}
Should I even consider testing such case if it is possible?
EDIT:
As requested this is the test coverage report:
And here you can find gist of all the tests that I do in order to achieve such coverage.
https://gist.github.com/kuskmen/df3f112b2b6d880741ee6ab559d64d53
I am assuming you are using the Moq framework for mocking. If this is the case you can do one of two things.
You can create your mock in strict mode
You can expect that when the plugin.Build(..) method is called that an exception is thrown.
A similar question was asked here: How to verify that method was NOT called in Moq?
edit:
I simulated the exact scenario that you are seeing and I have narrowed it down to the data type that you are iterating over. Due to the fact that you are using a list I would guess there is some internal workings of the list that are causing this problem. I changed all the list references to arrays and the branch coverage returned as expected.
Related
I am using the testing framework that comes with Visual Studio, along with NSubstitute to unit test a method that takes a system ID, and throws an exception if the system can't be found in the database...
public VRTSystem GetSystem(int systemID)
{
VRTSystem system = VrtSystemsRepository.GetVRTSystemByID(systemID);
if (system == null)
{
throw new Exception("System not found");
}
return system;
}
(In case this seems odd, there is a specific business case for this method that requires it to throw an exception, as returning a null system is not acceptable for its usage)
I want to write a test to check that an exception is thrown if the system doesn't exist. I currently have the following...
[TestMethod]
public void LicensingApplicationServiceBusinessLogic_GetSystem_SystemDoesntExist()
{
var bll = new LicensingApplicationServiceBusinessLogic();
try
{
VRTSystem systemReturned = bll.GetSystem(613);
Assert.Fail("Should have thrown an exception, but didn't.);
}
catch () { }
}
By not mocking the repository, the system returned by VrtSystemsRepository.GetVRTSystemByID() will be null, and the exception thrown. Whilst this works, it looks wrong to me. I wouldn't have expected to need a try/catch block in a test.
The NSubstitute docs have an example that implies I should be able to test this as follows...
[TestMethod]
public void GetSystem_SystemDoesntExist()
{
var bll = new LicensingApplicationServiceBusinessLogic();
Assert.Throws<Exception>(() => bll.GetSystem(613));
}
However, if I try this in my test code, I get Throws highlighted in red, with the error message "Assert does not contain definition for Throws"
Now, I'm not actually sure that the sample on that page covers my scenario, as the test code specifies that the method under test throws an exception, which I don't really understand, as I thought the idea of testing was to leave the method under test alone, and test what happens under various scenarios. However, even without that, I don't understand why the Assert.Throws method doesn't exist.
Anyone any ideas?
Edit: DavidG pointed out that Assert.Throws is probably part of NUnit, not the MS framework, which would explain why it's not recognised. If so, is the way I'm currently testing the right way to do it?
As mentioned by DavidG The referenced documentation is using NUnit for assertions.
If not using that framework you can use the ExpectedExceptionAttribute Class
[TestMethod]
[ExpectedException(typeof(<<Your expected exception here>>))]
public void GetSystem_SystemDoesntExist() {
var bll = new LicensingApplicationServiceBusinessLogic();
bll.GetSystem(613);
}
which would fail if the expected exception is not thrown.
So I've been trying to write a test for mass transit using the in-memory feature. I wondered what peoples approach was to waiting for consumers to execute. In the example below a use a sleep or I've also tried a while loop, but not a fan of either, any better ideas? I basically want to check that the consumer is executed.
[Fact]
public async Task SomeTest()
{
var busControl = Bus.Factory.CreateUsingInMemory(cfg =>
{
cfg.ReceiveEndpoint("commands", ec =>
{
ec.LoadFrom(context);
});
});
var address = new Uri(bus.Address, "commands")
await sendEndpoint.Send(MyExampleCommand());
Thread.Sleep(2000);
//Check nsubstitute mock received
}
Look at the test harness features that are built into MassTransit. They should give you some good ideas of how to test them.
You can look at the harness tests to see how they should be used. Note that they work with any test framework, not just NUnit.
https://github.com/MassTransit/MassTransit/blob/develop/tests/MassTransit.Tests/Testing/ConsumerTest_Specs.cs
The Testing documentation explains how to use the test harnesses.
I'm trying to figure out how to have tests fail as 'Inconclusive' if some required setup that also exists as a test fails.
This is an overly simplified example of what I'm trying to do, but hopefully it illustrates the point:
[Test] public void TestSetter() {
Assert.That(() => myInstance.Property = "test", Throws.Nothing);
}
[Test] public void TestGetter() {
try { TestSetter(); } catch (AssertionException) {
Assert.Inconclusive("Unable to test, prereq failed");
}
Assert.That(myInstance.Property, Is.EqualTo("test"));
}
This doesn't seem to work though, if I force the TestSetter test to fail, both still show the 'Failed' result, instead of the TestGetter resulting in 'Inconclusive'. I stepped through the code, it's definitely hitting the Assert.Inconclusive call, but it seems the earlier AssertionException is still getting precedence.
Is there any way of getting this to correctly report 'Inconclusive'?
Using C#7, with NUnit 3.6.0
Try upgrading to NUnit Framework v3.6.1, and see if that runs as you expect.
There was a change in v3.6.0 relating to catching AssertionExceptions, which was later reverted.
See https://github.com/nunit/nunit/issues/2043 for details. To summarise - you should be aware that catching AssertionException's isn't strictly a supported interface to write tests to, and it may be worthwhile refactoring, to test for setup success in a different way.
NUnit 2.6.4.
I have a VS/C# project that introduces async methods. It has many tests like this that pass:
[Test]
public async void NullProcThrows_Async()
{
var keyList = new KeyList<int>();
Assert.Throws<ArgumentNullException>(async () => await keyList.LoadAsync((IDBProcedure)null, "ID", CancellationToken.None));
}
I have merged this into our trunk (no conflicts) and now this test fails. I am trying to figure out the difference.
When I trace the code in the trunk I see two exceptions thrown:
The first is the ArgumentNullException I am expecting.
The second is
NUnit.Framework.AssertionException saying Expected
<System.ArgumentNullException> But was: null
When I run the test on the branch version I only see the one exception and the test passes.
What could be different between the two projects?
There seems to be a few issues with the code provided, consider the following:
[Test, ExpectedException(typeof(ArgumentNullException)]
public async Task NullProcThrows_Async()
{
var keyList = new KeyList<int>();
await keyList.LoadAsync((IDBProcedure)null, "ID", CancellationToken.None);
Assert.Fail("This should never be executed as we expected the above to throw.");
}
According to the NUnit documentation, you should be using the ExpectedException attribute instead of Assert.Throws. So I added that attribute and then remove the Assert.Throws and instead add Assert.Fail. Also, I made the method Task returning and this prevented the async void. Finally, doing it this way prevents the async lambda.
I use SeleniumWebdriver C# to build automated tests.
I also use NUnit to mark test methods (so I can run them).
In each of the tests there are several verification and when the first verification fails then the test stops executing (an exception is thrown).
I want the test to continue executing so more errors could be found!
Please guys, give me a clue how to do it right.
I think about something like that:
Instead of throwing Exception in verification I'll add an error to a stack
When a test ends I check if my stack is not empty
If the stack is not empty I push to console all errors and fail the test
If the stack is empty then the test passed successfully.
Are these steps good? Is there a better way?
I think this should work for you.
class Asserts
{
private static StringBuilder _stack;
[SetUp]
public void SetUp()
{
_stack = new StringBuilder();
}
[TearDown]
public void TearDown()
{
if (_stack.Lenght != 0) Assert.Fail(_stack.ToString());
}
[Test]
public void Test()
{
AssertHelper(() => Assert.AreEqual(0, 0));
AssertHelper(() => Assert.IsNotNull(null));
AssertHelper(() => Assert.AreEqual(3, 4));
AssertHelper(() => Assert.AreEqual(1, 1));
}
private static void AssertHelper(Action assert)
{
try
{
assert();
}
catch (AssertionException e)
{
_stack.Append(e.Message);
}
}
}
But the good idea is to keep one verification per one test.
Generally you want your tests to only be verifying one thing at a time. If there's 2 steps to an action that require asserting, considering having tests like:
UserForm_FirstNameMissing_ThrowsException
UserForm_SecondNameMissing_ThrowsException
UserForm_AgeTooLarge_ThrowsException
instead of having a single test called something like:
UserForm_TestValidation
This means that - like you wanted - if one stage of validation fails, you can continue to test the rest. Your best bet for this may be a singular test with a dataset, however keeping the answer simple yet relevant, a test per failure state is good enough for what you want.
I solved my problem in the next way:
I've created a set of methods that are performed once prior to executing any of the tests ([TestFixtureSetUpAttribute] in NUnit);
In this set of methods I do the main actions results of which I'll check (in my case I buy a lottery ticket);
In each test ([Test] in NUnit) I check one assertion.
So now I have a main action that doesn't include any verification. After this actions is completed I check all needed assertions.