Develop Reference

Error `Async test method must have non-void return type` when upgrading from NUnit 2 to NUnit 3

I have to refactor am unit test from NUNIT 2 to NUNIT 3 and the following syntax throws an error:
var expectedResponseMessage = new HttpResponseMessage();
Func<Task<HttpResponseMessage>> continuation =
() => Task.Factory.StartNew(() => expectedResponseMessage);
Error:
Async test method must have non-void return type
How may I rewrite this? I have tried many syntaxes but no luck.

The error
Async test method must have non-void return type
means that in NUnit 3+, an async Unit Test itself may not have a void return type (i.e. the method decorated with [Test] / [TestCase] etc). Instead, you can return an empty Task (Correct way in newer versions of NUnit with async test support):
[Test]
public async Task EnsureFoo()
{
// Arrange
// Act
var myResult = await classBeingTested.DoSomethingAsync();
// Assert
Assert.IsNotNull(myResult);
...
}
In NUnit 2.x, this wasn't checked, so a async void unit test could slip into your unit test code base, i.e. of the form (Bad, don't do this)
[Test]
public async void Foo() // < -- Error : Async test method must have non-void return type
{
var myResult = await classBeingTested.DoSomethingAsync();
// Post continuation assertions here.
}
this is rather dangerous - the test can't be awaited*, and would return before any continuations completed - e.g. any failures in the Asserts done in the continuation might not be reported.
Re : Your fake Task
Scheduling a Task just to return a fake response seems overkill, i.e. in most tests you should be able to use Task.FromResult to replace:
Func<Task<HttpResponseMessage>> continuation =
() => Task.Factory.StartNew(() => expectedResponseMessage);
With the cheaper:
Func<Task<HttpResponseMessage>> continuation =
() => Task.FromResult(expectedResponseMessage);
Task.FromResult returns an already completed task with the given return value - in most cases, this should suffice for your unit testing purposes, unless you really do want an independent Task to be executed on the threadpool.
* Actually, seemingly even earlier versions such as NUnit 2.6.4 had already identified the issue with async void tests, and incorporated a workaround

Unit testing break in foreach loop

I'm trying to unit test the cancel execution scenario in a class conceptually similar to the following:
public class ContextExecutor
{
public ContextExecutor(IContextRunner runner, IExecutionCanceler canceler)
{
this.runner = runner;
this.canceler = canceler;
}
public void Execute(IEnumerable<IContext> contexts)
{
foreach (var ctx in contexts)
{
if (canceler.IsCanceled)
{
break;
}
runner.Run(ctx);
}
}
readonly IContextRunner runner;
readonly IExecutionCanceler canceler;
}
public interface IContextRunner
{
void Run(IContext context);
}
public interface IExecutionCanceler
{
bool IsCanceled { get; }
}
The test case I was after should go through the following steps:
start ContextExecutor.Execute() asynchronously somehow;
put that method execution on hold until something unlocks it from unit test code;
unlock execution and let it perform 1 (..or 2, or..) loop runs, anyway less than full enumerable length;
invoke canceling by setting canceler.IsCanceled = true;
unlock loop execution free;
wait synchronously for method completion;
assert that loop has been invoked the expected nr of times.
I got tangled up with controlling loop execution locking/unlocking from unit test code. Apart from starting Execute() in a new thread, I avoided using threading synchronization primitives (e.g. semaphores and locks). I also had to discard a Task-based approach, as I could not change signatures to apply async/await constructs. I tried to play with the following, but with no luck:
inject a yield-powered function as IEnumerable<IContext> input parameter to hold loop on foreach() line, to release loop everytime another yield is hit, and try to control that from unit test code.
inject a IContextRunner runner powered by a Reactive Extension Subject to hold loop on runner.Run line, to release loop everytime another Subject.OnNext is hit, and try to control that from unit test code.
For that matters, unit testing framework is NUnit, while NSubstitute is the mocking framework and FluentAssertion is the assertion library of choice. I know how to arrange/act/assert with those.
What is so evident that I missing? Thanks
EDIT
To provide an example of what has been tried, this is a Task-based approach made after posting question and reading #Peter Duniho helpful comment:
// in unit test class
ContextExecutor executor;
IContextRunner runner;
IExecutionCanceler canceler;
IRunnableContext[] runnableContexts;
int totalNrOfContexts;
int nrOfContextToRun; // this will be < totalNrOfContexts
int actualNrOfContextRan;
[SetUp]
public virtual void before_each()
{
// create instance under test, mock dependencies, dummy input data
Initialize();
RunScenarioAsync().Wait();
}
async Task RunScenarioAsync()
{
// prepare mock IContextRunner so that for each input context:
// * there's a related TaskCompletionSource<Object>
// * Run() increments actualNrOfContextRan
// * Run() performs taskSource.Task.Wait();
List<TaskCompletionSource<Object>> runTaskSources = PrepareMockContextRunner();
canceler.IsCanceled.Returns(false); // let execution go initially
// queue up method under test to be processed asynchronously
var executeTask = Task.Run(() =>
{
executor.Execute(runnableContexts);
};
// "unlock" some IContextRunner.Run() invocations,
// for when they will be invoked
for (int i = 0; i < nrOfContextToRun; i++)
{
runTaskSources[i].SetResult(null);
await Task.Delay(0); // tried also with Delay(1) and without this line at all
}
// flag to cancel execution
canceler.IsCanceled.Returns(true);
// unlock all remaining IContextRunner.Run() invocations,
// again for when/if they will be invoked
for (int i = nrOfContextToRun; i < totalNrOfContexts; i++)
{
runTaskSources[i].SetResult(null);
await Task.Delay(0);
}
// wait until method under test completes
await executeTask;
}
[Test]
public void it_should_only_run_until_cancel()
{
int expected = nrOfContextToRun;
int actual = actualNrOfContextRan;
actual.Should().Be(expected);
}
The problem I have here (and similar to other approaches tried) is about giving and regain control to/from the method under test in a predictable way (that is, synchronizing).
Here, if there's no await Task.Delay() or if delay is 0ms, only 1 context is actually ran: the method under test has no chance to run the 2nd and 3rd one, it finds the canceling flag too soon. If delay is 1ms, method executes more context than expected before actually detecting the flag. Also tried with ticks instead of ms, but in my experience playing with delays usually means you're doing something wrong.

How to await a Task inside of a test method

In my repository's unit test project, I'm creating a cleanup method to delete all of the records that were created by tests. Here is what I'm trying to do:
private void DeleteTestExtensions()
{
var authorization = _extensionRepository.GetAuthorizationByTaxAccountIdAndYear(_testTaxAccountId, _testTaxAccountYear);
var extensions = await _extensionRepository.GetExtensionsByUserIdAsync(_testUserId);
foreach (var extension in extensions)
{
try
{
_extensionRepository.DeleteExtension(extension.ExtensionID);
}
}
_extensionRepository.DeleteAuthorization(authorization.ExtensionAuthorizationID);
}
The first one is synchronous and works as expected, but the second method is asynchronous and tells me the await operator can only be used on an async method, but whenever I hover over GetExtensionsByUserIdAsync, I can see (plus I know) that it's defined as
(awaitable) Task<List<Extension>> IExtensionRepository.GetExtensionsByUserIdAsync(Guid userID)
So why would it tell me that I can't use await when it's clearly async?

The message is actually relating to DeleteTestExtensions which must be marked as async, not GetExtensionsByUserIdAsync.
So your method signature would become:
private async Task DeleteTestExtensions()
Note marking the method as returning Task means it can in turn be awaited.

How to use a breakpoint after await in unit test?

I am surprised my breakpoint after awaiting an async method that is on a line that references the awaited Task<T> is never hit:
[Test]
public async void GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
var result = await service.TryGetSomethingAsync(20);
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
}
Putting a breakpoint on the first Assert line is never hit but the test passes.
How to I break when await returns?
UPDATE: I guess it is because the test framework is not awaiting the invocation of the test method, I am using NUnit 2.6.3 and that claims async support, however whether that entails breaking after an await like I am trying to do, I am not sure...

The problem is that your method is async void. That has fire-and-forget semantics.
Conceptually what your method is doing with the async-await usage looks like this:
[Test]
public void GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
service.TryGetSomethingAsync(20).ContinueWith(t =>
{
var result = t.Result;
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
});
}
Now it should be clear what the problem is. Your test method immediately returns as soon as TryGetSomethingAsync returns its Task. So the test immediately finishes. Since no exceptions were thrown, it is a success.
If your test framework supports Task-returning tests, you can fix your test to do what you want by simply changing its return type to Task instead of void.
[Test]
public async Task GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
var result = await service.TryGetSomethingAsync(20);
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
}
This will conceptually translate to the following.
[Test]
public Task GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
return service.TryGetSomethingAsync(20).ContinueWith(t =>
{
var result = t.Result;
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
});
}
Notice how the Task continuation is returned, so that the test framework can now wait on it, ensuring that all the test's code has time to run before the test is considered finished.
(Technically a framework could be made to work in the async void case as well, but I don't know of any reason why that would be a good feature, so I expect most don't handle it.)
If your test framework does not support Task-returning tests, you can fix your test by using .Result instead of await.
[Test]
public void GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
var result = service.TryGetSomethingAsync(20).Result;
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
}
This will simply block the current thread until the Task returned by TryGetSomethingAsync is completed.

Unexpected behavior when passing async Actions around

I'm pretty familiar with the async/await pattern, but I'm bumping into some behavior that strikes me as odd. I'm sure there's a perfectly valid reason why it's happening, and I'd love to understand the behavior.
The background here is that I'm developing a Windows Store app, and since I'm a cautious, conscientious developer, I'm unit testing everything. I discovered pretty quickly that the ExpectedExceptionAttribute doesn't exist for WSAs. Weird, right? Well, no problem! I can more-or-less replicate the behavior with an extension method! So I wrote this:
public static class TestHelpers
{
// There's no ExpectedExceptionAttribute for Windows Store apps! Why must Microsoft make my life so hard?!
public static void AssertThrowsExpectedException<T>(this Action a) where T : Exception
{
try
{
a();
}
catch (T)
{
return;
}
Assert.Fail("The expected exception was not thrown");
}
}
And lo, it works beautifully.
So I continued happily writing my unit tests, until I hit an async method that I wanted to confirm throws an exception under certain circumstances. "No problem," I thought to myself, "I can just pass in an async lambda!"
So I wrote this test method:
[TestMethod]
public async Task Network_Interface_Being_Unavailable_Throws_Exception()
{
var webManager = new FakeWebManager
{
IsNetworkAvailable = false
};
var am = new AuthenticationManager(webManager);
Action authenticate = async () => await am.Authenticate("foo", "bar");
authenticate.AssertThrowsExpectedException<LoginFailedException>();
}
This, surprisingly, throws a runtime error. It actually crashes the test-runner!
I made an overload of my AssertThrowsExpectedException method:
public static async Task AssertThrowsExpectedException<TException>(this Func<Task> a) where TException : Exception
{
try
{
await a();
}
catch (TException)
{
return;
}
Assert.Fail("The expected exception was not thrown");
}
and I tweaked my test:
[TestMethod]
public async Task Network_Interface_Being_Unavailable_Throws_Exception()
{
var webManager = new FakeWebManager
{
IsNetworkAvailable = false
};
var am = new AuthenticationManager(webManager);
Func<Task> authenticate = async () => await am.Authenticate("foo", "bar");
await authenticate.AssertThrowsExpectedException<LoginFailedException>();
}
I'm fine with my solution, I'm just wondering exactly why everything goes pear-shaped when I try to invoke the async Action. I'm guessing because, as far as the runtime is concerned, it's not an Action, I'm just cramming the lambda into it. I know the lambda will happily be assigned to either Action or Func<Task>.

It is not surprising that it may crash the tester, in your second code fragment scenario:
Action authenticate = async () => await am.Authenticate("foo", "bar");
authenticate.AssertThrowsExpectedException<LoginFailedException>();
It's actually a fire-and-forget invocation of an async void method, when you call the action:
try
{
a();
}
The a() returns instantly, and so does the AssertThrowsExpectedException method. At the same time, some activity started inside am.Authenticate may continue executing in the background, possibly on a pool thread. What's exactly going on there depends on the implementation of am.Authenticate, but it may crash your tester later, when such async operation is completed and it throws LoginFailedException. I'm not sure what is the synchronization context of the unit test execution environment, but if it uses the default SynchronizationContext, the exception may indeed be thrown unobserved on a different thread in this case.
VS2012 automatically supports asynchronous unit tests, as long as the test method signatures are async Task. So, I think you've answered your own question by using await and Func<T> for your test.