We currently have an integration test that tests a front-end api call. The call is async therefore we force it until the call is ready to compare the result with what we expect.
However, the problem is that the code contains also an another async call which is not awaited. So we have nested async calls. This also has the cause that the test sometimes succeeds and sometimes it fails (race condition).
For now we have fixed the issue by added a Thread.Sleep (10000) to the code. However, this is not a good solution.
The code likes like this:
[TestMethod]
public void Integration_Test_Example()
{
// Arrange
...
// Act
var request = CreateRequest(#"testfile.xml");
var task = target.SendMessage(request);
Thread.Sleep(10000); // dirty fix
task.Wait();
// Assert
...
}
Does anyone have advice on how we can best solve this?
If I understand correctly SendMessage returns task, so you should be able to await it before moving on with your test.
[TestMethod]
public async Task Integration_Test_Example()
{
var request = CreateRequest(#"testfile.xml");
var result = await target.SendMessage(request);
}
You can change the test to be awaitable
[TestMethod]
public async Task Integration_Test_Example()
{
// Arrange
...
// Act
var request = CreateRequest(#"testfile.xml");
await target.SendMessage(request);
// Assert
...
}
Related
I have a unit test that is using Moq to mock interfaces and verify calls. Here is the test code:
[Fact]
public void NewBlank_InvokesManagerAdd()
{
// ReSharper disable once AssignNullToNotNullAttribute
var newPath = Path.Combine(_testSaveDirectory, "InvokeBlank.txt");
_dbManagerMock.Setup(manager => manager.KeynoteDBs).Returns(new ObservableCollection<KeynoteDBVM>());
_dialogMock.Setup(dialog => dialog.GetSaveFileDialogResult(It.IsAny<SaveFileDialogData>()))
.Returns(newPath);
_commands.CmdNewBlank.Execute(null);
_dbManagerMock
.Verify(manager => manager.AddDB(It.IsAny<KeynoteDB>(), It.IsAny<int>()),
Times.Once);
}
However when I run this I get this test failure:
Moq.MockException
Expected invocation on the mock once, but was 0 times: manager => manager.AddDB(It.IsAny<KeynoteDB>(), It.IsAny<int>())
Performed invocations:
Mock<IDBManager:1> (manager):
IDBManager.KeynoteDBs
IDBManager.IsLoading = True
IDBManager.ActiveDB
IDBManager.AddDB(KeynoteDB, -1)
IDBManager.ActiveDB
IDBManager.IsLoading = False
at Moq.Mock.Verify(Mock mock, LambdaExpression expression, Times times, String failMessage)
at Moq.Mock`1.Verify[TResult](Expression`1 expression, Func`1 times)
at KMCore_Tests.AppCommandsTests.NewBlank_InvokesManagerAdd() in *my path*\AppCommandsTests.cs:line 140
Is it not listing that invocation 1 time in that list of invocations right in the middle? How is it saying it has 0 invocations? What am I missing? I feel like I must be missing something dumb but I can't see it...
EDIT
Ok so this turned out to be a race condition issue because the commands are async. The command execution calls an async void method and it must have been hitting the assert before it hit the actual call (or at least that's all I can think of). I put in a await Task.Delay(500) after calling execute and before the assert and it is passing now.
Is there a better way to test this situation? The commands are essentially button handlers so I think async void is correct here from my understanding, but that means I can't await it in the unit test...
You should avoid using async void anyway.
async void means fire and forget. In your case, you don't seem to want that because you need to wait until the async method is completed.
If you can't change the method signature, you could move the body of the async void method to a Task returning method.
So for example, change
public async void MyAsyncVoidMethod()
{
await Task.Delay(500);
MethodToBeCalled();
}
to
// Wait this task in unit test
public Task MyAsyncTask { get; private set; }
public async void MyAsyncVoidMethod()
{
MyAsyncTask = MyAsyncTaskMethod();
await MyAsyncTask;
}
public async Task MyAsyncTaskMethod()
{
await Task.Delay(500);
MethodToBeCalled();
}
Alternatively, if there is any event fired after the command is completed, you could also hook into one of those events to stay notified in your tests.
I have async methods which doesn't have any return type , it just has Task type.
How can I write valid unit test method for just Task ?
Async Methods:
public async Task UploadFileAsync(FileDetails data)
{
await _fileUpload.UploadImageAsync(data);
}
public async Task UploadImageAsync(FileDetails data)
{
// does a async operation to upload a stream to azure blob storage
await azureBlob.UploadFromStreamAsync(streamData).ConfigureAwait(false)
}
Test Method:
[TestMethod]
public async Task UploadFileAsync()
{
//-->Arrange
_fileUpload.Setup(e => e.UploadImageAsync().Returns(new Task<Task>(() => null));
//-->Act
Task result = _uploadManager.UploadFileAsync(GetFakeUploadData());
// Task doesn't return any result ?
//-->Assert
}
Update:
Finally I call this method to upload file to azure blob storage.
You can just change your Act section to:
//-->Arrange
_fileUpload
.Setup(e => e.UploadImageAsync(It.IsAny<FileDetails>()))
.Returns(Task.FromResult<object>((object)null))
.Verifiable(); //<-- Allows you to verify that the mock was invoked correctly
//Act
await _uploadManager.UploadFileAsync(GetFakeUploadData());
//Assert
_fileUpload.Verify();
Reference Moq Quickstart to get a better understanding of how to use the Moq framework.
An async method returning Task is similar to a regular void method. You can await the task to ensure that the method runs before your test completes:
Task result = _uploadManager.UploadFileAsync(GetFakeUploadData());
await result.ConfigureAwait(false);
There is no assertion to make here - all you could say about the method is that it runs without triggering an error.
If the server returns some token to your application when upload is complete (say, some proprietary resource identifier) than it would be beneficial to change the signature of UploadFileAsync to make that identifier available to the caller:
public async Task<UploadedResourceIdentifier> UploadFileAsync(FileDetails data) {
var ret = await _fileUpload.UploadImageAsync(data).ConfigureAwait(false);
return new UploadedResourceIdentifier(ret); // Parses the return, and constructs resource identifier
}
Now the test would have something to assert:
var result = await _uploadManager
.UploadFileAsync(GetFakeUploadData())
.ConfigureAwait(false);
Assert.That(result.IsValid, Is.True);
I would suggest using the ContinueWith method so you can inspect the IsFaulted member.
result.ContinueWith(t => Assert.IsFalse(t.IsFaulted));
await _uploadManager.UploadFileAsync(GetFakeUploadData());
to execute the operation and wait for it to complete, and then you should somehow access the uploaded file and verify it's there. If you can upload you can probably also download, so use that.
I am surprised my breakpoint after awaiting an async method that is on a line that references the awaited Task<T> is never hit:
[Test]
public async void GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
var result = await service.TryGetSomethingAsync(20);
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
}
Putting a breakpoint on the first Assert line is never hit but the test passes.
How to I break when await returns?
UPDATE: I guess it is because the test framework is not awaiting the invocation of the test method, I am using NUnit 2.6.3 and that claims async support, however whether that entails breaking after an await like I am trying to do, I am not sure...
The problem is that your method is async void. That has fire-and-forget semantics.
Conceptually what your method is doing with the async-await usage looks like this:
[Test]
public void GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
service.TryGetSomethingAsync(20).ContinueWith(t =>
{
var result = t.Result;
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
});
}
Now it should be clear what the problem is. Your test method immediately returns as soon as TryGetSomethingAsync returns its Task. So the test immediately finishes. Since no exceptions were thrown, it is a success.
If your test framework supports Task-returning tests, you can fix your test to do what you want by simply changing its return type to Task instead of void.
[Test]
public async Task GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
var result = await service.TryGetSomethingAsync(20);
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
}
This will conceptually translate to the following.
[Test]
public Task GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
return service.TryGetSomethingAsync(20).ContinueWith(t =>
{
var result = t.Result;
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
});
}
Notice how the Task continuation is returned, so that the test framework can now wait on it, ensuring that all the test's code has time to run before the test is considered finished.
(Technically a framework could be made to work in the async void case as well, but I don't know of any reason why that would be a good feature, so I expect most don't handle it.)
If your test framework does not support Task-returning tests, you can fix your test by using .Result instead of await.
[Test]
public void GetSomethingTest()
{
var service = SimpleIoc.Default.GetInstance<IService>();
var result = service.TryGetSomethingAsync(20).Result;
Assert.IsTrue(result.IsSuccess);
Assert.IsNotNull(result.ReturnValue);
}
This will simply block the current thread until the Task returned by TryGetSomethingAsync is completed.
I'm pretty familiar with the async/await pattern, but I'm bumping into some behavior that strikes me as odd. I'm sure there's a perfectly valid reason why it's happening, and I'd love to understand the behavior.
The background here is that I'm developing a Windows Store app, and since I'm a cautious, conscientious developer, I'm unit testing everything. I discovered pretty quickly that the ExpectedExceptionAttribute doesn't exist for WSAs. Weird, right? Well, no problem! I can more-or-less replicate the behavior with an extension method! So I wrote this:
public static class TestHelpers
{
// There's no ExpectedExceptionAttribute for Windows Store apps! Why must Microsoft make my life so hard?!
public static void AssertThrowsExpectedException<T>(this Action a) where T : Exception
{
try
{
a();
}
catch (T)
{
return;
}
Assert.Fail("The expected exception was not thrown");
}
}
And lo, it works beautifully.
So I continued happily writing my unit tests, until I hit an async method that I wanted to confirm throws an exception under certain circumstances. "No problem," I thought to myself, "I can just pass in an async lambda!"
So I wrote this test method:
[TestMethod]
public async Task Network_Interface_Being_Unavailable_Throws_Exception()
{
var webManager = new FakeWebManager
{
IsNetworkAvailable = false
};
var am = new AuthenticationManager(webManager);
Action authenticate = async () => await am.Authenticate("foo", "bar");
authenticate.AssertThrowsExpectedException<LoginFailedException>();
}
This, surprisingly, throws a runtime error. It actually crashes the test-runner!
I made an overload of my AssertThrowsExpectedException method:
public static async Task AssertThrowsExpectedException<TException>(this Func<Task> a) where TException : Exception
{
try
{
await a();
}
catch (TException)
{
return;
}
Assert.Fail("The expected exception was not thrown");
}
and I tweaked my test:
[TestMethod]
public async Task Network_Interface_Being_Unavailable_Throws_Exception()
{
var webManager = new FakeWebManager
{
IsNetworkAvailable = false
};
var am = new AuthenticationManager(webManager);
Func<Task> authenticate = async () => await am.Authenticate("foo", "bar");
await authenticate.AssertThrowsExpectedException<LoginFailedException>();
}
I'm fine with my solution, I'm just wondering exactly why everything goes pear-shaped when I try to invoke the async Action. I'm guessing because, as far as the runtime is concerned, it's not an Action, I'm just cramming the lambda into it. I know the lambda will happily be assigned to either Action or Func<Task>.
It is not surprising that it may crash the tester, in your second code fragment scenario:
Action authenticate = async () => await am.Authenticate("foo", "bar");
authenticate.AssertThrowsExpectedException<LoginFailedException>();
It's actually a fire-and-forget invocation of an async void method, when you call the action:
try
{
a();
}
The a() returns instantly, and so does the AssertThrowsExpectedException method. At the same time, some activity started inside am.Authenticate may continue executing in the background, possibly on a pool thread. What's exactly going on there depends on the implementation of am.Authenticate, but it may crash your tester later, when such async operation is completed and it throws LoginFailedException. I'm not sure what is the synchronization context of the unit test execution environment, but if it uses the default SynchronizationContext, the exception may indeed be thrown unobserved on a different thread in this case.
VS2012 automatically supports asynchronous unit tests, as long as the test method signatures are async Task. So, I think you've answered your own question by using await and Func<T> for your test.
While on my local box the following code works:
public async Task<GameStatistic> LoadReport()
{
var folder = await ApplicationData.Current.LocalFolder.CreateFolderAsync(FolderName, CreationCollisionOption.OpenIfExists);
var file = await folder.GetFileAsync(FileName);
GameStatistic returnValue;
using (var inStream = await file.OpenSequentialReadAsync())
{
var serializer = new DataContractJsonSerializer(typeof (GameStatistic));
returnValue = serializer.ReadObject(inStream.AsStreamForRead()) as GameStatistic;
}
return returnValue;
}
Code that calls the above method:
public GameStatistic GetReportData()
{
var repo = new GameRepository();
var gameStatTask = repo.LoadReport(); //(awaitable) Task<GameStatistic>
gameStatTask.Wait(); //this seems to make no difference
return gameStatTask.Result;
}
But When I move to code to my Surface Pro and run the application (no debugger), the folder.GetFileAsync(FileName) fails because the async call to get the folder hasn't returned yet.
When I debug the application on my Surface Pro (via Remote Machine) and slowly walk the debugger past the first line of code and wait a few seconds, and then step again, everything works.
I don't like the idea of trying to put a thread to sleep for an arbitrary length of time, but I am not sure what else I can do here.
Is there something I am doing wrong or something I should be doing that I am not doing at all?
Is there a common practice that would really wait until the CreateFolderAsync returns so that when I call folder.GetFileAsync that I could be sure the preceding line was complete?
Thanks for any help you may be able to provide.
As #J.B points out you need to use await instead of wait. Also, any function that calls an async method should itself be async (there is at least one exception to this). So almost your entire call stack up to the UI must be changed to be some variation of async Task<...>...:
async public Task<GameStatistic> GetReportData()
{
var repo = new GameRepository();
return await repo.LoadReport(); //(awaitable) Task<GameStatistic>
}
Caller of above (just an arbitrary method):
async public Task<MyResultClass> GetReportAndResult()
{
var gameStat = await GetReportData();
return ReportDataToMyResult(gameStat);
}
Top of call chain (event handler) must be async void:
async void GetReportData_ButtonClick(...)
{
var result = await GetReportAndResult();
// do something with result
// ...
}