I am using Timeout attribute for nunit test case as below:
[Test, TestCaseSource("TestCases"), Retry(2), Timeout(10000)
public void test(){
// Some code that runs for more than 10 seconds
}
I have gone through the documentation of nunit but it said that apart from assertion error, retry will not work but I have a situation where test times out.
I want this test to execute again as it is timing out but it executes only once using above code. Please help.
We had the same issue (E2E UI testing, which is finicky so test throws exceptions and retry doesnt work)
You can do a workaround and wrap your test code IE
protected void ExecuteTest(Action test)
{
try
{
test();
}
catch (Exception ex)
{
//If the caught exception is not an assert exception but an unhandled exception.
if (!(ex is AssertionException))
Assert.Fail(ex.Message);
}
}
Which for a test youd want to retry even if it throws would look like
[Test, Retry(3)]
public void TestCase()
{
ExecuteTest(() =>{
<test code>
});
}
Im unsure how the nunit timeout attribute would work (Im assuming the test() call would just throw a timeout exception in which case this solution would work) but it this solution doesnt work for that you can switch to a task or action and WaitOne or something and have a default param for execute test be 1000 for your timeout IE
protected void ExecuteTest(Action test, int timeoutSeconds = 10)
{
try
{
var task = Task.Run(test);
if (!task.Wait(TimeSpan.FromSeconds(timeoutSeconds)))
throw new TimeoutException("Timed out");
test.BeginInvoke(null,null);
}
catch (Exception ex)
{
//If the caught exception is not an assert exception but an unhandled exception.
if (!(ex is AssertionException))
Assert.Fail(ex.Message);
}
}
This looked like our best solution so it's what we currently have implemented and seems to work fine
Related
I have read the Microsoft Documentation but it doesn't mention CallActivityWithRetryAsync.
My IDurableOrchestrationContext is mocked for the call I'm making:
mockContext.Setup(c => c.CallActivityWithRetryAsync(nameof(SerialiseXml), It.IsAny<RetryOptions>(),
It.IsAny<InboundOrchestrationData>()))
.ThrowsAsync(new IOException());
and I can put a breakpoint at the calling point:
try
{
await context.CallActivityWithRetryAsync("SerialiseXml",
new RetryOptions(TimeSpan.FromMinutes(1), 3), data); // breakpoint
}
catch (Exception e)
{
log.LogError("Problem serialising xml.");
}
but my code only ever breaks here once when I would expect it to break three times according to the RetryOptions.
If I pass in a Mock<ILogger<Class>> and check the invocations there is also only one. I have also configured my mock using SetupSequence() and multiple .ThrowsAsync.
Do mind that if you start mocking
mockContext.Setup(c => c.CallActivityWithRetryAsync(nameof(SerialiseXml), It.IsAny<RetryOptions>(),
It.IsAny<InboundOrchestrationData>()))
.ThrowsAsync(new IOException());
there won't be any retries performed at all since it is mocked. There is no real implementation that is executed.
Inside the implementation of CallActivityWithRetryAsync there will be a retry of the specified activity call in case of a retryable failure. Only after all retry attempts have failed an exception will bubble up to your code. So I ever expect just one call to CallActivityWithRetryAsync.
Your exception handling code will also be just called once, and only when all retries have failed or there is another reason for the call to fail.
The implementation of CallActivityWithRetryAsync will look like this (simplified):
async Task<int> CallActivityWithRetryAsync(string activity, RetryOptions retryOptions)
{
int attempts = 1;
while (true)
{
try
{
return await CallActivity(activity);
}
catch
{
++attempts;
if (attempts > retryOptions.RetryCount)
{
throw;
}
}
}
}
Given this pseudocode you can easily see why the breakpoint hits only one.
Using Nunit 3
Test Case: that a thread I created and started, throws an ThreadAbortException when I abort the the thread
Expected Result: to pass (for test to confirm that ThreadAbortException happened)
Result: is failed with error
NUnit.Framework.AssertionException: ' Expected: <System.Threading.ThreadAbortException>
But was: null
Nunit 3 test code:
[SetUp]
public void Setup()
{
_threadViewModel.CreateThread();
}
[Test]
public void TestThreadThrowsAbortedException()
{
try
{
_threadViewModel.RunThread();
Assert.Throws<ThreadAbortException>(() => _threadViewModel.AbortThread());
}
catch (ThreadAbortException e)
{
}
}
Visual Studio Output Window: the output window is correct
System.Threading.ThreadAbortException: Thread was being aborted. at Multthreading.ThreadRunner.WriteY()
Problem: the nunit 3 test does not confirm for me that the exception was thrown
The CLR will automatically re-raise the ThreadAbortException after your catch block (see this answer for more information).
You can try to use the Thread.ResetAbort() method in your test code's catch block - be sure to read the remarks.
[Test]
public void TestThreadThrowsAbortedException()
{
try
{
_threadViewModel.RunThread();
Assert.Throws<ThreadAbortException>(() => _threadViewModel.AbortThread());
}
catch (ThreadAbortException e)
{
Thread.ResetAbort();
}
}
It works for me with my test runner. YMMV.
A little confusion as to the behaviour of Moq with MsTest.
Edit: This is not a question of "How do I test?" or "How do I assert?", this is a scratch pad to see how MoQ works so don't focus on the exception type etc.
I think a better question may be => "Does Moq Throws<> behave similar to MsTest ExpectedExceptionAttribute?" That is, they're expecting an exception in the test or the SUT?
I'd like to know just how MoQ "Throws" works when used with MsTest. Is it better to not use the MsTest expected exception attribute? Is it better to perform a try..catch within the test? I have a few more questions surrounding this.
I am Mocking a database call and when an error occurs I would like to return zero (0).
The TestMethod is straight forward with the MsTest exception attribute, and the throws exception with Moq. It only works when I throw an exception within the SaveCart method and not when I return zero.
I would like to understand the underlying behaviour because it feels as though I shouldn't, nor want to throw an exception within the SaveCart method.
Here is the Test under question:
[TestMethod]
[ExpectedException(typeof(ApplicationException))]
public void CartRepoSaveCartExceptionShouldReturnZero()
{
_cartDatabaseMock.Setup(c => c.SaveCart(_cart))
.Throws<ApplicationException>();
var result = _cartRepository.SaveCart(_cart);
Assert.AreEqual(result, _cartSaveExceptionValue);
}
Here is the basic SaveCart which does NOT throw an exception causing the test to fail:
public long SaveCart(Cart cart )
{
long returnValue;
try
{
returnValue = _cartDatabase.SaveCart(cart);
}
catch (Exception)
{
return 0;
}
return returnValue;
}
Here is a basic SaveCart where the test works because it's throwing an exception:
public long SaveCart(Cart cart )
{
long returnValue;
try
{
returnValue = _cartDatabase.SaveCart(cart);
}
catch (Exception)
{
throw new ApplicationException();
}
return returnValue;
}
Feel free to suggest a better title for the question if it doesn't quite explain it clearly.
You should use ExpectedExceptionAttribute when the unit under test throws an exception.
In your first example the method didn't throw any exception therefore the test failed.
Since your method under test doesn't throw any exception you don't need to use this attribute at all...(just verify the return value in this scenario)
When you want to verify that exception was thrown and you want to verify that some additional operations occurred, use the following pattern:
[TestMethod]
[ExpectedException(typeof(<The specific exception>))]
public void FooTest()
{
//arrange
try
{
// act
}
catch(<the specific exception>)
{
// some asserts
throw;
}
}
The above snippet will failed if:
wrong exception raised
exception was not raised
one of your asserts failed.
BTW, since your catch in the method is no Exception instead of ApplicationException, I offer you to change the setup to:
_cartDatabaseMock.Setup(c => c.SaveCart(_cart)).Throws<Exception>();
You are right - the second test "SaveCart" works because it's throwing an exception and the the first test fail because you are turning 0. From your response to previous answers, I am sure you already know all of this. If you are asking for the behavior how it failed your first test... it goes like this:
SaveCart is called
It returns an exception (result of your moq setup)
Your try catch caught the exception (you did this on purpose to alter the result)
Your try catch returns 0 (result is now 0 as you intended to alter it)
Assert checks your result against _cartSaveExceptionValue
You get a fail test stating something similar to this "Message: Assert.AreEqual failed. Expected. Actual<0 (System.Int32)>."
If you want to double check this... you can try the following test
comment out the [ExpectedException(typeof())]
change the Assert.AreEqual(result, _cartSaveExceptionValue) to Assert.AreEqual(result, 0);
the test should pass because you are comparing "result" (aka 0) to 0
I hope this answer your question.
catch (Exception)
{
return 0;
}
you are not throwing the exception, rather swallowing the exception, so why would you expect exception? It has nothing to do with MOQ. Your test and code are not in sync.
This is a bad practice btw, to swallow exception.
catch (Exception)
{
throw new ApplicationException();
}
That's also a code smell. You are catching all kinds of exception and then throwing a different type.
This question already has answers here:
How to test that no exception is thrown?
(20 answers)
Closed 8 years ago.
I m using NUnit, I have following code which will be tested.
public class StudentPresenter
{
IView myview;
Repository myrepo;
public StudentPresenter(IView vw, Data.Repository rep)
{
this.myview = vw;
this.myrepo = rep;
this.myview.ButtonClick += myview_ButtonClick;
}
public void myview_ButtonClick()
{
try
{
var d = this.myrepo.GetById(int.Parse(myview.GivenId));
this.myview.GetStudent(d);
}
catch(Exception e)
{
}
}
}
I need to test myview_ButonClick()
Lets suppose I will test this method and it will throw exception if myview.GivenId is null?
So I write unit test as below:
[Test]
public void Button1Click_NullText_ThrowException()
{
var mock = Substitute.For<IView>();
StudentPresenter sp = new StudentPresenter(mock, repo);
mock.GivenId = string.Empty;
Assert.Throws<Exception>(()=>sp.myview_ButtonClick());
}
But test failed.. Why? (becouse no throw in my catch block). But I dont want to throw anything, I just want that it has to ability to catch. So is it possible to test?
You cannot have a unit test that checks if a code block has a "catch" block. The only way to (indirectly) do it would be to test that the test does not throw when given input that would normally cause it to throw.
NUnit has a DoesNotThrow assert that should be useful here.
Of course, that's no guarantee of the existence of a try/catch, but its about the best you can do.
myview_ButtonClick() catches all exceptions, and so it will never throw an exception that could be detected from outside the method, hence your test fails.
If you want to catch the exception, do something with it, and then throw it in order for a caller to catch it, or the test to pass, then simply
catch (Exception e)
{
//actions....
throw;
}
Bear in mind too that catching all exceptions is rarely a good idea. The calling code will continue oblivious to the exception state, and this could cause untold problems down the line, which could be a nightmare to debug and track down.
I've a method test that does not have a clear assert expression. The returned tested value is a very long string that has to be inspected by a programmer to check if it is correct or not. For this reason, if the code executes without exceptions, I'm calling 'Assert.Inconclusive'.
However, if some kind of Exception is thrown, I want to call 'Assert.Fail' with the exception message. Something like this:
[TestMethod()]
public void Test()
{
try {
string toBeChecked = MethodToBeTested();
//the string is so particular that no clear
//assertion can be made about it.
Console.WriteLine(toBeChecked);
Assert.Inconclusive("Check the console output.");
} catch(Exception e) {
Assert.Fail(e.Message);
}
}
The problem with this code is that if no regular exception is thrown, the Assert.Inconclusive method also throws an exception that is catched, so Assert.Fail is called, and from the IDE test result pane it seems that the test has failed. This is not what I want.
Is there a way to filter the exceptions, such as catching every exception but the Assert-like ones?
(I'm using .NET framework 3.5SP1)
Why not just leave out the Assert.Inconclusive()? In fact, why catch any exceptions at all - if the code throws an exception the unit test framework will mark it as failed. Less is more:
[TestMethod()]
public void Test()
{
string toBeChecked = MethodToBeTested();
Console.WriteLine(toBeChecked);
}
But this is a poor unit test if we can not automatically check the result. All we are checking is that no exception is thrown.
Are there no asserts you can make about the resulting string?
For example, that is is not null or empty? Do we expect it to contain a certain sub-string that we can test?
At least give the test a helpful name, which includes something like: ManualAssertRequired
The Assert.Inconclusive method should throw a AssertInconclusiveException so you can either mark the Test as ExcpectedException(typeof(AssertInconclusiveExcpetion)) or use something like this:
[TestMethod()]
public void Test()
{
try {
string toBeChecked = MethodToBeTested();
//the string is so particular that no clear
//assertion can be made about it.
Console.WriteLine(toBeChecked);
Assert.Inconclusive("Check the console output.");
} catch(AsssertInconclusiveException) {
/* Do nothing */
}
} catch(Exception e) {
Assert.Fail(e.Message);
}
}
Try to catch specific exception type instead of Exception or add another catch for nunit exception that is caused by Assert.Inconclusivemethod which is AssertInconclusiveException...
For example modify it like this:
[TestMethod()]
public void Test()
{
try {
string toBeChecked = MethodToBeTested();
//the string is so particular that no clear
//assertion can be made about it.
Console.WriteLine(toBeChecked);
Assert.Inconclusive("Check the console output.");
} catch(AssertInconclusiveException e) {
// do nothing...
} catch(Exception e) {
Assert.Fail(e.Message);
}
}