I've noticed that NUnit's TestContext.CurrentContext.Result.Outcome.Status is always Inconclusive at the end of a test run. The CurrentContext is also unaware that any assertions have taken place.
Is it possible to get the status of a test before the [TearDown]?
I was hoping to use the value during the Dispose() of my test management class to capture metrics and other data for post-test diagnosis.
Code example from a new .NET Framework 4.6.1 project that only has the NuGet packages NUnit and FluentAssertions:
namespace TestProject
{
using FluentAssertions;
using NUnit.Framework;
[TestFixture]
public class Class1
{
[Test]
public void test1()
{
var a = 1;
var b = 2;
var c = 1;
var context = TestContext.CurrentContext;
a.Should().Be(c);
Assert.AreEqual(a, c);
}
}
}
A test result starts out as Inconclusive. If the test is skipped or ignored, then the result changes, but of course it is never executed.
If it is executed, the Outcome is Inconclusive until the test is over. Clearly, while you are still executing the test, it is not yet finished. When teardown begins, the outcome of the test is known, so it will vary according to whether the test method itself was successful. Of course, an exception in teardown may change the result to an Error state.
Bottom line, the Outcome field is not useful while the test method itself is still running. In any case, if you are executing code in the test method, the test has not yet failed. Otherwise, you would not have continued execution!
You say you can't use TearDown but the link you provide doesn't deal with the issue of accessing the test result. Can you explain further? What exactly do you want to do after checking the test result?
You could try using AssertionScope along with the AssertionScope.Succeeded flag. Although the intelisense on that property specifies:
Gets a value indicating whether or not the last assertion executed through this scope succeeded.
Example usage of AssertionScope
[Test]
public void Test()
{
var a = 1;
var b = 2;
var c = 1;
var context = new AssertionScope();
try
{
throw new Exception("Test");
}
catch (Exception e)
{
context.FailWith(e.ToString());
}
var strings = context.Discard();
Console.WriteLine(strings.StringJoin(","));
context.Succeeded.Should().BeFalse();
var someOtherContext = new AssertionScope();
try
{
c.Should().Be(a);
}
catch (Exception e)
{
someOtherContext.FailWith(e.ToString());
}
var discard = someOtherContext.Discard();
Console.WriteLine(discard.StringJoin(","));
someOtherContext.Succeeded.Should().BeTrue();
}
Related
I am using Visual Studio Unit Test Cases.
I have written the Unit test case where Argument Exception is expected from the method under test MethodUnderTest. Suppose if any other part of the test case (Setup part) throws the expected exception ArgumentException, Then I want to enforce that my test case should fail. It should pass only in case Setup is correct and instance.MethodUnderTest(); line of code throws ArgumentException.
I can achieve using try catch, but I want to know is there any better approach to achieve this.
[ExpectedException(typeof(ArgumentException))]
public void TestCaseMethod()
{
// Set up
Mock<ITestClass> testM = new Mock<ITestClass>();
AnimalClass instance = new AnimalClass(testM.Object);
// call the method under test
instance.MethodUnderTest();
}
If you use a more advanced unit testing framework, like NUnit. you can do things like:
// Act
var result = Assert.Throws<Exception>(() => instance.MethodUnderTest));
// Assert
Assert.IsInstanceOf<ArgumentException>(result);
I don't know of any built in way, but you could wrap up the method in an assert exception
private void AssertException<T>(Action method)
where T : Exception
{
try
{
method();
Assert.Fail();
}
catch (T e)
{
Assert.IsTrue(true);
}
}
Then call with
[TestMethod]
public void TestCaseMethod()
{
// Set up
Mock<ITestClass> testM = new Mock<ITestClass>();
AnimalClass instance = new AnimalClass(testM.Object);
// call the method under test
AssertException<ArgumentException>(instance.MethodUnderTest)
}
Or, if your method takes in parameters or returns values
AssertException<MyException>(() => instance.ParameterisedFunction(a, b));
I have the following code for which I am writing a unit test
protected virtual string GetTokenFromTheWebrequest(WebRequest httpWebRequestObject)
{
try
{
HttpWebResponse responseHttpPostForToken = (HttpWebResponse)httpWebRequestObject.GetResponse();
string tokenJson = GetContentResponse(responseHttpPostForToken);
AccessToken accesstokenObj = DeserializeToAccessToken(tokenJson);
return accesstokenObj.Token;
}
catch (Exception Error)
{
Log.Debug("error");//I want to test this method
throw Error;
}
}
So,For this I have this unit test
[Test]
[ExpectedException(typeof(JsonReaderException))]
public void GetTokenFromTheWebrequestTestwithInValidHttpWebResponseOjectJsonReader()
{
var mockHttpWebResponse = new Mock<HttpWebResponse>();
var mockHttpWebRequest = new Mock<HttpWebRequest>();
var mockLog = new Mock<ILog>();
mockHttpWebRequest.Setup(c => c.GetResponse()).Returns(mockHttpWebResponse.Object);
mockedSurveyMonkeyAPIService.Setup(y => y.GetContentResponse(mockHttpWebResponse.Object)).Returns(TestData.TestData.SampleResponseStream);
mockedSurveyMonkeyAPIService.Setup(z => z.DeserializeToAccessToken(TestData.TestData.SampleResponseStream)).Throws(new JsonReaderException());
//mockLog.Setup(x => x.Debug("error")).Throws(new WebException());
var token = mockedSurveyMonkeyAPIService.Object.GetTokenFromTheWebrequest(mockHttpWebRequest.Object);
mockLog.VerifySet(x => x.Debug("error"),Times.Once); //its not going till this when i debug
}
Can anyone suggest how to check whether Log.Debug is called properly.
You don't seem to be testing any of your own code... you've mocked pretty much everything being used in that method. This begs the question what is actually being tested and the answer is that your implementation details are being tested, not any logic. You have even mocked the logger, so really you are testing the .NET try-catch mechanism here.
If you make any minor change to this method, your test will break even though the behaviour is the same. This is not the kind of test you want to shackle yourself to.
There are better ways of testing. For example, if you want to test your logger - test it in isolation of this method so you know your logger works. Don't test the logger by testing calls to it everywhere it is used.
I experience a strange issue while testing my Nhibernate repositories.
I have 10 unit-tests like the ones below. Everytime a run them in a batch the first fails and the rest succeeds. If a run them one by one they all fail. If a restart MSDTC before my testrun it sometimes behaves like before and sometimes all tests succeeds. I can´t find a pattern why it behaves like that.
I want the transaction to rollback so that a start with a clean DB for every test, therefore the transaction disposal.
The test/tests are failing due to this error:
System.Data.SqlClient.SqlException: MSDTC on server 'MYCOMPUTERNAME\SQLEXPRESS' is unavailable.
My tests looks like this:
[TestInitialize]
public void MyTestInitialize()
{
_transactionScope = new TransactionScope();
}
[TestCleanup]
public void MyTestCleanup()
{
if (_transactionScope != null)
{
_transactionScope.Dispose();
_transactionScope = null;
}
}
[TestMethod]
[TestCategory("RepositoryTests")]
public void RepositoryCanSaveAProduct()
{
var platform = ProductObjectMother.CreatePlatform("100010", "Supplier 10");
var mainsegment = ProductObjectMother.CreateMainSegment("123");
var application = ProductObjectMother.CreateApplication("Foo");
var productfamily = ProductObjectMother.CreateProductFamily("X99");
Engine i = ProductObjectMother.CreateEngine(platform, productfamily, application, mainsegment);
var repository = new ProductRepository();
repository.Save(i);
repository.Flush();
}
Problem seems to be with transaction that is neither committed using _transactionScope.Complete() or roll back by throwing exception.
Also I notice one strange thing, test usually fails or run successfully by "Assert" functions(equals, not equal, exists etc from assert) that is missing from your test. :)
is there a MsTest Equivalent of Assert.Warning in MbUnit ?
The closest match is Assert.Inconclusive() - it doesn't make the test fail as such, but it doesn't succeed either. It fall into a third stage called Inconclusive.
A single Inconclusive test will cause an entire test suite to be Inconclusive.
There are overloads that supports custom messages as well:
Assert.Inconclusive("Ploeh");
I have a similar issue as I use NUnit for some projects. Try using
Console.Write("Some Warning");
You may want to use a custom exception.
The trouble with Assert.Inconclusive is that Test Explorer states the test wasn't even run. This may be misleading when running the test in the future, particularly if the test is run by other developers:
The way I've come to prefer is as follows. Firstly, define a custom UnitTestWarningException. I've given mine an additional constructor so I can pass my warning message String.Format-style with arguments:
public class UnitTestWarningException : Exception
{
public UnitTestWarningException(string Message) : base(Message) { }
public UnitTestWarningException(string Format, params object[] Args) : base(string.Format(Format, Args)) { }
}
Then, at the point where you want to end a unit test with a warning, throw a UnitTestWarningException instead:
[TestMethod]
public void TestMethod1()
{
.
.
.
try
{
WorkflowInvoker.Invoke(workflow1, inputDictionary);
}
catch (SqlException ex)
{
if (ex.Errors.Count > 0
&& ex.Errors[0].Procedure == "proc_AVAILABLEPLACEMENTNOTIFICATIONInsert")
{
//Likely to occur if we try to repeat an insert during development/debugging.
//Probably not interested--the mail has already been sent if we got as far as that proc.
throw new UnitTestWarningException("Note: after sending the mail, proc_AVAILABLEPLACEMENTNOTIFICATIONInsert threw an exception. This may be expected depending on test conditions. The exception was: {0}", ex.Message);
}
}
}
The result: Test Explorer then shows that the test has been executed, but failed with a UnitTestWarningException that shows your warning:
Here is my hack on how to have warnings with nunit ( i know this question was about mstest, but this should work too). As always, I am interested in any improvements. This method is working for me.
Background: I have code which checks the tests themselves for correct comments and has logic to detect if someone has copied and pasted another test without changing comments. These are warnings I want to be shown to the developer without normal Assert.Inconclusive blocking the actual test from running. Some are focused on the test and the cleanup refactorings phase is to remove the warnings.
Mission: to have warnings after all other asserts are run. This means even showing the warnings after Assert.Fail that normally occur in tests during development.
Implementation: (best to create a base class for all test files):
public class BaseTestClass
{
public static StringBuilder Warnings;
[SetUp]
public virtual void Test_SetUp()
{
Warnings = new StringBuilder();
}
[TearDown]
public virtual void Test_TearDown()
{
if (Warnings.Length > 0)
{
string warningMessage = Warnings.ToString();
//-- cleared if there is more than one test running in the session
Warnings = new StringBuilder();
if (TestContext.CurrentContext.Result.Status == TestStatus.Failed)
{
Assert.Fail(warningMessage);
}
else
{
Assert.Inconclusive(warningMessage);
}
}
}
Testing Usage
[Test]
public void Sample_Test()
{
if (condition) Warning.AppendLine("Developer warning");
Assert.Fail("This Test Failed!");
}
Actual Result:
"This Test Failed!"
"Developer warning"
Status of test is failed - RED
If the test passed and there was a warning, you will then get the status of Inconclusive - YELLOW.
I would like to have my TearDown method check whether the previous test was a success before it applies some logic. Is there an easy way to do this?
This has been already solved in Ran's answer to similar SO question. Quoting Ran:
Since version 2.5.7, NUnit allows Teardown to detect if last test failed.
A new TestContext class allows tests to access information about themselves including the TestStauts.
For more details, please refer to http://nunit.org/?p=releaseNotes&r=2.5.7
[TearDown]
public void TearDown()
{
if (TestContext.CurrentContext.Result.Status == TestStatus.Failed)
{
PerformCleanUpFromTest();
}
}
If you want to use TearDown to detect status of last test with NUnit 3.5 it should be:
[TearDown]
public void TearDown()
{
if (TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Failed)
{
//your code
}
}
sounds like a dangerous idea unless it's an integration test, with say data to remove say. Why not do it in the test itself?
Obviously a private flag in the class could be set.
This is what Charlie Poole himself has suggested if you must
Only if you do this manually. In fact you even won't know which tests are intend to run. In NUnit IDE one can enable some tests and disable some other. If you want to know if some specific test has run you could include code like this in your test class:
enum TestStateEnum { DISABLED, FAILED, SUCCEDED };
TestStateEnum test1State = TestStateEnum.DISABLED;
[Test]
void Test1()
{
test1State = TestStateEnum.FAILED; // On the beginning of your test
...
test1State = TestStateEnum.SUCCEDED; // On the End of your Test
}
Then you can check the test1State variable. If the test throws an exception it won't set the SUCCEDED. you can also put this in a try catch finally block in your tests with a slightly different logic:
[Test]
void Test1()
{
test1State = TestStateEnum.SUCCEDED; // On the beginning of your test
try
{
... // Your Test
}
catch( Exception )
{
test1State = TestStateEnum.FAILED;
throw; // Rethrows the Exception
}
}
[OneTimeTearDown]
public void AfterEachTest()
{
if (TestContext.CurrentContext.Result.Outcome.Status.Equals(TestStatus.Failed))
{
Console.WriteLine("FAILS");
}
else if (TestContext.CurrentContext.Result.Outcome.Equals(ResultState.Success))
{
Console.WriteLine("SUCESS");
}
}
IMHO tear down logic should be independent of test results.
Ideally you should avoid using setup and teardown completely, a al xunit.net. See here for more info.