Hi all I'm making a Chess AI as hobby project with Test Driven Development. But for some reason the only message my tests give is "test has thrown an exception: ...". And therefore omitting the only thing that matters. Instead of directly reading the error I now have to right-click and view test result details. I have tried adding and removing columns, but I cannot get the whole message to be shown directly.
Can VS2010 be setup so the exception message gets shown directly for each unit test?
edit: I am using standaard VS unit tests:
[TestClass]
public class MiniMaxTest
{
[TestMethod]
public void TestConstructor()
{
throw new Exception("Must I click view details to see this?");
}
}
Why these questions? You guys can reproduce these things. Debug or Run tests give the same messages:
No, I don't believe you can configure VS to show it differently. The IDE shows the first line of the exception message, and there's a newline character in the full message text, so you'll need to click through to the details to view the whole thing.
What you can do however, is abuse the MSTest framework to show your own messages.
MS Test Assertions are all implemented by throwing exceptions. All the MS Test Assert functions throw exceptions which are derived from UnitTestAssertException. Visual studio has special handling for these kinds of exceptions.
For example: If you write this test:
[TestMethod]
public void AllAboard()
{
throw new AssertFailedException("Failboat");
}
AssertFailedException is the standard base class for most other assertion failures.
You'll note that VS2010 does not print the generic "test threw an exception" message, instead it just prints "Failboat".
Now what you can do is surround your tests in things that convert normal exceptions into AssertFailedException and then you can print whatever messages you like.
[TestMethod]
public void TestStuff()
{
try
{
string o = null; // pretend this came from some real code
var x = o.Split(new[] { ',' }); // this will throw NullRefException
}
catch (Exception e)
{
throw new AssertFailedException(e.Message, e);
}
}
Of course, I wouldn't recommend actually doing this... it's verbose, and more importantly, you lose the call stack... but hey, now you've got one more tool in your toolbelt
Related
I'm writing API tests and am using the mstest framework. My setup is pretty simple, my cleanup and initialize functions look like below -- stand up and shut down logging instaces. My problem is that some tests throw exceptions that are not caught (this is usually due to test code that is still in development). This causes the test to fail, which I would expect, but I can't manage to get the actual exception object and log it which makes debugging a little difficult. The obvious answer is "wrap your test in a try catch block that handles the logging and gracefully fails the test", but I don't really like that because it just adds another responsibility that future test authors (probably me) poorly implementing the catch, log, and rethrow logic.
TL;DR : How can I, using TestContext (or another solution) get access to an exception that was thrown but not caught by the actual testing logic in the test cleanup method?
public TestContext TestContext { get; set; }
[TestInitialize]
public void TestInitialize()
{
Log.RegisterLogger(new TestLog() {
TestCaseName = TestContext.TestName,
TestCastStart = DateTime.UtcNow
});
}
[TestCleanup]
public void TestCleanup()
{
/*I want to be able to get the details of an uncaught exception here*/
Log.FinalizeLogging();
}
I am using the testing framework that comes with Visual Studio, along with NSubstitute to unit test a method that takes a system ID, and throws an exception if the system can't be found in the database...
public VRTSystem GetSystem(int systemID)
{
VRTSystem system = VrtSystemsRepository.GetVRTSystemByID(systemID);
if (system == null)
{
throw new Exception("System not found");
}
return system;
}
(In case this seems odd, there is a specific business case for this method that requires it to throw an exception, as returning a null system is not acceptable for its usage)
I want to write a test to check that an exception is thrown if the system doesn't exist. I currently have the following...
[TestMethod]
public void LicensingApplicationServiceBusinessLogic_GetSystem_SystemDoesntExist()
{
var bll = new LicensingApplicationServiceBusinessLogic();
try
{
VRTSystem systemReturned = bll.GetSystem(613);
Assert.Fail("Should have thrown an exception, but didn't.);
}
catch () { }
}
By not mocking the repository, the system returned by VrtSystemsRepository.GetVRTSystemByID() will be null, and the exception thrown. Whilst this works, it looks wrong to me. I wouldn't have expected to need a try/catch block in a test.
The NSubstitute docs have an example that implies I should be able to test this as follows...
[TestMethod]
public void GetSystem_SystemDoesntExist()
{
var bll = new LicensingApplicationServiceBusinessLogic();
Assert.Throws<Exception>(() => bll.GetSystem(613));
}
However, if I try this in my test code, I get Throws highlighted in red, with the error message "Assert does not contain definition for Throws"
Now, I'm not actually sure that the sample on that page covers my scenario, as the test code specifies that the method under test throws an exception, which I don't really understand, as I thought the idea of testing was to leave the method under test alone, and test what happens under various scenarios. However, even without that, I don't understand why the Assert.Throws method doesn't exist.
Anyone any ideas?
Edit: DavidG pointed out that Assert.Throws is probably part of NUnit, not the MS framework, which would explain why it's not recognised. If so, is the way I'm currently testing the right way to do it?
As mentioned by DavidG The referenced documentation is using NUnit for assertions.
If not using that framework you can use the ExpectedExceptionAttribute Class
[TestMethod]
[ExpectedException(typeof(<<Your expected exception here>>))]
public void GetSystem_SystemDoesntExist() {
var bll = new LicensingApplicationServiceBusinessLogic();
bll.GetSystem(613);
}
which would fail if the expected exception is not thrown.
I found, that simple Unit test
[TestMethod()]
[ExpectedException(typeof(ArgumentException))]
public void UT_UU()
{
}
gives fault with
Message: Test method did not throw expected exception
System.ArgumentException.
I successfully use [ExpetedException] to test error message output, however, any checking with Assert of side variables in the Unit Test gives it faulted.
Could I somehow decorate my test to avoid it? Or it is "political error"?
with [ExpectedException(typeof(ArgumentException))] your unit test will pass only when the method under test generates an exception of similar type you specified. In your case ArgumentException else it will fail with the message you have posted.
So essentially, you will have to fail your test method with that injection by means of injecting that exception. Currently your test method doesn't throw the expected exception (it actually doesn't perform anything)
For example below test method would pass
[TestMethod()]
[ExpectedException(typeof(ArgumentException))]
public void UT_UU()
{
throw new ArgumentException();
}
Using Assert statements in the body of the test that is decorated with ExpectedException doesn't really fit the pattern well if your test is isolated enough to only be testing one thing. If you follow the "Arrange, Act, Assert" pattern, then Assert is in this case handled by the ExpectedExceptionAttribute itself and the last line of code in the body of the test itself would be the "Act" because it should cause the exception to occur. If you need to know something more specific about the thrown exception to ensure that the system is meeting behavioral expectations, then I would use a try/catch within the test itself to be able to provide more granular inspection:
[TestMethod]
public void UT_UU()
{
// Arrange
var subject = new Foo();
try
{
// Act
subject.Bar();
}
catch(ArgumentException ae)
{
Assert.AreEqual("my message", ae.Message);
return;
}
catch(Exception e)
{
Assert.Fail("Thrown exception was of wrong type"); // would provide more detail here
}
Assert.Fail("exception should have been thrown");
}
Obviously just using ExpectedException is nice because it lets you right a very clean test without extra noise. Alternatively, if you were going to re-use this type of test you could write a derivative of ExpectedException that lets you specify other characteristics (like Message) to check instead of just the type.
I have an application for smoke testing several key services. I want these tests to be easily written and self discovering. To this end, I have created an attribute for each method in the style of [TestMethod] from MSTest. I have also created a object that will find all these methods and execute them within a try-catch. If the method throws an exception I report as a failure, otherwise as a success.
This is all very unit test familiar and that is the intention. So a test would look like...
[MyTestAttribute]
public void TestingTimesAhead()
{
var d = DateTime.MaxValue.AddDays(1);
}
The magic happens in the test object, this has an Action property and a Run method...
public Action TestAction { get; private set; }
public override ITestResult RunTest()
{
try
{
this.TestAction.Invoke();
return new BaseTestResult() { Result = TestResultStatus.Success };
}
catch(Exception ex)
{
return new BaseTestResult() { Result = TestResultStatus.Failure, FailureException = ex};
}
}
When ran in isolation the above test, being wrapped as the Action, will cause an exception and the test fails. Perfect.
However, when I wrap the attributes and test running object up into a dll and reference from a new project, VS Debugger breaks for the exception.
I am presented with an option in that dialog to toggle 'break when this exception type is user-unhandled'.
I can suppress exceptions on a type by type basis, but that isn't going to work for reuse. Also, if I run the release build from the BIN output, the expected behaviour is there - it's just a really awkward development experience.
Is it possible to manage this exception setting from my code instead? Or if anyone can suggest a way around the problem, I'm open to ideas.
I'm working on a project at the moment where I need to inter-operate with code that swallows exceptions. In particular, I'm writing NUnit unit tests. There are some places where I want to embed assertions within code that gets passed as a delegate, as part of mocking a particular behavior. The problem I'm having is that the AssertionException gets swallowed by the code calling the delegate, which means the test passes, even though the test Assert failed.
Is there any way to inform NUnit that a test should fail that can't be circumvented by catching AssertionException? I can't modify the code that swallows the exceptions, as I don't have full ownership and it's already in semi-production use. I'm hoping there's a clean way to accomplish this.
The best I've come up with is something like this:
private static string _assertionFailure;
public static void AssertWrapper(Action action)
{
try
{
action();
}
catch (AssertionException ex)
{
_assertionFailure = ex.Message;
throw;
}
}
[Test]
[ExpectedException(typeof(AssertionException))]
public void TestDefeatSwallowing()
{
Action failure = () => AssertWrapper(() => Assert.Fail("This is a failure"));
EvilSwallowingMethod(failure);
if (_assertionFailure != null)
Assert.Fail(_assertionFailure);
}
private void EvilSwallowingMethod(Action action)
{
try
{
action();
}
catch
{
}
}
It works, but it's pretty ugly. I have to wrap every Assert call and I have to check at the end of every test if an assertion was swallowed.
So you're doing something like this? (this is using Moq syntax)
var dependency1 = new Mock<IDependency1>();
dependency1.Setup(d => d.CalledMethod([Args])
.Callback(TestOutArgsAndPossiblyThrow);
var objectUnderTest = new TestedObject(dependency1.Object);
objectUnderTest.MethodThatCallsIDependency1dotCalledMethod();
And you've got TestOutArgsAndPossiblyThrow encapsulated in your AssertWrapper class?
Unless that's way off, I'd say you're doing it just about right. You have execution re-entering your test at a point where you can record the state of the call to the dependency. Whether that's done via catching exceptions and analyzing them or just directly inspecting the values of the method parameters, you've just gotta do the work. And if you're swallowing exceptions inside the black box, you're going to have to monitor them before they get back into the black box.
I still say you'd be much better off with appropriate logging and notification (you don't have to notify the end users, necessarily). To #TrueWill's point - what do you do when there's an IOException or the database isn't available?
DISCUSSION EDIT
Is your scenario structured like this?
TEST -> TESTED CODE -> SWALLOWING CODE -> THROWING MOCK