Why does Assert.Multiple 'remember' an already caught assertion failure? - c#

The following code:
using NUnit.Framework;
namespace Sandbox {
public class Program {
public static void Main(string[] args) {
try {
Assert.Fail("Fail!");
} catch (Exception e) {
Console.WriteLine("Expected exception");
}
try {
Assert.Multiple(() => { Assert.True(true); });
} catch (Exception e) {
Console.WriteLine("Unexpected exception: " + e.Message);
}
}
}
}
prints
Expected exception
Unexpected exception: Fail!
Why is the AssertionException, which is caught in the first block, picked up a second time by Assert.Multiple? If I just do Assert.True(true);, no exception is thrown.
I don't think it matters, but I'm using .NET 6.0.

In (very) old versions, NUnit used an exception in order to (1) report errors and then (2) terminate the tests for the current test class.
Back in 2016, however, these two functions were separated. This had to be done in order to support Assert.Multiple and to a lesser extent the warning asserts Warn.If and Warn.Unless. Those features require execution to be continued after the error is reported.
It should be noted that users have been advised to avoid catching NUnit exceptions for about 20 years. That's because those exceptions are an internal implementation detail, subject to change. I should say that the NUnit developers have been giving that advice for many years. Other people, unfortunately, have often published code that involved catching them. :-)
Anyway, six or seven years ago the inevitable happened: that implementation detail changed.
It's a general principal that you should never try to handle or suppress an exception unless you understand what it means. The meaning of the assertion thrown by a test failure is (now) that the error has been recorded and the fixture is about to be terminated.
That answers your "Why" question but most likely isn't very satisfying. :-)
I suspect this may be an XY problem. Perhaps you might post another question about exactly what you are trying to accomplish and we could have another go at it.

Related

Odd behavior: Catching ThreadAbortException and throwing different exception

As a result of investigating this question: Rethrowing exception in Task doesn't make the Task to go to faulted state, I noticed some very odd behavior with the ThreadAbortException that I can't make sense of.
Now, I know that ThreadAbortException is a very special kind of exception to begin with. And the documentation is pretty clear about that when it says:
ThreadAbortException is a special exception that can be caught, but it will automatically be raised again at the end of the catch block.
Scenario #1: Documented behavior.
static void Main(string[] args)
{
try
{
Thread.CurrentThread.Abort();
}
catch (Exception tae)
{
Console.WriteLine("caught exception: " + tae.GetType().Name);
}
Console.WriteLine("will never be reached");
}
As expected, the ThreadAbortException is rethrown automatically, resulting in the following output:
caught exception: ThreadAbortException
Scenario #2: Where it gets interesting is when I decide to throw a different exception in the catch block:
static void Main(string[] args)
{
try
{
Thread.CurrentThread.Abort();
}
catch (Exception tae)
{
Console.WriteLine("caught exception: " + tae.GetType().Name);
throw new ApplicationException(); // will ThreadAbortException take precedence?
}
Console.WriteLine("will never be reached");
}
In this case, I assumed that, despite the throwing of the ApplicationException, that the ThreadAbortException would be rethrown anyways to ensure that the documented behavior was preserved. To my surprise, this is the output that resulted:
caught exception: ThreadAbortException
Unhandled Exception: System.ApplicationException: Error in the application.
at ConsoleApplication1.Program.Main(String[] args) in C:\projects\ConsoleApplication1\Program.cs:line 193
Did the ApplicationException actually replace and prevent the ThreadAbortException from being thrown?!?
Scenario #3: And finally, to make matters more interesting, I wrap my existing exception handling with one more try-catch layer:
static void Main(string[] args)
{
try
{
try
{
Thread.CurrentThread.Abort();
}
catch (Exception tae)
{
Console.WriteLine("caught exception: " + tae.GetType().Name);
throw new ApplicationException();
}
}
catch (Exception outerEx)
{
Console.WriteLine("caught outer exception: " + outerEx.GetType().Name);
}
Console.WriteLine("will never be reached");
}
Now I'm not too sure what to expect. Which exception will be caught in the outer catch block? Will it be ApplicationException? And if so, does that mean that I'll be able to swallow the exception and actually manage to print out the will never be reached string after all?
This is the actual output:
caught exception: ThreadAbortException
caught outer exception: ApplicationException
From the above output, it looks like the outer catch block does actually catch an ApplicationException. But by the end of that catch block, the ThreadAbortException now all of a sudden reappears out of thin air and gets rethrown?
Question(s): Can someone explain and reconcile scenarios #2 and #3? How is it that in scenario #2, it looks as though the ThreadAbortException is, unexpectedly, replaced by a different exception? Yet, in scenario #3, it looks like ThreadAbortException was still there all along? How is this happening? Is this behavior documented somewhere?
The behavior is implementation specific.
From Ecma-335 VI Annex E, Portability Considerations:
This clause gathers together information about areas where this Standard deliberately leaves leeway to implementations. This leeway is intended to allow compliant implementations to make choices that provide better performance or add value in other ways. But this leeway inherently makes programs non-portable. ...
and further (emphasis mine):
V I . E . 1 Uncontrollable Behavior
The following aspects of program behavior are implementation dependent. >Many of these items
will be familiar to programmers used to writing code designed for >portability (for example, the
fact that the CLI does not impose a minimum size for heap or stack).
Size of heap and stack aren't required to have minimum sizes
Behavior relative to asynchronous exceptions (see System.Thread.Abort)
Globalization is not supported, so every implementation specifies its culture information including such user-visible features as sort order for strings.
Threads cannot be assumed to be either pre -emptively or non-pre-emptively scheduled. This decision is implementation specific.
Locating assemblies is an implementation-specific mechanism.
Security policy is an implemenation-specific mechanism.
File names are implementation-specific.
Timer resolution (granularity) is implementation -specific, although the unit is specified.

Why do my C# unit tests for exceptions fail if I use try/catch?

I'm using C# with nUnit to unit test my project and I'm trying to write some unit tests that ensure a certain Exception is called but my tests keep failing. After a lot of testing to work out the issue, I managed to identify the cause of it and created a small code example that shows my issuew below.
My unit test code:
[Test]
public void TestExceptionIsRaised()
{
var ex = Assert.Throws<UnauthorizedAccessException>(() => TestExceptionMethod());
StringAssert.Contains("Attempted to perform an unauthorized operation", ex.Message);
}
My method with no error checking:
public void TestExceptionMethod()
{
throw new UnauthorizedAccessException();
}
Now if I run that unit test with the above method... it throws the exception, nUnit detects it and the test passes. However, this code is a problem because it has no error handling and will crash if I release it like this.
So to solve that problem, I add some error handling into my method like this:
public void TestExceptionMethod()
{
try
{
throw new UnauthorizedAccessException();
}
catch (UnauthorizedAccessException ex)
{
// Do something about the error
}
}
But now when I run the unit test it fails with this error:
Expected: System.UnauthorizedAccessException
But was: null
So rather than having to choose between a unit test or proper error handling, I tried to re-throw the error like this:
public void TestExceptionMethod()
{
try
{
throw new UnauthorizedAccessException();
}
catch (UnauthorizedAccessException ex)
{
// Do something about the error
throw;
}
}
Finally, everything now works. The unit test passes every time. And my method can still catch the error and do something about it.
However from what I've heard it isn't good practice to keep re-throwing errors like this??? So I'm just wondering is my solution "correct"? Or is there a better way of unit testing for an exception without having to remove the try/catch blocks from the original method?
However from what I've heard it isn't good practice to keep
re-throwing errors like this???
There is nothing wrong with re-throwing exceptions. It's actually a best practice to re-throw the exception after your application runs some logging code without recovering your application state.
Or is there a better way of unit testing for an exception without
having to remove the try/catch blocks from the original method?
I'm not sure I understand what you mean. But if you expect an exception thrown from your method, there must be an exception for your test to pass no matter it's thrown directly or re-thrown. If you try/catch like this:
public void TestExceptionMethod()
{
try
{
throw new UnauthorizedAccessException();
}
catch (UnauthorizedAccessException ex)
{
// Do something about the error
}
}
The test will fail for sure because the exception is swallowed. The test cannot detect there was an exception inside the method.
Translating your test into English you are saying:
"I expect the method TestExceptionMethod to throw a
UnauthorizedAccessException. The message of which will contain a
string with more details."
By not throwing the exception this test will never pass. If you don't want the exception to be thrown you need to change your test. If you do want the exception to be thrown, what you're doing is correct.
Is this correct?
It depends. If what you're doing is an exceptional issue, then throw an exception. Let the program crash. Fix the bug.
On the other hand if this is something that is to be expected such as bad user input, boundary conditions, handle it if you can. If you cannot recover from the problem then throw an exception.
Exception handling done poorly can make debugging more difficult especially when the exception is swallowed, or when a new exception instance is created and thrown losing the stack trace of the original exception. Sometimes the latter is desirable when you need a more abstract exception higher up the call stack, in which case you can add the original exception as an inner exception to keep access to the stack trace.

Unhandled Exception in C# Console Application causing AppCrash

I have a Windows Console application built in Visual Studio 2010 and it keeps crashing but the error is not caught by the visual studio debugging tool nor by try/catch statements in my code.
I have managed to locate the WER file on my system and would like to be able to understand the contents of the file so I can pinpoint exactally what is causing the unhandled exception.
I would be greatful if anyone can offer some idea on how I can use the following information to locate the process causing me this problem and also what the exception may be...
The information from the WER file is:
Version=1
EventType=APPCRASH
EventTime=129973086237604286
ReportType=2
Consent=1
ReportIdentifier=91331e8b-2dc8-11e2-977b-080027f7e5bb
IntegratorReportIdentifier=91331e8a-2dc8-11e2-977b-080027f7e5bb
WOW64=1
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=SAGE_TESTING.vshost.exe
Sig[1].Name=Application Version
Sig[1].Value=10.0.30319.1
Sig[2].Name=Application Timestamp
Sig[2].Value=4ba2084b
Sig[3].Name=Fault Module Name
Sig[3].Value=ntdll.dll
Sig[4].Name=Fault Module Version
Sig[4].Value=6.1.7600.16385
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=4a5bdb3b
Sig[6].Name=Exception Code
Sig[6].Value=c015000f
Sig[7].Name=Exception Offset
Sig[7].Value=000845bb
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=6.1.7600.2.0.0.272.7
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=2057
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=0a9e
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=0a9e372d3b4ad19135b953a78882e789
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=0a9e
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=0a9e372d3b4ad19135b953a78882e789
Here is the section of code I believe to be causing the exception to be thrown:
//Data from the project linked to the split data
if (oSplitData.Project != null)
{
oProject = oSplitData.Project as SageDataObject190.Project;
oBasicDetail.ProjectID = oProject.ProjectID;
oBasicDetail.ProjectReference = oProject.Reference.ToString();
}
else
{
oBasicDetail.ProjectID = -1;
oBasicDetail.ProjectReference = "NO_PROJECT";
}
To add to all the above I seem to have found that there is a general exception that is being thrown but it doesn't help me out much - if anyone can put some light on this it would be great:
Unhandled exception at 0x78bc7361 in SAGE_TESTING.exe: 0xC0000005: Access violation reading location 0xfeeefeee.
If your program is multi-threaded and the exception is thrown in one of the spawned threads, the Exception may not be caught depending on how you do exception handling in your program.
You can add a catch-all exception handler like this:
class Program
{
static void Main(string[] args)
{
AppDomain.CurrentDomain.UnhandledException += UnhandledExceptionHandler;
// Your code here
}
static void UnhandledExceptionHandler(object sender, UnhandledExceptionEventArgs e)
{
Console.WriteLine(e.ExceptionObject.ToString());
Environment.Exit(1);
}
}
UPDATE
Based on the code you posted, here are some things to look at
Put a try/catch block around the code you posted.
Are you sure that oSplitData is not null?
In the following line, oProject will be null if oSplitData.Project is not of type SageDataObject190.Project. Test for null.
oProject = oSplitData.Project as SageDataObject190.Project;
You are probably dealing with so-called corrupted state exceptions. These exceptions corrupt the process in a way so it is usually more safe to kill the process since it is very difficult to impossible to recover from such an error, even if it would be only for running a short catch-clause. Examples are StackOverflowExceptions, OutOfMemoryExceptions or AccessViolationExceptions.
There is an extensive and generally interesting explanation on corrupted state exceptions in this article.
What is helpful on getting a hand on such exceptions is to use DebugDiag. With this tool from Microsoft (download on this page) you can define a crash rule which generates a crashdump for your failed process. You can easily open these dump files in Visual Studio, where you may find the source of the exception that lead to the failure. This is not guaranteed but it often helped me in the past to nail down some nasty errors.
Are you invoking non-managed C++ or other code?
I'd try something like
static void Main()
{
try
{
DoSomethingUseful() ;
}
catch ( Exception e )
{
// managed exceptions caught here
}
catch
{
// non-managed C++ or other code can throw non-exception objects
// they are caught here.
}
return ;
}
See Will CLR handle both CLS-Complaint and non-CLS complaint exceptions?
Also C++ try, catch and throw statements at msdn: http://msdn.microsoft.com/en-us/library/6dekhbbc(v=vs.100).aspx
And MSIL opcode throw (0x7A) allows the throwing any object reference. C#, however, does not allow it.
But it looks like they improved things with .Net 2.0 and started wrapping oddball stuff in an RuntimeWrappedException.

Unit test throws exception but cannot see the message

Hi all I'm making a Chess AI as hobby project with Test Driven Development. But for some reason the only message my tests give is "test has thrown an exception: ...". And therefore omitting the only thing that matters. Instead of directly reading the error I now have to right-click and view test result details. I have tried adding and removing columns, but I cannot get the whole message to be shown directly.
Can VS2010 be setup so the exception message gets shown directly for each unit test?
edit: I am using standaard VS unit tests:
[TestClass]
public class MiniMaxTest
{
[TestMethod]
public void TestConstructor()
{
throw new Exception("Must I click view details to see this?");
}
}
Why these questions? You guys can reproduce these things. Debug or Run tests give the same messages:
No, I don't believe you can configure VS to show it differently. The IDE shows the first line of the exception message, and there's a newline character in the full message text, so you'll need to click through to the details to view the whole thing.
What you can do however, is abuse the MSTest framework to show your own messages.
MS Test Assertions are all implemented by throwing exceptions. All the MS Test Assert functions throw exceptions which are derived from UnitTestAssertException. Visual studio has special handling for these kinds of exceptions.
For example: If you write this test:
[TestMethod]
public void AllAboard()
{
throw new AssertFailedException("Failboat");
}
AssertFailedException is the standard base class for most other assertion failures.
You'll note that VS2010 does not print the generic "test threw an exception" message, instead it just prints "Failboat".
Now what you can do is surround your tests in things that convert normal exceptions into AssertFailedException and then you can print whatever messages you like.
[TestMethod]
public void TestStuff()
{
try
{
string o = null; // pretend this came from some real code
var x = o.Split(new[] { ',' }); // this will throw NullRefException
}
catch (Exception e)
{
throw new AssertFailedException(e.Message, e);
}
}
Of course, I wouldn't recommend actually doing this... it's verbose, and more importantly, you lose the call stack... but hey, now you've got one more tool in your toolbelt

Common programming mistakes in .Net when handling exceptions? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
What are some of the most common mistakes you've seen made when handling exceptions?
It seems like exception handling can be one of the hardest things to learn how to do "right" in .Net. Especially considering the currently #1 ranked answer to Common programming mistakes for .NET developers to avoid? is related to exception handling.
Hopefully by listing some of the most common mistakes we can all learn to handle exceptions better.
What are some of the most common mistakes you've seen made when handling exceptions?
I can think of lots.
First read my article on categorization of exceptions into vexing, boneheaded, fatal and exogenous:
http://ericlippert.com/2008/09/10/vexing-exceptions/
Some common errors:
Failure to handle exogenous exceptions.
Failure to handle vexing exceptions.
Construction of methods that throw vexing exceptions.
Handling exceptions that you actually cannot handle, like fatal exceptions.
Handling exceptions that hide bugs in your code; don't handle a boneheaded exception, fix the bug so that it isn't thrown in the first place
Security error: failure to the unsafe mode
try
{
result = CheckPassword();
if (result == BadPassword) throw BadPasswordException();
}
catch(BadPasswordException ex) { ReportError(ex); return; }
catch(Exception ex) { LogException(ex); }
AccessUserData();
See what happened? We failed to the unsafe mode. If CheckPassword threw NetworkDriverIsAllMessedUpException then we caught it, logged it, and accessed the user's data regardless of whether the password was correct. Fail to the safe mode; when you get any exception, assume the worst.
Security error: production of exceptions which leak sensitive information, directly or indirectly.
This isn't exactly about handling exceptions in your code, it's about producing exceptions which are handled by hostile code.
Funny story. Before .NET 1.0 shipped to customers we found a bug where it was possible to call a method that threw the exception "the assembly which called this method does not have permission to determine the name of file C:\foo.txt". Great. Thanks for letting me know. What is stopping said assembly from catching the exception and interrogating its message to get the file name? Nothing. We fixed that before we shipped.
That's a direct problem. An indirect problem would be a problem I implemented in LoadPicture, in VBScript. It gave a different error message depending upon whether the incorrect argument is a directory, a file that isn't a picture, or a file that doesn't exist. Which means you could use it as a very slow disk browser! By trying a whole bunch of different things you could gradually build up a picture of what files and directories were on someone's hard disk. Exceptions should be designed so that if they are handled by untrustworthy code, that code learns none of the user's private information from whatever they did to cause the exception. (LoadPicture now gives much less helpful error messages.)
Security and resource management error: Handlers which do not clean up resources are resource leaks waiting to happen. Resource leaks can be used as denial-of-service attacks by hostile partial trust code which deliberately creates exceptions-producing situations.
Robustness error: Handlers must assume that program state is messed up unless handling a specific exogenous exception. This is particularly true of finally blocks. When you're handling an unexpected exception, it is entirely possible and even likely that something is deeply messed up in your program. You have no idea if any of your subsystems are working, and if they are, whether calling them will make the situation better or worse. Concentrate on logging the error and saving user data if possible and shut down as cleanly as you can. Assume that nothing works right.
Security error: temporary global state mutations that have security impacts need to be undone before any code that might be hostile can run. Hostile code can run before finally blocks run! See my article on this for details:
http://blogs.msdn.com/ericlippert/archive/2004/09/01/224064.aspx
Re-throwing exceptions like this:
try
{
// some code here
}
catch(Exception ex)
{
// logging, etc
throw ex;
}
This kills the stack trace, making is far less usable. The correct way to rethrow would be like this:
try
{
// some code here
}
catch(Exception ex)
{
// logging, etc
throw;
}
Catching all exceptions when in many cases you should attempt to catch specific exceptions:
try {
// Do something.
} catch (Exception exc) {
// Do something.
}
Rather than:
try {
// Do something.
} catch (IOException exc) {
// Do something.
}
Exceptions should be ordered from most specific to least.
Rethrowing an exception with a meaningless message.
try
{
...
}
catch (Exception ex)
{
throw new Exception("An error ocurred when saving database changes").
}
You won't believe how often I see code like this running in production.
Nobody is talking about seeing empty catch blocks like these....
try{
//do something
}
catch(SQLException sqex){
// do nothing
}
Also never use Exception handling for creating alternate method flows...
try{
//do something
}catch(SQLException sqex){
//do something else
}
Not using using on IDisposable objects:
File myFile = File.Open("some file");
callSomeMethodWhichThrowsException(myFile);
myFile.Close();
myFile does not get closed until myFile's finalizer is called (which may be never) because an exception was thrown before myFile.Close() was called.
The proper way to do this is
using(File myFile = File.Open("some file"))
{
callSomeMethodWhichThrowsException(myFile);
}
This gets translated by the compiler into something like:
File myFile = File.Open("some file");
try
{
callSomeMethodWhichThrowsException(myFile);
}
finally
{
if(myFile != null)
myFile.Dispose(); //Dispose() calls Close()
}
So the file gets closed even in the face of exceptions.
Forget to set the inner exception when rethrowing a catched exception
try
{
...
}
catch (IOException ioException)
{
throw new AppSpecificException("It was not possible to save exportation file.")
// instead of
throw new AppSpecificException("It was not possible to save exportation file.", ioException);
}
When I posted this answer, I forget to mention that we should always consider when to include inner exception or not due to security reasons. As Eric Lippert pointed out on another answer for this topic, some exceptions can provide sensitive information about the implementation details of the server. Thus, if the caller who will be handling the exception is not trusted, it is not a good idea to include the inner exception information.
Empty catch:
//What's the point?
catch()
{}
Rethrowing:
//Exceptions are for *adding* detail up the stack
catch (Exception ex)
{throw ex;}
Assuming an exception that covers many scenarios was something specific. A real life scenario was a web app where the exception handling always assumed all errors were session time outs and logged and reported all errors as session time outs.
Another example:
try
{
Insert(data);
}
catch (SqlException e)
{
//oh this is a duplicate row, lets change to update
Update(data);
}
To log Exception.Message instead of Exception.ToString()
Many times, I see code logging only the exception message while it should log the return of ToString method. ToString provides much more information about the exception than the Message. It includes information like inner exception and stack trace besides the message.
Trying to catch OutOfMemoryException or StackOverflowException - those lead to a shutdown of the runtime, hence to way to catch them from within the same Process (or even from the CLR as a whole?)
OutOfMemoryException: The exception that is thrown when there is not enough memory to continue the execution of a program.
"Starting with the .NET Framework version 2.0, a StackOverflowException object cannot be caught by a try-catch block and the corresponding process is terminated by default. Consequently, users are advised to write their code to detect and prevent a stack overflow."
Failing to catch possible exceptions inside a catch handler. This can cause the wrong exception to be propagated upwards.
For example:
try
{
DoImportantWork();
}
catch
{
Cleanup();
throw;
}
What happens if Cleanup() throws an exception? You don't want to see an Exception pointing to the Cleanup() method in this catch handler. You want the original error. You could try to log the cleanup error, but even your logging code needs exception handling to avoid throwing exceptions from it.
try
{
DoImportantWork();
}
catch
{
try
{
Cleanup();
}
catch
{
// We did our best to clean up, and even that failed.
// If you try to log this error, the logging may throw yet another Exception.
}
throw;
}
Wrong
try
{
// Do something stupid
}
catch
{
// ignore the resulting error because I'm lazy, and later spend
// a week trying to figure out why my app is crashing all over
// the place.
}
Better
try
{
/// do something silly.
}
catch (InvalidOperationException ex)
{
/// respond, or log it.
}
catch (Exception e)
{
/// log it.
}
Using exceptions for normal flow control. Exceptions should exceptional. If it's a good / expected operation, use return values, etc.

Categories

Resources