I don't need a lesson in switching from recursive to non-recursive means, I just want to know why we can't deal with this type of exception. Regardless, I'm using recursive functions on very large lists.
I have written code to attempt to catch StackOverFlowExceptions:
try { recursiveFxn(100000); }
catch(Exception){}
private void recursiveFxn(int countdown)
{
if (countdown > 0)
recursiveFxn(countdown - 1);
else
throw new Exception("lol. Forced exception.");
}
But still I get program crashes (in both NUnit and a webpage I'm running). Why isn't the exception caught?
Since .NET Framework 2.0, StackOverflowException cannot be caught. This is because it is considered a bad practice. Quoting the MSDN documentation:
Starting with the .NET Framework
version 2.0, a StackOverflowException
object cannot be caught by a try-catch
block and the corresponding process is
terminated by default. Consequently,
users are advised to write their code
to detect and prevent a stack
overflow. For example, if your
application depends on recursion, use
a counter or a state condition to
terminate the recursive loop.
Now, the only way to catch a StackOverflowException is when it was thrown by user code, as explained in a blog by Jared Parsons. Other than that, by hosting the CLR, you can handle (but not catch) a StackOverflowException and devise a way to let the execution of your program continue.
Note that because the stack is unwound when an exception occurs, in pre-2.0 versions of .Net the stack would actually be much shorter when the StackOverflowException is handled, making it possible to do so without generating another StackOverflowException.
You can't catch a stack overflow exception because when it happens it kills the thread dead. Try... catch... is performed by the same thread so that won't work. There may be some lower level APIs that you could P/Invoke and have another thread catch it.
There may also be some lower level APIs to change the maximum stack size, but I don't see anything in the .NET Framework to help with that so again you would need to P/Invoke something.
Related
I have a recursive call to a method that throws a stack overflow exception. The first call is surrounded by a try catch block but the exception is not caught.
Does the stack overflow exception behave in a special way? Can I catch/handle the exception properly?
Not sure if relevant, but additional information:
the exception is not thrown in the main thread
the object where the code is throwing the exception is manually loaded by Assembly.LoadFrom(...).CreateInstance(...)
Starting with 2.0 a StackOverflow Exception can only be caught in the following circumstances.
The CLR is being run in a hosted environment* where the host specifically allows for StackOverflow exceptions to be handled
The stackoverflow exception is thrown by user code and not due to an actual stack overflow situation (Reference)
*"hosted environment" as in "my code hosts CLR and I configure CLR's options" and not "my code runs on shared hosting"
The right way is to fix the overflow, but....
You can give yourself a bigger stack:-
using System.Threading;
Thread T = new Thread(threadDelegate, stackSizeInBytes);
T.Start();
You can use System.Diagnostics.StackTrace FrameCount property to count the frames you've used and throw your own exception when a frame limit is reached.
Or, you can calculate the size of the stack remaining and throw your own exception when it falls below a threshold:-
class Program
{
static int n;
static int topOfStack;
const int stackSize = 1000000; // Default?
// The func is 76 bytes, but we need space to unwind the exception.
const int spaceRequired = 18*1024;
unsafe static void Main(string[] args)
{
int var;
topOfStack = (int)&var;
n=0;
recurse();
}
unsafe static void recurse()
{
int remaining;
remaining = stackSize - (topOfStack - (int)&remaining);
if (remaining < spaceRequired)
throw new Exception("Cheese");
n++;
recurse();
}
}
Just catch the Cheese. ;)
From the MSDN page on StackOverflowExceptions:
In prior versions of the .NET
Framework, your application could
catch a StackOverflowException object
(for example, to recover from
unbounded recursion). However, that
practice is currently discouraged
because significant additional code is
required to reliably catch a stack
overflow exception and continue
program execution.
Starting with the .NET Framework
version 2.0, a StackOverflowException
object cannot be caught by a try-catch
block and the corresponding process is
terminated by default. Consequently,
users are advised to write their code
to detect and prevent a stack
overflow. For example, if your
application depends on recursion, use
a counter or a state condition to
terminate the recursive loop. Note
that an application that hosts the
common language runtime (CLR) can
specify that the CLR unload the
application domain where the stack
overflow exception occurs and let the
corresponding process continue. For
more information, see
ICLRPolicyManager Interface and
Hosting the Common Language Runtime.
As several users have already said, you can't catch the exception. However, if you're struggling to find out where it's happening, you may want to configure visual studio to break when it's thrown.
To do that, you need to open Exception Settings from the 'Debug' menu. In older versions of Visual Studio, this is at 'Debug' - 'Exceptions'; in newer versions, it's at 'Debug' - 'Windows' - 'Exception Settings'.
Once you have the settings open, expand 'Common Language Runtime Exceptions', expand 'System', scroll down and check 'System.StackOverflowException'. Then you can look at the call stack and look for the repeating pattern of calls. That should give you an idea of where to look to fix the code that's causing the stack overflow.
As mentioned above several times, it's not possible to catch a StackOverflowException that was raised by the System due to corrupted process-state. But there's a way to notice the exception as an event:
http://msdn.microsoft.com/en-us/library/system.appdomain.unhandledexception.aspx
Starting with the .NET Framework version 4, this event is not raised for exceptions that corrupt the state of the process, such as stack overflows or access violations, unless the event handler is security-critical and has the HandleProcessCorruptedStateExceptionsAttribute attribute.
Nevertheless your application will terminate after exiting the event-function (a VERY dirty workaround, was to restart the app within this event haha, havn't done so and never will do). But it's good enough for logging!
In the .NET Framework versions 1.0 and 1.1, an unhandled exception that occurs in a thread other than the main application thread is caught by the runtime and therefore does not cause the application to terminate. Thus, it is possible for the UnhandledException event to be raised without the application terminating. Starting with the .NET Framework version 2.0, this backstop for unhandled exceptions in child threads was removed, because the cumulative effect of such silent failures included performance degradation, corrupted data, and lockups, all of which were difficult to debug. For more information, including a list of cases in which the runtime does not terminate, see Exceptions in Managed Threads.
Yes from CLR 2.0 stack overflow is considered a non-recoverable situation. So the runtime still shut down the process.
For details please see the documentation http://msdn.microsoft.com/en-us/library/system.stackoverflowexception.aspx
You can't. The CLR won't let you. A stack overflow is a fatal error and can't be recovered from.
You can't as most of the posts are explaining, let me add another area:
On many websites you will find people saying that the way to avoid this is using a different AppDomain so if this happens the domain will be unloaded. That is absolutely wrong (unless you host your CLR) as the default behavior of the CLR will raise a KillProcess event, bringing down your default AppDomain.
It's impossible, and for a good reason (for one, think about all those catch(Exception){} around).
If you want to continue execution after stack overflow, run dangerous code in a different AppDomain. CLR policies can be set to terminate current AppDomain on overflow without affecting original domain.
I have a recursive call to a method that throws a stack overflow exception. The first call is surrounded by a try catch block but the exception is not caught.
Does the stack overflow exception behave in a special way? Can I catch/handle the exception properly?
Not sure if relevant, but additional information:
the exception is not thrown in the main thread
the object where the code is throwing the exception is manually loaded by Assembly.LoadFrom(...).CreateInstance(...)
Starting with 2.0 a StackOverflow Exception can only be caught in the following circumstances.
The CLR is being run in a hosted environment* where the host specifically allows for StackOverflow exceptions to be handled
The stackoverflow exception is thrown by user code and not due to an actual stack overflow situation (Reference)
*"hosted environment" as in "my code hosts CLR and I configure CLR's options" and not "my code runs on shared hosting"
The right way is to fix the overflow, but....
You can give yourself a bigger stack:-
using System.Threading;
Thread T = new Thread(threadDelegate, stackSizeInBytes);
T.Start();
You can use System.Diagnostics.StackTrace FrameCount property to count the frames you've used and throw your own exception when a frame limit is reached.
Or, you can calculate the size of the stack remaining and throw your own exception when it falls below a threshold:-
class Program
{
static int n;
static int topOfStack;
const int stackSize = 1000000; // Default?
// The func is 76 bytes, but we need space to unwind the exception.
const int spaceRequired = 18*1024;
unsafe static void Main(string[] args)
{
int var;
topOfStack = (int)&var;
n=0;
recurse();
}
unsafe static void recurse()
{
int remaining;
remaining = stackSize - (topOfStack - (int)&remaining);
if (remaining < spaceRequired)
throw new Exception("Cheese");
n++;
recurse();
}
}
Just catch the Cheese. ;)
From the MSDN page on StackOverflowExceptions:
In prior versions of the .NET
Framework, your application could
catch a StackOverflowException object
(for example, to recover from
unbounded recursion). However, that
practice is currently discouraged
because significant additional code is
required to reliably catch a stack
overflow exception and continue
program execution.
Starting with the .NET Framework
version 2.0, a StackOverflowException
object cannot be caught by a try-catch
block and the corresponding process is
terminated by default. Consequently,
users are advised to write their code
to detect and prevent a stack
overflow. For example, if your
application depends on recursion, use
a counter or a state condition to
terminate the recursive loop. Note
that an application that hosts the
common language runtime (CLR) can
specify that the CLR unload the
application domain where the stack
overflow exception occurs and let the
corresponding process continue. For
more information, see
ICLRPolicyManager Interface and
Hosting the Common Language Runtime.
As several users have already said, you can't catch the exception. However, if you're struggling to find out where it's happening, you may want to configure visual studio to break when it's thrown.
To do that, you need to open Exception Settings from the 'Debug' menu. In older versions of Visual Studio, this is at 'Debug' - 'Exceptions'; in newer versions, it's at 'Debug' - 'Windows' - 'Exception Settings'.
Once you have the settings open, expand 'Common Language Runtime Exceptions', expand 'System', scroll down and check 'System.StackOverflowException'. Then you can look at the call stack and look for the repeating pattern of calls. That should give you an idea of where to look to fix the code that's causing the stack overflow.
As mentioned above several times, it's not possible to catch a StackOverflowException that was raised by the System due to corrupted process-state. But there's a way to notice the exception as an event:
http://msdn.microsoft.com/en-us/library/system.appdomain.unhandledexception.aspx
Starting with the .NET Framework version 4, this event is not raised for exceptions that corrupt the state of the process, such as stack overflows or access violations, unless the event handler is security-critical and has the HandleProcessCorruptedStateExceptionsAttribute attribute.
Nevertheless your application will terminate after exiting the event-function (a VERY dirty workaround, was to restart the app within this event haha, havn't done so and never will do). But it's good enough for logging!
In the .NET Framework versions 1.0 and 1.1, an unhandled exception that occurs in a thread other than the main application thread is caught by the runtime and therefore does not cause the application to terminate. Thus, it is possible for the UnhandledException event to be raised without the application terminating. Starting with the .NET Framework version 2.0, this backstop for unhandled exceptions in child threads was removed, because the cumulative effect of such silent failures included performance degradation, corrupted data, and lockups, all of which were difficult to debug. For more information, including a list of cases in which the runtime does not terminate, see Exceptions in Managed Threads.
Yes from CLR 2.0 stack overflow is considered a non-recoverable situation. So the runtime still shut down the process.
For details please see the documentation http://msdn.microsoft.com/en-us/library/system.stackoverflowexception.aspx
You can't. The CLR won't let you. A stack overflow is a fatal error and can't be recovered from.
You can't as most of the posts are explaining, let me add another area:
On many websites you will find people saying that the way to avoid this is using a different AppDomain so if this happens the domain will be unloaded. That is absolutely wrong (unless you host your CLR) as the default behavior of the CLR will raise a KillProcess event, bringing down your default AppDomain.
It's impossible, and for a good reason (for one, think about all those catch(Exception){} around).
If you want to continue execution after stack overflow, run dangerous code in a different AppDomain. CLR policies can be set to terminate current AppDomain on overflow without affecting original domain.
It seems I have stumbled into a situation where an exception is still available as after catch block. It is 100% reproducable in the situation I encounter it in, but I cannot make a test that exactly reproduces this scenario.
See this image for a debugging situation:
Situation is as follows:
ASync task is executed an awaited.
Task is in set to a faulted state and throws an exception.
Code steps through the catch block, handling the exception. <- As expected
Exception is not cleared from stack after catch block has finished. <- Not as expected
Final result: Marshall.GetExceptionCode() != 0 in all code that follows after the catch block.
Does anyone have an idea what is happening here?
The stack doesn't belong to you, it belongs to the runtime. Your .NET code shouldn't depend on the precise way the stack is handled.
Marshal.GetExceptionCode is used exclusively for interop and compiler support, and shouldn't really be used in .NET user code anyway, since you have no idea what happened between two (seemingly innocuous) lines of .NET code.
Stick to the specs, and stop relying on undefined behaviour. It's not like there's a huge amount of undefined behaviour in .NET.
I'm aware that StackOverflowExceptions in .NET can't be caught, take down their process, and have no stack trace. This is officially documented on MSDN. However, I'm wondering what the technical (or other) reasons are behind the behavior. All MSDN says is:
In prior versions of the .NET Framework, your application could catch
a StackOverflowException object (for example, to recover from
unbounded recursion). However, that practice is currently discouraged
because significant additional code is required to reliably catch a
stack overflow exception and continue program execution.
What is this "significant additional code"? Are there other documented reasons for this behavior? Even if we can't catch SOE, why can't we at least get a stack trace? Several co-workers and I just sunk several hours into debugging a production StackOverflowException that would have taken minutes with a stack trace, so I'm wondering if there is a good reason for my suffering.
The stack of a thread is created by Windows. It uses so-called guard pages to be able to detect a stack overflow. A feature that's generally available to user mode code as described in this MSDN Library article. The basic idea is that the last two pages of the stack (2 x 4096 = 8192 bytes) are reserved and any processor access to them triggers a page fault that's turned into an SEH exception, STATUS_GUARD_PAGE_VIOLATION.
This is intercepted by the kernel in the case of those pages belonging to a thread stack. It changes the protection attributes of the first of those 2 pages, thus giving the thread some emergency stack space to deal with the mishap, then re-raises a STATUS_STACK_OVERFLOW exception.
This exception is in turn intercepted by the CLR. At that point there's about 3 kilobytes of stack space left. This is, for one, not enough to run the Just-in-time compiler (JITter) to compile the code that could deal with the exception in your program, the JITter needs much more space than that. The CLR therefore cannot do anything else but rudely abort the thread. And by .NET 2.0 policy that also terminates the process.
Note how this is less of a problem in Java, it has a bytecode interpreter so there's a guarantee that executable user code can run. Or in a non-managed program written in languages like C, C++ or Delphi, code is generated at build time. It is however still a very difficult mishap to deal with, the emergency space in the stack is blown so there is no scenario where continuing to run code on the thread is safe to do. The likelihood that a program can continue operating correctly with a thread aborted at a completely random location and rather corrupted state is quite unlikely.
If there was any effort at all in considering raising an event on another thread or in removing the restriction in the winapi (the number of guard pages is not configurable) then that's either a very well-kept secret or just wasn't considered useful. I suspect the latter, don't know it for a fact.
The stack is where virtually everything about the state of a program is stored. The address of each return site when methods are called, local variables, method parameters, etc. If a method overflows the stack, its execution must, by necessity, stop immediately (since there is no more stack space left for it to continue running). Then, to gracefully recover, somebody needs to clean up whatever that method did to the stack before it died. This means knowing what the stack looked like before the method was called. This incurs some overhead.
And if you can't clean up the stack, then you can't get a stack trace either, because the information required to generate the trace comes from "unrolling" the stack to discover which methods were called.
To handle stack overflow or out-of-memory conditions gracefully, it is necessary to trigger an exception somewhat before the stack has actually overflowed or heap memory is totally exhausted, at a time when the available stack and heap resources will be adequate to execute any cleanup code that will need to run before the exceptions are caught. In the case of stack-overflow exceptions, handling them cleanly would basically require checking the stack pointer on entry to each method (which shouldn't really be all that expensive). Normally, they're handled by setting an access-violation trap just beyond the end of the stack, but the problem with doing that is that the trap won't fire until it's already too late to handle things cleanly. One could set the trap to fire on the last memory block of the stack, rather than the one past, and have the system change the trap to the block past the stack once it fires and triggers a StackOverflowException, but the problem is there would be no nice way to ensure that the "almost out of stack" trap got re-enabled once the stack had unwound that far.
That having been said, an alternative approach would be to allow threads to set a delegate for what should happen if the thread blows its stack, and then say that in case of StackOverflowException the thread's stack will be cleared and it will run the supplied delegate. The trap could be re-instated before running the delegate (the stack would be empty at that point), and code could maintain a thread-status object that the delegate could use to know whether any important finally blocks got skipped.
My understanding is based on this long, but fantastic, article which supports the behavior listed in the C# specification.
The CLI standard (EMCA-335) shows that if there is no suitable catch, the runtime should terminate immediately. The .NET runtime does not do this, instead it seems to lean toward the behavior of the C# specification (EMCA-334).
First, I find it strange that a language specification is appears to be defining framework behavior. Secondly, They seem to contradict.
Do they contradict each other, or am I getting the wrong meaning of the document?
Does a runtime have to go about exception handling in this way to be compliant with the standard?
As an optional question, which one is the "correct" one, as in, if I were to write my own implementation of the CLI which one should I use? Note that EMCA-335 (CLI) document was updated two months ago, where EMCA-334 (C#) was updated back in 2006.
ECMA-335 Partition I Section 12.4.2.5
When an exception occurs, the CLI searches the array for the first protected block that
Protects a region including the current instruction pointer and
Is a catch handler block and
Whose filter wishes to handle the exception
If a match is not found in the current method, the calling method is searched, and so on. If no match is found the CLI will dump a stack trace and abort the program.
If a match is found, the CLI walks the stack back to the point just located, but this time calling the finally and fault handlers. It then starts the corresponding exception handler.
C# Specification §15.9.5 and §15.10 (§8.9.5 and §8.10 on MSDN)
The main difference between it and the CLI standard, is that whether or not a catch block is found, the application will not just exist, but will still unwind the stack, and take care of finally handlers.
I would suggest reading the standard itself to get a better meaning of this, since below is a very crude summary. It outlines step-by-step how a try statement is executed with each possible scenario.
In the function that raises the exception:
Looks for a matching catch clause in each try statement
Executes the catch statement if it exists
A finally block is executed if it exists
If there was no handler, the above steps are repeated in the calling function
If the exception processing terminates all function member invocations in the current thread, indicating that the thread has no handler for the exception, then the thread is itself terminated. The impact of such termination is implementation-defined.
There's no conflict here. The C# language specification is worded like this:
If the try statement has no catch clauses or if no catch clause matches the exception:
• If the try statement has a finally block, the finally block is executed.
• The exception is propagated to the next enclosing try statement.
Bullet 2 here specially doesn't say what happens when there is no next enclosing try statement. For that, turn to the end of 8.9.5:
If the exception processing terminates all function member invocations in the current thread, indicating that the thread has no handler for the exception, then the thread is itself terminated. The impact of such termination is implementation-defined.
It certainly is implementation-defined. Beyond the Ecma 335 spec, the exception handling policy is a configurable item in the Microsoft CLR. Controlled by ICLRPolicyManager::SetActionOnFailure(). In turn configurable in the default host with the <legacyUnhandledExceptionPolicy> app.exe.config file element. The default for CLR version 2.0 and up is to immediately terminate the program.
This is otherwise fairly unproductive biblical hermeneutics. None of this should come as a surprise to a C# programmer, especially given how easy it is to test.
I think this might just be a matter of a vague wording.
If a match is not found in the current method, the calling method is searched, and so on. If no match is found the CLI will dump a stack trace and abort the program.
Okay, that's true in C#. We all know that if we don't have a catch then an exception will bring down our program.
If a match is found, the CLI walks the stack back to the point just located, but this time calling the finally and fault handlers. It then starts the corresponding exception handler.
And that matches what we know from C# too. If there are some finally (we don't get to see fault) blocks to deal with as we go up the stack from the exception being thrown until our catch block, they get processed, but it stops there and goes no further up the stack.
A lot hangs on how we read the "If" that starts that second excerpt I just quoted. You're reading it as "if ... then ... otherwise no such thing". It could be read though as the first excerpt identifying the point in the stack that will be walked to: If there was a catch, then it's walked to that point. If there is no catch, then it's walked to the very top of the stack and we get a dump and abort. The finally handlers (and fault handlers) still get called, but the point is not that of a matching catch handler.
Your reading is the most literal, and mine the one that stretches things a bit. However, mine does match with the description of finally elsewhere in the same standard, most closely
The cited article in the O.P. has an incorrect underlying assumption:
Of course, we can’t talk about managed exceptions without first considering Windows
Structured Exception Handling (SEH). And we also need to look at the C++ exception
model. That’s because both managed exceptions and C++ exceptions are implemented
on top of the underlying SEH mechanism, and because managed exceptions must
interoperate with both SEH and C++ exceptions.
The CLR standard (ISO 23271/ECMA 335) is intentionally platform-agnostic. Microsoft's implementation is one of many possible implementations (Mono, of course, being another).
Interoperability with Windows Structured Exception Handling and C++ exception handling is, I'm pretty sure, Microsoft's choice and not an ISO 23271 requirement.