Related
I've built a program that
takes in a list of record data from a file
parses and cleans up each record in a parsing object
outputs it to an output file
So far this has worked on a single thread, but considering the fact that records can exceed 1 million in some cases, we want to implement this in a multi threading context. Multi threading is new to me in .Net, and I've given it a shot but its not working. Below I will provide more details and code:
Main Class (simplified):
public class MainClass
{
parseObject[] parseObjects;
Thread[] threads;
List<InputLineItem> inputList = new List<InputLineItem>();
FileUtils fileUtils = new FileUtils();
public GenParseUtilsThreaded(int threadCount)
{
this.threadCount = threadCount;
Init();
}
public void Init()
{
inputList = fileUtils.GetInputList();
parseObjects = new parseObject[threadCount - 1];
threads = new Thread[threadCount - 1];
InitParseObjects();
Parse();
}
private void InitParseObjects()
{
//using a ref of fileUtils to use as my lock expression
parseObjects[0] = new ParseObject(ref fileUtils);
parseObjects[0].InitValues();
for (int i = 1; i < threadCount - 1; i++)
{
parseObjects[i] = new parseObject(ref fileUtils);
parseObjects[i].InitValues();
}
}
private void InitThreads()
{
for (int i = 0; i < threadCount - 1; i++)
{
Thread t = new Thread(new ThreadStart(parseObjects[0].CleanupAndParseInput));
threads[i] = t;
}
}
public void Parse()
{
try
{
InitThreads();
int objectIndex = 0;
foreach (InputLineItem inputLineItem in inputList)
{
parseObjects[0].inputLineItem = inputLineItem;
threads[objectIndex].Start();
objectIndex++;
if (objectIndex == threadCount)
{
objectIndex = 0;
InitThreads(); //do i need to re-init the threads after I've already used them all once?
}
}
}
catch (Exception e)
{
Console.WriteLine("(286) The following error occured: " + e);
}
}
}
}
And my Parse object class (also simplified):
public class ParseObject
{
public ParserLibrary parser { get; set; }
public FileUtils fileUtils { get; set; }
public InputLineItem inputLineItem { get; set; }
public ParseObject( ref FileUtils fileUtils)
{
this.fileUtils = fileUtils;
}
public void InitValues()
{
//relevant config of parser library object occurs here
}
public void CleanupFields()
{
parser.Clean(inputLineItem.nameValue);
inputLineItem.nameValue = GetCleanupUpValueFromParser();
}
private string GetCleanupFieldValue()
{
//code to extract cleanup up value from parses
}
public void CleanupAndParseInput()
{
CleanupFields();
ParseInput();
}
public void ParseInput()
{
try
{
parser.Parse(InputLineItem.NameValue);
}
catch (Exception e)
{
}
try
{
lock (fileUtils)
{
WriteOutputToFile(inputLineItem);
}
}
catch (Exception e)
{
Console.WriteLine("(414) Failed to write to output: " + e);
}
}
public void WriteOutputToFile(InputLineItem inputLineItem)
{
//writes updated value to output file
}
}
The error I get is when trying to run the Parse function, I get this message:
An unhandled exception of type 'System.AccessViolationException' occurred in GenParse.NET.dll
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
That being said, I feel like there's a whole lot more that I'm doing wrong here aside from what is causing that error.
I also have further questions:
Do I create multiple parse objects and iteratively feed them to each thread as I'm attempting to do, or should I use one Parse object that gets shared or cloned across each thread?
If, outside the thread, I change a value in the object that I'm passing to the thread, will that change reflect in the object passed to the thread? i.e, is the object passed by value or reference?
Is there a more efficient way for each record to be assigned to a thread and its parse object than I am currently doing with the objectIndex iterator?
THANKS!
Do I create multiple parse objects and iteratively feed them to each thread as I'm attempting to do, or should I use one Parse object that gets shared or cloned across each thread?
You initialize each thread with new ThreadStart(parseObjects[0].CleanupAndParseInput) so all threads will share the same parse object. It is a fairly safe bet that the parse objects are not threadsafe. So each thread should have a separate object. Note that this might not be sufficient, if the parse library uses any global fields it might be non-threadsafe even when using separate objects.
If, outside the thread, I change a value in the object that I'm passing to the thread, will that change reflect in the object passed to the thread? i.e, is the object passed by value or reference?
Objects (i.e. classes) are passed by reference. But any changes to an object are not guaranteed to be visible in other threads unless a memoryBarrier is issued. Most synchronization code (like lock) will issue memory barriers. Keep in mind that any non-atomic operation is unsafe if a field is written an read concurrently.
Is there a more efficient way for each record to be assigned to a thread and its parse object than I am currently doing with the objectIndex iterator?
Using manual threads in this way is very old-school. The modern, easier, and probably faster way is to use a parallel-for loop. This will try to be smart about how many threads it will use and try to adapt chunk sizes to keep the synchronization overhead low.
var items = new List<int>();
ParseObject LocalInit()
{
// Do initalization, This is run once for each thread used
return new ParseObject();
}
ParseObject ThreadMain(int value, ParallelLoopState state, ParseObject threadLocalObject)
{
// Do whatever you need to do
// This is run on multiple threads
return threadLocalObject;
}
void LocalFinally(ParseObject obj)
{
// Do Cleanup for each thread
}
Parallel.ForEach(items, LocalInit, ThreadMain, LocalFinally);
As a final note, I would advice against using multithreading unless you are familiar with the potential dangers and pitfalls it involves, at least for any project where the result is important. There are many ways to screw up and make a program that will work 99.9% of the time, and silently corrupt data the remaining 0.1% of the time.
I would like to either prevent or handle a StackOverflowException that I am getting from a call to the XslCompiledTransform.Transform method within an Xsl Editor I am writing. The problem seems to be that the user can write an Xsl script that is infinitely recursive, and it just blows up on the call to the Transform method. (That is, the problem is not just the typical programmatic error, which is usually the cause of such an exception.)
Is there a way to detect and/or limit how many recursions are allowed? Or any other ideas to keep this code from just blowing up on me?
From Microsoft:
Starting with the .NET Framework
version 2.0, a StackOverflowException
object cannot be caught by a try-catch
block and the corresponding process is
terminated by default. Consequently,
users are advised to write their code
to detect and prevent a stack
overflow. For example, if your
application depends on recursion, use
a counter or a state condition to
terminate the recursive loop.
I'm assuming the exception is happening within an internal .NET method, and not in your code.
You can do a couple things.
Write code that checks the xsl for infinite recursion and notifies the user prior to applying a transform (Ugh).
Load the XslTransform code into a separate process (Hacky, but less work).
You can use the Process class to load the assembly that will apply the transform into a separate process, and alert the user of the failure if it dies, without killing your main app.
EDIT: I just tested, here is how to do it:
MainProcess:
// This is just an example, obviously you'll want to pass args to this.
Process p1 = new Process();
p1.StartInfo.FileName = "ApplyTransform.exe";
p1.StartInfo.UseShellExecute = false;
p1.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
p1.Start();
p1.WaitForExit();
if (p1.ExitCode == 1)
Console.WriteLine("StackOverflow was thrown");
ApplyTransform Process:
class Program
{
static void Main(string[] args)
{
AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(CurrentDomain_UnhandledException);
throw new StackOverflowException();
}
// We trap this, we can't save the process,
// but we can prevent the "ILLEGAL OPERATION" window
static void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
if (e.IsTerminating)
{
Environment.Exit(1);
}
}
}
NOTE The question in the bounty by #WilliamJockusch and the original question are different.
This answer is about StackOverflow's in the general case of third-party libraries and what you can/can't do with them. If you're looking about the special case with XslTransform, see the accepted answer.
Stack overflows happen because the data on the stack exceeds a certain limit (in bytes). The details of how this detection works can be found here.
I'm wondering if there is a general way to track down StackOverflowExceptions. In other words, suppose I have infinite recursion somewhere in my code, but I have no idea where. I want to track it down by some means that is easier than stepping through code all over the place until I see it happening. I don't care how hackish it is.
As I mentioned in the link, detecting a stack overflow from static code analysis would require solving the halting problem which is undecidable. Now that we've established that there is no silver bullet, I can show you a few tricks that I think helps track down the problem.
I think this question can be interpreted in different ways, and since I'm a bit bored :-), I'll break it down into different variations.
Detecting a stack overflow in a test environment
Basically the problem here is that you have a (limited) test environment and want to detect a stack overflow in an (expanded) production environment.
Instead of detecting the SO itself, I solve this by exploiting the fact that the stack depth can be set. The debugger will give you all the information you need. Most languages allow you to specify the stack size or the max recursion depth.
Basically I try to force a SO by making the stack depth as small as possible. If it doesn't overflow, I can always make it bigger (=in this case: safer) for the production environment. The moment you get a stack overflow, you can manually decide if it's a 'valid' one or not.
To do this, pass the stack size (in our case: a small value) to a Thread parameter, and see what happens. The default stack size in .NET is 1 MB, we're going to use a way smaller value:
class StackOverflowDetector
{
static int Recur()
{
int variable = 1;
return variable + Recur();
}
static void Start()
{
int depth = 1 + Recur();
}
static void Main(string[] args)
{
Thread t = new Thread(Start, 1);
t.Start();
t.Join();
Console.WriteLine();
Console.ReadLine();
}
}
Note: we're going to use this code below as well.
Once it overflows, you can set it to a bigger value until you get a SO that makes sense.
Creating exceptions before you SO
The StackOverflowException is not catchable. This means there's not much you can do when it has happened. So, if you believe something is bound to go wrong in your code, you can make your own exception in some cases. The only thing you need for this is the current stack depth; there's no need for a counter, you can use the real values from .NET:
class StackOverflowDetector
{
static void CheckStackDepth()
{
if (new StackTrace().FrameCount > 10) // some arbitrary limit
{
throw new StackOverflowException("Bad thread.");
}
}
static int Recur()
{
CheckStackDepth();
int variable = 1;
return variable + Recur();
}
static void Main(string[] args)
{
try
{
int depth = 1 + Recur();
}
catch (ThreadAbortException e)
{
Console.WriteLine("We've been a {0}", e.ExceptionState);
}
Console.WriteLine();
Console.ReadLine();
}
}
Note that this approach also works if you are dealing with third-party components that use a callback mechanism. The only thing required is that you can intercept some calls in the stack trace.
Detection in a separate thread
You explicitly suggested this, so here goes this one.
You can try detecting a SO in a separate thread.. but it probably won't do you any good. A stack overflow can happen fast, even before you get a context switch. This means that this mechanism isn't reliable at all... I wouldn't recommend actually using it. It was fun to build though, so here's the code :-)
class StackOverflowDetector
{
static int Recur()
{
Thread.Sleep(1); // simulate that we're actually doing something :-)
int variable = 1;
return variable + Recur();
}
static void Start()
{
try
{
int depth = 1 + Recur();
}
catch (ThreadAbortException e)
{
Console.WriteLine("We've been a {0}", e.ExceptionState);
}
}
static void Main(string[] args)
{
// Prepare the execution thread
Thread t = new Thread(Start);
t.Priority = ThreadPriority.Lowest;
// Create the watch thread
Thread watcher = new Thread(Watcher);
watcher.Priority = ThreadPriority.Highest;
watcher.Start(t);
// Start the execution thread
t.Start();
t.Join();
watcher.Abort();
Console.WriteLine();
Console.ReadLine();
}
private static void Watcher(object o)
{
Thread towatch = (Thread)o;
while (true)
{
if (towatch.ThreadState == System.Threading.ThreadState.Running)
{
towatch.Suspend();
var frames = new System.Diagnostics.StackTrace(towatch, false);
if (frames.FrameCount > 20)
{
towatch.Resume();
towatch.Abort("Bad bad thread!");
}
else
{
towatch.Resume();
}
}
}
}
}
Run this in the debugger and have fun of what happens.
Using the characteristics of a stack overflow
Another interpretation of your question is: "Where are the pieces of code that could potentially cause a stack overflow exception?". Obviously the answer of this is: all code with recursion. For each piece of code, you can then do some manual analysis.
It's also possible to determine this using static code analysis. What you need to do for that is to decompile all methods and figure out if they contain an infinite recursion. Here's some code that does that for you:
// A simple decompiler that extracts all method tokens (that is: call, callvirt, newobj in IL)
internal class Decompiler
{
private Decompiler() { }
static Decompiler()
{
singleByteOpcodes = new OpCode[0x100];
multiByteOpcodes = new OpCode[0x100];
FieldInfo[] infoArray1 = typeof(OpCodes).GetFields();
for (int num1 = 0; num1 < infoArray1.Length; num1++)
{
FieldInfo info1 = infoArray1[num1];
if (info1.FieldType == typeof(OpCode))
{
OpCode code1 = (OpCode)info1.GetValue(null);
ushort num2 = (ushort)code1.Value;
if (num2 < 0x100)
{
singleByteOpcodes[(int)num2] = code1;
}
else
{
if ((num2 & 0xff00) != 0xfe00)
{
throw new Exception("Invalid opcode: " + num2.ToString());
}
multiByteOpcodes[num2 & 0xff] = code1;
}
}
}
}
private static OpCode[] singleByteOpcodes;
private static OpCode[] multiByteOpcodes;
public static MethodBase[] Decompile(MethodBase mi, byte[] ildata)
{
HashSet<MethodBase> result = new HashSet<MethodBase>();
Module module = mi.Module;
int position = 0;
while (position < ildata.Length)
{
OpCode code = OpCodes.Nop;
ushort b = ildata[position++];
if (b != 0xfe)
{
code = singleByteOpcodes[b];
}
else
{
b = ildata[position++];
code = multiByteOpcodes[b];
b |= (ushort)(0xfe00);
}
switch (code.OperandType)
{
case OperandType.InlineNone:
break;
case OperandType.ShortInlineBrTarget:
case OperandType.ShortInlineI:
case OperandType.ShortInlineVar:
position += 1;
break;
case OperandType.InlineVar:
position += 2;
break;
case OperandType.InlineBrTarget:
case OperandType.InlineField:
case OperandType.InlineI:
case OperandType.InlineSig:
case OperandType.InlineString:
case OperandType.InlineTok:
case OperandType.InlineType:
case OperandType.ShortInlineR:
position += 4;
break;
case OperandType.InlineR:
case OperandType.InlineI8:
position += 8;
break;
case OperandType.InlineSwitch:
int count = BitConverter.ToInt32(ildata, position);
position += count * 4 + 4;
break;
case OperandType.InlineMethod:
int methodId = BitConverter.ToInt32(ildata, position);
position += 4;
try
{
if (mi is ConstructorInfo)
{
result.Add((MethodBase)module.ResolveMember(methodId, mi.DeclaringType.GetGenericArguments(), Type.EmptyTypes));
}
else
{
result.Add((MethodBase)module.ResolveMember(methodId, mi.DeclaringType.GetGenericArguments(), mi.GetGenericArguments()));
}
}
catch { }
break;
default:
throw new Exception("Unknown instruction operand; cannot continue. Operand type: " + code.OperandType);
}
}
return result.ToArray();
}
}
class StackOverflowDetector
{
// This method will be found:
static int Recur()
{
CheckStackDepth();
int variable = 1;
return variable + Recur();
}
static void Main(string[] args)
{
RecursionDetector();
Console.WriteLine();
Console.ReadLine();
}
static void RecursionDetector()
{
// First decompile all methods in the assembly:
Dictionary<MethodBase, MethodBase[]> calling = new Dictionary<MethodBase, MethodBase[]>();
var assembly = typeof(StackOverflowDetector).Assembly;
foreach (var type in assembly.GetTypes())
{
foreach (var member in type.GetMembers(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.Instance).OfType<MethodBase>())
{
var body = member.GetMethodBody();
if (body!=null)
{
var bytes = body.GetILAsByteArray();
if (bytes != null)
{
// Store all the calls of this method:
var calls = Decompiler.Decompile(member, bytes);
calling[member] = calls;
}
}
}
}
// Check every method:
foreach (var method in calling.Keys)
{
// If method A -> ... -> method A, we have a possible infinite recursion
CheckRecursion(method, calling, new HashSet<MethodBase>());
}
}
Now, the fact that a method cycle contains recursion, is by no means a guarantee that a stack overflow will happen - it's just the most likely precondition for your stack overflow exception. In short, this means that this code will determine the pieces of code where a stack overflow can occur, which should narrow down most code considerably.
Yet other approaches
There are some other approaches you can try that I haven't described here.
Handling the stack overflow by hosting the CLR process and handling it. Note that you still cannot 'catch' it.
Changing all IL code, building another DLL, adding checks on recursion. Yes, that's quite possible (I've implemented it in the past :-); it's just difficult and involves a lot of code to get it right.
Use the .NET profiling API to capture all method calls and use that to figure out stack overflows. For example, you can implement checks that if you encounter the same method X times in your call tree, you give a signal. There's a project clrprofiler that will give you a head start.
I would suggest creating a wrapper around XmlWriter object, so it would count amount of calls to WriteStartElement/WriteEndElement, and if you limit amount of tags to some number (f.e. 100), you would be able to throw a different exception, for example - InvalidOperation.
That should solve the problem in the majority of the cases
public class LimitedDepthXmlWriter : XmlWriter
{
private readonly XmlWriter _innerWriter;
private readonly int _maxDepth;
private int _depth;
public LimitedDepthXmlWriter(XmlWriter innerWriter): this(innerWriter, 100)
{
}
public LimitedDepthXmlWriter(XmlWriter innerWriter, int maxDepth)
{
_maxDepth = maxDepth;
_innerWriter = innerWriter;
}
public override void Close()
{
_innerWriter.Close();
}
public override void Flush()
{
_innerWriter.Flush();
}
public override string LookupPrefix(string ns)
{
return _innerWriter.LookupPrefix(ns);
}
public override void WriteBase64(byte[] buffer, int index, int count)
{
_innerWriter.WriteBase64(buffer, index, count);
}
public override void WriteCData(string text)
{
_innerWriter.WriteCData(text);
}
public override void WriteCharEntity(char ch)
{
_innerWriter.WriteCharEntity(ch);
}
public override void WriteChars(char[] buffer, int index, int count)
{
_innerWriter.WriteChars(buffer, index, count);
}
public override void WriteComment(string text)
{
_innerWriter.WriteComment(text);
}
public override void WriteDocType(string name, string pubid, string sysid, string subset)
{
_innerWriter.WriteDocType(name, pubid, sysid, subset);
}
public override void WriteEndAttribute()
{
_innerWriter.WriteEndAttribute();
}
public override void WriteEndDocument()
{
_innerWriter.WriteEndDocument();
}
public override void WriteEndElement()
{
_depth--;
_innerWriter.WriteEndElement();
}
public override void WriteEntityRef(string name)
{
_innerWriter.WriteEntityRef(name);
}
public override void WriteFullEndElement()
{
_innerWriter.WriteFullEndElement();
}
public override void WriteProcessingInstruction(string name, string text)
{
_innerWriter.WriteProcessingInstruction(name, text);
}
public override void WriteRaw(string data)
{
_innerWriter.WriteRaw(data);
}
public override void WriteRaw(char[] buffer, int index, int count)
{
_innerWriter.WriteRaw(buffer, index, count);
}
public override void WriteStartAttribute(string prefix, string localName, string ns)
{
_innerWriter.WriteStartAttribute(prefix, localName, ns);
}
public override void WriteStartDocument(bool standalone)
{
_innerWriter.WriteStartDocument(standalone);
}
public override void WriteStartDocument()
{
_innerWriter.WriteStartDocument();
}
public override void WriteStartElement(string prefix, string localName, string ns)
{
if (_depth++ > _maxDepth) ThrowException();
_innerWriter.WriteStartElement(prefix, localName, ns);
}
public override WriteState WriteState
{
get { return _innerWriter.WriteState; }
}
public override void WriteString(string text)
{
_innerWriter.WriteString(text);
}
public override void WriteSurrogateCharEntity(char lowChar, char highChar)
{
_innerWriter.WriteSurrogateCharEntity(lowChar, highChar);
}
public override void WriteWhitespace(string ws)
{
_innerWriter.WriteWhitespace(ws);
}
private void ThrowException()
{
throw new InvalidOperationException(string.Format("Result xml has more than {0} nested tags. It is possible that xslt transformation contains an endless recursive call.", _maxDepth));
}
}
This answer is for #WilliamJockusch.
I'm wondering if there is a general way to track down
StackOverflowExceptions. In other words, suppose I have infinite
recursion somewhere in my code, but I have no idea where. I want to
track it down by some means that is easier than stepping through code
all over the place until I see it happening. I don't care how hackish
it is. For example, It would be great to have a module I could
activate, perhaps even from another thread, that polled the stack
depth and complained if it got to a level I considered "too high." For
example, I might set "too high" to 600 frames, figuring that if the
stack were too deep, that has to be a problem. Is something like that
possible. Another example would be to log every 1000th method call
within my code to the debug output. The chances this would get some
evidence of the overlow would be pretty good, and it likely would not
blow up the output too badly. The key is that it cannot involve
writing a check wherever the overflow is happening. Because the entire
problem is that I don't know where that is. Preferrably the solution
should not depend on what my development environment looks like; i.e,
it should not assumet that I am using C# via a specific toolset (e.g.
VS).
It sounds like you're keen to hear some debugging techniques to catch this StackOverflow so I thought I would share a couple for you to try.
1. Memory Dumps.
Pro's: Memory Dumps are a sure fire way to work out the cause of a Stack Overflow. A C# MVP & I worked together troubleshooting a SO and he went on to blog about it here.
This method is the fastest way to track down the problem.
This method wont require you to reproduce problems by following steps seen in logs.
Con's: Memory Dumps are very large and you have to attach AdPlus/procdump the process.
2. Aspect Orientated Programming.
Pro's: This is probably the easiest way for you to implement code that checks the size of the call stack from any method without writing code in every method of your application. There are a bunch of AOP Frameworks that allow you to Intercept before and after calls.
Will tell you the methods that are causing the Stack Overflow.
Allows you to check the StackTrace().FrameCount at the entry and exit of all methods in your application.
Con's: It will have a performance impact - the hooks are embedded into the IL for every method and you cant really "de-activate" it out.
It somewhat depends on your development environment tool set.
3. Logging User Activity.
A week ago I was trying to hunt down several hard to reproduce problems. I posted this QA User Activity Logging, Telemetry (and Variables in Global Exception Handlers) . The conclusion I came to was a really simple user-actions-logger to see how to reproduce problems in a debugger when any unhandled exception occurs.
Pro's: You can turn it on or off at will (ie subscribing to events).
Tracking the user actions doesn't require intercepting every method.
You can count the number of events methods are subscribed too far more simply than with AOP.
The log files are relatively small and focus on what actions you need to perform to reproduce the problem.
It can help you to understand how users are using your application.
Con's: Isn't suited to a Windows Service and I'm sure there are better tools like this for web apps.
Doesn't necessarily tell you the methods that cause the Stack Overflow.
Requires you to step through logs manually reproducing problems rather than a Memory Dump where you can get it and debug it straight away.
Maybe you might try all techniques I mention above and some that #atlaste posted and tell us which one's you found were the easiest/quickest/dirtiest/most acceptable to run in a PROD environment/etc.
Anyway good luck tracking down this SO.
If you application depends on 3d-party code (in Xsl-scripts) then you have to decide first do you want to defend from bugs in them or not.
If you really want to defend then I think you should execute your logic which prone to external errors in separate AppDomains.
Catching StackOverflowException is not good.
Check also this question.
I had a stackoverflow today and i read some of your posts and decided to help out the Garbage Collecter.
I used to have a near infinite loop like this:
class Foo
{
public Foo()
{
Go();
}
public void Go()
{
for (float i = float.MinValue; i < float.MaxValue; i+= 0.000000000000001f)
{
byte[] b = new byte[1]; // Causes stackoverflow
}
}
}
Instead let the resource run out of scope like this:
class Foo
{
public Foo()
{
GoHelper();
}
public void GoHelper()
{
for (float i = float.MinValue; i < float.MaxValue; i+= 0.000000000000001f)
{
Go();
}
}
public void Go()
{
byte[] b = new byte[1]; // Will get cleaned by GC
} // right now
}
It worked for me, hope it helps someone.
With .NET 4.0 You can add the HandleProcessCorruptedStateExceptions attribute from System.Runtime.ExceptionServices to the method containing the try/catch block. This really worked! Maybe not recommended but works.
using System;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Runtime.ExceptionServices;
namespace ExceptionCatching
{
public class Test
{
public void StackOverflow()
{
StackOverflow();
}
public void CustomException()
{
throw new Exception();
}
public unsafe void AccessViolation()
{
byte b = *(byte*)(8762765876);
}
}
class Program
{
[HandleProcessCorruptedStateExceptions]
static void Main(string[] args)
{
Test test = new Test();
try {
//test.StackOverflow();
test.AccessViolation();
//test.CustomException();
}
catch
{
Console.WriteLine("Caught.");
}
Console.WriteLine("End of program");
}
}
}
#WilliamJockusch, if I understood correctly your concern, it's not possible (from a mathematical point of view) to always identify an infinite recursion as it would mean to solve the Halting problem. To solve it you'd need a Super-recursive algorithm (like Trial-and-error predicates for example) or a machine that can hypercompute (an example is explained in the following section - available as preview - of this book).
From a practical point of view, you'd have to know:
How much stack memory you have left at the given time
How much stack memory your recursive method will need at the given time for the specific output.
Keep in mind that, with the current machines, this data is extremely mutable due to multitasking and I haven't heard of a software that does the task.
Let me know if something is unclear.
By the looks of it, apart from starting another process, there doesn't seem to be any way of handling a StackOverflowException. Before anyone else asks, I tried using AppDomain, but that didn't work:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Threading;
namespace StackOverflowExceptionAppDomainTest
{
class Program
{
static void recrusiveAlgorithm()
{
recrusiveAlgorithm();
}
static void Main(string[] args)
{
if(args.Length>0&&args[0]=="--child")
{
recrusiveAlgorithm();
}
else
{
var domain = AppDomain.CreateDomain("Child domain to test StackOverflowException in.");
domain.ExecuteAssembly(Assembly.GetEntryAssembly().CodeBase, new[] { "--child" });
domain.UnhandledException += (object sender, UnhandledExceptionEventArgs e) =>
{
Console.WriteLine("Detected unhandled exception: " + e.ExceptionObject.ToString());
};
while (true)
{
Console.WriteLine("*");
Thread.Sleep(1000);
}
}
}
}
}
If you do end up using the separate-process solution, however, I would recommend using Process.Exited and Process.StandardOutput and handle the errors yourself, to give your users a better experience.
You can read up this property every few calls, Environment.StackTrace , and if the stacktrace exceded a specific threshold that you preset, you can return the function.
You should also try to replace some recursive functions with loops.
As I am developing my small game, I have made a lot of progress yet frustrated about a lot of things. The latest thing was creating a list of required items and for you to understand that I will provide you with both Explanation as well as Code which I created but obviously doesn't work...
I - Explanation
In order for the player to build a building he must have done some required researches with each research requires more researches for it to be researched... It is like a tree of researches that the player will go through them by exploring the game and doing some tasks...
So to imagine it more accurately you can look at my small code here
II - Code
//Available Main Elements
var carbon = new Element {Name = "Carbon"};
var hydrogen = new Element {Name = "Hydrogen"};
var oxygen = new Element {Name = "Oxygen"};
var nitrogen = new Element {Name = "Nitrogen"};
//Example Research
var steam = new Research(name : "Steam", requiredElements: null, requiredResearches: /*Fire*/ & /*Water*/ & /*Iron*/);
So from the last snippet of code [which is just to explain] the player want to research the Steam that for instance needs 3 more researches in order to be researched... one of which the Iron also needs 1 more research to be researched and so on [maybe less maybe more or maybe no requirements at all]...
Concluding that the Question is : How could I create such nesting so that when a player tries to do a research the system quickly looks at the researches he have done and the research he wants to do [including it's nested ones] and If the player did not meet the requirements it just returns a tree with what things he want's to achieve ?
After all, I just want to thank you in advance and I am waiting for your very valuable support...
I would remove the requirement logic out of the Research objects themselves. For simplicity say it was this:
public class Research
{
public string Name { get; set; }
}
Then, I would keep a list of the requirements in a Dictonary where each Research contains a bucket of other Researches:
Dictionary<Research, List<Research>> requiredResearches =
new Dictionary<Research, List<Research>>();
// the list of Researches the player has completed
List<Research> playersResearched = new List<Research>;
For example, "Steam" would contain "Fire", "Water" and "Iron". And maybe "Iron" contains "Tin".
Next, given a Research, we could look at all of it's requierements including requirements of requirements:
// e.g. research is "Steam" and returns "Fire", "Water", "Iron", "Tin"
var chainOfRequirements = GetReq(requiredResearches, research);
That calls a recursive function like this:
public IList<Research> GetReq(Dictionary<Research, List<Research>> reqs,
Research target)
{
var chain = new List<Research>();
if(reqs.ContainsKey(target))
{
foreach(var item in reqs[target])
{
chain.Add(item);
chain.AddRange(GetReq(reqs, item));
}
}
return chain;
}
What you are returned is a flat list of requirements (including requirements of requirements). At that point, a little query against the players list of Researches can return to you which ones that are missing:
var missing = chainOfRequirements.Where (c =>
playerResearches.Where (r => r == c).Any () == false).Distinct();
TO ENABLE COMPARING DICTIONARY USING "NAME"
public sealed class NameEqualityComparer : IEqualityComparer<Research>
{
public bool Equals(Research x, Research y)
{
if (ReferenceEquals(x, y)) return true;
if (ReferenceEquals(x, null)) return false;
if (ReferenceEquals(y, null)) return false;
if (x.GetType() != y.GetType()) return false;
return string.Equals(x.Name, y.Name);
}
public int GetHashCode(Research obj)
{
return (obj.Name != null ? obj.Name.GetHashCode() : 0);
}
}
Here is proof of concept.
I don't know C# well enough to provide code, but you could make each Research have an Array of type Research, that would hold the Research that must be done to be able to make the actual Research, then you only need to iterate the Array checking if all of them are completed.
You don't even need to check the requirements of the requirements, as if they are completed, they already have all requirements completed.
If the Research don't have any requirements you just let the Array empty.
I suggest that you make your own requirementsTree class, which would need to be a linked list with at least the following members:
a requirementsTree[] prerequisiteResearch property, containing all of the research required for that particular item.
a bool isResearched property, indicating if the player has in fact researched the item.
a method, say, bool isEligibleToResearch(), which loops through all of the items in prerequisiteResearch to check if the player has already researched them.
With this I think you'd have a well-structured and extensible requirements class.
I can't think of any specific things that would prevent this from happening, but I can see why you'd be hesitant to just jump into coding it without having thought through the cases. To try to give some pointers, here are some functions I would imagine you having by the end:
// Having a function to retrieve a unique variable by its name (backed by a Dictionary<string, Research>)
// is often handy for me, especially if you decide to script the requirements tree in a text file.
// If you have enough references passed around that you can do this without a static method, all
// the better.
Research.getResearchByName(string name)
// A recursive function. If this research has not been completed, it adds itself to the set, as well
// as any prerequisites (by calling this function on its prerequisites). The top-level, public
// version of this function would create the set, and then return it after performing this check. If
// the Set is empty, then the player can start the research.
Research.addUnresearched_Internal(Set<Research> unresearched)
I think the main issue with my approach here is that I only thought to use a Set, rather than a Tree;
but with some bit of variation, you might be able to do better than me.
One possibility would be to add bool completed to the Element and Research classes. You could then have a function that checks for completed == false for any of its sub-researches.
bool CanCompleteResearch(Research r)
{
for (Research sub in r.requiredResearches)
{
// No need to recursively check 'sub'; in order for 'sub' to be marked as
// completed, its sub-researches must've been already completed.
if (!sub.completed)
return false;
}
return true;
}
I'm sure there's some one-liner trick you can do with LINQ.
EDIT: this only works if a research tree is not being used in more than one place at once (i.e. if you had the research objects shared across multiple player instances).
EDIT 2: In order to figure out all the missing research, you can use a simple recursive function:
void ListMissingResearch(Research r)
{
for (Research sub in r.requiredResearches)
{
if (!sub.completed)
{
// print message saying "Research <r> requires research <sub>"
ListMissingResearch(sub); // list any missing sub-sub-research
}
}
}
It sounds like you want to do something like this:
public class Research
{
private Collection<Research> prerequisites
public Collection<Research> Prerequisites
{
get { return this.prerequisistes; }
}
public bool IsComplete { get; set; }
public IEnumerable<Research> RequiredResearch()
{
if (this.IsComplete)
{
return new Research[0];
}
else
{
return new[] { this }.Concat(
this.Prerequisites.SelectMany(r => r.RequiredResearch()));
}
}
}
If you're making a multi-player, it might have to be something like this:
public class Player
{
private Collection<Research> completedResearch
public Collection<Research> CompletedResearch
{
get { return this.completedResearch; }
}
}
public class Research
{
private Collection<Research> prerequisites
public Collection<Research> Prerequisites
{
get { return this.prerequisistes; }
}
public IEnumerable<Research> RequiredResearch(Player player)
{
if (player.CompletedResearch.Contains(this))
{
return new Research[0];
}
else
{
return new[] { this }.Concat(
this.Prerequisites.SelectMany(r => r.RequiredResearch(player)));
}
}
}
Both these methods will return the current research if it has not been completed (instead of just it's prerequisites). You could simply do a .Skip(1) on the result to ignore the current one, though.
You could use a function like this in your research class that just takes in an enumerable list of Researchs you use to check that the player has the prerequisits. This assumes there is a local list or some other enumerable of Researches called Prereqs in the Research class.
public bool CanHas(IEnumerable<Research> researches)
{
return Prereqs.All((pr) => researches.Contains(pr) && pr.CanHas(researches));
}
The function just recursivly checks each Prereq to see if it is in the list passed in, and then checks that prereqs prereqs. All is a linq extention method that just returns true if all of the elements in the enumerable meet the criteria. If there are no elements, it returns true (all zero of the elements meet the criteria) so this will terminate when you get down to a Research with no Prereqs.
Note: This does a depth first search and is not smart about the same research being a sub-research of more then one prerequisite.
I'm doing practice problems from MCTS Exam 70-536 Microsft .Net Framework Application Dev Foundation, and one of the problems is to create two classes, one generic, one object type that both perform the same thing; in which a loop uses the class and iterated over thousand times. And using the timer, time the performance of both. There was another post at C# generics question that seeks the same questoion but nonone replied.
Basically if in my code I run the generic class first it takes loger to process. If I run the object class first than the object class takes longer to process. The whole idea was to prove that generics perform faster.
I used the original users code to save me some time. I didn't particularly see anything wrong with the code and was puzzled by the outcome. Can some one explain why the unusual results?
Thanks,
Risho
Here is the code:
class Program
{
class Object_Sample
{
public Object_Sample()
{
Console.WriteLine("Object_Sample Class");
}
public long getTicks()
{
return DateTime.Now.Ticks;
}
public void display(Object a)
{
Console.WriteLine("{0}", a);
}
}
class Generics_Samle<T>
{
public Generics_Samle()
{
Console.WriteLine("Generics_Sample Class");
}
public long getTicks()
{
return DateTime.Now.Ticks;
}
public void display(T a)
{
Console.WriteLine("{0}", a);
}
}
static void Main(string[] args)
{
long ticks_initial, ticks_final, diff_generics, diff_object;
Object_Sample OS = new Object_Sample();
Generics_Samle<int> GS = new Generics_Samle<int>();
//Generic Sample
ticks_initial = 0;
ticks_final = 0;
ticks_initial = GS.getTicks();
for (int i = 0; i < 50000; i++)
{
GS.display(i);
}
ticks_final = GS.getTicks();
diff_generics = ticks_final - ticks_initial;
//Object Sample
ticks_initial = 0;
ticks_final = 0;
ticks_initial = OS.getTicks();
for (int j = 0; j < 50000; j++)
{
OS.display(j);
}
ticks_final = OS.getTicks();
diff_object = ticks_final - ticks_initial;
Console.WriteLine("\nPerformance of Generics {0}", diff_generics);
Console.WriteLine("Performance of Object {0}", diff_object);
Console.ReadKey();
}
}
Well, the first problem I can see is that you're using the DateTime object to measure time in your application (for a very small interval).
You should be using the Stopwatch class. It offers better precision when trying to benchmark code.
The second problem is that you're not allowing for JIT (Just-In-Time compilation). The first call to your code is going to take longer simply because it has to be JIT'd. After that, you'll get your results.
I would make a single call in to your code before you start timing things so you can get an accurate idea of what is happening during the loop.
You should run both classes a separate time before timing it to allow the JITter to run.
Your test is incorrect. Here are your methods:
public void display(T a)
{
Console.WriteLine("{0}", a); // Console.WriteLine(string format, params object[] args) <- boxing is performed here
}
public void display(Object a)// <- boxing is performed here
{
Console.WriteLine("{0}", a);
}
So, in both cases you are using boxing. Much better would be if your class, for example, will count total sum of values, like:
public void add(long a)
{
Total += a;
}
public void display(Object a)// <- boxing is performed here
{
Total += (long) a;// <- unboxing is performed here
}
Your timed code includes a Console.WriteLine(). That will take up 99.999999% of the time.
Your assumption that generic will be faster in this situation is wrong. You may have misinterpreted a remark about non-generic collection classes.
This won't be on he exam.
why would it be faster? both ints must be boxed in order to use Console.WriteLine(string, object)
edit: ToString() itself does not seem to cause boxing
http://weblogs.asp.net/ngur/archive/2003/12/16/43856.aspx
so when you use Console.WriteLine(a); which would call Console.WriteLine(Int32) that should work i guess (i would need to look into reflector to confirm this)
In my multithreading application I am using some variables that can be altered by many instances in the same time. It is weird but it has worked fine without any problem..but of course I need to make it thread-safe. I am just beginning with locks so I would appretiate your advice:
When client connects, class Client is created where each Client has its own "A" variable.
Sometimes, Client calls method like that:
Client selectedClient SelectOtherClientClassByID(sentID);
selectedClient.A=5;
No problems until now with that even when 5 classes were doing at the same time (threadpool), but I was thinking what about adding locks to A properties?
Like:
A {
get { return mA; }
set {
// use lock here for settting A to some value
}
}
Would it be OK?
You need to use locks in BOTH get and set. This lock must be the same object. For example:
private object mylock = new object();
public int A {
get {
int result;
lock(mylock) {
result = mA;
}
return result;
}
set {
lock(mylock) {
mA = value;
}
}
}
Locking access to properties inside of accessors may lead to bogus results. For the example, look at the following code:
class C {
private object mylock = new object();
public int A {
get {
int result;
lock(mylock) {
result = mA;
}
return result;
}
set {
lock(mylock) {
mA = value;
}
}
}
}
C obj = new C;
C.A++;
(yes, I've copied it from the first answer)
There is a race condition here! Operation "C.A++" actually requires two separate accesses to A, one to get the value and the other to set the updated value. Nothing ensures that these two accesses will be carried out as together without context switch between them. Classical scenario for race condition!
So, beware! It's not a good idea to put locks inside accessors, locks should be explicitly obtained, like the previous answer suggests (though it doesn't have to be with SyncRoots, any object will do)
It's very rare when all you need is just set a single property. More often selectedClient.A = 5 will be a part of a much bigger logical operation, which involves several assignments/evaluations/etc. During that whole operation you'd rather prefer selectedClient to be in a consistent state and not to introduce deadlocks/race conditions. Therefore, it will be much better to expose SyncRoot property in your Client class and lock on that from the calling code:
Client selectedClient = GetClient(...);
lock(selectedClient.SyncRoot)
{
selectedClient.A = 5;
selectedClient.B = selectedClient.A * 54;
}