Develop Reference

Why does deadlock occur on calling Semaphore.WaitOne?

The run() function in the following code is called from other threads simultaneously. At anytime, on any line, a ThreadAbortException might occur according to the general design of the application, which I cannot change.
I sometimes get SemaphoreFullException while calling pool.Release(). I think this occurs if a thread abort exception occurs while calling "pool.WaitOne()". During my debug tries, after SemaphoreFullException has occurred, there is no problem in running the code. After that exception, pool.WaitOne() calls and other things work just as expected.
I haven't been able to get a deadlock situation during my local debug sessions. However, in a remote computer, I have a deadlock with this code. I attach that process using remote debugger and see that the execution is locked on the line pool.WaitOne();.
I can't figure out how this would happen, and what I'm doing wrong. Any help is very appreciated.
private static object poolLocker = new object();
private static Semaphore _pool;
private static Semaphore pool
{
get
{
if (_pool == null)
lock (poolLocker)
if (_pool == null)
{
int count = myMaximumThreadCount;
_pool = new Semaphore(count, count);
}
return _pool;
}
}
private void run()
{
try
{
pool.WaitOne();
do_something_that_may_throw_exception();
}
finally
{
try
{
pool.Release();
}
catch (SemaphoreFullException) { }
}
}

Try to change the initialization of the semaphore object in pool property to:
private static Semaphore pool
{
get
{
if (_pool == null)
lock (poolLocker)
if (_pool == null)
{
int count = myMaximumThreadCount;
_pool = new Semaphore(0, count);
}
return _pool;
}
}
An initial count for this semaphore should be set to zero.

I have found the cause of the deadlock; and it has nothing to do with the question I've asked, so this is a bad question, sorry for that. There seems to be no problem in the code in the question.
The cause: In the do_something_that_may_throw_exception() function, an extern function of a C++ library is being called. When an error occurs in the C++ function, a SEHException is thrown. However, in my tries this exception can only be caught in a function that has HandleProcessCorruptedStateExceptions and SecurityCritical attributes. And that function happens to call the run() function of the question. However, the finally part of the run() function is newer executed! Also, if you have a using(IDisposable object){ ... } and the SEHException occurs inside it; object's Dispose() function won't be called.
I've used the following function for calling the C++ function; and everything worked fine:
SafeCall(()=> call_external_cpp_function());
[HandleProcessCorruptedStateExceptions]
[SecurityCritical]
internal static void SafeCall(Action action)
{
try
{
action();
}
catch (System.Threading.ThreadAbortException) { throw; }
catch (System.Threading.ThreadInterruptedException) { throw; }
catch (Exception ex)
{
throw new Exception(ex.Message);
}
}

Gracefully handling corrupted state exceptions

Related to this question, I would like to force CLR to let my .NET 4.5.2 app catch Corrupted State Exceptions, for the sole purpose of logging them and then terminating the application. What's the correct way to do this, if I have catch (Exception ex) at several places around the app?
So, after I specify the <legacyCorruptedStateExceptionsPolicy> attribute, if I understood correctly, all the catch (Exception ex) handlers will catch exceptions like AccessViolationException and happily continue.
Yeah, I know catch (Exception ex) is a Bad Idea™, but if CLR would at least put the correct stack trace into the Event Log, I would be more than happy to explain to the customer that his server app failing fast at 1AM and being offline for the night is a good thing. But unfortunately, CLR logs an unrelated exception into the Event Log and then closes the process so that I cannot find out what actually happened.
The question is, how to make this happen, process wide:
if the exception thrown is a Corrupted State Exception:
- write the message to the log file
- end the process
(Update)
In other words, this would probably work for most exceptions in a simple app:
[HandleProcessCorruptedStateExceptions]
[SecurityCritical]
static void Main() // main entry point
{
try
{
}
catch (Exception ex)
{
// this will catch CSEs
}
}
But, it won't work for:
Unhandled app domain exceptions (i.e. thrown on non-foreground threads)
Windows Service apps (which don't have an actual Main entry point)
So it seems like <legacyCorruptedStateExceptionsPolicy> is the only way to make this work, in which case I don't know how to fail after logging the CSE?

Instead of using <legacyCorruptedStateExceptionsPolicy> it would be better to use [HandleProcessCorruptedStateExceptions] (and [SecurityCritical]) as stated here:
https://msdn.microsoft.com/en-us/magazine/dd419661.aspx
Following that, your Main method should look something like this:
[HandleProcessCorruptedStateExceptions, SecurityCritical]
static void Main(string[] args)
{
try
{
...
}
catch (Exception ex)
{
// Log the CSE.
}
}
But be aware that this doesn't catch the more serious exceptions like StackOverflowException and ExecutionEngineException.
Also finally of involved try blocks will not be executed:
https://csharp.2000things.com/2013/08/30/920-a-finally-block-is-not-executed-when-a-corrupted-state-exception-occurs/
For other unhandled appdomain exceptions you can use :
AppDomain.CurrentDomain.UnhandledException
Application.Current.DispatcherUnhandledException
TaskScheduler.UnobservedTaskException
(Please do a search for the details when a specific handler is appropriate for your situation. TaskScheduler.UnobservedTaskException for example is a bit tricky.)
If you don't have access to the Main method, you can also mark your AppDomain exception handler to catch the CSE:
AppDomain.CurrentDomain.UnhandledException += CurrentDomain_UnhandledException;
...
[HandleProcessCorruptedStateExceptions, SecurityCritical]
private static void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
// AccessViolationExceptions will get caught here but you cannot stop
// the termination of the process if e.IsTerminating is true.
}
The last line of defense could be an unmanaged UnhandledExceptionFilter like this:
[DllImport("kernel32"), SuppressUnmanagedCodeSecurity]
private static extern int SetUnhandledExceptionFilter(Callback cb);
// This has to be an own non generic delegate because generic delegates cannot be marshalled to unmanaged code.
private delegate uint Callback(IntPtr ptrToExceptionInfo);
And then somewhere at the beginning of your process:
SetUnhandledExceptionFilter(ptrToExceptionInfo =>
{
var errorCode = "0x" + Marshal.GetExceptionCode().ToString("x2");
...
return 1;
});
You can find more information about the possible return codes here:
https://msdn.microsoft.com/en-us/library/ms680634(VS.85).aspx
A "specialty" of the UnhandledExceptionFilter is that it isn't called if a debugger is attached. (At least not in my case of having a WPF app.) So be aware of that.
If you set all the appropriate ExceptionHandlers from above, you should be logging all exceptions that can be logged. For the more serious exceptions (like StackOverflowException and ExecutionEngineException) you have to find another way because the whole process is unusable after they happened. A possible way could perhaps be another process that watches the main process and logs any fatal errors.
Additional hints:
In the AppDomain.CurrentDomain.UnhandledException you can safely cast the e.ExceptionObject to Exception without having to worry - at least if you don't have any IL code that throws other objects than Exception: Why is UnhandledExceptionEventArgs.ExceptionObject an object and not an Exception?
If you want to suppress the Windows Error Reporting dialog, you can take a look here: How to terminate a program when it crashes? (which should just fail a unit test instead of getting stuck forever)
If you have a WPF application with multiple dispatchers you can also use a Dispatcher.UnhandledException for the other dispatchers.

Thanks to #haindl for pointing out that you can also decorate handler methods with the [HandleProcessCorruptedStateExceptions]1 attribute, so I made a little test app just to confirm if things really work as they are supposed to.
1 Note: Most answers state that I should also include the [SecurityCritical] attribute, although in the tests below omitting it didn't change the behavior (the [HandleProcessCorruptedStateExceptions] alone seemed to work just fine). However, I will leave both attributes below since I am presuming all these folks knew what they were saying. That's a school example of "Copied from StackOverflow" pattern in action.
The idea is, obviously, to remove the <legacyCorruptedStateExceptionsPolicy> setting from app.config, i.e. only allow our outermost (entry-level) handler(s) to catch the exception, log it, and then fail. Adding the setting will allow your app to continue, if you catch the exception in some inner handler, and this is not what you want: the idea is just to get the accurate exception info and then die miserably.
I used the following method to throw the exception:
static void DoSomeAccessViolation()
{
// if you have any questions about why this throws,
// the answer is "42", of course
var ptr = new IntPtr(42);
Marshal.StructureToPtr(42, ptr, true);
}
1. Catching exceptions from Main:
[SecurityCritical]
[HandleProcessCorruptedStateExceptions]
static void Main(string[] args)
{
try
{
DoSomeAccessViolation();
}
catch (Exception ex)
{
// this will catch all CSEs in the main thread
Log(ex);
}
}
2. Catching all exceptions, including background threads/tasks:
// no need to add attributes here
static void Main(string[] args)
{
AppDomain.CurrentDomain.UnhandledException += UnhandledException;
// throw on a background thread
var t = new Task(DoSomeAccessViolation);
t.Start();
t.Wait();
}
// but it's important that this method is marked
[SecurityCritical]
[HandleProcessCorruptedStateExceptions]
private static void UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
// this will catch all unhandled exceptions, including CSEs
Log(e.ExceptionObject as Exception);
}
I would recommend using just the latter approach, and removing the [HandleProcessCorruptedStateExceptions] from all other places to make sure the exception doesn't get caught at the wrong place. I.e. if you have a try/catch block somewhere and an AccessViolationException is thrown, you want CLR to skip the catch block and propagate to the UnhandledException before ending the app.

Is party over? not so fast
Microsoft: "Use application domains to isolate tasks that might bring down a process."
The program below will protect your main application/thread from unrecoverable failures without risks associated with use of HandleProcessCorruptedStateExceptions and <legacyCorruptedStateExceptionsPolicy>
public class BoundaryLessExecHelper : MarshalByRefObject
{
public void DoSomething(MethodParams parms, Action action)
{
if (action != null)
action();
parms.BeenThere = true; // example of return value
}
}
public struct MethodParams
{
public bool BeenThere { get; set; }
}
class Program
{
static void InvokeCse()
{
IntPtr ptr = new IntPtr(123);
System.Runtime.InteropServices.Marshal.StructureToPtr(123, ptr, true);
}
// This is a plain code that will prove that CSE is thrown and not handled
// this method is not a solution. Solution is below
private static void ExecInThisDomain()
{
try
{
var o = new BoundaryLessExecHelper();
var p = new MethodParams() { BeenThere = false };
Console.WriteLine("Before call");
o.DoSomething(p, CausesAccessViolation);
Console.WriteLine("After call. param been there? : " + p.BeenThere.ToString()); //never stops here
}
catch (Exception exc)
{
Console.WriteLine($"CSE: {exc.ToString()}");
}
Console.ReadLine();
}
// This is a solution for CSE not to break your app.
private static void ExecInAnotherDomain()
{
AppDomain dom = null;
try
{
dom = AppDomain.CreateDomain("newDomain");
var p = new MethodParams() { BeenThere = false };
var o = (BoundaryLessExecHelper)dom.CreateInstanceAndUnwrap(typeof(BoundaryLessExecHelper).Assembly.FullName, typeof(BoundaryLessExecHelper).FullName);
Console.WriteLine("Before call");
o.DoSomething(p, CausesAccessViolation);
Console.WriteLine("After call. param been there? : " + p.BeenThere.ToString()); // never gets to here
}
catch (Exception exc)
{
Console.WriteLine($"CSE: {exc.ToString()}");
}
finally
{
AppDomain.Unload(dom);
}
Console.ReadLine();
}
static void Main(string[] args)
{
ExecInAnotherDomain(); // this will not break app
ExecInThisDomain(); // this will
}
}

How to create a generic timeout object for various code blocks?

I have a series of code blocks that are taking too long. I don't need any finesse when it fails. In fact, I want to throw an exception when these blocks take too long, and just fall out through our standard error handling. I would prefer to NOT create methods out of each block (which are the only suggestions I've seen so far), as it would require a major rewrite of the code base.
Here's what I would LIKE to create, if possible.
public void MyMethod( ... )
{
...
using (MyTimeoutObject mto = new MyTimeoutObject(new TimeSpan(0,0,30)))
{
// Everything in here must complete within the timespan
// or mto will throw an exception. When the using block
// disposes of mto, then the timer is disabled and
// disaster is averted.
}
...
}
I've created a simple object to do this using the Timer class. (NOTE for those that like to copy/paste: THIS CODE DOES NOT WORK!!)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Timers;
public class MyTimeoutObject : IDisposable
{
private Timer timer = null;
public MyTimeoutObject (TimeSpan ts)
{
timer = new Timer();
timer.Elapsed += timer_Elapsed;
timer.Interval = ts.TotalMilliseconds;
timer.Start();
}
void timer_Elapsed(object sender, ElapsedEventArgs e)
{
throw new TimeoutException("A code block has timed out.");
}
public void Dispose()
{
if (timer != null)
{
timer.Stop();
}
}
}
It does not work because the System.Timers.Timer class captures, absorbs and ignores any exceptions thrown within, which -- as I've discovered -- defeats my design. Any other way of creating this class/functionality without a total redesign?
This seemed so simple two hours ago, but is causing me much headache.

OK, I've spent some time on this one and I think I have a solution that will work for you without having to change your code all that much.
The following is how you would use the Timebox class that I created.
public void MyMethod( ... ) {
// some stuff
// instead of this
// using(...){ /* your code here */ }
// you can use this
var timebox = new Timebox(TimeSpan.FromSeconds(1));
timebox.Execute(() =>
{
/* your code here */
});
// some more stuff
}
Here's how Timebox works.
A Timebox object is created with a given Timespan
When Execute is called, the Timebox creates a child AppDomain to hold a TimeboxRuntime object reference, and returns a proxy to it
The TimeboxRuntime object in the child AppDomain takes an Action as input to execute within the child domain
Timebox then creates a task to call the TimeboxRuntime proxy
The task is started (and the action execution starts), and the "main" thread waits for for as long as the given TimeSpan
After the given TimeSpan (or when the task completes), the child AppDomain is unloaded whether the Action was completed or not.
A TimeoutException is thrown if action times out, otherwise if action throws an exception, it is caught by the child AppDomain and returned for the calling AppDomain to throw
A downside is that your program will need elevated enough permissions to create an AppDomain.
Here is a sample program which demonstrates how it works (I believe you can copy-paste this, if you include the correct usings). I also created this gist if you are interested.
public class Program
{
public static void Main()
{
try
{
var timebox = new Timebox(TimeSpan.FromSeconds(1));
timebox.Execute(() =>
{
// do your thing
for (var i = 0; i < 1000; i++)
{
Console.WriteLine(i);
}
});
Console.WriteLine("Didn't Time Out");
}
catch (TimeoutException e)
{
Console.WriteLine("Timed Out");
// handle it
}
catch(Exception e)
{
Console.WriteLine("Another exception was thrown in your timeboxed function");
// handle it
}
Console.WriteLine("Program Finished");
Console.ReadLine();
}
}
public class Timebox
{
private readonly TimeSpan _ts;
public Timebox(TimeSpan ts)
{
_ts = ts;
}
public void Execute(Action func)
{
AppDomain childDomain = null;
try
{
// Construct and initialize settings for a second AppDomain. Perhaps some of
// this is unnecessary but perhaps not.
var domainSetup = new AppDomainSetup()
{
ApplicationBase = AppDomain.CurrentDomain.SetupInformation.ApplicationBase,
ConfigurationFile = AppDomain.CurrentDomain.SetupInformation.ConfigurationFile,
ApplicationName = AppDomain.CurrentDomain.SetupInformation.ApplicationName,
LoaderOptimization = LoaderOptimization.MultiDomainHost
};
// Create the child AppDomain
childDomain = AppDomain.CreateDomain("Timebox Domain", null, domainSetup);
// Create an instance of the timebox runtime child AppDomain
var timeboxRuntime = (ITimeboxRuntime)childDomain.CreateInstanceAndUnwrap(
typeof(TimeboxRuntime).Assembly.FullName, typeof(TimeboxRuntime).FullName);
// Start the runtime, by passing it the function we're timboxing
Exception ex = null;
var timeoutOccurred = true;
var task = new Task(() =>
{
ex = timeboxRuntime.Run(func);
timeoutOccurred = false;
});
// start task, and wait for the alloted timespan. If the method doesn't finish
// by then, then we kill the childDomain and throw a TimeoutException
task.Start();
task.Wait(_ts);
// if the timeout occurred then we throw the exception for the caller to handle.
if(timeoutOccurred)
{
throw new TimeoutException("The child domain timed out");
}
// If no timeout occurred, then throw whatever exception was thrown
// by our child AppDomain, so that calling code "sees" the exception
// thrown by the code that it passes in.
if(ex != null)
{
throw ex;
}
}
finally
{
// kill the child domain whether or not the function has completed
if(childDomain != null) AppDomain.Unload(childDomain);
}
}
// don't strictly need this, but I prefer having an interface point to the proxy
private interface ITimeboxRuntime
{
Exception Run(Action action);
}
// Need to derive from MarshalByRefObject... proxy is returned across AppDomain boundary.
private class TimeboxRuntime : MarshalByRefObject, ITimeboxRuntime
{
public Exception Run(Action action)
{
try
{
// Nike: just do it!
action();
}
catch(Exception e)
{
// return the exception to be thrown in the calling AppDomain
return e;
}
return null;
}
}
}
EDIT:
The reason I went with an AppDomain instead of Threads or Tasks only, is because there is no bullet proof way for terminating Threads or Tasks for arbitrary code [1][2][3]. An AppDomain, for your requirements, seemed like the best approach to me.

Here's an async implementation of timeouts:
...
private readonly semaphore = new SemaphoreSlim(1,1);
...
// total time allowed here is 100ms
var tokenSource = new CancellationTokenSource(100);
try{
await WorkMethod(parameters, tokenSource.Token); // work
} catch (OperationCancelledException ocx){
// gracefully handle cancellations:
label.Text = "Operation timed out";
}
...
public async Task WorkMethod(object prm, CancellationToken ct){
try{
await sem.WaitAsync(ct); // equivalent to lock(object){...}
// synchronized work,
// call tokenSource.Token.ThrowIfCancellationRequested() or
// check tokenSource.IsCancellationRequested in long-running blocks
// and pass ct to other tasks, such as async HTTP or stream operations
} finally {
sem.Release();
}
}
NOT that I advise it, but you could pass the tokenSource instead of its Token into WorkMethod and periodically do tokenSource.CancelAfter(200) to add more time if you're certain you're not at a spot that can be dead-locked (waiting on an HTTP call) but I think that would be an esoteric approach to multithreading.
Instead your threads should be as fast as possible (minimum IO) and one thread can serialize the resources (producer) while others process a queue (consumers) if you need to deal with IO multithreading (say file compression, downloads etc) and avoid deadlock possibility altogether.

I really liked the visual idea of a using statement. However, that is not a viable solution. Why? Well, a sub-thread (the object/thread/timer within the using statement) cannot disrupt the main thread and inject an exception, thus causing it to stop what it was doing and jump to the nearest try/catch. That's what it all boils down to. The more I sat and worked with this, the more that came to light.
In short, it can't be done the way I wanted to do it.
However, I've taken Pieter's approach and mangled my code a bit. It does introduce some readability issues, but I've tried to mitigate them with comments and such.
public void MyMethod( ... )
{
...
// Placeholder for thread to kill if the action times out.
Thread threadToKill = null;
Action wrappedAction = () =>
{
// Take note of the action's thread. We may need to kill it later.
threadToKill = Thread.CurrentThread;
...
/* DO STUFF HERE */
...
};
// Now, execute the action. We'll deal with the action timeouts below.
IAsyncResult result = wrappedAction.BeginInvoke(null, null);
// Set the timeout to 10 minutes.
if (result.AsyncWaitHandle.WaitOne(10 * 60 * 1000))
{
// Everything was successful. Just clean up the invoke and get out.
wrappedAction.EndInvoke(result);
}
else
{
// We have timed out. We need to abort the thread!!
// Don't let it continue to try to do work. Something may be stuck.
threadToKill.Abort();
throw new TimeoutException("This code block timed out");
}
...
}
Since I'm doing this in three or four places per major section, this does get harder to read over. However, it works quite well.

Finally is not executed when in a Thread running in a Windows Service

Can anyone explain why this finally block is not executed? I have read posts about when to expect finally block not be executed, but this seems to be another case. This code needs TopShelf and log4net. I am running .net 4.5
I guess it must be the Windows Service engine that kicks in on unhandled exceptions, but why is it running before the finally block has finished?
using log4net;
using log4net.Config;
using System;
using System.Threading;
using Topshelf;
namespace ConsoleApplication1
{
public class HostMain
{
static void Main(string[] args)
{
HostFactory.Run(x =>
{
x.Service<HostMain>(s =>
{
s.ConstructUsing(name => new HostMain());
s.WhenStarted(tc => tc.Start());
s.WhenStopped(tc => tc.Stop());
});
x.RunAsLocalSystem();
x.SetServiceName("TimerTest");
});
}
public void Stop()
{
LogManager.GetLogger("MyLog").Info("stopping");
}
public void Start()
{
XmlConfigurator.Configure();
LogManager.GetLogger("MyLog").Info("starting");
new Thread(StartServiceCode).Start();
}
public void StartServiceCode()
{
try
{
LogManager.GetLogger("MyLog").Info("throwing");
throw new ApplicationException();
}
finally
{
LogManager.GetLogger("MyLog").Info("finally");
}
}
}
}
outputs
starting
throwing
stopping
EDIT: Please comment why you are downgrading, maybe you don't understand the problem? I see a big problem here. You write some domain logic that does important stuff in the finally clause on Exception. Then if you host the logic in a Windows Service the design suddenly is broken.

From MDSN try-finally (C# Reference)
Within a handled exception, the associated finally block is guaranteed to be run. However, if the exception is unhandled, execution of the finally block is dependent on how the exception unwind operation is triggered. That, in turn, is dependent on how your computer is set up. For more information, see Unhandled Exception Processing in the CLR.
Usually, when an unhandled exception ends an application, whether or not the finally block is run is not important
This is by design, .NET has chosen to terminate your application, reason is, there is something terribly wrong, something didn't work as expected, by calling finally, we don't want to do more damage, so best is to end the application.
What if finally throws one more exception, where does that go? If application is about to close, it may have closed or started closing managed resources and accessing them for logging in finally could turn out to be fatal as well.

Sorry about this being an answer, but couldn't comment.
I couldn't find anything specific about the windows service, but I'm assuming it uses background/foreground threading to execute the code.
And in terms of threading, the finally block is sometimes voided (if the thread is aborted or interrupted unexpectedly) -
http://blog.goyello.com/2014/01/21/threading-in-c-7-things-you-should-always-remember-about/
Or for a more official post - (Look for the foreground/background threading section)
https://msdn.microsoft.com/en-us/library/orm-9780596527570-03-19.aspx
Hopefully it helps you a little

Have you made sure that the logger is getting a chance to flush to disk before the logger is destroyed when the service stops?
Edit
When a service starts it happens on a new thread. Within the Topshelf code there is an AppDomain.CurrentDomain.UnhandledException += CatchUnhandledException; handler.
void CatchUnhandledException(object sender, UnhandledExceptionEventArgs e)
{
_log.Fatal("The service threw an unhandled exception", (Exception)e.ExceptionObject);
HostLogger.Shutdown();
if (e.IsTerminating)
{
_exitCode = TopshelfExitCode.UnhandledServiceException;
_exit.Set();
#if !NET35
// it isn't likely that a TPL thread should land here, but if it does let's no block it
if (Task.CurrentId.HasValue)
{
return;
}
#endif
// this is evil, but perhaps a good thing to let us clean up properly.
int deadThreadId = Interlocked.Increment(ref _deadThread);
Thread.CurrentThread.IsBackground = true;
Thread.CurrentThread.Name = "Unhandled Exception " + deadThreadId.ToString();
while (true)
Thread.Sleep(TimeSpan.FromHours(1));
}
}
This catches the unhandled exception, and stops the service by setting the manualresetevent (this is the only thing that is blocking the service from ending).
After sleep is called, the thread is signalled and your finally block, which is on the service thread is killed.
The code then exits.
This is wired up in the Run() method in ConsoleRunHost.
public TopshelfExitCode Run()
{
Directory.SetCurrentDirectory(AppDomain.CurrentDomain.BaseDirectory);
AppDomain.CurrentDomain.UnhandledException += CatchUnhandledException;
if (_environment.IsServiceInstalled(_settings.ServiceName))
{
if (!_environment.IsServiceStopped(_settings.ServiceName))
{
_log.ErrorFormat("The {0} service is running and must be stopped before running via the console",
_settings.ServiceName);
return TopshelfExitCode.ServiceAlreadyRunning;
}
}
bool started = false;
try
{
_log.Debug("Starting up as a console application");
_log.Debug("Thread.CurrentThread.Name");
_log.Debug(Thread.CurrentThread.Name);
_exit = new ManualResetEvent(false);
_exitCode = TopshelfExitCode.Ok;
Console.Title = _settings.DisplayName;
Console.CancelKeyPress += HandleCancelKeyPress;
if (!_serviceHandle.Start(this))
throw new TopshelfException("The service failed to start (return false).");
started = true;
_log.InfoFormat("The {0} service is now running, press Control+C to exit.", _settings.ServiceName);
_exit.WaitOne();
}
catch (Exception ex)
{
_log.Error("An exception occurred", ex);
return TopshelfExitCode.AbnormalExit;
}
finally
{
if (started)
StopService();
_exit.Close();
(_exit as IDisposable).Dispose();
HostLogger.Shutdown();
}
return _exitCode;
}
There is no guarantee that finally will be called for certain exceptions.

Since this program runs as a Windows service it is managed by Windows. Windows detects that something went wrong because of the ApplicationException call and it sends Stop to the service which abort the thread before the finally block is executed.
The "finally" block is never executed because Windows pulls the rug from under. This is pefectly logical when you remind how exception handling works :
try {
// Do stuff
} catch (Exception e) {
// Executed first
} finally {
// Executed last
}
Since you didn't provide a catch block the ApplicationException is propagated up to the other layers and ultimately to Windows service management which handle it by sending the stop request thus aborting the thread.
Side notes :
Unmanaged exception in a service is very bad : obviously you should add a catch block and log exceptions.
Normally the Stop function is used to tell the working thread it needs to stop. This will give the thread a chance to stop in clean way. Here is a good example.
Edit :
Here is a sample of what I would do. It is more like pseudo-code but you should get the idea.
public void StartServiceCode(object state)
{
bool stopTimer = false;
try
{
LogManager.GetLogger("MyLog").Info("Locking");
lock (thisLock) {
LogManager.GetLogger("MyLog").Info("Throwing");
throw new ApplicationException();
}
} catch (Exception e) {
// The lock is relased automatically
// Logging the error (best practice)
LogManager.GetLogger("MyLog").Info("Exception occurred...");
// If severe, we need to stop the timer
if (e is StackOverflowException || e is OutOfMemoryException) stopTimer = true;
} finally {
// Always clean up
LogManager.GetLogger("MyLog").Info("finally");
}
// Do we need to stop?
if (stopTimer) {
LogManager.GetLogger("MyLog").Info("Severe exception : stopping");
// You need to keep a reference to the timer. (yes, a timer can stop itself)
timer.Stop();
}
}

The linked article explains why the finally block of a method run into a windows service provided by TopShelf library that raises an unhandled exception, it isn't executed: https://lowleveldesign.wordpress.com/2012/12/03/try-finally-topshelf-winsvc/
The problem seems related to a portion of code in the topshelf library that sleeps the thread that has raised the exception.
Follows an excerpt of the code responsible for the sleep call on the thread, this method belongs to TopShelf library
...
void CatchUnhandledException(object sender, UnhandledExceptionEventArgs e)
{
_log.Error("The service threw an unhandled exception", (Exception)e.ExceptionObject);
...
int deadThreadId = Interlocked.Increment(ref _deadThread);
Thread.CurrentThread.IsBackground = true;
Thread.CurrentThread.Name = "Unhandled Exception " + deadThreadId.ToString();
while (true)
Thread.Sleep(TimeSpan.FromHours(1));
}
...

C# Enqueue Failure

I have a simple logging mechanism that should be thread safe. It works most of the time, but every now and then I get an exception on this line, "_logQ.Enqueue(s);" that the queue is not long enough. Looking in the debugger there are sometimes just hundreds of items, so I can't see it being resources. The queue is supposed to expand as needed. If I catch the exception as opposed to letting the debugger pause at the exception I see the same error. Is there something not thread safe here? I don't even know how to start debugging this.
static void ProcessLogQ(object state)
{
try
{
while (_logQ.Count > 0)
{
var s = _logQ.Dequeue();
string dir="";
Type t=Type.GetType("Mono.Runtime");
if (t!=null)
{
dir ="/var/log";
}else
{
dir = #"c:\log";
if (!Directory.Exists(dir))
Directory.CreateDirectory(dir);
}
if (Directory.Exists(dir))
{
File.AppendAllText(Path.Combine(dir, "admin.log"), DateTime.Now.ToString("hh:mm:ss ") + s + Environment.NewLine);
}
}
}
catch (Exception)
{
}
finally
{
_isProcessingLogQ = false;
}
}
public static void Log(string s) {
if (_logQ == null)
_logQ = new Queue<string> { };
lock (_logQ)
_logQ.Enqueue(s);
if (!_isProcessingLogQ) {
_isProcessingLogQ = true;
ThreadPool.QueueUserWorkItem(ProcessLogQ);
}
}
Note that the threads all call Log(string s). ProcessLogQ is private to the logger class.
* Edit *
I made a mistake in not mentioning that this is in a .NET 3.5 environment, therefore I can't use Task or ConcurrentQueue. I am working on fixes for the current example within .NET 3.5 constraints.
** Edit *
I believe I have a thread-safe version for .NET 3.5 listed below. I start the logger thread once from a single thread at program start, so there is only one thread running to log to the file (t is a static Thread):
static void ProcessLogQ()
{
while (true) {
try {
lock (_logQ);
while (_logQ.Count > 0) {
var s = _logQ.Dequeue ();
string dir = "../../log";
if (!Directory.Exists (dir))
Directory.CreateDirectory (dir);
if (Directory.Exists (dir)) {
File.AppendAllText (Path.Combine (dir, "s3ol.log"), DateTime.Now.ToString ("hh:mm:ss ") + s + Environment.NewLine);
}
}
} catch (Exception ex) {
Console.WriteLine (ex.Message);
} finally {
}
Thread.Sleep (1000);
}
}
public static void startLogger(){
lock (t) {
if (t.ThreadState != ThreadState.Running)
t.Start ();
}
}
private static void multiThreadLog(string msg){
lock (_logQ)
_logQ.Enqueue(msg);
}

Look at the TaskParallel Library. All the hard work is already done for you. If you're doing this to learn about multithreading read up on locking techniques and pros and cons of each.
Further, you're checking if _logQ is null outside your lock statement, from what I can deduce it's a static field that you're not initializing inside a static constructor. You can avoid doing this null check (which should be inside a lock, it's critical code!) you can ensure thread-safety by making it a static readonly and initializing it inside the static constructor.
Further, you're not properly handling queue states. Since there's no lock during the check of the queue count it could vary on every iteration. You're missing a lock as your dequeuing items.
Excellent resource:
http://www.yoda.arachsys.com/csharp/threads/

For a thread-safe queue, you should use the ConcurrentQueue instead:
https://msdn.microsoft.com/en-us/library/dd267265(v=vs.110).aspx