This is not about terminating a system-process but killing "myself". I have several parallel theads, which CAN hang because of different reasons.
I already created a watchdog when a thread is taking too long:
TimerCallback timerDelegate = new TimerCallback(CheckProcessStatus);
System.Threading.Timer watchDogTimer = new Timer(timerDelegate, new ProcessHealth(plog), 1000 * 60, 1000 * 60);
try
{
// lots of code here
} finally
{
watchDogTimer.Dispose();
}
Watchdog:
public void CheckProcessStatus(Object timerState) {
ProcessHealth ph = (ProcessHealth)timerState;
System.writeLine(string.Format("process runs for {0} minutes!", ph.WaitingTime);
if (ph.WaitingTime>60) {
// KILL THE PROCESS
}
}
When "lots of code here" takes too long I want to terminate the thread no matter what state it is in. (at "Kill the process").
What would be the best approach?
Thread.CurrentThread.Interrupt()
OR
Thread.CurrentThread.Abort()?
Or are there even better approaches? (I cannot use "simple" mechanisms like boolean "stop"-variables as the "lots of code here" is VERY Dynamic calling other classes via reflection etc.
Does that even work? Or do I just kill the watchdog-thread, NOT the thread to be watched?
Thread.Abort attempts to terminate the target thread by injecting an out-of-band (asynchronous) exception. It is unsafe because the exception gets injected at unpredictable points in the execution sequence. This can (and often does) lead to some type of corruption in the application domain because of interrupted writes to data structures.
Thread.Interrupt causes most blocking calls in the BCL (like Thread.Sleep, WaitHandle.WaitOne, etc.) to bailout immediately. Unlike aborting a thread, interrupting a thread can be made completely safe because the exception is injected at predictable points in the execution sequence. A crafty programmer can make sure these points are considered "safe points".
So, if "lots of code here" will respond to Thread.Interrupt then that might be an acceptable approach to use. But, I would like to steer you more towards the cooperative cancellation pattern. Basically, this means your code must periodically poll for a cancellation request. The TPL already has a framework in place for doing this via CancellationToken. But, you could easily accomplish the same thing with a ManualResetEvent or a simple volatile bool variable.
Now, if "lots of code here" is not under your control or if the cooperative cancellation pattern will not work (perhaps because you are using a faulty 3rd party library) then you pretty much have no other choice but to spin up a completely separate process to run the risky code. Use WCF to communicate with the process and if it does not respond then you can kill it without corrupting the main process. It is a lot of work, but it may be your only option.
Aborting a thread, when it is in an unknown state, is not advisable. Say, the thread is currently executing a static constructor. The static ctor will be aborted and never run again (because faulting static ctor's never run again). You have effectively destroyed global state in your AppDomain without a way to ever recover.
There are lots of other hazards as well. That just doesn't fly.
There are two production-ready choices aborting threads:
Cooperatively (set an event or a boolean flag in combination with Thread.MemoryBarrier)
Don't abort the thread, but the entire AppDomain or process. Thread level granularity is too small. You need to delete all state related to that thread, doo.
I want to stress that you cannot make this work any other way. You will have the strangest faults in production if you insist on aborting threads non-cooperatively.
Related
Let's say i have .NET Core 2.0/2.1 program.
There is a thread executing the following method. I want to stop it forcefully.
Important notes:
Cooperative multitasking (for example, with CancellationToken) is a good thing, but not the case
XY problem (https://en.wikipedia.org/wiki/XY_problem) does exist, but i just want to know if stopping this thread is actually possible
while (true)
{
var i = 0;
try
{
Console.WriteLine($"Still alive {i++}");
}
catch (Exception e)
{
Console.WriteLine($"Caught {e.GetType().Name}");
}
}
Tried several options:
Thread.Abort - throws PlatformNotSupportedException, not an option
Thread.Interrupt - only works for threads in WaitSleepJoin state, which is not the case
Calling native API methods such as TerminateThread from kernel32.dll on Windows. This approach has a lot of problems like non-released locks (https://msdn.microsoft.com/en-us/library/windows/desktop/ms686717(v=vs.85).aspx)
Concerns, from most important to least:
Releasing locks
Disposing objects in using directives
Actually collecting allocated objects
(as a corner case we can assume that out thread does not perform any heap allocations at all)
Use a ManualResetEventSlim. The instance will need to be available to both the thread you are trying to stop and the thread which will cause the stop.
In your while(true) loop, do something like this:
var shouldTerminate = mres.Wait(100);
if (shouldTerminate) { break; }
What this does is wait until the ManualResetEvent is put into a Set state, or 100ms, whichever comes first. The value returned indicates if the event is Set or Unset. You'll start off with the MRE in an Unset state, and when the control thread wishes to terminate the worker thread, it will call the Set method, and then it can Join the worker thread to wait for it to finish. This is important as in your loop you could perhaps be waiting on a network call to finish, and the worker won't actually terminate until you are back at the top of the loop again. If you need to, you could check the MRE with Wait at multiple points in the worker thread to prevent further expensive operations from continuing.
I have created a semaphore instance on top of this class
public static SemaphoreSlim _zReportSemaphore = new SemaphoreSlim(1, 500);
And somewhere in my code i need to retrieve and send some data.
while (_isRunning)
{
try
{
xBsonDocument = null;
//I think its very clear in this line...
MongoDBDAO.xGetInstance().GetZReportData(ref xBsonDocument);
foreach (BsonDocument item in xBsonDocument)
{
try
{
ThreadObject xThreadObject = new ThreadObject();
xThreadObject.m_strTerminalId = item.GetValue("_id")["TERMINAL_ID"].ToString();
xThreadObject.m_strZNo = item.GetValue("_id")["Z_NO"].ToString();
m_xBuildAndSendZReportThread =
new Thread(new ParameterizedThreadStart(vBuildAndSendZReport));
m_xBuildAndSendZReportThread.Start(xThreadObject);
}
catch (Exception xException)
{
xException.TraceError();
continue;
}
Thread.Sleep(m_litleStepQTime);
}
}
catch (Exception xException)
{
Thread.Sleep(m_bigStepQTime);
Trace.vInsertError(xException);
continue;
}
Thread.Sleep(m_iSleepTime);
}
This thread targeting to send files to ftp
private void vBuildAndSendZReport(object prm_objParameters)
{
_zReportSemaphore.Wait();
RetriveDataFromMongoAndSend();
_zReportSemaphore.Release();
}
In this structure; if i don't use a semaphore it has working great but sometimes thread count overloading the CPU or Memory usage and machine has been crushing.
1- How can i provide control over data usage (ballancing, isolating threads etc.) with this slim semaphore?
2- Can I use SemaphoreSlim for this type of job in production? What can be the advantages and disadvantages of using such a workflow organization like this?
Does it improve performance? in my special case
3- Is there another alternative that will provide system resource management and will wrap up the technical exception management
Update:
I asked this question during a job I did a long time ago. After solving the problem, I realized that I did not return.
In the above example, the report sending job was happening in the file sharing environment. Other solutions are possible, such as using a CDN.
The question was: Why should I use a thread if it can't keep me informed about what it's doing, if it doesn't tell me if it has had successful results? Why should I use SemaphoreSlim for example!?
yes, of course it can be done with async programming. but I didn't want to include this library in related environment. It had to be. I'm sure this situation is needed in many codes.
my solution was this: I eliminated the possibility of the exception in the code that was throwing the exception. so i synced the conflict with the thread outside the app. I made something like a threadpool. It worked flawlessly as a consumer. I did this by setting up a custom timing mechanism.
Regardless, I still agree. A thread should be set up to carry information about the job it is doing. I'm not talking about writing a Mutex object in between. Thread itself can carry this information.
By the way, I gave points to those who answered. Because they made the right comments according the question.
This is the first hit on Google for "Semaphore and SemaphoreSlim usage Best Practices", so I would like to add 1 remark:
At least, this code
semaphore.Wait();
DoSomeThing();
semaphore.Release();
should be at the minimum
semaphore.Wait();
try
{
DoSomeThing();
}
finally
{
semaphore.Release();
}
Or else you might end up in NEVER releasing the semaphore again if an exceptions occurs in DoSomeThing...
And in async programming, consider using
await semaphore.WaitAsync();
Is there any event when semaphore ends its all threads
No. It's not even clear what that might mean. For example, what do you want to happen if, due to thread-scheduling issues, you have just one running thread in the semaphore at the moment, and that thread completes, releasing the semaphore, before one or more other threads even get to try to acquire the semaphore?
The semaphore has no way to detect this condition as different from every thread being done.
If you want to know when some collection of asynchronous operations has completed, you'll need to wait on that specifically. You have a number of options in .NET, including:
Call Thread.Join() on all of the thread objects you've started.
Use Task to run your asynchronous tasks instead of Thread, and use Task.WhenAll() (or less preferably, Task.WaitAll()) to wait for them to complete.
Use CountdownEvent. Call AddCount() for each task you start, have each task call Signal() when it's done, and then wait on the CountdownEvent.
By the way, the code you posted is suspect in other ways:
Why are you specifying a maximum count for the SemaphoreSlim, and why is this maximum not the same as your initial count? Do you actually expect to call Release() more often than you call Wait()?
Code that calls Thread.Sleep() is often incorrect. It's not clear why you are doing that, but it's likely there are better ways to solve whatever issue you're trying to address with those calls.
Without a good Minimal, Complete, and Verifiable example, I can't say for sure that those things are wrong. But there's a low likelihood of them being right. :)
I am using BackgroundWorker for processing a long running external operation. However the user have option to cancel the background operation. Since my custom BackgroundWorker supports Thread.Abort(), all I am doing is BackgroundWorker.Abort() when user triggers Cancel from main thread.
But the thread is not actually terminating, it is still completing the external process. Is there any way I can terminate a thread instantly.
I do not have control on the external processing, so cannot send any flag for approach like while (checkThreadCancelled){}.
Below is my pseudo code.
Any help?
AbortableBackgroundWorker _bgWorker;
void MainThreadFunc()
{
_bgWorker = new AbortableBackgroundWorker();
_bgWorker.DoWork += new DoWorkEventHandler(bg_DoWork);
_bgWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler
( bg_RunWorkerCompleted );
_bgWorker.WorkerSupportsCancellation = true;
_bgWorker.RunWorkerAsync();
}
void bg_DoWork()
{
//Call external dll function for processing
}
void bg_RunWorkerCompleted()
{
//Process completed code
}
void CancelBgProcess()
{
if(_bgWorker != null)
_bgWorker.Abort();
}
The Abort method relies on worker thread cooperating with it. Ultimately it causes the CLR to throw an exception indicating that the thread is to abort, which the thread is free to deal with as it pleases.
As your worker thread is executing something in a DLL, the CLR isn't in control and therefore it does not have the option to throw an exception.
You have the option of using the Win32 TerminateThread API, but doing so is severe and may or may not lead to corruption within your process. TerminateThread is not really an option that you should ever choose.
Since you cannot modify the library that you are calling, you are left with two options. The first and easiest approach, lower the priority of the background thread and ignore the fact that it continues to run after cancellation.
The second is to launch your background operation in a separate process rather than thread. At which point, you may terminate the entire process if the operation is cancelled. If you go this route, you will need to pick some form of IPC to communicate the input and output parameters of the library.
Tasks and CancellationTokens ultimately will not help you in this situation as you will end up in the same place: executing library code that will not cooperate with you in order to be cancelled.
You don't want to use Thread.Abort, it is typically considered bad practice. There are many questions asked on SO that provide some very good explanations. For example: Timeout Pattern - How bad is Thread.Abort really?
Try looking at Tasks and CancellationTokens. See this MSDN article: http://msdn.microsoft.com/en-us/library/dd997396.aspx
Try this:
if (_bgWorker.IsBusy)
{
_bgWorker.WorkerSupportsCancellation = true;
//To cancel the Thread if Closing the Application
//while the Background Thread Worker is Still running on Background.
_bgWorker.CancelAsync();
}
It will stop the current thread process and will cancel the ongoing operation on that thread.
May be it helps you
We are using the TPL to queue long-running tasks into the threadpool.
Some of the tasks can block for some time, so we are using the following pattern to cancel them:
private void RunAction(Action action, CancellationTokenSourceWithException cts)
{
try
{
s_logger.Info("Starting action on thread ID: {0}", Utils.GetCurrentNativeThreadId());
Thread taskThread = Thread.CurrentThread;
cts.Token.Register(() => InterruptTask(taskThread));
s_logger.Info("Running next action");
action();
}
catch (Exception e)
{
cts.Cancel(e);
throw;
}
This way, calling cts.Cancel() will cause the task thread to be interrupted in case it is blocking.
This, however, has led to a problem: we don't know if the thread actually got the ThreadInterruptedException or not. It is possible that we call Thread.Interrupt() on it, but the thread will run to completion and the task will simply end. In that case, the threadpool thread will have a ticking bomb in the form of the ThreadInterruptedException, and whenver another task runs on this thread and attempts to block, it will get this exception.
A Thread.ResetInterrupted() method (similar to Thread.ResetAbort()) would be helpful here, but it does not seem to exist. We can use something like the following:
try
{
someEvent.Wait(10);
}
catch (ThreadInterruptedException) {}
To swallow the ThreadInterruptedException, but it looks ugly.
Can anyone suggest an alternative? Are we wrong to be calling Thread.Interrupt on threadpool threads? It seems like the easiest way to cancel tasks: cooperative cancellation using events etc. are much more cumbersome to use, and have to propagate into all classes that we use from the task.
You cannot do this because you don't know if/when the thread pool's threads will block when not running your own code!
Apart from the problems you mentioned, if a thread decides to block while not running your own code then the ThreadInterruptException will be unhandled and the app will immediately terminate. This is something you cannot work around with a try/block/catch guard because there is a race condition: the guard might have just completed when Thread.Interrupt is called, so if the runtime decides to have the thread block at that point you 'll get a crash.
So using Thread.Interrupt is not a viable option and you will definitely have to set up cooperative cancellation.
Apart from that, you should probably not be using the thread pool for these tasks in the first place (although there's not enough data to be . Quoting the docs (emphasis mine):
If you have short tasks that require background processing, the
managed thread pool is an easy way to take advantage of multiple
threads.
There are several scenarios in which it is appropriate to create and
manage your own threads instead of using thread pool threads:
...
You have tasks that cause the thread to block for long periods of time. The thread pool has a maximum number of threads, so a large
number of blocked thread pool threads might prevent tasks from
starting.
...
You might therefore want to consider using a thread pool of your own (there is an apparently very reputable implementation here).
Simple. You need to pass a CancellationToken to the action being called and act on it when cancellation is signalled. Messing with TPL threads with Interrupt is definitely the wrong action to take and will leave TPL in a "confused" state. Adopt the cancellation pattern all the way.
I'm currently writing a web services based front-end to an existing application. To do that, I'm using the WCF LOB Adapter SDK, which allows one to create custom WCF bindings that expose external data and operations as web services.
The SDK provides a few interfaces to implement, and some of their methods are time-constrained: the implementation is expected to complete its work within a specified timespan or throw a TimeoutException.
Investigations led me to the question "Implement C# Generic Timeout", which wisely advises to use a worker thread. Armed with that knowledge, I can write:
public MetadataRetrievalNode[] Browse(string nodeId, int childStartIndex,
int maxChildNodes, TimeSpan timeout)
{
Func<MetadataRetrievalNode[]> work = () => {
// Return computed metadata...
};
IAsyncResult result = work.BeginInvoke(null, null);
if (result.AsyncWaitHandle.WaitOne(timeout)) {
return work.EndInvoke(result);
} else {
throw new TimeoutException();
}
}
However, the consensus is not clear about what to do with the worker thread if it times out. One can just forget about it, like the code above does, or one can abort it:
public MetadataRetrievalNode[] Browse(string nodeId, int childStartIndex,
int maxChildNodes, TimeSpan timeout)
{
Thread workerThread = null;
Func<MetadataRetrievalNode[]> work = () => {
workerThread = Thread.CurrentThread;
// Return computed metadata...
};
IAsyncResult result = work.BeginInvoke(null, null);
if (result.AsyncWaitHandle.WaitOne(timeout)) {
return work.EndInvoke(result);
} else {
workerThread.Abort();
throw new TimeoutException();
}
}
Now, aborting a thread is widely considered as wrong. It breaks work in progress, leaks resources, messes with locking and does not even guarantee the thread will actually stop running. That said, HttpResponse.Redirect() aborts a thread every time it's called, and IIS seems to be perfectly happy with that. Maybe it's prepared to deal with it somehow. My external application probably isn't.
On the other hand, if I let the worker thread run its course, apart from the resource contention increase (less available threads in the pool), wouldn't memory be leaked anyway, because work.EndInvoke() never gets called? More specifically, wouldn't the MetadataRetrievalNode[] array returned by work remain around forever?
Is this only a matter of choosing the lesser of two evils, or is there a way not to abort the worker thread and still reclaim the memory used by BeginInvoke()?
Well, first off Thread.Abort is not nearly as bad as it used it to be. There were several improvements made to the CLR in 2.0 that fixed several of the major issues with aborting threads. It is still bad, mind you, so avoiding it is the best course of action. If you must resort to aborting threads then at the very least you should consider tearing down the application domain from where the abort originated. That is going to be incredibly invasive in most scenarios and would not resolve the possible corruption of unmanaged resources.
Aside from that, aborting in this case is going to have other implications. The most important being that you are attempting to abort a ThreadPool thread. I am really not sure what the end result of that would be and it could be different depending on which version of the framework is in play.
The best course of action is to have your Func<MetadataRetrievalNode[]> delegate poll a variable at safe points to see if it should terminate execution on its own.
public MetadataRetrievalNode[] Browse(string nodeId, int childStartIndex, int maxChildNodes, TimeSpan timeout)
{
bool terminate = false;
Func<MetadataRetrievalNode[]> work =
() =>
{
// Do some work.
Thread.MemoryBarrier(); // Ensure a fresh read of the terminate variable.
if (terminate) throw new InvalidOperationException();
// Do some work.
Thread.MemoryBarrier(); // Ensure a fresh read of the terminate variable.
if (terminate) throw new InvalidOperationException();
// Return computed metadata...
};
IAsyncResult result = work.BeginInvoke(null, null);
terminate = !result.AsyncWaitHandle.WaitOne(timeout);
return work.EndInvoke(result); // This blocks until the delegate completes.
}
The tricky part is how to deal with blocking calls inside your delegate. Obviously, you cannot check the terminate flag if the delegate is in the middle of a blocking call. But, assuming the blocking call is initiated from one of the canned BCL waiting mechansisms (WaitHandle.WaitOne, Monitor.Wait, etc.) then you could use Thread.Interrupt to "poke" it and that should immediately unblock it.
The answer depends on the type of work your worker thread is performing. My guess is it's working with external resources like a data connection. Thread.Abort() is indeed evil in any case of threads working with hooks to unmanaged resources, no matter how well-wrapped.
Basically, you want your service to give up if it times out. At this point, theoretically, the caller no longer cares how long the thread's going to take; it only cares that it's "too long", and should move on. Barring a bug in the worker thread's running method, it WILL end eventually; the caller just no longer cares when because it's not waiting any longer.
Now, if the reason the thread timed out is because it's caught in an infinite loop, or is told to wait forever on some other operation like a service call, then you have a problem that you should fix, but the fix is not to kill the thread. That would be analagous to sending your kid into a grocery store to buy bread while you wait in the car. If your kid keeps spending 15 minutes in the store when you think it should take 5, you eventually get curious, go in and find out what they're doing. If it's not what you thought they should be doing, like they've spent all the time looking at pots & pans, you "correct" their behavior for future occasions. If you go in and see your kid standing in a long checkout line, then you just start waiting longer. In neither of these cases should you press the button that detonates the explosive vest they're wearing; that just makes a big mess that will likely interfere with the next kid's ability to do the same errand later.