I have a multi thread application written by c#, my max thread number is 256 and this application gets the performance counters of the computers in an Ip interval(192.168.1.0 -192.168.205.255)
it works fine and turns many times in a day. because I have to get reports.
But the problem is some times one machine keeps a thread and never finishes its work so my loop doesnt turn...
Are there any way to create threads with a countdown parameter. when I start the threads in foreach?
foreach(Thread t in threads)
{
t.start(); -----> t.start(countdownParameter) etc....
}
coundown parameter is the max life of each threads. This mean if a thread cant reach a machine it have to be abort. for example 60 seconds.. no not 256 machines, I meant 256 threads... there are about 5000 ip and 600 of them are alive. soo I am using 256 threads to read their values. and the other thing is loop. my loop is working as while all off the ipies finish it starts from beginning.
You can't specify a timeout for thread execution. However, you can try to Join each thread with a timeout, and abort it if it doesn't exit.
foreach(Thread t in threads)
{
t.Start();
}
TimeSpan timeOut = TimeSpan.FromSeconds(10);
foreach(Thread t in threads)
{
if (!t.Join(timeOut))
{
// Still not complete after 10 seconds, abort
t.Abort();
}
}
There are of course more elegant ways to do it, like using WaitHandles with the WaitAll method (note that WaitAll is limited to 64 handles at a time on most implementations, and doesn't work on STA threads, like the UI thread)
You should not terminate the thread from the outside. (Never kill a thread, make it commit suicide). Killing a thread can easily corrupt the state of an appdomain if you're not very careful.
You should rewrite the network code in the threads to either time out once the time-limit has been reached, or use asynchronous network code.
Usually a thread gets stuck on a blocking call (unless of course you have a bug causing an infinite loop). You need to identify which call is blocking and "poke" it to get it to unblock. It could be that your thread is waiting inside one of the .NET BCL waiting calls (WaitHandle.WaitOne, etc.) in which case you could use Thread.Interrupt to unblock it. But, in your case it is more likely that the API managing the communication with the remote computers is hung. Sometimes you can simply close the connection from a separate thread and that will unblock the hung method (as is the case with the Socket class). If all else fails then you really might have to fall back on the method of last of calling Thread.Abort. Just keep in mind that if you abort a thread it might corrupt the state of the app domain in which the abort originated or even the entire process itself. There were a lot of provisions added in .NET 2.0 that make aborts a lot safer than they were before, but there is still some risk.
You can use smth like this:
public static T Exec<T>(Func<t> F, int Timeout, out bool Completed)
{
T result = default(T);
Thread thread = new Thread(() => result = F());
thread.Start();
Completed = thread.Join(Timeout);
if(!Completed) thread.Abort();
return result;
}
Related
Let's say i have .NET Core 2.0/2.1 program.
There is a thread executing the following method. I want to stop it forcefully.
Important notes:
Cooperative multitasking (for example, with CancellationToken) is a good thing, but not the case
XY problem (https://en.wikipedia.org/wiki/XY_problem) does exist, but i just want to know if stopping this thread is actually possible
while (true)
{
var i = 0;
try
{
Console.WriteLine($"Still alive {i++}");
}
catch (Exception e)
{
Console.WriteLine($"Caught {e.GetType().Name}");
}
}
Tried several options:
Thread.Abort - throws PlatformNotSupportedException, not an option
Thread.Interrupt - only works for threads in WaitSleepJoin state, which is not the case
Calling native API methods such as TerminateThread from kernel32.dll on Windows. This approach has a lot of problems like non-released locks (https://msdn.microsoft.com/en-us/library/windows/desktop/ms686717(v=vs.85).aspx)
Concerns, from most important to least:
Releasing locks
Disposing objects in using directives
Actually collecting allocated objects
(as a corner case we can assume that out thread does not perform any heap allocations at all)
Use a ManualResetEventSlim. The instance will need to be available to both the thread you are trying to stop and the thread which will cause the stop.
In your while(true) loop, do something like this:
var shouldTerminate = mres.Wait(100);
if (shouldTerminate) { break; }
What this does is wait until the ManualResetEvent is put into a Set state, or 100ms, whichever comes first. The value returned indicates if the event is Set or Unset. You'll start off with the MRE in an Unset state, and when the control thread wishes to terminate the worker thread, it will call the Set method, and then it can Join the worker thread to wait for it to finish. This is important as in your loop you could perhaps be waiting on a network call to finish, and the worker won't actually terminate until you are back at the top of the loop again. If you need to, you could check the MRE with Wait at multiple points in the worker thread to prevent further expensive operations from continuing.
I wrote a multithreaded app which uses async/await extensively. It is supposed to download some stuff at a scheduled time. To achieve that, it uses 'await Task.Delay'. Sometimes it sends thousands requests every minute.
It works as expected, but sometimes my program needs to log something big. When it does, it serializes many objects and saves them to a file. During that time, I noticed that my scheduled tasks are executed too late. I've put all the logging to a separate thread with the lowest priority and the problem doesn't occur that often anymore, but it still happens. The things is, I want to know when it happens and in order to know that I have to use something like that:
var delayTestDate = DateTime.Now;
await Task.Delay(5000);
if((DateTime.Now - delayTestDate).TotalMilliseconds > 6000/*delays up to 1 second are tolerated*/) Console.WriteLine("The task has been delayed!");
Moreover, I have found that 'Task.Run', which I also use, can also cause delays. To monitor that, I have to use even more ugly code:
var delayTestDate = DateTime.Now;
await Task.Run(() =>
{
if((DateTime.Now - delayTestDate).TotalMilliseconds > 1000/*delays up to 1 second are tolerated*/) Console.WriteLine("The task has been delayed!");
//do some stuff
delayTestDate = DateTime.Now;
});
if((DateTime.Now - delayTestDate).TotalMilliseconds > 1000/*delays up to 1 second are tolerated*/) Console.WriteLine("The task has been delayed!");
I have to use it before and after every await and Task.Run and inside every async function, which is ugly and inconvenient. I can't put it into a separate function, since it would have to be async and I would have to await it anyway. Does anybody have an idea of a more elegant solution?
EDIT:
Some information I provided in the comments:
As #YuvalItzchakov noticed, the problem may be caused by Thread Pool starvation. That's why I used System.Threading.Thread to take care of the logging outside of the Thread Pool, but as I said, the problem still sometimes occur.
I have a processor with four cores and by subtracting results of ThreadPool.GetAvailableThreads from ThreadPool.GetMaxThreads I get 0 busy worker threads and 1-2 busy completion port threads. Process.GetCurrentProcess().Threads.Count usually returns about 30. It's a Windows Forms app and although it only has a tray icon with a menu, it starts with 11 threads. When it gets to sending thousands requests per minute, it quickly gets up to 30.
As #Noseratio suggested, I tried to play with ThreadPool.SetMinThreads and ThreadPool.SetMaxThreads, but it didn't even change the numbers of busy threads mentioned above.
When you execute Task.Run it uses Thread Pool threads to execute those tasks. When you have long running tasks, you are causing starvation to the Thread Pool, since its resources are currently occupied with long running tasks.
2 Suggestions:
When running long running tasks, make sure to use Task.Factory.Startnew with TaskCreationOptions.LongRunning, which will trigger a new thread creation. You must be cautious here as well, as spinning too many new threads will cause excessive context switches which will cause your app to slow down
Use true async where you have to do IO Bound work, use apis that support the TAP such as HttpClient and Stream, which wont cause a new thread to execute blocking work.
There are overheads in async/await, as well as the tasks themselves being executed at a lower priority. If you need something to happen reliably at an accurate interval, async/await / TPL is not the interface to use.
Try creating an independent background thread that loops until it is scheduled to do work. This way you can control the priority and timing directly without going through TPL / async.
Thread backgroundThread = new Thread(BackgroundWork);
DateTime nextInterval = DateTime.Now;
public void BackgroundWork()
{
if(DateTime.Now > nextInterval){
DoWork();
nextInterval = nextInterval.Add(new TimeSpan(0,0,0,10)); // 10 seconds
}
Thread.Sleep(100);
}
Adjust the Sleep(..) and interval values as needed.
I think you're experiencing the situation described by Joe Duffy in his "CLR thread pool injection, stuttering problems" blog post:
One silly thing our thread pool currently does has to do with how it
creates new threads. Namely, it severely throttles creation of new
threads once you surpass the “minimum” number of threads, which, by
default, is the number of CPUs on the machine. We limit ourselves to
at most one new thread per 500ms once we reach or surpass this number.
One solution might be to explicitly increase the minimum number of thread pool threads before making any use of TPL, e.g.:
ThreadPool.SetMaxThreads(workerThreads: 200, completionPortThreads: 200);
ThreadPool.SetMinThreads(workerThreads: 100, completionPortThreads: 100);
Try playing with these numbers and see if the problem goes away.
I am using VS 2012, .Net 4.5.
Execute this code (just upgrade some sample from article about threading):
using System.Threading;
class BasicWaitHandle
{
static EventWaitHandle wh = new AutoResetEvent(false);
static void Main()
{
new Thread(Waiter).Start();
new Thread(Waiter).Start();
Thread.Sleep(1000); // Подождать некоторое время...
wh.Set(); // OK – можно разбудить
wh.Set();
Console.ReadLine();
}
static void Waiter()
{
Console.WriteLine("Avait..."+Thread.CurrentThread.ManagedThreadId);
wh.WaitOne(); // Ожидать сигнала
Console.WriteLine("Got a signal"+Thread.CurrentThread.ManagedThreadId);
}
}
I Debug it few times, but usually (not always) get wrong result. At first (once or more times) it correct:
Avait...10
Avait...11
Got a signal 11
Got a signal 10
But then it just start skipping one thread (somethimes first? somethimes second):
Avait...10
Avait...11
Got a signal 11 (or 10)
And program just does not react. In a few minutes it gives some correct results, but then go wrong again...
Moreover, when i debugging it step-by-step it always acting correctly.
So, maybe I should choose another approach? But this looks like what I expected, even if threads got signals in random order...
I am pretty unsure you can use same AutoResetEvent for multiple awaters, because Set is not waiting for first thread to complete its Wait:
There is no guarantee that every call to the Set method will release a thread from an EventWaitHandle whose reset mode is EventResetMode.AutoReset. If two calls are too close together, so that the second call occurs before a thread has been released, only one thread is released. It is as if the second call did not happen. Also, if Set is called when there are no threads waiting and the EventWaitHandle is already signaled, the call has no effect.
I'd go with ManualResetEvent and synchronization during setting signal (to ensure, what waiting thread receive signal) or (better) use dedicated event for each waiting function (every thread would start with its own event to wait for, you will need kind of manager for those thread to create waiting event and to have Set method what will signal all these events).
p.s.: can repeat said above in russian btw ^^
Both threads start and run until they block on the WaitHandle. When the WaitHandle is set, one thread will wake up and the event will reset.
You can't guarantee which thread will wake up, so the order isn't ensured. When running correctly, either 10 or 11 will wake up, followed by the other, every time.
In the case where your application hangs, the problem is the execution order. The main thread is executing both calls to Event.Set() prior to the first thread waking up. The AutoResetEvent is not a counter, it is either set or unset, so the second call to Set() is lost.
If you Sleep() between calls to Set(), you will yield to the other threads and give one of them time to wake up and reset the event.
In the case where it works correctly, you are just getting lucky and the waiting threads are getting a chance to run between calls to Set(). This is referred to as a race condition.
There are 5 threads running in an infinite loop.
2 of them will send messages when queue is not empty.
4 of them will keep sending heartbeat within 5 minutes.
1 of them is to request data from another source.
When it utilizes 100% of the CPU, I can not use any other applications in the window. The whole window becomes very slow.
EDIT: can sleep be put after WaitOne?
if(autoEvent.WaitOne())
{
}
else
{
}
Thread.Sleep(100);
Can sleep be put after subscriber.Recv() which is ZeroMQ ?
all threads i put a sleep if no Recv(), however there is one thread i do not dare to put a sleep in realtime datafeed thread which has only client.Send, will just one thread cause 100% ?
Q: How to make a program not utilize 100% CPU?
A: Don't create a busy loop!!!!
Blocking is Good. There are lots of ways to accomplish "block until there's something to do". Including using an alarm signal or timer (poor, but a definite improvement), doing a socket read with a timeout (if you happen to be notified with a network socket) or using a Windows Event object with a timeout.
Failing all else, you can always use a "Sleep()". I would discourage using "Sleep" if you can avoid it - there are almost always much better design strategies. But it will keep you from a 100% CPU busy loop ;)
=======================================
Addendum: you posted some code (thank you!)
You're using xxx.WaitOne().
Just use WaitOne() (a blocking call), with a timeout. This is an IDEAL solution: no busy loop, no "Sleep" required!
http://msdn.microsoft.com/en-us/library/aa332441%28v=vs.71%29.aspx
Put System.Threading.Thread.Sleep(100) (100 milliseconds sleep = time for system to do something else) in your infinite loops.
For the threads that send messages, when the queue is emtpy, use a ResetEvent
DeliverMessageThread_DoWork
{
while(true)
{
if(GetNextMessage() == null)
MyAutoResetEvent.WaitOne(); // The thread will suspend here until the ARE is signalled
else
{
DeliverMessage();
Thread.Sleep(10); // Give something else a chance to do something
}
}
}
MessageGenerator_NewMessageArrived(object sender, EventArgs e)
{
MyAutoResetEvent.Set(); // If the deliver message thread is suspended, it will carry on now until there are no more messages to send
}
This way, you won't have 2 threads chewing up all of the CPU cycles all of the time
We are using the TPL to queue long-running tasks into the threadpool.
Some of the tasks can block for some time, so we are using the following pattern to cancel them:
private void RunAction(Action action, CancellationTokenSourceWithException cts)
{
try
{
s_logger.Info("Starting action on thread ID: {0}", Utils.GetCurrentNativeThreadId());
Thread taskThread = Thread.CurrentThread;
cts.Token.Register(() => InterruptTask(taskThread));
s_logger.Info("Running next action");
action();
}
catch (Exception e)
{
cts.Cancel(e);
throw;
}
This way, calling cts.Cancel() will cause the task thread to be interrupted in case it is blocking.
This, however, has led to a problem: we don't know if the thread actually got the ThreadInterruptedException or not. It is possible that we call Thread.Interrupt() on it, but the thread will run to completion and the task will simply end. In that case, the threadpool thread will have a ticking bomb in the form of the ThreadInterruptedException, and whenver another task runs on this thread and attempts to block, it will get this exception.
A Thread.ResetInterrupted() method (similar to Thread.ResetAbort()) would be helpful here, but it does not seem to exist. We can use something like the following:
try
{
someEvent.Wait(10);
}
catch (ThreadInterruptedException) {}
To swallow the ThreadInterruptedException, but it looks ugly.
Can anyone suggest an alternative? Are we wrong to be calling Thread.Interrupt on threadpool threads? It seems like the easiest way to cancel tasks: cooperative cancellation using events etc. are much more cumbersome to use, and have to propagate into all classes that we use from the task.
You cannot do this because you don't know if/when the thread pool's threads will block when not running your own code!
Apart from the problems you mentioned, if a thread decides to block while not running your own code then the ThreadInterruptException will be unhandled and the app will immediately terminate. This is something you cannot work around with a try/block/catch guard because there is a race condition: the guard might have just completed when Thread.Interrupt is called, so if the runtime decides to have the thread block at that point you 'll get a crash.
So using Thread.Interrupt is not a viable option and you will definitely have to set up cooperative cancellation.
Apart from that, you should probably not be using the thread pool for these tasks in the first place (although there's not enough data to be . Quoting the docs (emphasis mine):
If you have short tasks that require background processing, the
managed thread pool is an easy way to take advantage of multiple
threads.
There are several scenarios in which it is appropriate to create and
manage your own threads instead of using thread pool threads:
...
You have tasks that cause the thread to block for long periods of time. The thread pool has a maximum number of threads, so a large
number of blocked thread pool threads might prevent tasks from
starting.
...
You might therefore want to consider using a thread pool of your own (there is an apparently very reputable implementation here).
Simple. You need to pass a CancellationToken to the action being called and act on it when cancellation is signalled. Messing with TPL threads with Interrupt is definitely the wrong action to take and will leave TPL in a "confused" state. Adopt the cancellation pattern all the way.