I've written a program which uses all available cores by using Parallel.ForEach. The list for the ForEach contains ~1000 objects and the computation for each object take some time (~10 sec).
In this scenario I setup a timer like this:
timer = new System.Timers.Timer();
timer.Elapsed += TimerHandler;
timer.Interval = 15000;
timer.Enabled = true;
private void TimerHandler(object source, ElapsedEventArgs e)
{
Console.WriteLine(DateTime.Now + ": Timer fired");
}
At the moment the TimerHandler method is a stub to make sure the problem isn't caused by this method.
My expectation was that the TimerHandler method will be executed every ~15 seconds. However, the time between two calls to this method even reaches 40 seconds, so 25 seconds too much.
By using new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount -1 } for the Parallel.ForEach method this doesn't happen and the expected interval of 15 seconds is seen.
Is it intended that I have to make sure that there is always one core available per active timer? Seems to be a bit odd even more, because the "reserved" could be a valuable resource for my computation.
Edit: As indicated by Yuval setting a fixed minimum of threads in the pool by ThreadPool.SetMinThreads solved the problem. I also tried new ParallelOptions { MaxDegreeOfParallelism = Environment.ProcessorCount } (so without the -1 in the initial question) for the Parallel.ForEach method and this also solves the problem. However, I have no good explanation why these modifications solved the problem. Maybe there were so many threads created that the timer thread just got "lost" for a "long" time until it was executed again.
The Parallel class uses an internal TPL facility called self replicating tasks. They are meant to consume all available thread resources. I don't know what kind of limits are in place but it seems that it's all-consuming. I have answered basically the same question a few days ago.
The Parallel class is prone to spawn insane amounts of tasks without any limit. It is easy to provoke it to literally spawn unlimited threads (2 per second). I consider the Parallel class unusable without a manually specified max-DOP. It is a time bomb that explodes in production randomly under load.
Parallel is especially poisonous in ASP.NET scenarios where many requests share one thread-pool.
Update: I forgot to make the key point. Timer ticks are queued to the thread pool. If the pool is saturated they get in line and execute at a later time. (This is the reason why timer ticks can happen concurrently or after timers are stopped.) This explains what you are seeing. The way to fix that is to fix the pool overloading.
The best fix for this particular scenario would be a custom task scheduler with a fixed amount of threads. Parallel can be made to use that task scheduler. The Parallel Extension Extras have such a scheduler. Get that work off the globally shared thread pool. Normally, I would recomment PLINQ but that is not able to take a scheduler. In a sense both Parallel and PLINQ are needlessly crippled APIs.
Don't use ThreadPool.SetMinThreads. Don't mess with global process wide settings. Just leave the poor thread pool alone.
Also, don't use Environment.ProcessorCount -1 because that wastes one core.
the timer is already executed in its own thread
The timer is a data structure in the OS kernel. There is no thread until a tick must be queued. Not sure how exactly that works but the tick is queued to the thread pool in .NET eventually. That's when the problem starts.
As a solution you could start a thread that sleeps in a loop in order to simulate a timer. That's a hack, though, because it does not fix the root cause: The overloaded thread pool.
Related
I've found a ton of related content that all beat around the bush and I've never been able to find an answer. I'm almost 100% certain that Task.Delay(int) does not use a thread, because I can run this code on my machine with only 16 logical processors:
var tasks = new List<Task>();
for(int i = 1; i < 100000; i++) tasks.Add(Task.Delay(10000));
await Task.WhenAll(tasks);
And it takes ten seconds to complete. If it were using roughly a hundred thousand threads it would take quite a bit longer, I'd think.
So my question is how does Task.Delay(int) work? Not in the manner that this poorly-entitled SO question indicates, but from a threading and hardware resources standpoint.
In the current implementation of .NET, there is a single "timer thread" that just keeps track of managed timer instances and raises their events at the appropriate times. This timer thread will block on its control signal with a timeout set to the next timer's due time. The control signal is used to add/remove/change timers, so when this blocking request times out, the timer thread knows the next timer has fired. This is a normal thread blocking operation, so internally, the thread is idled and removed from the scheduler queue until that blocking operation completes or is timed out. The timing out of those operations is handled by the OS scheduler's timer interrupt.
So technically there is a thread, but it's only one thread per process, not one thread per Task.Delay.
I again stress that this is in the current implementation of .NET. Other solutions have been proposed, such as one timer thread per CPU, or a dynamic pool of timer threads. Perhaps they were experimented with and rejected for some reason, or perhaps an alternative solution will be adopted in the future. AFAIK this is not officially documented anywhere, so this is an implementation detail.
Context: we have a task which might take from 30 seconds to 5 minutes depending on a service we are consuming in some Azure Functions.
We are planning to monitor the current status of that task object to make sure it's running and has not been cancelled/faulted.
There are two ways to go around it:
Create a Task, run it and then cancel it when the main task is finished. Alternatively, maybe use Task.Delay along with a while with a condition.
Create a Thread, run it and wait for it to finish (with a while condition to avoid a while that runs forever).
We have done some research and have realised that both have pros and cons. But we are still not sure about which one would be the best approach and why.
In a similar scenario, what would you use? A task, a thread, or something else?
Using a thread is a bit wasteful, but slightly more reliable.
It is wasteful because each thread allocates 1 MB of memory just for its mere existence.
It is more reliable because it doesn't depend on the availability of ThreadPool threads for running a timer event. A sudden burst in demand for ThreadPool threads could leave the ThreadPool starved for several seconds, or even minutes (in extreme scenarios).
So if wasting 1 MB of memory is a non-issue for the app, use a thread. On the other hand if the absolute precision in the timing of the events is something unimportant, use a task.
You could also use a task started with the option LongRunning, but this is essentially a thread in disguise.
I've a c# single threaded application and currently working on to make it multi-threaded with the use of thread pools. I am stuck in deciding which model would work for my problem.
Here's my current scenario
While(1)
{
do_sometask();
wait(time);
}
And this is repeated almost forever. The new scenario has multiple threads which does the above. I could easily implement it by spawning number of threads based on the tasks I have to perform, where all the threads perform some task and wait forever.
The issue here is I may not know the number of tasks, so I can't just blindly spawn 500 threads. I thought about using threadpool, but because almost every thread loops forever and won't ever be freed up for new tasks in the queue, am not sure which other model to use.
I am looking for an idea or solution where I could break the loop in the thread and free it up instead of waiting, but come back and resume the same task after the wait(when the time gets elapsed, using something like a timer/checking timestamp of when the last task is performed).
With this I could use a limited number of threads (like in a thread pool) and serve the tasks which are coming in during the time old threads waits(virtually).
Any help is really appreciated.
If all you have is a bunch of things that happen periodically, it sounds what you want is a bunch of timers. Create a timer for each task, to fire when appropriate. So if you have two different tasks:
using System.Threading;
// Task1 happens once per minute
Timer task1Timer = new Timer(
s => DoTask1(),
null,
TimeSpan.FromMinutes(1),
TimeSpan.FromMinutes(1));
// Task2 happens once every 47 seconds
Timer task2Timer = new Timer(
s => DoTask2(),
null,
TimeSpan.FromSeconds(47),
TimeSpan.FromSeconds(47);
The timer is a pretty lightweight object, so having a whole bunch of them isn't really a problem. The timer only takes CPU resources when it fires. The callback method will be executed on a pool thread.
There is one potential problem. If you have a whole lot of timers all with the same period, then the callbacks will all be called at the same time. The threadpool should handle that gracefully by limiting the number of concurrent tasks, but I can't say for sure. But if your wait times are staggered, this is going to work well.
If you have small wait times (less than a second), then you probably need a different technique. I'll detail that if required.
With this design, you only have one thread blocked at any time.
Have one thread (the master thread) waiting on a concurrent blocking collection, such as the BlockingCollection. This thread will be blocked by a call to TryTake until something is placed in the collection, or after a certain amount of time has passed via a timeout passed into the call (more on this later).
Once it is unblocked, it may have a unit of work to be processed. It checks to see if there is one (i.e., the TryTake call didn't time out), then if there is capacity to perform this work, and if so, queues up a thread (pool, Task or whatevs) to service the work. This master thread then goes back to the blocking collection and tries to take another unit of work. The cycle continues.
As a unit of work is begun, it will be noted so that the main thread can see how many threads are working. Once this unit is completed, the notation will be removed. The thread is then freed.
You want to use a timeout so that if it is judged that too many operations are running concurrently, you will be able to re-evaluate this a set period of time down the road. Otherwise, that unit of work sits in the blocking collection until a new unit is added, which is not optimal.
Outside users of this instance can queue up new units of work by simply dropping them in the collection.
You can use a cancellation token to immediately unblock the thread when it's time to shut down operations. Have the worker operations take cancellation tokens as well so they can halt on shutdown.
I could implement it with the help of a threadpool and few conditions to check the last activity of the task before adding it to the threadpool queue.
I'm using a System.Threading.Timer which is configured to expire in 100ms. Typically it will call the callback method within 10ms of the expected time, however, the callback is frequently called up to 500ms late. By frequent, I mean around 25% of the time.
Can anybody explain this?
Well, 500 msec is in fact a magic number in a .NET program. That's how often the threadpool manager considers adding another thread to the pool because the existing ones appear to be stuck and not making progress. The Timer callback is made on a tp thread.
So, sight unseen, a conclusion you can draw is that the callback cannot run soon enough because you have far too many threadpool threads active in your program. So the timer callback only gets a chance to run when the tp manager forcibly adds another thread to the pool.
If accurate, this is pretty unhealthy. Could be that you have a lot of tp threads that are burning 100% core. Easy to see from Task Manager, you'll see the CPU usage completely pegged at 100%. But far more common is that they are not being used effectively, instead of executing code they are blocking. Most typically on an I/O request, a socket read or dbase query for example. Such code should not run on a tp thread, it should run on a regular Thread or a Task that was configured with TaskCreationOptions.LongRunning. Or made more efficient by making it asynchronous, the C# v5 asych/await keywords can make that a lot easier.
A sledge-hammer solution is to call ThreadPool.SetMinThreads() and bump up the minimum. Only ever consider this if you cannot afford the time to do it right.
It's quite simple:
1) your OS is not RTOS (RealTime OS)
2) System.Threading.Timer executes its callback on thread from ThreadPool and it can guarantee only the fact that callback will be called after time interval elapsed.
I am cleaning up some old code converting it to work asynchronously.
psDelegate.GetStops decStops = psLoadRetrieve.GetLoadStopsByLoadID;
var arStops = decStops.BeginInvoke(loadID, null, null);
WaitHandle.WaitAll(new WaitHandle[] { arStops.AsyncWaitHandle });
var stops = decStops.EndInvoke(arStops);
Above is a single example of what I am doing for asynchronous work. My plan is to have close to 20 different delegates running. All will call BeginInvoke and wait until they are all complete before calling EndInvoke.
My question is will having so many delegates running cause problems? I understand that BeginInvoke uses the ThreadPool to do work and that has a limit of 25 threads. 20 is under that limit but it is very likely that other parts of the system could be using any number of threads from the ThreadPool as well.
Thanks!
No, the ThreadPool manager was designed to deal with this situation. It won't let all thread pool threads run at the same time. It starts off allowing as many threads to run as you have CPU cores. As soon as one completes, it allows another one to run.
Every half second, it steps in if the active threads are not completing. It assumes they are stuck and allows another one to run. On a 2 core CPU, you'd now have 3 threads running.
Getting to the maximum, 500 threads on a 2 core CPU, would take quite a while. You would have to have threads that don't complete for over 4 minutes. If your threads behave that way then you don't want to use threadpool threads.
The current default MaxThreads is 250 per processor, so effectively there should be no limit unless your application is just spewing out calls to BeginInvoke. See http://msdn.microsoft.com/en-us/library/system.threading.threadpool.aspx. Also, the thread pool will try to reuse existing threads before creating new ones in order to reduce the overhead of creating new threads. If you invokes are all fast you will probably not see the thread pool create a lot of threads.
For long running tasks or blocking tasks it is usually better to avoid the thread pool and managed the threads yourself.
However, trying to schedule that many threads will probably not yield the best results on most current machines.