I am cleaning up some old code converting it to work asynchronously.
psDelegate.GetStops decStops = psLoadRetrieve.GetLoadStopsByLoadID;
var arStops = decStops.BeginInvoke(loadID, null, null);
WaitHandle.WaitAll(new WaitHandle[] { arStops.AsyncWaitHandle });
var stops = decStops.EndInvoke(arStops);
Above is a single example of what I am doing for asynchronous work. My plan is to have close to 20 different delegates running. All will call BeginInvoke and wait until they are all complete before calling EndInvoke.
My question is will having so many delegates running cause problems? I understand that BeginInvoke uses the ThreadPool to do work and that has a limit of 25 threads. 20 is under that limit but it is very likely that other parts of the system could be using any number of threads from the ThreadPool as well.
Thanks!
No, the ThreadPool manager was designed to deal with this situation. It won't let all thread pool threads run at the same time. It starts off allowing as many threads to run as you have CPU cores. As soon as one completes, it allows another one to run.
Every half second, it steps in if the active threads are not completing. It assumes they are stuck and allows another one to run. On a 2 core CPU, you'd now have 3 threads running.
Getting to the maximum, 500 threads on a 2 core CPU, would take quite a while. You would have to have threads that don't complete for over 4 minutes. If your threads behave that way then you don't want to use threadpool threads.
The current default MaxThreads is 250 per processor, so effectively there should be no limit unless your application is just spewing out calls to BeginInvoke. See http://msdn.microsoft.com/en-us/library/system.threading.threadpool.aspx. Also, the thread pool will try to reuse existing threads before creating new ones in order to reduce the overhead of creating new threads. If you invokes are all fast you will probably not see the thread pool create a lot of threads.
For long running tasks or blocking tasks it is usually better to avoid the thread pool and managed the threads yourself.
However, trying to schedule that many threads will probably not yield the best results on most current machines.
Related
I have been tasked to take over an old bit of code that uses delegates.
SearchDelegate[] dlgt = new SearchDelegate[numSearches];
IAsyncResult[] ar = new IAsyncResult[numSearches];
It then does a loop to start multiple delegate functions
for (int i = 0; i < numSearches; i++)
{
ar[i] = dlgt[i].BeginInvoke(....);
}
It then does a timed loop to get the results from the ar object.
It all seems to work fine. The issue I am having is that sometimes some of these delegate functions can take 3 to 4 seconds to start, even longer if the count goes above 10. Is this a common problem, or is there a setting I can tweak?
This is running on IIS. I can replicate the issue locally with the minimal machine resources being used.
Thanks all.
Daz
can take 3 to 4 seconds to start
is caused by the threadpool. When all threads are busy it only slowly (2/second) creates new threads.
You could up the min amount of threads in the pool but especially for a web app you should research, test and measure that extensively. ASP.NET also is a big stakeholder in the threadpool.
BeginInvoke method dispatches actual work to a thread pool, as it's written in this article. It may take some time actually, when there are no available idle threads. Thread pool may decide to wait for some work items completion or to add additional threads, accounting min and max limits.
Some additional info may be found here The managed threadpool and there Simple description of worker and IO threads in net and at remarks section of this article as well ThreadPool.SetMinThreads.
You should be aware that the same thread pool is used for HTTP requests processing, therefore it's usually senseless to offload custom non IO-bound work to the thread pool in web apps, as it won't give you any benefits and may even hurt performance due to additional thread switches. While BeginInvoke doesn't look as an invocation of asynchronous IO operation.
It doesn't actually matter which concrete thread executes the work — client still have to wait for the response the same amount of time. Looks, like you may, probably, win some time by performing work in parallel, but it won't be possible under load as far as there won't be available threads at thread pool for processing both HTTP requests and your custom work items.
You may want to check this thread for some additional details on this theme. It's related to Task, but this doesn't matter as far as both BeginInvoke and Task.Run are using the same thread pool under the hood.
I'm reading Essential C# 5.0 which says,
The thread pool also assumes that all the work will be relatively
short-running (that is, consuming milliseconds or seconds of processor
time, not hours or days). By making this assumption, it can ensure
that each processor is working full out on a task, and not
inefficiently time slicing multiple tasks. The thread pool attempts to
prevent excessive time slicing by ensuring that thread creation is
"throttled" and so that no one processor is "oversubscrived" with too
many threads.
I've always thought one of the benefits of multithreading was time slicing.
If you >1 processor, than you can concurrently run threads and achieve true multithreading. But unless that's the case, you'd have to resort to time slicing for applications with multiple threads right?
So, if the ThreadPool in C# doesn't time slice, then,
a. Does it mean the ThreadPool is only used to get around the overhead in creating new threads?
b. Does it mean the ThreadPool can't run multiple threads simultaneously unless the processor has multiple cores, where each core can run a single process?
The .NET thread pool will create multiple threads per core, but has heuristics to keep the number of threads as low as possible while performing the maximum amount of work.
This means if your code is CPU-bound, you may end up with a single thread per core. If your code is I/O-bound and blocks, or queues up a huge amount of work items, you may end up with many threads per core.
It's not just thread creation that is expensive: context switching between hundreds of threads takes up a lot of time that you'd rather be spent running your own code. More threads is almost never better.
The quote you mention refers to the rate at which the Threadpool creates new threads when all of the threads that it has already created are already allocated to a task. The new thread creation rate is throttled (and new tasks are queued) to avoid creating many threads (and swamping the CPU) when a large burst of tasks are put on the Threadpool.
The current algorithm does indeed create many threads per CPU core, but it creates them relatively slowly, in the hope that the current backlog of tasks will be quickly satisfied by the threads that it has already created, and adding threads will not be needed.
Q. Does it mean the ThreadPool is only used to get around the overhead in creating new threads?
A. No. Thread creation is usually not the problem. The goal is make your CPU work at 100%, while using as as few concurrently executing threads as it is possible. The low number of threads will avoid excessive context switching, and improve overall execution time.
Q. Does it mean the ThreadPool can't run multiple threads simultaneously unless the processor has multiple cores, where each core can run a single process?
A. It runs multiple threads simultaneously. Ideally it runs exactly one CPU-intensive task per core, in which case you CPU works at 100%
I have read in several blogs that we should create our own threads for long running, or blocking tasks and not consume from the thread pool.
My question: if I set setmaxthreads to 250 and I have 25 long running tasks, should I still create my own threads? I still have the remainder threads for other small tasks.
If they are long-running tasks, you should not use the ThreadPool at all. You really should not usually tweak the thread pool settings; certainly not to avoid this. Note that the thread pool size is limited for a reason; too many threads running at one time is a bad thing, too.
So, let the ThreadPool do what it's supposed to do, and just create your own thread for your long-running tasks. (assuming you aren't creating dozens or hundreds of these; in which case you have a different problem)
When I am using the .Net 4 Task class which uses the ThreadPool, what happens if all Threads are busy?
Does the TaskScheduler create a new thread and extend the ThreadPool maximum number of threads or does it sit and wait until a thread is available?
The maximum number of threads in the ThreadPool is set to around 1000 threads on a .NET4.0 32-bit system. It's less for older versions of .NET. If you have 1000 threads going, say they're blocking for some reason, when you queue the 1001st task, it won't ever execute.
You will never hit the max thread count in a 32 bit process. Keep in mind that each thread takes at least 1MB of memory (that's the size of the user-mode stack), plus any other overhead. You already lose a lot of memory from the CLR and native DLLs loaded, so you'll hit an OutOfMemoryException before you use that many threads.
You can change the number of threads the ThreadPool can use, by calling the ThreadPool.SetMaxThreads method. However, if you're expecting to use that many threads, you have a far larger problem with your code. I do NOT recommend you mess around with ThreadPool configurations like that. You'll most likely just get worse performance.
Keep in mind with Task and ThreadPool.QueueUserWorkItem, the threads are re-used when they're done. If you create a task or queue a threadpool thread, it may or may not create a new thread to execute your code. If there are available threads already in the pool, it will re-use one of those instead of creating (an expensive) new thread. Only if the methods you're executing in the tasks never return, should you worry about running out of threads, but like I said, that's an entirely different problem with your code.
By default, the MaxThreads of the ThreadPool is very high. Usually you'll never get there, your app will crash first.
So when all threads are busy the new tasks are queued and slowly, at most 1 per 500 ms, the TP will allocate new threads.
It won't increase MaxThreads. When there are more tasks than available worker threads, some tasks will be queued and wait until the thread pool provides an available thread. It does some pretty advanced stuff to scale with a large number of cores (work-stealing, thread injection, etc).
protected override void OnStart(String[] args)
{
ResultManager.PrepareCache();
ThreadPool.QueueUserWorkItem(ResultQueue.Process);
ThreadPool.QueueUserWorkItem(StatusUpdater.UpdateStatus);
ThreadPool.QueueUserWorkItem(GeneralQueue.RestartHungTests);
ThreadPool.QueueUserWorkItem(ResultManager.SyncroniseResultsTable);
ThreadPool.QueueUserWorkItem(GeneralQueue.RecoverLostResults);
ThreadPool.QueueUserWorkItem(BrowserTestStartInfo.FillQueues);
ThreadPool.QueueUserWorkItem(MailAppAccount.FillQueues);
}
Each of these tasks run for the duration of the life of the Windows Service. I've always stuck to the ThreadPool for this kind of thing, should I be just starting normal threads? Can I be certain the ThreadPool will have enough threads available to run each task? If I SetMaxThreads to 7, will I run into issues later because these threads never abort? Is it safe to set it to something much higher?
Edit:
I always want all 7 threads to be running simultaneously, and they never abort - should I even be using threads? Is there something else more suitable for this kind of perpetually running task?
Each task runs a particular method every x minutes.
This is not an appropriate use of the thread pool. Just create normal threads, since they're long-lived. The overhead of creating the threads won't matter, since you'll only be creating them once.
As John says, this is not a good idea. The reason is that the threadpool has a limited number of threads, and you are using a large number of them and never returning them. The threadpool is designed to pool thread usage for short lived threads.
You don't really need to manage the threads since they live the lifetime of the app.