Develop Reference

Parallel.ForEach on a BlockingCollection causes steady increase of threads

I observed a strange behavior while experimenting with the Parallel.ForEach method and the BlockingCollection<T> class. Apparently calling the two lines below on a separate thread, is enough to cause an ever increasing number of threads in the ThreadPool:
var queue = new BlockingCollection<int>();
Parallel.ForEach(queue.GetConsumingEnumerable(), _ => { });
The queue contains no elements. I was expecting that the Parallel.ForEach loop would be idle, waiting for items to be added in the queue. Apparently it's not idle, because the ThreadPool.ThreadCount is increasing by one every second. Here is a minimal demo:
public class Program
{
public static void Main()
{
new Thread(() =>
{
var queue = new BlockingCollection<int>();
Parallel.ForEach(queue.GetConsumingEnumerable(), _ => { });
})
{ IsBackground = true }.Start();
Stopwatch stopwatch = Stopwatch.StartNew();
while (true)
{
Console.WriteLine($"ThreadCount: {ThreadPool.ThreadCount}");
if (stopwatch.ElapsedMilliseconds > 8000) break;
Thread.Sleep(1000);
}
Console.WriteLine("Finished");
}
}
Output:
ThreadCount: 0
ThreadCount: 4
ThreadCount: 5
ThreadCount: 6
ThreadCount: 7
ThreadCount: 8
ThreadCount: 10
ThreadCount: 11
ThreadCount: 12
Finished
Online demo.
Can anyone explain why is this happening, and how to prevent it from happening? Ideally I would like the Parallel.ForEach to consume at most one ThreadPool thread, while the queue is empty.
I am searching for a solution applicable on .NET Core 3.1 and later.

Reason
The Parallel.ForEach method partitions the enumerable first, then it uses TaskReplicator.Run to run the action on each partition.
The actual runner is the method Replica.Execute. In the beginning of the method, because the enumerable is blocked, the replica won't stop, it will create its replica until _remainingConcurrency is 0.
The _remainingConcurrency field is initialed from the ParallelOptions.EffectiveMaxConcurrencyLevel option, its default value is -1, which means unlimited. So you saw the ThreadCount kept increasing.
Solution
Because we know the reason, we just need to specific a reasonable option to limit the iteration count.
var option = new ParallelOptions
{
MaxDegreeOfParallelism = 5
};
Parallel.ForEach(queue.GetConsumingEnumerable(), option, _ => { });

How to limit number of parallel processes executed from C# program?

I need to create a code in C# that will execute a single process 1000 times (each time with different command line arguments), and I want to limit the number of processes that run in parallel to 4. In such a way so that all the rest will wait until at least one has finished, and then the next to start.
How can I do that?

you have couple options here
one is to use Parallel
Parallel.For(0, 1000, new ParallelOptions { MaxDegreeOfParallelism = 4 },
i =>
{
//do something
});
another one is to use Semaphore
Semaphore pool = new Semaphore(0, 4);
You would call pool.WaitOne() before starting the worker and pool.Release() when your worker finishes
Or you might be able to limit the thread pool's size (not recommended)

You can set the maximum number of Threads in the ThreadPool to 4:
ThreadPool.SetMaxThreads(4, 4);
Then you can queue up the 1000 processes in the ThreadPool:
for (int i = 0; i < 1000; i++)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(LaunchProcess));
}
Here is the method that will run on the background thread and run your process. It uses WaitForExit() to ensure the process finishes before the thread will finish.
static void LaunchProcess(object o)
{
var p = new Process();
p.StartInfo = new ProcessStartInfo("myprocess.exe");
p.Start();
p.WaitForExit(); // Critical to wait here for the process to finish
}
That should limit the number of active processes to 4, I believe. Haven't tested.

Define current size of thread pool in .NET

I want to know the number of active threads (iocp and worker) in threadpool. I use this technique:
ThreadPool.GetAvailableThreads(out var workerAvailable, out var iocpAvailable);
ThreadPool.GetMinThreads(out var workerMin, out var iocpMin);
ThreadPool.GetMaxThreads(out var workerMax, out var iocpMax);
var total = Process.GetCurrentProcess().Threads.Count;
var workerCurrent = workerMax - workerAvailable;
var iocpCurrent = iocpMax - iocpAvailable;
var other = total - worker - iocp;
There are quite strange numbers in logs: 8 worker, 3 iocp and 150 other threads.
I made a dump using procdump tool and inspected it using ClrMd (ClrThread.IsThreadpoolCompletionPort and ClrThread.IsThreadpoolWorker properties). Finally I got a different result from the dump: 99 worker and 14 iocp threads.
Why does the first approach return such strange result?
UPD: I suppose ThreadPool.GetAvailableThreads returns max minus currently active (not idle) threads.

Suggest a way in threading that always fixed number of threads must be used in computing?

I have an array of filepath in List<string> with thousands of files. I want to process them in a function parallel with 8 threads.
ParallelOptions opt= new ParallelOptions();
opt.TaskScheduler = null;
opt.MaxDegreeOfParallelism = 8;
Parallel.ForEach(fileList, opt, item => DoSomething(item));
This code works fine for me but it guarantees to run max 8 threads and I want to run 8 threads always. CLR decides the number of threads to be use as per CPU load.
Please suggest a way in threading that always 8 threads are used in computing with minimum overhead.

Use a producer / consumer model. Create one producer and 8 consumers. For example:
BlockingCollection<string> _filesToProcess = new BlockingCollection<string>();
// start 8 tasks to do the processing
List<Task> _consumers = new List<Task>();
for (int i = 0; i < 8; ++i)
{
var t = Task.Factory.StartNew(ProcessFiles, TaskCreationOptions.LongRunning);
_consumers.Add(t);
}
// Populate the queue
foreach (var filename in filelist)
{
_filesToProcess.Add(filename);
}
// Mark the collection as complete for adding
_filesToProcess.CompleteAdding();
// wait for consumers to finish
Task.WaitAll(_consumers.ToArray(), Timeout.Infinite);
Your processing method removes things from the BlockingCollection and processes them:
void ProcessFiles()
{
foreach (var filename in _filesToProcess.GetConsumingEnumerable())
{
// do something with the file name
}
}
That will keep 8 threads running until the collection is empty. Assuming, of course, you have 8 cores on which to run the threads. If you have fewer available cores, then there will be a lot of context switching, which will cost you.
See BlockingCollection for more information.

Within a static counter, you might be able to get the number of current threads.
Every time you call start a task there is the possibility to use the Task.ContinueWith (http://msdn.microsoft.com/en-us/library/dd270696.aspx) to notify that it's over and you can start another one.
This way there is going to be always 8 tasks running.

OrderablePartitioner<Tuple<int, int>> chunkPart = Partitioner.Create(0, fileList.Count, 1);//Partition the list in chunk of 1 entry
ParallelOptions opt= new ParallelOptions();
opt.TaskScheduler = null;
opt.MaxDegreeOfParallelism = 8;
Parallel.ForEach(chunkPart, opt, chunkRange =>
{
for (int i = chunkRange.Item1; i < chunkRange.Item2; i++)
{
DoSomething(fileList[i].FullName);
}
});

Does Parallel.ForEach limit the number of active threads?

Given this code:
var arrayStrings = new string[1000];
Parallel.ForEach<string>(arrayStrings, someString =>
{
DoSomething(someString);
});
Will all 1000 threads spawn almost simultaneously?

No, it won't start 1000 threads - yes, it will limit how many threads are used. Parallel Extensions uses an appropriate number of cores, based on how many you physically have and how many are already busy. It allocates work for each core and then uses a technique called work stealing to let each thread process its own queue efficiently and only need to do any expensive cross-thread access when it really needs to.
Have a look at the PFX Team Blog for loads of information about how it allocates work and all kinds of other topics.
Note that in some cases you can specify the degree of parallelism you want, too.

On a single core machine... Parallel.ForEach partitions (chunks) of the collection it's working on between a number of threads, but that number is calculated based on an algorithm that takes into account and appears to continually monitor the work done by the threads it's allocating to the ForEach. So if the body part of the ForEach calls out to long running IO-bound/blocking functions which would leave the thread waiting around, the algorithm will spawn up more threads and repartition the collection between them. If the threads complete quickly and don't block on IO threads for example, such as simply calculating some numbers, the algorithm will ramp up (or indeed down) the number of threads to a point where the algorithm considers optimum for throughput (average completion time of each iteration).
Basically the thread pool behind all the various Parallel library functions, will work out an optimum number of threads to use. The number of physical processor cores forms only part of the equation. There is NOT a simple one to one relationship between the number of cores and the number of threads spawned.
I don't find the documentation around the cancellation and handling of synchronizing threads very helpful. Hopefully MS can supply better examples in MSDN.
Don't forget, the body code must be written to run on multiple threads, along with all the usual thread safety considerations, the framework does not abstract that factor... yet.

Great question. In your example, the level of parallelization is pretty low even on a quad core processor, but with some waiting the level of parallelization can get quite high.
// Max concurrency: 5
[Test]
public void Memory_Operations()
{
ConcurrentBag<int> monitor = new ConcurrentBag<int>();
ConcurrentBag<int> monitorOut = new ConcurrentBag<int>();
var arrayStrings = new string[1000];
Parallel.ForEach<string>(arrayStrings, someString =>
{
monitor.Add(monitor.Count);
monitor.TryTake(out int result);
monitorOut.Add(result);
});
Console.WriteLine("Max concurrency: " + monitorOut.OrderByDescending(x => x).First());
}
Now look what happens when a waiting operation is added to simulate an HTTP request.
// Max concurrency: 34
[Test]
public void Waiting_Operations()
{
ConcurrentBag<int> monitor = new ConcurrentBag<int>();
ConcurrentBag<int> monitorOut = new ConcurrentBag<int>();
var arrayStrings = new string[1000];
Parallel.ForEach<string>(arrayStrings, someString =>
{
monitor.Add(monitor.Count);
System.Threading.Thread.Sleep(1000);
monitor.TryTake(out int result);
monitorOut.Add(result);
});
Console.WriteLine("Max concurrency: " + monitorOut.OrderByDescending(x => x).First());
}
I haven't made any changes yet and the level of concurrency/parallelization has jumped drammatically. Concurrency can have its limit increased with ParallelOptions.MaxDegreeOfParallelism.
// Max concurrency: 43
[Test]
public void Test()
{
ConcurrentBag<int> monitor = new ConcurrentBag<int>();
ConcurrentBag<int> monitorOut = new ConcurrentBag<int>();
var arrayStrings = new string[1000];
var options = new ParallelOptions {MaxDegreeOfParallelism = int.MaxValue};
Parallel.ForEach<string>(arrayStrings, options, someString =>
{
monitor.Add(monitor.Count);
System.Threading.Thread.Sleep(1000);
monitor.TryTake(out int result);
monitorOut.Add(result);
});
Console.WriteLine("Max concurrency: " + monitorOut.OrderByDescending(x => x).First());
}
// Max concurrency: 391
[Test]
public void Test()
{
ConcurrentBag<int> monitor = new ConcurrentBag<int>();
ConcurrentBag<int> monitorOut = new ConcurrentBag<int>();
var arrayStrings = new string[1000];
var options = new ParallelOptions {MaxDegreeOfParallelism = int.MaxValue};
Parallel.ForEach<string>(arrayStrings, options, someString =>
{
monitor.Add(monitor.Count);
System.Threading.Thread.Sleep(100000);
monitor.TryTake(out int result);
monitorOut.Add(result);
});
Console.WriteLine("Max concurrency: " + monitorOut.OrderByDescending(x => x).First());
}
I reccommend setting ParallelOptions.MaxDegreeOfParallelism. It will not necessarily increase the number of threads in use, but it will ensure you only start a sane number of threads, which seems to be your concern.
Lastly to answer your question, no you will not get all threads to start at once. Use Parallel.Invoke if you are looking to invoke in parallel perfectly e.g. testing race conditions.
// 636462943623363344
// 636462943623363344
// 636462943623363344
// 636462943623363344
// 636462943623363344
// 636462943623368346
// 636462943623368346
// 636462943623373351
// 636462943623393364
// 636462943623393364
[Test]
public void Test()
{
ConcurrentBag<string> monitor = new ConcurrentBag<string>();
ConcurrentBag<string> monitorOut = new ConcurrentBag<string>();
var arrayStrings = new string[1000];
var options = new ParallelOptions {MaxDegreeOfParallelism = int.MaxValue};
Parallel.ForEach<string>(arrayStrings, options, someString =>
{
monitor.Add(DateTime.UtcNow.Ticks.ToString());
monitor.TryTake(out string result);
monitorOut.Add(result);
});
var startTimes = monitorOut.OrderBy(x => x.ToString()).ToList();
Console.WriteLine(string.Join(Environment.NewLine, startTimes.Take(10)));
}

It works out an optimal number of threads based on the number of processors/cores. They will not all spawn at once.

See Does Parallel.For use one Task per iteration? for an idea of a "mental model" to use. However the author does state that "At the end of the day, it’s important to remember that implementation details may change at any time."

Does Parallel.ForEach limit the number of active threads?
If you don't configure the Parallel.ForEach with a positive MaxDegreeOfParallelism, the answer is no. With the default configuration, and assuming a source sequence with sufficiently large size, the Parallel.ForEach will use all the threads that are immediately available in the ThreadPool, and will continuously ask for more. By itself the Parallel.ForEach imposes zero restrictions to the number of threads. It is only limited by the capabilities of the associated TaskScheduler.
The default ParallelOptions.MaxDegreeOfParallelism is -1, which means unlimited parallelism.
The default ParallelOptions.TaskScheduler is the ThreadPoolTaskScheduler, better known as TaskScheduler.Default.
So if you want to understand the behavior of an unconfigured Parallel.ForEach, you have to know the behavior of the ThreadPool. Which is simple enough so that it can be described in a single paragraph. The ThreadPool satisfies immediately all requests for work, by starting new threads, up to a soft limit which be default is Environment.ProcessorCount. Upon reaching this limit, further requests are queued, and new threads are created with a rate of one new thread per second to satisfy the demand¹. There is also a hard limit on the number of threads, which in my machine is 32,767 threads. The soft limit is configurable with the ThreadPool.SetMinThreads method. The ThreadPool also retires threads, in case there are too many and there is no queued work, at about the same rate (1 per second).
Below is an experimental demonstration that the Parallel.ForEach uses all the available threads in the ThreadPool. The number of available threads
is configured up front with the ThreadPool.SetMinThreads, and then the Parallel.ForEach kicks in and takes all of them:
ThreadPool.SetMinThreads(workerThreads: 100, completionPortThreads: 10);
HashSet<Thread> threads = new();
int concurrency = 0;
int maxConcurrency = 0;
Parallel.ForEach(Enumerable.Range(1, 1500), n =>
{
lock (threads) maxConcurrency = Math.Max(maxConcurrency, ++concurrency);
lock (threads) threads.Add(Thread.CurrentThread);
// Simulate a CPU-bound operation that takes 200 msec
Stopwatch stopwatch = Stopwatch.StartNew();
while (stopwatch.ElapsedMilliseconds < 200) { }
lock (threads) --concurrency;
});
Console.WriteLine($"Number of unique threads: {threads.Count}");
Console.WriteLine($"Maximum concurrency: {maxConcurrency}");
Output (after waiting for ~5 seconds):
Number of unique threads: 102
Maximum concurrency: 102
Online demo.
The number of completionPortThreads is irrelevant for this test. The Parallel.ForEach uses the threads designated as "workerThreads". The 102 threads are:
The 100 ThreadPool threads that were created immediately on demand.
One more ThreadPool thread that was injected after 1 sec delay.
The main thread of the console application.
¹ The injection rate of the ThreadPool is not documented. As of .NET 7 it is one thread per second, but this could change in future .NET versions.