I understand that thread pool priority should/can not be changed by the running process, but is the priority of particular task running on the thread pool somewhat priced-in with the calling process priority?
in other words, do all tasks in thread pool run in the same priority regardless the calling process priority?
thank you
update 1: i should have been more specific, i refer to thread inside Parallel.ForEach
I understand that thread pool priority should/can not be changed by the running process,
That's not exact. You can change Thread Pool's thread priority (inside delegate itself) and it'll run with new priority but default one will be restored when its task finishes and it'll be send back to pool.
ThreadPool.QueueUserWorkItem(delegate(object state) {
Thread.CurrentThread.Priority = ThreadPriority.Highest;
// Code in this function will run with Highest priority
});
is the priority of particular task running on the thread pool somewhat priced-in with the calling process priority?
Yes, and it doesn't apply to Thread Pool's threads only. In Windows process' priority is given by its class (from IDLE_PRIORITY_CLASS to REALTIME_PRIORITY_CLASS). Together with thread's priority (from THREAD_PRIORITY_IDLE to THREAD_PRIORITY_TIME_CRITICAL) it'll be used to calculate thread's final priority.
From MSDN:
The process priority class and thread priority level are combined to form the base priority of each thread.
Note that it's not simply a base priority plus an offset:
NORMAL_PRIORITY_CLASS + THREAD_PRIORITY_IDLE == 1
NORMAL_PRIORITY_CLASS + THREAD_PRIORITY_TIME_CRITICAL == 15
But:
REALTIME_PRIORITY_CLASS + THREAD_PRIORITY_IDLE == 16
REALTIME_PRIORITY_CLASS + THREAD_PRIORITY_TIME_CRITICAL == 31
Moreover threads can have a temporary boost (decided and managed by Windows Scheduler). Be aware that a process can also change its own priority class.
in other words, do all tasks in thread pool run in the same priority regardless the calling process priority?
No, thread's priority depends on process' priority (see previous paragraph) and each thread in pool can temporary have a different priority. Also note that thread priority isn't affected by caller thread's priority:
ThreadPool.QueueUserWorkItem(delegate(object s1) {
Thread.CurrentThread.Priority = ThreadPriority.Highest;
ThreadPool.QueueUserWorkItem(delegate(object s2) {
// This code is executed with ThreadPriority.Normal
Thread.CurrentThread.Priority = ThreadPriority.Lowest;
// This code is executed with ThreadPriority.Lowest
});
// This code is executed with ThreadPriority.Highest
});
EDIT: .NET tasks uses Thread Pool than what wrote above still applies. If, for example, you're enumerating a collecition with Parallel.ForEach to increase thread priority you have to do it inside your loop:
Parallel.ForEach(items, item => {
Thread.CurrentThread.Priority = ThreadPriority.Highest;
// Your code here...
});
Just a warning: be careful when you change priorities. If, for example, two threads use a shared resource (protected by a lock), there are many races to acquire that resources and one of them has highest priority then you may end with a very high CPU usage (because of spinning behavior of Monitor.Enter). This is just one issue, please refer to MSDN for more details (increasing thread's priority may even result is worse performance).
do all tasks in thread pool run in the same priority regardless the calling process priority?
They have to. The only thing dropped off at the pool is a delegate. That holds a reference to an object but not to the Thread that dropped it off.
The ones that are currently running have the same priority. But there's a queue for the ones that aren't running yet - so in practice, there is a "priority". Even more confusingly, thread priorities can be boosted (and limited) by the OS, for example when two threads in the threadpool depend on each other (e.g. one is blocking the other). And of course, any time something is blocking on the threadpool, you're wasting resources :D
That said, you shouldn't really change thread priorities at all, threadpool or not. You don't really need to, and thread (and process) priorities don't work the way you probably expect them to - it's just not worth it. Keep everything at normal, and just ignore that there's a Priority property, and you'll avoid a lot of unnecessary problems :)
You'll find a lot of nice explanations on the internet - for example, http://blog.codinghorror.com/thread-priorities-are-evil/. Those are usually dated, of course - but so is the concept of thread priorities, really - they were designed for single-core machines at a time when OSes weren't all that good at pre-emptive multitasking, really.
Related
My question is a bit nit-picky on definitions:
Can the code below be described as "busy waiting"? Despite the fact that it uses Thread.Sleep() to allow for context switching?
while (true) {
if (work_is_ready){
doWork();
}
Thread.Sleep(A_FEW_MILLISECONDS);
}
PS - The current definition for busy waiting in Wikipedia suggests that it is a "less wasteful" form of busy waiting.
Any polling loop, regardless of the time between polling operations, is a busy wait. Granted, sleeping a few milliseconds is a lot less "busy" than no sleep at all, but it still involves processing: thread context switches and some minimal condition checking.
A non-busy wait is a blocking call. The non-busy version of your example would involve waiting on a synchronization primitive such as an event or a condition variable. For example, this pseudocode:
// initialize an event to be set when work is ready
Event word_is_ready;
work_is_ready.Reset();
// in code that processes work items
while (true)
{
work_is_ready.Wait(); // non-busy wait for work item
do_work();
}
The difference here is that there is no periodic polling. The Wait call blocks and the thread is never scheduled until the event is set.
That's not busy waiting. Busy waiting, or spinning, involves the opposite: avoiding context switching.
If you want to allow other threads to run, if and only if other threads are ready to run, to avoid deadlock scenarios in single threaded CPUs (e.g., the current thread needs work_is_ready to be set to true, but if this thread doesn't give up the processor and lets others run, it will never be set to true), you can use Thread.Sleep(0).
A much better option would be to use SpinWait.SpinUntil
SpinWait.SpinUntil(() => work_is_ready);
doWork();
SpinWait emits a special rep; nop (repeat no-op) or pause instruction that lets the processor know you're busy waiting, and is optimized for HyperThreading CPUs.
Also, in single core CPUs, this will yield the processor immediately (because busy waiting is completely useless if there's only one core).
But spinning is only useful if you're absolutely sure you won't be waiting on a condition for longer than it would take the processor to switch the context out and back in again. I.e., no more than a few microseconds.
If you want to poll for a condition every few milliseconds, then you should use a blocking synchronization primitive, as the wiki page suggests. For your scenario, I'd recommend an AutoResetEvent, which blocks the thread upon calling WaitOne until the event has been signaled (i.e, the condition has become true).
Read also: Overview of Synchronization Primitives
It depends on the operating system and the exact number of milliseconds you are sleeping. If the sleep is sufficiently long that the operating system can switch to another task, populate its caches, and usefully run that task until your task is ready-to-run again, then it's not busy waiting. If not, then it is.
To criticize this code, I would say something like this: "This code may busy wait if the sleep is too small to allow the core to do useful work between checks. It should be changed so that the code that makes this code need to do work triggers that response."
This poor design creates a needless design problem -- how long should the sleep be? If it's too short, you busy wait. If it's too long, the work sits undone. Even if it's long enough that you don't busy wait, you force needless context switches.
When your code is sleeping for a moment, technically it will be in sleep state freeing up a CPU. While in busy waiting your code is holding the CPU until condition is met.
Can the code below be described as "busy waiting"? Despite the fact that it uses Thread.Sleep() to allow for context switching?
It is not busy waiting, rather polling which is more performant that busy waiting. There is a difference between both
Simply put, Busy-waiting is blocking where as Polling is non-blocking.
Busy waiting is something like this:
for(;;) {
if (condition) {
break;
}
}
The condition could be "checking the current time" (for example performance counter polling). With this you can get a very accurate pause in your thread. This is useful for example for low level I/O (toggling GPIOs etc.). Because of this your thread is running all the time, and if you are on cooperative multi threading, the you are fully in control, how long the thread will stay in that wait for you. Usually this kind of threads have a high priority set and are uninterruptible.
Now a non-busy waiting means, the thread is non-busy. It allows another threads to execute, so there is a context switch. To allow a context switch, in most languages and OS you can simply use a sleep(). There are another similar functions, like yield(), wait(), select(), etc. It depends on OS and language, if they are non-busy or busy implemented. But in my experience in all cases a sleep > 0 was always non-busy.
Advantage of non-busy waiting is allowing another threads to run, which includes idle threads. With this your CPU can go into power saving mode, clock down, etc. It can also run another tasks. After the specified time the scheduler tries to go back to your thread. But is is just a try. It is not exact and it may be a little bit longer, than your sleep defines.
I think. This is clear now.
And now the big question: Is this busy, or non-busy waiting:
for(;;) {
if (condition) {
break;
}
sleep(1);
}
The answer is: is is a non-busy waiting. sleep(1) allows the thread to perform a context-switch.
Now the next question: Is the second for() busy, or non-busy waiting:
function wait() {
for(;;) {
if (condition) {
break;
}
}
}
for(;;) {
wait();
if (condition) {
break;
}
sleep(1);
}
It is hard to say. It depends on the real execution time of the wait() function. If it does nothing, then the CPU is almost the entire time in sleep(1). And this would be a non-blocking for-loop. But if wait() is a heavy calculation function without allowing a thread context switch, then this whole for-loop may become a blocking function, even if there is a sleep(1). Think of the worst-case: the wait() function is never returning back to caller, because the condition isn't hit for a long time.
This here is hard to answer, because we don't know the conditions. You can imagine the problem, where you cannot answer the question, because you don't know the conditions, in the following way:
if (unkonwnCondition) {
for(;;) {
if (condition) {
break;
}
}
} else {
for(;;) {
if (condition) {
break;
}
sleep(1);
}
}
As you see, its the same: because you don't know the conditions, you cannot say if the wait is busy or non-busy.
Joe Duffy (author of Concurrent Programming on Windows) writes in this blog article that Thread.Sleep(1) is preferred over Thread.Sleep(0) because it will suspend for same and lower priority threads, not just equal priority threads as for Thread.Sleep(0).
The .NET version of MSDN says that Thread.Sleep(0) is special, it will suspend this thread and allow other waiting threads to execute. But it says nothing about Thread.Sleep(1) (for any .NET version).
So, is Thread.Sleep(1) actually doing anything special?
Background:
I'm refreshing my knowledge of concurrent programming. I wrote some C# code to visibly show that pre/post increments and decrements are non-atomic and therefore not thread-safe.
To avoid needing to create hundreds of threads I place a Thread.Sleep(0) after incrementing a shared variable to force the scheduler to run another thread. This regular swapping of threads makes the non-atomic nature of pre/post increment/decrement more obvious.
Thread.Sleep(0) appears to causes no additional delay, as expected. However if I change this to Thread.Sleep(1), it appears to revert to normal sleep behaviour (eg. I get roughly a minimum of 1ms delay).
This would mean that while Thread.Sleep(1) may be preferred, any code that uses it in a loop would run much slower.
This SO question "Could someone explain this interesting behaviour with Sleep(1)?" is sort of relevant, but it is C++ focused and just repeats the guidance in Joe Duffy's blog article.
Here's my code for anyone interested (copied from LinqPad, so you may need to add a class around it):
int x = 0;
void Main()
{
List<Thread> threadList=new List<Thread>();
Stopwatch sw=new Stopwatch();
for(int i=0; i<20; i++)
{
threadList.Add(new Thread(Go));
threadList[i].Priority=ThreadPriority.Lowest;
}
sw.Start();
foreach (Thread thread in threadList)
{
thread.Start();
}
foreach (Thread thread in threadList)
{
thread.Join();
}
sw.Stop();
Console.WriteLine(sw.ElapsedMilliseconds);
Thread.Sleep(200);
Console.WriteLine(x);
}
void Go()
{
for(int i=0;i<10000;i++)
{
x++;
Thread.Sleep(0);
}
}
You no longer need to use Sleep(1) instead of Sleep(0) because Microsoft changed the implementation of the Windows API Sleep().
From the MSDN documentation for Sleep(), this is what happens now with Sleep(0):
A value of zero causes the thread to relinquish the remainder of its time slice to any other thread that is ready to run. If there are no other threads ready to run, the function returns immediately, and the thread continues execution.
This is what used to happen in Windows XP:
A value of zero causes the thread to relinquish the remainder of its time slice to any other thread of equal priority that is ready to run. If there are no other threads of equal priority ready to run, the function returns immediately, and the thread continues execution. This behavior changed starting with Windows Server 2003.
Note the difference between "any other thread" and "any other thread of equal priority".
The only reason that Joe Duffy suggests using Sleep(1) rather than Sleep(0) is because it is the shortest Sleep() value that will prevent the Sleep() from returning immediately if there are no other threads of equal priority ready to run, when running on Windows XP.
You don't need to worry about this for OS versions after Windows Server 2003 because of the change in behaviour of Sleep().
I draw your attention to this part of Joe's blog:
And even though there's an explicit Sleep in there, issuing it doesn't allow the producer to be scheduled because it's at a lower priority.
In XP, lower priority threads would be starved even if the main thread (of higher priority) did Sleep(0). Post-XP, this will no longer happen because Sleep(0) will allow the lower priority threads to run.
In my HFT trading application I have several places where I receive data from network. In most cases this is just a thread that only receives and process data. Below is part of such processing:
public Reciver(IPAddress mcastGroup, int mcastPort, IPAddress ipSource)
{
thread = new Thread(ReceiveData);
s = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
s.ReceiveBufferSize = ReceiveBufferSize;
var ipPort = new IPEndPoint(LISTEN_INTERFACE/* IPAddress.Any*/, mcastPort);
s.Bind(ipPort);
option = new byte[12];
Buffer.BlockCopy(mcastGroup.GetAddressBytes(), 0, option, 0, 4);
Buffer.BlockCopy(ipSource.GetAddressBytes(), 0, option, 4, 4);
Buffer.BlockCopy(/*IPAddress.Any.GetAddressBytes()*/LISTEN_INTERFACE.GetAddressBytes(), 0, option, 8, 4);
}
public void ReceiveData()
{
byte[] byteIn = new byte[4096];
while (needReceive)
{
if (IsConnected)
{
int count = 0;
try
{
count = s.Receive(byteIn);
}
catch (Exception e6)
{
Console.WriteLine(e6.Message);
Log.Push(LogItemType.Error, e6.Message);
return;
}
if (count > 0)
{
OnNewMessage(new NewMessageEventArgs(byteIn, count));
}
}
}
}
This thread works forever once created. I just wonder if I should configure this thread to run on certain core? As I need lowest latency I want to avoid context switch. As I want to avoid context switch I better to run the same thread on the same processor core, right?
Taking into account that i need lowest latency is that correct that:
It would be better to set "thread afinity" for the most part of the "long-running" threads?
It would be better to set "thread afinity" for the thread from my example above?
I rewriting above code to c++ right now to port to Linux later if this is important however I assume that my question is more about hardware than language or OS.
I think the algorithm that has as little latency as possible would be to pin your threads to one core and set them to realtime priority (or whatever is the highest one).
This will cause the OS to evict any other thread which happens to use that core.
Hopefully the CPU cache will still contain useful data when your thread gets scheduled there. For that reason I like the idea of pinning to a core.
You should probably set your entire process to a high priority class and minimize other activity on your box. Also turn off unused hardware because it might generate interrupts. Fix your NIC's interrupts to a different CPU core (some better NICs can do that).
As I want to avoid context switch I better to run the same thread on the same processor core, right?
No. A context switch will not necessarily be avoided by setting affinity to one CPU. You have no control over context switches, they are in the hands of the OS thread scheduler. They occur when a thread quantum (time slice) has elapsed or when a higher priority thread interrupts your thread.
Latency you talk about, I assume is network or memory latency, is not at all avoided by setting thread affinity. Memory latency can be avoided by making your code cache friendly (ie it can all be in the L1 - L2 caches, for example). Network latency is really just part of any network, and not something I suspect you can do much about.
As Tony The Lion has already answered your question, I would like to address your comment:
"why not setting thread afinity to my code? why thread from my example need to travel between cores?"
Your thread doesn't travel anywhere.
Context switch happens when OS thread scheduler decides to give your thread a slice of time to execute. Then the environment is prepared for your thread, e.g. the CPU registers
are set up to correct values etc. This is called context switch.
So regardless of thread affinity, the same CPU setup work has to be done, whether it is the same CPU/core which was used in previous slice when your thread was running or another one. And at this moments, your computer has more info to do it properly then you do at compile time.
You seem to believe that thread somehow resides on the CPU, but it is not so. What you use is a logical thread and there can be hundreds or even thousands of them. Common CPUs, OTOH, usually have 1 or 2 hardware threads per core, and your logical thread gets mapped to one of these every time it is scheduled, even if OS always picks the same HW thread.
EDIT: it seems that you have already picked the answer you want to hear and I don't like long discussion threads on answers so I will put it here.
you should try and measure it. I believe that you will be dissapointed
running some threads on high priority thread might easily mess up other processes
you are worried about context switch latency, but you have no problems that GC thread will freeze your thread? BTW, on which core will your GC thread run? :)
what if your highest priority thread blocks GC thread? memory leaks? do you know what is priority of that thread so you are sure it would work?
really, why not C or hand optimized assembly if microseconds are important?
as someone suggested, you should use an RTOS if you want to control this aspect of execution
it doesn't seem likely that your data travels through data center just 4-5 times slower than it takes to setup a thread context on one machine, but who knows...
I was reading over some threading basics and on the msdn website I found this snippet of code.
// Put the main thread to sleep for 1 millisecond to
// allow the worker thread to do some work:
Thread.Sleep(1);
Here is a link to the the page: http://msdn.microsoft.com/en-us/library/7a2f3ay4(v=vs.80).aspx.
Why does the main thread have sleep for 1 millisecond? Will the secondary thread not start its tasks if the main thread is continuously running? Or is the example meant for a task that takes 1 millisecond to do? As in if the task generally takes 5 seconds to complete the main thread should sleep for 5000 milliseconds?
If this is solely regarding CPU usage, here is a similar Question about Thread.Sleep.
Any comments would be appreciated.
Thanks.
The 1 in that code is not terribly special; it will always end up sleeping longer than that, as things aren't so precise, and giving up your time slice does not equal any guarantee from the OS when you will get it back.
The purpose of the time parameter in Thread.Sleep() is that your thread will yield for at least that amount of time, roughly.
So that code is just explicitly giving up its time slot. Generally speaking, such a bit of code should not be needed, as the OS will manage your threads for you, preemptively interrupting them to work on other threads.
This kind of code is often used in "threading examples", where the writer wants to force some artificial occurrence to prove some race condition, or the like (that appears to be the case in your example)
As noted in Jon Hanna's answer to this same question, there is a subtle but important difference between Sleep(0) and Sleep(1) (or any other non-zero number), and as ChrisF alludes to, this can be important in some threading situations.
Both of those involve thread priorities; Threads can be given higher/lower priorities, such that lower priority threads will never execute as long as there are higher priority threads that have any work to do. In such a case, Sleep(1) can be required... However...
Low-priority threads are also subject to what other processes are doing on the same system; so while your process might have no higher-priority threads running, if any others do, yours still won't run.
This isn't usually something you ever need to worry about, though; the default priority is the 'normal' priority, and under most circumstances, you should not change it. Raising or lowering it has numerous implications.
Thread.Sleep(0) will give up the rest of a thread's time-slice if a thread of equal priority is ready to schedule.
Thread.Sleep(1) (or any other value, but 1 is the lowest to have this effect) will give up the rest of the thread's time-slice unconditionally. If it wants to make sure that even threads with lower priority have a chance to run (and such a thread could be doing something that is blocking this thread, it has to), then it's the one to go for.
http://www.bluebytesoftware.com/blog/PermaLink,guid,1c013d42-c983-4102-9233-ca54b8f3d1a1.aspx has more on this.
If the main thread doesn't sleep at all then the other threads will not be able to run at all.
Inserting a Sleep of any length allows the other threads some processing time. Using a small value (of 1 millisecond in this case) means that the main thread doesn't appear to lock up. You can use Sleep(0), but as Jon Hanna points out that has a different meaning to Sleep(1) (or indeed any positive value) as it only allows threads of equal priority to run.
If the task takes 5 seconds then the main thread will sleep for a total of 5,000 milliseconds, but spread out over a longer period.
It's only for the sake of the example- they want to make sure that the worker thread has the chance to print "worker thread: working..." at least once before the main thread kills it.
As Andrew implied, this is important in the example especially because if you were running on a single-processor machine, the main thread may not give up the processor, killing the background thread before it has a chance to iterate even once.
Interesting thing I noticed today. Interrupting a thread throws a ThreadInterruptedException. I was trying to catch the exception but could not for some reason. My coworker recommended that I put Thread.Sleep(1) prior to the catch statement and that allowed me to catch the ThreadInterruptedException.
// Start the listener
tcpListener_ = new TcpListener(ipAddress[0], int.Parse(portNumber_));
tcpListener_.Start();
try
{
// Wait for client connection
while (true)
{
// Wait for the new connection from the client
if (tcpListener_.Pending())
{
socket_ = tcpListener_.AcceptSocket();
changeState(InstrumentState.Connected);
readSocket();
}
Thread.Sleep(1);
}
}
catch (ThreadInterruptedException) { }
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Contineo", MessageBoxButtons.OK, MessageBoxIcon.Error);
Console.WriteLine(ex.StackTrace);
}
Some other class...
if (instrumentThread_ != null)
{
instrumentThread_.Interrupt();
instrumentThread_ = null;
}
I set the max thread to 10. Then I added 22000 task using ThreadPool.QueueUserWorkItem.
It is very likely that not all the 22000 task was completed after running the program. Is there a limitation how many task can be queued for avaiable threads?
If you need to wait for all of the tasks to process, you need to handle that yourself. The ThreadPool threads are all background threads, and will not keep the application alive.
This is a relatively clean way to handle this type of situation:
using (var mre = new ManualResetEvent(false))
{
int remainingToProcess = workItems.Count(); // Assuming workItems is a collection of "tasks"
foreach(var item in workItems)
{
// Delegate closure (in C# 4 and earlier) below will
// capture a reference to 'item', resulting in
// the incorrect item sent to ProcessTask each iteration. Use a local copy
// of the 'item' variable instead.
// C# 5/VS2012 will not require the local here.
var localItem = item;
ThreadPool.QueueUserWorkItem(delegate
{
// Replace this with your "work"
ProcessTask(localItem);
// This will (safely) decrement the remaining count, and allow the main thread to continue when we're done
if (Interlocked.Decrement(ref remainingToProcess) == 0)
mre.Set();
});
}
mre.WaitOne();
}
That being said, it's usually better to "group" together your work items if you have thousands of them, and not treat them as separate Work Items for the threadpool. This is some overhead involved in managing the list of items, and since you won't be able to process 22000 at a time, you're better off grouping these into blocks. Having single work items each process 50 or so will probably help your overall throughput quite a bit...
The queue has no practical limit however the pool itself will not exceed 64 wait handles, ie total threads active.
This is an implementation dependent question and the implementation of this function has changed a bit over time. But in .Net 4.0, you're essentially limited by the amount of memory in the system as the tasks are stored in an in memory queue. You can see this by digging through the implementation in reflector.
From the documentation of ThreadPool:
Note: The threads in the managed thread pool are background threads. That is, their IsBackground properties are true. This means that a ThreadPool thread will not keep an application running after all foreground threads have exited.
Is it possible that you're exiting before all tasks have been processed?