When coding an app for Windows (c++, c#), is there a way to lock a certain amount of cpu percentage or cores or threads so they cannot be used by other programs or processes when said app is running? I know you could tinker with CPU priority and affinity in task manager, but I don't know if that prevents other programs to 'steal' cpu power when they need it.
The app is very cpu intensive and dependant on 'real time' operation so when the usage reaches 100%, cpu cannot deal with all the load and errors occur.
So ideally the code would make sure that, if the app is currently working nicely and using 80% of the cpu, no other proccess would ever be allowed to take the remaining 20% (allowing only 10% usage, for example). I guess you could call that 'safety overhead'? I hope I made myself clear.
I am trying to figure out if such a concept exists at all, I couldn't be sure of the keywords or find a thread to start pulling.
If that is not possible in Windows c++ c#, is it a thing in other enviroments?
Thanks!
To my knowledge there is no nice way of doing such things, especially since what you are trying to use is a custom scheduler, but as these are usually hard-coded into the operating system I don't see much hope for you.
If real-time functionality is your main concern, I would either recommend using a real-time-operating-system or optimizing your software, so it doesn't need 80% of your CPU, if possible. You could also just upgrade your CPU (if money is not really a concern), so it can handle your software.
Under other operating systems there are ways to encourage the scheduler to like your software more (look up "nice value") but that's similar to changing priority in Task Manager (on steroids however).
I also remember from my operating systems lecture that there are operating systems that allow the scheduler to be modified, this might be further than you wanted to go, but that is a possibility if you become desperate enough.
And as my last idea: If you have something really computationally intense, it is often doing something over and over again. Assuming that these steps are (partially) independent of each other, it could be a massive performance gain to move work from the CPU to the GPU, from my experience (n=1) saving 50% is possible. From C++ with an Nvidia GPU you want to look up CUDA, for everything else you likely want OpenCL.
Related
This a VERY open question.
Basically, I have a computing application that launches test combinations for N Scenarios.
Each test is conducted in a single dedicated thread, and involves reading large binary data, processing it, and dropping results to DB.
If the number of threads is too large, the app gets rogue and eats out all available memory and hangs out..
What is the most efficient way to exploit all CPU+RAM capabilities (High Performance computing i.e 12Cores/16GB RAM) without putting the system down to its knees (which happens if "too many" simultaneous threads are launched, "too many" being a relative notion of course)
I have to specify that I have a workers buffer queue with N workers, every time one finishes and dies a new one is launched via a Queue. This works pretty fine as of now. But I would like to avoid "manually" and "empirically" setting the number of simultaneous threads and have an intelligent scalable system that drops as many threads at a time that the system can properly handle, and stop at a "reasonable" memory usage (the target server is dedicated to the app so there is no problem regarding other applications except the system)
PS : I know that .Net 3.5 comes with Thread Pools and .Net 4 has interesting TPL capabilites, that I am still considering right now (I never went very deep into this so far).
PS 2 : After reading this post I was a bit puzzled by the "don't do this" answers. Though I think such request is fair for a memory-demanding computing program.
EDIT
After reading this post I will to try to use WMI features
All built-in threading capabilities in .NET do not support adjusting according to memory usage. You need to build this yourself.
You can either predict memory usage or react to low memory conditions. Alternatives:
Look at the amount of free memory on the system before launching a new task. If it is below 500mb, wait until enough has been freed.
Launch tasks as they come and throttle as soon as some of them start to fail because of OOM. Restart them later. This alternative sucks big time because your process will do garbage collections like crazy to avoid the OOMs.
I recommend (1).
You can either look at free system memory or your own processes memory usage. In order to get the memory usage I recommend looking at private bytes using the Process class.
If you set aside 1GB of buffer on your 16GB system you run at 94% efficiency and are pretty safe.
I have a small list of rather large files that I want to process, which got me thinking...
In C#, I was thinking of using Parallel.ForEach of TPL to take advantage of modern multi-core CPUs, but my question is more of a hypothetical character;
Does the use of multi-threading in practicality mean that it would take longer time to load the files in parallel (using as many CPU-cores as possible), as opposed to loading each file sequentially (but with probably less CPU-utilization)?
Or to put it in another way (:
What is the point of multi-threading? More tasks in parallel but at a slower rate, as opposed to focusing all computing resources on one task at a time?
In order to not increase latency, parallel computational programs typically only create one thread per core. Applications which aren't purely computational tend to add more threads so that the number of runnable threads is the number of cores (the others are in I/O wait, and not competing for CPU time).
Now, parallelism on disk-I/O bound programs may well cause performance to decrease, if the disk has a non-negligible seek time then much more time will be wasted performing seeks and less time actually reading. This is called "churning" or "thrashing". Elevator sorting helps somewhat, true random access (such as solid state memories) helps more.
Parallelism does almost always increase the total raw work done, but this is only important if battery life is of foremost importance (and by the time you account for power used by other components, such as the screen backlight, completing quicker is often still more efficient overall).
You asked multiple questions, so I've broken up my response into multiple answers:
Multithreading may have no effect on loading speed, depending on what your bottleneck during loading is. If you're loading a lot of data off disk or a database, I/O may be your limiting factor. On the other hand if 'loading' involves doing a lot of CPU work with some data, you may get a speed up from using multithreading.
Generally speaking you can't focus "all computing resources on one task." Some multicore processors have the ability to overclock a single core in exchange for disabling other cores, but this speed boost is not equal to the potential performance benefit you would get from fully utilizing all of the cores using multithreading/multiprocessing. In other words it's asymmetrical -- if you have a 4 core 1Ghz CPU, it won't be able to overclock a single core all the way to 4ghz in exchange for disabling the others. In fact, that's the reason the industry is going multicore in the first place -- at least for now we've hit limits on how fast we can make a single CPU run, so instead we've gone the route of adding more CPUs.
There are 2 reasons for multithreading. The first is that you want to tasks to run at the same time simply because it's desirable for both to be able to happen simultaneously -- e.g. you want your GUI to continue to respond to clicks or keyboard presses while it's doing other work (event loops are another way to accomplish this though). The second is to utilize multiple cores to get a performance boost.
For loading files from disk, this is likely to make things much slower. What happens is the operating system tries to lay out files on disk such that you should only need to do an expensive disk seek once for each file. If you have a lot of threads reading a lot of files, you're gonna have contention over which thread has access to the disk, and you'll have to seek back to the right place in the file every time the next thread gets a turn.
What you can do is use exactly two threads. Set one to load all of the files in the background, and let the other remain available for other tasks, like handling user input. In C# winforms, you can do this easily with a BackgroundWorker control.
Multi-threading is useful for highly parallelizable tasks. CPU intensive tasks are perfect. Your CPU has many cores, many threads can use many cores. They'll use more CPU time, but in the end they'll use less "user" time. If your app is I/O bounded, then multithreading isn't always the solution (but it COULD help)
It might be helpful to first understand the difference between Multithreading and Parallelism, as more often than not I see them being used rather interchangeably. Joseph Albahari has written a quite interesting guide about the subject: Threading in C# - Part 5 - Parallelism
As with all great programming endeavors, it depends. By and large, you'll be requesting files from one physical store, or one physical controller which will serialize the requests anyhow (or worse, cause a LOT of head back-and-forth on a classical hard drive) and slow down the already slow I/O.
OTOH, if the controllers and the medium are separate, multiple cores loading data from them should be improved over a sequential method.
System.Environment.ProcessorCount shows me N Processors (N in my case = 8), which I want to make use of. Now the problem is, that the windows resourcemanager sais, that 4 of my CPU's are 'parked', and the 8 Threads i start just seperate up to the 4 unblocked CPUs.
Now is there a way to use the parked CPU's, too?
When Windows "parks" a CPU core, it means that there is not enough work for that core to do so it puts that core in a low-power state. In order to "unpark" the CPU, you just have to create enough work.
If you are starting 8 threads and Windows isn't unparking the CPUs, the threads probably are doing I/O, blocking, or completing too quickly. If you post what your threads are doing, maybe somebody can explain why they're not running on the parked cores.
Usually, you should be able to do it this way:
Process.GetCurrentProcess().ProcessorAffinity = (IntPtr)0x00FF;
see documentation for it here:
http://msdn.microsoft.com/en-us/library/system.diagnostics.process.processoraffinity.aspx
but it also says that, by default your process is assigned to all cores.
On the other hand, you could try ProcessThread.ProcessorAffinity and try to set it manually (if you want to force each thread to use another core).
Win7/2K8R2 won't unpark cores until the other ones are saturated or near saturation.
The whole point of parking cores is to consolidate work. It's more power efficient to use 4 cores at 80% than 8 cores at 40%. Also, the performance difference should be almost non-existent.
Also, depending on how much data is shared, consolidating the work will actually be faster because there would be less sync overhead because there are fewer hardware threads involved. Recent data changes from one thread will be more likely in cache.
So, common worst case is about same performance and less power used and common best case is better performance and less power used.
The parking is not controlled by the CPU affinity setting of your process, it is done automatically by the Windows CPU Scheduler. Adjustments to your CPU affinity can perhaps force utilization of certain cores, but then Windows will just park different cores. The parking is turned on or off dynamically, very quickly, in accordance with system load. It is actually surprisingly aggressive by default (maybe too much so on some platforms). You can watch it in the Resource Monitor, as you saw.
Setting your own CPU affinity is something you should do with extreme caution. You must consider HyperThreaded cores, or in the case of AMD Bulldozer, paired cores that share computational units (their HyperThreading without being HyperThreading ;p). You don't want to end up 'stuck' on a Hyper-Threaded core that offers a mere fraction of the performance of a real core. The CPU scheduler is aware of such things, so usually the affinity is best left to it -- unless you know what you're doing, and have checked that system's CPU.
However, you can enable/disable or tweak CPU Parking very easily, without rebooting. I wrote a HOW-TO, complete with a simple GUI, here: How to Enable/Disable or Tweak CPU Parking Without a Reboot, and without Registry Edits
It also includes more information about CPU Parking, and how to tweak it using PowerCfg.exe. You can actually make the option show up in the standard Advanced Power Profile settings in Windows, but it takes some tweaking I won't get into here.
Basically, I'm wondering if threading is useful or necessary, or possibly more specifically the uses and situations in which you would use it. I don't know much about threading, and have never used it (I primarily use C#) and have wondered if there are any gains to performance or stability if you use them. If anyone would be so kind to explain, I would be grateful.
In the world of desktop applications (my domain), threading is a vital construct in creating responsive user interfaces. Whenever a time-or-computationally-intensive operation needs to run, it's almost essential to run that operation in a separate thread. Otherwise, the user interface locks up and, in some cases, Windows will decide that the whole application has become unresponsive.
Threading is also a vital tool in animation, audio and communications. Basically, any situation in which you find yourself needing to do several things at once lends itself to the use of threads.
there is definitely no gains to stability :). I would suggest you get a basic understanding of threading but don't jump to use it in any real production application until you have a real need. you have C# so not sure if you are building websites or winforms.
Usually the firsty threading use case for winforms is when a user click a button and you want to run some expensive operation (database or webservice call) but you dont want the screen to freeze up . .
a good tutorial to deal with that situation is to look at the backgroundworker class in c# as this will give you a first flavor into this space and then you can go from there
There was a time when our applications would speed up when we deploy them on new CPU. And that speed up was by large extent because CPU speed (clock) was incremented by large factors.
But several years ago, CPU manufacturers stopped increasing CPU clocks because of physical limits (e.g. heat dissipation). And instead they started adding additional cores to CPUs.
Now, if your application runs only on one thread it cannot take advantage of complete CPU (e.g. of 4 cores it uses only 1).
So today to fully utilize CPU we must take effort and divide task on multiple treads.
For ASP.NET this is already done for us by ASP.NET architecture and IIS.
Look here The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software
Here is a simple example of how threading can improve performance. You have a n numbers that all needed to be added together. In a single threaded application, it will take a n time units to add all of the numbers together for the final sum. However, if you broke your numbers into 2 groups, you could have the same operation running side by side with, each with a group of n/2 numbers. Each would take n/2 time units to find their respective sums, and then an additional unit to find the full sum. By creating two threads, you have effectively cut the compute time in half.
Technically on a single core processor, there is no such thing as multi-threading, just the illusion that multiple tasks are happening in parallel since each task gets a small amount of time.
However, that being said, threading is very useful if you have to do some work that takes a long time but you want your application to be responsive (i.e. be able to do other things) while you wait for that task to finish. A good example is GUI applications.
On multi-core / multi-processor systems, you can have one process doing many things at once so the performance gain there is obvious :)
I have to build a simulator with C#. This simulator should be able to run a second thread with configureable CPU speed and limited RAM size, e.g. 144MHz and 50 MB.
Of course I know that a simulator can never be as accurate as the real hardware. But I try to get almost similar performance.
At the moment I'm thinking about creating a thread which I will stop/sleep from time to time. Depending on the desired CPU speed the simulator should adjust the sleep time of this thread and therefore simulate different cpu frequency. To measure the achieved speed I though about using PerformanceCounters. But with this approach I have the problem that I don't know how to limit the RAM size the thread could use.
Do you have any ideas how to realize such a simulator?
Thanks in advance!!
Limit memory is easy with the virtual machines like vmware. You can change cpu speed with some overclocking tools. For example http://cpu.rightmark.org/products/rmclock.shtml
Good luck!
CPU speed limiting? You should check this, perhaps it will useful (to some degree at least).
CPU Emulation and locking to a specific clock speed
If you are concerned with simulating an operating system environment then one answer would be to use a virtual machines environment where you can control memory and CPU parameters, etc.
The threading pause\stop may help you to simulate CPU frequency, but this is going to be terribly inaccurate as when you pause the thread it will be de-scheduled, then it's up to the operating system to re-schedule it at some "random" point in time i.e. a point which you have no control over.
As for limiting the memory, starting a new process that will host your code is an option, and then limiting the memory of that process, e.g.:
http://www.codeproject.com/KB/threads/Setting_Max_Memory_Limit.aspx
This will not really simulate overall OS memory limitations though.
A thread to sleep down the software execution of your guest opcodes ?
I think it works but a little weird, like fast-forward, pause, ff, pause, etc...
If you just want to speed down a process, try this: use the cpu single step features, and "debug" the process. You have to wrote a custom handler for the cpu single stepping trap. Your handler job is only a big loop of NOPs.
You have a fine delay between each instruction.