im create a windows application C# like a download Manager
when run this application i found the CPU is 99% and i write a threading
application how i can start to solve this Problem
thank you
Look for any tight loops you've got in your code - it's almost certainly due to one of those. Something like this:
while (!finished)
{
progressBar.Value = DownloadProgress;
}
Without seeing your code though, it's hard to guess any more accurately than that.
I suggest you start by profiling your application to identify hot spots and then rework through the code to eliminate the same. Profile your application - number of active threads - CPU consumed by the different threads, profile functions to catch any CPU heavy function.
Yes, profiling is a good idea. Use e.g. Red Gate ANTS Performance Profiler which is free to test for 14 days.
This is my favorite place for threading, I don't think you can find a simpler article on threading strategies.
http://www.albahari.com/threading/
I would try with Jons answer and then implement Muek answer.
You are having tight loops somewhere in your code. Despite low priority (if you used it) - if you have a loop that doesn't release the control back to the kernel thread scheduler it will consume all of the CPU until control is TAKEN from it. For example you can have:
while (!_shouldStop)
{
DoProcessing();
}
This really IS bad, because it will certainly use all of your CPU.
To solve this, easiest way is to use either Sleep(100) or Sleep(0) inside your loop, something like:
while (!_shouldStop)
{
DoProcessing();
Thread.Sleep(0);
}
There are also better (and somehow more complicated ways) to do this - events for example - but for start, your app will behave much better with that here.
Related
The more I use Parallel.ForEach and PLINQ in my code, the more faces and code review push backs I am getting. So I wonder is there any reason for me NOT to use PLINQ, at extreme, on each LINQ statement? Can the runtime not be smart enough to start spawning so many threads (or consuming so many threads from the thread pool) that the app performance would actually degrade instead of improve? The same question applies to Parallel library.
I do understand implications related to thread-safety and overhead of using multi-threading. I also realize not everything is good for parallelizing. All I am wondering about if I should stop defending my approaches and just give up on these two fine things because my peers think I'd better do thread control myself instead of relying on .NET facilities?
UPDATE: please assume the hardware is sufficiently good to satisfy prerequisites for use of multithreading.
It all comes down to two things:
Is the extra work required to partition the collection and synchronize the threads greater than the performance gain compared to a regular foreach?
Are all the threads going to use a shared resource that will become a bottle neck?
An example of the second case is doing a Parallel.ForEach over the results of a Linq to Sql statement. In that case, if your results are coming from the DB very slowly, each thread may spend more time waiting for data to process than actually doing something.
See: http://msdn.microsoft.com/en-us/library/dd997392.aspx
To set the number of worker threads you can use .WithDegreeOfParallelism(N)
eg
var query = from item in source.AsParallel().WithDegreeOfParallelism(2)
where Compute(item) > 42
select item;
See http://msdn.microsoft.com/en-us/library/dd997425.aspx
When dig into performance questions this deep, I think the best thing to do is... measure, measure and measure. Even if somebody answered that PLINK is great and will boost the performance of your application, would you trust that without verifing it with profiling? Although general answers may exists you cannot spare the effort to measure the performance in your exact case. The overall performance depends on so many things and it can be that PLINK helps in one case but not in the other.My personal experiences with PLINK is that after swicthing every LINQ query into PLINK the response times are way better when the load is small, and there is no difference when the load is around its maximum. But I can imagine a case where PLINK hurts the overall performance under a huge load. Have to check it for your own particular case.Well... and if you want to convince other people that you are walking the right path, what else would be better than measurement results?
I am interning for a company this summer, and I got passed down this program which is a total piece. It does very computationally intensive operations throughout most of its duration. It takes about 5 minutes to complete a run on a small job, and the guy I work with said that the larger jobs have taken up to 4 days to run. My job is to find a way to make it go faster. My idea was that I could split the input in half and pass the halves to two new threads or processes, I was wondering if I could get some feedback on how effective that might be and whether threads or processes are the way to go.
Any inputs would be welcomed.
Hunter
I'd take a strong look at TPL that was introduced in .net4 :) PLINQ might be especially useful for easy speedups.
Genereally speaking, splitting into diffrent processes(exefiles) is inadvicable for perfomance since starting processes is expensive. It does have other merits such as isolation(if part of a program crashes) though, but i dont think they are applicable for your problem.
If the jobs are splittable, then going multithreaded/multiprocessed will bring better speed. That is assuming, of course, that the computer they run on actually has multiple cores/cpus.
Threads or processes doesn't really matter regarding speed (if the threads don't share data). The only reason to use processes that I know of is when a job is likely to crash an entire process, which is not likely in .NET.
Use threads if theres lots of memory sharing in your code but if you think you'd like to scale the program to run across multiple computers (when required cores > 16) then develop it using processes with a client/server model.
Best way when optimising code, always, is to Profile it to find out where the Logjam's are IMO.
Sometimes you can find non obvious huge speed increases with little effort.
Eqatec, and SlimTune are two free C# profilers which may be worth trying out.
(Of course the other comments about which parallelization architecture to use are spot on - it's just I prefer analysis first....
Have a look at the Task Parallel Library -- this sounds like a prime candidate problem for using it.
As for the threads vs processes dilemma: threads are fine unless there is a specific reason to use processes (e.g. if you were using buggy code that you couldn't fix, and you did not want a bad crash in that code to bring down your whole process).
Well if the problem has a parallel solution then this is the right way to (ideally) significantly (but not always) increase performance.
However, you don't control making additional processes except for running an app that launches multiple mini apps ... which is not going to help you with this problem.
You are going to need to utilize multiple threads. There is a pretty cool library added to .NET for parallel programming you should take a look at. I believe its namespace is System.Threading.Tasks or System.Threading with the Parallel class.
Edit: I would definitely suggest though, that you think about whether or not a linear solution may fit better. Sometimes parallel solutions would taken even longer. It all depends on the problem in question.
If you need to communicate/pass data, go with threads (and if you can go .Net 4, use the Task Parallel Library as others have suggested). If you don't need to pass info that much, I suggest processes (scales a bit better on multiple cores, you get the ability to do multiple computers in a client/server setup [server passes info to clients and gets a response, but other than that not much info passing], etc.).
Personally, I would invest my effort into profiling the application first. You can gain a much better awareness of where the problem spots are before attempting a fix. You can parallelize this problem all day long, but it will only give you a linear improvement in speed (assuming that it can be parallelized at all). But, if you can figure out how to transform the solution into something that only takes O(n) operations instead of O(n^2), for example, then you have hit the jackpot. I guess what I am saying is that you should not necessarily focus on parallelization.
You might find spots that are looping through collections to find specific items. Instead you can transform these loops into hash table lookups. You might find spots that do frequent sorting. Instead you could convert those frequent sorting operations into a single binary search tree (SortedDictionary) which maintains a sorted collection efficiently through the many add/remove operations. And maybe you will find spots that repeatedly make the same calculations. You can cache the results of already made calculations and look them up later if necessary.
Good afternoon,
As I said earlier in another post, I have to calculate some 8,000,000 shortest- time/path distances between some points in the map, the coordinates of which are know. The problem is that, while straight-line distances were easy (and quick) to calculate, someone told me that a single-threaded application can have problems calculating this number of distances using MapPoint. The question is that I know nothing about multi-threading... I am currently working on a i7 - 720QM environment, so I would like to use all the 4 cores to make these calculations... Is there any easy way of doing this in C# or C++?
Thank you very much.
If you are totally new to the Multithreading than my advice start with BackGroundWorker component as a starting point and gradually switch to more garnular threading concepts.
and if you are using ..net 4.0 than Task Parallel Library gives you easy way to start with.
See Links Below
TPL
BackGroundWorker
That might have been me who said it would take a long time. MapPoint's COM API is single threaded. The way to get it to compute multiple routes in parallel is to start multiple MapPoint's, each on its own thread.
So for your quad core, you will start 2-3 threads. Each thread starts its own MapPoint, and then uses it for routing. You will NOT have one MapPoint per core. As well OS overhead and your I/O overhead, if you watch a single MapPoint compute a route, you will find that later versions are partially internally multi-threaded and can take about 1.5 cores if they are available.
There are also a lot of gotchas to watch out for. MapPoint's own garbage collection is not optimized for batch route calculation. The easiest workaround for this is to simply restart each MapPoint application at periodic intervals (at least once a day but probably more frequently).
Also, some operations (File Open seems to be the main one) cannot be called by multiple MapPoints at once. Probably because they are trying to open the same file, but I have not investigated further. You will need to implement your own locking mechanism to avoid this.
Saurabh's advice for .NET 4 sounds good: I have yet to use .NET 4's multi-threading in anger - my MapPoint/.NET threading experience is with .NET 2.
I don't know what your app is, but did you know that I sell a product that uses multi-processor MapPoint for batch route distance/time calculation... :-)
I need an accurate timer, and DateTime.Now seems not accurate enough. From the descriptions I read, System.Diagnostics.Stopwatch seems to be exactly what I want.
But I have a phobia. I'm nervous about using anything from System.Diagnostics in actual production code. (I use it extensively for debugging with Asserts and PrintLns etc, but never yet for production stuff.) I'm not merely trying to use a timer to benchmark my functions - my app needs an actual timer. I've read on another forum that System.Diagnostics.StopWatch is only for benchmarking, and shouldn't be used in retail code, though there was no reason given. Is this correct, or am I (and whoever posted that advice) being too closed minded about System.Diagnostics? ie, is it ok to use System.Diagnostics.Stopwatch in production code?
Thanks
Adrian
Under the hood, pretty much all Stopwatch does is wrap QueryPerformanceCounter. As I understand it, Stopwatch is there to provide access to the high-resolution timer - if you need this resolution in production code I don't see anything wrong with using it.
Yes, System.Diagnostics does sound like it is for debugging only, but don't let the name deceive you. The System.Diagnostics namespace may seem a bit scary sounding for use in production code at first (it did for me), but there are plenty of useful things in that namespace.
Some things, such as the Process class, are useful for interacting with the system. With Process.Start you can start other applications, launch a website for the user, open a file or folder, etc.
Others things, such as the Trace class, can help you track down bugs in production code. Granted, you will not always use them in production code, but they are very useful for logging and tracking down that elusive bug on a remote machine.
Don't worry about the name.
You say you've read on another forum not to use classes from System.Diagnostics in production. But the only source you should worry about is Microsoft, who created the code. They say that the StopWatch class:
Provides a set of methods and properties that you can use to accurately measure elapsed time.
They don't say, "except in production".
Afaik StopWatch is a shell over QueryPerformanceCounter functionality. This function is the basis of a lot of performance counters related measurements. QPF is very fast to call and perfectly safe. IF you feel paranoid about the Diagnostics namespace, pInvoke the QPF directly.
The stopwatch is basically a neat wrapper around the native QueryPerformanceCounter and QueryPerformanceFrequency methods. If you don't feel comfortable using the System.Diagnostic namespace, you can access these directly.
Using the Performance Counter is very common, there is nothing wrong with that. AFAIK, there is no higher timer precision available. Note the QPF might lead to problems with multi processor machines, but the MSDN Article linked before gives some additional information on that. It is advisable to make sure System.Diagnostics.Stopwatch does that in the background or to call the SetThreadAffinity manually - otherwise your timer might jump back in time!
Note that for very high precision measurements, there are some subtleties that need to be taken into account. If you need this much precision, these might be of some concern.
There are several different timer classes in the .NET base class library - which one is best suited to your needs can only be determined by you.
Here is a good article from MSDN magazine on the subject (Comparing the Timer Classes in the .NET Framework Class Library).
Depending on what you're using the timer for, you may have other issues to consider. Windows does not provide guarantees on timing of execution, so you shouldn't rely on it for any real-time processing (there are real-time extensions you can get for Windows that provide hard real-time scheduling). I also suspect you could lose precision as a result of context switching after you capture the time interval and before you do something with it that depends on its precision. In principle, this could be an arbitrarily long period of time; in practice it should be on the order of milliseconds. It really depends on how mission-critical this timing is.
I'm writing a plug-in for another program in C#.NET, and am having performance issues where commands take a lot longer then I would. The plug-in reacts to events in the host program, and also depends on utility methods of the the host program SDK. My plug-in has a lot of recursive functions because I'm doing a lot of reading and writing to a tree structure. Plus I have a lot of event subscriptions between my plugin and the host application, as well as event subscriptions between classes in my plug-in.
How can I figure out what is taking so long for a task to complete? I can't use regular breakpoint style debugging, because it's not that it doesn't work it's just that it's too slow. I have setup a static "LogWriter" class that I can reference from all my classes that will allow me to write out timestamped lines to a log file from my code. Is there another way? Does visual studio keep some kind of timestamped log that I could use instead? Is there someway to view the call stack after the application has closed?
You need to use profiler. Here link to good one: ANTS Performance Profiler.
Update: You can also write messages in control points using Debug.Write. Then you need to load DebugView application that displays all your debug string with precise time stamp. It is freeware and very good for quick debugging and profiling.
My Profiler List includes ANTS, dotTrace, and AQtime.
However, looking more closely at your question, it seems to me that you should do some unit testing at the same time you're doing profiling. Maybe start by doing a quick overall performance scan, just to see which areas need most attention. Then start writing some unit tests for those areas. You can then run the profiler while running those unit tests, so that you'll get consistent results.
In my experience, the best method is also the simplest. Get it running, and while it is being slow, hit the "pause" button in the IDE. Then make a record of the call stack. Repeat this several times. (Here's a more detailed example and explanation.)
What you are looking for is any statement that appears on more than one stack sample that isn't strictly necessary. The more samples it appears on, the more time it takes. The way to tell if the statement is necessary is to look up the stack, because that tells you why it is being done.
Anything that causes a significant amount of time to be consumed will be revealed by this method, and recursion does not bother it.
People seem to tackle problems like this in one of two ways:
Try to get good measurements before doing anything.
Just find something big that you can get rid of, rip it out, and repeat.
I prefer the latter, because it's fast, and because you don't have to know precisely how big a tumor is to know it's big enough to remove. What you do need to know is exactly where it is, and that's what this method tells you.
Sounds like you want a code 'profiler'. http://en.wikipedia.org/wiki/Code_profiler#Use_of_profilers
I'm unfamiliar with which profilers are the best for C#, but I came across this link after a quick google which has a list of free open-source offerings. I'm sure someone else will know which ones are worth considering :)
http://csharp-source.net/open-source/profilers
Despite the title of this topic I must argue that the "best" way is subjective, we can only suggest possible solutions.
I have had experience using Redgate ANTS Performance Profiler which will show you where the bottlenecks are in your application. It's definitely worth checking out.
Visual Studio Team System has a profiler baked in, its far from perfect, but for simple applications you can kind of get it to work.
Recently I have had the most success with EQATECs free profiler, or rolling my own tiny profiling class where needed.
Also, there have been quite a few questions about profilers in that past see: http://www.google.com.au/search?hl=en&q=site:stackoverflow.com+.net+profiler&btnG=Google+Search&meta=&aq=f&oq=
Don't ever forget Rico Mariani's advice on how to carry out a good perf investigation.
You can also use performance counter for asp.net applications.