I have a method in c# code behind..which needs to be executed 10000+ lines from Assemblies as well as in Child group Methods. My Question is How to optimize it? It is taking more than 40 Seconds Load to 500 Rows in my page my own gridview which is designed by myself.
Profile your code. That will help you identify where its slow. From reading your post , optimizing might take you a long time since you have alot of code and data.
Virtualize as much as you can. Instead of loading 500 rows, can you try loading 50 rows first, show your UI then load the remainder 450 rows asynchronously ? This doesnt speed up your application, but at least it seems working much quicker than waiting 40 seconds.
This method is very simple, but it can pinpoint the activities that would profit the most by optimizing.
If there is a way to speed up your program, it is taking some fraction of time, like 60%.
If you randomly interrupt it, under a debugger, you have a 60% chance of catching it in the act.
If you examine the stack, and maybe some of the state variables, it will tell you with great precision just what the problem is.
If you do it 10 times, you will see the problem on about 6 samples.
Related
I'm currently getting into PerfView for performance analysis for my (C#) apps.
But typically those apps use a lot of database calls.
So I asked myself questions like:
- How much time is spent in Repositories?
- (How much time is spent waiting for SQL Queries to return?) -> I don't know if this is even possible to discover with PerfView
But from my traces I get barely any useful results. In the "Any Stacks" View it tells me (when I use grouping on my Repository) that 1,5 seconds are spent in my Repsoitory (the whole call is about 45 seconds). And i know this is not really true, because the repositories calls the database A LOT.
Is it just that CPU metric is not captured when waiting for SQL Queries to complete because CPU has nothing to do in this period of time and therefore my times are just including data transformation times etc in the repository?
Thanks for any help!
EDIT:
What i missed is turning on thread times option to get times of blocked code (which is what's happening during database calls i suppose). I got all the stacks now, just have filter out the uninteresting things. But i don't seem to get anywhere.
What's especially interesting for me when using "Thread Time" is the BLOCKED_TIME. But with it the times are off i think. When you look at the screenshot, it tells me that CPU_TIME is 28,384. Which is milliseconds (afaik), but BLOCKED_TIME is 2,314,732, which can't be milliseconds. So percentage for CPU_TIME is very low with 1.2% but 28 out of 70 seconds are still a lot. So the Inclusive Percentage time is comparing apples and oranges here. Can somebody explain?
So, i succeeded.
What I missed (and Vance Morrison was explaining it in his video tutorial actually) is: When doing a wall clock time analysis with perfview, you get accumulated time from all the threads that have been "waiting around" in what is called "BLOCKED_TIME". Which means for a 70 seconds time, alone the finalizer thread adds 70 seconds to this "BLOCKED_TIME" because it was sitting there not doing anything (at least almost anything in my case).
So when doing wall clock time analysis it is important to filter out what you're interested in. For example search for the thread that was taking the most CPU-time and just include this one in your analysis and go further down the stack to find pieces of your code that are expensive (and also might lead to DB or Service Calls). As soon as you a analysis from the point of view of a method you are really getting the times that were spent in this method and the accumulated "BLOCK_TIME" is out of the picture.
What I found most useful is searching for methods in my own code that "seemed time consuming", i switched to the callers view for this method. Which shed some light from where it's called and in the callees view what is responsible for the consuming time further down the stack (a DB call in a repository or service calls for getting some data).
Little bit hard to explain, but as soon as i had understand really the basics of wall clock time analysis, it all made sense at some point.
I recommend this video tutorial: http://channel9.msdn.com/Series/PerfView-Tutorial
Again, great and very powerful tool!
I'm new to profiling. I'm trying to profile a C# application which connects to an SQLite database and retrieve data. The database contains 146856400 rows and the select query retrieves 428800 rows after execution.
On the first execution the main thread takes 246686 ms
On second execution of the same code the main thread takes only 4296 ms
After restarting the system
On the first execution the main thread takes 244533 ms
On the second execution of the same code the main thread takes only 4053 ms
Questions:
1) Why is there a big difference between the first execution timing and the second execution timing
2) After restarting the system why I'm not getting the same results.
Pls help
You experience the difference between cold and warm execution of your query. Cold means the first time and warm all subsequent invocations of your db query.
The first time everything is "cold"
OS file system cache is empty.
SQLLite cache is empty.
ORM dynamic query compilation is not done and cached yet.
ORM Mapper cache is empty.
Garbage Collector needs to tune your working set
....
When you execute your query a second time all these first time initializations (caching) are done and you are measuring the effects of different cache levels as long as there is enough memory available to cache a substantial amount of your requested data.
A performance difference between 4 minutes and 4s is impressive. Both numbers are valid. Measuring something is easy. Telling someone else what exactly you have measured and how the performance can be improved by changing this or that is much harder.
The performance game goes often like this:
Customer: It is slow
Dev: I cannot repro your issue.
Customer: Here is my scenario ....
Dev: I still cannot repro it. Can you give me data set you use and the exact steps you did perform?
Customer: Sure. Here is the data and the test steps.
Dev: Ahh I see. I can make it 10 times faster.
Customer: That is great. Can I have the fix?
Dev: Sure here it is.
Customer: **Very Angry** It has become faster yes. But I cannot read my old data!
Dev: Ups. We need to migrate all your old data to the new much more efficient format.
We need to develop a a conversion tool which will take 3 weeks and your site will
have 3 days downtime while the conversion tool is running.
Or
We keep the old inefficient data format. But then we can make it only 9 times faster.
Customer: I want to access my data faster without data conversion!
Dev: Here is the fix which is 10% slower with no schema changes.
Customer: Finally. The fix does not break anything but it has not become faster?
Dev: I have measured your use case. It is only slow for the first time.
All later data retrievals are 9 times faster than before.
Customer: Did I mention that in my use case I read always different data?
Dev: No you did not.
Customer: Fix it!
Dev: That is not really possible without a major rewrite of large portions of our software.
Customer: The data I want to access is stored in a list. I want to process it sequentially.
Dev: In that case we can preload the data in the background while you are working the current data set. You will only experience a delay for the first data set on each working day.
Customer: Can I have the fix?
Dev: Sure here it is.
Customer: Perfect. It works!
Performance is hard to grasp since most of the time you deal with perceived performance which is subjective. Bringing it down to quantitative measurements is a good start but you need to tune your metrics to reflect actual customer use cases or you will likely optimize at the wrong places like above. A complete understanding of customer requirements and use cases is a must. On the other hand you need to understand your complete system (profile it as hell) to be able to tell the difference between cold and warm query execution and where you can tune the whole thing. These caches become useless if you query for different data all of the time (not likely). Perhaps you need a different index to speed up queries or you buy a SSD or you keep all of the data in memory and do all subsequent queries in memory....
I want to compute the shortest path by computing the distance between school and student locations, here is my code, It works only for the first row in the database but I do not Know why! and there is no error while running but takes a lot of time!It seems like there is an infinity loop !!but I do not Know where is the error??!!
the value of counter is romper of students
Oh man, beginners ;) Funny.
but takes a lot of time!It seems like there is an infinity loop
You do realize that common sense says that an infinite loop does not take a LOT of time, but INFINITE time? SO if that thing comes back after 15 minutes it is likely atotally crappy algorithm, but it is NOT an infinite loop. Infinite is not slow, it is STOP HERE NEVER EXIT.
Given that you run 2 loops on data we do not know and that you run statements that make zero sense, I Would suggest you get a tutorial on how to use the debugger and start - debugging. That is developer baseline info and if you do not learn it now, rather give up programming - there is no chance to write more complex code without regularly using a debuger.
For pure performane there are profilers. Again, unless you hate programming and do not want to make anyhting more than small samples - you WILL need to use a profiler more or less regularly. Sp ypi cam start dpomg so now.
I think you will find out that you do a LOT of SQL stuff. THat is a select / update every time - and as such it is very inefficient (unless it happens ot often, but hey, without the data, how should I know?). I doubt you have so many busses (like: more than a couple of millions), it may make more sense to pull the data into memory once and then not hit the database until your loop is ready. Any select will take some time, and if you do that a lot - welcome to performance hell.
I have a function that runs for about 0.7 seconds on my not-so-new development machine. (it runs for about 3 seconds on another machine I tested)
I want to show the user some pre-message about half a second before the above function is done.
I don't want to show the message too long before the function is done as it will be annoying to just look at it and wait. On the other hand, I would rather not wait until the function is done because the whole thing starts from a user action and I don't want to waste time - it's better if I can show that message while the other function is doing its job.
I've already added a loop with a short Thread.sleep() to let the pre-message hang if the function was "too fast" but I'm afraid that usually won't be the case... And so, I want to see if I could roughly estimate the execution time based on current machine specifications and even by the current CPU usage and do that before running the function. Also, since we are talking about seconds and milliseconds, if getting this information will take more than a few milliseconds then it's not worth it. In this case, I might calculate it only once when the application is loaded.
Does anybody have an idea how to do that?
Estimating time is pointless, and most likely impossible to be acurate. (Though there might be some scenarios where it could be done).
Look at Windows file copying in the past "10 seconds left.... 2 minutes left.... 5 seconds left" It kept changing its estimate based on whatever metrics it used. Better to just show a spnning image, or message, to let the user know something is going on.
If you are processing a list of items, then it will be much easier for you, as the message could be:
Processing item 4 of 100.
At least then the user can know, roughly, what the code is doing. If you have nothing like this to inform the user, then I would cut your losses and show a simple "Processing...." message, or some icon, whatever takes your fancy for your solution.
I've got collection of 50k records, and i get it in one second, but loading it to database take about 10 seconds.
How to increase loading data?
Everything what I make now is:
dgvCars.DataSource=cars;
Data-binding 50k rows is going to take a while. I would first look at reducing the data volume (what is any user really going to do with 50k rows). But otherwise: "virtual mode" (what | how).
Edit; I suspect most of the time is being spent doing things like building control trees and other structures, but it might be that the reflection-based member-access is slowing this down; if so, maybe HyperDescriptor could help (simply by adding a 1 line call in your code to enable it for the associated type).
Its typically more work, but you might look into asynchronous querying. Its more work, but as you can get a subset batch of data back, display it to the grid. Then on continued background results getting returned, just add to the underlying table being displayed in the grid on an as needed basis. Don't worry about pulling the full 50k records down.