When I say list, I mean List, array, HashTable, things like that where you can iterate through with an IndexOf method.
I have a program that wants to make many repeated lookups to its lists, which is obviously slow if you have a list format that iterates through every value starting at 0 every time you do an IndexOf (and to a lesser extent slower than a direct reference when using hashtables with many many lookups). Before I do this, I want to ask;
Is there an existing IEnumerable that is optimized for repeated lookups?
Assuming this hypothetical IEnumerable keeps a reference of the lookup, (giving a key to the object doing the lookup to be stored), is this even a good idea in terms of locking in items that the garbage collector could pick up? Is there a way to mark fields as not "not important to keep in memory despite a reference existing" and "garbage-collectible"? (This isn't a problem I don't know how to fix as much as it would be something I would rather implement rather than working around)
If the answer to either of these is "no", then I'll make an IList that keeps a reference of the last index of a lookup, which wouldn't bog up the garbage collector.
The program I'm working on is a UI system, where Widgets tend to take up a decent amount of memory as well as changing a lot, leaving a garbage collector as my only means freeing up memory. Willy nilly references laying around are dangerous to account for.
Related
I could see that CoreCLR and CoreFx implicitly use array for most of the generic collections. what is the main driving factor to go with arrays and how it handles any side effects of LOH fragmentation.
What other then arrays should collections be?
More importnatly, what other then arrays could collections be?
In use collection boils down to "arrays - and stuff we wrap around arrays, for ease of use.":
The pure thing (arrays), wich do offer some conveniences like bounds checks in C#/.NET
Self growing arrays (Lists)
Two synchronized arrays that allow the mapping of any any input to any element (Dictionaries key/value pair)
Three synchornized array: Key, Value and a Hashvalue to quickly identify not-matching keys (HastTable).
Below the hood - regardless of how hard .NET makes it to use pointers - it all boils down to some code doing C/C++ style pointer arythmethic to get the next element.
Edit 1: As I learned in another place, .NET Dictionaries are actually implemented as HashLists. The HashList class is just the pre-generics version. Object has a GetHashCode function with sensible default behavior wich can be used, but also fully overwritten.
Fragmentation wise the "best" would be a array of references. It can be as small as the reference width (a Pointer or slightly bigger) and the GC can move around the instances to defragment memory. Of course then you get the slight overhead of accessing references rather the just counting/mathing up a pointer, so as usualy it is a memory vs speed tradeoff. However this might go into Speed Rant Territory of detail.
Edit 2: As Markus Appel pointed out in the comments, there is something even better for fragmentation avoidance: Linked lists. Even that single array of references - if you just make it big enough - will take quite some memory in one indivisible chunk. So it might run into object size limits or array indexer limits. A linked list will do neither. But as a result the performance is around a disk that was never defragmented.
Generics is just a convience to have typesafety in collections/other places. It avoids you having to use the dreaded Object as type, wich ruins all compile-time typesafety. Afaik they add nothing else to this situation. List<string> works the same as a StringList would.
Array access is faster as it is a linear storage. If Arrays can solve a problem well enough they are a better storage for traversal rather than always identifying where the next object is stored. For Large data structures this performance benefit will also be amplified.
Using arrays can cause fragmentation if used carelessly. In the general case though, the performance gains outweigh the cost.
When the buffer runs out, the collection allocates a new one with double the size. If the code inserts a lot of items without specifying a capacity, this results in log2(N) reallocations. If the code does specify a capacity though, even a very rough approximation, there may be no fragmentation issues at all.
Removal is another expensive case as the collection will have to move the items after the deleted item(s) to the left.
In general though, array storage offers far better performance than other storage structures though, both for reading, inserting and allocating memory. Deletions are rare in most cases.
For example, inserting N items in a linked list requires allocating N objects to hold that value and storing N pointers. That cost will be paid for every insertion, while the GC will have a lot more objects to track and collect. Inserting 100K items in a linked list would allocate 100K node objects that would need tracking.
With an array there won't be any allocations unless the buffer runs out. In the majority of cases insertion means simply writing to a buffer location and updating a count. When the buffer runs out there will be a single reallocation and an (expensive) copy operation. For 100K items, that's 17 allocations. In most cases, that's an acceptable cost.
To reduce or even get rid of allocations, the code can specify a capacity that's used as the initial buffer size. Specifying even a very rough estimate can reduce allocations a lot. Specifying 1024 as the initial capacity for 100K items would reduce reallocations to 7.
Sorry for my last question, my code was so stupid.
My base situation is: I want to construct a state tree which has 8! items in the last state. so the total count of iterations is about 100.000 (8!*2 + 7! + 6! + ... )
it currently takes less than one second, i need to construct it every time my artificial intelligence is making a move. Of course, the alpha/beta search is a solution but before thinking of that i want to optimize my code so i really have the best possible performance.
what i already did:
tried to replace every LINQ function with precalculations or collections with faster access (Dictionary), more precalculations for skipping whole operations, of course, some approximations to spare heavy calculations, using List constructors only when there's actually a change, if not, just use the reference.
there'll be more calculations coming so i really need more ideas for reducing. maybe something about what collection is fastest for my purpose.
My code
It's about the BuildChildNodes function and the called TryCollect function. My Constructor is doing some little precalculations. my state tree knows everything, even the cards which aren't actually shown.
as the comment came up: i'm not asking you to read and understand my code to provide content-wise advices. i'm asking you about the functions, operators, data types and classes i'm using and if there could be make a replacement which runs a bit faster. e.g. if there's a faster collection for my purpose or if you have a better idea to replace the collections constructor with a faster method reagarding of adding and removing afterwards.
Edit: okay List is definitely the best type i can use. i tried [] Arrays and even Dictionaries () and last of all i even tried LinkedLists. All with a significant loss.
I can see that RemoveAt() could be expensive at it is proportional to the size of the list.
You can always use the Visual Studio performance profiler to find out where you should optimize your code the most.
If you can find a way to use fixed-size arrays that you allocate when your program starts, instead of dynamically-allocated data structures like List, you will save of lot on memory allocation management overhead.
Ok, I want to do the following to me it seems like a good idea so if there's no way to do what I'm asking, I'm sure there's a reasonable alternative.
Anyways, I have a sparse matrix. It's pretty big and mostly empty. I have a class called MatrixNode that's basically a wrapper around each of the cells in the matrix. Through it you can get and set the value of that cell. It also has Up, Down, Left and Right properties that return a new MatrixNode that points to the corresponding cell.
Now, since the matrix is mostly empty, having a live node for each cell, including the empty ones, is an unacceptable memory overhead. The other solution is to make new instances of MatrixNode every time a node is requested. This will make sure that only the needed nodes are kept in the memory and the rest will be collected. What I don't like about it is that a new object has to be created every time. I'm scared about it being too slow.
So here's what I've come up with. Have a dictionary of weak references to nodes. When a node is requested, if it doesn't exist, the dictionary creates it and stores it as a weak reference. If the node does already exist (probably referenced somewhere), it just returns it.
Then, if the node doesn't have any live references left, instead of it being collected, I want to store it in a pool. Later, when a new node is needed, I want to first check if the pool is empty and only make a new node if there isn't one already available that can just have it's data swapped out.
Can this be done?
A better question would be, does .NET already do this for me? Am I right in worrying about the performance of creating single use objects in large numbers?
Instead of guessing, you should make a performance test to see if there are any issues at all. You may be surprised to know that managed memory allocation can often outperform explicit allocation because your code doesn't have to pay for deallocation when your data goes out of scope.
Performance may become an issue only when you are allocating new objects so frequently that the garbage collector has no chance to collect them.
That said, there are sparse array implementations in C# already, like Math.NET and MetaNumerics. These libraries are already optimized for performance and will probably avoid performance issues you will run into if you start your implementation from stratch
An SO search for c# and sparse-matrix will return many related questions, including answers pointing to commercial libraries like ILNumerics (has a community edition), NMath and Extreme Optimization's libraries
Most sparse matrix implementations use one of a few well-known schemes for their data; I generally recommend CSR or CSC, as those are efficient for common operations.
If that seems too complex, you can start using COO. What this means in your code is that you will not store anything for empty members; however, you have an item for every non-empty one. A simple implementation might be:
public struct SparseMatrixItem
{
int Row;
int Col;
double Value;
}
And your matrix would generally be a simple container:
public interface SparseMatrix
{
public IList<SparseMatrixItem> Items { get; }
}
You should make sure that the Items list stays sorted according to the row and col indices, because then you can use binary search to quickly find out if an item exists for a specific (i,j).
The idea of having a pool of objects that people use and then return to the pool is used for really expensive objects. Objects representing a network connection, a new thread, etc. It sounds like your object is very small and easy to create. Given that, you're almost certainly going to harm performance pooling it; the overhead of managing the pool will be greater than the cost of just creating a new one each time.
Having lots of short lived very small objects is the exact case that the GC is designed to handle quickly. Creating a new object is dirt cheap; it's just moving a pointer up and clearing out the bits for that object. The real overhead for objects comes in when a new garbage collection happens; for that it needs to find all "alive" objects and move them around, leaving all "dead" objects in their place. If your small object doesn't live through a single collection it has added almost no overhead. Keeping the objects around for a long time (like, say, by pooling them so you can reuse them) means copying them through several collections, consuming a fair bit of resources.
This question already has answers here:
Using the "clear" method vs. New Object
(5 answers)
Closed 8 years ago.
I have a list of elements that steadily grows, until I dump all the data from that list into a file. I then want to reuse that list for the same purpose again. Is it bad practice to simply assign it to a new list, instead of removing all the elements from the list? It seems garbage collection should take care of the old list, and that way I don't have to worry about removing the elements.
For example:
var myList = new List<element>();
myList.Add(someElement);
myList.Add(anotherElement);
// dumps the elements into a file
myList = new List<element>();
Edit: Even if there are easy ways around this, I was wondering too about the philosophical side of it. Is it bad to let something be garbage collected if there is a way around it? What are the costs of allowing garbage collection vs deleting the elements and reusing the same memory?
It depends a bit on how many elements are in the list. If the array backing the list is large enough to be on the large object heap, then you might be better off clearing the list and reusing it. This will reduce the number of large memory allocations, and will help reduce the problem of large object heap fragmentation. (See http://msdn.microsoft.com/en-us/magazine/cc534993.aspx and http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/ for more information; see http://blogs.msdn.com/b/dotnet/archive/2011/10/04/large-object-heap-improvements-in-net-4-5.aspx for improvements due with .NET 4.5)
If the lists are small, you might be better off just creating a new list, or you might get better performance calling Clear(). When in doubt, measure the performance.
Edit: In response to the philosophical question you pose in your edit, here are two reasons to create a new list:
In general, code is cleaner and easier to reason about if you do not reuse objects. The cost of garbage collection is low, the cost of confusing code is high.
Consider what happens if the code dumping the list's contents is in another function, as it most likely is. Once you've passed that list out of its local context, it's possible that there are non-local references to the same list. Other code might be modifying the list, or might be assuming (incorrectly) that you're not modifying it.
myList.Clear() is even easier to code than myList = new List<element>();
msdn: List.Clear Method
Each element in the list is a different object itself, and will need to be garbage collected whether you clear the list, or recreate a new list, or remove the items one at a time. What will NOT need to be garbage collected if you just clear the list and reuse it is the list itself. Unless your list is huge, containing hundreds of thousands of items, it will be difficult to measure a performance difference one way or the other. Fortunately, the garbage collector is highly optimized and it's a rare occurrence where developers need to consider what it is doing.
(As others have pointed out, there are various factors involved, such as...how many elements will you be adding to the new list? vs how many elements were in the old list? ...but the point is: the garbage collection of the list itself isn't relevant when it comes to collecting the elements of the list.)
I'm no expert, but:
Making a new list expecting that the GC will "take care" of the old one is probably a bad idea because it's a bad practice & probably inefficient.
Although it's a micro-optimization, I'd say that "setting" the new values until you reach list.Count, and the continuing to list.Add is the best way, because then you don't clear nor allocate unnecessary new memory (unless it's large lists which you want to clear for space)
Anyway, I would recommend using List.Clear() - it saves you and the GC trouble.
It sounds like you're asking two different questions. One is whether it's okay to set it to a new object or just clear it, which I think Eric answered pretty well. The second is whether you should just ignore the GC and let it work without trying to "help" it - to that, I'd say absolutely YES. Let the framework do what the framework does and stay out of its way until you have to.
A lot of programmers want to dig in too deep, and most of the time it causes more problems than it helps. The GC is designed to collect these things and clean them up for you. Unless you are seeing a very specific problem, you should write the code that works and pay ignore when something will be collected (with the exception of the using keyword when appropriate).
The important perspective is clean code.
when you create a new list the old one will be removed by the GC (if there are no other reference to it.)
I would rather to use List.Clear() to remove all the elements for re-use. The Capacity remain unchanged so there shouldn't have additional overhead cost and letting GC to handle the memory garbage collection so you can maintain clean code.
There is an established guideline that getting a hashcode should not allocate memory because this will negatively impact hash table lookups by invoking the garbage collector.
Yet this exact failing is what I see what I profile my application which uses a System.Collections.Generic.Dictionary
Way deep down in a very tight loop I find the following in my profiler results:
[3.47%] TryGetValue(TKey, TValue&) (...Dictionary)
[3.47%] FindEntry(TKey) (...Dictionary)
[3.47%] GetHashCode(string) (System.CultureAwareComparer)
[3.46%] GetHashCodeOfString(String, CompareOptions) (System.Globalization.CompareInfo)
[3.39%] [Garbage Collection]
[0.01%] [Thread Suspendended]
That's the whole sub-tree accounting from the profiler.
I'm not a seasoned expert in this specific sort of work, so I could be reading these tea leaves incorrectly. But it looks to me like GetHashCodeOfString "must be" allocating memory and inviting the garbage collector to interrupt my program in the middle of this loop I want REALLY TUNED AND TIGHT, and this is accounting for the staggering majority of the cost of this loop.
As an aside, here is an additional piece of evidence suggesting this code allocates memory
My next step will be to initialize the Dictionary with the ordinal comparer and re-run my tests.
But I want to know if there is existing wisdom out there around this issue. It seems like dictionaries with string keys are common, and the costs of such a common thing may be well explored. I found the following analysis, but it focuses on the actual comparison as the cause for woe, and not the hash code method allocating memory.
Can anyone suggest the proper way to use a dictionary with string keys that avoids this problem?
Specific questions I have include:
If I use the ordinal comparitor will the allocation go away?
If not, do I need to write my own comparitor, and will THAT make the allocation go away?
If I do make the comparitor go away, can I really expect a real improvement, as per the MSFT recommendation link I started with?
EDIT: Crud, my bad, but this is not with the default comparer properties, we have it set to ignoreCase. Not sure if this impacts the results, but since ignoreCase would impact the equality, it must therefor have some impact on the hash.
UPDATE: Ran another test using the ordinal comparer (still with IgnoreCase), and recast the original results output to 100% cost = TryGetValue so it would be more apples to apples
Original:
100% TryGetValue
100% FindEntry
99.5% CultureAwareComparer.GetHashCode
99.5% CompareInfo.GetHashCodeOfString
95.86% [Garbage Collection]
3.31% [Thread Suspended]
0.5% CultureAwareComparer.Equals
0.5% Compare
0.5% [garbage collection]
Ordinal:
100% TryGetValue
100% FindEntry
47.22% CultureAwareComparer.Equals
47.22% [Garbage Collection]
There also appeared to be a dramatic decrease in the overall time spend in TryGetValue. I was not careful to make sure all else was equal, but this accounted for 46 seconds out of a 10 minute stress test in the first run, and in the orindal run it accounted for 252 milliseconds. Consider that anecdotal, not an expected relative cost.
It seems like the entire cost of the hash, which used to be 99+% of the cost, is now so "free" that it fails to even appear in the profiler, which I think is running in sampling mode.
I guess this seconds the word on the street that you should use ordinal comparison.
I still can't PROVE to myself why the GC cost is contributing so heavily to the first profile result, but from the comments below I suppose I have to believe it does NOT allocate managed heap memory, but that because it's slow, it tends to be the function that is "randomly" GCed by other activities on other threads, as this process is indeed using server mode gc.
Maybe this indicates that this tight loop tends to be concurrent with allocation-happy code someplace else.
By default, when you use string keys, string.GetHashCode() is used. This method doesn't allocate any memory on the heap, and should be pretty fast.
But since you're using ignore case, CultureAwareComparer.GetHashCode() is used instead. That method calls (as can be seen from your profile results) CompareInfo.GetHashCodeOfString(), which in turn calls the unmanaged function InternalGetGlobalizedHashCode(). Neither of the two managed methods makes any heap allocations (as you can see if you look at them in a decompiler). I can't say what InternalGetGlobalizedHashCode() does, but since it is unmanaged, I doubt it makes any allocations on the managed heap. In any case, it has to be quite a lot more complex than the default hash code computation, especially since it is culture-aware and has to keep in mind issues like the Turkish İ.
What this means is that you probably have some other code that allocates memory on the heap, which causes the garbage collection.
And if you are going for maximum performance, you should avoid “ignore case”, and especially its culture-aware variants.