Develop Reference

Best practice for iterating over an ad-hoc list of strings in C# [duplicate]

MyClass[] array;
List<MyClass> list;
What are the scenarios when one is preferable over the other? And why?

It is rare, in reality, that you would want to use an array. Definitely use a List<T> any time you want to add/remove data, since resizing arrays is expensive. If you know the data is fixed length, and you want to micro-optimise for some very specific reason (after benchmarking), then an array may be useful.
List<T> offers a lot more functionality than an array (although LINQ evens it up a bit), and is almost always the right choice. Except for params arguments, of course. ;-p
As a counter - List<T> is one-dimensional; where-as you have have rectangular (etc) arrays like int[,] or string[,,] - but there are other ways of modelling such data (if you need) in an object model.
See also:
How/When to abandon the use of Arrays in c#.net?
Arrays, What's the point?
That said, I make a lot of use of arrays in my protobuf-net project; entirely for performance:
it does a lot of bit-shifting, so a byte[] is pretty much essential for encoding;
I use a local rolling byte[] buffer which I fill before sending down to the underlying stream (and v.v.); quicker than BufferedStream etc;
it internally uses an array-based model of objects (Foo[] rather than List<Foo>), since the size is fixed once built, and needs to be very fast.
But this is definitely an exception; for general line-of-business processing, a List<T> wins every time.

Really just answering to add a link which I'm surprised hasn't been mentioned yet: Eric's Lippert's blog entry on "Arrays considered somewhat harmful."
You can judge from the title that it's suggesting using collections wherever practical - but as Marc rightly points out, there are plenty of places where an array really is the only practical solution.

Notwithstanding the other answers recommending List<T>, you'll want to use arrays when handling:
image bitmap data
other low-level data-structures (i.e. network protocols)

Unless you are really concerned with performance, and by that I mean, "Why are you using .Net instead of C++?" you should stick with List<>. It's easier to maintain and does all the dirty work of resizing an array behind the scenes for you. (If necessary, List<> is pretty smart about choosing array sizes so it doesn't need to usually.)

Arrays should be used in preference to List when the immutability of the collection itself is part of the contract between the client & provider code (not necessarily immutability of the items within the collection) AND when IEnumerable is not suitable.
For example,
var str = "This is a string";
var strChars = str.ToCharArray(); // returns array
It is clear that modification of "strChars" will not mutate the original "str" object, irrespective implementation-level knowledge of "str"'s underlying type.
But suppose that
var str = "This is a string";
var strChars = str.ToCharList(); // returns List<char>
strChars.Insert(0, 'X');
In this case, it's not clear from that code-snippet alone if the insert method will or will not mutate the original "str" object. It requires implementation level knowledge of String to make that determination, which breaks Design by Contract approach. In the case of String, it's not a big deal, but it can be a big deal in almost every other case. Setting the List to read-only does help but results in run-time errors, not compile-time.

If I know exactly how many elements I'm going to need, say I need 5 elements and only ever 5 elements then I use an array. Otherwise I just use a List<T>.

Arrays Vs. Lists is a classic maintainability vs. performance problem. The rule of thumb that nearly all developers follow is that you should shoot for both, but when they come in to conflict, choose maintainability over performance. The exception to that rule is when performance has already proven to be an issue. If you carry this principle in to Arrays Vs. Lists, then what you get is this:
Use strongly typed lists until you hit performance problems. If you hit a performance problem, make a decision as to whether dropping out to arrays will benefit your solution with performance more than it will be a detriment to your solution in terms of maintenance.

Most of the times, using a List would suffice. A List uses an internal array to handle its data, and automatically resizes the array when adding more elements to the List than its current capacity, which makes it more easy to use than an array, where you need to know the capacity beforehand.
See http://msdn.microsoft.com/en-us/library/ms379570(v=vs.80).aspx#datastructures20_1_topic5 for more information about Lists in C# or just decompile System.Collections.Generic.List<T>.
If you need multidimensional data (for example using a matrix or in graphics programming), you would probably go with an array instead.
As always, if memory or performance is an issue, measure it! Otherwise you could be making false assumptions about the code.

Another situation not yet mentioned is when one will have a large number of items, each of which consists of a fixed bunch of related-but-independent variables stuck together (e.g. the coordinates of a point, or the vertices of a 3d triangle). An array of exposed-field structures will allow the its elements to be efficiently modified "in place"--something which is not possible with any other collection type. Because an array of structures holds its elements consecutively in RAM, sequential accesses to array elements can be very fast. In situations where code will need to make many sequential passes through an array, an array of structures may outperform an array or other collection of class object references by a factor of 2:1; further, the ability to update elements in place may allow an array of structures to outperform any other kind of collection of structures.
Although arrays are not resizable, it is not difficult to have code store an array reference along with the number of elements that are in use, and replace the array with a larger one as required. Alternatively, one could easily write code for a type which behaved much like a List<T> but exposed its backing store, thus allowing one to say either MyPoints.Add(nextPoint); or MyPoints.Items[23].X += 5;. Note that the latter would not necessarily throw an exception if code tried to access beyond the end of the list, but usage would otherwise be conceptually quite similar to List<T>.

Rather than going through a comparison of the features of each data type, I think the most pragmatic answer is "the differences probably aren't that important for what you need to accomplish, especially since they both implement IEnumerable, so follow popular convention and use a List until you have a reason not to, at which point you probably will have your reason for using an array over a List."
Most of the time in managed code you're going to want to favor collections being as easy to work with as possible over worrying about micro-optimizations.

Lists in .NET are wrappers over arrays, and use an array internally. The time complexity of operations on lists is the same as would be with arrays, however there is a little more overhead with all the added functionality / ease of use of lists (such as automatic resizing and the methods that come with the list class). Pretty much, I would recommend using lists in all cases unless there is a compelling reason not to do so, such as if you need to write extremely optimized code, or are working with other code that is built around arrays.

Since no one mention: In C#, an array is a list. MyClass[] and List<MyClass> both implement IList<MyClass>. (e.g. void Foo(IList<int> foo) can be called like Foo(new[] { 1, 2, 3 }) or Foo(new List<int> { 1, 2, 3 }) )
So, if you are writing a method that accepts a List<MyClass> as an argument, but uses only subset of features, you may want to declare as IList<MyClass> instead for callers' convenience.
Details:
Why array implements IList?
How do arrays in C# partially implement IList<T>?

They may be unpopular, but I am a fan of Arrays in game projects.
- Iteration speed can be important in some cases, foreach on an Array has significantly less overhead if you are not doing much per element
- Adding and removing is not that hard with helper functions
- Its slower, but in cases where you only build it once it may not matter
- In most cases, less extra memory is wasted (only really significant with Arrays of structs)
- Slightly less garbage and pointers and pointer chasing
That being said, I use List far more often than Arrays in practice, but they each have their place.
It would be nice if List where a built in type so that they could optimize out the wrapper and enumeration overhead.

Populating a list is easier than an array. For arrays, you need to know the exact length of data, but for lists, data size can be any. And, you can convert a list into an array.
List<URLDTO> urls = new List<URLDTO>();
urls.Add(new URLDTO() {
key = "wiki",
url = "https://...",
});
urls.Add(new URLDTO()
{
key = "url",
url = "http://...",
});
urls.Add(new URLDTO()
{
key = "dir",
url = "https://...",
});
// convert a list into an array: URLDTO[]
return urls.ToArray();

Keep in mind that with List is not possible to do this:
List<string> arr = new List<string>();
arr.Add("string a");
arr.Add("string b");
arr.Add("string c");
arr.Add("string d");
arr[10] = "new string";
It generates an Exception.
Instead with arrays:
string[] strArr = new string[20];
strArr[0] = "string a";
strArr[1] = "string b";
strArr[2] = "string c";
strArr[3] = "string d";
strArr[10] = "new string";
But with Arrays there is not an automatic data structure resizing. You have to manage it manually or with Array.Resize method.
A trick could be initialize a List with an empty array.
List<string> arr = new List<string>(new string[100]);
arr[10] = "new string";
But in this case if you put a new element using Add method it will be injected in the end of the List.
List<string> arr = new List<string>(new string[100]);
arr[10] = "new string";
arr.Add("bla bla bla"); // this will be in the end of List

It completely depends on the contexts in which the data structure is needed. For example, if you are creating items to be used by other functions or services using List is the perfect way to accomplish it.
Now if you have a list of items and you just want to display them, say on a web page array is the container you need to use.

Push Item to the end of an array

No, I can't use generic Collections. What I am trying to do is pretty simple actually. In php I would do something like this
$foo = [];
$foo[] = 1;
What I have in C# is this
var foo = new int [10];
// yeah that's pretty much it
Now I can do something like foo[foo.length - 1] = 1 but that obviously wont work. Another option is foo[foo.Count(x => x.HasValue)] = 1 along with a nullable int during declaration. But there has to be a simpler way around this trivial task.
This is homework and I don't want to explain to my teacher (and possibly the entire class) what foo[foo.Count(x => x.HasValue)] = 1 is and why it works etc.

The simplest way is to create a new class that holds the index of the inserted item:
public class PushPopIntArray
{
private int[] _vals = new int[10];
private int _nextIndex = 0;
public void Push(int val)
{
if (_nextIndex >= _vals.Length)
throw new InvalidOperationException("No more values left to push");
_vals[_nextIndex] = val;
_nextIndex++;
}
public int Pop()
{
if (_nextIndex <= 0)
throw new InvalidOperationException("No more values left to pop");
_nextIndex--;
return _vals[_nextIndex];
}
}
You could add overloads to get the entire array, or to index directly into it if you wanted. You could also add overloads or constructors to create different sized arrays, etc.

In C#, arrays cannot be resized dynamically. You can use Array.Resize (but this will probably be bad for performance) or substitute for ArrayList type instead.

But there has to be a simpler way around this trivial task.
Nope. Not all languages do everything as easy as each other, this is why Collections were invented. C# <> python <> php <> java. Pick whichever suits you better, but equivalent effort isn't always the case when moving from one language to another.

foo[foo.Length] won't work because foo.Length index is outside the array.
Last item is at index foo.Length - 1
After that an array is a fixed size structure if you expect it to work the same as in php you're just plainly wrong

Originally I wrote this as a comment, but I think it contains enough important points to warrant writing it as an answer.
You seem to be under the impression that C# is an awkward language because you stubbornly insist on using an array while having the requirement that you should "push items onto the end", as evidenced by this comment:
Isn't pushing items into the array kind of the entire purpose of the data structure?
To answer that: no, the purpose of the array data structure is to have a contiguous block of pre-allocated memory to mimic the original array structure in C(++) that you can easily index and perform pointer arithmetic on.
If you want a data structure that supports certain operations, such as pushing elements onto the end, consider a System.Collections.Generic.List<T>, or, if you insist on avoiding generics, a System.Collections.List. There are specializations that specify the underlying storage structure (such as ArrayList) but in general the whole point of the C# library is that you don't want to concern yourself with such details: the List<T> class has certain guarantees on its operations (e.g. insertion is O(n), retrieval is O(1) -- just like an array) and whether there is an array or some linked list that actually holds the data is irrelevant and is in fact dynamically decided based on the size and use case of the list at runtime.
Don't try to compare PHP and C# by comparing PHP arrays with C# arrays - they have different programming paradigms and the way to solve a problem in one does not necessarily carry over to the other.
To answer the question as written, I see two options then:
Use arrays the awkward way. Either create an array of Nullable<int>s and accept some boxing / unboxing and unpleasant LINQ statements for insertion; or keep an additional counter (preferably wrapped up in a class together with the array) to keep track of the last assigned element.
Use a proper data structure with appropriate guarantees on the operations that matter, such as List<T> which is effectively the (much better, optimised) built-in version of the second option above.
I understand that the latter option is not feasible for you because of the constraints imposed by your teacher, but then do not be surprised that things are harder than the canonical way in another language, if you are not allowed to use the canonical way in this language.
Afterthought:
A hybrid alternative that just came to mind, is using a List for storage and then just calling .ToArray on it. In your insert method, just Add to the list and return the new array.

List<T> vs HashSet<T> - dynamic collection choice is efficient or not?

var usedIds = list.Count > 20 ? new HashSet<int>() as ICollection<int> : new List<int>();
Assuming that List is more performant with 20 or less items and HashSet is more performant with greater items amount (from this post), is it efficient approach to use different collection types dynamicaly based on the predictable items count?
All of the actions for each of the collection types will be the same.
PS: Also i have found HybridCollection Class which seems to do the same thing automaticaly, but i've never used it so i have no info on its performance either.
EDIT: My collection is mostly used as the buffer with many inserts and gets.

In theory, it could be, depending on how many and what type of operations you are performing on the collections. In practice, it would be a pretty rare case where such micro-optimization would justify the added complexity.
Also consider what type of data you are working with. If you are using int as the collection item as the first line of your question suggests, then the threshold is going to be quite a bit less than 20 where List is no longer faster than HashSet for many operations.
In any case, if you are going to do that, I would create a new collection class to handle it, something along the lines of the HybridDictionary, and expose it to your user code with some generic interface like IDictionary.
And make sure you profile it to be sure that your use case actually benefits from it.
There may even be a better option than either of those collections, depending on what exactly it is you are doing. i.e. if you are doing a lot of "before or after" inserts and traversals, then LinkedList might work better for you.

Hashtables like Hashset<T> and Dictionary<K,T> are faster at searching and inserting items in any order.
Arrays T[] are best used if you always have a fixed size and a lot of indexing operations. Adding items to a array is slower than adding into a list due to the covariance of arrays in c#.
List<T> are best used for dynamic sized collections whith indexing operations.
I don't think it is a good idea to write something like the hybrid collection better use a collection dependent on your requirements. If you have a buffer with a lof of index based operations i would not suggest a Hashtable, as somebody already quoted a Hashtable by design uses more memory

HashSet is for faster access, but List is for insert. If you don't plan adding new items, use HashSet, otherwise List.

If you collection is very small then the performance is virtually always going to be a non-issue. If you know that n is always less than 20, O(n) is, by definition, O(1). Everything is fast for small n.
Use the data structure that most appropriate represents how you are conceptually treating the data, the type of operations that you need to perform, and the type of operations that should be most efficient.

is it efficient approach to use different collection types dynamicaly based on the predictable items count?
It can be depending on what you mean by "efficiency" (MS offers HybridDictionary class for that, though unfortunately it is non generic). But irrespective of that its mostly a bad choice. I will explain both.
From an efficiency standpoint:
Addition will be always faster in a List<T>, since a HashSet<T> will have to precompute hash code and store it. Even though removal and lookup will be faster with a HashSet<T> as size grows up, addition to the end is where List<T> wins. You will have to decide which is more important to you.
HashSet<T> will come up with a memory overhead compared to List<T>. See this for some illustration.
But however, from a usability standpoint it need not make sense. A HashSet<T> is a set, unlike a bag which List<T> is. They are very different, and their uses are very different. For:
HashSet<T> cannot have duplicates.
HashSet<T> will not care about any order.
So when you return a hybrid ICollection<T>, your requirement goes like this: "It doesn't matter whether duplicates can be added or not. Sometimes let it be added, sometimes not. Of course iteration order is not important anyway" - very rarely useful.
Good q, and +1.

HashSet is better, because it will probably use less space, and you will have faster access to elements.

Efficiency: Creating an array of doubles incrementally?

Consider the following code:
List<double> l = new List<double>();
//add unknown number of values to the list
l.Add(0.1); //assume we don't have these values ahead of time.
l.Add(0.11);
l.Add(0.1);
l.ToArray(); //ultimately we want an array of doubles
Anything wrong with this approach? Is there a more appropriate way to build an array, without knowing the size, or elements ahead of time?

There's nothing wrong with your approach. You are using the correct data type for the purpose.

After some observations you can get a better idea of the total elements in that list. Then you can create a new list with an initial capacity in the constructor:
List<double> l = new List<double>(capacity);
Other than this, it's the proper technique and data structure.
UPDATE:
If you:
Need only the Add and ToArray functions of the List<T> structure,
And you can't really predict the total capacity
And you end up with more than 1K elements
And better performance is really really (really!) your goal
Then you might want to write your own interface:
public interface IArrayBuilder<T>
{
void Add(T item);
T[] ToArray();
}
And then write your own implementation, which might be better than List<T>. Why is that? because List<T> holds a single array internally, and it increases its size when needed. The procedure of increasing the inner array costs, in terms of performance, since it allocates new memory (and perhaps copies the elements from the old array to the new one, I don't remember). However, if all of the conditions described above are true, all you need is to build an array, you don't really need all of the data to be stored in a single array internally.
I know it's a long shot, but I think it's better sharing such thoughts...

As others have already pointed out: This is the correct approach. I'll just add that if you can somehow avoid the array and use List<T> directly or perhaps IEnumerable<T>, you'll avoid copying the array as ToArray actually copies the internal array of the list instance.
Eric Lippert has a great post about arrays, that you may find relevant.

A dynamic data structure like a List is the correct way to implement this. The only real advantage arrays have over a List is the O(1) access performance (compared to O(n) in List). The flexibility more than makes up for this performance loss imho

C# List<T>.ToArray performance is bad?

I'm using .Net 3.5 (C#) and I've heard the performance of C# List<T>.ToArray is "bad", since it memory copies for all elements to form a new array. Is that true?

No that's not true. Performance is good since all it does is memory copy all elements (*) to form a new array.
Of course it depends on what you define as "good" or "bad" performance.
(*) references for reference types, values for value types.
EDIT
In response to your comment, using Reflector is a good way to check the implementation (see below). Or just think for a couple of minutes about how you would implement it, and take it on trust that Microsoft's engineers won't come up with a worse solution.
public T[] ToArray()
{
T[] destinationArray = new T[this._size];
Array.Copy(this._items, 0, destinationArray, 0, this._size);
return destinationArray;
}
Of course, "good" or "bad" performance only has a meaning relative to some alternative. If in your specific case, there is an alternative technique to achieve your goal that is measurably faster, then you can consider performance to be "bad". If there is no such alternative, then performance is "good" (or "good enough").
EDIT 2
In response to the comment: "No re-construction of objects?" :
No reconstruction for reference types. For value types the values are copied, which could loosely be described as reconstruction.

Reasons to call ToArray()
If the returned value is not meant to be modified, returning it as an array makes that fact a bit clearer.
If the caller is expected to perform many non-sequential accesses to the data, there can be a performance benefit to an array over a List<>.
If you know you will need to pass the returned value to a third-party function that expects an array.
Compatibility with calling functions that need to work with .NET version 1 or 1.1. These versions don't have the List<> type (or any generic types, for that matter).
Reasons not to call ToArray()
If the caller ever does need to add or remove elements, a List<> is absolutely required.
The performance benefits are not necessarily guaranteed, especially if the caller is accessing the data in a sequential fashion. There is also the additional step of converting from List<> to array, which takes processing time.
The caller can always convert the list to an array themselves.
taken from here

Yes, it's true that it does a memory copy of all elements. Is it a performance problem? That depends on your performance requirements.
A List contains an array internally to hold all the elements. The array grows if the capacity is no longer sufficient for the list. Any time that happens, the list will copy all elements into a new array. That happens all the time, and for most people that is no performance problem.
E.g. a list with a default constructor starts at capacity 16, and when you .Add() the 17th element, it creates a new array of size 32, copies the 16 old values and adds the 17th.
The size difference is also the reason why ToArray() returns a new array instance instead of passing the private reference.

This is what Microsoft's official documentation says about List.ToArray's time complexity
The elements are copied using Array.Copy, which is an O(n) operation, where n is Count.
Then, looking at Array.Copy, we see that it is usually not cloning the data but instead using references:
If sourceArray and destinationArray are both reference-type arrays or are both arrays of type Object, a shallow copy is performed. A shallow copy of an Array is a new Array containing references to the same elements as the original Array. The elements themselves or anything referenced by the elements are not copied. In contrast, a deep copy of an Array copies the elements and everything directly or indirectly referenced by the elements.
So in conclusion, this is a pretty efficient way of getting an array from a list.

it creates new references in an array, but that's just the only thing that that method could and should do...

Performance has to be understood in relative terms. Converting an array to a List involves copying the array, and the cost of that will depend on the size of the array. But you have to compare that cost to other other things your program is doing. How did you obtain the information to put into the array in the first place? If it was by reading from the disk, or a network connection, or a database, then an array copy in memory is very unlikely to make a detectable difference to the time taken.

For any kind of List/ICollection where it knows the length, it can allocate an array of exactly the right size from the start.
T[] destinationArray = new T[this._size];
Array.Copy(this._items, 0, destinationArray, 0, this._size);
return destinationArray;
If your source type is IEnumerable (not a List/Collection) then the source is:
items = new TElement[4];
..
if (no more space) {
TElement[] newItems = new TElement[checked(count * 2)];
Array.Copy(items, 0, newItems, 0, count);
items = newItems;
It starts at size 4 and grows exponentially, doubling each time it runs out of space. Each time it doubles, it has to reallocate memory and copy the data over.
If we know the source-data size, we can avoid this slight overhead. However in most cases eg array size <=1024, it will execute so quickly, that we don't even need to think about this implementation detail.
References: Enumerable.cs, List.cs (F12ing into them), Joe's answer