Develop Reference

C# - Accessing Lists from another Class or copying it? - Performance & Best Practice

I have a general question, concerning performance and best practice.
When working with a List (or any other datatype) from a different Class, which is better practice? Copying it at the beginning, working with the local and then re-copying it to the original, or always access the original?
An Example:
access the original:
public class A
{
public static List<int> list = new List<int>();
}
public class B
{
public static void insertString(int i)
{
// insert at right place
int count = A.list.Count;
if (count == 0)
{
A.list.Add(i);
}
else
{
for (int j = 0; j < count; j++)
{
if (A.list[j] >= i)
{
A.list.Insert(j, i);
break;
}
if (j == count - 1)
{
A.list.Add(i);
}
}
}
}
}
As you see I access the original List A.list several times. Here the alternative:
Copying:
public class A
{
public static List<int> list = new List<int>();
}
public class B
{
public static void insertString(int i)
{
List<int> localList = A.list;
// insert at right place
int count = localList.Count;
if (count == 0)
{
localList.Add(i);
}
else
{
for (int j = 0; j < count; j++)
{
if (localList[j] >= i)
{
localList.Insert(j, i);
break;
}
if (j == count - 1)
{
localList.Add(i);
}
}
}
A.list = localList;
}
}
Here I access the the list in the other class only twice (getting it at the beginning and setting it at the end). Which would be better.
Please note that this is a general question and that the algorithm is only an example.

I won't bother thinking about performance here and instead focus on best practice:
Giving out the whole List violates encapsulation. B can modify the List and all its elements without A noticing (This is not a problem if A never uses the List itself but then A wouldn't even need to store it).
A simple example: A creates the List and immediately adds one element. Subsequently, A never bothers to check List.Count, because it knows that the List cannot be empty. Now B comes along and empties the List...
So any time B is changed, you need to also check A to see if all the assumptions of A are still correct. This is enough of a headache if you have full control over the code. If another programmer uses your class A, he may do something unexpected with the List and never check if that's ok.
Solution(s):
If B only needs to iterate over the elements, write an IEnumerable accessor. If B mustn't modify the elements, make the accessor deliver copies.
If B needs to modify the List (add/remove elements), either give B a copy of the List (containing copies of the elements if they needn't be modified) and accept a new List from B or use an accessor as before and implement the necessary List operations. In both cases, A will know if B modifies the List and can react accordingly.
Example:
class A
{
private List<ItemType> internalList;
public IEnumerable<ItemType> Items()
{
foreach (var item in internalList)
yield return item;
// or maybe item.Copy();
// new ItemType(item);
// depending on ItemType
}
public RemoveFromList(ItemType toRemove)
{
internalList.Remove(toRemove);
// do other things necessary to keep A in a consistent state
}
}

C# data structure, list which can dynamically resize up to a given limit, and allows fast access to any index

I'm implementing a memory system for an AI agent. It needs to have an internal list of state transitions which is capped at some number, say 10000.
If at capacity, adding a new memory should automatically remove the oldest memory.
Importantly, I should also need to be able to quickly access any item in this list.
A wrapper for Queue at first seemed obvious, but Queue does not allow fast access of any element. (O(n))
Similarly, remove an item from the beginning of a List structure takes O(n).
LinkedLists allow fast additions and removals, but again do not allow quick access to every index.
An array would allow random access but obviously it's not dynamically resizeable and deletion is problematic.
I've seen a HashMap being suggested but I'm ensure how that might be implemented.
Suggestions?

If you want the queue to be a fixed length, you could use a circular buffer which enables O(1) enqueue, dequeue and indexing operations and automatically overwrites old entries when the queue is full.

Try using a Dictionary with a LinkedList. The keys of the Dictionary are the indexes of the LinkedList nodes and the values of the Dictionary are of type LinkedListNode; that is, the LinkedList nodes.
The Dictionary would give you almost an O(1) on its operations and removing/adding LinkedListNode(s) to the beginning or end of a LinkedList is of O(1) as well.
Another alternative is to use a HashTable. However, in this case you have to know the capacity of the table beforehand (See Hashtable.Add Method) in order to get the O(1) performance:
If Count is less than the capacity of the Hashtable, this method is an O(1) operation. If the capacity needs to be increased to accommodate the new element, this method becomes an O(n) operation, where n is Count.
In the first solution, no matter what's the capcity of the LinkedList or the Dictionary you would still get almost an O(1) from both the Dictionary and the LinkedList. Of course that's going to be an O(3) or O(4) depending on the total number of operations that you perform on both the Dictionary and the LinkedList to do an add or remove operation inside your memory class. The search access is going to be always an O(1) because you will be using the Dictionary only.

HashMap is for Java, so the closest equivalent is Dictionary. C# Java HashMap equivalent. But I wouldn't say that this is the ultimate answer.
If you implement it as Dictionary, which key == the content, then you can search the content with O(1). However, you cannot have same key. Also, because it is not ordered, you may not know which the 1st content is.
If you implement it as Dictionary, which key == index, and value == the content, searching for the content still takes O(n) because you don't know the location of content.
A List or an Array will cost O(1) if you search the content by index reference. So, please double check your statement that it takes O(n)
If you search by index is sufficient, then circular array/ buffer which #Lee mentioned is good enough.
Otherwise, similar to DB, you might want to maintain in 2 separate data: 1 for storing the data (Circular Array) and the other one for search (Hash).

EDIT: #Lee has it right. A circular buffer seems to give you what you want. Answer left in place though.
I think the data structure you want might be a priority queue -- it depends on what you mean by 'quickly access any item'. If you mean 'able to enumerate all items in O(N)', then a priority queue fits the bill. If you mean 'enumerate the list in historical order', then it won't.
Assuming you need these operations;
add an item and associate with a time
remove the oldest item
enumerate all existing items in arbitrary order
Then you could easily extend this priority queue implementation I wrote a little while ago.
You'll want implement IEnumerable as a loop through the T[] data array from 0 to cursor. This will give you your enumeration.
Implement a GetItem(i) function which returns this.data[i] so long as i <= cursor.
Implement an automatic size limit by putting this into the Push() method;
if (queue.Size => 10000) {
queue.Pop();
}
I think this is O(ln n) for push and pop, and O(N) to enumerate ALL items, or O(i) to find ANY item, so long as you don't need them in order.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Mindfire.DataStructures
{
public class PiorityQueue<T>
{
private int[] priorities;
private T[] data;
private int cursor;
private int capacity;
public int Size
{
get
{
return cursor+1;
}
}
public PiorityQueue(int capacity)
{
this.cursor = -1;
this.capacity = capacity;
this.priorities = new int[this.capacity];
this.data = new T[this.capacity];
}
public T Pop()
{
if (this.Size == 0)
{
throw new InvalidOperationException($"The {this.GetType().Name} is Empty");
}
var result = this.data[0];
this.data[0] = this.data[cursor];
this.priorities[0] = this.priorities[cursor];
this.cursor--;
var loc = 0;
while (true)
{
var l = loc * 2;
var r = loc * 2 + 1;
var leftIsBigger = l <= cursor && this.priorities[loc] < this.priorities[l];
var rightIsBigger = r <= cursor && this.priorities[loc] < this.priorities[r];
if (leftIsBigger)
{
Swap(loc, l);
loc = l;
}
else if (rightIsBigger)
{
Swap(loc, r);
loc = r;
}
else
{
break;
}
}
return result;
}
public void Push(int priority, T v)
{
this.cursor++;
if (this.cursor == this.capacity)
{
Resize(this.capacity * 2);
};
this.data[this.cursor] = v;
this.priorities[this.cursor] = priority;
var loc = (this.cursor -1)/ 2;
while (this.priorities[loc] < this.priorities[cursor])
{
// swap
this.Swap(loc, cursor);
}
}
private void Swap(int a, int b)
{
if (a == b) { return; }
var data = this.data[b];
var priority = this.priorities[b];
this.data[b] = this.data[a];
this.priorities[b] = this.priorities[a];
this.priorities[a] = priority;
this.data[a] = data;
}
private void Resize(int newCapacity)
{
var newPriorities = new int[newCapacity];
var newData = new T[newCapacity];
this.priorities.CopyTo(newPriorities, 0);
this.data.CopyTo(newData, 0);
this.data = newData;
this.priorities = newPriorities;
this.capacity = newCapacity;
}
public PiorityQueue() : this(1)
{
}
public T Peek()
{
if (this.cursor > 0)
{
return this.data[0];
}
else
{
return default(T);
}
}
public void Push(T item, int priority)
{
}
}
}

Way to pad an array to avoid index outside of bounds of array error

I expect to have at least 183 items in my list when I query it, but sometimes the result from my extract results in items count lower than 183. My current fix supposedly pads the array in the case that the count is less than 183.
if (extractArray.Count() < 183) {
int arraysize= extractArray.Count();
var tempArr = new String[183 - arraysize];
List<string> itemsList = extractArray.ToList<string>();
itemsList.AddRange(tempArr);
var values = itemsList.ToArray();
//-- Process the new array that is now at least 183 in length
}
But it seems my solution is not the best. I would appreciate any other solutions that could help ensure I get at least 183 items whenever the extract happens please.

I'd probably follow others' suggestions, and use a list. Use the "capacity" constructor for added performance:
var list = new List<string>(183);
Then, whenever you get a new array, do this (replace " " with whatever value you use to pad the array):
list.Clear();
list.AddRange(array);
// logically, you can do this without the if, but it saves an object allocation when the array is full
if (array.Length < 183)
list.AddRange(Enumerable.Repeat(" ", 183 - array.Length));
This way, the list is always reusing the same internal array, reducing allocations and GC pressure.
Or, you could use an extension method:
public static class ArrayExtensions
{
public static T ElementOrDefault<T>(this T[] array, int index)
{
return ElementOrDefault(array, index, default(T));
}
public static T ElementOrDefault<T>(this T[] array, int index, T defaultValue)
{
return index < array.Length ? array[index] : defaultValue;
}
}
Then code like this:
items.Zero = array[0];
items.One = array[1];
//...
Becomes this:
items.Zero = array.ElementOrDefault(0);
items.One = array.ElementOrDefault(1);
//...
Finally, this is the rather cumbersome idea with which I started writing this answer: You could wrap the array in an IList implementation that's guaranteed to have 183 indexes (I've omitted most of the interface member implementations for brevity):
class ConstantSizeReadOnlyArrayWrapper<T> : IList<T>
{
private readonly T[] _array;
private readonly int _constantSize;
private readonly T _padValue;
public ConstantSizeReadOnlyArrayWrapper(T[] array, int constantSize, T padValue)
{
//parameter validation omitted for brevity
_array = array;
_constantSize = constantSize;
_padValue = padValue;
}
private int MissingItemCount
{
get { return _constantSize - _array.Length; }
}
public IEnumerator<T> GetEnumerator()
{
//maybe you don't need to implement this, or maybe just returning _array.GetEnumerator() would suffice.
return _array.Concat(Enumerable.Repeat(_padValue, MissingItemCount)).GetEnumerator();
}
public int Count
{
get { return _constantSize; }
}
public bool IsReadOnly
{
get { return true; }
}
public int IndexOf(T item)
{
var arrayIndex = Array.IndexOf(_array, item);
if (arrayIndex < 0 && item.Equals(_padValue))
return _array.Length;
return arrayIndex;
}
public T this[int index]
{
get
{
if (index < 0 || index >= _constantSize)
throw new IndexOutOfRangeException();
return index < _array.Length ? _array[index] : _padValue;
}
set { throw new NotSupportedException(); }
}
}
Ack.

The Array base class implements the Resize method
if(extractArray.Length < 183)
Array.Resize<string>(ref extractArray, 183);
However, keep in mind that resizing is problematic for performance, thus this method is useful only if you require the array for some reason. If you can switch to a List
And, I suppose you have an unidimensional array of strings here, so I use the Length property to check the effective number of items in the array.

Since you've stated that you need to ensure there's 183 indexes, and that you need to pad it if there is not, I would suggest using a List instead of an array. You can do something like:
while (extractList.Count < 183)
{
extractList.Add(" "); // just add a space
}
If you ABSOLUTELY have to go back to an array you can using something similar.

I can't say that I would recommend this solution, but I won't let that stop me from posting it! Whether they like to admit it or not, everyone likes linq solutions!
Using linq, given an array with X elements in it, you can generate an array with exactly Y (183 in your case) elements in it like this:
var items183exactly = extractArray.Length == 183 ? extractArray :
extractArray.Take(183)
.Concat(Enumerable.Repeat(string.Empty, Math.Max(0, 183 - extractArray.Length)))
.ToArray();
If there are fewer than 183 elements, the array will be padded with empty strings. If there are more than 183 elements, the array will be truncated. If there are exactly 183 elements, the array is used as is.
I don't claim that this is efficient or that it is necessarily a good idea. However, it does use linq (yippee!) and it is fun.

Is one of these for loops faster than the other?

for (var keyValue = 0; keyValue < dwhSessionDto.KeyValues.Count; keyValue++)
{...}
var count = dwhSessionDto.KeyValues.Count;
for (var keyValue = 0; keyValue < count; keyValue++)
{...}
I know there's a difference between the two, but is one of them faster than the other? I would think the second is faster.

Yes, the first version is much slower. After all, I'm assuming you're dealing with types like this:
public class SlowCountProvider
{
public int Count
{
get
{
Thread.Sleep(1000);
return 10;
}
}
}
public class KeyValuesWithSlowCountProvider
{
public SlowCountProvider KeyValues
{
get { return new SlowCountProvider(); }
}
}
Here, your first loop will take ~10 seconds, whereas your second loop will take ~1 second.
Of course, you might argue that the assumption that you're using this code is unjustified - but my point is that the right answer will depend on the types involved, and the question doesn't state what those types are.
Now if you're actually dealing with a type where accessing KeyValues and Count is cheap (which is quite likely) I wouldn't expect there to be much difference. Mind you, I'd almost always prefer to use foreach where possible:
foreach (var pair in dwhSessionDto.KeyValues)
{
// Use pair here
}
That way you never need the count. But then, you haven't said what you're trying to do inside the loop either. (Hint: to get more useful answers, provide more information.)

it depends how difficult it is to compute dwhSessionDto.KeyValues.Count if its just a pointer to an int then the speed of each version will be the same. However, if the Count value needs to be calculated, then it will be calculated every time, and therefore impede perfomance.
EDIT -- heres some code to demonstrate that the condition is always re-evaluated
public class Temp
{
public int Count { get; set; }
}
static void Main(string[] args)
{
var t = new Temp() {Count = 5};
for (int i = 0; i < t.Count; i++)
{
Console.WriteLine(i);
t.Count--;
}
Console.ReadLine();
}
The output is 0, 1, 2 - only !

See comments for reasons why this answer is wrong.
If there is a difference, it’s the other way round: Indeed, the first one might be faster. That’s because the compiler recognizes that you are iterating from 0 to the end of the array, and it can therefore elide bounds checks within the loop (i.e. when you access dwhSessionDTo.KeyValues[i]).
However, I believe the compiler only applies this optimization to arrays so there probably will be no difference here.

It is impossible to say without knowing the implementation of dwhSessionDto.KeyValues.Count and the loop body.
Assume a global variable bool foo = false; and then following implementations:
/* Loop body... */
{
if(foo) Thread.Sleep(1000);
}
/* ... */
public int Count
{
get
{
foo = !foo;
return 10;
}
}
/* ... */
Now, the first loop will perform approximately twice as fast as the second ;D
However, assuming non-moronic implementation, the second one is indeed more likely to be faster.

No. There is no performance difference between these two loops. With JIT and Code Optimization, it does not make any difference.

There is no difference but why you think that thereis difference , can you please post your findings?
if you see the implementation of insert item in Dictionary using reflector
private void Insert(TKey key, TValue value, bool add)
{
int freeList;
if (key == null)
{
ThrowHelper.ThrowArgumentNullException(ExceptionArgument.key);
}
if (this.buckets == null)
{
this.Initialize(0);
}
int num = this.comparer.GetHashCode(key) & 0x7fffffff;
int index = num % this.buckets.Length;
for (int i = this.buckets[index]; i >= 0; i = this.entries[i].next)
{
if ((this.entries[i].hashCode == num) && this.comparer.Equals(this.entries[i].key, key))
{
if (add)
{
ThrowHelper.ThrowArgumentException(ExceptionResource.Argument_AddingDuplicate);
}
this.entries[i].value = value;
this.version++;
return;
}
}
if (this.freeCount > 0)
{
freeList = this.freeList;
this.freeList = this.entries[freeList].next;
this.freeCount--;
}
else
{
if (this.count == this.entries.Length)
{
this.Resize();
index = num % this.buckets.Length;
}
freeList = this.count;
this.count++;
}
this.entries[freeList].hashCode = num;
this.entries[freeList].next = this.buckets[index];
this.entries[freeList].key = key;
this.entries[freeList].value = value;
this.buckets[index] = freeList;
this.version++;
}
Count is a internal member to this class which is incremented each item you insert an item into dictionary
so i beleive that there is no differenct at all.

The second version can be faster, sometimes. The point is that the condition is reevaluated after every iteration, so if e.g. the getter of "Count" actually counts the elements in an IEnumerable, or interogates a database /etc, this will slow things down.
So I'd say that if you dont affect the value of "Count" in the "for", the second version is safer.

Access Enumerator within a foreach loop?

I have a List class, and I would like to override GetEnumerator() to return my own Enumerator class. This Enumerator class would have two additional properties that would be updated as the Enumerator is used.
For simplicity (this isn't the exact business case), let's say those properties were CurrentIndex and RunningTotal.
I could manage these properties within the foreach loop manually, but I would rather encapsulate this functionality for reuse, and the Enumerator seems to be the right spot.
The problem: foreach hides all the Enumerator business, so is there a way to, within a foreach statement, access the current Enumerator so I can retrieve my properties? Or would I have to foreach, use a nasty old while loop, and manipulate the Enumerator myself?

Strictly speaking, I would say that if you want to do exactly what you're saying, then yes, you would need to call GetEnumerator and control the enumerator yourself with a while loop.
Without knowing too much about your business requirement, you might be able to take advantage of an iterator function, such as something like this:
public static IEnumerable<decimal> IgnoreSmallValues(List<decimal> list)
{
decimal runningTotal = 0M;
foreach (decimal value in list)
{
// if the value is less than 1% of the running total, then ignore it
if (runningTotal == 0M || value >= 0.01M * runningTotal)
{
runningTotal += value;
yield return value;
}
}
}
Then you can do this:
List<decimal> payments = new List<decimal>() {
123.45M,
234.56M,
.01M,
345.67M,
1.23M,
456.78M
};
foreach (decimal largePayment in IgnoreSmallValues(payments))
{
// handle the large payments so that I can divert all the small payments to my own bank account. Mwahaha!
}
Updated:
Ok, so here's a follow-up with what I've termed my "fishing hook" solution. Now, let me add a disclaimer that I can't really think of a good reason to do something this way, but your situation may differ.
The idea is that you simply create a "fishing hook" object (reference type) that you pass to your iterator function. The iterator function manipulates your fishing hook object, and since you still have a reference to it in your code outside, you have visibility into what's going on:
public class FishingHook
{
public int Index { get; set; }
public decimal RunningTotal { get; set; }
public Func<decimal, bool> Criteria { get; set; }
}
public static IEnumerable<decimal> FishingHookIteration(IEnumerable<decimal> list, FishingHook hook)
{
hook.Index = 0;
hook.RunningTotal = 0;
foreach(decimal value in list)
{
// the hook object may define a Criteria delegate that
// determines whether to skip the current value
if (hook.Criteria == null || hook.Criteria(value))
{
hook.RunningTotal += value;
yield return value;
hook.Index++;
}
}
}
You would utilize it like this:
List<decimal> payments = new List<decimal>() {
123.45M,
.01M,
345.67M,
234.56M,
1.23M,
456.78M
};
FishingHook hook = new FishingHook();
decimal min = 0;
hook.Criteria = x => x > min; // exclude any values that are less than/equal to the defined minimum
foreach (decimal value in FishingHookIteration(payments, hook))
{
// update the minimum
if (value > min) min = value;
Console.WriteLine("Index: {0}, Value: {1}, Running Total: {2}", hook.Index, value, hook.RunningTotal);
}
// Resultint output is:
//Index: 0, Value: 123.45, Running Total: 123.45
//Index: 1, Value: 345.67, Running Total: 469.12
//Index: 2, Value: 456.78, Running Total: 925.90
// we've skipped the values .01, 234.56, and 1.23
Essentially, the FishingHook object gives you some control over how the iterator executes. The impression I got from the question was that you needed some way to access the inner workings of the iterator so that you could manipulate how it iterates while you are in the middle of iterating, but if this is not the case, then this solution might be overkill for what you need.

With foreach you indeed can't get the enumerator - you could, however, have the enumerator return (yield) a tuple that includes that data; in fact, you could probably use LINQ to do it for you...
(I couldn't cleanly get the index using LINQ - can get the total and current value via Aggregate, though; so here's the tuple approach)
using System.Collections;
using System.Collections.Generic;
using System;
class MyTuple
{
public int Value {get;private set;}
public int Index { get; private set; }
public int RunningTotal { get; private set; }
public MyTuple(int value, int index, int runningTotal)
{
Value = value; Index = index; RunningTotal = runningTotal;
}
static IEnumerable<MyTuple> SomeMethod(IEnumerable<int> data)
{
int index = 0, total = 0;
foreach (int value in data)
{
yield return new MyTuple(value, index++,
total = total + value);
}
}
static void Main()
{
int[] data = { 1, 2, 3 };
foreach (var tuple in SomeMethod(data))
{
Console.WriteLine("{0}: {1} ; {2}", tuple.Index,
tuple.Value, tuple.RunningTotal);
}
}
}

You can also do something like this in a more Functional way, depending on your requirements. What you are asking can be though of as "zipping" together multiple sequences, and then iterating through them all at once. The three sequences for the example you gave would be:
The "value" sequence
The "index" sequence
The "Running Total" Sequence
The next step would be to specify each of these sequences seperately:
List<decimal> ValueList
var Indexes = Enumerable.Range(0, ValueList.Count)
The last one is more fun... the two methods I can think of are to either have a temporary variable used to sum up the sequence, or to recalculate the sum for each item. The second is obviously much less performant, I would rather use the temporary:
decimal Sum = 0;
var RunningTotals = ValueList.Select(v => Sum = Sum + v);
The last step would be to zip these all together. .Net 4 will have the Zip operator built in, in which case it will look like this:
var ZippedSequence = ValueList.Zip(Indexes, (value, index) => new {value, index}).Zip(RunningTotals, (temp, total) => new {temp.value, temp.index, total});
This obviously gets noisier the more things you try to zip together.
In the last link, there is source for implementing the Zip function yourself. It really is a simple little bit of code.