Develop Reference

How to truncate an array in place in C#

I mean is it really possible? MSDN says that arrays are fixed-size and the only way to resize is "copy-to-new-place". But maybe it is possible with unsafe/some magic with internal CLR structures, they all are written in C++ where we have a full memory control and can call realloc and so on.
I have no code provided for this question, because I don't even know if it can exist.
I'm not talking about Array.Resize methods and so on, because they obviosly do not have needed behaviour.
Assume that we have a standard x86 process with 2GB ram, and I have 1.9GB filled by single array. Then I want to release half of it. So I want to write something like:
MagicClass.ResizeArray(ref arr, n)
And do not get OutOfMemoryException. Array.Resize will try to allocate another gigabyte of RAM and will fail with 1.9+1 > 2GB OutOfMemory.

You can try Array.Resize():
int[] myArray = new int[] { 1, 2, 3, 4 };
int myNewSize = 1;
Array.Resize(ref myArray, myNewSize);
// Test: 1
Console.Write(myArray.Length);

realloc will attempt to do the inplace resize - but it reserves the right to copy the whole thing elsewhere and return a pointer that's completely different.
Pretty much the same outward behaviour is exposed by .NET's List<T> class - which you should be using anyway if you find yourself changing array sizes often. It hides the actual array reference from you so that the change is propagated throughout all of the references to the same list. As you remove items from the end, only the length of the list changes while the inner array stays the same - avoiding the copying.
It doesn't release the memory (you can always do that explicitly with Capacity = XXX, but that makes a new copy of the array), but then again, unless you're working with large arrays, neither does realloc - and if you're working with large arrays, yada, yada - we've been there :)
realloc doesn't really make sense in the kind of memory model .NET has anyway - the heap is continously collected and compacted over time. So if you're trying to use it to avoid the copies when just trimming an array, while also keeping memory usage low... don't bother. At the next heap compaction, the whole memory above your array is going to be moved to fill in the blanks. Even if it were possible to do the realloc, the only benefit you have over simply copying the array is that you would keep your array in the old-living heap - and that isn't necessarily what you want anyway.

Neither array type in BCL supports what you want. That being said - you can implement your own type that would support what you need. It can be backed by standard array, but would implement own Length and indexer properties, that would 'hide' portion of array from you.
public class MyTruncatableArray<T>
{
private T[] _array;
private int _length;
public MyTruncatableArray(int size)
{
_array = new T[size];
_length = size;
}
public T this[int index]
{
get
{
CheckIndex(index, _length);
return _array[index];
}
set
{
CheckIndex(index, _length);
_array[index] = value;
}
}
public int Length
{
get { return _length; }
set
{
CheckIndex(value);
_length = value;
}
}
private void CheckIndex(int index)
{
this.CheckIndex(index, _array.Length);
}
private void CheckIndex(int index, int maxValue)
{
if (index < 0 || index > maxValue)
{
throw new ArgumentException("New array length must be positive and lower or equal to original size");
}
}
}
It really depend what exactly do need. (E.g. do you need to truncate just so that you can easier use it from your code. Or is perf/GC/memory consumption a concern? If the latter is the case - did you perform any measurements that proves standard Array.Resize method unusable for your case?)

How to properly Clear a Queue containing structs?

I have declared a basic struct like this
private struct ValLine {
public string val;
public ulong linenum;
}
and declared a Queue like this
Queue<ValLine> check = new Queue<ValLine>();
Then in a using StreamReader setup where I'm reading through the lines of an input file using ReadLine in a while loop, among other things, I'm doing this to populate the Queue:
check.Enqueue(new ValLine { val = line, linenum = linenum });
("line" is a string containing the text of each line, "linenum" is just a counter that is initialized at 0 and is incremented each time through the loop.)
The purpose of the "check" Queue is that if a particular line meets some criteria, then I store that line in "check" along with the line number that it occurs on in the input file.
After I've finished reading through the input file, I use "check" for various things, but then when I'm finished using it I clear it out in the obvious manner:
check.Clear();
(Alternatively, in my final loop through "check" I could just use .Dequeue(), instead of foreach'ing it.)
But then I got to thinking - wait a minute, what about all those "new ValLine" I generated when populating the Queue in the first place??? Have I created a memory leak? I've pretty new to C#, so it's not coming clear to me how to deal with this - or even if it should be dealt with (perhaps .Clear() or .Dequeue() deals with the now obsoleted structs automatically?). I've spent over an hour with our dear friend Google, and just not finding any specific discussion of this kind of example in regard to the clearing of a collection of structs.
So... In C# do we need to deal with wiping out the individual structs before clearing the queue (or as we are dequeueing), or not? And if so, then what is the proper way to do this?
(Just in case it's relevant, I'm using .NET 4.5 in Visual Studio 2013.)
UPDATE: This is for future reference (you know, like if this page comes up in a Google search) in regard to proper coding. To make the struct immutable as per recommendation, this is what I've ended up with:
private struct ValLine {
private readonly string _val;
private readonly ulong _linenum;
public string val { get { return _val; } }
public ulong linenum { get { return _linenum; } }
public ValLine(string x, ulong n) { _val = x; _linenum = n; }
}
Corresponding to that change, the queue population line is now this:
check.Enqueue(new ValLine(line,linenum));
Also, though not strictly necessary, I did get rid of my foreach on the queue (and the check.Clear();, and changed it to this
while (check.Count > 0) {
ValLine ll = check.Dequeue();
writer.WriteLine("[{0}] {1}", ll.linenum, ll.val);
}
so that the queue is emptied out as the information is output.
UPDATE 2: Okay, yes, I'm still a C# newbie (less than a year). I learn a lot from the Internet, but of course, I'm often looking at examples from more than a year ago. I have changed my struct so now it looks like this:
private struct ValLine {
public string val { get; private set; }
public ulong linenum { get; private set; }
public ValLine(string x, ulong n): this()
{ this.val = x; this.linenum = n; }
}
Interestingly enough, I had actually tried exactly this off the top of my head before coming up with what's in the first update (above), but got a compile error (because I did not have the : this() with the constructor). Upon further suggestion, I checked further and found a recent example showing that : this() for making it work like I tried before, plugged that in, and - Wa La! - clean compile. I like the cleaner look of the code. What the private variables are called is irrelevant to me.

No, you won't have created a memory leak. Calling Clear or Dequeue will clear the memory appropriately - for example, if you had a List<T> then a clear operation might use:
for (int i = 0; i < capacity; i++)
{
array[i] = default(T);
}
I don't know offhand whether Queue<T> is implemented with a circular buffer built on an array, or a linked list - but either way, you'll be fine.
Having said that, I would strongly recommend against using mutable structs as you're doing here, along with mutable fields. While it's not causing the particular problem you're envisaging, they can behave in confusing ways.

how to avoid objects allocations? how to reuse objects instead of allocating them?

My financical software processes constantly almost the same objects. For example I have such data online:
HP 100 1
HP 100 2
HP 100.1 1
etc.
I've about 1000 updates every second.
Each update is stored in object - but i do not want to allocate these objects on the fly to improve latency.
I use objects only in short period of time - i recive them, apply and free. Once object is free it actually can be reused for another pack of data.
So I need some storage (likely ring-buffer) that allocates required number of objects once and them allow to "obtain" and "free" them. What is the best way to do that in c#?
Each object has id and i assign id's sequentially and free them sequentially too.
For example i receive id's 1 2 and 3, then I free 1, 2, 3. So any FIFO collection would work, but i'm looking for some library class that cover's required functionality.
I.e. I need FIFO collection that do not allocates objects, but reuse them and allows to reconfigure them.
upd
I've added my implementation of what I want. This is not tested code and probably has bugs.
Idea is simple. Writer should call Obtain Commit methods. Reader should call TryGet method. Reader and writer can access this structure from different threads:
public sealed class ArrayPool<T> where T : class
{
readonly T[] array;
private readonly uint MASK;
private volatile uint curWriteNum;
private volatile uint curReadNum;
public ArrayPool(uint length = 1024) // length must be power of 2
{
if (length <= 0) throw new ArgumentOutOfRangeException("length");
array = new T[length];
MASK = length - 1;
}
/// <summary>
/// TryGet() itself is not thread safe and should be called from one thread.
/// However TryGet() and Obtain/Commit can be called from different threads
/// </summary>
/// <returns></returns>
public T TryGet()
{
if (curReadNum == curWriteNum)
{
return null;
}
T result = array[curReadNum & MASK];
curReadNum++;
return result;
}
public T Obtain()
{
return array[curWriteNum & MASK];
}
public void Commit()
{
curWriteNum++;
}
}
Comments about my implementation are welcome and probably some library method can replace this simple class?

I don't think you should leap at this, as per my comments on the question - however, a simple approach would be something like:
public sealed class MicroPool<T> where T : class
{
readonly T[] array;
public MicroPool(int length = 10)
{
if (length <= 0) throw new ArgumentOutOfRangeException("length");
array = new T[length];
}
public T TryGet()
{
T item;
for (int i = 0; i < array.Length; i++)
{
if ((item = Interlocked.Exchange(ref array[i], null)) != null)
return item;
}
return null;
}
public void Recycle(T item)
{
if(item == null) return;
for (int i = 0; i < array.Length; i++)
{
if (Interlocked.CompareExchange(ref array[i], item, null) == null)
return;
}
using (item as IDisposable) { } // cleaup if needed
}
}

If the loads come in burst, you may be able to use the GC's latency modes to offset the overhead by delaying collects. This is not a silver bullet, but in some cases it can be very helpful.

I am not sure, if this is what you need, but you could always make a pool of objects that are going to be used. Initialize a List of the object type. Then when you need to use an object remove it from the list and add it back when you are done with it.
http://www.codeproject.com/Articles/20848/C-Object-Pooling Is a good start.
Hope I've helped even if a little :)

If you are just worried about the time taken for the GC to run, then don't be - it can't be beaten by anything you can do yourself.
However, if your objects' constructors do some work it might be quicker to cache them.
A fairly straightforward way to do this is to use a ConcurrentBag
Essentially what you do is to pre-populate it with a set of objects using ConcurrentBag.Add() (that is if you want - or you can start with it empty and let it grow).
Then when you need a new object you use ConcurrentBag.TryTake() to grab an object.
If TryTake() fails then you just create a new object and use that instead.
Regardless of whether you grabbed an object from the bag or created a new one, once you're done with it you just put that object back into the bag using ConcurrentBag.Add()
Generally your bag will get to a certain size but no larger (but you might want to instrument things just to check it).
In any case, I would always do some timings to see if changes like this actually make any difference. Unless the object constructors are doing a fair bit of work, chances are it won't make much difference.

C#, how to handle constant tables

It seems strange that the language apparently includes no suitable functionality.
I find myself with data that would best be expressed as a multi-dimensional array but it's utterly constant, there is no way anyone could want to change it without also changing the associated code. Faced with such stuff in Delphi the answer is obvious--a constant whose value is the table. However, C# doesn't seem to support anything like this.
Google shows many people griping about this, no good answers.
How do people handle this sort of situation?
(And don't say that constants don't belong in code--the last one I bumped into was all possible permutations of 4 items. Unless the very nature of spacetime changes this is set in stone.)
What happened?? There was an answer that came pretty close, I was asking about a detail and it vanished! Simply declaring an array sort of does the job--the only problem is that the array allocation is going to run every time. The one in front of me contains 96 values--how do I get it to initialize only once? Do I just have to accept scoping it far wider than it should be? (As it stands it's in one 3-line routine that's inside what amounts to an O(n^3) routine.)

ReadOnlyCollection

There's a page in in the C# FAQ about this specific thing.
They suggest using a static readonly array:
static readonly int[,] constIntArray = new int[,] { { 1, 2, 3 }, { 4, 5, 6 }};
However, be aware that this is only sort of constant - you can still reassign individual elements within the array. Also, this has to be specified on the class level since it's a static, but it will work fairly well.

You could use a readonly Hashtable. The only downside is that readonly does not prevent you from changing the value of a particular item in the Hashtable. So it is not truly const.
readonly Hashtable table = new Hashtable(){{1,"One"},{2,"Two"}};
Or an array
public readonly string[,] arry = new string[,]{{"1","2"},{"2","4"}};
Yes, you will need to declare the variable in the appropriate scope so it does not get initialized more than once.

Like they say, just add another layer of indirection. C# doesn't need to provide a specialized data structure as a language primitive, although one does, at times, wish there was a way to make any class immutable, but that's another discussion.
Now you didn't mention if you need to store different things in there. In fact you didn't mention anything other than multi-dimensional and no ability to change the values or the arrays. I don't even know if the access pattern (a single int,int,int indexer) is appropriate.
But in general, for a 3-dimensional jagged array, the following works (but it isn't pretty).
One caveat is the type you construct it with also needs to be immutable, but that's your problem. You can just create your own read-only wrapper.
public static readonly ReadOnlyThreeDimensions<int> MyGlobalThree
= new ReadOnlyThreeDimensions<int>(IntInitializer);
public class ReadOnlyThreeDimensions<T>
{
private T[][][] _arrayOfT;
public ReadOnlyThreeDimensions(Func<T[][][]> initializer)
{
_arrayOfT = initializer();
}
public ReadOnlyThreeDimensions(T[][][] arrayOfT)
{
_arrayOfT = arrayOfT;
}
public T this [int x, int y, int z]
{
get
{
return _arrayOfT[x][y][z];
}
}
}
And then you just need to provide some initializer method, or assign it in a static constructor.
public static int[][][] IntInitializer()
{
return xyz // something that constructs a [][][]
}

Enumerations, surely.

Well, I've taken the approach of the following, it's a little nasty to read but easy to edit.
public struct Something
{
public readonly int Number;
public readonly string Name;
public Something(int num, string name) { this.Number = num; this.Name = name; }
}
public readonly Something[] GlobalCollection = new Something[]
{
new Something(1, "One"),
new Something(2, "Two"),
};

How to initialize a List<T> to a given size (as opposed to capacity)?

.NET offers a generic list container whose performance is almost identical (see Performance of Arrays vs. Lists question). However they are quite different in initialization.
Arrays are very easy to initialize with a default value, and by definition they already have certain size:
string[] Ar = new string[10];
Which allows one to safely assign random items, say:
Ar[5]="hello";
with list things are more tricky. I can see two ways of doing the same initialization, neither of which is what you would call elegant:
List<string> L = new List<string>(10);
for (int i=0;i<10;i++) L.Add(null);
or
string[] Ar = new string[10];
List<string> L = new List<string>(Ar);
What would be a cleaner way?
EDIT: The answers so far refer to capacity, which is something else than pre-populating a list. For example, on a list just created with a capacity of 10, one cannot do L[2]="somevalue"
EDIT 2: People wonder why I want to use lists this way, as it is not the way they are intended to be used. I can see two reasons:
One could quite convincingly argue that lists are the "next generation" arrays, adding flexibility with almost no penalty. Therefore one should use them by default. I'm pointing out they might not be as easy to initialize.
What I'm currently writing is a base class offering default functionality as part of a bigger framework. In the default functionality I offer, the size of the List is known in advanced and therefore I could have used an array. However, I want to offer any base class the chance to dynamically extend it and therefore I opt for a list.

List<string> L = new List<string> ( new string[10] );

I can't say I need this very often - could you give more details as to why you want this? I'd probably put it as a static method in a helper class:
public static class Lists
{
public static List<T> RepeatedDefault<T>(int count)
{
return Repeated(default(T), count);
}
public static List<T> Repeated<T>(T value, int count)
{
List<T> ret = new List<T>(count);
ret.AddRange(Enumerable.Repeat(value, count));
return ret;
}
}
You could use Enumerable.Repeat(default(T), count).ToList() but that would be inefficient due to buffer resizing.
Note that if T is a reference type, it will store count copies of the reference passed for the value parameter - so they will all refer to the same object. That may or may not be what you want, depending on your use case.
EDIT: As noted in comments, you could make Repeated use a loop to populate the list if you wanted to. That would be slightly faster too. Personally I find the code using Repeat more descriptive, and suspect that in the real world the performance difference would be irrelevant, but your mileage may vary.

Use the constructor which takes an int ("capacity") as an argument:
List<string> = new List<string>(10);
EDIT: I should add that I agree with Frederik. You are using the List in a way that goes against the entire reasoning behind using it in the first place.
EDIT2:
EDIT 2: What I'm currently writing is a base class offering default functionality as part of a bigger framework. In the default functionality I offer, the size of the List is known in advanced and therefore I could have used an array. However, I want to offer any base class the chance to dynamically extend it and therefore I opt for a list.
Why would anyone need to know the size of a List with all null values? If there are no real values in the list, I would expect the length to be 0. Anyhow, the fact that this is cludgy demonstrates that it is going against the intended use of the class.

Create an array with the number of items you want first and then convert the array in to a List.
int[] fakeArray = new int[10];
List<int> list = fakeArray.ToList();

If you want to initialize the list with N elements of some fixed value:
public List<T> InitList<T>(int count, T initValue)
{
return Enumerable.Repeat(initValue, count).ToList();
}

Why are you using a List if you want to initialize it with a fixed value ?
I can understand that -for the sake of performance- you want to give it an initial capacity, but isn't one of the advantages of a list over a regular array that it can grow when needed ?
When you do this:
List<int> = new List<int>(100);
You create a list whose capacity is 100 integers. This means that your List won't need to 'grow' until you add the 101th item.
The underlying array of the list will be initialized with a length of 100.

This is an old question, but I have two solutions. One is fast and dirty reflection; the other is a solution that actually answers the question (set the size not the capacity) while still being performant, which none of the answers here do.
Reflection
This is quick and dirty, and should be pretty obvious what the code does. If you want to speed it up, cache the result of GetField, or create a DynamicMethod to do it:
public static void SetSize<T>(this List<T> l, int newSize) =>
l.GetType().GetField("_size", BindingFlags.NonPublic | BindingFlags.Instance).SetValue(l, newSize);
Obviously a lot of people will be hesitant to put such code into production.
ICollection<T>
This solution is based around the fact that the constructor List(IEnumerable<T> collection) optimizes for ICollection<T> and immediately adjusts the size to the correct amount, without iterating it. It then calls the collections CopyTo to do the copy.
The code for the List<T> constructor is as follows:
public List(IEnumerable<T> collection) {
....
ICollection<T> c = collection as ICollection<T>;
if (collection is ICollection<T> c)
{
int count = c.Count;
if (count == 0)
{
_items = s_emptyArray;
}
else {
_items = new T[count];
c.CopyTo(_items, 0);
_size = count;
}
}
So we can completely optimally pre-initialize the List to the correct size, without any extra copying.
How so? By creating an ICollection<T> object that does nothing other than return a Count. Specifically, we will not implement anything in CopyTo which is the only other function called.
private struct SizeCollection<T> : ICollection<T>
{
public SizeCollection(int size) =>
Count = size;
public void Add(T i){}
public void Clear(){}
public bool Contains(T i)=>true;
public void CopyTo(T[]a, int i){}
public bool Remove(T i)=>true;
public int Count {get;}
public bool IsReadOnly=>true;
public IEnumerator<T> GetEnumerator()=>null;
IEnumerator IEnumerable.GetEnumerator()=>null;
}
public List<T> InitializedList<T>(int size) =>
new List<T>(new SizeCollection<T>(size));
We could in theory do the same thing for AddRange/InsertRange for an existing array, which also accounts for ICollection<T>, but the code there creates a new array for the supposed items, then copies them in. In such case, it would be faster to just empty-loop Add:
public void SetSize<T>(this List<T> l, int size)
{
if(size < l.Count)
l.RemoveRange(size, l.Count - size);
else
for(size -= l.Count; size > 0; size--)
l.Add(default(T));
}

Initializing the contents of a list like that isn't really what lists are for. Lists are designed to hold objects. If you want to map particular numbers to particular objects, consider using a key-value pair structure like a hash table or dictionary instead of a list.

You seem to be emphasizing the need for a positional association with your data, so wouldn't an associative array be more fitting?
Dictionary<int, string> foo = new Dictionary<int, string>();
foo[2] = "string";

The accepted answer (the one with the green check mark) has an issue.
The problem:
var result = Lists.Repeated(new MyType(), sizeOfList);
// each item in the list references the same MyType() object
// if you edit item 1 in the list, you are also editing item 2 in the list
I recommend changing the line above to perform a copy of the object. There are many different articles about that:
String.MemberwiseClone() method called through reflection doesn't work, why?
https://code.msdn.microsoft.com/windowsdesktop/CSDeepCloneObject-8a53311e
If you want to initialize every item in your list with the default constructor, rather than NULL, then add the following method:
public static List<T> RepeatedDefaultInstance<T>(int count)
{
List<T> ret = new List<T>(count);
for (var i = 0; i < count; i++)
{
ret.Add((T)Activator.CreateInstance(typeof(T)));
}
return ret;
}

You can use Linq to cleverly initialize your list with a default value. (Similar to David B's answer.)
var defaultStrings = (new int[10]).Select(x => "my value").ToList();
Go one step farther and initialize each string with distinct values "string 1", "string 2", "string 3", etc:
int x = 1;
var numberedStrings = (new int[10]).Select(x => "string " + x++).ToList();

string [] temp = new string[] {"1","2","3"};
List<string> temp2 = temp.ToList();

After thinking again, I had found the non-reflection answer to the OP question, but Charlieface beat me to it. So I believe that the correct and complete answer is https://stackoverflow.com/a/65766955/4572240
My old answer:
If I understand correctly, you want the List<T> version of new T[size], without the overhead of adding values to it.
If you are not afraid the implementation of List<T> will change dramatically in the future (and in this case I believe the probability is close to 0), you can use reflection:
public static List<T> NewOfSize<T>(int size) {
var list = new List<T>(size);
var sizeField = list.GetType().GetField("_size",BindingFlags.Instance|BindingFlags.NonPublic);
sizeField.SetValue(list, size);
return list;
}
Note that this takes into account the default functionality of the underlying array to prefill with the default value of the item type. All int arrays will have values of 0 and all reference type arrays will have values of null. Also note that for a list of reference types, only the space for the pointer to each item is created.
If you, for some reason, decide on not using reflection, I would have liked to offer an option of AddRange with a generator method, but underneath List<T> just calls Insert a zillion times, which doesn't serve.
I would also like to point out that the Array class has a static method called ResizeArray, if you want to go the other way around and start from Array.
To end, I really hate when I ask a question and everybody points out that it's the wrong question. Maybe it is, and thanks for the info, but I would still like an answer, because you have no idea why I am asking it. That being said, if you want to create a framework that has an optimal use of resources, List<T> is a pretty inefficient class for anything than holding and adding stuff to the end of a collection.

A notice about IList:
MSDN IList Remarks:
"IList implementations fall into three categories: read-only, fixed-size, and variable-size. (...). For the generic version of this interface, see
System.Collections.Generic.IList<T>."
IList<T> does NOT inherits from IList (but List<T> does implement both IList<T> and IList), but is always variable-size.
Since .NET 4.5, we have also IReadOnlyList<T> but AFAIK, there is no fixed-size generic List which would be what you are looking for.

This is a sample I used for my unit test. I created a list of class object. Then I used forloop to add 'X' number of objects that I am expecting from the service.
This way you can add/initialize a List for any given size.
public void TestMethod1()
{
var expected = new List<DotaViewer.Interface.DotaHero>();
for (int i = 0; i < 22; i++)//You add empty initialization here
{
var temp = new DotaViewer.Interface.DotaHero();
expected.Add(temp);
}
var nw = new DotaHeroCsvService();
var items = nw.GetHero();
CollectionAssert.AreEqual(expected,items);
}
Hope I was of help to you guys.

A bit late but first solution you proposed seems far cleaner to me : you dont allocate memory twice.
Even List constrcutor needs to loop through array in order to copy it; it doesn't even know by advance there is only null elements inside.
1.
- allocate N
- loop N
Cost: 1 * allocate(N) + N * loop_iteration
2.
- allocate N
- allocate N + loop ()
Cost : 2 * allocate(N) + N * loop_iteration
However List's allocation an loops might be faster since List is a built-in class, but C# is jit-compiled sooo...