Related
I'm trying to vectorize a n-dimensional array as 1-dimensional array in C# to later ease working using linear indexing (this whatever the type of the elements).
So far I was using Buffer.BlockCopy to do that (and even reshaping from n-dimensions to m-dimensions as long as the number of elements was not changing) but unfortunately I came across having to reshape arrays whose elements are not primitive types (double, single, int) and in this case Buffer.BlockCopy does not work (example array of string or whatever other non primitive type).
Currently the solution I have is to make special-case for non-primitive types:
/// <summary>Vectorize ND-array</summary>
/// <param name="arrayNd">ND-Array to vectorize.</param>
/// <returns>Surface copy as 1D array.</returns>
public static Array Vectorize(Array arrayNd)
{
// Check arguments
if (arrayNd == null) { return null; }
var elementCount = arrayNd.Length;
// Create 1D array
var tarray = arrayNd.GetType();
var telem = tarray.GetElementType();
var array1D = Array.CreateInstance(telem, elementCount);
// Surface copy
if (telem.IsPrimitive)
{
// Block copy only works for array whose elements are primitive types (double, single, ...)
var numberOfBytes = Buffer.ByteLength(arrayNd);
Buffer.BlockCopy(arrayNd, 0, array1D, 0, numberOfBytes);
}
else
{
// Slow version for other element types
// NB: arrayNd.GetValue(...) does not support linear indexing so need to compute indices for each dimension (very slow !!)
var indices = new int[arrayNd.Rank];
for (var i = 0; i < elementCount; i++)
{
var idx = i;
for (var d = arrayNd.Rank - 1; d >= 0; d--)
{
var l = arrayNd.GetLength(d);
indices[d] = idx % l;
idx /= l;
}
array1D.SetValue(arrayNd.GetValue(indices), i);
}
}
// Return as 1D
return array1D;
}
So this works now all types:
var double1D = Vectorize(new double[3, 2, 5]); // Fast BlockCopy
var string1D = Vectorize(new string[3, 2, 5]); // Slow solution
I already have an NEnumerator class of my own to speed up computing indices (instead of using modulo as above) but maybe there is really fast way for just making this sort of "surface memcpy" ?
NB1: I'd like to avoid unsafe code but if it's the only way ...
NB2: I really want to work with System.Array (eventually I'll later do a bunch of T[] Vectorize(T[,,,,] array) overloads but that's not the issue)
In my experience, Multidimensional arrays are kind of a pain to work with, in large part since it is so difficult to access the backing data. As far as I know there is no direct way to just copy all the elements for arbitrary types.
Because of this I tend to prefer a custom type for my 2D types that uses a linear array as backing storage, and index like myArray[y * width + x]. With this model the whole exercise becomes a no-op, and you can get a pointer to pass to native code, it works better with serialization etc.
For 3D/4D arrays you could use the same mode, but it seems like the best option for performance is allocate slices independently, i.e. myArray[z][y * width + x], at least for large arrays. I have not worked with 4D arrays, but in general, I would avoid multidimensional arrays if performance is a concern. There might also be libraries out there that might suit your needs, but I'm not aware of any specific one.
However, looking at your code I would expect there to be some possible improvements. You are currently doing N calls to GetLength, modulus & divisions for each element. So I would expect something like this to be a bit faster:
public static Array MultidimensionalToLinear(Array arr)
{
var rank = arr.Rank;
var lengths = new int[rank];
for (int i = 0; i < rank; i++)
{
lengths[i] = arr.GetLength(i);
}
var linearLength = arr.Length;
var result = Array.CreateInstance(arr.GetType().GetElementType(), linearLength);
var index = new int[rank];
var linearIndex = 0;
CopyRecursive(0, index, result, ref linearIndex);
void CopyRecursive(int rank, int[] index, Array result, ref int linearIndex)
{
var lastIndex = index.Length - 1;
if (rank == lastIndex)
{
for (int i = 0; i < lengths[lastIndex]; i++)
{
index[lastIndex] = i;
result.SetValue(arr.GetValue(index), linearIndex);
linearIndex++;
}
}
else
{
for (int i = 0; i < lengths[rank]; i++)
{
index[rank] = i;
CopyRecursive(rank +1, index, result, ref linearIndex);
}
}
}
return result;
}
However, when measuring it seem like the performance improvement is fairly small. Probably due the code in GetValue dominating the runtime.
I am implementing a library where I am extensively using the .Net BitArray class and need an equivalent to the Java BitSet.Cardinality() method, i.e. a method which returns the number of bits set. I was thinking of implementing it as an extension method for the BitArray class. The trivial implementation is to iterate and count the bits set (like below), but I wanted a faster implementation as I would be performing thousands of set operations and counting the answer. Is there a faster way than the example below?
count = 0;
for (int i = 0; i < mybitarray.Length; i++)
{
if (mybitarray [i])
count++;
}
This is my solution based on the "best bit counting method" from http://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel
public static Int32 GetCardinality(BitArray bitArray)
{
Int32[] ints = new Int32[(bitArray.Count >> 5) + 1];
bitArray.CopyTo(ints, 0);
Int32 count = 0;
// fix for not truncated bits in last integer that may have been set to true with SetAll()
ints[ints.Length - 1] &= ~(-1 << (bitArray.Count % 32));
for (Int32 i = 0; i < ints.Length; i++)
{
Int32 c = ints[i];
// magic (http://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel)
unchecked
{
c = c - ((c >> 1) & 0x55555555);
c = (c & 0x33333333) + ((c >> 2) & 0x33333333);
c = ((c + (c >> 4) & 0xF0F0F0F) * 0x1010101) >> 24;
}
count += c;
}
return count;
}
According to my tests, this is around 60 times faster than the simple foreach loop and still 30 times faster than the Kernighan approach with around 50% bits set to true in a BitArray with 1000 bits. I also have a VB version of this if needed.
you can accomplish this pretty easily with Linq
BitArray ba = new BitArray(new[] { true, false, true, false, false });
var numOnes = (from bool m in ba
where m
select m).Count();
BitArray myBitArray = new BitArray(...
int
bits = myBitArray.Count,
size = ((bits - 1) >> 3) + 1,
counter = 0,
x,
c;
byte[] buffer = new byte[size];
myBitArray.CopyTo(buffer, 0);
for (x = 0; x < size; x++)
for (c = 0; buffer[x] > 0; buffer[x] >>= 1)
counter += buffer[x] & 1;
Taken from "Counting bits set, Brian Kernighan's way" and adapted for bytes. I'm using it for bit arrays of 1 000 000+ bits and it's superb.
If your bits are not n*8 then you can count the mod byte manually.
I had the same issue, but had more than just the one Cardinality method to convert. So, I opted to port the entire BitSet class. Fortunately it was self-contained.
Here is the Gist of the C# port.
I would appreciate if people would report any bugs that are found - I am not a Java developer, and have limited experience with bit logic, so I might have translated some of it incorrectly.
Faster and simpler version than the accepted answer thanks to the use of System.Numerics.BitOperations.PopCount
C#
Int32[] ints = new Int32[(bitArray.Count >> 5) + 1];
bitArray.CopyTo(ints, 0);
Int32 count = 0;
for (Int32 i = 0; i < ints.Length; i++) {
count += BitOperations.PopCount(ints[i]);
}
Console.WriteLine(count);
F#
let ints = Array.create ((bitArray.Count >>> 5) + 1) 0u
bitArray.CopyTo(ints, 0)
ints
|> Array.sumBy BitOperations.PopCount
|> printfn "%d"
See more details in Is BitOperations.PopCount the best way to compute the BitArray cardinality in .NET?
You could use Linq, but it would be useless and slower:
var sum = mybitarray.OfType<bool>().Count(p => p);
There is no faster way with using BitArray - What it comes down to is you will have to count them - you could use LINQ to do that or do your own loop, but there is no method offered by BitArray and the underlying data structure is an int[] array (as seen with Reflector) - so this will always be O(n), n being the number of bits in the array.
The only way I could think of making it faster is using reflection to get a hold of the underlying m_array field, then you can get around the boundary checks that Get() uses on every call (see below) - but this is kinda dirty, and might only be worth it on very large arrays since reflection is expensive.
public bool Get(int index)
{
if ((index < 0) || (index >= this.Length))
{
throw new ArgumentOutOfRangeException("index", Environment.GetResourceString("ArgumentOutOfRange_Index"));
}
return ((this.m_array[index / 0x20] & (((int) 1) << (index % 0x20))) != 0);
}
If this optimization is really important to you, you should create your own class for bit manipulation, that internally could use BitArray, but keeps track of the number of bits set and offers the appropriate methods (mostly delegate to BitArray but add methods to get number of bits currently set) - then of course this would be O(1).
If you really want to maximize the speed, you could pre-compute a lookup table where given a byte-value you have the cardinality, but BitArray is not the most ideal structure for this, since you'd need to use reflection to pull the underlying storage out of it and operate on the integral types - see this question for a better explanation of that technique.
Another, perhaps more useful technique, is to use something like the Kernighan trick, which is O(m) for an n-bit value of cardinality m.
static readonly ZERO = new BitArray (0);
static readonly NOT_ONE = new BitArray (1).Not ();
public static int GetCardinality (this BitArray bits)
{
int c = 0;
var tmp = new BitArray (myBitArray);
for (c; tmp != ZERO; c++)
tmp = tmp.And (tmp.And (NOT_ONE));
return c;
}
This too is a bit more cumbersome than it would be in say C, because there are no operations defined between integer types and BitArrays, (tmp &= tmp - 1, for example, to clear the least significant set bit, has been translated to tmp &= (tmp & ~0x1).
I have no idea if this ends up being any faster than naively iterating for the case of the BCL BitArray, but algorithmically speaking it should be superior.
EDIT: cited where I discovered the Kernighan trick, with a more in-depth explanation
If you don't mind to copy the code of System.Collections.BitArray to your project and Edit it,you can write as fellow:
(I think it's the fastest. And I've tried use BitVector32[] to implement my BitArray, but it's still so slow.)
public void Set(int index, bool value)
{
if ((index < 0) || (index >= this.m_length))
{
throw new ArgumentOutOfRangeException("index", "Index Out Of Range");
}
SetWithOutAuth(index,value);
}
//When in batch setting values,we need one method that won't auth the index range
private void SetWithOutAuth(int index, bool value)
{
int v = ((int)1) << (index % 0x20);
index = index / 0x20;
bool NotSet = (this.m_array[index] & v) == 0;
if (value && NotSet)
{
CountOfTrue++;//Count the True values
this.m_array[index] |= v;
}
else if (!value && !NotSet)
{
CountOfTrue--;//Count the True values
this.m_array[index] &= ~v;
}
else
return;
this._version++;
}
public int CountOfTrue { get; internal set; }
public void BatchSet(int start, int length, bool value)
{
if (start < 0 || start >= this.m_length || length <= 0)
return;
for (int i = start; i < length && i < this.m_length; i++)
{
SetWithOutAuth(i,value);
}
}
I wrote my version of after not finding one that uses a look-up table:
private int[] _bitCountLookup;
private void InitLookupTable()
{
_bitCountLookup = new int[256];
for (var byteValue = 0; byteValue < 256; byteValue++)
{
var count = 0;
for (var bitIndex = 0; bitIndex < 8; bitIndex++)
{
count += (byteValue >> bitIndex) & 1;
}
_bitCountLookup[byteValue] = count;
}
}
private int CountSetBits(BitArray bitArray)
{
var result = 0;
var numberOfFullBytes = bitArray.Length / 8;
var numberOfTailBits = bitArray.Length % 8;
var tailByte = numberOfTailBits > 0 ? 1 : 0;
var bitArrayInBytes = new byte[numberOfFullBytes + tailByte];
bitArray.CopyTo(bitArrayInBytes, 0);
for (var i = 0; i < numberOfFullBytes; i++)
{
result += _bitCountLookup[bitArrayInBytes[i]];
}
for (var i = (numberOfFullBytes * 8); i < bitArray.Length; i++)
{
if (bitArray[i])
{
result++;
}
}
return result;
}
The problem is naturally O(n), as a result your solution is probably the most efficient.
Since you are trying to count an arbitrary subset of bits you cannot count the bits when they are set (would would provide a speed boost if you are not setting the bits too often).
You could check to see if the processor you are using has a command which will return the number of set bits. For example a processor with SSE4 could use the POPCNT according to this post. This would probably not work for you since .Net does not allow assembly (because it is platform independent). Also, ARM processors probably do not have an equivalent.
Probably the best solution would be a look up table (or switch if you could guarantee the switch will compiled to a single jump to currentLocation + byteValue). This would give you the count for the whole byte. Of course BitArray does not give access to the underlying data type so you would have to make your own BitArray. You would also have to guarantee that all the bits in the byte will always be part of the intersection which does not sound likely.
Another option would be to use an array of booleans instead of a BitArray. This has the advantage not needing to extract the bit from the others in the byte. The disadvantage is the array will take up 8x as much space in memory meaning not only wasted space, but also more data push as you iterate through the array to perform your count.
The difference between a standard array look up and a BitArray look up is as follows:
Array:
offset = index * indexSize
Get memory at location + offset and save to value
BitArray:
index = index/indexSize
offset = index * indexSize
Get memory at location + offset and save to value
position = index%indexSize
Shift value position bits
value = value and 1
With the exception of #2 for Arrays and #3 most of these commands take 1 processor cycle to complete. Some of the commands can be combined into 1 command using x86/x64 processors, though probably not with ARM since it uses a reduced set of instructions.
Which of the two (array or BitArray) perform better will be specific to your platform (processor speed, processor instructions, processor cache sizes, processor cache speed, amount of system memory (Ram), speed of system memory (CAS), speed of connection between processor and RAM) as well as the spread of indexes you want to count (are the intersections most often clustered or are they randomly distributed).
To summarize: you could probably find a way to make it faster, but your solution is the fastest you will get for your data set using a bit per boolean model in .NET.
Edit: make sure you are accessing the indexes you want to count in order. If you access indexes 200, 5, 150, 151, 311, 6 in that order then you will increase the amount of cache misses resulting in more time spent waiting for values to be retrieved from RAM.
I'm trying to initialize an array in three dimension to load a voxel world.
The total size of the map should be (2048/1024/2048). I tried to initialize an jagged array of "int" but I throw a memory exception. What is the size limit?
Size of my table: 2048 * 1024 * 2048 = 4'191'893'824
Anyone know there a way around this problem?
// System.OutOfMemoryException here !
int[][][] matrice = CreateJaggedArray<int[][][]>(2048,1024,2048);
// if i try normal Initialization I also throws the exception
int[, ,] matrice = new int[2048,1024,2048];
static T CreateJaggedArray<T>(params int[] lengths)
{
return (T)InitializeJaggedArray(typeof(T).GetElementType(), 0, lengths);
}
static object InitializeJaggedArray(Type type, int index, int[] lengths)
{
Array array = Array.CreateInstance(type, lengths[index]);
Type elementType = type.GetElementType();
if (elementType != null)
{
for (int i = 0; i < lengths[index]; i++)
{
array.SetValue(
InitializeJaggedArray(elementType, index + 1, lengths), i);
}
}
return array;
}
The maximum size of a single object in C# is 2GB. Since you are creating a multi-dimensional array rather than a jagged array (despite the name of your method) it is a single object that needs to contain all of those items, not several. If you actually used a jagged array then you wouldn't have a single item with all of that data (even though the total memory footprint would be a tad larger, not smaller, it's just spread out more).
Thank you so much to all the staff who tried to help me in understanding and solving my problem.
I tried several solution to be able to load a lot of data and stored in a table.
After two days, here are my tests and finally the solution which can store 4'191'893'824 entry into one array
I add my final solution, hoping someone could help
the goal
I recall the goal: Initialize an integer array [2048/1024/2048] for storing 4'191'893'824 data
Test 1: with JaggedArray method (failure)
system out of memory exception thrown
/* ******************** */
/* Jagged Array method */
/* ******************** */
// allocate the first dimension;
bigData = new int[2048][][];
for (int x = 0; x < 2048; x++)
{
// allocate the second dimension;
bigData[x] = new int[1024][];
for (int y = 0; y < 1024; y++)
{
// the last dimension allocation
bigData[x][y] = new int[2048];
}
}
Test 2: with List method (failure)
system out of memory exception thrown (divide the big array into several small array .. Does not work because "List <>" allows a maximum of "2GB" Ram allocution like a simple array unfortunately.)
/* ******************** */
/* List method */
/* ******************** */
List<int[,,]> bigData = new List<int[,,]>(512);
for (int a = 0; a < 512; a++)
{
bigData.Add(new int[256, 128, 256]);
}
Test 3: with MemoryMappedFile (Solution)
I finally finally found the solution!
Use the class "Memory Mapped File" contains the contents of a file in virtual memory.
MemoryMappedFile MSDN
Use with custom class that I found on codeproject here. The initialization is long but it works well!
/* ************************ */
/* MemoryMappedFile method */
/* ************************ */
string path = AppDomain.CurrentDomain.BaseDirectory;
var myList = new GenericMemoryMappedArray<int>(2048L*1024L*2048L, path);
using (myList)
{
myList.AutoGrow = false;
/*
for (int a = 0; a < (2048L * 1024L * 2048L); a++)
{
myList[a] = a;
}
*/
myList[12456] = 8;
myList[1939848234] = 1;
// etc...
}
From the MSDN documentation on Arrays (emphasis added)
By default, the maximum size of an Array is 2 gigabytes (GB). In a
64-bit environment, you can avoid the size restriction by setting the
enabled attribute of the gcAllowVeryLargeObjects configuration element
to true in the run-time environment. However, the array will still be
limited to a total of 4 billion elements, and to a maximum index of
0X7FEFFFFF in any given dimension (0X7FFFFFC7 for byte arrays and
arrays of single-byte structures).
So despite the above answers, even if you set the flag to allow a larger object size, the array is still limited to the 32bit limit of the number of elements.
EDIT: You'll likely have to redesign to eliminate the need for a multidimensional array as you're currently using it (as others have suggested, there are a few ways to do this between using actual jagged arrays, or some other collection of dimensions). Given the scale of the number of elements, it may be best to use a design that dynamically allocates objects/memory as used instead of arrays that have to pre-allocate it. (unless you don't mind using many gigabytes of memory) EDITx2: That is, perhaps you can define data structures that define filled content rather than defining every possible voxel in the world, even the "empty" ones. (I'm assuming the vast majority of voxels are "empty" rather than "filled")
EDIT: Although not trivial, especially if most of the space is considered "empty", then your best bet would be to introduce some sort of spatial tree that will let you efficiently query your world to see what objects are in a particular area. For example: Octrees (as Eric suggested) or RTrees
Creating this object as described, either as a standard array or as a jagged array, is going to destroy the locality of reference that allows your CPU to be performant. I recommend you use a structure like this instead:
class BigArray
{
ArrayCell[,,] arrayCell = new ArrayCell[32,16,32];
public int this[int i, int j, int k]
{
get { return (arrayCell[i/64, j/64, k/64])[i%64, j%64, k%16]; }
}
}
class ArrayCell
{
int[,,] cell = new int[64,64,64];
public int this[int i, int j, int k]
{
get { return cell[i,j,k]; }
}
}
I'm porting some C Code to C#. I'm stuck at a piece where I don't quite understand the Author's intention of writing code in unfamiliar fashion.
The Code is:
typedef struct{
Int32 window[2][8];
Int32 windowF[2][8];
short Index;
}BLOCK_SWITCHING_CONTROL;
maxWindow = SrchMaxWithIndex( &blockSwitchingControl->window[0][8-1],
&blockSwitchingControl->Index, 8);
*****************************************************************************
*
* function name: SrchMaxWithIndex
* description: search for the biggest value in an array
* returns: the max value
*
**********************************************************************************/
static Int32 SrchMaxWithIndex(const Int32 in[], Int16 *index, Int16 n)
{
Int32 max;
Int32 i, idx;
/* Search maximum value in array and return index and value */
max = 0;
idx = 0;
for (i = 0; i < n; i++) {
if (in[i+1] > max) {
max = in[i+1];
idx = i;
}
}
*index = idx;
return(max);
}
As you can see, when SrchMaxWithIndex is being called, not an array but a single Int32 is being passed as its first parameter which is of course wrong. But because I know for sure the C code has nothing wrong with it, I'm convinced that I'm missing something here.
What am I missing? What was the Author's intention to pass a single Int32 instead of an array?
So far I've ported the above to C# in the following manner:
static class BLOCK_SWITCHING_CONTROL{
Int32[][] window = new int[2]{new int[8], new int[8]};
Int32[][] windowF = new int[2]{new int[8], new int[8]};
short Index;
};
maxWindow = SrchMaxWithIndex( blockSwitchingControl.window[0]/*[8-1]*/,
out blockSwitchingControl.Index);
*****************************************************************************
*
* function name: SrchMaxWithIndex
* description: search for the biggest value in an array
* returns: the max value
*
**********************************************************************************/
static Int32 SrchMaxWithIndex(Int32 _in[], out Int16 index)
{
Int32 max;
Int32 i, idx;
/* Search maximum value in array and return index and value */
max = 0;
idx = 0;
for (i = 0; i < _in.Length; i++) {
if (in[i+1] > max) {
max = in[i+1];
idx = i;
}
}
index = idx;
return(max);
}
But it is just to remove the errors in C#.
The C code is not passing a single integer. It's passing the address of an integer, using the & prefix operator.
There does seem to be some kind of typo though, since the C code references a windowN member in the struct which does not seem to exist.
Assuming it means windowF, this code:
maxWindow = SrchMaxWithIndex(&blockSwitchingControl->windowF[0][8-1],
&blockSwitchingControl->Index, 8);
Tells the called function to treat the given address as an array of 8 integers. I think this will overflow into windowF[1][]. It's very scary code, but if it were as wrong as you state, it wouldn't compile. You can't in general pass an integer to a function expecting a pointer, in C.
OK, first, I have to assume you have a mistake in the code:
blockSwitchingControl->windowN
Does not exist in the BLOCK_SWITCHING_CONTROL structure. So I'll answer this assuming you ment something like windowF
Now to your question, the author did pass an array... sort of.
Presumably blockSwitchingControl is an structure of type BLOCK_SWITCHING_CONTROL. What we're doing here:
&blockSwitchingControl->windowF[0][8-1]
is passing the address of windowF's [0][7]'th element. A multi-dimensional array is linear (continuous) memory so Int32 windowF[2][8] is properly sized for 16 Int32s to be stored in a "row" in memory. Something like:
windowF[2][8] => [0][1][2][3][4][5][6][7][8][9][A][B][C][D][E][F][10]
Thus, if I were to pass the address of windowF's [0][7] element, I'm really passing part of the array:
windowF[0][7] [0][1][2][3][4][5][6][7][8][9][A][B][C][D][E][F][10] //full array
-----------------------------^
So now inside the SrchMaxWithIndex() I have _in[] = an array of Int32's which is equivalent to part of windowF's array. So you can see they're passing an "array's worth of values", even if it's not how you'd expect.
The address of an element of an array is being passed (note the & in &blockSwitchingControl->windowN[0][8-1]).
So in[i+1] will be equivalent to blockSwitchingControl->windowN[0][8], which is, presumably, a valid item in the array.
This question already has answers here:
What is the equivalent of memset in C#?
(17 answers)
Closed 8 years ago.
I'm busy rewriting an old project that was done in C++, to C#.
My task is to rewrite the program so that it functions as close to the original as possible.
During a bunch of file-handling the previous developer who wrote this program creates a structure containing a ton of fields that correspond to the set format that a file has to be written in, so all that work is already done for me.
These fields are all byte arrays. What the C++ code then does is use memset to set this entire structure to all spaces characters (0x20). One line of code. Easy.
This is very important as the utility that this file eventually goes to is expecting the file in this format. What I've had to do is change this struct to a class in C#, but I cannot find a way to easily initialize each of these byte arrays to all space characters.
What I've ended up having to do is this in the class constructor:
//Initialize all of the variables to spaces.
int index = 0;
foreach (byte b in UserCode)
{
UserCode[index] = 0x20;
index++;
}
This works fine, but I'm sure there must be a simpler way to do this. When the array is set to UserCode = new byte[6] in the constructor the byte array gets automatically initialized to the default null values. Is there no way that I can make it become all spaces upon declaration, so that when I call my class' constructor that it is initialized straight away like this? Or some memset-like function?
For small arrays use array initialisation syntax:
var sevenItems = new byte[] { 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20 };
For larger arrays use a standard for loop. This is the most readable and efficient way to do it:
var sevenThousandItems = new byte[7000];
for (int i = 0; i < sevenThousandItems.Length; i++)
{
sevenThousandItems[i] = 0x20;
}
Of course, if you need to do this a lot then you could create a helper method to help keep your code concise:
byte[] sevenItems = CreateSpecialByteArray(7);
byte[] sevenThousandItems = CreateSpecialByteArray(7000);
// ...
public static byte[] CreateSpecialByteArray(int length)
{
var arr = new byte[length];
for (int i = 0; i < arr.Length; i++)
{
arr[i] = 0x20;
}
return arr;
}
Use this to create the array in the first place:
byte[] array = Enumerable.Repeat((byte)0x20, <number of elements>).ToArray();
Replace <number of elements> with the desired array size.
You can use Enumerable.Repeat()
Enumerable.Repeat generates a sequence that contains one repeated value.
Array of 100 items initialized to 0x20:
byte[] arr1 = Enumerable.Repeat((byte)0x20,100).ToArray();
var array = Encoding.ASCII.GetBytes(new string(' ', 100));
If you need to initialise a small array you can use:
byte[] smallArray = new byte[] { 0x20, 0x20, 0x20, 0x20, 0x20, 0x20, 0x20 };
If you have a larger array, then you could use:
byte[] bitBiggerArray Enumerable.Repeat(0x20, 7000).ToArray();
Which is simple, and easy for the next guy/girl to read. And will be fast enough 99.9% of the time.
(Normally will be the BestOption™)
However if you really really need super speed, calling out to the optimized memset method, using P/invoke, is for you:
(Here wrapped up in a nice to use class)
public static class Superfast
{
[DllImport("msvcrt.dll",
EntryPoint = "memset",
CallingConvention = CallingConvention.Cdecl,
SetLastError = false)]
private static extern IntPtr MemSet(IntPtr dest, int c, int count);
//If you need super speed, calling out to M$ memset optimized method using P/invoke
public static byte[] InitByteArray(byte fillWith, int size)
{
byte[] arrayBytes = new byte[size];
GCHandle gch = GCHandle.Alloc(arrayBytes, GCHandleType.Pinned);
MemSet(gch.AddrOfPinnedObject(), fillWith, arrayBytes.Length);
gch.Free();
return arrayBytes;
}
}
Usage:
byte[] oneofManyBigArrays = Superfast.InitByteArray(0x20,700000);
Maybe these could be helpful?
What is the equivalent of memset in C#?
http://techmikael.blogspot.com/2009/12/filling-array-with-default-value.html
Guys before me gave you your answer. I just want to point out your misuse of foreach loop. See, since you have to increment index standard "for loop" would be not only more compact, but also more efficient ("foreach" does many things under the hood):
for (int index = 0; index < UserCode.Length; ++index)
{
UserCode[index] = 0x20;
}
This is a faster version of the code from the post marked as the answer.
All of the benchmarks that I have performed show that a simple for loop that only contains something like an array fill is typically twice as fast if it is decrementing versus if it is incrementing.
Also, the array Length property is already passed as the parameter so it doesn't need to be retrieved from the array properties. It should also be pre-calculated and assigned to a local variable.
Loop bounds calculations that involve a property accessor will re-compute the value of the bounds before each iteration of the loop.
public static byte[] CreateSpecialByteArray(int length)
{
byte[] array = new byte[length];
int len = length - 1;
for (int i = len; i >= 0; i--)
{
array[i] = 0x20;
}
return array;
}
Just to expand on my answer a neater way of doing this multiple times would probably be:
PopulateByteArray(UserCode, 0x20);
which calls:
public static void PopulateByteArray(byte[] byteArray, byte value)
{
for (int i = 0; i < byteArray.Length; i++)
{
byteArray[i] = value;
}
}
This has the advantage of a nice efficient for loop (mention to gwiazdorrr's answer) as well as a nice neat looking call if it is being used a lot. And a lot mroe at a glance readable than the enumeration one I personally think. :)
The fastest way to do this is to use the api:
bR = 0xFF;
RtlFillMemory(pBuffer, nFileLen, bR);
using a pointer to a buffer, the length to write, and the encoded byte. I think the fastest way to do it in managed code (much slower), is to create a small block of initialized bytes, then use Buffer.Blockcopy to write them to the byte array in a loop. I threw this together but haven't tested it, but you get the idea:
long size = GetFileSize(FileName);
// zero byte
const int blocksize = 1024;
// 1's array
byte[] ntemp = new byte[blocksize];
byte[] nbyte = new byte[size];
// init 1's array
for (int i = 0; i < blocksize; i++)
ntemp[i] = 0xff;
// get dimensions
int blocks = (int)(size / blocksize);
int remainder = (int)(size - (blocks * blocksize));
int count = 0;
// copy to the buffer
do
{
Buffer.BlockCopy(ntemp, 0, nbyte, blocksize * count, blocksize);
count++;
} while (count < blocks);
// copy remaining bytes
Buffer.BlockCopy(ntemp, 0, nbyte, blocksize * count, remainder);
This function is way faster than a for loop for filling an array.
The Array.Copy command is a very fast memory copy function. This function takes advantage of that by repeatedly calling the Array.Copy command and doubling the size of what we copy until the array is full.
I discuss this on my blog at https://grax32.com/2013/06/fast-array-fill-function-revisited.html (Link updated 12/16/2019). Also see Nuget package that provides this extension method. http://sites.grax32.com/ArrayExtensions/
Note that this would be easy to make into an extension method by just adding the word "this" to the method declarations i.e. public static void ArrayFill<T>(this T[] arrayToFill ...
public static void ArrayFill<T>(T[] arrayToFill, T fillValue)
{
// if called with a single value, wrap the value in an array and call the main function
ArrayFill(arrayToFill, new T[] { fillValue });
}
public static void ArrayFill<T>(T[] arrayToFill, T[] fillValue)
{
if (fillValue.Length >= arrayToFill.Length)
{
throw new ArgumentException("fillValue array length must be smaller than length of arrayToFill");
}
// set the initial array value
Array.Copy(fillValue, arrayToFill, fillValue.Length);
int arrayToFillHalfLength = arrayToFill.Length / 2;
for (int i = fillValue.Length; i < arrayToFill.Length; i *= 2)
{
int copyLength = i;
if (i > arrayToFillHalfLength)
{
copyLength = arrayToFill.Length - i;
}
Array.Copy(arrayToFill, 0, arrayToFill, i, copyLength);
}
}
You can use a collection initializer:
UserCode = new byte[]{0x20,0x20,0x20,0x20,0x20,0x20};
This will work better than Repeat if the values are not identical.
You could speed up the initialization and simplify the code by using the the Parallel class (.NET 4 and newer):
public static void PopulateByteArray(byte[] byteArray, byte value)
{
Parallel.For(0, byteArray.Length, i => byteArray[i] = value);
}
Of course you can create the array at the same time:
public static byte[] CreateSpecialByteArray(int length, byte value)
{
var byteArray = new byte[length];
Parallel.For(0, length, i => byteArray[i] = value);
return byteArray;
}