I have some large arrays of 2D data elements. A and B aren't equally sized dimensions.
A) is between 5 and 20
B) is between 1000 and 100000
The initialization time is no problem as its only going to be lookup tables for realtime application, so performance on indexing elements from knowing value A and B is crucial. The data stored is currently a single byte-value.
I was thinking around these solutions:
byte[A][B] datalist1a;
or
byte[B][A] datalist2a;
or
byte[A,B] datalist1b;
or
byte[B,A] datalist2b;
or perhaps loosing the multidimension as I know the fixed size and just multiply the to values before looking it up.
byte[A*Bmax + B] datalist3;
or
byte[B*Amax + A] datalist4;
What I need is to know, what datatype/array structure to use for most efficient lookup in C# when I have this setup.
Edit 1
the first two solutions were supposed to be multidimensional, not multi arrays.
Edit 2
All data in the smallest dimension is read at each lookup, but the large one is only used for indexing once at a time.
So its something like - Grab all A's from sample B.
I'd bet on the jagged arrays, unless the Amax or Bmax are a power of 2.
I'd say so, because a jagged array needs two indexed accesses, thus very fast. The other forms implies a multiplication, either implicit or explicit. Unless that multiplication is a simple shift, I think could be a bit heavier than a couple of indexed accesses.
EDIT: Here is the small program used for the test:
class Program
{
private static int A = 10;
private static int B = 100;
private static byte[] _linear;
private static byte[,] _square;
private static byte[][] _jagged;
unsafe static void Main(string[] args)
{
//init arrays
_linear = new byte[A * B];
_square = new byte[A, B];
_jagged = new byte[A][];
for (int i = 0; i < A; i++)
_jagged[i] = new byte[B];
//set-up the params
var sw = new Stopwatch();
byte b;
const int N = 100000;
//one-dim array (buffer)
sw.Restart();
for (int i = 0; i < N; i++)
{
for (int r = 0; r < A; r++)
{
for (int c = 0; c < B; c++)
{
b = _linear[r * B + c];
}
}
}
sw.Stop();
Console.WriteLine("linear={0}", sw.ElapsedMilliseconds);
//two-dim array
sw.Restart();
for (int i = 0; i < N; i++)
{
for (int r = 0; r < A; r++)
{
for (int c = 0; c < B; c++)
{
b = _square[r, c];
}
}
}
sw.Stop();
Console.WriteLine("square={0}", sw.ElapsedMilliseconds);
//jagged array
sw.Restart();
for (int i = 0; i < N; i++)
{
for (int r = 0; r < A; r++)
{
for (int c = 0; c < B; c++)
{
b = _jagged[r][c];
}
}
}
sw.Stop();
Console.WriteLine("jagged={0}", sw.ElapsedMilliseconds);
//one-dim array within unsafe access (and context)
sw.Restart();
for (int i = 0; i < N; i++)
{
for (int r = 0; r < A; r++)
{
fixed (byte* offset = &_linear[r * B])
{
for (int c = 0; c < B; c++)
{
b = *(byte*)(offset + c);
}
}
}
}
sw.Stop();
Console.WriteLine("unsafe={0}", sw.ElapsedMilliseconds);
Console.Write("Press any key...");
Console.ReadKey();
Console.WriteLine();
}
}
Multidimensional ([,]) arrays are nearly always the slowest, unless under a heavy random access scenario. In theory they shouldn't be, but it's one of the CLR oddities.
Jagged arrays ([][]) are nearly always faster than multidimensional arrays; even under random access scenarios. These have a memory overhead.
Singledimensional ([]) and algebraic arrays ([y * stride + x]) are the fastest for random access in safe code.
Unsafe code is, normally, fastest in all cases (provided you don't pin it repeatedly).
The only useful answer to "which X is faster" (for all X) is: you have to do performance tests that reflect your requirements.
And remember to consider, in general*:
Maintenance of the program. If this is not a quick one off, a slightly slower but maintainable program is a better option in most cases.
Micro benchmarks can be deceptive. For instance a tight loop just reading from a collection might be optimised away in ways not possible when real work is being done.
Additionally consider that you need to look at the complete program to decide where to optimise. Speeding up a loop by 1% might be useful for that loop, but if it is only 1% of the complete runtime then it is not making much differences.
* But all rules have exceptions.
On most modern computers, arithmetic operations are far, far faster than memory lookups.
If you fetch a memory address that isn't in a cache or where the out of order execution pulls from the wrong place you are looking at 10-100 clocks, a pipelined multiply is 1 clock.
The other issue is cache locality.
byte[BAmax + A] datalist4; seems like the best bet if you are accessing with A's varying sequentially.
When datalist4[bAmax + a] is accessed, the computer will usually start pulling in datalist4[bAmax + a+ 64/sizeof(dataListType)], ... +128 ... etc, or if it detects a reverse iteration, datalist4[bAmax + a - 64/sizeof(dataListType)]
Hope that helps!
May be best way for u will be use HashMap
Dictionary?
Related
I'm doing brainteasers and exercises on hackerrank to prepare for an interview test and came across some weird behaviour.
The taks in question is Minimum Swaps 2 from the Interview Preparation Kit's Arrays section.
Depending on which of the two elements I put into my temporary variable, it either passes or fails on timeout on most test cases.
Version A:
static int minimumSwaps(int[] arr) {
int swaps=0;
int tmp=0;
int n = arr.Length;
for(int i = 0; i < n; i++){
if(arr[i]==i+1) continue;
tmp = arr[arr[i]-1];
arr[arr[i]-1] = arr[i];
arr[i] = tmp;
i--;
swaps++;
}
return swaps;
}
Version B
static int minimumSwaps(int[] arr) {
int swaps=0;
int tmp=0;
int n = arr.Length;
for(int i = 0; i < n; i++){
if(arr[i]==i+1) continue;
tmp = arr[i];
arr[i] = arr[arr[i]-1];
arr[arr[i]-1]=tmp;
i--;
swaps++;
}
return swaps;
}
A passes, B times out on all but one test.
What it actually does is find the minimum amount of swaps to sort an array of consecutive integers. That's why i can be used to compare values and indices.
If arr[i] is on the corresponding index, we continue, if it's wrong, we swap it with the index it's supposed to be at, decrement i and repeat.
I'm honestly at a loss as to why storing arr[arr[i]-1] in tmp is less complicated than storing arr[i] first. It must have something to do with the intermediate machine code it get's compiled to, right?
I'd be grateful for any explanation so I don't run into similar issues in the future.
I have a problem that I don't understand, in that code:
ilProbekUcz= valuesUcz.Count; //valuesUcz is the list of <float[]>
for (int i = 0; i < ilWezlowDanych; i++) nodesValueArrayUcz[i] = new BitArray(ilProbekUcz);
Stopwatch sw = new Stopwatch();
sw.Start();
for (int i = 0; i < ilProbekUcz; i++)
{
int index = 0;
linia = (float[])valuesUcz[i];//removing this line not solve problem
for (int a = 0; a < ileRazem; a++)
for (int b = 0; b < ileRazem; b++)
if (a != b)
{
bool value = linia[a] >= linia[b];
nodesValueArrayUcz[index][i] = value;
nodesValueArrayUcz[ilWezlowDanychP2 + index][i] = !value;
index++;
}
}
sw.Stop();
When i increase size of valuesUcz 2x, time of execution is 4x bigger
When i increase size of valuesUcz 4x, time of execution is 8x bigger
etc ...
(ileRazem,ilWezlowDanych is the same)
I understand: increase of ilProbekUcz increases size of BitArrays but i test it many times and it is no problem - time should grow linearly - in code:
ilProbekUcz= valuesUcz.Count; //valuesTest is the list of float[]
for (int i = 0; i < ilWezlowDanych; i++) nodesValueArrayUcz[i] = new BitArray(ilProbekUcz);
BitArray test1 = nodesValueArrayUcz[10];
BitArray test2 = nodesValueArrayUcz[20];
Stopwatch sw = new Stopwatch();
sw.Start();
for (int i = 0; i < ilProbekUcz; i++)
{
int index = 0;
linia = (float[])valuesUcz[i];//removing this line not solve problem
for (int a = 0; a < ileRazem; a++)
for (int b = 0; b < ileRazem; b++)
if (a != b)
{
bool value = linia[a] >= linia[b];
test1[i] = value;
test2[i] = !value;
index++;
}
}
time grows linearly, so the problem is to take a BitArray from the array...
Is any method to do it faster ? (i want time to grow linearly)
You have to understand that measuring time there are many factors that makes them inacurate. The biggest factor when you have huuuuuge arrays as in your example is cashe misses. Many times the same thing written when taking account of cashe, can be as much as 2-5 or more times faster. Two words how cashe works, very roughly. Cache is memory inside cpu. It is waaaaaaaaaaaaaay faster than ram so when you want to fetch a variable from memory you want to make sure this variable is stored in cache and not in ram. If it is stored in cache we say we have a hit otherwise a miss. Some times, not so often, a program is so big that it stores variables in hard drive. In that case you have a huuuuuuuuuuuge hit in delay when you fetch these! An example of cache:
Lets say we have an array of 10 elements in memory(ram)
when you get the first element testArray[0], because testArray[0] is not in cache the cpu brings this value along with a number(lets say 3, the number depends on the cpu) of adjacent elements of the array eg it stores to cache testArray[0], testArray[1], testArray[2], testArray[3]
Now when we get testArray[1] it is in cache so we have a hit. The same with testArray[2] and testArray[3]. testArray[4] isn't in cache so it gets testArray[4] along with another 3 testArray[5], testArray[6], testArray[7]
and so on...
Cache misses are very costly. That means you may expect an array of double the size is going to be accessible double the time. But this is not true. Bigger arrays more misses
and the time may increase 2 or 3 or 4 or more times from what you expect. This is normal. In your example that is what is happening. From 100 million elemensts(first array) you go t0 400 million (second one). The missesare not double but waaay more as you saw. A very cool trick has to do with the way you access an array. In your example ba1[j][i] = (j % 2) == 0; is way worse than ba1[i][j] = (j % 2) == 0;. The same with ba2[j][i] = (j % 2) == 0; and ba1[i][j] = (j % 2) == 0;. You can test it. Just reverse i and j. It has to do with the way the 2D array is stored in memory so in the second case you have more hits that the first one.
as you can read from the title I'm trying to store all the numbers between two numbers in an array.
For example store the numbers between 21 and 43 (22,23,24,25,26,27,28,29...) in an array.
This is the code, I don't know why but it prints only the higher number minus one.
class Program
{
static void Main(string[] args)
{
int higher = 43;
int lower = 21;
int[] numbers = new int[22]; //the numbers between 21 and 43 are 22
for (int i = lower; i < higher;i++)
{
for (int a = 0; a < 22; a++)
{
numbers[a] = i;
}
}
for (int c = 0; c < 22; c++)
{
Console.WriteLine(numbers[c]);
}
Console.ReadLine();
}
}
This is the code, I don't know why but it prints only the higher number minus one.
This question will attract answers giving you a half dozen solutions you can cut and paste to do your assignment.
I note you did not ask a question in your question -- next time, please format your question in the form of a question. The right question to ask here is how do I learn how to spot mistakes in code I've written? because that is the vital skill you lack. Answers that give you the code will not answer that question.
I already gave you a link to a recent answer where I explain that in detail, so study that.
In particular, in your case you have to read the program you wrote as though you had not written it. As though you were coming fresh to the program that someone else wrote and trying to figure out what it does.
The first thing I would do is look at the inner loop and say to myself "what does this do, in words?"
for (int a = 0; a < 22; a++)
{
numbers[a] = i;
}
That is "put the value i in every slot of the array. Now look at the outer loop:
for (int i = lower; i < higher;i++)
{
put the value i in every slot of the array
}
Now the technique to use here is to logically "unroll" the loop. A loop just does something multiple times so write that out. It starts with lower, it goes to higher-1, so that loop does this:
put the value lower in every slot of the array
put the value lower+1 in every slot of the array
…
put the value higher-1 in every slot of the array
What does the third loop do?
print every item in the array
And now you know why it prints the highest number minus one multiple times. Because that's what the program does. We just reasoned it out.
Incidentally the answers posted so far are correct, but some are not the best.
You have a technique that you understand for "do something to every member of an array, and that is:
loop an indexer from 0 to the array size minus one
do something to the array slot at the indexer
But the solutions the other answers are proposing are the opposite:
loop an indexer from the lower to the higher value
compute an index
do something to the array slot at that index
It's important to understand that both are correct, but my feeling is that for the beginner you should stick with the pattern you know. How would we
loop an indexer from 0 to the array size minus one
do something to the array slot at the indexer
for your problem? Let's start with giving you a much better technique for looping the indexer:
for (int i = 0; i < numbers.Length; ++i)
That's a better technique because when you change the size of the array, you don't have to change the loop! And also you are guaranteed that every slot in the array is covered. Design your loops so that they are robust to changes and have good invariants.
Now you have to work out what the right loop body is:
{
int number = i + lower;
numbers[i] = number;
}
Now you know that your loop invariant is "when the loop is done, the array is full of consecutive numbers starting at lower".
For everytime you loop through i, you put that number in every slot of the array. The inner loop is what is causing your issue. A better solution would be:
int higher = 43;
int lower = 21;
int[] numbers = new int[21];
int index = 0;
for (int i = lower + 1; i < higher; i++) // if you want to store everything
// between 21 and 43, you need to
// start with 22, thus lower + 1
{
numbers[index] = i;
index++;
}
for (int c = 0; c < 21; c++)
{
Console.WriteLine(numbers[c]);
}
Console.ReadLine();
Replace a with a direct translation of i
for (int i = lower; i < higher;i++)
{
numbers[i-lower] = i;
}
Use below
int higher = 43;
int lower = 21;
int[] numbers = new int[22]; //the numbers between 21 and 43 are 22
for (int i = lower+1; i < higher; i++)
{
numbers[i-lower] = i;
}
for (int c = 1; c < 21; c++)
{
Console.WriteLine(numbers[c]);
}
Console.ReadLine();
I think higher & lower are variables so following will give you output
for any higher and lower numbers
class Program
{
static void Main(string[] args)
{
int higher = 43;
int lower = 21;
int numDiff = higher - lower - 1;
int[] numbers = new int[numDiff]; //the numbers between 21 and 43 are 22
for(int i = 0; i<numbers.Length; i++)
{
numbers[i] = numDiff + i + 1;
}
for(int b = 0; b<numbers.Length; b++)
{
Console.WriteLine(numbers[b]);
}
Console.ReadLine();
}
}
Is it faster to process the unwrapping of Tuple and using a few variables instead of just using Tuple as it is. Consider this example :
Tuple<int,int> test = new Tuple<int,int>;
int numberOne = int.Parse(Console.Readline());
int numberTwo = int.Parse(Console.Readline());
test.Item1 = numberOne;
test.Item2 = numberTwo;
for(int i = 0; i < 10; i++)
{
if(test.Item1 * i > test.Item2 * i)
{// do stuff}
else
{// do stuff}
}
VS
Tuple<int,int> test = new Tuple<int,int>;
int numberOne = int.Parse(Console.Readline());
int numberTwo = int.Parse(Console.Readline());
test.Item1 = numberOne;
test.Item2 = numberTwo;
for(int i = 0; i < 1000; i ++)
{
int tempItem1 = test.Item1;
int tempItem2 = test.Item2;
if(tempItem1 * i > tempItem2 * i)
{// do stuff}
else
{// do stuff}
}
I also want to know if unwrapping the Tuple with 2 variables is faster than using item1 and item2. Will the result still be the same if we were using more than 2 variables ?
Thanks in advance
Both versions load the tuple items from memory into a temporary value. The only difference is whether it's a temporary on the IL execution stack or an actual IL local. Normally, this should be within reach of the JIT.
If this was a C compiler it would be 100% certain that the performance is the same. Any case where that was not so would be a bug.
The .NET JIT has a very poor optimizer so you might always get unlucky and step into an optimizer hole.
Please check below code, this code try to compute birthday conflict possibility. To my surprise, if i execute those code with sequence, the result is expected around 0.44; but if try on PLinq, the result is 0.99.
Anyone can explain the result?
public static void BirthdayConflict(int num = 5, int people = 300) {
int N = 100000;
int act = 0;
Random r = new Random();
Action<int> action = (a) => {
List<int> p = new List<int>();
for (int i = 0; i < people; i++)
{
p.Add(r.Next(364) + 1);
}
p.Sort();
bool b = false;
for (int i = 0; i < 300; i++)
{
if (i + num -1 >= people) break;
if (p[i] == p[i + num -1])
b = true;
}
if (b)
Interlocked.Increment(ref act);
// act++;
};
// Result is around 0.99 - which is not OK
// Parallel.For( 0, N, action);
//Result is around 0.44 - which is OK
for (int i = 0; i < N; i++)
{
action(0);
}
Console.WriteLine(act / 100000.0);
Console.ReadLine();
}
You're using a shared (between threads) instance System.Random. It's not thread-safe then you're getting wrong results (well actually it just doesn't work and it'll return 0). From MSDN:
If your app calls Random methods from multiple threads, you must use a synchronization object to ensure that only one thread can access the random number generator at a time. If you don't ensure that the Random object is accessed in a thread-safe way, calls to methods that return random numbers return 0.
Simple (but not so efficient for parallel execution) solution is to use a lock:
lock (r)
{
for (int i = 0; i < people; i++)
{
p.Add(r.Next(364) + 1);
}
}
To improve performance (but you should measure) you may use multiple instances of System.Random, be careful to initialize each one with a different seed.
I find a useful explanation why random does not work under multi-thread, although it was original for Java, still can be benefitical.