Consider a simple equation: 5 = 2 * a + 4 * b - 3 * c
Is it a better way to loop thru the variables than multiple for loops?
This has multiple answers, but in order to find answers to the equation I'm using multiple for loops like
for(int a = 1; a < 50; a++) {
for(int b = 1; b < 50; b++) {
for(int c = 1; c < 50; c++) {
//validate
}
}
}
Now, for this example this would not take much time. But if this was going thru a dataset of thousands of entries and the goal of the for loop is see If I can find a optimized set of variables then its going to take some time. Maybe there are more than 3. The equation above is just an example.
Is there a alternative better way to do this? A code pattern maybe? I'm also interested to see how I can clean this up as there is a lot of nesting.
My validation logic is already thrown inside a BackgroundWorker and I limit the count so I can utilize 100% of the CPU, so I'm mainly looking into not doing for-loop nesting if possible.
The nested loop is the most efficient way to do it, and you can parallelize it pretty easily by Parallel.Foring the outer loop.
int solutionsCount = 0;
Parallel.For(1, 50, a =>
{
for (int b = 1; b < 50; b++)
for (int c = 1; c < 50; c++)
if (2 * a + 4 * b - 3 * c == 0) Interlocked.Increment(ref solutionsCount);
});
If you want to get fancy you can create a custom iterator that will produce all the permutations:
private static IEnumerable<(int a, int b, int c)> Loop(int to1, int to2, int to3)
{
for (int a = 1; a < to1; a++)
for (int b = 1; b < to2; b++)
for (int c = 1; c < to3; c++)
yield return (a, b, c); // this is a ValueTuple<int, int, int>
}
And use it like this:
foreach (var p in Loop(50, 50, 50))
{
// Do something with p.a, p.b and p.c
}
You can even use LINQ to get the solutions directly:
var solutions = Loop(50, 50, 50)
.Where(p => 2 * p.a + 4 * p.b - 3 * p.c == 0);
Console.WriteLine($"Solutions: {String.Join(", ", solutions)}");
...but it is 10 times slower.
You could even go pure LINQ like this:
var solutions = Enumerable.Range(1, 50 - 1)
.SelectMany(a => Enumerable.Range(1, 50 - 1)
.SelectMany(b => Enumerable.Range(1, 50 - 1)
.Where(c => 2 * a + 4 * b - 3 * c == 0)));
...which has about the same performance as the previous one. It is also parallelizable by chaining AsParallel() in the query (after the first Enumerable.Range).
Related
I have a float array containing 1M floats
I want to do sampling: for each 4 floats I want to take only 1. So i am doing this :
for(int i = 0; i< floatArray.Length; i++) {
if(i % 4 == 0) {
resultFloat.Add(floatArray[i])
}
}
This works fine, but it takes much time to run through all the elements , is there any other methods to make it with better results (if there are any)
I can see two factors that might be slowing down performance.
As you have already been offered, you should set the step to 4:
for (int i = 0; i < floatArray.Length; i += 4)
{
resultFloat.Add(floatArray[i]);
}
Looks like resultFloat is a list of float. I suggest to use array instead of list, like this:
int m = (floatArray.Length + 3) / 4;
float[] resultFloat = new float[m];
for (int i = 0, k = 0; i < floatArray.Length; i += 4, k++)
{
resultFloat[k] = floatArray[i];
}
Just increment your loop by 4 each iteration instead of by 1:
for(int i = 0; i< floatArray.Length; i+=4)
{
resultFloat.Add(floatArray[i]);
}
If you really have an issue with performance, then you'd be even better off not using a dynamic container for the results, but a statically sized array.
float[] resultFloat = new float[(floatArray.Length + 3) >> 2];
for(int i = 0; i < resultFloat.Length; i++)
resultFloat[i] = floatArray[i << 2];
Usually performance isn't an issue thow, and you shouldn't optimize until a profiler gave you proof that you should. In all other cases the more readable code is preferrable.
Just to add another option, if you want this to be the fastest, use Parallel.For instead of a normal for loop.
int resultLength = (floatArray.Length + 3) / 4;
var resultFloat = new float[resultLength];
Parallel.For(0, resultLength, i =>
{
resultFloat[i] = floatArray[i * 4];
});
List<decimal> list = new List<decimal>();
list.Add(1);
list.Add(2);
list.Add(3);
list.Add(4);
list.Add(5);
list.Add(6);
list.Add(7);
list.Add(8);
list.Add(9);
list.Add(10);
list.Add(11);
list.Add(12);
list.Add(13);
list.Add(14);
list.Add(15);
list.Add(16);
var sampleData = list.Where((x, i) => (i + 1) % (4) == 0).ToList();
public static int n;
public static int w;
public static int[] s;
public static int[] p;
static void Main(string[] args)
{
n = 5;
w = 5;
s = new int[n + 1];
p = new int[n + 1];
Random rnd = new Random();
for (int i = 1; i <= n; i++)
{
s[i] = rnd.Next(1, 10);
p[i] = rnd.Next(1, 10);
}
Console.WriteLine(F_recursion(n, w));
Console.WriteLine(DP(n, w));
}
// recursive approach
public static int F_recursion(int n, int w)
{
if (n == 0 || w == 0)
return 0;
else if (s[n] > w)
return F_recursion(n - 1, w);
else
{
return Math.Max(F_recursion(n - 1, w), (p[n] + F_recursion(n - 1, w - s[n])));
}
}
// iterative approach
public static int DP(int n, int w)
{
int result = 0;
for (int i = 1; i <= n; i++)
{
if (s[i] > w)
{
continue;
}
else
{
result += p[i];
w = w - s[i];
}
}
return result;
}
I need to convert F_recursion function to iterative. I currently written following function DP that sometimes works but not always. I learned that problem is in F_recursion(n - 1, w - s[n]) I have no idea how to make w - s[n] work correctly in iterative solution. If change w - s[n] and w - s[i] to only w then program always work.
In Console:
s[i] = 2 p[i] = 3
-------
s[i] = 3 p[i] = 4
-------
s[i] = 5 p[i] = 3
-------
s[i] = 3 p[i] = 8
-------
s[i] = 6 p[i] = 6
-------
Recursive:11
Iteration:7
but sometimes it works
s[i] = 5 p[i] = 6
-------
s[i] = 8 p[i] = 1
-------
s[i] = 3 p[i] = 5
-------
s[i] = 3 p[i] = 1
-------
s[i] = 7 p[i] = 7
-------
Recursive:6
Iteration:6
The following approach might be useful, when bigger numbers are involved (specially for s) and consequently a 2 dimensional array would be unnecessary big and only a few w values would actually be used in computing the result.
The idea: precompute possible w values, by starting at w and for each i in [n, n-1, ..., 1] determine the values w_[i], where w_[i+1] >= s[i] without duplicates.
Then iterate i_n over n and compute sub-results only for valid w_[i] values.
I chose an array of Dictionary as datastructure, since it's relatively easy to design sparse data this way.
public static int DP(int n, int w)
{
// compute possible w values for each iteration from 0 to n
Stack<HashSet<int>> validW = new Stack<HashSet<int>>();
validW.Push(new HashSet<int>() { w });
for (int i = n; i > 0; i--)
{
HashSet<int> validW_i = new HashSet<int>();
foreach (var prevValid in validW.Peek())
{
validW_i.Add(prevValid);
if (prevValid >= s[i])
{
validW_i.Add(prevValid - s[i]);
}
}
validW.Push(validW_i);
}
// compute sub-results for all possible n,w values.
Dictionary<int, int>[] value = new Dictionary<int,int>[n + 1];
for (int n_i = 0; n_i <= n; n_i++)
{
value[n_i] = new Dictionary<int, int>();
HashSet<int> validSubtractW_i = validW.Pop();
foreach (var w_j in validSubtractW_i)
{
if (n_i == 0 || w_j == 0)
value[n_i][w_j] = 0;
else if (s[n_i] > w_j)
value[n_i][w_j] = value[n_i - 1][w_j];
else
value[n_i][w_j] = Math.Max(value[n_i - 1][w_j], (p[n_i] + value[n_i - 1][w_j - s[n_i]]));
}
}
return value[n][w];
}
It's important to understand that some space and computation is "wasted" in order to precompute possible w values and to support the sparse data structures. So this approach might perform bad for large data sets with small values in s, where most w values will be possible sub-results.
After some more thought I realized, if space is a concern, you can actually throw away the sub-results of everything except the previous outer loop iteration, since the recursion in this algorithm follows a strict n-1 pattern. However, I'm not including this into my code for now.
Your approach does not work because your dynamic programmig state space (which apparently is only one variable) does not match the signature of the recursive method. The goal of the dynamic programming approach should be to define and fill a state space such that all results for evaluation are available when needed. On inspection of the recursive method, notice that the recursive calls of F_recursion may change both arguments, n and w. This is an indication that a two-dimensional state space should be used.
The first argument (which apparently limits the range of items) can range from 0 to n while the second argument (which apparently is some bound for the total of an item property) can range from 0 to w.
You should define a two dimensional state space
int[,] value = new int[n,w];
for accomodation of the values. Next, you should initialize the values to undefined; you can use the value Int32.MaxValue for this, because it will behave in a suitable way if the minimum with some different value is calculated.
Next, the iterative version of the algorithm shoud use two loops which iterate in a forwad manner, unlike the recursive iteration which decreases the arguments.
for (int i = 0; i < n; i++)
{
for (int j = 0; j < w; j++)
{
// logic for the recurrence relation goes here
}
}
In the innermost block you can use a modified version of the recurrence relation. Instead of using recursive calls, you access values which are stored in value; instead of returning values, you write the values to value.
Semantically this is the same as memoization, but instead of using actual recursive calls, the order of evaluation asserts that necessary values always exist, making additional logic unneccessary.
Once the state space is filled, you have to examine its last state (namely the part of the array where the first index is n-1) to determine the maximal value for the entire input.
So I converted a recursive function to iterative and then used Parallel.ForEach but when I was running it through VTune it was only really using 2 logical cores at for the majority of its run time.
I decided to attempt to use managed threads instead, and converted this code:
for (int N = 2; N <= length; N <<= 1)
{
int maxThreads = 4;
var workGroup = Enumerable.Range(0, maxThreads);
Parallel.ForEach(workGroup, i =>
{
for (int j = ((i / maxThreads) * length); j < (((i + 1) / maxThreads) * length); j += N)
{
for (int k = 0; k < N / 2; k++)
{
int evenIndex = j + k;
int oddIndex = j + k + (N / 2);
var even = output[evenIndex];
var odd = output[oddIndex];
output[evenIndex] = even + odd * twiddles[k * (length / N)];
output[oddIndex] = even + odd * twiddles[(k + (N / 2)) * (length / N)];
}
}
});
}
Into this:
for (int N = 2; N <= length; N <<= 1)
{
int maxThreads = 4;
Thread one = new Thread(() => calculateChunk(0, maxThreads, length, N, output));
Thread two = new Thread(() => calculateChunk(1, maxThreads, length, N, output));
Thread three = new Thread(() => calculateChunk(2, maxThreads, length, N, output));
Thread four = new Thread(() => calculateChunk(3, maxThreads, length, N, output));
one.Start();
two.Start();
three.Start();
four.Start();
}
public void calculateChunk(int i, int maxThreads, int length, int N, Complex[] output)
{
for (int j = ((i / maxThreads) * length); j < (((i + 1) / maxThreads) * length); j += N)
{
for (int k = 0; k < N / 2; k++)
{
int evenIndex = j + k;
int oddIndex = j + k + (N / 2);
var even = output[evenIndex];
var odd = output[oddIndex];
output[evenIndex] = even + odd * twiddles[k * (length / N)];
output[oddIndex] = even + odd * twiddles[(k + (N / 2)) * (length / N)];
}
}
}
The issue is in the fourth thread on the last iteration of the N loop I get a index out of bounds exception for the output array where the index is attempting access the equivalent of the length.
I can not pinpoint the cause using debugging, but I believe it is to do with the threads, I ran the code without the threads and it worked as intended.
If any of the code needs changing let me know, I usually have a few people suggest edits. Thanks for your help, I have tried to sort it myself and am fairly certain the problem is occurring in my threading but I can not see how.
PS: The intended purpose is to parallelize this segment of code.
The observed behaviour is almost certainly due to the use of a captured loop iteration variable N. I can reproduce your situation with a simple test:
ConcurrentBag<int> numbers = new ConcurrentBag<int>();
for (int i = 0; i < 10000; i++)
{
Thread t = new Thread(() => numbers.Add(i));
t.Start();
//t.Join(); // Uncomment this to get expected behaviour.
}
// You'd not expect this assert to be true, but most of the time it will be.
Assert.True(numbers.Contains(10000));
Put simply, your for loop is racing to increment N before the value of N can be copied by the delegate that executes the calculateChunk call. As a result calculateChunk sees almost random values of N going up to (and including) length <<= 1 - that's what's causing your IndexOutOfRangeException.
The output values you'll get will be rubbish too as you can never rely on the value of N being correct.
If you want to safely rewrite the original code to utilize more cores, move Parallel.ForEach from the inner loop to the outer loop. If the number of outer loop iterations is high, the load balancer will be able to do its job properly (which it can't with your current workGroup count of 4 - that number of elements is simply too low).
I have this for loop. TicketList starts with 109 tickets. nColumns = 100. I calculate the number of rows I will need depending on the number of tickets. So in this case I need 2 rows. Row one will be full and row two will only have 9 entries. I have the loop below. It only runs one time for the NumOfRows and fills the first 100 and never loops.
What am I missing?
for (int j = 0; j < NumOfRows; j++)
{
for (int i = 0; i < nColumns; i++)
{
if (TicketList.Count() > 0)
{
t = rand.Next(0, TicketList.Count() - 1);
numbers[i, j] = TicketList[t];
TicketList.Remove(TicketList[t]);
}
}
}
Try changing your code to use a more LINQ-like, functional approach. If might make the logic easier. Something like this:
TicketList
.OrderBy(x => rand.Next())
.Select((ticket, n) => new
{
ticket,
j = n / NumOfRows,
i = n % NumOfRows
})
.ToList()
.ForEach(x =>
{
numbers[x.i, x.j] = x.ticket;
});
You may need to flip around x.i & x.j or use nColumns instead of NumOfRows - I wasn't sure what your logic was looking for - but this code might work better.
Other than a few poor choices, your loops appear to be fine. I would venture that NumOfRows is not being calculated correctly.
The expression NumOfRows = (TotalTickets + (Columns - 1)) / Columns; should calculate the correct number of rows.
Also, you should use the property version of Count rather than the Linq extension method and use IList<T>.RemoveAt() or List<T>.RemoveAt rather than Remove(TicketList[T]).
Using Remove() requires that the list be enumerated to locate the element to remove, which may not be the same index that you are targeting. Not to mention that you will scan 50% (on average) of the list for each Remove call, when you already know the correct index to remove.
The functional approach listed earlier seems like overkill.
I've attempted to replicate your issue, assuming certain facts about the various variables in use. The loop repeats the expected number of times.
static void TestMe ()
{
List<object> TicketList = new List<object>();
for (int index = 0; index < 109; index++)
TicketList.Add(new object());
var rand = new Random();
int nColumns = 100;
int NumOfRows = (TicketList.Count + (nColumns - 1)) / nColumns;
object[,] numbers;
int t;
numbers = new object[nColumns, NumOfRows];
for (int j = 0; j < NumOfRows; j++)
{
Console.WriteLine("OuterLoop");
for (int i = 0; i < nColumns; i++)
{
if (TicketList.Count > 0)
{
t = rand.Next(0, TicketList.Count - 1);
numbers[i, j] = TicketList[t];
TicketList.RemoveAt(t);
}
}
}
}
The problem that you are seeing must be the result of something that you have not included in your sample.
I heard about Counting Sort and wrote my version of it based on what I understood.
public void my_counting_sort(int[] arr)
{
int range = 100;
int[] count = new int[range];
for (int i = 0; i < arr.Length; i++) count[arr[i]]++;
int index = 0;
for (int i = 0; i < count.Length; i++)
{
while (count[i] != 0)
{
arr[index++] = i;
count[i]--;
}
}
}
The above code works perfectly.
However, the algorithm given in CLRS is different. Below is my implementation
public int[] counting_sort(int[] arr)
{
int k = 100;
int[] count = new int[k + 1];
for (int i = 0; i < arr.Length; i++)
count[arr[i]]++;
for (int i = 1; i <= k; i++)
count[i] = count[i] + count[i - 1];
int[] b = new int[arr.Length];
for (int i = arr.Length - 1; i >= 0; i--)
{
b[count[arr[i]]] = arr[i];
count[arr[i]]--;
}
return b;
}
I've directly translated this from pseudocode to C#. The code doesn't work and I get an IndexOutOfRange Exception.
So my questions are:
What's wrong with the second piece of code ?
What's the difference algorithm wise between my naive implementation and the one given in the book ?
The problem with your version is that it won't work if the elements have satellite data.
CLRS version would work and it's stable.
EDIT:
Here's an implementation of the CLRS version in Python, which sorts pairs (key, value) by key:
def sort(a):
B = 101
count = [0] * B
for (k, v) in a:
count[k] += 1
for i in range(1, B):
count[i] += count[i-1]
b = [None] * len(a)
for i in range(len(a) - 1, -1, -1):
(k, v) = a[i]
count[k] -= 1
b[count[k]] = a[i]
return b
>>> print sort([(3,'b'),(2,'a'),(3,'l'),(1,'s'),(1,'t'),(3,'e')])
[(1, 's'), (1, 't'), (2, 'a'), (3, 'b'), (3, 'l'), (3, 'e')]
It should be
b[count[arr[i]]-1] = arr[i];
I'll leave it to you to track down why ;-).
I don't think they perform any differently. The second just pushes the correlation of counts out of the loop so that it's simplified a bit within the final loop. That's not necessary as far as I'm concerned. Your way is just as straightforward and probably more readable. In fact (I don't know about C# since I'm a Java guy) I would expect that you could replace that inner while-loop with a library array fill; something like this:
for (int i = 0; i < count.Length; i++)
{
arrayFill(arr, index, count[i], i);
index += count[i];
}
In Java the method is java.util.Arrays.fill(...).
The problem is that you have hard-coded the length of the array that you are using to 100. The length of the array should be m + 1 where m is the maximum element on the original array. This is the first reason that you would think using counting-sort, if you have information about the elements of the array are all minor that some constant and it would work great.