I need an 1D Convolution against 2 big arrays. I'm using this code in C# but it takes a loooong time to run.
I know, i know! FFT convolutions is very fast. But in this project i CAN'T use it.
It is a constraint of the project to not use FFT (please don't ask why :/).
This is my code in C# (ported from matlab, by the way):
var result = new double[input.Length + filter.Length - 1];
for (var i = 0; i < input.Length; i++)
{
for (var j = 0; j < filter.Length; j++)
{
result[i + j] += input[i] * filter[j];
}
}
So, anyone knows any fast convolution algorithm widthout FFT?
Convolution is numerically the same as a polynomial multiplication with an extra wrap-around step. Therefore, all the polynomial and large integer multiplication algorithms can be used to perform convolution.
FFT is the only way to get the fast O(n log(n)) run-time. But you can still get sub-quadratic run-time using the divide-and-conquer approaches like Karatsuba's algorithm.
Karatsuba's algorithm is fairly easy to implement once you understand how it works. It runs in O(n^1.585), and will probably be faster than trying to super-optimize the classic O(n^2) approach.
You could reduce the number of indexed accesses to result, as well as the Length properties:
int inputLength = filter.Length;
int filterLength = filter.Length;
var result = new double[inputLength + filterLength - 1];
for (int i = resultLength; i >= 0; i--)
{
double sum = 0;
// max(i - input.Length + 1,0)
int n1 = i < inputLength ? 0 : i - inputLength + 1;
// min(i, filter.Length - 1)
int n2 = i < filterLength ? i : filterLength - 1;
for (int j = n1; j <= n2; j++)
{
sum += input[i - j] * filter[j];
}
result[i] = sum;
}
If you further split the outer loop, you can get rid of some repeating conditionals. (This assumes 0 < filterLength ≤ inputLength ≤ resultLength)
int inputLength = filter.Length;
int filterLength = filter.Length;
int resultLength = inputLength + filterLength - 1;
var result = new double[resultLength];
for (int i = 0; i < filterLength; i++)
{
double sum = 0;
for (int j = i; j >= 0; j--)
{
sum += input[i - j] * filter[j];
}
result[i] = sum;
}
for (int i = filterLength; i < inputLength; i++)
{
double sum = 0;
for (int j = filterLength - 1; j >= 0; j--)
{
sum += input[i - j] * filter[j];
}
result[i] = sum;
}
for (int i = inputLength; i < resultLength; i++)
{
double sum = 0;
for (int j = i - inputLength + 1; j < filterLength; j++)
{
sum += input[i - j] * filter[j];
}
result[i] = sum;
}
You can use a special IIR filter.
Then process that as like:
y(n)= a1*y(n-1)+b1*y(n-2)...+a2*x(n-1)+b2*x(n-2)......
I think it's faster.
Here are two possibilities that may give minor speedups, but you'd need to test to be sure.
Unroll the inner loop to remove some tests. This will be easier if you know the filter length will always be a multiple if some N.
Reverse the order of the loops. Do filter.length passes over the whole array. This does less dereferencing in the inner loop but may have worse caching behavior.
Related
So I wrote bucket sort implementation (I use rnd.NextDouble() to fill my array, so all the elements are in the range between 0 and 1). And I have number of experiments (10 experiments for each array size) and different array sizes (I start with 1000 and then +1000, etc. till 300.000). And sometimes it works fine, but sometimes it gives me OutOfRange exception:
public static void BucketSortImplement(ref int countOfElements, ref float[] array)
{
List<float>[] buckets = new List<float>[countOfElements];
for (int i = 0; i < countOfElements; i++)
{
buckets[i] = new List<float>();
}
for (int i = 0; i < countOfElements; i++)
{
float indexOfElement = array[i] * countOfElements;
buckets[(int)indexOfElement].Add(array[i]); // right here
}
for (int i = 0; i < countOfElements; i++)
{
for (int j = 0; j < buckets[i].Count; j++)
{
float keyElement = buckets[i][j];
int k = j - 1;
while (k >= 0 && buckets[i][k] > keyElement)
{
buckets[i][k + 1] = buckets[i][k];
k -= 1;
}
buckets[i][k + 1] = keyElement;
}
}
int arrayIndex = 0;
for (int i = 0; i < countOfElements; i++)
{
for (int j = 0; j < buckets[i].Count; j++)
{
array[arrayIndex++] = buckets[i][j];
}
}
}
I am a bit confused, because the algorithm itself looks fine, that is I have to calculate array[i] * countOfElements to get the index where I can put my element. Could you please direct me?
If you array contain value '1' it will lead to OutOfRange, because if you have a size of the list equal to 3 (for example) then valid indexes will be in the range [0, 2].
Basically i'm trying to multiply each element of the first array with each element of the second array and then store it all at the end as a total. I'm pretty new to coding so just trying to learn and this one really has me stuck. This is the example of what it should eventually do.
ExampleArray1 = 5,6,7,8
ExampleArray2 = 2,3,4
(5*2)+(5*3)+(5*4) + (6*2)+(6*3)+(6*4) + (7*2)+(7*3)+(7*4) + (8*2)+(8*3)+(8*4) = 234
My code
int[] firstArray = { 5, 6, 7, 8 };
int[] secondArray = { 2, 3, 4 };
int[] thirdArray = new int[firstArray.Length * secondArray.Length];
for (int i = 0; i < firstArray.Length; i++)
for (int j = 0; j < secondArray.Length; j++)
{
thirdArray[i * firstArray.Length + j] = firstArray[i] * secondArray[j];
Console.WriteLine(thirdArray[i * firstArray.Length + j]);
}
You dont need a third array, you can just sum the results
var total = 0;
for (int i = 0; i < firstArray.Length; i++)
for (int j = 0; j < secondArray.Length; j++)
{
total += (firstArray[i] * secondArray[j]);
}
Console.WriteLine(total);
However you forgot to minus one form the length, in the third array index.
i.e to get the index you need i * (firstArray.Length - 1) + j
int[] thirdArray = new int[firstArray.Length * secondArray.Length];
for (int i = 0; i < firstArray.Length; i++)
for (int j = 0; j < secondArray.Length; j++)
{
thirdArray[i * (firstArray.Length - 1) + j] = firstArray[i] * secondArray[j];
}
Console.WriteLine(thirdArray.Sum());
You can apply some basic algebra to simplify this:
var total = 0;
var array1Total = 0;
var array2Total = 0;
for (int i = 0; i < firstArray.Length; i++)
{
array1Total += firstArray[i];
}
for (int j = 0; j < secondArray.Length; j++)
{
array2Total += secondArray[j];
}
total = array1Total * array2Total;
Console.WriteLine(total);
The reason is that if you expand (x0+x1+x2+x3...)*(y0+y1+y2+...) then you can see that you will multiply x0 by each of the y, then multiply x1 by each of the y, and so on. So you'll get each of the elements in the first brackets multiplied by each of the elements in the second brackets.
While this may not seem much different it will be considerably better for large arrays. if the length of your arrays are m and n then by the nested loops method you'll have m*n loop iterations. With the above technique you will have m+n. For small values this isn't a big deal. If you have arrays of thousands of items then the above will be significantly faster.
Of course, there is an even easier way to do the above:
var total = firstArray.Sum()*secondArray.Sum();
How can i make this so the user types positions in console (table[4,5]) i want that user types that?
int[,] table = new int[8, 8];
for (int i = 0; i < 8; i++)
{
for (int j = 0; j < 8; j++)
{
if ((i + j) % 2 == 0)
{
table[i, j] = 0;
}
else
{
table[i, j] = 1;
}
}
}
Console.WriteLine("4 - king");
Console.WriteLine("3 - queen");
Console.WriteLine("4 - hunter");
table[4,5] = 2;
table[6,7] = 3;
table[2,2] = 4;
for (int i = 0; i < 8; i++)
{
for (int j = 0; j < 8; j++)
{
Console.Write(table[i, j] + " ");
}
Console.WriteLine();
}
What should i do to make this work?And if i type this it doesnt work:
Thats why i have to put figures in different rows or columns how do i fix this
table[4,5] = 2;
table[4,7] = 3;
table[2,2] = 4;
You are confusing the assignment operator "=" with the logical comparison operator "==". Your second line is just comparing table[4,5] with 2 and probably returning false.
Change it to:
table[4,5] = 2;
Also, even if you manage to assign a value to table[4,5], you will overwrite it in the next lines. You should move that line to the end of the first nested loop. Just before the second "for (int i = 0; i < 8; i++)"
I've run into a stall trying to put together some code to average out 10x10 subarrays of a 2D multidimensional array.
Given a multidimensional array
var myArray = new byte[100, 100];
How should I go about creating 100 subarrays of 100 bytes (10x10) each.
Here are some examples of the value indexes the subarrays from the multidimensional would contain.
[x1,y1,x2,y2]
Subarray1[0,0][9,9]
Subarray2[10,10][19,19]
Subarray3[20,20][29,29]
Given these subarrays, I would then need to average the subarray values to create a byte[10,10] from the original byte[100,100].
I realize this is not unbelievably difficult, but after spending 4 days debugging very low-level code and now getting stuck on this would appreciate some fresh eyes.
Use this as a reference. I used ints just for ease of use. Code is untested. but the idea is there.
var rowSize = 100;
var colSize = 100;
var arr = new int[rowSize, colSize];
var r = new Random();
for (int i = 0; i < rowSize; i++)
for (int j = 0; j < colSize; j++)
arr[i, j] = r.Next(20);
for (var subcol = 0; subcol < colSize / 10; subcol++)
{
for (var subrow = 0; subrow < colSize/10; subrow++)
{
var startX = subcol*10;
var startY = subrow*10;
var avg = 0;
for (var x=0; x<10; x++)
for (var y = 0; y < 10; y++)
avg += arr[startX + x, startY + y];
avg /= 10*10;
Console.WriteLine(avg);
}
}
It looks like you're new to SO. Next time try to post your attempt at the problem; it's better to fix your code.
The only challenge is figuring out the function, that given the subarray index we're trying to populate, would give you the correct row and column indexes in your original 100x100 array; the rest would just be a matter of copying the values:
// psuedocode
// given a subarrayIndex of 0 to 99, these will calculate the correct indices
rowIndexIn100x100Array = (subarrayIndex / 10) * 10 + subArrayRowIndexToPopulate;
colIndexIn100x100Array = (subarrayIndex % 10) * 10 + subArrayColIndexToPopulate;
I'll leave it as an exercise to you to deduce why the above functions correctly calculate the indices.
With the above, we can easily map the values:
var subArrays = new List<byte[,]>();
for (int subarrayIndex = 0; subarrayIndex < 100; subarrayIndex++)
{
var subarray = new byte[10, 10];
for (int i = 0; i < 10; i++)
for (int j = 0; j < 10; j++)
{
int rowIndexIn100x100Array = (subarrayIndex / 10) * 10 + i;
int colIndexIn100x100Array = (subarrayIndex % 10) * 10 + j;
subarray[i, j] = originalArray[rowIndexIn100x100Array, colIndexIn100x100Array];
}
subArrays.Add(subarray);
}
Once we have the 10x10 arrays, calculating the average would be trivial using LINQ:
var averages = new byte[10, 10];
for (int i = 0; i < 10; i++)
for (int j = 0; j < 10; j++)
{
averages[i, j] = (byte)subArrays[(i * 10) + j].Cast<byte>().Average(b => b);
}
Fiddle.
I come looking for general tips about the program I'm writing now.
The goal is:
Use neural network program to recognize 3 letters [D,O,M] (or display "nothing is recognized" if i input anything other than those 3).
Here's what I have so far:
A class for my single neuron
public class neuron
{
double[] weights;
public neuron()
{
weights = null;
}
public neuron(int size)
{
weights = new double[size + 1];
Random r = new Random();
for (int i = 0; i <= size; i++)
{
weights[i] = r.NextDouble() / 5 - 0.1;
}
}
public double output(double[] wej)
{
double s = 0.0;
for (int i = 0; i < weights.Length; i++) s += weights[i] * wej[i];
s = 1 / (1 + Math.Exp(s));
return s;
}
}
A class for a layer:
public class layer
{
neuron[] tab;
public layer()
{
tab = null;
}
public layer(int numNeurons, int numInputs)
{
tab = new neuron[numNeurons];
for (int i = 0; i < numNeurons; i++)
{
tab[i] = new neuron(numInputs);
}
}
public double[] compute(double[] wejscia)
{
double[] output = new double[tab.Length + 1];
output[0] = 1;
for (int i = 1; i <= tab.Length; i++)
{
output[i] = tab[i - 1].output(wejscia);
}
return output;
}
}
And finally a class for a network
public class network
{
layer[] layers = null;
public network(int numLayers, int numInputs, int[] npl)
{
layers = new layer[numLayers];
for (int i = 0; i < numLayers; i++)
{
layers[i] = new layer(npl[i], (i == 0) ? numInputs : (npl[i - 1]));
}
}
double[] compute(double[] inputs)
{
double[] output = layers[0].compute(inputs);
for (int i = 1; i < layers.Length; i++)
{
output = layers[i].compute(output);
}
return output;
}
}
Now for the algorythm I chose:
I have a picture box, size 200x200, where you can draw a letter (or read one from jpg file).
I then convert it to my first array(get the whole picture) and 2nd one(cut the non relevant background around it) like so:
Bitmap bmp2 = new Bitmap(this.pictureBox1.Image);
int[,] binaryfrom = new int[bmp2.Width, bmp2.Height];
int minrow=0, maxrow=0, mincol=0, maxcol=0;
for (int i = 0; i < bmp2.Height; i++)
{
for (int j = 0; j < bmp2.Width; j++)
{
if (bmp2.GetPixel(j, i).R == 0)
{
binaryfrom[i, j] = 1;
if (minrow == 0) minrow = i;
if (maxrow < i) maxrow = i;
if (mincol == 0) mincol = j;
else if (mincol > j) mincol = j;
if (maxcol < j) maxcol = j;
}
else
{
binaryfrom[i, j] = 0;
}
}
}
int[,] boundaries = new int[binaryfrom.GetLength(0)-minrow-(binaryfrom.GetLength(0)-(maxrow+1)),binaryfrom.GetLength(1)-mincol-(binaryfrom.GetLength(1)-(maxcol+1))];
for(int i = 0; i < boundaries.GetLength(0); i++)
{
for(int j = 0; j < boundaries.GetLength(1); j++)
{
boundaries[i, j] = binaryfrom[i + minrow, j + mincol];
}
}
And convert it to my final array of 12x8 like so (i know I could shorten this a fair bit, but wanted to have every step in different loop so I can see what went wrong easier[if anything did]):
int[,] finalnet = new int[12, 8];
int k = 1;
int l = 1;
for (int i = 0; i < finalnet.GetLength(0); i++)
{
for (int j = 0; j < finalnet.GetLength(1); j++)
{
finalnet[i, j] = 0;
}
}
while (k <= finalnet.GetLength(0))
{
while (l <= finalnet.GetLength(1))
{
for (int i = (int)(boundaries.GetLength(0) / finalnet.GetLength(0)) * (k - 1); i < (int)(boundaries.GetLength(0) / finalnet.GetLength(0)) * k; i++)
{
for (int j = (int)(boundaries.GetLength(1) / finalnet.GetLength(1)) * (l - 1); j < (int)(boundaries.GetLength(1) / finalnet.GetLength(1)) * l; j++)
{
if (boundaries[i, j] == 1) finalnet[k-1, l-1] = 1;
}
}
l++;
}
l = 1;
k++;
}
int a = boundaries.GetLength(0);
int b = finalnet.GetLength(1);
if((a%b) != 0){
k = 1;
while (k <= finalnet.GetLength(1))
{
for (int i = (int)(boundaries.GetLength(0) / finalnet.GetLength(0)) * finalnet.GetLength(0); i < boundaries.GetLength(0); i++)
{
for (int j = (int)(boundaries.GetLength(1) / finalnet.GetLength(1)) * (k - 1); j < (int)(boundaries.GetLength(1) / finalnet.GetLength(1)) * k; j++)
{
if (boundaries[i, j] == 1) finalnet[finalnet.GetLength(0) - 1, k - 1] = 1;
}
}
k++;
}
}
if (boundaries.GetLength(1) % finalnet.GetLength(1) != 0)
{
k = 1;
while (k <= finalnet.GetLength(0))
{
for (int i = (int)(boundaries.GetLength(0) / finalnet.GetLength(0)) * (k - 1); i < (int)(boundaries.GetLength(0) / finalnet.GetLength(0)) * k; i++)
{
for (int j = (int)(boundaries.GetLength(1) / finalnet.GetLength(1)) * finalnet.GetLength(1); j < boundaries.GetLength(1); j++)
{
if (boundaries[i, j] == 1) finalnet[k - 1, finalnet.GetLength(1) - 1] = 1;
}
}
k++;
}
for (int i = (int)(boundaries.GetLength(0) / finalnet.GetLength(0)) * finalnet.GetLength(0); i < boundaries.GetLength(0); i++)
{
for (int j = (int)(boundaries.GetLength(1) / finalnet.GetLength(1)) * finalnet.GetLength(1); j < boundaries.GetLength(1); j++)
{
if (boundaries[i, j] == 1) finalnet[finalnet.GetLength(0) - 1, finalnet.GetLength(1) - 1] = 1;
}
}
}
The result is a 12x8 (I can change it in the code to get it from some form controls) array of 0 and 1, where 1 form the rough shape of a letter you drawn.
Now my questions are:
Is this a correct algorythm?
Is my function
1/(1+Math.Exp(x))
good one to use here?
What should be the topology? 2 or 3 layers, and if 3, how many neurons in hidden layer? I have 96 inputs (every field of the finalnet array), so should I also take 96 neurons in the first layer? Should I have 3 neurons in the final layer or 4(to take into account the "not recognized" case), or is it not necessary?
Thank you for your help.
EDIT: Oh, and I forgot to add, I'm gonna train my network using Backpropagation algorythm.
You may need 4 layers at least to get accurate results using back propagation method. 1 input, 2 middle layers, and an output layer.
12 * 8 matrix is too small(and you may end up in data loss which will result in total failure) - try some thing 16 * 16. If you want to reduce the size then you have to peel out the outer layers of black pixels further.
Think about training the network with your reference characters.
Remember that you have to feed back the output back to the input layer again and iterate it multiple times.
A while back I created a neural net to recognize digits 0-9 (python, sorry), so based on my (short) experience, 3 layers are ok and 96/50/3 topology will probably do a good job. As for the output layer, it's your choice; you can either backpropagate all 0s when the input image is not a D, O or M or use the fourth output neuron to indicate that the letter was not recognized. I think that the first option would be the best one because it's simpler (shorter training time, less problems debugging the net...), you just need to apply a threshold under which you classify the image as 'not recognized'.
I also used the sigmoid as activation function, I didn't try others but it worked :)