Develop Reference

How to use self defined aggregate function with LINQ? (without using extension methods)

In the question Calculate the sum of numbers from 1 to n using LINQ, the function sum was preexisting, what about when we want to define our own function ? for example SquareSum function, where the the square of numbers are to be added, or some other non preexisting function?
Edit : the use of extension method was commented, I was actually looking for a way to do it without extension methods.

You can write it like this
IEnumerable<int> range = Enumerable.Range(1, n);
var sum = range.Aggregate(0, (x, y) => x + y);
Sum of squares:
IEnumerable<int> range = Enumerable.Range(1, n);
var sum = range.Aggregate(0, (x, y) => x + y * y);

Please try the below one.
If you have already an IEnumerable type which contains the numbers, you don't need any extension methods. You can directly use the Aggregate extention method like below.
num.Aggregate((a, b) => (a * a) + (b * b));
Edit: Folks, My apologies for the above wrong answer (num.Aggregate((a, b) => (a * a) + (b * b));). I just posted it without trying. One gentleman already posted the answer which is the correct one. However I find one more way for the same output
num.Select(x => x * x).Sum();

Why is HashSet<Point> so much slower than HashSet<string>?

I wanted to store some pixels locations without allowing duplicates, so the first thing comes to mind is HashSet<Point> or similar classes. However this seems to be very slow compared to something like HashSet<string>.
For example, this code:
HashSet<Point> points = new HashSet<Point>();
using (Bitmap img = new Bitmap(1000, 1000))
{
for (int x = 0; x < img.Width; x++)
{
for (int y = 0; y < img.Height; y++)
{
points.Add(new Point(x, y));
}
}
}
takes about 22.5 seconds.
While the following code (which is not a good choice for obvious reasons) takes only 1.6 seconds:
HashSet<string> points = new HashSet<string>();
using (Bitmap img = new Bitmap(1000, 1000))
{
for (int x = 0; x < img.Width; x++)
{
for (int y = 0; y < img.Height; y++)
{
points.Add(x + "," + y);
}
}
}
So, my questions are:
Is there a reason for that? I checked this answer, but 22.5 sec is way more than the numbers shown in that answer.
Is there a better way to store points without duplicates?

There are two perf problems induced by the Point struct. Something you can see when you add Console.WriteLine(GC.CollectionCount(0)); to the test code. You'll see that the Point test requires ~3720 collections but the string test only needs ~18 collections. Not for free. When you see a value type induce so many collections then you need to conclude "uh-oh, too much boxing".
At issue is that HashSet<T> needs an IEqualityComparer<T> to get its job done. Since you did not provide one, it needs to fall back to one returned by EqualityComparer.Default<T>(). That method can do a good job for string, it implements IEquatable. But not for Point, it is a type that harks from .NET 1.0 and never got the generics love. All it can do is use the Object methods.
The other issue is that Point.GetHashCode() does not do a stellar job in this test, too many collisions, so it hammers Object.Equals() pretty heavily. String has an excellent GetHashCode implementation.
You can solve both problems by providing the HashSet with a good comparer. Like this one:
class PointComparer : IEqualityComparer<Point> {
public bool Equals(Point x, Point y) {
return x.X == y.X && x.Y == y.Y;
}
public int GetHashCode(Point obj) {
// Perfect hash for practical bitmaps, their width/height is never >= 65536
return (obj.Y << 16) ^ obj.X;
}
}
And use it:
HashSet<Point> list = new HashSet<Point>(new PointComparer());
And it is now about 150 times faster, easily beating the string test.

The main reason for the performance drop is all the boxing going on (as already explained in Hans Passant's answer).
Apart from that, the hash code algorithm worsens the problem, because it causes more calls to Equals(object obj) thus increasing the amount of boxing conversions.
Also note that the hash code of Point is computed by x ^ y. This produces very little dispersion in your data range, and therefore the buckets of the HashSet are overpopulated — something that doesn't happen with string, where the dispersion of the hashes is much larger.
You can solve that problem by implementing your own Point struct (trivial) and using a better hash algorithm for your expected data range, e.g. by shifting the coordinates:
(x << 16) ^ y
For some good advice when it comes to hash codes, read Eric Lippert's blog post on the subject.

Can't get cost function for logistic regression to work

I'm trying to implement logistic regression by myself writing code in C#. I found a library (Accord.NET) that I use to minimize the cost function. However I'm always getting different minimums. Therefore I think something may be wrong in the cost function that I wrote.
static double costfunction(double[] thetas)
{
int i = 0;
double sum = 0;
double[][] theta_matrix_transposed = MatrixCreate(1, thetas.Length);
while(i!=thetas.Length) { theta_matrix_transposed[0][i] = thetas[i]; i++;}
i = 0;
while (i != m) // m is the number of examples
{
int z = 0;
double[][] x_matrix = MatrixCreate(thetas.Length, 1);
while (z != thetas.Length) { x_matrix[z][0] = x[z][i]; z++; } //Put values from the training set to the matrix
double p = MatrixProduct(theta_matrix_transposed, x_matrix)[0][0];
sum += y[i] * Math.Log(sigmoid(p)) + (1 - y[i]) * Math.Log(1 - sigmoid(p));
i++;
}
double value = (-1 / m) * sum;
return value;
}
static double sigmoid(double z)
{
return 1 / (1 + Math.Exp(-z));
}
x is a list of lists that represent the training set, one list for each feature. What's wrong with the code? Why am I getting different results every time I run the L-BFGS? Thank you for your patience, I'm just getting started with machine learning!

That is very common with these optimization algorithms - the minima you arrive at depends on your weight initialization. The fact that you are getting different minimums doesn't necessarily mean something is wrong with your implementation. Instead, check your gradients to make sure they are correct using the finite differences method, and also look at your train/validation/test accuracy to see if they are also acceptable.

Is there a better performing functional version of this iterative algorithm in C#?

I was hoping to figure out a way to write the below in a functional style with extension functions. Ideally this functional style would perform well compared to the iterative/loop version. I'm guessing that there isn't a way. Probably because of the many additional function calls and stack allocations, etc.
Fundamentally I think the pattern which is making it troublesome is that it both calculates a value to use for the Predicate and then needs that calculated value again as part of the resulting collection.
// This is what is passed to each function.
// Do not assume the array is in order.
var a = (0).To(999999).ToArray().Shuffle();
// Approx times in release mode (on my machine):
// Functional is avg 20ms per call
// Iterative is avg 5ms per call
// Linq is avg 14ms per call
private static List<int> Iterative(int[] a)
{
var squares = new List<int>(a.Length);
for (int i = 0; i < a.Length; i++)
{
var n = a[i];
if (n % 2 == 0)
{
int square = n * n;
if (square < 1000000)
{
squares.Add(square);
}
}
}
return squares;
}
private static List<int> Functional(int[] a)
{
return
a
.Where(x => x % 2 == 0 && x * x < 1000000)
.Select(x => x * x)
.ToList();
}
private static List<int> Linq(int[] a)
{
var squares =
from num in a
where num % 2 == 0 && num * num < 1000000
select num * num;
return squares.ToList();
}

An alternative to Konrad's suggestion. This avoids the double calculation, but also avoids even calculating the square when it doesn't have to:
return a.Where(x => x % 2 == 0)
.Select(x => x * x)
.Where(square => square < 1000000)
.ToList();
Personally, I wouldn't sweat the difference in performance until I'd seen it be significant in a larger context.
(I'm assuming that this is just an example, by the way. Normally you'd possibly compute the square root of 1000000 once and then just compare n with that, to shave off a few milliseconds. It does require two comparisons or an Abs operation though, of course.)
EDIT: Note that a more functional version would avoid using ToList at all. Return IEnumerable<int> instead, and let the caller transform it into a List<T> if they want to. If they don't, they don't take the hit. If they only want the first 5 values, they can call Take(5). That laziness could be a big performance win over the original version, depending on the context.

Just solving your problem of the double calculation:
return (from x in a
let sq = x * x
where x % 2 == 0 && sq < 1000000
select sq).ToList();
That said, I’m not sure that this will lead to much performance improvement. Is the functional variant actually noticeably faster than the iterative one? The code offers quite a lot of potential for automated optimisation.

How about some parallel processing? Or does the solution have to be LINQ (which I consider to be slow).
var squares = new List<int>(a.Length);
Parallel.ForEach(a, n =>
{
if(n < 1000 && n % 2 == 0) squares.Add(n * n);
}
The Linq version would be:
return a.AsParallel()
.Where(n => n < 1000 && n % 2 == 0)
.Select(n => n * n)
.ToList();

I don't think there's a functional solution that will be completely on-par with the iterative solution performance-wise. In my timings (see below) the 'functional' implementation from the OP appears to be around twice as slow as the iterative implementation.
Micro-benchmarks like this one are prone to all manner of issues. A common tactic in dealing with variability problems is to repeatedly call the method being timed and compute an average time per call - like this:
// from main
Time(Functional, "Functional", a);
Time(Linq, "Linq", a);
Time(Iterative, "Iterative", a);
// ...
static int reps = 1000;
private static List<int> Time(Func<int[],List<int>> func, string name, int[] a)
{
var sw = System.Diagnostics.Stopwatch.StartNew();
List<int> ret = null;
for(int i = 0; i < reps; ++i)
{
ret = func(a);
}
sw.Stop();
Console.WriteLine(
"{0} per call timings - {1} ticks, {2} ms",
name,
sw.ElapsedTicks/(double)reps,
sw.ElapsedMilliseconds/(double)reps);
return ret;
}
Here are the timings from one session:
Functional per call timings - 46493.541 ticks, 16.945 ms
Linq per call timings - 46526.734 ticks, 16.958 ms
Iterative per call timings - 21971.274 ticks, 8.008 ms
There are a host of other challenges as well: strobe-effects with the timer use, how and when the just-in-time compiler does its thing, the garbage collector running its collections, the order that competing algorithms are run, the type of cpu, the OS swapping other processes in and out, etc.
I tried my hand at a little optimization. I removed the square from the test (num * num < 1000000) - changing it to (num < 1000) - which seemed safe since there are no negatives in the input - that is, I took the square root of both sides of the inequality. Surprisingly, I got different results as compared to the methods in the OP - there were only 500 items in my optimized output as compared to the 241,849 from the three implementations in the OP implementations. So why the difference? Much of the input when squared overflows 32 bit integers, so those extra 241,349 items came from numbers that when squared overflowed to either negative numbers or numbers under 1 million while still passing our evenness test.
optimized (functional) timing:
Optimized per call timings - 16849.529 ticks, 6.141 ms
This was one of the functional implementations altered as suggested. It output the 500 items passing the criteria as expected. It is deceptively "faster" only because it output fewer items than the iterative solution.
We can make the original implementations blow up with an OverflowException by adding a checked block around their implementations. Here is a checked block added to the "Iterative" method:
private static List<int> Iterative(int[] a)
{
checked
{
var squares = new List<int>(a.Length);
// rest of method omitted for brevity...
return squares;
}
}

What is wrong with this fourier transform implementation

I'm trying to implement a discrete fourier transform, but it's not working. I'm probably have written a bug somewhere, but I haven't found it yet.
Based on the following formula:
This function does the first loop, looping over X0 - Xn-1...
public Complex[] Transform(Complex[] data, bool reverse)
{
var transformed = new Complex[data.Length];
for(var i = 0; i < data.Length; i++)
{
//I create a method to calculate a single value
transformed[i] = TransformSingle(i, data, reverse);
}
return transformed;
}
And the actual calculating, this is probably where the bug is.
private Complex TransformSingle(int k, Complex[] data, bool reverse)
{
var sign = reverse ? 1.0: -1.0;
var transformed = Complex.Zero;
var argument = sign*2.0*Math.PI*k/data.Length;
for(var i = 0; i < data.Length; i++)
{
transformed += data[i]*Complex.FromPolarCoordinates(1, argument*i);
}
return transformed;
}
Next the explaination of the rest of the code:
var sign = reverse ? 1.0: -1.0; The reversed DFT will not have -1 in the argument, while a regular DFT does have a -1 in the argument.
var argument = sign*2.0*Math.PI*k/data.Length; is the argument of the algorithm. This part:
then the last part
transformed += data[i]*Complex.FromPolarCoordinates(1, argument*i);
I think I carefully copied the algorithm, so I don't see where I made the mistake...
Additional information
As Adam Gritt has shown in his answer, there is a nice implementation of this algorithm by AForge.net. I can probably solve this problem in 30 seconds by just copying their code. However, I still don't know what I have done wrong in my implementation.
I'm really curious where my flaw is, and what I have interpreted wrong.

My days of doing complex mathematics are a ways behind me right now so I may be missing something myself. However, it appears to me that you are doing the following line:
transformed += data[i]*Complex.FromPolarCoordinates(1, argument*i);
when it should probably be more like:
transformed += data[i]*Math.Pow(Math.E, Complex.FromPolarCoordinates(1, argument*i));
Unless you have this wrapped up into the method FromPolarCoordinates()
UPDATE:
I found the following bit of code in the AForge.NET Framework library and it shows additional Cos/Sin operations being done that are not being handled in your code. This code can be found in full context in the Sources\Math\FourierTransform.cs: DFT method.
for ( int i = 0; i < n; i++ )
{
dst[i] = Complex.Zero;
arg = - (int) direction * 2.0 * System.Math.PI * (double) i / (double) n;
// sum source elements
for ( int j = 0; j < n; j++ )
{
cos = System.Math.Cos( j * arg );
sin = System.Math.Sin( j * arg );
dst[i].Re += ( data[j].Re * cos - data[j].Im * sin );
dst[i].Im += ( data[j].Re * sin + data[j].Im * cos );
}
}
It is using a custom Complex class (as it was pre-4.0). Most of the math is similar to what you have implemented but the inner iteration is doing additional mathematical operations on the Real and Imaginary portions.
FURTHER UPDATE:
After some implementation and testing I found that the code above and the code provided in the question produce the same results. I also found, based on the comments what the difference is between what is generated from this code and what is produced by WolframAlpha. The difference in the results is that it would appear that Wolfram is applying a normalization of 1/sqrt(N) to the results. In the Wolfram Link provided if each value is multiplied by Sqrt(2) then the values are the same as those generated by the above code (rounding errors aside). I tested this by passing 3, 4, and 5 values into Wolfram and found that my results were different by Sqrt(3), Sqrt(4) and Sqrt(5) respectfully. Based on the Discrete Fourier Transform information provided by wikipedia it does mention a normalization to make the transforms for DFT and IDFT unitary. This might be the avenue that you need to look down to either modify your code or understand what Wolfram may be doing.

Your code is actually almost right (you are missing an 1/N on the inverse transform). The thing is, the formula you used is typically used for computations because it's lighter, but on purely theorical environments (and in Wolfram), you would use a normalization by 1/sqrt(N) to make the transforms unitary.
i.e. your formulas would be:
Xk = 1/sqrt(N) * sum(x[n] * exp(-i*2*pi/N * k*n))
x[n] = 1/sqrt(N) * sum(Xk * exp(i*2*pi/N * k*n))
It's just a matter of convention in normalisation, only amplitudes change so your results weren't that bad (if you hadn't forgotten the 1/N in the reverse transform).
Cheers