Develop Reference

Possibilities to improve performance using vectorization for the following function in C#?

I have a function that estimates correlation between two input arrays.
The input is feeded by a dataDict which is of type Dictionary<string, double[]> which has 153 keys with values as double array of size 1500.
For each individual key, I need to estimate its correlation with all other keys and store the result to a double[,] that has a size of double[dataDict.Count(), dataDict.Count()]
The following function prepares two double[] arrays whose correlation needs to be estimated.
public double[,] CalculateCorrelation(Dictionary<string, double?[]> dataDict, string corrMethod = "kendall")
{
CorrelationLogicModule correlationLogicModule = new CorrelationLogicModule();
double[,] correlationMatrix = new double[dataDict.Count(), dataDict.Count()];
for (int i = 0; i < dataDict.Count; i++)
{
for (int j = 0; j < dataDict.Count; j++)
{
var arrayA = dataDict[dataDict.ElementAt(i).Key].Cast<double>().ToArray();
var arrayB = dataDict[dataDict.ElementAt(j).Key].Cast<double>().ToArray();
correlationMatrix[i, j] = correlationLogicModule.Kendall_Formula(arrayA, arrayB);
}
}
return correlationMatrix;
}
The following function (I found it on internet from here) finds correlation between two input arrays using 'Kendall's' method.
public double Kendall_Formula(double[] Ticker1, double[] Ticker2)
{
double NbrConcord, NbrDiscord, S;
NbrConcord = 0;
NbrDiscord = 0;
S = 0;
for (int i = 0; i < Ticker1.Length - 1; i++)
{
for (int j = i + 1; j < Ticker1.Length; j++)
{
//Compute the number of concordant pairs
if (((Ticker1[i] < Ticker1[j]) & (Ticker2[i] < Ticker2[j])) | ((Ticker1[i] > Ticker1[j]) & (Ticker2[i] > Ticker2[j])))
{
NbrConcord++;
}
//Compute the number of discordant pairs
else if (((Ticker1[i] > Ticker1[j]) & (Ticker2[i] < Ticker2[j])) | ((Ticker1[i] < Ticker1[j]) & (Ticker2[i] > Ticker2[j])))
{
NbrDiscord++;
}
}
}
S = NbrConcord - NbrDiscord;
//Proportion with the total pairs
return 2 * S / (Ticker1.Length * (Ticker1.Length - 1));
}
Moving this way forward, takes a very long time to calculate the correlations for all the keys.
is there a possible way to optimize the performance?.
I am new to C# but I have been using Python for a long time and in Python using 'Numpys' and 'Pandas' I am sure the above operation would take seconds to compute. For e.g. lets say I had the above data in form of a pandas dataframe, then data[[list of columns]].corr('method') would lead the result in seconds. This is because pandas uses numpy under the hood which takes benefit from vectorization. I would like to learn how can I take benefit from vectorization to improve the performance of the above code in C# and if there are other factors I need to consider. Thank you!

You are using dataDict[dataDict.ElementAt(i).Key] to access the dictionary values in an undefined order. I don't know if that's what you intended, but the following code should give the the same results.
If you call dataDict.Values.ToArray(); you will get the dictionary values in the same order as you would when using foreach to iterate over it. That means that it will be the same as the order when using dataDict[dataDict.ElementAt(i).Key].
Therefore this code should be equivalent, and it should be faster:
public double[,] CalculateCorrelation(Dictionary<string, double?[]> dataDict, string corrMethod = "kendall")
{
CorrelationLogicModule correlationLogicModule = new CorrelationLogicModule();
var values = dataDict.Values.Select(array => array.Cast<double>().ToArray()).ToArray();
double[,] correlationMatrix = new double[dataDict.Count, dataDict.Count];
for (int i = 0; i < dataDict.Count; i++)
{
for (int j = 0; j < dataDict.Count; j++)
{
var arrayA = values[i];
var arrayB = values[j];
correlationMatrix[i, j] = correlationLogicModule.Kendall_Formula(arrayA, arrayB);
}
}
return correlationMatrix;
}
Note that the .ElementAt() call in your original code is a Linq extension, not a member of Dictionary<TKey,TValue>. It iterates from the start of the dictionary EVERY TIME you call it - and it also returns items in an unspecified order. From the documentation: For purposes of enumeration, each item in the dictionary is treated as a KeyValuePair<TKey,TValue> structure representing a value and its key. The order in which the items are returned is undefined.
Also:
You should change the bitwise & to logical && in your conditions. The use of & will prevent the compiler applying a boolean short-circuit optimisation, meaning that all the < / > comparisons will be performed, even if the first condition is false.

Can I have an if statement change what variable is accessed in a loop

Hey I'm in a situation where I have a for loop that does some stuff and I want to make a line of code either call a function passing in an array indexed by the for loops index, or run a single (not array) variable for every call of that function, I know I could do that by putting an if statement inside the for loop but i'd be repeating the same if statement over and over getting the same result. So is there a way good way I can run the if statement before the for loop and the result of that if statement run the same for loop but that one call passes in the array or the variable?
Code Example
for (int i = 0; i < CurrentVerticalList.Count; i++)
{
GuiGeneral CGroup = CurrentVerticalList[i];
CGroup.ResizeUsingStandard(ForcedResize[i]); //I want the condition before the for
//loop to have ForcedResize[i] here if
//true and another variable here of the
//same type but not an array if false.
for (int j = 0; j < 2; j++)
{
GlobalListIndex[j]++;
}
CGroup.MoveElementTo(CCoord, false);
CCoord.y += CGroup.ElementRect.WidthHeight.y;
}

Here you go, moving your condition check out of your for loop:
Func<int, double> GetResizeFromForcedResize = (index => ForcedResize[index]);
Func<int, double> GetResizeFromVariable = (index => fixVariable);
var GetResizeValue = condition? GetResizeFromForcedResize : GetResizeFromVariable;
for (int i = 0; i < CurrentVerticalList.Count; i++)
{
GuiGeneral CGroup = CurrentVerticalList[i];
CGroup.ResizeUsingStandard(GetResizeValue(i));
for (int j = 0; j < 2; j++)
{
GlobalListIndex[j]++;
}
CGroup.MoveElementTo(CCoord, false);
CCoord.y += CGroup.ElementRect.WidthHeight.y;
}
Edit: Wanted to let you know that the other answers here are still doing a check at every iteration, but not this one.

Eh, hard to understand the question to me but i recon what you want is something along these lines, could've helped with more types supplied, but you could make the intend achievable using a local function:
ForcedResizeArrayType other = new object(); //TODO: Define return type
bool condition = ResolveCondition(); //TODO: Define condition to be true or false
ForcedResizeArrayType GetOneOr(int i, bool condition,
ForcedResizeArrayType[] forcedResizeArray)
{
return condition ? forcedResizeArray[i] : other;
}
for (int i = 0; i < CurrentVerticalList.Count; i++)
{
CGroup.ResizeUsingStandard(GetOneOr(i, condition, ForcedResize));
}
It varies from week to week if i love or hate those local functions, but they have uses

Wrapping C# primitives in struct performance implications

I am writing some code on geometry processing, delaunay triangulation to be more specific, and I need it to be fast, so I use simple arrays of primitive as data structure to represent my triangulation information, here is a sample of it
private readonly float2[] points;
private readonly int[] pointsHalfEdgeStartCount;
private readonly int[] pointsIncomingHalfEdgeIndexes;
So let's say I want to iterate fast through all the incoming half-edge of the point of index p, I just do this using the precomputed arrays:
int count = pointsHalfEdgeStartCount[p * 2 + 1];
for (int i = 0; i < count; i++)
{
var e = pointsIncomingHalfEdgeIndexes[pointsHalfEdgeStartCount[p * 2] + i]
}
// pointsHalfEdgeStartCount[p * 2] is the start index
And this is fast enought, but does not feel safe or very clear. So I had the idea of wrapping my index into struct to make it clearer while retaining the performance, somthing like that:
public readonly struct Point
{
public readonly int index;
public readonly DelaunayTriangulation delaunay
public Point(int index, DelaunayTriangulation delaunay)
{
this.index = index;
this.delaunay = delaunay;
}
public int GetIncomingHalfEdgeCount() => delaunay.pointsEdgeStartCount[index * 2 + 1];
public HalfEdge GetIncomingHalfEdge(int i)
{
return new HalfEdge(
delaunay,
delaunay.pointsIncomingHalfEdgeIndexes[delaunay.pointsEdgeStartCount[index * 2] + i]
);
}
//... other methods
}
Then I can just do so:
int count = p.GetIncomingHalfEdgeCount();
for (int i = 0; i < count; i++)
{
var e = p.GetIncomingHalfEdge(i);
}
However it was kind of killing my performance, being a lot slower (around 10 times) on a benchmark I did, iterating over all the points and iterating over all their incoming half-edge. I guess because storing a reference to the delaunay triangulaiton in each point struct was an obvious waste and slowed down all the operations involving points, having twice the amount of data to move.
I could make the DelaunayTriangulation a static class but it was not practical for other reasons, so I did that:
public readonly struct Point
{
public readonly int index;
public Point(int index) => this.index = index;
public int GetIncomingHalfEdgeCount(DelaunayTriangulation delaunay) => delaunay.pointsEdgeStartCount[index * 2 + 1];
public HalfEdge GetIncomingHalfEdge(DelaunayTriangulation delaunay, int i)
{
return new HalfEdge(
delaunay.pointsIncomingHalfEdgeIndexes[delaunay.pointsEdgeStartCount[index * 2] + i]
);
}
//... other methods
}
I can just do so:
int count = p.GetIncomingHalfEdgeCount(delaunay);
for (int i = 0; i < count; i++)
{
var e = p.GetIncomingHalfEdge(delaunay, i);
}
It was quite a lot faster, but still 2.5 times slower than the first method using simple int. I wondered if it could be because I was getting int in the first method while I got HalfEdge struct in the other methods (A struct similar to the Point struct, contains only an index as data and a couple of methods), and difference between plain int and the faster struct vanished when I used the e int to instantiate a new HalfEdge struct. Though I am not sure why is that so costly.Weirder still, I explored for clarity sake the option of wrinting the method inside the Delaunay class instead of the Point struct:
// In the DelaunayTriangulation class:
public int GetPointIncomingHalfEdgeCount(Point p) => pointsEdgeStartCount[p.index * 2 + 1];
public HalfEdge GetPointIncomingHalfEdge(Point p, int i)
{
return new HalfEdge(
pointsIncomingHalfEdgeIndexes[pointsEdgeStartCount[p.index * 2] + i]
);
}
And I used it like this:
int count = delaunay.GetPointIncomingHalfEdgeCount(p);
for (int i = 0; i < count; i++)
{
var e = delaunay.GetPointIncomingHalfEdge(p, i);
}
And it was 3 times slower than the previous method! I have no idea why.
I tried to use disassembly to see what machine code was generated but I failed to do so (I am working with Unity3D). Am I condemned to rely on plain int in arrays and sane variable naming and to renounce on trying to have some compile-time type checking (is this int really a point index ?)
I am not even bringing up other questions such as, why it is even slower when I try to use IEnumerable types with yields like so:
public IEnumerable<int> GetPointIncomingHalfEdges(Point p)
{
int start = pointsEdgeStartCount[p.index * 2]; // this should be a slight optimization right ?
int count = pointsEdgeStartCount[p.index * 2 + 1];
for (int i = 0; i < count; i++)
{
yield pointsIncomingHalfEdgeIndexes[start + i];
}
}

I have added a compiler directive for aggressive inlining and it seems to make up for the discrepencies in time! For some reason the compiler fails to inline correctly for:
var e = delaunay.GetPointIncomingHalfEdge(p, i);
While it managed to do so with
var e = p.GetIncomingHalfEdge(delaunay, i);
Why ? I do not know. However It would be far easier if I was able to see how the code is compiled and I could not find how to do that. I will search that, maybe open another question and if I find a better explaination I will come back!

type conversion error, c#

I am using Advanced Matrix Library in C#. NET#
http://www.codeproject.com/KB/recipes/AdvancedMatrixLibrary.aspx?msg=4042613#xx4042613xx.
library css file is like
using System;
namespace MatrixLibrary
{
public class Matrix
{
private double[,] in_Mat;
public Matrix(int noRows, int noCols)
{
this.in_Mat = new double[noRows, noCols];
}
public Matrix(double [,] Mat)
{
this.in_Mat = (double[,])Mat.Clone();
}
public static double[,] Identity(int n)
{
double[,] temp = new double[n,n];
for (int i=0; i<n;i++) temp[i,i] = 1;
return temp;
}
public static double[,] ScalarDivide(double Value, double[,] Mat)
{
int i, j, Rows, Cols;
double[,] sol;
try {Find_R_C(Mat, out Rows, out Cols);}
catch{throw new MatrixNullException();}
sol = new double[Rows+1, Cols+1];
for (i = 0; i<=Rows;i++)
for (j = 0; j<=Cols;j++)
sol[i, j] = Mat[i, j] / Value;
return sol;
}
}}
I am trying to get identity matrix and getting error of type conversion. Could some one please guide me.
Matrix id_mat = MatrixLibrary.Matrix.Identity(6);
Can't implicity convert typedouble[,] to Matrix.
But
Matrix B = new Matrix(4, 4);
Random rnd = new Random();
for (int i = 0; i < B.NoRows; i++)
for (int j = 0; j < B.NoCols; j++)
B[i, j] = 2 * rnd.NextDouble();
Matrix E = Matrix.ScalarDivide(2, B);
It works and I can Have a matrix. Please guide me?
regards,

Read the error message.
You have a method that returns double[,] and you are trying to store the reference in a variable for Matrix. There is no implicit conversion from a multidimensional array of doubles to a Matrix.
To use that method, you would write
double[,] id_max = MatrixLibrary.Maxtrix.Identify(6);
If you actually need to store it as a Matrix, you need to define the appropriate conversion to do so.

The function returns a 2D array "double[,]" and you're trying to assign it to a "Matrix". There isn't an implicit conversion that does this. Does Matrix have a constructor that accepts a "double[,]"? Maybe you could use that.
EDIT: You might also find that a different matrix library would suit you better than one pulled off of CodeProject. Math.NET Numerics is a good one that I've used that's also open source: http://numerics.mathdotnet.com/

You need an implicit cast.
public static implicit operator Matrix(double[,] m)
{
// code to convert
}

Array.Reverse algorithm?

What kind of algorithm Array.Reverse(string a), uses behind the scene to reverse the string?

UPDATE: See the bottom of this answer for one truly horrifying ramification of reversing a string in-place in .NET.
"Good" Answer
In .NET, there's no Array.Reverse overload that takes a string. That said, here's how one might be implemented if it were to exist:
static string ReverseString(string str) {
char[] reversed = new char[str.Length];
for (int i = 0; i < reversed.Length; ++i)
reversed[i] = str[str.Length - 1 - i];
return new string(reversed);
}
Note that in .NET this method has to return a string, since the System.String type is immutable and so you couldn't reverse one in-place.
Scary Answer
OK, actually, it is possible to reverse a string in-place in .NET.
Here's one way, which requires compiling in unsafe mode:
static unsafe void ReverseString(string str) {
int i = 0;
int j = str.Length - 1;
fixed (char* fstr = str) {
while (i < j) {
char temp = fstr[j];
fstr[j--] = fstr[i];
fstr[i++] = temp;
}
}
}
And here's another way, which uses reflection and does not need to be compiled in unsafe mode:
static void ReverseString(string str) {
int i = 0;
int j = str.Length - 1;
// what a tricky bastard!
MethodInfo setter = typeof(string).GetMethod(
"SetChar",
BindingFlags.Instance | BindingFlags.NonPublic
);
while (i < j) {
char temp = str[j];
setter.Invoke(str, new object[] { j--, str[i] });
setter.Invoke(str, new object[] { i++, temp });
}
}
Totally inadvisable and reckless, yes -- not to mention that it would likely have horrendous performance. But possible nonetheless.
The Horror
Oh, and by the way, in case there's any doubt in your mind whatsoever that you should never do anything like this: be aware that either of the ReverseString methods I've provided above will actually allow you to write the following utterly bizarre program:
ReverseString("Hello!");
Console.WriteLine("Hello!");
The above code will output, believe it or not*:
!olleH
So yeah, unless you want all hell to break loose in your code, don't reverse a string in-place. Even though technically you can ;)
*You can try it for yourself if you don't believe me.

Probably a standard in-place reversal algorithm.
function reverse-in-place(a[0..n])
for i from 0 to floor(n/2)
swap(a[i], a[n-i])
Sources
Wikipedia/In-place algorithm

The algorithm is probably using two pointers i and j that start at 0 and length-1 respectively. Then the characters at position i and j are swapped (with the help of a temporal variable) and i is incremented and j decremented by 1. These steps are repeated until both pointers reach each other (i ≥ j).
In pseudo-code:
i := 0;
j := a.length-1;
while (i < j) do
tmp := a[i];
a[i] := a[j];
a[j] := tmp;
i := i+1;
j := j-1;
endwhile;

According to Reflector, Array.Reverse(Array) (there's no string variation) first calls something called TrySZReverse, for which I can't find the implementation. I assume it's some sort of heavily optimized native method..
If that fails, it does something like this:
int num = index;
int num2 = (index + length) - 1;
while (num < num2)
{
object obj2 = objArray[num];
objArray[num] = objArray[num2];
objArray[num2] = obj2;
num++;
num2--;
}
So, an in place algorithm, where it swaps the values at each end, then moves inward, repeatedly.

Here's a general-purpose, language-independent, question-appropriate answer: It copies the input string to the output string, reading from one end and writing to the other.

My suggestion:
private string Reverse(string text)
{
char[] c = text.ToCharArray(0, text.Length);
Array.Reverse(c);
return new string(c);
}