I have a function that estimates correlation between two input arrays.
The input is feeded by a dataDict which is of type Dictionary<string, double[]> which has 153 keys with values as double array of size 1500.
For each individual key, I need to estimate its correlation with all other keys and store the result to a double[,] that has a size of double[dataDict.Count(), dataDict.Count()]
The following function prepares two double[] arrays whose correlation needs to be estimated.
public double[,] CalculateCorrelation(Dictionary<string, double?[]> dataDict, string corrMethod = "kendall")
{
CorrelationLogicModule correlationLogicModule = new CorrelationLogicModule();
double[,] correlationMatrix = new double[dataDict.Count(), dataDict.Count()];
for (int i = 0; i < dataDict.Count; i++)
{
for (int j = 0; j < dataDict.Count; j++)
{
var arrayA = dataDict[dataDict.ElementAt(i).Key].Cast<double>().ToArray();
var arrayB = dataDict[dataDict.ElementAt(j).Key].Cast<double>().ToArray();
correlationMatrix[i, j] = correlationLogicModule.Kendall_Formula(arrayA, arrayB);
}
}
return correlationMatrix;
}
The following function (I found it on internet from here) finds correlation between two input arrays using 'Kendall's' method.
public double Kendall_Formula(double[] Ticker1, double[] Ticker2)
{
double NbrConcord, NbrDiscord, S;
NbrConcord = 0;
NbrDiscord = 0;
S = 0;
for (int i = 0; i < Ticker1.Length - 1; i++)
{
for (int j = i + 1; j < Ticker1.Length; j++)
{
//Compute the number of concordant pairs
if (((Ticker1[i] < Ticker1[j]) & (Ticker2[i] < Ticker2[j])) | ((Ticker1[i] > Ticker1[j]) & (Ticker2[i] > Ticker2[j])))
{
NbrConcord++;
}
//Compute the number of discordant pairs
else if (((Ticker1[i] > Ticker1[j]) & (Ticker2[i] < Ticker2[j])) | ((Ticker1[i] < Ticker1[j]) & (Ticker2[i] > Ticker2[j])))
{
NbrDiscord++;
}
}
}
S = NbrConcord - NbrDiscord;
//Proportion with the total pairs
return 2 * S / (Ticker1.Length * (Ticker1.Length - 1));
}
Moving this way forward, takes a very long time to calculate the correlations for all the keys.
is there a possible way to optimize the performance?.
I am new to C# but I have been using Python for a long time and in Python using 'Numpys' and 'Pandas' I am sure the above operation would take seconds to compute. For e.g. lets say I had the above data in form of a pandas dataframe, then data[[list of columns]].corr('method') would lead the result in seconds. This is because pandas uses numpy under the hood which takes benefit from vectorization. I would like to learn how can I take benefit from vectorization to improve the performance of the above code in C# and if there are other factors I need to consider. Thank you!
You are using dataDict[dataDict.ElementAt(i).Key] to access the dictionary values in an undefined order. I don't know if that's what you intended, but the following code should give the the same results.
If you call dataDict.Values.ToArray(); you will get the dictionary values in the same order as you would when using foreach to iterate over it. That means that it will be the same as the order when using dataDict[dataDict.ElementAt(i).Key].
Therefore this code should be equivalent, and it should be faster:
public double[,] CalculateCorrelation(Dictionary<string, double?[]> dataDict, string corrMethod = "kendall")
{
CorrelationLogicModule correlationLogicModule = new CorrelationLogicModule();
var values = dataDict.Values.Select(array => array.Cast<double>().ToArray()).ToArray();
double[,] correlationMatrix = new double[dataDict.Count, dataDict.Count];
for (int i = 0; i < dataDict.Count; i++)
{
for (int j = 0; j < dataDict.Count; j++)
{
var arrayA = values[i];
var arrayB = values[j];
correlationMatrix[i, j] = correlationLogicModule.Kendall_Formula(arrayA, arrayB);
}
}
return correlationMatrix;
}
Note that the .ElementAt() call in your original code is a Linq extension, not a member of Dictionary<TKey,TValue>. It iterates from the start of the dictionary EVERY TIME you call it - and it also returns items in an unspecified order. From the documentation: For purposes of enumeration, each item in the dictionary is treated as a KeyValuePair<TKey,TValue> structure representing a value and its key. The order in which the items are returned is undefined.
Also:
You should change the bitwise & to logical && in your conditions. The use of & will prevent the compiler applying a boolean short-circuit optimisation, meaning that all the < / > comparisons will be performed, even if the first condition is false.
I'm trying to create some pairs of unique numbers using pretty simple algorithm.
For some unknown reason after compiling Unity goes into an endless "not responding" state. Seems like it's stuck in a do..while loop, but I don't see any reason for that.
//Creating two lists to store random numbers
List<int> xList = new List<int>();
List<int> yList = new List<int>();
int rx, ry;
for(int i = 0; i < 10; i++)
{
// look for numbers until they are unique(while they are in lists)
do
{
rx = rand.Next(0, width);
ry = rand.Next(0, height);
}
while(xList.Contains(rx) || yList.Contains(ry));
//add them to lists
xList.Add(rx);
yList.Add(ry);
Debug.Log(rx + ", " + ry);
// some actions with these numbers
gridArray[rx,ry].isBomb = true;
gridArray[rx,ry].changeSprite(bombSprite);
}
As mentioned the issue is that once all unique numbers have been used once you are stuck in the do - while loop.
Instead you should rather simply
generate the plain index lists for all possible pairs.
I will use the Unit built-in type Vector2Int but you could do the same using your own struct/class
For each bomb to place pick a random entry from the list of pairs
Remove according random picked item from the pairs so it is not available anymore in the next go
Something like
// create the plain pair list
var pairs = new List<Vector2Int>(width * height);
for(var x = 0; x < width; x++)
{
for(var y = 0; y < height; y++)
{
pairs.Add(new Vector2Int(x,y));
}
}
// so now you have all possible permutations in one list
if(pairs.Count < BOMB_AMOUNT_TO_PLACE)
{
Debug.LogError("You are trying more bombs than there are fields in the grid!");
return;
}
// Now place your bombs one by one on a random spot in the grid
for(var i = 0; i < BOMB_AMOUNT_TO_PLACE; i++)
{
// now all you need to do is pick one random index from the possible entries
var randomIndexInPairs = Random.Range(0, pairs.Count);
var randomPair = pairs[randomIndexInPairs];
// and at the same time remove the according entry
pairs.RemoveAt(randomIndexInPairs);
// Now you have completely unique but random index pairs
var rx = randomPair.x;
var ry = randomPair.y;
gridArray[rx, ry].isBomb = true;
gridArray[rx, ry].changeSprite(bombSprite);
}
Depending on your use-case as alternative to generate the pairs list and then remove entries again you could also generate it once and then use
if(pairs.Count < BOMB_AMOUNT_TO_PLACE)
{
Debug.LogError("You are trying more bombs than there are fields in the grid!");
return;
}
var random = new System.Random();
var shuffledPairs = pairs.OrderBy(e => random.Next());
for(var i = 0; i < BOMB_AMOUNT_TO_PLACE; i++)
{
// then you can directly use
var randomPair = shuffledPairs[i];
// Now you have completely unique but random index pairs
var rx = randomPair.x;
var ry = randomPair.y;
gridArray[rx, ry].isBomb = true;
gridArray[rx, ry].changeSprite(bombSprite);
}
Although your algorithm is maybe not the best way to generate ten bombs in a grid, it should work.
The problem is that your while condition is using a OR statement, which means that if you have a bomb in the first line (in any column), it will not be able to add another bomb in that line.
Therefore you will pretty soon end up with an infinite loop because for every bomb you lock the line and column.
If you put an AND condition, you make sure the pair is unique because you lock only that cell.
Provided of course that width x height is more than ten.
I have the following code. It's roughly analogous in concept to the python reshape function. It successfully loads 1-dimensional data into a multi-dimensional array, the dimensions of which are not known until runtime. For example {209,64,64,3}. I have to iterate over the 1-dimensional data and create the correct indexes for each dimension of the array.
private void InitializeData()
{
var imageData = ImageData.Load(txtFileName.Text); // one dimensional array
var dimensions = txtDimensions.Text.Split(',').Select(d => int.Parse(d)).ToArray(); // e.g., {-1,64,64,3}
int elements = 1;
foreach (var dim in dimensions.Skip(1))
{
elements *= dim;
}
dimensions[0] = imageData.Length / elements; // {209,64,64,3}
// create multipliers
var multipliers = new int[dimensions.Length - 1];
for (var dimension = 1; dimension < dimensions.Length; dimension++)
{
var multiplier = 1;
for (var followingdimension = dimension; followingdimension < dimensions.Length; followingdimension++)
{
multiplier *= dimensions[followingdimension];
}
multipliers[dimension - 1] = multiplier;
}
// load data
var dataArray = Array.CreateInstance(typeof(int), dimensions);
var indexes = new int[dimensions.Length];
for (var imageDataIndex = 0; imageDataIndex < imageData.Length; imageDataIndex++)
{
indexes[0] = imageDataIndex / multipliers[0];
indexes[dimensions.Length - 1] = imageDataIndex % multipliers[multipliers.Length - 1];
for (var multiplier = 1; multiplier < dimensions.Length - 1; multiplier++)
indexes[multiplier] = (imageDataIndex / multipliers[multiplier]) % dimensions[multiplier];
dataArray.SetValue(imageData[imageDataIndex], indexes);
}
}
Is there a faster or more elegant way of doing this? I do realize those are two different things. I'll do bench-marking on the elegant suggestions, but I'd still like to see them. Because this is just too ugly to look at and was too painful to write to be the best way.
Note (Please)
The data may not always be image data, so I am not looking for bitmap operations. That just happens here but it's not necessarily a typical case. And, my goal is not to get a bitmap, but an array.
I have a partial answer thanks to How to reshape an Array in c#
The code can be replaced with just this:
var imageData = ImageData.Load(txtFileName.Text); // one dimensional array
// e.g., {209,64,64,3}
var dimensions = txtDimensions.Text.Split(',').Select(d => int.Parse(d)).ToArray();
int elements = 1;
foreach (var dim in dimensions.Skip(1))
{
elements *= dim;
}
dimensions[0] = imageData.Length / elements;
// load data
var dataArray = Array.CreateInstance(typeof(int), dimensions);
Buffer.BlockCopy(imageData, 0, dataArray, 0, imageData.Length * sizeof(int));
I would be surprised if there's a faster way to do the actual load then Buffer.BlockCopy, or a simpler one. It turns out whatever dimensional form your original data is in, BlockCopy handles it as long as you can specify your target dimensions as part of a target array.
I'll keep looking for ways to further refine the rest of the original code.
I'm working on a FFT (fast fourier transform) function in C#.
I've found AForge library but when using it I keep getting the error message:
Incorrect data length.
The data I'm putting in is a list of doubles and the size can vary, depending on the signal I'm sending in.
What to do about this?
This is what my function looks like so far
private void FastFoulierMethod(ISignal signal, List<IMarker> markers)
{
AForge.Math.Complex[] complex = new AForge.Math.Complex[samples.Count];
for (int i = 0; i < samples.Count-1; i++)
{
complex[i] = new AForge.Math.Complex(samples[i].GetTimeInSec(),0);
}
AForge.Math.Complex[] complex2 = complex;
FourierTransform.DFT(complex, FourierTransform.Direction.Backward);
FourierTransform.FFT(complex2, FourierTransform.Direction.Backward);
}
As per AForge documentation:
The method accepts data array of 2n size only, where n may vary in the [1, 14] range
So you would need to make sure the input size is correctly padded to a length that is a power of 2, and in the specified range:
double logLength = Math.Ceiling(Math.Log((double)sample.Count, 2.0));
int paddedLength = (int) Math.Pow(2.0, Math.Min(Math.Max(1.0, logLength), 14.0));
AForge.Math.Complex[] complex = new AForge.Math.Complex[paddedLength];
// copy all input samples
int i = 0;
for (; i < sample.Count; i++)
{
complex[i] = new AForge.Math.Complex(samples[i].GetTimeInSec(),0);
}
// pad with zeros
for (; i < paddedLength; i++)
{
complex[i] = new AForge.Math.Complex(0,0);
}
If you look at AForge FourierTransform source code you will find ReorderData method(at the bottom of file) which is called from FFT method. In this method there is check which rises error:
private static void ReorderData( Complex[] data )
{
int len = data.Length;
// check data length
if ( ( len < minLength ) || ( len > maxLength ) || ( !Tools.IsPowerOf2( len ) ) )
throw new ArgumentException( "Incorrect data length." );
//rest of code...
}
So the answer to your question is - your array length must be power of 2 and be between 2 and 16384.
Probably a really simple one this - I'm starting out with C# and need to add values to an array, for example:
int[] terms;
for(int runs = 0; runs < 400; runs++)
{
terms[] = runs;
}
For those who have used PHP, here's what I'm trying to do in C#:
$arr = array();
for ($i = 0; $i < 10; $i++) {
$arr[] = $i;
}
You can do this way -
int[] terms = new int[400];
for (int runs = 0; runs < 400; runs++)
{
terms[runs] = value;
}
Alternatively, you can use Lists - the advantage with lists being, you don't need to know the array size when instantiating the list.
List<int> termsList = new List<int>();
for (int runs = 0; runs < 400; runs++)
{
termsList.Add(value);
}
// You can convert it back to an array if you would like to
int[] terms = termsList.ToArray();
Edit: a) for loops on List<T> are a bit more than 2 times cheaper than foreach loops on List<T>, b) Looping on array is around 2 times cheaper than looping on List<T>, c) looping on array using for is 5 times cheaper than looping on List<T> using foreach (which most of us do).
Using Linq's method Concat makes this simple
int[] array = new int[] { 3, 4 };
array = array.Concat(new int[] { 2 }).ToArray();
result
3,4,2
If you're writing in C# 3, you can do it with a one-liner:
int[] terms = Enumerable.Range(0, 400).ToArray();
This code snippet assumes that you have a using directive for System.Linq at the top of your file.
On the other hand, if you're looking for something that can be dynamically resized, as it appears is the case for PHP (I've never actually learned it), then you may want to use a List instead of an int[]. Here's what that code would look like:
List<int> terms = Enumerable.Range(0, 400).ToList();
Note, however, that you cannot simply add a 401st element by setting terms[400] to a value. You'd instead need to call Add() like this:
terms.Add(1337);
By 2019 you can use Append, Prepend using LinQ in just one line
using System.Linq;
and then in NET 6.0:
terms = terms.Append(21);
or versions lower than NET 6.0
terms = terms.Append(21).ToArray();
Answers on how to do it using an array are provided here.
However, C# has a very handy thing called System.Collections
Collections are fancy alternatives to using an array, though many of them use an array internally.
For example, C# has a collection called List that functions very similar to the PHP array.
using System.Collections.Generic;
// Create a List, and it can only contain integers.
List<int> list = new List<int>();
for (int i = 0; i < 400; i++)
{
list.Add(i);
}
Using a List as an intermediary is the easiest way, as others have described, but since your input is an array and you don't just want to keep the data in a List, I presume you might be concerned about performance.
The most efficient method is likely allocating a new array and then using Array.Copy or Array.CopyTo. This is not hard if you just want to add an item to the end of the list:
public static T[] Add<T>(this T[] target, T item)
{
if (target == null)
{
//TODO: Return null or throw ArgumentNullException;
}
T[] result = new T[target.Length + 1];
target.CopyTo(result, 0);
result[target.Length] = item;
return result;
}
I can also post code for an Insert extension method that takes a destination index as input, if desired. It's a little more complicated and uses the static method Array.Copy 1-2 times.
Based on the answer of Thracx (I don't have enough points to answer):
public static T[] Add<T>(this T[] target, params T[] items)
{
// Validate the parameters
if (target == null) {
target = new T[] { };
}
if (items== null) {
items = new T[] { };
}
// Join the arrays
T[] result = new T[target.Length + items.Length];
target.CopyTo(result, 0);
items.CopyTo(result, target.Length);
return result;
}
This allows to add more than just one item to the array, or just pass an array as a parameter to join two arrays.
You have to allocate the array first:
int [] terms = new int[400]; // allocate an array of 400 ints
for(int runs = 0; runs < terms.Length; runs++) // Use Length property rather than the 400 magic number again
{
terms[runs] = value;
}
int ArraySize = 400;
int[] terms = new int[ArraySize];
for(int runs = 0; runs < ArraySize; runs++)
{
terms[runs] = runs;
}
That would be how I'd code it.
C# arrays are fixed length and always indexed. Go with Motti's solution:
int [] terms = new int[400];
for(int runs = 0; runs < 400; runs++)
{
terms[runs] = value;
}
Note that this array is a dense array, a contiguous block of 400 bytes where you can drop things. If you want a dynamically sized array, use a List<int>.
List<int> terms = new List<int>();
for(int runs = 0; runs < 400; runs ++)
{
terms.Add(runs);
}
Neither int[] nor List<int> is an associative array -- that would be a Dictionary<> in C#. Both arrays and lists are dense.
You can't just add an element to an array easily. You can set the element at a given position as fallen888 outlined, but I recommend to use a List<int> or a Collection<int> instead, and use ToArray() if you need it converted into an array.
If you really need an array the following is probly the simplest:
using System.Collections.Generic;
// Create a List, and it can only contain integers.
List<int> list = new List<int>();
for (int i = 0; i < 400; i++)
{
list.Add(i);
}
int [] terms = list.ToArray();
one approach is to fill an array via LINQ
if you want to fill an array with one element
you can simply write
string[] arrayToBeFilled;
arrayToBeFilled= arrayToBeFilled.Append("str").ToArray();
furthermore, If you want to fill an array with multiple elements you can use the
previous code in a loop
//the array you want to fill values in
string[] arrayToBeFilled;
//list of values that you want to fill inside an array
List<string> listToFill = new List<string> { "a1", "a2", "a3" };
//looping through list to start filling the array
foreach (string str in listToFill){
// here are the LINQ extensions
arrayToBeFilled= arrayToBeFilled.Append(str).ToArray();
}
Array Push Example
public void ArrayPush<T>(ref T[] table, object value)
{
Array.Resize(ref table, table.Length + 1); // Resizing the array for the cloned length (+-) (+1)
table.SetValue(value, table.Length - 1); // Setting the value for the new element
}
int[] terms = new int[10]; //create 10 empty index in array terms
//fill value = 400 for every index (run) in the array
//terms.Length is the total length of the array, it is equal to 10 in this case
for (int run = 0; run < terms.Length; run++)
{
terms[run] = 400;
}
//print value from each of the index
for (int run = 0; run < terms.Length; run++)
{
Console.WriteLine("Value in index {0}:\t{1}",run, terms[run]);
}
Console.ReadLine();
/*Output：
Value in index 0: 400
Value in index 1: 400
Value in index 2: 400
Value in index 3: 400
Value in index 4: 400
Value in index 5: 400
Value in index 6: 400
Value in index 7: 400
Value in index 8: 400
Value in index 9: 400
*/
If you don't know the size of the Array or already have an existing array that you are adding to. You can go about this in two ways. The first is using a generic List<T>:
To do this you will want convert the array to a var termsList = terms.ToList(); and use the Add method. Then when done use the var terms = termsList.ToArray(); method to convert back to an array.
var terms = default(int[]);
var termsList = terms == null ? new List<int>() : terms.ToList();
for(var i = 0; i < 400; i++)
termsList.Add(i);
terms = termsList.ToArray();
The second way is resizing the current array:
var terms = default(int[]);
for(var i = 0; i < 400; i++)
{
if(terms == null)
terms = new int[1];
else
Array.Resize<int>(ref terms, terms.Length + 1);
terms[terms.Length - 1] = i;
}
If you are using .NET 3.5 Array.Add(...);
Both of these will allow you to do it dynamically. If you will be adding lots of items then just use a List<T>. If it's just a couple of items then it will have better performance resizing the array. This is because you take more of a hit for creating the List<T> object.
Times in ticks:
3 items
Array Resize Time: 6
List Add Time: 16
400 items
Array Resize Time: 305
List Add Time: 20
I will add this for a another variant. I prefer this type of functional coding lines more.
Enumerable.Range(0, 400).Select(x => x).ToArray();
You can't do this directly. However, you can use Linq to do this:
List<int> termsLst=new List<int>();
for (int runs = 0; runs < 400; runs++)
{
termsLst.Add(runs);
}
int[] terms = termsLst.ToArray();
If the array terms wasn't empty in the beginning, you can convert it to List first then do your stuf. Like:
List<int> termsLst = terms.ToList();
for (int runs = 0; runs < 400; runs++)
{
termsLst.Add(runs);
}
terms = termsLst.ToArray();
Note: don't miss adding 'using System.Linq;' at the begaining of the file.
This seems like a lot less trouble to me:
var usageList = usageArray.ToList();
usageList.Add("newstuff");
usageArray = usageList.ToArray();
Just a different approach:
int runs = 0;
bool batting = true;
string scorecard;
while (batting = runs < 400)
scorecard += "!" + runs++;
return scorecard.Split("!");
int[] terms = new int[400];
for(int runs = 0; runs < 400; runs++)
{
terms[runs] = value;
}
static void Main(string[] args)
{
int[] arrayname = new int[5];/*arrayname is an array of 5 integer [5] mean in array [0],[1],[2],[3],[4],[5] because array starts with zero*/
int i, j;
/*initialize elements of array arrayname*/
for (i = 0; i < 5; i++)
{
arrayname[i] = i + 100;
}
/*output each array element value*/
for (j = 0; j < 5; j++)
{
Console.WriteLine("Element and output value [{0}]={1}",j,arrayname[j]);
}
Console.ReadKey();/*Obtains the next character or function key pressed by the user.
The pressed key is displayed in the console window.*/
}
/*arrayname is an array of 5 integer*/
int[] arrayname = new int[5];
int i, j;
/*initialize elements of array arrayname*/
for (i = 0; i < 5; i++)
{
arrayname[i] = i + 100;
}
To add the list values to string array using C# without using ToArray() method
List<string> list = new List<string>();
list.Add("one");
list.Add("two");
list.Add("three");
list.Add("four");
list.Add("five");
string[] values = new string[list.Count];//assigning the count for array
for(int i=0;i<list.Count;i++)
{
values[i] = list[i].ToString();
}
Output of the value array contains:
one
two
three
four
five
You can do this is with a list. here is how
List<string> info = new List<string>();
info.Add("finally worked");
and if you need to return this array do
return info.ToArray();
Here is one way how to deal with adding new numbers and strings to Array:
int[] ids = new int[10];
ids[0] = 1;
string[] names = new string[10];
do
{
for (int i = 0; i < names.Length; i++)
{
Console.WriteLine("Enter Name");
names[i] = Convert.ToString(Console.ReadLine());
Console.WriteLine($"The Name is: {names[i]}");
Console.WriteLine($"the index of name is: {i}");
Console.WriteLine("Enter ID");
ids[i] = Convert.ToInt32(Console.ReadLine());
Console.WriteLine($"The number is: {ids[i]}");
Console.WriteLine($"the index is: {i}");
}
} while (names.Length <= 10);