I'm working on a project right now that requires me to work with and store millions of different doubles. So I figure the most effective way to store all of these values would be in sets of arrays within arrays. but i have no idea how to do this. Plus i have to feed values into the lower level arrays and don't yet know an effective way of doing this.
The below is an attempt at explaining what I'm going for (This is not code, but the website thought it was):
L01 = [1,2,3]
L02 = [3,2,1]
L03 = [2,3,1]
L04 = [1,3,2]
L11 = [L01,L02]
L12 = [L03,L04]
L2 = [L11,L12]
So far I have only come up with this, and you should see why it doesn't work. I'm still pretty new to programming so please explain what I should be doing:
//Stores the weight values attached to an individual neuron
public double[] NeuronWeights = new double[321];
//Stores the NeuronWeights[] for an individual layer with incoming weights
public double[][] LayerWeight = new double[321][];
//Stores all of the LayerWeight[] in the network
public double[][][] TotalWeights = new double[11][][];
public void InitializWeights()
{
for (int i = 0; i < TotalWeights.Length; i++)
{
for(int j = 0; j < LayerWeight.Length; j++)
{
for(int k = 0; k < NeuronWeights.Length; k++)
{
Random r = new Random();
//Creating randome values to fill first level
if (r.Next(0, 2) > 0)
{
NeuronWeights[k] = r.NextDouble() * 2 - 1;
}
else NeuronWeights[k] = 0.0;
}
LayerWeight[j][] = NeuronWeights[];
}
TotalWeights[i][][] = LayerWeight[][];
}
}
}
To add more detail; I'm trying to generate and store 321 doubles ,within the range of (-1, 1), 321 time per "layer". Then do that for all 11 "layers". this information then needs to be called on to assign values for 321 other doubles, 11 times.
Multi-dimensional arrays are not the same as arrays of arrays. Try using List<> instead:
var L01 = new List<int> { 1,2,3 };
var L02 = new List<int> { 3,2,1 };
var L03 = new List<int> { 2,3,1 };
var L04 = new List<int> { 1,3,2 };
var L11 = new List<List<int>> { L01,L02 };
var L12 = new List<List<int>> { L03,L04 };
var L2 = new List<List<List<int>>> { L11,L12 };
But depending on what you are trying to accomplish, maybe you don't need to nest your Lists like that.
It looks like you are trying to create a structure where there are 11 layers, each with 321 neuron weights? That doesn't seem to make sense with your initial goal of storing and working with millions of doubles.
If you simply wanted to store a variable number of layers, each with a variable number of recorded weights, you'd set up something like this:
List<List<double>> weightData = new List<List<double>>();
You can think of a List as a variable-sized array that is expandable as you are going along in your process. So for each Layer, you'd
List<double> currentLayer = new List<double>();
weightData.add(currentLayer);
Then for each weight you wanted to add to the layer you'd do something like:
double neuronWeight = 17.0;
currentLayer.add(neuronWeight);
at the end of the day, weightData will have all your layers and weights within the layers. If you plan to manipulate the data after loading it, you will need to come up with a scheme to reference the various layers and weights, but I'm not sure what your use case for that is.
Good luck!
What about something like this, which uses one multi-dimensional array?
int totalWeightLength = 11;
int layerWeightLength = 321;
int neuronWeights = 321;
double[,,] TotalWeights = new double[totalWeightLength, layerWeightLength, neuronWeights];
void InitializWeights()
{
for (int i = 0; i < totalWeightLength; i++)
{
for (int j = 0; j < layerWeightLength; j++)
{
for (int k = 0; k < neuronWeights; k++)
{
Random r = new Random();
//Creating randome values to fill first level
if (r.Next(0, 2) > 0)
{
TotalWeights[i,j,k] = r.NextDouble() * 2 - 1;
}
else TotalWeights[i,j,k] = 0.0;
}
}
}
}
InitializWeights();
Console.WriteLine(TotalWeights[1,34,23]);
Console.WriteLine(TotalWeights[2,84,123]);
Console.WriteLine(TotalWeights[3,39,24]);
Console.WriteLine(TotalWeights[4,27,36]);
Related
i'm trying to write a desktop app using c# .NET , but currently i'm stuck in the following problem:
Assume you have two Lists:
List1=[1,2,3,4]
List2=[1,4,1,3]
And i want a new list that is filled with "n" zeros between two consecutive elements of list1, been "n" the "i" element of list2, the new list should look like this:
List3=[1,0,2,0,0,0,0,3,0,4,0,0,0]
My Code:
List<int> idID = new List<int>();
idID.Add(1);
idID.Add(2);
idID.Add(3);
idID.Add(4);
List<int> Nevent = new List<int>();
Nevent.Add(1);
Nevent.Add(4);
Nevent.Add(1);
Nevent.Add(3);
int total = Nevent.Count;
for (int j = 0; j < total; j++)
{
for (int i = 1; i <= Nevent[j]; i++)
{
idID.Insert(i, 0); //modify (____,0) of this line???
}
}
string IDS = String.Join(",", idID);
Console.WriteLine(IDS);
I think that i should change the part with idID.Insert(i, 0); and replace the i with some kind of sequence form like N0,N0+N1,N0+N1+N2,... been Ni the i element of list2 (that i name Nevent in the code) but i do not know how to do that.
How should I proceed? or there is a better way to achive what i want?
Thank you in advance.
If we assume the lists are the same length, you can make use of Zip to zip the 2 arrays together, and then a bit of LINQ to build up and select the resulting array
var result = idID.Zip(Nevent,
(x,y) => new[]{ x }.Concat(Enumerable.Repeat(0,y)) )
.SelectMany(x => x);
result is
1,0,2,0,0,0,0,3,0,4,0,0,0
Live example: https://dotnetfiddle.net/PvtOVv
If you want/need to have result as a list just tag ToList() on the end
Create new list, and treat idID and Nevent as input only. With LINQ we can use Enumerable.Repeat, otherwise we would have to use inner loop to add zeros.
using System.Linq;
...
List<int> result = new List<int>();
for (int j = 0; j < idID.Count; j++)
{
result.Add(idID[j]);
result.AddRange(Enumerable.Repeat(0, Nevent[j]));
}
Another approach is to process the list backwards. Like this the inserted zeroes do not alter the indexes of elements to be processed.
var idID = new List<int> { 1, 2, 3, 4 };
var Nevent = new List<int> { 1, 4, 1, 3 };
for (int i = idID.Count - 1; i >= 0; i--) {
for (int n = 0; n < Nevent[i]; n++) {
idID.Insert(i + 1, 0);
}
}
Console.WriteLine(String.Join(",", idID));
prints
1,0,2,0,0,0,0,3,0,4,0,0,0
I wanted to implement a simple method to sample from a multinomial distribution in C# (the first argument is an array of integers we want to sample and the second one is the probabilities of selecting each of those integers).
When I do this with numpy in python, the results make sense.
np.random.choice(np.array([1,2,3,4,5,6]),p=np.array([.624,.23,.08,.04, .02, .006]),size=len(b))
I get a lot of 1's (probability 62%), a bunch of 2's, some 3's etc.
However, when I try the implementation below in C# (pretty straightforward inverse transform sampling for multinomial, only relies on a uniform random variable), I get really weird results. For all 1000 samples, I'll often find all 1's. Sometimes, I'll find all 3's (!!??). The results never look like what you would expect (and what you get from the python function - try running it yourself a few times). This is really scary since we rely on these primitives. Does anyone have insight into what might be wrong with the C# version?
static void Main(string[] args)
{
int[] iis = new int[7];
int[] itms = new int[] { 1, 2, 3, 4, 5, 6 };
double[] probs = new double[] { .624, .23, .08, .04, .02, .006 };
for (int i = 0; i < 1000; i++)
{
iis[MultinomialSample(itms, probs)] += 1;
}
foreach (var ii in iis)
{
Console.Write(ii + ",");
}
Console.Read();
}
private static int MultinomialSample(int[] s, double[] ps)
{
double[] cumProbs = new double[ps.Length];
cumProbs[0] = ps[0];
for (int i = 1; i < ps.Length; i++)
{
cumProbs[i] = cumProbs[i - 1] + ps[i];
}
Random random = new Random();
double u = random.NextDouble();
for (int i = 0; i < cumProbs.Length - 1; i++)
{
if (u < cumProbs[i])
{
return s[i];
}
}
return s[s.Length - 1];
}
You're initializing Random each time you call MultinomialSample. If these calls are very close together, Random will be initialized with the same seed (based on the system clock). Try either making Random a private class field: private static Random random = new Random(); or pass it into the method as an argument from Main, where it would be initialized only once:
private static Random random = new Random();
private static int MultinomialSample(IReadOnlyList<int> sample,
IReadOnlyList<double> probabilities)
{
var cumProbs = new double[probabilities.Count];
cumProbs[0] = probabilities[0];
for (var i = 1; i < probabilities.Count; i++)
{
cumProbs[i] = cumProbs[i - 1] + probabilities[i];
}
for (var i = 0; i < cumProbs.Length - 1; i++)
{
if (random.NextDouble() < cumProbs[i])
{
return sample[i];
}
}
return sample[sample.Count - 1];
}
private static void Main()
{
var iis = new int[7];
var items = new[] {1, 2, 3, 4, 5, 6};
var probabilities = new[] {.624, .23, .08, .04, .02, .006};
for (int i = 0; i < 1000; i++)
{
iis[MultinomialSample(items, probabilities)] ++;
}
Console.WriteLine(string.Join(", ", iis));
Console.WriteLine("\nDone!\nPress any key to exit...");
Console.ReadKey();
}
I used Rufus' code in a simulation I was working on and noticed there is still a problem, even after seeding the random number generator just once (which is the correct thing to do). You will notice that as we are iterating, the call to random.NextDouble() generates a new random number each time. This is wrong.
for (var i = 0; i < cumProbs.Length - 1; i++)
{
if (random.NextDouble() < cumProbs[i])
{
return sample[i];
}
}
The random number should be generated outside of the loop, as follows:
var r = random.NextDouble();
for (var i = 0; i < cumProbs.Length - 1; i++)
{
if (r < cumProbs[i])
{
return sample[i];
}
}
You can compare it to the Excel algorithm given on Wikipedia: https://en.wikipedia.org/wiki/Multinomial_distribution. When I made the above change to Rufus' code, I got the desired frequency distribution as specified by the probabilities array.
I have a 2D array for a lottery I am creating. Basically it's a set of 2 integers:
int[,] coupon = new int[rowAmount, numAmount];
Where rowAmount is the amount of rows, and numAmount is the amount of numbers in that row.
Now I need to select the numbers for each row, however there may not be duplicates of a number within a specific row.
for (int r = 0; r < rowAmount; ++r)
{
for (int n = 0; n < numAmount; ++n)
{
userNum = lotRng.Next(1, numAmount * rngMult);
while (COUPON CONTAINS DUPLICATE NUMBER ON SECOND SPOT )
{
userNum = lotRng.Next(1, numAmount * rngMult);
}
coupon[r, n] = userNum;
}
}
My issue is the while part, I cannot figure out how to check if coupon contains the userNum on the second slot(The numAmount slot). For lists and stuff I used to just do list.Contains() but that doesn't seem to work on here.
Depending on the size of your array is wether it makes sense to optimize performance.
Depending on that one possibility would be to sort the array and use Array.BinarySearch .
You have to sort your array for that.
https://msdn.microsoft.com/en-us/library/2cy9f6wb(v=vs.110).aspx
So you have a number of possibilities to optimize data structure.
Solution with an array of lists is one of my favourites for this. It's very similar to my other with jagged arrays but faster- because List search is most efficient and Linq searches are not.
const int rowAmount = 1000;
const int numAmount=1000;
const int rngMult = 10;
Random lotRng = new Random();
var coupon = new List<int>[rowAmount];
int userNum;
for (int r = 0; r < rowAmount; r++)
{
coupon[r]= new List<int>();
for (int n = 0; n < numAmount; ++n)
{
do userNum = lotRng.Next(1, numAmount * rngMult);
while (coupon[r].Contains(userNum));
coupon[r].Add(userNum);
}
}
Of course it would be also possible to use a list of lists (kind of 2D lists), if necessary.
var couponLoL = new List<List<int>>();
The following quick and dirty way show a possible way of copying a 2D array to a list, but not to recommend here for several reasons (loop, boxing for value types):
var coupon= new int[rowAmount,numAmount];
[..]
do userNum = lotRng.Next(1, numAmount * rngMult);
while (coupon.Cast<int>().ToList().Contains(userNum));
In this special case it makes even less sense, because this would look in the whole 2D array for the double value. But it is worth knowing how to convert from 2D to 1D array (and then in a list).
Solution with 2D jagged arrays: If you want to access rows and columns in C#, a jagged array is very convenient and unless you care very much how effient the array ist stored internally, jagged array are to recommend strongly for that.
Jagged arrays are simply arrays of arrays.
const int rowAmount = 1000;
const int numAmount=1000;
const int rngMult = 10;
int userNum;
Random lotRng = new Random();
var coupon = new int[rowAmount][];
for (int r = 0; r < rowAmount; r++)
{
coupon[r] = new int[numAmount];
for (int n = 0; n < numAmount; ++n)
{
do userNum = lotRng.Next(1, numAmount * rngMult);
while (Array.Exists(coupon[r], x => x == userNum));
coupon[r, n] = userNum;
}
}
The above function Array.Exists works only for 1 dimension what is enough here, and needs no Linq. The same as above with Linq method .Any :
while (coupon[r].Any(x => x == userNum));
If you would have to search in two dimensions for a double value, you would need a loop more again, but still on nested loop level less than without this.
Linq is elegant, but normally not the fastest method (but you would have to handle with very big arrays of multi-million sizes for that to matter).
For other possibilities of using Linq, look for example here:
How to use LINQ with a 2 dimensional array
Another idea would be to make one-dimensional array of size rowAmount*numAmount.
It needs a little bit of thinking, but allows most simple and fastest access of searching.
Solution with loops in an array. Not elegant, but you could refactor the search loops in an own method to look better. But as a second point, in this case a linear search like shown is not a really fast solution either.
Only inner part of the 2 for-loops, not full code (as in other answers by me here):
bool foundDup;
do
{
userNum = lotRng.Next(1, numAmount * rngMult);
foundDup = false;
for (var x = 0; x < coupon.GetLength(1); x++) //Iterate over second dimension only
if (coupon[r, x] == userNum)
{ foundDup = true;
break;
}
} while (foundDup);
coupon[r, n] = userNum;
In the special context of this question, you can optimize the loop:
for (var x = 0; x < n; x++)
As you say in your comment you need to check all n fields vs. your new userNum. You can solve this with the following code:
for (int r = 0; r < rowAmount; ++r)
{
for (int n = 0; n < numAmount; ++n)
{
userNum = lotRng.Next(1, numAmount * rngMult);
for (int x = 0; x < coupon.GetLength(1); x++) //Iterate over your second dimension again
{
while (coupon[r,x] == userNum)
{
userNum = lotRng.Next(1, numAmount * rngMult);
}
}
coupon[r, n] = userNum;
}
}
This question already has answers here:
List items changes as I add new data... why?
(6 answers)
Closed 6 years ago.
I need to create a dynamically sized list that contains pairs of points for a window's form using C#. The list size will change depending on the data that is loaded.
A simplified version of my approach, to simply convey the problem, is as follows (in the real app, the for-loops will iterate over sizes dependent on loaded data):
int[,] dummyInt = new int[1, 2];
List<int[,]> test = new List<int[,]>();
for (int i = 0; i < 100; i++)
{
dummyInt[0, 0] = i;
for (int j = 0; j < 5; j++)
{
dummyInt[0, 1] = j;
test.Add(dummyInt);
}
}
//Show the values in the list for debugging
foreach (int[,] value in test)
{
MessageBox.Show(value.ToString("G"));
}
Using this approach, all 500 values in the list are [99,4].
What I was expecting/hoping to get was
value 1 [0,0]
value 2 [0,1]
...
value 500 [99,4]
Seems like the list is storing the actual variable, and then changing the value with every iteration of the for loop. How can I store just the value of dummyInt as a new object to the list?
I searched for this, but I'm not sure I know the appropriate vocabulary to nail down the search.
Your List object is storing reference to the dummyInt object. If you want to store different values in List you have to create new int array every time you are adding it to List.
Reference:
https://msdn.microsoft.com/en-gb/library/4d43ts61(v=vs.90).aspx
Firstly, you don't need a 2-dimensional array if you're just storing a pair of coordinates. The first coordinate can go in the first element of a 1-dimensional array, and the second coordinate can go in the second element of the array. Secondly, the Clone method can be used to make a copy of an array object if you want to force a separate copy of the whole array to exist.
int[] dummyInt = new int[2];
List<int[]> test = new List<int[]>();
for (int i = 0; i < 100; i++)
{
dummyInt[0] = i;
for (int j = 0; j < 5; j++)
{
dummyInt[1] = j;
test.Add((int[])dummyInt.Clone());
}
}
foreach (int[] value in test)
{
Console.WriteLine("({0},{1})", value[0], value[1]);
}
And finally, an array might not be the best way to store a pair of coordinates. You might want to use a tuple or make your own structure. If you use a Value type (struct) instead of a Reference type (class), you don't need to clone each one.
struct Pair
{
public int x;
public int y;
}
public class Test
{
public static void Main()
{
Pair dummyInt = new Pair();
List<Pair> test = new List<Pair>();
for (int i = 0; i < 100; i++)
{
dummyInt.x = i;
for (int j = 0; j < 5; j++)
{
dummyInt.y = j;
test.Add(dummyInt);
}
}
foreach (Pair value in test)
{
Console.WriteLine("({0},{1})", value.x, value.y);
}
}
}
Note how the result is different if you change the word struct at the beginning to class.
I feel like this task should not be done this way...
My sequenceY length is not equal to number of steam numbers because you can't assign int[] length with int that have 0 as a starting value.
Therefore my sequenceY have a lot of 0 inside and I can't print the whole sequence. I even tried adding this after for loop:
sequenceY = new int[steamnumbercounter];
But it didn't work ... Why ?
My other question is how do programmers deal with sequences that have unknown length?
I managed to print only steam numbers but the task says print sequenceY not only part of it.
// 4. sequenceX[20] is made of random numbers from 1 to 30 ,
// sequenceY is made of steam numbers from sequenceX. Print sequneceY.
int[] nizx = new int[20];
int[] nizy = new int[20];
int n = 0;
int steamnumbercounter = 0;
Random rnd = new Random();
for (int i = 0; i < nizx.Length; i++)
{
nizx[i] = rnd.Next(1, 30);
if (nizx[i]%2==0)
{
nizy[n] = nizx[i];
n++;
steamnumbercounter++;
}
Console.Write("{0} , ", nizx[i]);
}
for (int i = 0; i < steamnumbercounter; i++)
{
Console.WriteLine("{0} , ",nizy[i]);
}
Partial code review along with an answer.
But it didn't work ... Why ?
That code didn't work because you're reassigning sequenceY to a completely new value.
My other question is how do programmers deal with sequences that have unknown length?
So, with that known we can do a few things here: create an array and use Array.Resize, use a List<T>, fill the initial array then swap it for one of the right size which is filled.
I'm going to assume a "steam" number is an even one.
Your naming is not good: nizx and nizy don't convey the meaning or line up with the problem.
I'm going to demonstrate the last option (since you stated that you don't know how to use many of the moderately complex parts of .NET in this class yet, which is fine): fill the initial array and swap it for a new one. This will run in O(n^2) time (sorta).
So, let's start with our source array.
int[] sequenceX = new int[20];
Next we'll define our destination array to be the same size as our source array. (This is the maximum number of values that could be stored in it, we'll shrink it later.)
int[] sequenceY = new int[sequenceX.Length];
Then we need a variable to hold how many numbers we found that meet our criteria:
int steamNumbers = 0;
And lastly, our Random.
Random random = new Random();
Then, we look through all our sequenceX as you did, but we'll update the logic a bit.
for (int i = 0; i < sequenceX.Length; i++)
{
sequenceX[i] = random.Next(1, 30);
if (sequenceX[i] % 2 == 0)
{
sequenceY[steamNumbers] = sequenceX[i];
steamNumbers++;
}
}
So our code looks almost the same as yours, but we have one more thing to do: since you only want sequenceY to contain steamNumbers we have to shrink it or something.
int[] tempSequenceY = sequenceY;
sequenceY = new int[steamNumbers];
for (int i = 0; i < steamNumbers; i++)
{
sequenceY[i] = tempSequenceY[i];
}
Now sequenceY only has your steam numbers in it.
Final code:
int[] sequenceX = new int[20];
int[] sequenceY = new int[sequenceX.Length];
int steamNumbers = 0;
Random random = new Random();
for (int i = 0; i < sequenceX.Length; i++)
{
sequenceX[i] = random.Next(1, 30);
if (sequenceX[i] % 2 == 0)
{
sequenceY[steamNumbers] = sequenceX[i];
steamNumbers++;
}
}
int[] tempSequenceY = sequenceY;
sequenceY = new int[steamNumbers];
for (int i = 0; i < steamNumbers; i++)
{
sequenceY[i] = tempSequenceY[i];
}
// Print your `sequenceY` here.
You could extract this to a method pretty easily as well:
public int[] GetSteamNumbers(int sequenceCount, int randomMinimum, int randomMaximum)
{
int[] sequenceX = new int[sequenceCount];
int[] sequenceY = new int[sequenceX.Length];
int steamNumbers = 0;
Random random = new Random();
for (int i = 0; i < sequenceX.Length; i++)
{
sequenceX[i] = random.Next(randomMinimum, randomMaximum);
if (sequenceX[i] % 2 == 0)
{
sequenceY[steamNumbers] = sequenceX[i];
steamNumbers++;
}
}
int[] tempSequenceY = sequenceY;
sequenceY = new int[steamNumbers];
for (int i = 0; i < steamNumbers; i++)
{
sequenceY[i] = tempSequenceY[i];
}
return sequenceY;
}
And then call it with:
int[] steamNumbers = GetSteamNumbers(20, 1, 30);
Of course, for the more advanced users (this doesn't help you, but it may help others) we can do something as follows using LINQ:
var random = new Random();
var sequenceY = Enumerable.Range(1, 20)
.Select(x => random.Next(1, 30))
.Where(x => x % 2 == 0)
.ToArray();
Which should have the same effect. (Just demonstrating that there are still things in C# to look forward to in the future.)
Disclaimer: I wrote this entire answer outside of the IDE and without actually compiling it, I make no guarantees to the accuracy of the code but the procedure itself should be fairly straight forward.
The thing with arrays in C# is that they are of fixed size.
You'll have to iterate through and re-create it or use a IEnumerable that has dynamic sizes, such as Lists.
Solution here would be to use a List that contains your integers and then you would use nizx.Add(rnd.Next(1, 30));
Elaborating on my comment above: You can create a 'fake' list by concatenating the values you need in a string, separated by commas. The string.Split(',') will give you the resulting array that you need.
Given a string of form "a,b,c,d" string.Split(',') will create the array ["a","b,"c","d"]. The code:
{
int[] nizx = new int[20];
string numberString = string.Empty;
int n = 0;
Random rnd = new Random();
for (int i = 0; i < nizx.Length; i++)
{
nizx[i] = rnd.Next(1, 30);
if (nizx[i] % 2 == 0)
{
numberString += nizx[i] + ",";
n++;
}
}
var numberArray = numberString.Split(',');
for (int i = 0; i < n; i++)
{
Console.WriteLine("{0} , ", numberArray[i]);
}
}