Develop Reference

Most efficient way to distribute non unique elements across multiple lists

Suppose that I have a list of integer or whatever
List<int> motherlist = { 1, 1, 2, 5, 7, 2, 2, 2, 6, 1 }
Console.WriteLine(children.Count); // 10
I would like to find all duplicates and not remove them from the list but to distribute them across other lists so the final count of all childrens should be the same as motherlist:
List<List<int>> children = { { 1, 2, 5, 7, 6 }, { 1, 2 }, { 1, 2 }, { 2 }}
Console.WriteLine(children.Sum(l => l.Count())); // 10 same as mother
I tried so far a brute force approach by looping through all elements of mother, comparing the elements with all other elements and to check for duplicates, If duplicate found I add it to a list of buckets (List of Lists) and so forth until the last elements.
But the brute force approach takes 7 CPU seconds for only a mother list of 300 items.
I imagine that if I had 1000 items this would take forever.
Is there a faster way to do this in C# .NET ?

I suggest grouping duplicates and then loop taking into account size of the groups:
public static IEnumerable<List<T>> MyDo<T>(IEnumerable<T> source,
IEqualityComparer<T> comparer = null) {
if (null == source)
throw new ArgumentNullException(nameof(source));
var groups = new Dictionary<T, List<T>>(comparer ?? EqualityComparer<T>.Default);
int maxLength = 0;
foreach (T item in source) {
if (!groups.TryGetValue(item, out var list))
groups.Add(item, list = new List<T>());
list.Add(item);
maxLength = Math.Max(maxLength, list.Count);
}
for (int i = 0; i < maxLength; ++i) {
List<T> result = new List<T>();
foreach (var value in groups.Values)
if (i < value.Count)
result.Add(value[i]);
yield return result;
}
}
Demo:
int[] source = new int[] { 1, 1, 2, 5, 7, 2, 2, 2, 6, 1 };
var result = MyDo(source).ToList();
string report = string.Join(Environment.NewLine, result
.Select(line => $"[{string.Join(", ", line)}]"));
Console.Write(report);
Outcome:
[1, 2, 5, 7, 6]
[1, 2]
[1, 2]
[2]
Stress Demo:
Random random = new Random(1234); // seed, the results to be reproducible
// We don't want 1000 items be forever; let's try 1_000_000 items
int[] source = Enumerable
.Range(1, 1_000_000)
.Select(x => random.Next(1, 1000))
.ToArray();
Stopwatch sw = new Stopwatch();
sw.Start();
var result = MyDo(source).ToList();
sw.Stop();
Console.WriteLine($"Time: {sw.ElapsedMilliseconds} ms");
Outcome: (may vary from workstation to workstation)
Time: 50 ms

I would GroupBy the elements of the list, and then use the count of elements to know the number of sublists an element has to be added in
List<int> motherlist = new List<int> { 1, 1, 2, 5, 7, 2, 2, 2, 6, 1 };
var childrens = motherlist.GroupBy(x => x).OrderByDescending(x => x.Count());
var result = new List<List<int>>();
foreach (var children in childrens)
{
for (var i = 0; i < children.Count(); i++)
{
if (result.Count() <= i) result.Add(new List<int>());
result[i].Add(children.Key);
}
}
Console.WriteLine("{");
foreach (var res in result)
{
Console.WriteLine($"\t{{ { string.Join(", ", res) } }}");
}
Console.WriteLine("}");
This outputs :
{
{ 2, 1, 5, 7, 6 }
{ 2, 1 }
{ 2, 1 }
{ 2 }
}

Just a quick shot, but it seems to work quite well...
using System;
using System.Collections.Generic;
using System.Linq;
namespace ConsoleApp2
{
class Program
{
static void Main(string[] args)
{
List<int> motherlist = new List<int> { 1, 1, 2, 5, 7, 2, 2, 2, 6, 1 };
var rnd = new Random(1);
for (int i = 0; i < 1000; i++)
{
motherlist.Add(rnd.Next(1, 200));
}
var resultLists = new List<IEnumerable<int>>();
while (motherlist.Any())
{
var subList = motherlist.Distinct().OrderBy(x => x).ToList();
subList.ForEach(x => motherlist.Remove(x));
resultLists.Add(subList);
}
}
}
}

You can use a Dictionary<int, int> to keep track of the number of occurrences of each element and build the child lists in a single iteration with O(n) time complexity(most of the time) and without any LINQ:
var motherlist = new List<int>() { 1, 1, 2, 5, 7, 2, 2, 2, 6, 1 };
var counts = new Dictionary<int, int>();
var children = new List<List<int>>();
foreach(var element in motherlist)
{
counts.TryGetValue(element, out int count);
counts[element] = ++count;
if (children.Count < count)
{
children.Add(new List<int>() { element });
}
else
{
children[count - 1].Add(element);
}
}
OUTPUT
{ 1, 2, 5, 7, 6 }
{ 1, 2 }
{ 2, 1 }
{ 2 }

optimization of an equation parameters using genetic sharp genetic algorithm

i have some data for example:
x=1, 2, 3, 4, 5
y=5, 4, 3, 2, 1
z=1, 2, 1, 1, 3
t=9, 12, 9, 9, 15.
i know that t=ax+by+c*z. here i know a=1,b=1 and c=3 and i need to find them by genetic algorithm by Genetic Sharp. i wrote the below code but i get the boundary values all times of -100,-100,-100.
can you help me on it?
NOTE that i need the GA to search on a large boundary for example here i selected -100 to 100 for each of a,b and c. in real case i need to bound it to -100000 to 10000 for example. also i need the at least 10 fraction digits also.
double[] x = new double[] { 1, 2, 3, 4, 5 };
double[] y = new double[] { 5, 4, 3, 2, 1 };
double[] z = new double[] { 1, 2, 1, 1, 3 };
double[] t = new double[] { 9, 12, 9, 9, 15 };
var selection = new EliteSelection();
var crossover = new UniformCrossover();
var mutation = new FlipBitMutation();
var fitness = new FuncFitness((c) =>
{
var vals = (c as FloatingPointChromosome).ToFloatingPoints();
double err = 0;
for (int i = 0; i < x.Count(); i++)
{
err += Math.Pow((t[i] - (vals[0] * x[i] + vals[1] * y[i] + vals[2] * z[i])), 2);
}
return err;
});
var chromosome = new FloatingPointChromosome(
new double[] { -100, -100, -100 },
new double[] { 100, 100, 100 },
new int[] { 64, 64, 64 },
new int[] { 10, 10, 10 });
var population = new Population(100, 100, chromosome);
var ga = new GeneticAlgorithm(population, fitness, selection, crossover, mutation);
ga.Termination = new GenerationNumberTermination(300);
Console.WriteLine("GA running...");
ga.Start();

I think that is because the algorith wants to maximize the Fitness Function and in the way that you wrote it, the algo will find the worst possible values for your variables.
I changed the error calculation in the Fitness Function to
err -= Math.Pow((t[i] - (vals[0] * x[i] + vals[1] * y[i] + vals[2] * z[i])), 2);
so now the maximum possible value for err is zero and the algorith will try to find the variable values for it.
Got very close results running this code, although couldn't get the exact ones.

In this problem, by changing the fitness function and mutation the correct answer could be achieved using GeneticSharp. the modified code is as follows:
// x = 1, 2, 3, 4, 5
// y = 5, 4, 3, 2, 1
// z = 1, 2, 1, 1, 3
// t = 9, 12, 9, 9, 15.
// t = a*x + b*y + c*z
// a,b,c =???
double[] x = new double[] { 1, 2, 3, 4, 5 };
double[] y = new double[] { 5, 4, 3, 2, 1 };
double[] z = new double[] { 1, 2, 1, 1, 3 };
double[] t = new double[] { 9, 12, 9, 9, 15 };
var chromosome = new FloatingPointChromosome(
new double[] { -10, -10, -10 },
new double[] { 10.0, 10.0, 10.0 },
new int[] { 64, 64, 64 },
new int[] { 0, 0, 0 });
var fitness = new FuncFitness((c) =>
{
var fc = c as FloatingPointChromosome;
double err = 0;
var values = fc.ToFloatingPoints();
var pa = values[0];
var pb = values[1];
var pc = values[2];
var fitness_e = 0.0;
for (int i = 0; i < x.Count(); i++)
{
try
{
var expected_result = t[i];
var equations = (pa * x[i] + pb * y[i] + pc * z[i]);
var diff = Math.Abs(equations - expected_result);
fitness_e += diff;
}
catch (Exception)
{
return double.MinValue;
}
}
return fitness_e * -1;
});
var population = new Population(100, 500, chromosome);
var selection = new EliteSelection();
var crossover = new ThreeParentCrossover();
var mutation = new UniformMutation(true);
var ga = new GeneticAlgorithm(population, fitness, selection, crossover, mutation);
ga.Termination = new OrTermination(
new GenerationNumberTermination(5000),
new FitnessStagnationTermination(2000));
ga.CrossoverProbability = 0.1f;
ga.MutationProbability = 0.4f;
ga.Reinsertion = new ElitistReinsertion();
Console.WriteLine("GA running...");
var latestFitness = 0.0;
ga.GenerationRan += (sender1, e1) =>
{
var bestChromosome = ga.BestChromosome as FloatingPointChromosome;
var bestFitness = bestChromosome.Fitness.Value;
if (bestFitness != latestFitness)
{
latestFitness = bestFitness;
var phenotype = bestChromosome.ToFloatingPoints();
Console.WriteLine(
"Generation {0}: {1},{2},{3} fitness: {4}",
ga.GenerationsNumber,
phenotype[0],
phenotype[1],
phenotype[2],
bestFitness
);
}
};
ga.Start();
Console.WriteLine("GA End");

How can I separate a multi-dimension (2D) array by rows into multiple 1D arrays?

I am programming in C# and I currently have the 2D array below:
int[,] results = {
{ 4, 7, 9, 3, 8, 6, 4},
{ 4, 8, 6, 4, 8, 5, 6},
{ 7, 3, 9, 2, 2, 1, 8}
};
I'd like to create a loop or function which outputs 3 separate arrays, copying the values from each row.
e.g. output:
row1 = {4, 7, 9, 3, 8, 6, 4}
row2 = {4, 8, 6, 4, 8, 5, 6}
etc
I have been able to copy the values of each row into a separate string which is then written in the console with this line of code:
for (int a = 1; a < (rows+1); a++)
{
for (int b = 1; b < (columns+1); b++)
{
scores = scores + " " + results[(a-1), (b-1)];
}
scores = "";
}

you need a convertion from a 2D array into a jagged array
int[,] array = { { 4, 7, 9, 3, 8, 6, 4},
{ 4, 8, 6, 4, 8, 5, 6},
{ 7, 3, 9, 2, 2, 1, 8}
};
int[][] jagged = new int[array.GetLength(0)][];
for (int i = 0; i < array.GetLength(0); i++)
{
int[] row = new int[array.GetLength(1)];
for (int j = 0; j < array.GetLength(1); j++)
{
row[j] = array[i, j];
}
jagged[i] = row;
}
int[] row1 = jagged[0];
int[] row2 = jagged[1];
int[] row3 = jagged[2];
difference between multidimensional array and jagged array:
//multidimensional array
int[,] multi = { { 7, 2, 6, 1 }, { 3, 5, 4, 8 } };
//array of arrays (jagged array)
int[][] jagged = new int[][] { new int[] { 7, 2, 6, 1 }, new int[] { 3, 5, 4, 8 } };

LINQ variant:
var m0 = new int[,] { { 1, 2 }, { 3, 4 } };
var rows = Enumerable.Range(0, m0.GetLength(0)).Select(i => Enumerable.Range(0, m0.GetLength(1)).Select(j => m0[i, j]));
foreach (var r in rows) Console.WriteLine(string.Join(" ", r));
Console.ReadLine();
a 'smarter' variant (flattern an array and take by N values):
var rows = m0.Cast<int>().Select((v, i) => new { i = i / m0.GetLength(1), v }).GroupBy(e => e.i).Select(g => g.Select(e => e.v));

Just a piece of cake using JsonSoft as
var json = JsonConvert.SerializeObject(results);
int[][] arr = JsonConvert.DeserializeObject<int[][]>(json);
int[] arr1 = arr[0];
int[] arr2 = arr[1];
int[] arr3 = arr[2];

Aggregate sub-lists using LINQ

I have a list of objects (i.e. integers) and I want to aggregate sub-lists with LINQ.
For example:
Original list: [ 1, 4, 5, 3, 4, 10, 4, 12 ]
Sub-lists: [ [1,4,5,3], [4,5,3,4], [5,3,4,10], [3,4,10,4], [4,10,4,12] ]
Result (Aggregated List): [ 5, 5, 10, 10, 12 ]
I want to create the maximum of a sub-list for each element containing itself and the following n = 3 elements. Is this possible with LINQ or do I need to create my own aggregation mechanism?
Thanks in advance,Christian

public IEnumerable<IEnumerable<int>> GetSubLists(int[] collection)
{
for(int i = 0; i< collection.Length - 3; i++)
yield return collection.Skip(i).Take(4);
}
GetSubLists(original).Select(l => l.Max());
Or in one line
int[] original = {1, 4, 5, 3, 4, 10, 4, 12 };
int chunkCount = 4;
Enumerable.Range(0, original.Length - chunkCount + 1).Select(i => original.Skip(i).Take(chunkCount))
.Select(l => l.Max());

var result = sublists.Select(sl => sl.Max());
// [5,5,10,10,12]
Creating sub-lists:
List<int> original = new List<int> { 1, 4, 5, 3, 4, 10, 4, 12 };
int sublistSize = 4;
// check if original size is greater than required sublistSize
var sublists = Enumerable.Range(0, original.Count - sublistSize + 1)
.Select(i => original.GetRange(i, sublistSize));
// [[1,4,5,3],[4,5,3,4],[5,3,4,10],[3,4,10,4],[4,10,4,12]]

IEnumerable<int[]> GetLists (int[] list, int size )
{
return Enumerable.Range(0, list.Length - size + 1).Select(x => list.Skip(x).Take(size).ToArray());
}
Sample:
var list = new[] {1, 4, 5, 3, 4, 10, 4, 12};
var max = GetLists(list, 4).Select(x => x.Max()).ToArray();

The Sub-lists intermediate result can be constructed with a "Sliding Window" function.
The desired Result then is the function Max() mapped over the windows with Select().
var originalList = new [] {1, 4, 5, 3, 4, 10, 4, 12};
var sublists = originalList.Window(4); // [ [1,4,5,3], [4,5,3,4], ... ]
var result = sublists.Select(Enumerable.Max); // [ 5, 5, 10, 10, 12 ]
Efficient Window function:
public static IEnumerable<IEnumerable<T>> Window<T>(this IEnumerable<T> source,
int windowSize)
{
if(windowSize < 1) throw new ArgumentException("windowSize must positive", "windowSize");
var q = new Queue<T>(windowSize);
foreach(var e in source)
{
q.Enqueue(e);
if(q.Count < windowSize) continue; // queue not full yet
yield return q;
q.Dequeue();
}
}

Linq on 2 arrays in 1 loop?

Is it possible to improve the efficiency of those linq requests? I use two different loops...
Can you help me to optimize this code?
double[] x = { 2, 3, 1, 5, 7, 2, 3 };
double[] y = { 1, 2, 3, 4, 5, 6, 7 };
IEnumerable<int> range = Enumerable.Range(0, x.Length);
double[] y_sorted = (from n in range orderby x[n] select y[n]).ToArray();
double[] x_sorted = (from n in range orderby x[n] select x[n]).ToArray();
This code in python is like that if you prefer:
x_index = argsort(x)
x_sorted = [x[i] for i in x_index]
y_sorted = [y[i] for i in x_index]
you will notice that, in this python code, i use only one sort. that's not the case of this c# code.
we should get at the end:
x_sorted = { 1, 2, 2, 3, 3, 5, 7 }
y_sorted = { 3, 1, 6, 2, 7, 4, 5 }
Fred
Edit:
I use the program of Diadistis (after a small correction)
So here we go:
Array.Sort(x, y) (0.05) is the fastest way following (0.18) by
int[] x_index = Enumerable.Range(0, x.Length).OrderBy(i => x[i]).ToArray();
double[] x_sorted = x_index.Select(i => x[i]).ToArray();
double[] y_sorted = x_index.Select(i => y[i]).ToArray();
The other solutions are quite equivalent (~0.35) in time consumption on my pc.
If someone have an interesting idea, I will profile it and update this post.

It's ok but I'd prefer a simpler syntax :
double[] x = { 2, 3, 1, 5, 7, 2, 3 };
double[] y = { 2, 3, 1, 5, 7, 2, 3 };
double[] x_sorted = x.OrderBy(d => d).ToArray();
double[] y_sorted = y.OrderBy(d => d).ToArray();
Edit:
Argh... I failed to spot that this was an associative array sort.
double[] x = { 2, 3, 1, 5, 7, 2, 3 };
double[] y = { 1, 2, 3, 4, 5, 6, 7 };
double[] y_sorted = y.Clone() as double[];
double[] x_sorted = x.Clone() as double[];
Array.Sort(x_sorted, y_sorted);
Edit 2 1/2
And some performance tests :
public class Program
{
delegate void SortMethod(double[] x, double[] y);
private const int ARRAY_SIZE = 3000000;
private static Random RandomNumberGenerator = new Random();
private static double[] x = GenerateTestData(ARRAY_SIZE);
private static double[] y = GenerateTestData(ARRAY_SIZE);
private static double[] GenerateTestData(int count)
{
var data = new double[count];
for (var i = 0; i < count; i++)
{
data[i] = RandomNumberGenerator.NextDouble();
}
return data;
}
private static void SortMethod1(double[] x, double[] y)
{
Array.Sort(x, y);
}
private static void SortMethod2(double[] x, double[] y)
{
IEnumerable<int> range = Enumerable.Range(0, x.Length);
x = (from n in range orderby x[n] select y[n]).ToArray();
y = (from n in range orderby x[n] select x[n]).ToArray();
}
private static void SortMethod3(double[] x, double[] y)
{
int[] x_index =
Enumerable.Range(0, x.Length).OrderBy(i => x[i]).ToArray();
x = x_index.Select(i => x[i]).ToArray();
y = x_index.Select(i => y[i]).ToArray();
}
private static void SortMethod4(double[] x, double[] y)
{
int[] range =
Enumerable.Range(0, x.Length).OrderBy(i => x[i]).ToArray();
var q = (
from n in range
orderby x[n]
select new { First = x[n], Second = y[n] }).ToArray();
x = q.Select(t => t.First).ToArray();
y = q.Select(t => t.Second).ToArray();
}
private static void SortMethodPerformanceTest(SortMethod sortMethod)
{
double[] y_sorted = y.Clone() as double[];
double[] x_sorted = x.Clone() as double[];
var sw = new Stopwatch();
sw.Start();
sortMethod.Invoke(x_sorted, y_sorted);
sw.Stop();
Console.WriteLine(
string.Format(
"{0} : {1}",
sortMethod.Method.Name,
sw.Elapsed));
}
static void Main(string[] args)
{
Console.WriteLine("For array length : " + ARRAY_SIZE);
Console.WriteLine("------------------------------");
SortMethodPerformanceTest(SortMethod1);
SortMethodPerformanceTest(SortMethod2);
SortMethodPerformanceTest(SortMethod3);
SortMethodPerformanceTest(SortMethod4);
Console.WriteLine("Press any key to continue...");
Console.ReadKey();
}
}
And the results :
For array length : 3000000
------------------------------
SortMethod1 : 00:00:00.6088503 // Array.Sort(Array, Array)
SortMethod2 : 00:00:07.9583779 // Original
SortMethod3 : 00:00:04.5023336 // dtb's Linq Alternative
SortMethod4 : 00:00:06.6115911 // Christian's Linq Alternative

If I read your code correctly, you're trying to sort two arrays by the elements of the first array.
The literal translation of your Python code to C# would be something like this:
int[] x_index = Enumerable.Range(0, x.Length).OrderBy(i => x[i]).ToArray();
double[] x_sorted = x_index.Select(i => x[i]).ToArray();
double[] y_sorted = x_index.Select(i => y[i]).ToArray();
Alternatively, you could zip the two arrays to an enumerable of tuples and then sort this by the first item:
var sorted = Enumerable.Zip(x, y, Tuple.Create<double, double>)
.OrderBy(t => t.Item1)
.ToArray();

This could be faster as you only sort once:
var q =
(from n in range
orderby x[n]
select new { First = x[n], Second = y[n] }).ToArray();
double[] x_sorted = q.Select(t => t.First).ToArray();
double[] y_sorted = q.Select(t => t.Second).ToArray();

You can also do following. Keeping in view that if you meant y[n] as in leppie comment
double[] x = { 2, 3, 1, 5, 7, 2, 3 };       
double[] y = { 2, 3, 1, 5, 7, 2, 3 };
Array.Sort<double>(x);
Array.Sort<double>(y);
Update
Should be as following to get correct result.
double[] x = { 2, 3, 1, 5, 7, 2, 3 };
double[] y = { 1, 2, 3, 4, 5, 6, 7 };
Array.Sort<double, double>(x, y);