I have this List<List<int>>:
{{1,2},{1,3},{1,4},{2,3},{2,4},{3,4}}
In this list there are 6 list, which contain numbers from 1 to 4, and the occurrence of each number is 3;
I want to filter it in order to get:
{{1,2}{1,3}{2,4}{3,4}}
here the occurrence of each number is 2;
the lists are generated dynamically and I want to be able to filter also dynamically, base on the occurrence;
Edit-More Details
I need to count how many times a number is contain in the List<List<int>>, for the above example is 3. Then I want to exclude lists from the List<List<int>> in order to reduce the number of times from 3 to 2,
The main issue for me was to find a way to not block my computer :), and also to get each number appear for 2 times (mandatory);
Well if it's always a combination of 2 numbers, and they have to appear N times on the list, it means that depending on the N You gonna have:
4 (different digits) x 2 (times hey have to appear) = 8 digits = 4 pairs
4 x 3 (times) = 12 = 6 (pairs)
4 x 4 = 16 = 8 pairs
That means - that from 6 pairs we know we must select 4 pairs that best match the criteria
so based on the basic combinatorics (https://www.khanacademy.org/math/probability/probability-and-combinatorics-topic/permutations/v/permutation-formula)
we have a 6!/2! = (6*5*4*3*2*1)/(2*1)= 360 possible permutations
basically You can have 360 different ways how You put the the second list together.
because it doesn't matter how You arrange the items in the list (the order of items in the list) then the number of possible combinations is 6!/(2!*4!) = 15
https://www.khanacademy.org/math/probability/probability-and-combinatorics-topic/combinations-combinatorics/v/combination-formula
so the thing is - you have 15 possible answers to Your question.
Which means - you only need to loop over it for 15 times.
There are only 15 ways to chose 4 items out of the list of 6
seems like this is a solution to Your - "killing the machine" question.
so next question - how do we find all the possible 'combination'
Let's define all the possible items that we can pick from the input array
for example 1-st, 2-nd, 3-rd and 4-th..
1,2,3,4....... 1,2,3,5...... 1,2,3,6 ...
All the combinations would be (from here https://stackoverflow.com/a/10629938/444149)
static IEnumerable<IEnumerable<T>> GetKCombs<T>(IEnumerable<T> list, int length) where T : IComparable
{
if (length == 1) return list.Select(t => new T[] { t });
return GetKCombs(list, length - 1)
.SelectMany(t => list.Where(o => o.CompareTo(t.Last()) > 0),
(t1, t2) => t1.Concat(new T[] { t2 }));
}
and invoke with (because there are 6 items to pick from, who's indexed are 0,1,2,3,4 and 5)
var possiblePicks = GetKCombs(new List<int> { 0, 1, 2, 3, 4, 5 }, 4);
we get 15 possible combinations
so now - we try taking 4 elements out of the first list, and check if they match the criteria.. if not.. then take another combination
var data = new List<List<int>>
{
new List<int> { 1,2 },
new List<int> { 1,3 },
new List<int> { 1,4 },
new List<int> { 2,3 },
new List<int> { 2,4 },
new List<int> { 3,4 }
};
foreach (var picks in possiblePicks)
{
var listToTest = new List<List<int>>(4);
foreach (var i in picks)
listToTest.Add(data[i]);
var ok = Check(listToTest, 2);
if (ok)
break;
}
private bool Check(List<List<int>> listToTest, int limit)
{
Dictionary<int, int> ret = new Dictionary<int, int>();
foreach (var inputElem in listToTest)
{
foreach (var z in inputElem)
{
var returnCount = ret.ContainsKey(z) ? ret[z] : 0;
if (!ret.ContainsKey(z))
ret.Add(z, returnCount + 1);
else
ret[z]++;
}
}
return ret.All(p => p.Value == limit);
}
I'm sure this can be further optimized to minimize the amount of iterations other the 'listToTest'
Also, this is a lazy implementation (Ienumerable) - so if it so happens that the very first (or second) combination is successful, it stop iterating.
I accepted the Marty's answer because fixed my issue, any way trying to use his method for larger lists, I found my self blocking again my computer so I start looking for another method and I end it up with this one:
var main = new List<HashSet<int>> {
new HashSet<int> {1,2},
new HashSet<int> {1,3},
new HashSet<int> {1,4},
new HashSet<int> {2,3},
new HashSet<int> {2,4},
new HashSet<int> {3,4} };
var items = new HashSet<int>(from l in main from p in l select p); //=>{1,2,3,4}
for (int i =main.Count-1;i-->0; )
{
var occurence=items.Select(a=> main.Where(x => x.Contains(a)).Count()).ToList();
var occurenceSum = 0;
foreach(var j in main[i])
{
occurenceSum += occurence[j - 1];
if (occurenceSum==6) //if both items have occurence=3, then the sum=6, then I can remove that list!
{
main.RemoveAt(i);
}
}
}
This is a weired problem, I have implemented simple quick sort as follows..
static void Main(string[] args)
{
List<int> unsorted = new List<int> { 1, 3, 5, 7, 9, 8, 6, 4, 2 };
List<int> sorted = quicksort(unsorted);
Console.WriteLine(string.Join(",", sorted));
Console.ReadKey();
}
private static List<T> quicksort<T>(List<T> arr) where T : IComparable<T>
{
List<T> loe = new List<T>(), gt = new List<T>();
if (arr.Count < 2)
return arr;
int pivot = arr.Count / 2;
T pivot_val = arr[pivot];
arr.RemoveAt(pivot);
foreach (T i in arr)
{
if (i.CompareTo(pivot_val) <= 0)
loe.Add(i);
else
gt.Add(i);
}
List<T> resultSet = new List<T>();
resultSet.AddRange(quicksort(loe));
gt.Add(pivot_val);
resultSet.AddRange(quicksort(gt));
return resultSet;
}
Output is : 1,2,3,4,5,6,7,8,9
But When I use any negative number in the unsorted list there is a stackoverflow error,
for example
if List unsorted = new List { 1, 3, 5, 7, 9, 8, 6, 4, 2, -1 };
Now there is a stackoverflow..
What's going on? Why this is not working ?
Your algorithm has a bug. Consider the simplest input list { 1, -1 }. Let's step through your logic.
You first choose a pivot index, Count / 2, which is 1.
You then remove the pivot element at index 1 (-1) from the arr list.
Next you compare each remaining element in the arr list (there's just the 1 at index 0) with the pivot.
The 1 is greater than the pivot (-1) so you add it to the gt list.
Next you quicksort the loe list, which is empty. That sort returns an empty list, which you add to the result set.
You then add the pivot value to the end of the gt list, so the gt list now looks like this: { 1, -1 }. Notice that this is the exact same list as you started with.
You then attempt to quicksort the gt list. Since you are calling the quicksort routine with the same input, the same sequence of steps happens again, until the stack overflows.
It seems the error in your logic is that you blindly add the pivot to the gt list without comparing it to anything. I'll leave it to you to figure out how to make it work.
Edited to add: I'm assuming this is a homework assignment, but if it's not, I would highly recommend using .NET's built in Sort() method on List<T>. It has been highly optimized and heavily tested, and will most likely perform better than anything home-brewed. Why reinvent the wheel?
if you don't have a debugger try this...
foreach (T i in arr)
{
if (i.CompareTo(pivot_val) <= 0)
{
loe.Add(i);
Console.WriteLine("loe.add " + i.ToString());
}
else
{
gt.Add(i);
Console.WriteLine("gt.add " + i.ToString());
}
}
This might sound lame, but I have not been able to find a really good explanation of Aggregate.
Good means short, descriptive, comprehensive with a small and clear example.
The easiest-to-understand definition of Aggregate is that it performs an operation on each element of the list taking into account the operations that have gone before. That is to say it performs the action on the first and second element and carries the result forward. Then it operates on the previous result and the third element and carries forward. etc.
Example 1. Summing numbers
var nums = new[]{1,2,3,4};
var sum = nums.Aggregate( (a,b) => a + b);
Console.WriteLine(sum); // output: 10 (1+2+3+4)
This adds 1 and 2 to make 3. Then adds 3 (result of previous) and 3 (next element in sequence) to make 6. Then adds 6 and 4 to make 10.
Example 2. create a csv from an array of strings
var chars = new []{"a","b","c","d"};
var csv = chars.Aggregate( (a,b) => a + ',' + b);
Console.WriteLine(csv); // Output a,b,c,d
This works in much the same way. Concatenate a a comma and b to make a,b. Then concatenates a,b with a comma and c to make a,b,c. and so on.
Example 3. Multiplying numbers using a seed
For completeness, there is an overload of Aggregate which takes a seed value.
var multipliers = new []{10,20,30,40};
var multiplied = multipliers.Aggregate(5, (a,b) => a * b);
Console.WriteLine(multiplied); //Output 1200000 ((((5*10)*20)*30)*40)
Much like the above examples, this starts with a value of 5 and multiplies it by the first element of the sequence 10 giving a result of 50. This result is carried forward and multiplied by the next number in the sequence 20 to give a result of 1000. This continues through the remaining 2 element of the sequence.
Live examples: http://rextester.com/ZXZ64749
Docs: http://msdn.microsoft.com/en-us/library/bb548651.aspx
Addendum
Example 2, above, uses string concatenation to create a list of values separated by a comma. This is a simplistic way to explain the use of Aggregate which was the intention of this answer. However, if using this technique to actually create a large amount of comma separated data, it would be more appropriate to use a StringBuilder, and this is entirely compatible with Aggregate using the seeded overload to initiate the StringBuilder.
var chars = new []{"a","b","c", "d"};
var csv = chars.Aggregate(new StringBuilder(), (a,b) => {
if(a.Length>0)
a.Append(",");
a.Append(b);
return a;
});
Console.WriteLine(csv);
Updated example: http://rextester.com/YZCVXV6464
It partly depends on which overload you're talking about, but the basic idea is:
Start with a seed as the "current value"
Iterate over the sequence. For each value in the sequence:
Apply a user-specified function to transform (currentValue, sequenceValue) into (nextValue)
Set currentValue = nextValue
Return the final currentValue
You may find the Aggregate post in my Edulinq series useful - it includes a more detailed description (including the various overloads) and implementations.
One simple example is using Aggregate as an alternative to Count:
// 0 is the seed, and for each item, we effectively increment the current value.
// In this case we can ignore "item" itself.
int count = sequence.Aggregate(0, (current, item) => current + 1);
Or perhaps summing all the lengths of strings in a sequence of strings:
int total = sequence.Aggregate(0, (current, item) => current + item.Length);
Personally I rarely find Aggregate useful - the "tailored" aggregation methods are usually good enough for me.
Super short
Aggregate works like fold in Haskell/ML/F#.
Slightly longer
.Max(), .Min(), .Sum(), .Average() all iterates over the elements in a sequence and aggregates them using the respective aggregate function. .Aggregate () is generalized aggregator in that it allows the developer to specify the start state (aka seed) and the aggregate function.
I know you asked for a short explaination but I figured as others gave a couple of short answers I figured you would perhaps be interested in a slightly longer one
Long version with code
One way to illustrate what does it could be show how you implement Sample Standard Deviation once using foreach and once using .Aggregate. Note: I haven't prioritized performance here so I iterate several times over the colleciton unnecessarily
First a helper function used to create a sum of quadratic distances:
static double SumOfQuadraticDistance (double average, int value, double state)
{
var diff = (value - average);
return state + diff * diff;
}
Then Sample Standard Deviation using ForEach:
static double SampleStandardDeviation_ForEach (
this IEnumerable<int> ints)
{
var length = ints.Count ();
if (length < 2)
{
return 0.0;
}
const double seed = 0.0;
var average = ints.Average ();
var state = seed;
foreach (var value in ints)
{
state = SumOfQuadraticDistance (average, value, state);
}
var sumOfQuadraticDistance = state;
return Math.Sqrt (sumOfQuadraticDistance / (length - 1));
}
Then once using .Aggregate:
static double SampleStandardDeviation_Aggregate (
this IEnumerable<int> ints)
{
var length = ints.Count ();
if (length < 2)
{
return 0.0;
}
const double seed = 0.0;
var average = ints.Average ();
var sumOfQuadraticDistance = ints
.Aggregate (
seed,
(state, value) => SumOfQuadraticDistance (average, value, state)
);
return Math.Sqrt (sumOfQuadraticDistance / (length - 1));
}
Note that these functions are identical except for how sumOfQuadraticDistance is calculated:
var state = seed;
foreach (var value in ints)
{
state = SumOfQuadraticDistance (average, value, state);
}
var sumOfQuadraticDistance = state;
Versus:
var sumOfQuadraticDistance = ints
.Aggregate (
seed,
(state, value) => SumOfQuadraticDistance (average, value, state)
);
So what .Aggregate does is that it encapsulates this aggregator pattern and I expect that the implementation of .Aggregate would look something like this:
public static TAggregate Aggregate<TAggregate, TValue> (
this IEnumerable<TValue> values,
TAggregate seed,
Func<TAggregate, TValue, TAggregate> aggregator
)
{
var state = seed;
foreach (var value in values)
{
state = aggregator (state, value);
}
return state;
}
Using the Standard deviation functions would look something like this:
var ints = new[] {3, 1, 4, 1, 5, 9, 2, 6, 5, 4};
var average = ints.Average ();
var sampleStandardDeviation = ints.SampleStandardDeviation_Aggregate ();
var sampleStandardDeviation2 = ints.SampleStandardDeviation_ForEach ();
Console.WriteLine (average);
Console.WriteLine (sampleStandardDeviation);
Console.WriteLine (sampleStandardDeviation2);
IMHO
So does .Aggregate help readability? In general I love LINQ because I think .Where, .Select, .OrderBy and so on greatly helps readability (if you avoid inlined hierarhical .Selects). Aggregate has to be in Linq for completeness reasons but personally I am not so convinced that .Aggregate adds readability compared to a well written foreach.
A picture is worth a thousand words
Reminder:
Func<X, Y, R> is a function with two inputs of type X and Y, that returns a result of type R.
Enumerable.Aggregate has three overloads:
Overload 1:
A Aggregate<A>(IEnumerable<A> a, Func<A, A, A> f)
Example:
new[]{1,2,3,4}.Aggregate((x, y) => x + y); // 10
This overload is simple, but it has the following limitations:
the sequence must contain at least one element,
otherwise the function will throw an InvalidOperationException.
elements and result must be of the same type.
Overload 2:
B Aggregate<A, B>(IEnumerable<A> a, B bIn, Func<B, A, B> f)
Example:
var hayStack = new[] {"straw", "needle", "straw", "straw", "needle"};
var nNeedles = hayStack.Aggregate(0, (n, e) => e == "needle" ? n+1 : n); // 2
This overload is more general:
a seed value must be provided (bIn).
the collection can be empty,
in this case, the function will yield the seed value as result.
elements and result can have different types.
Overload 3:
C Aggregate<A,B,C>(IEnumerable<A> a, B bIn, Func<B,A,B> f, Func<B,C> f2)
The third overload is not very useful IMO.
The same can be written more succinctly by using overload 2 followed by a function that transforms its result.
The illustrations are adapted from this excellent blogpost.
Aggregate is basically used to Group or Sum up data.
According to MSDN
"Aggregate Function Applies an accumulator function over a sequence."
Example 1: Add all the numbers in a array.
int[] numbers = new int[] { 1,2,3,4,5 };
int aggregatedValue = numbers.Aggregate((total, nextValue) => total + nextValue);
*important: The initial aggregate value by default is the 1 element in the sequence of collection.
i.e: the total variable initial value will be 1 by default.
variable explanation
total: it will hold the sum up value(aggregated value) returned by the func.
nextValue: it is the next value in the array sequence. This value is than added to the aggregated value i.e total.
Example 2: Add all items in an array. Also set the initial accumulator value to start adding with from 10.
int[] numbers = new int[] { 1,2,3,4,5 };
int aggregatedValue = numbers.Aggregate(10, (total, nextValue) => total + nextValue);
arguments explanation:
the first argument is the initial(starting value i.e seed value) which will be used to start addition with the next value in the array.
the second argument is a func which is a func that takes 2 int.
1.total: this will hold same as before the sum up value(aggregated value) returned by the func after the calculation.
2.nextValue: : it is the next value in the array sequence. This value is than added to the aggregated value i.e total.
Also debugging this code will give you a better understanding of how aggregate work.
In addition to all the great answers here already, I've also used it to walk an item through a series of transformation steps.
If a transformation is implemented as a Func<T,T>, you can add several transformations to a List<Func<T,T>> and use Aggregate to walk an instance of T through each step.
A more concrete example
You want to take a string value, and walk it through a series of text transformations that could be built programatically.
var transformationPipeLine = new List<Func<string, string>>();
transformationPipeLine.Add((input) => input.Trim());
transformationPipeLine.Add((input) => input.Substring(1));
transformationPipeLine.Add((input) => input.Substring(0, input.Length - 1));
transformationPipeLine.Add((input) => input.ToUpper());
var text = " cat ";
var output = transformationPipeLine.Aggregate(text, (input, transform)=> transform(input));
Console.WriteLine(output);
This will create a chain of transformations: Remove leading and trailing spaces -> remove first character -> remove last character -> convert to upper-case. Steps in this chain can be added, removed, or reordered as needed, to create whatever kind of transformation pipeline is required.
The end result of this specific pipeline, is that " cat " becomes "A".
This can become very powerful once you realize that T can be anything. This could be used for image transformations, like filters, using BitMap as an example;
Learned a lot from Jamiec's answer.
If the only need is to generate CSV string, you may try this.
var csv3 = string.Join(",",chars);
Here is a test with 1 million strings
0.28 seconds = Aggregate w/ String Builder
0.30 seconds = String.Join
Source code is here
Definition
Aggregate method is an extension method for generic collections. Aggregate method applies a function to each item of a collection. Not just only applies a function, but takes its result as initial value for the next iteration. So, as a result, we will get a computed value (min, max, avg, or other statistical value) from a collection.
Therefore, Aggregate method is a form of safe implementation of a recursive function.
Safe, because the recursion will iterate over each item of a collection and we can’t get any infinite loop suspension by wrong exit condition. Recursive, because the current function’s result is used as a parameter for the next function call.
Syntax:
collection.Aggregate(seed, func, resultSelector);
seed - initial value by default;
func - our recursive function. It can be a lambda-expression, a Func delegate or a function type T F(T result, T nextValue);
resultSelector - it can be a function like func or an expression to compute, transform, change, convert the final result.
How it works:
var nums = new[]{1, 2};
var result = nums.Aggregate(1, (result, n) => result + n); //result = (1 + 1) + 2 = 4
var result2 = nums.Aggregate(0, (result, n) => result + n, response => (decimal)response/2.0); //result2 = ((0 + 1) + 2)*1.0/2.0 = 3*1.0/2.0 = 3.0/2.0 = 1.5
Practical usage:
Find Factorial from a number n:
int n = 7;
var numbers = Enumerable.Range(1, n);
var factorial = numbers.Aggregate((result, x) => result * x);
which is doing the same thing as this function:
public static int Factorial(int n)
{
if (n < 1) return 1;
return n * Factorial(n - 1);
}
Aggregate() is one of the most powerful LINQ extension method, like Select() and Where(). We can use it to replace the Sum(), Min(). Max(), Avg() functionality, or to change it by implementing addition context:
var numbers = new[]{3, 2, 6, 4, 9, 5, 7};
var avg = numbers.Aggregate(0.0, (result, x) => result + x, response => (double)response/(double)numbers.Count());
var min = numbers.Aggregate((result, x) => (result < x)? result: x);
More complex usage of extension methods:
var path = #“c:\path-to-folder”;
string[] txtFiles = Directory.GetFiles(path).Where(f => f.EndsWith(“.txt”)).ToArray<string>();
var output = txtFiles.Select(f => File.ReadAllText(f, Encoding.Default)).Aggregate<string>((result, content) => result + content);
File.WriteAllText(path + “summary.txt”, output, Encoding.Default);
Console.WriteLine(“Text files merged into: {0}”, output); //or other log info
This is an explanation about using Aggregate on a Fluent API such as Linq Sorting.
var list = new List<Student>();
var sorted = list
.OrderBy(s => s.LastName)
.ThenBy(s => s.FirstName)
.ThenBy(s => s.Age)
.ThenBy(s => s.Grading)
.ThenBy(s => s.TotalCourses);
and lets see we want to implement a sort function that take a set of fields, this is very easy using Aggregate instead of a for-loop, like this:
public static IOrderedEnumerable<Student> MySort(
this List<Student> list,
params Func<Student, object>[] fields)
{
var firstField = fields.First();
var otherFields = fields.Skip(1);
var init = list.OrderBy(firstField);
return otherFields.Skip(1).Aggregate(init, (resultList, current) => resultList.ThenBy(current));
}
And we can use it like this:
var sorted = list.MySort(
s => s.LastName,
s => s.FirstName,
s => s.Age,
s => s.Grading,
s => s.TotalCourses);
Aggregate used to sum columns in a multi dimensional integer array
int[][] nonMagicSquare =
{
new int[] { 3, 1, 7, 8 },
new int[] { 2, 4, 16, 5 },
new int[] { 11, 6, 12, 15 },
new int[] { 9, 13, 10, 14 }
};
IEnumerable<int> rowSums = nonMagicSquare
.Select(row => row.Sum());
IEnumerable<int> colSums = nonMagicSquare
.Aggregate(
(priorSums, currentRow) =>
priorSums.Select((priorSum, index) => priorSum + currentRow[index]).ToArray()
);
Select with index is used within the Aggregate func to sum the matching columns and return a new Array; { 3 + 2 = 5, 1 + 4 = 5, 7 + 16 = 23, 8 + 5 = 13 }.
Console.WriteLine("rowSums: " + string.Join(", ", rowSums)); // rowSums: 19, 27, 44, 46
Console.WriteLine("colSums: " + string.Join(", ", colSums)); // colSums: 25, 24, 45, 42
But counting the number of trues in a Boolean array is more difficult since the accumulated type (int) differs from the source type (bool); here a seed is necessary in order to use the second overload.
bool[][] booleanTable =
{
new bool[] { true, true, true, false },
new bool[] { false, false, false, true },
new bool[] { true, false, false, true },
new bool[] { true, true, false, false }
};
IEnumerable<int> rowCounts = booleanTable
.Select(row => row.Select(value => value ? 1 : 0).Sum());
IEnumerable<int> seed = new int[booleanTable.First().Length];
IEnumerable<int> colCounts = booleanTable
.Aggregate(seed,
(priorSums, currentRow) =>
priorSums.Select((priorSum, index) => priorSum + (currentRow[index] ? 1 : 0)).ToArray()
);
Console.WriteLine("rowCounts: " + string.Join(", ", rowCounts)); // rowCounts: 3, 1, 2, 2
Console.WriteLine("colCounts: " + string.Join(", ", colCounts)); // colCounts: 3, 2, 1, 2
Everyone has given his explanation. My explanation is like that.
Aggregate method applies a function to each item of a collection. For example, let's have collection { 6, 2, 8, 3 } and the function Add (operator +) it does (((6+2)+8)+3) and returns 19
var numbers = new List<int> { 6, 2, 8, 3 };
int sum = numbers.Aggregate(func: (result, item) => result + item);
// sum: (((6+2)+8)+3) = 19
In this example there is passed named method Add instead of lambda expression.
var numbers = new List<int> { 6, 2, 8, 3 };
int sum = numbers.Aggregate(func: Add);
// sum: (((6+2)+8)+3) = 19
private static int Add(int x, int y) { return x + y; }
A short and essential definition might be this: Linq Aggregate extension method allows to declare a sort of recursive function applied on the elements of a list, the operands of whom are two: the elements in the order in which they are present into the list, one element at a time, and the result of the previous recursive iteration or nothing if not yet recursion.
In this way you can compute the factorial of numbers, or concatenate strings.