I need a quick algorithm to select 5 random elements from a generic list. For example, I'd like to get 5 random elements from a List<string>.
Using linq:
YourList.OrderBy(x => rnd.Next()).Take(5)
Iterate through and for each element make the probability of selection = (number needed)/(number left)
So if you had 40 items, the first would have a 5/40 chance of being selected. If it is, the next has a 4/39 chance, otherwise it has a 5/39 chance. By the time you get to the end you will have your 5 items, and often you'll have all of them before that.
This technique is called selection sampling, a special case of Reservoir Sampling. It's similar in performance to shuffling the input, but of course allows the sample to be generated without modifying the original data.
public static List<T> GetRandomElements<T>(this IEnumerable<T> list, int elementsCount)
{
return list.OrderBy(arg => Guid.NewGuid()).Take(elementsCount).ToList();
}
This is actually a harder problem than it sounds like, mainly because many mathematically-correct solutions will fail to actually allow you to hit all the possibilities (more on this below).
First, here are some easy-to-implement, correct-if-you-have-a-truly-random-number generator:
(0) Kyle's answer, which is O(n).
(1) Generate a list of n pairs [(0, rand), (1, rand), (2, rand), ...], sort them by the second coordinate, and use the first k (for you, k=5) indices to get your random subset. I think this is easy to implement, although it is O(n log n) time.
(2) Init an empty list s = [] that will grow to be the indices of k random elements. Choose a number r in {0, 1, 2, ..., n-1} at random, r = rand % n, and add this to s. Next take r = rand % (n-1) and stick in s; add to r the # elements less than it in s to avoid collisions. Next take r = rand % (n-2), and do the same thing, etc. until you have k distinct elements in s. This has worst-case running time O(k^2). So for k << n, this can be faster. If you keep s sorted and track which contiguous intervals it has, you can implement it in O(k log k), but it's more work.
#Kyle - you're right, on second thought I agree with your answer. I hastily read it at first, and mistakenly thought you were indicating to sequentially choose each element with fixed probability k/n, which would have been wrong - but your adaptive approach appears correct to me. Sorry about that.
Ok, and now for the kicker: asymptotically (for fixed k, n growing), there are n^k/k! choices of k element subset out of n elements [this is an approximation of (n choose k)]. If n is large, and k is not very small, then these numbers are huge. The best cycle length you can hope for in any standard 32 bit random number generator is 2^32 = 256^4. So if we have a list of 1000 elements, and we want to choose 5 at random, there's no way a standard random number generator will hit all the possibilities. However, as long as you're ok with a choice that works fine for smaller sets, and always "looks" random, then these algorithms should be ok.
Addendum: After writing this, I realized that it's tricky to implement idea (2) correctly, so I wanted to clarify this answer. To get O(k log k) time, you need an array-like structure that supports O(log m) searches and inserts - a balanced binary tree can do this. Using such a structure to build up an array called s, here is some pseudopython:
# Returns a container s with k distinct random numbers from {0, 1, ..., n-1}
def ChooseRandomSubset(n, k):
for i in range(k):
r = UniformRandom(0, n-i) # May be 0, must be < n-i
q = s.FirstIndexSuchThat( s[q] - q > r ) # This is the search.
s.InsertInOrder(q ? r + q : r + len(s)) # Inserts right before q.
return s
I suggest running through a few sample cases to see how this efficiently implements the above English explanation.
I think the selected answer is correct and pretty sweet. I implemented it differently though, as I also wanted the result in random order.
static IEnumerable<SomeType> PickSomeInRandomOrder<SomeType>(
IEnumerable<SomeType> someTypes,
int maxCount)
{
Random random = new Random(DateTime.Now.Millisecond);
Dictionary<double, SomeType> randomSortTable = new Dictionary<double,SomeType>();
foreach(SomeType someType in someTypes)
randomSortTable[random.NextDouble()] = someType;
return randomSortTable.OrderBy(KVP => KVP.Key).Take(maxCount).Select(KVP => KVP.Value);
}
I just ran into this problem, and some more google searching brought me to the problem of randomly shuffling a list: http://en.wikipedia.org/wiki/Fisher-Yates_shuffle
To completely randomly shuffle your list (in place) you do this:
To shuffle an array a of n elements (indices 0..n-1):
for i from n − 1 downto 1 do
j ← random integer with 0 ≤ j ≤ i
exchange a[j] and a[i]
If you only need the first 5 elements, then instead of running i all the way from n-1 to 1, you only need to run it to n-5 (ie: n-5)
Lets say you need k items,
This becomes:
for (i = n − 1; i >= n-k; i--)
{
j = random integer with 0 ≤ j ≤ i
exchange a[j] and a[i]
}
Each item that is selected is swapped toward the end of the array, so the k elements selected are the last k elements of the array.
This takes time O(k), where k is the number of randomly selected elements you need.
Further, if you don't want to modify your initial list, you can write down all your swaps in a temporary list, reverse that list, and apply them again, thus performing the inverse set of swaps and returning you your initial list without changing the O(k) running time.
Finally, for the real stickler, if (n == k), you should stop at 1, not n-k, as the randomly chosen integer will always be 0.
You can use this but the ordering will happen on client side
.AsEnumerable().OrderBy(n => Guid.NewGuid()).Take(5);
12 years on and the this question is still active, I didn't find an implementation of Kyle's solution I liked so here it is:
public IEnumerable<T> TakeRandom<T>(IEnumerable<T> collection, int take)
{
var random = new Random();
var available = collection.Count();
var needed = take;
foreach (var item in collection)
{
if (random.Next(available) < needed)
{
needed--;
yield return item;
if (needed == 0)
{
break;
}
}
available--;
}
}
From Dragons in the Algorithm, an interpretation in C#:
int k = 10; // items to select
var items = new List<int>(new[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 });
var selected = new List<int>();
double needed = k;
double available = items.Count;
var rand = new Random();
while (selected.Count < k) {
if( rand.NextDouble() < needed / available ) {
selected.Add(items[(int)available-1])
needed--;
}
available--;
}
This algorithm will select unique indicies of the items list.
Was thinking about comment by #JohnShedletsky on the accepted answer regarding (paraphrase):
you should be able to to this in O(subset.Length), rather than O(originalList.Length)
Basically, you should be able to generate subset random indices and then pluck them from the original list.
The Method
public static class EnumerableExtensions {
public static Random randomizer = new Random(); // you'd ideally be able to replace this with whatever makes you comfortable
public static IEnumerable<T> GetRandom<T>(this IEnumerable<T> list, int numItems) {
return (list as T[] ?? list.ToArray()).GetRandom(numItems);
// because ReSharper whined about duplicate enumeration...
/*
items.Add(list.ElementAt(randomizer.Next(list.Count()))) ) numItems--;
*/
}
// just because the parentheses were getting confusing
public static IEnumerable<T> GetRandom<T>(this T[] list, int numItems) {
var items = new HashSet<T>(); // don't want to add the same item twice; otherwise use a list
while (numItems > 0 )
// if we successfully added it, move on
if( items.Add(list[randomizer.Next(list.Length)]) ) numItems--;
return items;
}
// and because it's really fun; note -- you may get repetition
public static IEnumerable<T> PluckRandomly<T>(this IEnumerable<T> list) {
while( true )
yield return list.ElementAt(randomizer.Next(list.Count()));
}
}
If you wanted to be even more efficient, you would probably use a HashSet of the indices, not the actual list elements (in case you've got complex types or expensive comparisons);
The Unit Test
And to make sure we don't have any collisions, etc.
[TestClass]
public class RandomizingTests : UnitTestBase {
[TestMethod]
public void GetRandomFromList() {
this.testGetRandomFromList((list, num) => list.GetRandom(num));
}
[TestMethod]
public void PluckRandomly() {
this.testGetRandomFromList((list, num) => list.PluckRandomly().Take(num), requireDistinct:false);
}
private void testGetRandomFromList(Func<IEnumerable<int>, int, IEnumerable<int>> methodToGetRandomItems, int numToTake = 10, int repetitions = 100000, bool requireDistinct = true) {
var items = Enumerable.Range(0, 100);
IEnumerable<int> randomItems = null;
while( repetitions-- > 0 ) {
randomItems = methodToGetRandomItems(items, numToTake);
Assert.AreEqual(numToTake, randomItems.Count(),
"Did not get expected number of items {0}; failed at {1} repetition--", numToTake, repetitions);
if(requireDistinct) Assert.AreEqual(numToTake, randomItems.Distinct().Count(),
"Collisions (non-unique values) found, failed at {0} repetition--", repetitions);
Assert.IsTrue(randomItems.All(o => items.Contains(o)),
"Some unknown values found; failed at {0} repetition--", repetitions);
}
}
}
Selecting N random items from a group shouldn't have anything to do with order! Randomness is about unpredictability and not about shuffling positions in a group. All the answers that deal with some kinda ordering is bound to be less efficient than the ones that do not. Since efficiency is the key here, I will post something that doesn't change the order of items too much.
1) If you need true random values which means there is no restriction on which elements to choose from (ie, once chosen item can be reselected):
public static List<T> GetTrueRandom<T>(this IList<T> source, int count,
bool throwArgumentOutOfRangeException = true)
{
if (throwArgumentOutOfRangeException && count > source.Count)
throw new ArgumentOutOfRangeException();
var randoms = new List<T>(count);
randoms.AddRandomly(source, count);
return randoms;
}
If you set the exception flag off, then you can choose random items any number of times.
If you have { 1, 2, 3, 4 }, then it can give { 1, 4, 4 }, { 1, 4, 3 } etc for 3 items or even { 1, 4, 3, 2, 4 } for 5 items!
This should be pretty fast, as it has nothing to check.
2) If you need individual members from the group with no repetition, then I would rely on a dictionary (as many have pointed out already).
public static List<T> GetDistinctRandom<T>(this IList<T> source, int count)
{
if (count > source.Count)
throw new ArgumentOutOfRangeException();
if (count == source.Count)
return new List<T>(source);
var sourceDict = source.ToIndexedDictionary();
if (count > source.Count / 2)
{
while (sourceDict.Count > count)
sourceDict.Remove(source.GetRandomIndex());
return sourceDict.Select(kvp => kvp.Value).ToList();
}
var randomDict = new Dictionary<int, T>(count);
while (randomDict.Count < count)
{
int key = source.GetRandomIndex();
if (!randomDict.ContainsKey(key))
randomDict.Add(key, sourceDict[key]);
}
return randomDict.Select(kvp => kvp.Value).ToList();
}
The code is a bit lengthier than other dictionary approaches here because I'm not only adding, but also removing from list, so its kinda two loops. You can see here that I have not reordered anything at all when count becomes equal to source.Count. That's because I believe randomness should be in the returned set as a whole. I mean if you want 5 random items from 1, 2, 3, 4, 5, it shouldn't matter if its 1, 3, 4, 2, 5 or 1, 2, 3, 4, 5, but if you need 4 items from the same set, then it should unpredictably yield in 1, 2, 3, 4, 1, 3, 5, 2, 2, 3, 5, 4 etc. Secondly, when the count of random items to be returned is more than half of the original group, then its easier to remove source.Count - count items from the group than adding count items. For performance reasons I have used source instead of sourceDict to get then random index in the remove method.
So if you have { 1, 2, 3, 4 }, this can end up in { 1, 2, 3 }, { 3, 4, 1 } etc for 3 items.
3) If you need truly distinct random values from your group by taking into account the duplicates in the original group, then you may use the same approach as above, but a HashSet will be lighter than a dictionary.
public static List<T> GetTrueDistinctRandom<T>(this IList<T> source, int count,
bool throwArgumentOutOfRangeException = true)
{
if (count > source.Count)
throw new ArgumentOutOfRangeException();
var set = new HashSet<T>(source);
if (throwArgumentOutOfRangeException && count > set.Count)
throw new ArgumentOutOfRangeException();
List<T> list = hash.ToList();
if (count >= set.Count)
return list;
if (count > set.Count / 2)
{
while (set.Count > count)
set.Remove(list.GetRandom());
return set.ToList();
}
var randoms = new HashSet<T>();
randoms.AddRandomly(list, count);
return randoms.ToList();
}
The randoms variable is made a HashSet to avoid duplicates being added in the rarest of rarest cases where Random.Next can yield the same value, especially when input list is small.
So { 1, 2, 2, 4 } => 3 random items => { 1, 2, 4 } and never { 1, 2, 2}
{ 1, 2, 2, 4 } => 4 random items => exception!! or { 1, 2, 4 } depending on the flag set.
Some of the extension methods I have used:
static Random rnd = new Random();
public static int GetRandomIndex<T>(this ICollection<T> source)
{
return rnd.Next(source.Count);
}
public static T GetRandom<T>(this IList<T> source)
{
return source[source.GetRandomIndex()];
}
static void AddRandomly<T>(this ICollection<T> toCol, IList<T> fromList, int count)
{
while (toCol.Count < count)
toCol.Add(fromList.GetRandom());
}
public static Dictionary<int, T> ToIndexedDictionary<T>(this IEnumerable<T> lst)
{
return lst.ToIndexedDictionary(t => t);
}
public static Dictionary<int, T> ToIndexedDictionary<S, T>(this IEnumerable<S> lst,
Func<S, T> valueSelector)
{
int index = -1;
return lst.ToDictionary(t => ++index, valueSelector);
}
If its all about performance with tens of 1000s of items in the list having to be iterated 10000 times, then you may want to have faster random class than System.Random, but I don't think that's a big deal considering the latter most probably is never a bottleneck, its quite fast enough..
Edit: If you need to re-arrange order of returned items as well, then there's nothing that can beat dhakim's Fisher-Yates approach - short, sweet and simple..
I combined several of the above answers to create a Lazily-evaluated extension method. My testing showed that Kyle's approach (Order(N)) is many times slower than drzaus' use of a set to propose the random indices to choose (Order(K)). The former performs many more calls to the random number generator, plus iterates more times over the items.
The goals of my implementation were:
1) Do not realize the full list if given an IEnumerable that is not an IList. If I am given a sequence of a zillion items, I do not want to run out of memory. Use Kyle's approach for an on-line solution.
2) If I can tell that it is an IList, use drzaus' approach, with a twist. If K is more than half of N, I risk thrashing as I choose many random indices again and again and have to skip them. Thus I compose a list of the indices to NOT keep.
3) I guarantee that the items will be returned in the same order that they were encountered. Kyle's algorithm required no alteration. drzaus' algorithm required that I not emit items in the order that the random indices are chosen. I gather all the indices into a SortedSet, then emit items in sorted index order.
4) If K is large compared to N and I invert the sense of the set, then I enumerate all items and test if the index is not in the set. This means that
I lose the Order(K) run time, but since K is close to N in these cases, I do not lose much.
Here is the code:
/// <summary>
/// Takes k elements from the next n elements at random, preserving their order.
///
/// If there are fewer than n elements in items, this may return fewer than k elements.
/// </summary>
/// <typeparam name="TElem">Type of element in the items collection.</typeparam>
/// <param name="items">Items to be randomly selected.</param>
/// <param name="k">Number of items to pick.</param>
/// <param name="n">Total number of items to choose from.
/// If the items collection contains more than this number, the extra members will be skipped.
/// If the items collection contains fewer than this number, it is possible that fewer than k items will be returned.</param>
/// <returns>Enumerable over the retained items.
///
/// See http://stackoverflow.com/questions/48087/select-a-random-n-elements-from-listt-in-c-sharp for the commentary.
/// </returns>
public static IEnumerable<TElem> TakeRandom<TElem>(this IEnumerable<TElem> items, int k, int n)
{
var r = new FastRandom();
var itemsList = items as IList<TElem>;
if (k >= n || (itemsList != null && k >= itemsList.Count))
foreach (var item in items) yield return item;
else
{
// If we have a list, we can infer more information and choose a better algorithm.
// When using an IList, this is about 7 times faster (on one benchmark)!
if (itemsList != null && k < n/2)
{
// Since we have a List, we can use an algorithm suitable for Lists.
// If there are fewer than n elements, reduce n.
n = Math.Min(n, itemsList.Count);
// This algorithm picks K index-values randomly and directly chooses those items to be selected.
// If k is more than half of n, then we will spend a fair amount of time thrashing, picking
// indices that we have already picked and having to try again.
var invertSet = k >= n/2;
var positions = invertSet ? (ISet<int>) new HashSet<int>() : (ISet<int>) new SortedSet<int>();
var numbersNeeded = invertSet ? n - k : k;
while (numbersNeeded > 0)
if (positions.Add(r.Next(0, n))) numbersNeeded--;
if (invertSet)
{
// positions contains all the indices of elements to Skip.
for (var itemIndex = 0; itemIndex < n; itemIndex++)
{
if (!positions.Contains(itemIndex))
yield return itemsList[itemIndex];
}
}
else
{
// positions contains all the indices of elements to Take.
foreach (var itemIndex in positions)
yield return itemsList[itemIndex];
}
}
else
{
// Since we do not have a list, we will use an online algorithm.
// This permits is to skip the rest as soon as we have enough items.
var found = 0;
var scanned = 0;
foreach (var item in items)
{
var rand = r.Next(0,n-scanned);
if (rand < k - found)
{
yield return item;
found++;
}
scanned++;
if (found >= k || scanned >= n)
break;
}
}
}
}
I use a specialized random number generator, but you can just use C#'s Random if you want. (FastRandom was written by Colin Green and is part of SharpNEAT. It has a period of 2^128-1 which is better than many RNGs.)
Here are the unit tests:
[TestClass]
public class TakeRandomTests
{
/// <summary>
/// Ensure that when randomly choosing items from an array, all items are chosen with roughly equal probability.
/// </summary>
[TestMethod]
public void TakeRandom_Array_Uniformity()
{
const int numTrials = 2000000;
const int expectedCount = numTrials/20;
var timesChosen = new int[100];
var century = new int[100];
for (var i = 0; i < century.Length; i++)
century[i] = i;
for (var trial = 0; trial < numTrials; trial++)
{
foreach (var i in century.TakeRandom(5, 100))
timesChosen[i]++;
}
var avg = timesChosen.Average();
var max = timesChosen.Max();
var min = timesChosen.Min();
var allowedDifference = expectedCount/100;
AssertBetween(avg, expectedCount - 2, expectedCount + 2, "Average");
//AssertBetween(min, expectedCount - allowedDifference, expectedCount, "Min");
//AssertBetween(max, expectedCount, expectedCount + allowedDifference, "Max");
var countInRange = timesChosen.Count(i => i >= expectedCount - allowedDifference && i <= expectedCount + allowedDifference);
Assert.IsTrue(countInRange >= 90, String.Format("Not enough were in range: {0}", countInRange));
}
/// <summary>
/// Ensure that when randomly choosing items from an IEnumerable that is not an IList,
/// all items are chosen with roughly equal probability.
/// </summary>
[TestMethod]
public void TakeRandom_IEnumerable_Uniformity()
{
const int numTrials = 2000000;
const int expectedCount = numTrials / 20;
var timesChosen = new int[100];
for (var trial = 0; trial < numTrials; trial++)
{
foreach (var i in Range(0,100).TakeRandom(5, 100))
timesChosen[i]++;
}
var avg = timesChosen.Average();
var max = timesChosen.Max();
var min = timesChosen.Min();
var allowedDifference = expectedCount / 100;
var countInRange =
timesChosen.Count(i => i >= expectedCount - allowedDifference && i <= expectedCount + allowedDifference);
Assert.IsTrue(countInRange >= 90, String.Format("Not enough were in range: {0}", countInRange));
}
private IEnumerable<int> Range(int low, int count)
{
for (var i = low; i < low + count; i++)
yield return i;
}
private static void AssertBetween(int x, int low, int high, String message)
{
Assert.IsTrue(x > low, String.Format("Value {0} is less than lower limit of {1}. {2}", x, low, message));
Assert.IsTrue(x < high, String.Format("Value {0} is more than upper limit of {1}. {2}", x, high, message));
}
private static void AssertBetween(double x, double low, double high, String message)
{
Assert.IsTrue(x > low, String.Format("Value {0} is less than lower limit of {1}. {2}", x, low, message));
Assert.IsTrue(x < high, String.Format("Value {0} is more than upper limit of {1}. {2}", x, high, message));
}
}
Here you have one implementation based on Fisher-Yates Shuffle whose algorithm complexity is O(n) where n is the subset or sample size, instead of the list size, as John Shedletsky pointed out.
public static IEnumerable<T> GetRandomSample<T>(this IList<T> list, int sampleSize)
{
if (list == null) throw new ArgumentNullException("list");
if (sampleSize > list.Count) throw new ArgumentException("sampleSize may not be greater than list count", "sampleSize");
var indices = new Dictionary<int, int>(); int index;
var rnd = new Random();
for (int i = 0; i < sampleSize; i++)
{
int j = rnd.Next(i, list.Count);
if (!indices.TryGetValue(j, out index)) index = j;
yield return list[index];
if (!indices.TryGetValue(i, out index)) index = i;
indices[j] = index;
}
}
Extending from #ers's answer, if one is worried about possible different implementations of OrderBy, this should be safe:
// Instead of this
YourList.OrderBy(x => rnd.Next()).Take(5)
// Temporarily transform
YourList
.Select(v => new {v, i = rnd.Next()}) // Associate a random index to each entry
.OrderBy(x => x.i).Take(5) // Sort by (at this point fixed) random index
.Select(x => x.v); // Go back to enumerable of entry
The simple solution I use (probably not good for large lists):
Copy the list into temporary list, then in loop randomly select Item from temp list and put it in selected items list while removing it form temp list (so it can't be reselected).
Example:
List<Object> temp = OriginalList.ToList();
List<Object> selectedItems = new List<Object>();
Random rnd = new Random();
Object o;
int i = 0;
while (i < NumberOfSelectedItems)
{
o = temp[rnd.Next(temp.Count)];
selectedItems.Add(o);
temp.Remove(o);
i++;
}
This is the best I could come up with on a first cut:
public List<String> getRandomItemsFromList(int returnCount, List<String> list)
{
List<String> returnList = new List<String>();
Dictionary<int, int> randoms = new Dictionary<int, int>();
while (randoms.Count != returnCount)
{
//generate new random between one and total list count
int randomInt = new Random().Next(list.Count);
// store this in dictionary to ensure uniqueness
try
{
randoms.Add(randomInt, randomInt);
}
catch (ArgumentException aex)
{
Console.Write(aex.Message);
} //we can assume this element exists in the dictonary already
//check for randoms length and then iterate through the original list
//adding items we select via random to the return list
if (randoms.Count == returnCount)
{
foreach (int key in randoms.Keys)
returnList.Add(list[randoms[key]]);
break; //break out of _while_ loop
}
}
return returnList;
}
Using a list of randoms within a range of 1 - total list count and then simply pulling those items in the list seemed to be the best way, but using the Dictionary to ensure uniqueness is something I'm still mulling over.
Also note I used a string list, replace as needed.
Based on Kyle's answer, here's my c# implementation.
/// <summary>
/// Picks random selection of available game ID's
/// </summary>
private static List<int> GetRandomGameIDs(int count)
{
var gameIDs = (int[])HttpContext.Current.Application["NonDeletedArcadeGameIDs"];
var totalGameIDs = gameIDs.Count();
if (count > totalGameIDs) count = totalGameIDs;
var rnd = new Random();
var leftToPick = count;
var itemsLeft = totalGameIDs;
var arrPickIndex = 0;
var returnIDs = new List<int>();
while (leftToPick > 0)
{
if (rnd.Next(0, itemsLeft) < leftToPick)
{
returnIDs .Add(gameIDs[arrPickIndex]);
leftToPick--;
}
arrPickIndex++;
itemsLeft--;
}
return returnIDs ;
}
This method may be equivalent to Kyle's.
Say your list is of size n and you want k elements.
Random rand = new Random();
for(int i = 0; k>0; ++i)
{
int r = rand.Next(0, n-i);
if(r<k)
{
//include element i
k--;
}
}
Works like a charm :)
-Alex Gilbert
Here is a benchmark of three different methods:
The implementation of the accepted answer from Kyle.
An approach based on random index selection with HashSet duplication filtering, from drzaus.
A more academic approach posted by Jesús López, called Fisher–Yates shuffle.
The testing will consist of benchmarking the performance with multiple different list sizes and selection sizes.
I also included a measurement of the standard deviation of these three methods, i.e. how well distributed the random selection appears to be.
In a nutshell, drzaus's simple solution seems to be the best overall, from these three. The selected answer is great and elegant, but it's not that efficient, given that the time complexity is based on the sample size, not the selection size. Consequently, if you select a small number of items from a long list, it will take orders of magnitude more time. Of course it still performs better than the solutions based on complete reordering.
Curiously enough, this O(n) time complexity issue is true even if you only touch the list when you actually return an item, like I do in my implementation. The only thing I can thing of is that Random.Next() is pretty slow, and that performance benefits if you generate only one random number for each selected item.
And, also interestingly, the StdDev of Kyle's solution was significantly higher comparatively. I have no clue why; maybe the fault is in my implementation.
Sorry for the long code and output that will commence now; but I hope it's somewhat illuminative. Also, if you spot any issues in the tests or implementations, let me know and I'll fix it.
static void Main()
{
BenchmarkRunner.Run<Benchmarks>();
new Benchmarks() { ListSize = 100, SelectionSize = 10 }
.BenchmarkStdDev();
}
[MemoryDiagnoser]
public class Benchmarks
{
[Params(50, 500, 5000)]
public int ListSize;
[Params(5, 10, 25, 50)]
public int SelectionSize;
private Random _rnd;
private List<int> _list;
private int[] _hits;
[GlobalSetup]
public void Setup()
{
_rnd = new Random(12345);
_list = Enumerable.Range(0, ListSize).ToList();
_hits = new int[ListSize];
}
[Benchmark]
public void Test_IterateSelect()
=> Random_IterateSelect(_list, SelectionSize).ToList();
[Benchmark]
public void Test_RandomIndices()
=> Random_RandomIdices(_list, SelectionSize).ToList();
[Benchmark]
public void Test_FisherYates()
=> Random_FisherYates(_list, SelectionSize).ToList();
public void BenchmarkStdDev()
{
RunOnce(Random_IterateSelect, "IterateSelect");
RunOnce(Random_RandomIdices, "RandomIndices");
RunOnce(Random_FisherYates, "FisherYates");
void RunOnce(Func<IEnumerable<int>, int, IEnumerable<int>> method, string methodName)
{
Setup();
for (int i = 0; i < 1000000; i++)
{
var selected = method(_list, SelectionSize).ToList();
Debug.Assert(selected.Count() == SelectionSize);
foreach (var item in selected) _hits[item]++;
}
var stdDev = GetStdDev(_hits);
Console.WriteLine($"StdDev of {methodName}: {stdDev :n} (% of average: {stdDev / (_hits.Average() / 100) :n})");
}
double GetStdDev(IEnumerable<int> hits)
{
var average = hits.Average();
return Math.Sqrt(hits.Average(v => Math.Pow(v - average, 2)));
}
}
public IEnumerable<T> Random_IterateSelect<T>(IEnumerable<T> collection, int needed)
{
var count = collection.Count();
for (int i = 0; i < count; i++)
{
if (_rnd.Next(count - i) < needed)
{
yield return collection.ElementAt(i);
if (--needed == 0)
yield break;
}
}
}
public IEnumerable<T> Random_RandomIdices<T>(IEnumerable<T> list, int needed)
{
var selectedItems = new HashSet<T>();
var count = list.Count();
while (needed > 0)
if (selectedItems.Add(list.ElementAt(_rnd.Next(count))))
needed--;
return selectedItems;
}
public IEnumerable<T> Random_FisherYates<T>(IEnumerable<T> list, int sampleSize)
{
var count = list.Count();
if (sampleSize > count) throw new ArgumentException("sampleSize may not be greater than list count", "sampleSize");
var indices = new Dictionary<int, int>(); int index;
for (int i = 0; i < sampleSize; i++)
{
int j = _rnd.Next(i, count);
if (!indices.TryGetValue(j, out index)) index = j;
yield return list.ElementAt(index);
if (!indices.TryGetValue(i, out index)) index = i;
indices[j] = index;
}
}
}
Output:
| Method | ListSize | Select | Mean | Error | StdDev | Gen 0 | Allocated |
|-------------- |--------- |------- |------------:|----------:|----------:|-------:|----------:|
| IterateSelect | 50 | 5 | 711.5 ns | 5.19 ns | 4.85 ns | 0.0305 | 144 B |
| RandomIndices | 50 | 5 | 341.1 ns | 4.48 ns | 4.19 ns | 0.0644 | 304 B |
| FisherYates | 50 | 5 | 573.5 ns | 6.12 ns | 5.72 ns | 0.0944 | 447 B |
| IterateSelect | 50 | 10 | 967.2 ns | 4.64 ns | 3.87 ns | 0.0458 | 220 B |
| RandomIndices | 50 | 10 | 709.9 ns | 11.27 ns | 9.99 ns | 0.1307 | 621 B |
| FisherYates | 50 | 10 | 1,204.4 ns | 10.63 ns | 9.94 ns | 0.1850 | 875 B |
| IterateSelect | 50 | 25 | 1,358.5 ns | 7.97 ns | 6.65 ns | 0.0763 | 361 B |
| RandomIndices | 50 | 25 | 1,958.1 ns | 15.69 ns | 13.91 ns | 0.2747 | 1298 B |
| FisherYates | 50 | 25 | 2,878.9 ns | 31.42 ns | 29.39 ns | 0.3471 | 1653 B |
| IterateSelect | 50 | 50 | 1,739.1 ns | 15.86 ns | 14.06 ns | 0.1316 | 629 B |
| RandomIndices | 50 | 50 | 8,906.1 ns | 88.92 ns | 74.25 ns | 0.5951 | 2848 B |
| FisherYates | 50 | 50 | 4,899.9 ns | 38.10 ns | 33.78 ns | 0.4349 | 2063 B |
| IterateSelect | 500 | 5 | 4,775.3 ns | 46.96 ns | 41.63 ns | 0.0305 | 144 B |
| RandomIndices | 500 | 5 | 327.8 ns | 2.82 ns | 2.50 ns | 0.0644 | 304 B |
| FisherYates | 500 | 5 | 558.5 ns | 7.95 ns | 7.44 ns | 0.0944 | 449 B |
| IterateSelect | 500 | 10 | 5,387.1 ns | 44.57 ns | 41.69 ns | 0.0458 | 220 B |
| RandomIndices | 500 | 10 | 648.0 ns | 9.12 ns | 8.54 ns | 0.1307 | 621 B |
| FisherYates | 500 | 10 | 1,154.6 ns | 13.66 ns | 12.78 ns | 0.1869 | 889 B |
| IterateSelect | 500 | 25 | 6,442.3 ns | 48.90 ns | 40.83 ns | 0.0763 | 361 B |
| RandomIndices | 500 | 25 | 1,569.6 ns | 15.79 ns | 14.77 ns | 0.2747 | 1298 B |
| FisherYates | 500 | 25 | 2,726.1 ns | 25.32 ns | 22.44 ns | 0.3777 | 1795 B |
| IterateSelect | 500 | 50 | 7,775.4 ns | 35.47 ns | 31.45 ns | 0.1221 | 629 B |
| RandomIndices | 500 | 50 | 2,976.9 ns | 27.11 ns | 24.03 ns | 0.6027 | 2848 B |
| FisherYates | 500 | 50 | 5,383.2 ns | 36.49 ns | 32.35 ns | 0.8163 | 3870 B |
| IterateSelect | 5000 | 5 | 45,208.6 ns | 459.92 ns | 430.21 ns | - | 144 B |
| RandomIndices | 5000 | 5 | 328.7 ns | 5.15 ns | 4.81 ns | 0.0644 | 304 B |
| FisherYates | 5000 | 5 | 556.1 ns | 10.75 ns | 10.05 ns | 0.0944 | 449 B |
| IterateSelect | 5000 | 10 | 49,253.9 ns | 420.26 ns | 393.11 ns | - | 220 B |
| RandomIndices | 5000 | 10 | 642.9 ns | 4.95 ns | 4.13 ns | 0.1307 | 621 B |
| FisherYates | 5000 | 10 | 1,141.9 ns | 12.81 ns | 11.98 ns | 0.1869 | 889 B |
| IterateSelect | 5000 | 25 | 54,044.4 ns | 208.92 ns | 174.46 ns | 0.0610 | 361 B |
| RandomIndices | 5000 | 25 | 1,480.5 ns | 11.56 ns | 10.81 ns | 0.2747 | 1298 B |
| FisherYates | 5000 | 25 | 2,713.9 ns | 27.31 ns | 24.21 ns | 0.3777 | 1795 B |
| IterateSelect | 5000 | 50 | 54,418.2 ns | 329.62 ns | 308.32 ns | 0.1221 | 629 B |
| RandomIndices | 5000 | 50 | 2,886.4 ns | 36.53 ns | 34.17 ns | 0.6027 | 2848 B |
| FisherYates | 5000 | 50 | 5,347.2 ns | 59.45 ns | 55.61 ns | 0.8163 | 3870 B |
StdDev of IterateSelect: 671.88 (% of average: 0.67)
StdDev of RandomIndices: 296.07 (% of average: 0.30)
StdDev of FisherYates: 280.47 (% of average: 0.28)
It is a lot harder than one would think. See the great Article "Shuffling" from Jeff.
I did write a very short article on that subject including C# code:
Return random subset of N elements of a given array
Goal: Select N number of items from collection source without duplication.
I created an extension for any generic collection. Here's how I did it:
public static class CollectionExtension
{
public static IList<TSource> RandomizeCollection<TSource>(this IList<TSource> source, int maxItems)
{
int randomCount = source.Count > maxItems ? maxItems : source.Count;
int?[] randomizedIndices = new int?[randomCount];
Random random = new Random();
for (int i = 0; i < randomizedIndices.Length; i++)
{
int randomResult = -1;
while (randomizedIndices.Contains((randomResult = random.Next(0, source.Count))))
{
//0 -> since all list starts from index 0; source.Count -> maximum number of items that can be randomize
//continue looping while the generated random number is already in the list of randomizedIndices
}
randomizedIndices[i] = randomResult;
}
IList<TSource> result = new List<TSource>();
foreach (int index in randomizedIndices)
result.Add(source.ElementAt(index));
return result;
}
}
Short and simple. Hope this helps someone!
if (list.Count > maxListCount)
{
var rndList = new List<YourEntity>();
var r = new Random();
while (rndList.Count < maxListCount)
{
var addingElement = list[r.Next(list.Count)];
//element uniqueness checking
//choose your case
//if (rndList.Contains(addingElement))
//if (rndList.Any(p => p.Id == addingElement.Id))
continue;
rndList.Add(addingElement);
}
return rndList;
}
public static IEnumerable<TItem> RandomSample<TItem>(this IReadOnlyList<TItem> items, int count)
{
if (count < 1 || count > items.Count)
{
throw new ArgumentOutOfRangeException(nameof(count));
}
List<int> indexes = Enumerable.Range(0, items.Count).ToList();
int yieldedCount = 0;
while (yieldedCount < count)
{
int i = RandomNumberGenerator.GetInt32(indexes.Count);
int randomIndex = indexes[i];
yield return items[randomIndex];
// indexes.RemoveAt(i); // Avoid removing items from the middle of the list
indexes[i] = indexes[indexes.Count - 1]; // Replace yielded index with the last one
indexes.RemoveAt(indexes.Count - 1);
yieldedCount++;
}
}
public static IEnumerable<T> GetRandom<T>(IList<T> list, int count, Random random)
{
// Probably you should throw exception if count > list.Count
count = Math.Min(list.Count, count);
var selectedIndices = new SortedSet<int>();
// Random upper bound (exclusive)
int randomMax = list.Count;
while (selectedIndices.Count < count)
{
int randomIndex = random.Next(0, randomMax);
// skip over already selected indices
foreach (var selectedIndex in selectedIndices)
if (selectedIndex <= randomIndex)
++randomIndex;
else
break;
yield return list[randomIndex];
selectedIndices.Add(randomIndex);
--randomMax;
}
}
Memory: ~count
Complexity: O(count2)
I recently did this on my project using an idea similar to Tyler's point 1.
I was loading a bunch of questions and selecting five at random. Sorting was achieved using an IComparer.
aAll questions were loaded in the a QuestionSorter list, which was then sorted using the List's Sort function and the first k elements where selected.
private class QuestionSorter : IComparable<QuestionSorter>
{
public double SortingKey
{
get;
set;
}
public Question QuestionObject
{
get;
set;
}
public QuestionSorter(Question q)
{
this.SortingKey = RandomNumberGenerator.RandomDouble;
this.QuestionObject = q;
}
public int CompareTo(QuestionSorter other)
{
if (this.SortingKey < other.SortingKey)
{
return -1;
}
else if (this.SortingKey > other.SortingKey)
{
return 1;
}
else
{
return 0;
}
}
}
Usage:
List<QuestionSorter> unsortedQuestions = new List<QuestionSorter>();
// add the questions here
unsortedQuestions.Sort(unsortedQuestions as IComparer<QuestionSorter>);
// select the first k elements
why not something like this:
Dim ar As New ArrayList
Dim numToGet As Integer = 5
'hard code just to test
ar.Add("12")
ar.Add("11")
ar.Add("10")
ar.Add("15")
ar.Add("16")
ar.Add("17")
Dim randomListOfProductIds As New ArrayList
Dim toAdd As String = ""
For i = 0 To numToGet - 1
toAdd = ar(CInt((ar.Count - 1) * Rnd()))
randomListOfProductIds.Add(toAdd)
'remove from id list
ar.Remove(toAdd)
Next
'sorry i'm lazy and have to write vb at work :( and didn't feel like converting to c#
Here's my approach (full text here http://krkadev.blogspot.com/2010/08/random-numbers-without-repetition.html ).
It should run in O(K) instead of O(N), where K is the number of wanted elements and N is the size of the list to choose from:
public <T> List<T> take(List<T> source, int k) {
int n = source.size();
if (k > n) {
throw new IllegalStateException(
"Can not take " + k +
" elements from a list with " + n +
" elements");
}
List<T> result = new ArrayList<T>(k);
Map<Integer,Integer> used = new HashMap<Integer,Integer>();
int metric = 0;
for (int i = 0; i < k; i++) {
int off = random.nextInt(n - i);
while (true) {
metric++;
Integer redirect = used.put(off, n - i - 1);
if (redirect == null) {
break;
}
off = redirect;
}
result.add(source.get(off));
}
assert metric <= 2*k;
return result;
}
This isn't as elegant or efficient as the accepted solution, but it's quick to write up. First, permute the array randomly, then select the first K elements. In python,
import numpy
N = 20
K = 5
idx = np.arange(N)
numpy.random.shuffle(idx)
print idx[:K]
I would use an extension method.
public static IEnumerable<T> TakeRandom<T>(this IEnumerable<T> elements, int countToTake)
{
var random = new Random();
var internalList = elements.ToList();
var selected = new List<T>();
for (var i = 0; i < countToTake; ++i)
{
var next = random.Next(0, internalList.Count - selected.Count);
selected.Add(internalList[next]);
internalList[next] = internalList[internalList.Count - selected.Count];
}
return selected;
}
Using LINQ with large lists (when costly to touch each element) AND if you can live with the possibility of duplicates:
new int[5].Select(o => (int)(rnd.NextDouble() * maxIndex)).Select(i => YourIEnum.ElementAt(i))
For my use i had a list of 100.000 elements, and because of them being pulled from a DB I about halfed (or better) the time compared to a rnd on the whole list.
Having a large list will reduce the odds greatly for duplicates.
Related
I am trying to write a function that the returns one of the following multipliers randomly selected but following the frequency requirement. What below table defines is that for 1 Million calls to this function the 1500 will be returned once, 500 twice and so on.
|---------------------|------------------------------|
| Multiplier | Frequency Per Million |
|---------------------|------------------------------|
| 1500 | 1 |
|---------------------|------------------------------|
| 500 | 2 |
|---------------------|------------------------------|
| 200 | 50 |
|---------------------|------------------------------|
| 50 | 100 |
|---------------------|------------------------------|
| 25 | 20,000 |
|---------------------|------------------------------|
| 5 | 75,000 |
|---------------------|------------------------------|
| 3 | 414,326 |
|---------------------|------------------------------|
| 2 | 490521 |
|---------------------|------------------------------|
Wondering what would be the best way to approach implementing this.
First, let's declare the model:
static Dictionary<int, int> multipliers = new Dictionary<int, int>() {
{1500, 1},
{ 500, 2},
{ 200, 50},
{ 50, 100},
{ 25, 20_000},
{ 5, 75_000},
{ 3, 414_326},
{ 2, 490_521},
};
Then you can easily choose random multiplier:
// Easiest, but not thread safe
static Random random = new Random();
...
private static int RandomMultiplier() {
int r = random.Next(multipliers.Values.Sum());
s = 0;
foreach (var pair in multipliers.OrderByDescending(p => p.Key))
if (r < (s += pair.Value))
return pair.Key;
throw new InvalidOperationException($"{r} must not reach this line");
}
...
int multiplier = RandomMultiplier();
If the frequency and value that needs to be returned are set, then nothing complicated is needed. You just need to adjust for the previous numbers being handled in the if blocks by adding the frequency of the previous numbers.
private int GetRandomMultiplier()
{
var random = new Random();
var next = random.Next(1000000);
if (next < 1)
{
return 1500;
}
else if (next < 3)
{
return 500;
}
else if (next < 53)
{
return 200;
}
else if (next < 153)
{
return 50;
}
else if (next < 20153)
{
return 25;
}
else if (next < 95153)
{
return 5;
}
else if (next < 509479)
{
return 3;
}
return 2;
}
You don't want to create a new Random every time though, so create one once and use that.
I have a list like
| column1 | column2 |
| 1 | 72 |
| 2 | 30 |
| 3 | 27 |
| 3 | 38 |
| 4 | 72 |
As you can see, the list is already sorted on column1, my goal here is to perform an OrderByDescending on the second column only on equals columns 1. Basically, I want
| column1 | column2 |
| 1 | 72 |
| 2 | 30 |
| 3 | 38 |
| 3 | 27 |
| 4 | 72 |
I can't re-run the first OrderBy (that'll be hard to explain, we don't care, I just cant :D) so forget about
list.OrderBy(e => e.column1).ThenByDescending(e => e.column2)
In fact, I wouldn't have any problem if I could simply do a .ThenByDescending(e => e.column2) without having to do the .OrderBy (maybe I can run an "empty" OrderBy that won't change the sort ? Then I would be able to do the ThenByDescending ?)
list.OrderBy(e => e.column1).ThenByDescending(e => e.column2) is still the way to go.
The algorithm has to know it has to sort on e.column1 first, no matter if it actually changes something or not. It has to know it only has to sort e.column2 within the subset of the first sorting statement. You can't do that with 'just' sorting on column2.
list.GroupBy(i => i.column1).SelectMany(i => i.OrderByDescending(g => g.column2))
Will work with many providers, but some may not preserve the ordering in the GroupBy. In such a case:
list.AsEnumerable().GroupBy(i => i.column1).SelectMany(i => i.OrderByDescending(g => g.column2))
Will work by forcing the operation into memory (where the ordering is preserved by GroupBy), though with the disadvantage of all subsequent operations being done in-memory rather than on a DB etc.
If you really DO have a list rather then an IEnumerable, it is actually possible to do this using the overload of List.Sort() which lets you specify a subset of items to sort, along with a comparer.
What you have to do is an O(N) traversal of the list to determine where each subgroup occurs according to the already-sorted column. Then sort each subgroup of more than one item according to the secondary sort column.
There's a small amount of fiddlyness involved with selecting the keys with which to identify the subgroups and the keys with which to sort the subgroups.
Here's the implementation:
public static void SortSubgroupsBy<T>
(
List<T> items,
Func<T, T, bool> sortedColumnComparer, // Used to compare the already-sorted column.
Func<T, T, int> unsortedColumnComparer // Used to compare the unsorted column.
)
{
var unsortedComparer = Comparer<T>.Create(
(x, y) => unsortedColumnComparer(x, y));
for (int i = 0; i < items.Count; ++i)
{
int j = i + 1;
while (j < items.Count && sortedColumnComparer(items[i], items[j]))
++j;
if ((j - i) > 1)
items.Sort(i, j-i, unsortedComparer);
}
}
Here's a complete demonstration in a Console app:
using System;
using System.Collections.Generic;
namespace ConsoleApplication1
{
class Item
{
public Item(int column1, int column2)
{
Column1 = column1;
Column2 = column2;
}
public int Column1;
public int Column2;
public override string ToString()
{
return $"[{Column1}, {Column2}]";
}
}
class Program
{
static void Main()
{
List<Item> items = new List<Item>
{
new Item(1, 72),
new Item(2, 29),
new Item(2, 30),
new Item(3, 27),
new Item(3, 38),
new Item(3, 53),
new Item(4, 72),
new Item(4, 21),
new Item(4, 86),
new Item(4, 17),
new Item(5, 90)
};
SortSubgroupsBy(
items,
(x, y) => x.Column1 == y.Column1, // Compare sorted column.
(x, y) => y.Column2 - x.Column2); // Compare unsorted column.
Console.WriteLine(string.Join("\n", items));
}
public static void SortSubgroupsBy<T>
(
List<T> items,
Func<T, T, bool> sortedColumnComparer, // Used to compare the already-sorted column.
Func<T, T, int> unsortedColumnComparer // Used to compare the unsorted column.
)
{
var unsortedComparer = Comparer<T>.Create(
(x, y) => unsortedColumnComparer(x, y));
for (int i = 0; i < items.Count; ++i)
{
int j = i + 1;
while (j < items.Count && sortedColumnComparer(items[i], items[j]))
++j;
if ((j - i) > 1)
items.Sort(i, j-i, unsortedComparer);
}
}
}
}
I have a list a simple Player object, as follows
Name | Team | Position | Total
Tom Brady | Team 1 | QB | 200
Adrian Peterson | Team 1 | RB | 250
Calvin Johnson | Team 2 | WR | 260
LeVon Bell | Team 2 | RB | 220
Peyton Manning | Team 3 | QB | 220
Arian Foster | Team 3 | RB | 220
This is a simple sample, in reality there are about 200 records. What I want to do is to get all possible combinations of players per team, and sum their total, so the end product would be as follows
Possibilities
Teams | Players | Total
Team 1 | Tom Brady, Adrian Peterson | 450
Team 2 | Calvin Johnson, LeVon Bell | 480
Team 3 | Peyton Manning, Arian Foster | 440
Basically I am looking for trade possibilities, so I need to get combinations of players per team. The largest possible combination I am looking for is 5 players per team, where I would have the Players and their points combined in a new object. Right now I can get there with below.
var playerList = players.GroupBy(p => p.Team)
.Select(g => new
{
Team = g.Key,
g
}).ToList();
List<Possibilities> possibilities = new List<Possibilities>();
foreach (var a in playerList)
{
List<Possibilities> onePlayer = (from b in a.g
select new Possibilities
{
Players = b.Name,
Total = b.Total,
Team = a.Team
}).ToList();
List<Possibilities> twoPlayer = (from b in a.g
from c in a.g
select new Possibilities
{
Players = b.Name + ", " + c.Name,
Total = b.Total + c.Total,
Team = a.Team
}).ToList();
And this gives me all combinations of 1,2,3 players per team, but I want to add 4 and 5. This also does not remove duplicate combinations (Player 1, Player 2 and Player 2,Player1). Is there a cleaner way to do this?
You can generate all combinations of a limited set of items (where the number of items is <= 31) by counting using a binary number. Each set bit in the binary number represents an item being present in the combination.
For example, counting (in binary) for a set of 3 items:
000, 001, 010, 011, 100, 101, 110, 111
A 1 in the binary number indicates that the corresponding item in the set should be included in that combination.
If you want to ensure that the combination includes number of items in a certain range, you need to count the bits set in the binary number and check if that count is in range. There's an efficient way to do that using bit twiddling (see here for examples).
Putting that together gives code like the following. Run it and check the output. Hopefully you can see how to use it with your program.
This example produces all combinations of between 2 and 3 items taken from A, B, C, D. Its output is:
A,B
A,C
B,C
A,B,C
A,D
B,D
A,B,D
C,D
A,C,D
B,C,D
The code is:
using System;
using System.Collections.Generic;
namespace ConsoleApplication1
{
public class Program
{
public static void Main()
{
var data = new [] {"A", "B", "C", "D"};
// Get all the combinations of elements from A,B,C,D with between 2 and 3 values:
var combinations = Combinations(data, 2, 3);
// Combinations() has returned an IEnumerable<IEnumerable<T>>,
// that is, a sequence of subsequences where each subsequence is one combination.
foreach (var combination in combinations)
Console.WriteLine(string.Join(",", combination));
}
public static IEnumerable<IEnumerable<T>> Combinations<T>(T[] input, int minElements, int maxElements)
{
int numCombinations = 2 << (input.Length - 1);
for (int bits = 0; bits < numCombinations; ++bits)
{
int bitCount = NumBitsSet(bits);
if (minElements <= bitCount && bitCount <= maxElements)
yield return combination(input, bits);
}
}
private static IEnumerable<T> combination<T>(T[] input, int bits)
{
for (int bit = 1, i = 0; i < input.Length; ++i, bit <<= 1)
if ((bits & bit) != 0)
yield return input[i];
}
public static int NumBitsSet(int i)
{
i = i - ((i >> 1) & 0x55555555);
i = (i & 0x33333333) + ((i >> 2) & 0x33333333);
return (((i + (i >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24;
}
}
}
Although probably very inefficient, the below might work. Of course that's just a simple example and you'll need to adjust the code to your situation.
static void Main(string[] args)
{
var nums = new[] { 1, 2, 3, 4, 5, 6 };
var combinations = new List<int[]>();
int[] current;
foreach (int i in nums)
{
combinations.Add(new[] { i });
foreach (int j in nums.Where(n => n != i))
{
current = new[] { i, j };
if (!combinations.Any(c => current.Length == c.Length && current.All(n => c.Contains(n))))
{
combinations.Add(current);
}
foreach (int k in nums.Where(n => n != i && n != j))
{
current = new[] { i, j, k };
if (!combinations.Any(c => current.Length == c.Length && current.All(n => c.Contains(n))))
{
combinations.Add(current);
}
foreach (int l in nums.Where(n => n != i && n != j && n != k))
{
current = new[] { i, j, k, l };
if (!combinations.Any(c => current.Length == c.Length && current.All(n => c.Contains(n))))
{
combinations.Add(current);
}
foreach (int m in nums.Where(n => n != i && n != j && n != k && n != l))
{
current = new[] { i, j, k, l, m };
if (!combinations.Any(c => current.Length == c.Length && current.All(n => c.Contains(n))))
{
combinations.Add(current);
}
}
}
}
}
}
foreach (var c in combinations)
{
foreach (var num in c)
{
Console.Write(num + " ");
}
Console.WriteLine();
}
Console.ReadKey();
}
I would like to split a list in parts, without knowing how much items I will have in that list. The question is different from those of you who wants to split a list into chunk of fixed size.
int[] a = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 };
I would like the values to be splitted vertically.
Splitted in 2 :
-------------------
| item 1 | item 6 |
| item 2 | item 7 |
| item 3 | item 8 |
| item 4 | item 9 |
| item 5 | |
Splitted in 3:
| item 1 | item 4 | item 7 |
| item 2 | item 5 | item 8 |
| item 3 | item 6 | item 9 |
Splitted in 4:
| item 1 | item 4 | item 6 | item 8 |
| item 2 | item 5 | item 7 | item 9 |
| item 3 | | | |
I've found a few c# extensions that can do that but it doesn't distribute the value the way I want. Here's what I've found:
// this technic is an horizontal distribution
public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> list, int parts)
{
int i = 0;
var splits = from item in list
group item by i++ % parts into part
select part.AsEnumerable();
return splits;
}
the result is this but my problem is that the value are distributed horizontally:
| item 1 | item 2 |
| item 3 | item 4 |
| item 5 | item 6 |
| item 7 | item 8 |
| item 9 | |
or
| item 1 | item 2 | item 3 |
| item 4 | item 5 | item 6 |
| item 7 | item 8 | item 9 |
any idea how I can distribute my values vertically and have the possibility to choose the number of parts that i want?
In real life
For those of you who want to know in which situation I would like to split a list vertically, here's a screenshot of a section of my website:
With .Take() and .Skip() you can:
int[] a = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 };
int splitIndex = 4; // or (a.Length / 2) to split in the middle.
var list1 = a.Take(splitIndex).ToArray(); // Returns a specified number of contiguous elements from the start of a sequence.
var list2 = a.Skip(splitIndex).ToArray(); // Bypasses a specified number of elements in a sequence and then returns the remaining elements.
You can use .ToList() instead of .ToArray() if you want a List<int>.
EDIT:
After you changed (clarified maybe) your question a bit, I guess this is what you needed:
public static class Extensions
{
public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> source, int parts)
{
var list = new List<T>(source);
int defaultSize = (int)((double)list.Count / (double)parts);
int offset = list.Count % parts;
int position = 0;
for (int i = 0; i < parts; i++)
{
int size = defaultSize;
if (i < offset)
size++; // Just add one to the size (it's enough).
yield return list.GetRange(position, size);
// Set the new position after creating a part list, so that it always start with position zero on the first yield return above.
position += size;
}
}
}
Using it:
int[] a = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9 };
var lists = a.Split(2);
This would generate:
split in 2 : a.Split(2);
| item 1 | item 6 |
| item 2 | item 7 |
| item 3 | item 8 |
| item 4 | item 9 |
| item 5 | |
split in 3 : a.Split(3);
| item 1 | item 4 | item 7 |
| item 2 | item 5 | item 8 |
| item 3 | item 6 | item 9 |
split in 4 : a.Split(4);
| item 1 | item 4 | item 6 | item 8 |
| item 2 | item 5 | item 7 | item 9 |
| item 3 | | | |
Also, if you would have:
int[] b = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }; // 10 items
and split in 4 : b.Split(4);
| item 1 | item 4 | item 7 | item 9 |
| item 2 | item 5 | item 8 | item 10|
| item 3 | item 6 | | |
This seems to do the trick quite nicely. It can probably be done even more efficient, but this was puzzling enough... It's much easier to do:
1|4|7|10
2|5|8
3|6|9
Than:
1|4|7|9
2|5|8|10
3|6|
I ignored the LINQ request first, since I had trouble wrapping my head around it. The solution using normal array manipulation could result in something like this:
public static IEnumerable<IEnumerable<TListItem>> Split<TListItem>(this IEnumerable<TListItem> items, int parts)
where TListItem : struct
{
var itemsArray = items.ToArray();
int itemCount = itemsArray.Length;
int itemsOnlastRow = itemCount - ((itemCount / parts) * parts);
int numberOfRows = (int)(itemCount / (decimal)parts) + 1;
for (int row = 0; row < numberOfRows; row++)
{
yield return SplitToRow(itemsArray, parts, itemsOnlastRow, numberOfRows, row);
}
}
private static IEnumerable<TListItem> SplitToRow<TListItem>(TListItem[] items, int itemsOnFirstRows, int itemsOnlastRow,
int numberOfRows, int row)
{
for (int column = 0; column < itemsOnFirstRows; column++)
{
// Are we on the last row?
if (row == numberOfRows - 1)
{
// Are we within the number of items on that row?
if (column < itemsOnlastRow)
{
yield return items[(column + 1) * numberOfRows -1];
}
}
else
{
int firstblock = itemsOnlastRow * numberOfRows;
int index;
// are we in the first block?
if (column < itemsOnlastRow)
{
index = column*numberOfRows + ((row + 1)%numberOfRows) - 1;
}
else
{
index = firstblock + (column - itemsOnlastRow)*(numberOfRows - 1) + ((row + 1)%numberOfRows) - 1;
}
yield return
items[index];
}
}
}
The LINQ pseudo code would be:
//WARNING: DOES NOT WORK
public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> list, int parts)
{
int itemOnIndex = 0;
var splits = from item in list
group item by MethodToDefineRow(itemOnIndex++) into row
select row.AsEnumerable();
return splits;
}
But without knowing the number of items, there is no way to calculate the place where to put it.
So by doing a little pre-calculation, you can use LINQ to achieve the same thing as above this requires going through the IEnumerable twice, there doesn't seem to be a way around that. The trick is to calculate the row each value will be assigned to.
//WARNING: Iterates the IEnumerable twice
public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> list, int parts)
{
int itemOnIndex = 0;
int itemCount = list.Count();
int itemsOnlastRow = itemCount - ((itemCount / parts) * parts);
int numberOfRows = (int)(itemCount / (decimal)parts) + 1;
int firstblock = (numberOfRows*itemsOnlastRow);
var splits = from item in list
group item by (itemOnIndex++ < firstblock) ? ((itemOnIndex -1) % numberOfRows) : ((itemOnIndex - 1 - firstblock) % (numberOfRows - 1)) into row
orderby row.Key
select row.AsEnumerable();
return splits;
}
Use .Take(#OfElements) to specify number of elements you want to take.
You can also use .First and .Last
I need help to understand how a function is working;: it is a recursive function with yield return but I can't figure out how it works. It is used calculate a cumulative density function (approximate) over a set of data.
Thanks a lot to everyone.
/// Approximates the cumulative density through a recursive procedure
/// estimating counts of regions at different resolutions.
/// </summary>
/// <param name="data">Source collection of integer values</param>
/// <param name="maximum">The largest integer in the resulting cdf (it has to be a power of 2...</param>
/// <returns>A list of counts, where entry i is the number of records less than i</returns>
public static IEnumerable<int> FUNCT(IEnumerable<int> data, int max)
{
if (max == 1)
{
yield return data.Count();
}
else
{
var t = data.Where(x => x < max / 2);
var f = data.Where(x => x > max / 2);
foreach (var value in FUNCT(t, max / 2))
yield return value;
var count = t.Count();
f = f.Select(x => x - max / 2);
foreach (var value in FUNCT(f, max / 2))
yield return value + count;
}
}
In essence, IEnumerable functions that use yield return function slightly differently from traditional recursive functions. As a base case, suppose you have:
IEnumerable<int> F(int n)
{
if (n == 1)
{
yield return 1;
yield return 2;
// implied yield return break;
}
// Enter loop 1
foreach (var v in F(n - 1))
yield return v;
// End loop 1
int sum = 5;
// Enter loop 2
foreach (var v in F(n - 1))
yield return v + sum;
// End loop 2
// implied yield return break;
}
void Main()
{
foreach (var v in F(2))
Console.Write(v);
// implied return
}
F takes the basic orm of the original FUNCT. If we call F(2), then walking through the yields:
F(2)
| F(1)
| | yield return 1
| yield return 1
Console.Write(1);
| | yield return 2
| yield return 2
Console.Write(2)
| | RETURNS
| sum = 5;
| F(1)
| | yield return 1
| yield return 1 + 5
Console.Write(6)
| | yield return 2
| yield return 2 + 5
Console.Write(7)
| | RETURNS
| RETURNS
RETURNS
And 1267 is printed. Note that the yield return statement yields control to the caller, but that the next iteration causes the function to continue where it had previously yielded.
The CDF method does adds some additional complexity, but not much. The recursion splits the collection into two pieces, and computes the CDF of each piece, until max=1. Then the function counts the number of elements and yields it, with each yield propogating recursively to the enclosing loop.
To walk through FUNCT, suppose you run with data=[0,1,0,1,2,3,2,1] and max=4. Then running through the method, using the same Main function above as a driver, yields:
FUNCT([0,1,0,1,2,3,2,1], 4)
| max/2 = 2
| t = [0,1,0,1,1]
| f = [3] // (note: per my comment to the original question,
| // should be [2,3,2] to get true CDF. The 2s are
| // ignored since the method uses > max/2 rather than
| // >= max/2.)
| FUNCT(t,max/2) = FUNCT([0,1,0,1,1], 2)
| | max/2 = 1
| | t = [0,0]
| | f = [] // or [1,1,1]
| | FUNCT(t, max/2) = FUNCT([0,0], 1)
| | | max = 1
| | | yield return data.count = [0,0].count = 2
| | yield return 2
| yield return 2
Console.Write(2)
| | | RETURNS
| | count = t.count = 2
| | F(f, max/2) = FUNCT([], 1)
| | | max = 1
| | | yield return data.count = [].count = 0
| | yield return 0 + count = 2
| yield return 2
Console.Write(2)
| | | RETURNS
| | RETURNS
| count = t.Count() = 5
| f = f - max/2 = f - 2 = [1]
| FUNCT(f, max/2) = FUNCT([1], 2)
| | max = 2
| | max/2 = 1
| | t = []
| | f = [] // or [1]
| | FUNCT(t, max/2) = funct([], 1)
| | | max = 1
| | | yield return data.count = [].count = 0
| | yield return 0
| yield return 0 + count = 5
Console.Write(5)
| | | RETURNS
| | count = t.count = [].count = 0
| | f = f - max/2 = []
| | F(f, max/2) = funct([], 1)
| | | max = 1
| | | yield return data.count = [].count = 0
| | yield return 0 + count = 0 + 0 = 0
| yield return 0 + count = 0 + 5 = 5
Console.Write(5)
| | RETURNS
| RETURNS
RETURNS
So this returns the values (2,2,5,5). (using >= would yield the values (2,5,7,8) -- note that these are the exact values of a scaled CDF for non-negative integral data, rather than an approximation).
Interesting question. Assuming you understand how yield works, the comments on the function (in your question) are very helpful. I've commented the code as I understand it which might help:
public static IEnumerable<int> FUNCT(IEnumerable<int> data, int max)
{
if (max == 1)
{
// Effectively the end of the recursion.
yield return data.Count();
}
else
{
// Split the data into two sets
var t = data.Where(x => x < max / 2);
var f = data.Where(x => x > max / 2);
// In the set of smaller numbers, recurse to split it again
foreach (var value in FUNCT(t, max / 2))
yield return value;
// For the set of smaller numbers, get the count.
var count = t.Count();
// Shift the larger numbers so they are in the smaller half.
// This allows the recursive function to reach an end.
f = f.Select(x => x - max / 2);
// Recurse but add the count of smaller numbers. We already know there
// are at least 'count' values which are less than max / 2.
// Recurse to find out how many more there are.
foreach (var value in FUNCT(f, max / 2))
yield return value + count;
}
}