Say I have a type T:
class T {
public int identifier; //Arbitrary but unique for each character (Guids in real-life)
public char character; //In real life not a char, but I chose char here for easy demo purposes
}
And I have a predefined ordered sequence of identifiers:
int[] identifierSequence = new int[]{
9, 3, 4, 4, 7
};
I now need to order an IObservable<T> which produces the following sequence of objects:
{identifier: 3, character 'e'},
{identifier: 9, character 'h'},
{identifier: 4, character 'l'},
{identifier: 4, character 'l'},
{identifier: 7, character 'o'}
So that the resulting IObservable produces hello.
I don't want to use ToArray, as I want to receive objects as soon as they arrive and not wait until everything is observed.
More specifically, I would like to receive them like this:
Input: e h l l o
Output: he l l o
What would be the proper reactive way to do this?
The best I could come up with is this:
Dictionary<int, T> buffer = new Dictionary<int, T>();
int curIndex = 0;
inputObserable.SelectMany(item =>
{
buffer[item.identifier] = item;
IEnumerable<ReportTemplate> GetReadyElements()
{
while (true)
{
int nextItemIdentifier = identifierSequence[curIndex];
T nextItem;
if (buffer.TryGetValue(nextItemIdentifier, out nextItem))
{
buffer.Remove(nextItem.identifier);
curIndex++;
yield return nextItem;
}
else
{
break;
}
}
}
return GetReadyElements();
});
EDIT:
Schlomo raised some very valid issues with my code, which is why I marked his answer as correct. I made some modifications to his to code for it to be usable:
Generic identifier and object type
Iteration instead of recursion to prevent potential stackoverflow on very large observables
Convert the anonymous type to a real class for readability
Wherever possible, lookup a value in a dictionary only once and store as variable instead of looking it up multiple times
Fixed type
This gives me the following code:
public static IObservable<T> OrderByIdentifierSequence<T, TId>(this IObservable<T> source, IList<TId> identifierSequence, Func<T, TId> identifierFunc)
{
var initialState = new OrderByIdentifierSequenceState<T, TId>(0, ImmutableDictionary<TId, ImmutableList<T>>.Empty, Enumerable.Empty<T>());
return source.Scan(initialState, (oldState, item) =>
{
//Function to be called upon receiving new item
//If we can pattern match the first item, then it is moved into Output, and concatted continuously with the next possible item
//Otherwise, if nothing is available yet, just return the input state
OrderByIdentifierSequenceState<T, TId> GetOutput(OrderByIdentifierSequenceState<T, TId> state)
{
int index = state.Index;
ImmutableDictionary<TId, ImmutableList<T>> buffer = state.Buffer;
IList<T> output = new List<T>();
while (index < identifierSequence.Count)
{
TId key = identifierSequence[index];
ImmutableList<T> nextValues;
if (!buffer.TryGetValue(key, out nextValues) || nextValues.IsEmpty)
{
//No values available yet
break;
}
T toOutput = nextValues[nextValues.Count - 1];
output.Add(toOutput);
buffer = buffer.SetItem(key, nextValues.RemoveAt(nextValues.Count - 1));
index++;
}
return new OrderByIdentifierSequenceState<T, TId>(index, buffer, output);
}
//Before calling the recursive function, add the new item to the buffer
TId itemIdentifier = identifierFunc(item);
ImmutableList<T> valuesList;
if (!oldState.Buffer.TryGetValue(itemIdentifier, out valuesList))
{
valuesList = ImmutableList<T>.Empty;
}
var remodifiedBuffer = oldState.Buffer.SetItem(itemIdentifier, valuesList.Add(item));
return GetOutput(new OrderByIdentifierSequenceState<T, TId>(oldState.Index, remodifiedBuffer, Enumerable.Empty<T>()));
})
// Use Dematerialize/Notifications to detect and emit end of stream.
.SelectMany(output =>
{
var notifications = output.Output
.Select(item => Notification.CreateOnNext(item))
.ToList();
if (output.Index == identifierSequence.Count)
{
notifications.Add(Notification.CreateOnCompleted<T>());
}
return notifications;
})
.Dematerialize();
}
class OrderByIdentifierSequenceState<T, TId>
{
//Index shows what T we're waiting on
public int Index { get; }
//Buffer holds T that have arrived that we aren't ready yet for
public ImmutableDictionary<TId, ImmutableList<T>> Buffer { get; }
//Output holds T that can be safely emitted.
public IEnumerable<T> Output { get; }
public OrderByIdentifierSequenceState(int index, ImmutableDictionary<TId, ImmutableList<T>> buffer, IEnumerable<T> output)
{
this.Index = index;
this.Buffer = buffer;
this.Output = output;
}
}
However, this code still has a couple of problems:
Constant copying of the state (mainly the ImmutableDictionary), which can be very expensive. Possible solution: maintain a separate state per observer, instead of per item received.
When one or more of the elements in identifierSequence are not present in the source observable a problem appears. This currently blocks the ordered observable and it will never finish. Possible solutions: Timeout, throw exception when source observable is completed, return all available items when source observable is completed, ...
When the source observable contains more elements than identifierSequence, we get a memory leak. Items that are in the source observable, but not in identifierSequence currently get added to the dictionary, but will not be deleted before the source observable completes. This is a potential memory leak. Possible solutions: check whether the item is in identifierSequence before adding it to the dictionary, bypass code and immediately output the item, ...
MY SOLUTION:
/// <summary>
/// Takes the items from the source observable, and returns them in the order specified in identifierSequence.
/// If an item is missing from the source observable, the returned obserable returns items up until the missing item and then blocks until the source observable is completed.
/// All available items are then returned in order. Note that this means that while a correct order is guaranteed, there might be missing items in the result observable.
/// If there are items in the source observable that are not in identifierSequence, these items will be ignored.
/// </summary>
/// <typeparam name="T">The type that is produced by the source observable</typeparam>
/// <typeparam name="TId">The type of the identifiers used to uniquely identify a T</typeparam>
/// <param name="source">The source observable</param>
/// <param name="identifierSequence">A list of identifiers that defines the sequence in which the source observable is to be ordered</param>
/// <param name="identifierFunc">A function that takes a T and outputs its unique identifier</param>
/// <returns>An observable with the same elements as the source, but ordered by the sequence of items in identifierSequence</returns>
public static IObservable<T> OrderByIdentifierSequence<T, TId>(this IObservable<T> source, IList<TId> identifierSequence, Func<T, TId> identifierFunc)
{
if (source == null)
{
throw new ArgumentNullException(nameof(source));
}
if (identifierSequence == null)
{
throw new ArgumentNullException(nameof(identifierSequence));
}
if (identifierFunc == null)
{
throw new ArgumentNullException(nameof(identifierFunc));
}
if (identifierSequence.Count == 0)
{
return Observable.Empty<T>();
}
HashSet<TId> identifiersInSequence = new HashSet<TId>(identifierSequence);
return Observable.Create<T>(observer =>
{
//current index of pending item in identifierSequence
int index = 0;
//buffer of items we have received but are not ready for yet
Dictionary<TId, List<T>> buffer = new Dictionary<TId, List<T>>();
return source.Select(
item =>
{
//Function to be called upon receiving new item
//We search for the current pending item in the buffer. If it is available, we yield return it and repeat.
//If it is not available yet, stop.
IEnumerable<T> GetAvailableOutput()
{
while (index < identifierSequence.Count)
{
TId key = identifierSequence[index];
List<T> nextValues;
if (!buffer.TryGetValue(key, out nextValues) || nextValues.Count == 0)
{
//No values available yet
break;
}
yield return nextValues[nextValues.Count - 1];
nextValues.RemoveAt(nextValues.Count - 1);
index++;
}
}
//Get the identifier for this item
TId itemIdentifier = identifierFunc(item);
//If this item is not in identifiersInSequence, we ignore it.
if (!identifiersInSequence.Contains(itemIdentifier))
{
return Enumerable.Empty<T>();
}
//Add the new item to the buffer
List<T> valuesList;
if (!buffer.TryGetValue(itemIdentifier, out valuesList))
{
valuesList = new List<T>();
buffer[itemIdentifier] = valuesList;
}
valuesList.Add(item);
//Return all available items
return GetAvailableOutput();
})
.Subscribe(output =>
{
foreach (T cur in output)
{
observer.OnNext(cur);
}
if (index == identifierSequence.Count)
{
observer.OnCompleted();
}
},(ex) =>
{
observer.OnError(ex);
}, () =>
{
//When source observable is completed, return the remaining available items
while (index < identifierSequence.Count)
{
TId key = identifierSequence[index];
List<T> nextValues;
if (!buffer.TryGetValue(key, out nextValues) || nextValues.Count == 0)
{
//No values available
index++;
continue;
}
observer.OnNext(nextValues[nextValues.Count - 1]);
nextValues.RemoveAt(nextValues.Count - 1);
index++;
}
//Mark observable as completed
observer.OnCompleted();
});
});
}
Please note that your implementation has a few problems:
If the two 'l's come before their time, one gets swallowed, then holding up the whole sequence. Your dictionary should map to a collection, not a single item.
There's no OnCompleted message.
Multiple subscribers can screw up the state. Try this (where GetPatternMatchOriginal is your code):
-
var stateMachine = src.GetPatternMatchOriginal(new int[] { 9, 3, 4, 4, 7 });
stateMachine.Take(3).Dump(); //Linqpad
stateMachine.Take(3).Dump(); //Linqpad
The first ouptut is h e l the second output is l o. They should both output h e l.
This implementation fixes those problems, and also is side-effect free using immutable data structures:
public static class X
{
public static IObservable<T> GetStateMachine(this IObservable<T> source, string identifierSequence)
{
//State is held in an anonymous type:
// Index shows what character we're waiting on,
// Buffer holds characters that have arrived that we aren't ready yet for
// Output holds characters that can be safely emitted.
return source
.Scan(new { Index = 0, Buffer = ImmutableDictionary<int, ImmutableList<T>>.Empty, Output = Enumerable.Empty<T>() },
(state, item) =>
{
//Function to be called recursively upon receiving new item
//If we can pattern match the first item, then it is moved into Output, and concatted recursively with the next possible item
//Otherwise just return the inputs
(int Index, ImmutableDictionary<int, ImmutableList<T>> Buffer, IEnumerable<T> Output) GetOutput(int index, ImmutableDictionary<int, ImmutableList<T>> buffer, IEnumerable<T> results)
{
if (index == identifierSequence.Length)
return (index, buffer, results);
var key = identifierSequence[index];
if (buffer.ContainsKey(key) && buffer[key].Any())
{
var toOuptut = buffer[key][buffer[key].Count - 1];
return GetOutput(index + 1, buffer.SetItem(key, buffer[key].RemoveAt(buffer[key].Count - 1)), results.Concat(new[] { toOuptut }));
}
else
return (index, buffer, results);
}
//Before calling the recursive function, add the new item to the buffer
var modifiedBuffer = state.Buffer.ContainsKey(item.Identifier)
? state.Buffer
: state.Buffer.Add(item.Identifier, ImmutableList<T>.Empty);
var remodifiedBuffer = modifiedBuffer.SetItem(item.Identifier, modifiedBuffer[item.Identifier].Add(item));
var output = GetOutput(state.Index, remodifiedBuffer, Enumerable.Empty<T>());
return new { Index = output.Index, Buffer = output.Buffer, Output = output.Output };
})
// Use Dematerialize/Notifications to detect and emit end of stream.
.SelectMany(output =>
{
var notifications = output.Output
.Select(item => Notification.CreateOnNext(item))
.ToList();
if (output.Index == identifierSequence.Length)
notifications.Add(Notification.CreateOnCompleted<T>());
return notifications;
})
.Dematerialize();
}
}
then you can call it like so:
var stateMachine = src.GetStateMachine(new int[] { 9, 3, 4, 4, 7 });
stateMachine.Dump(); //LinqPad
src.OnNext(new T { Identifier = 4, Character = 'l' });
src.OnNext(new T { Identifier = 4, Character = 'l' });
src.OnNext(new T { Identifier = 7, Character = 'o' });
src.OnNext(new T { Identifier = 3, Character = 'e' });
src.OnNext(new T { Identifier = 9, Character = 'h' });
Given you have this:
IObservable<T> source = new []
{
new T() { identifier = 3, character = 'e' },
new T() { identifier = 9, character = 'h'},
new T() { identifier = 4, character = 'l'},
new T() { identifier = 4, character = 'l'},
new T() { identifier = 7, character = 'o'}
}.ToObservable();
int[] identifierSequence = new int[]
{
9, 3, 4, 4, 7
};
...then this works:
IObservable<T> query =
source
.Scan(new { index = 0, pendings = new List<T>(), outputs = new List<T>() }, (a, t) =>
{
var i = a.index;
var o = new List<T>();
a.pendings.Add(t);
var r = a.pendings.Where(x => x.identifier == identifierSequence[i]).FirstOrDefault();
while (r != null)
{
o.Add(r);
a.pendings.Remove(r);
i++;
r = a.pendings.Where(x => x.identifier == identifierSequence[i]).FirstOrDefault();
}
return new { index = i, a.pendings, outputs = o };
})
.SelectMany(x => x.outputs);
Nice question :-)
Given the multiple identical keys, it looks like pattern matching in an arbitrary order to me. Here's what I come up with:
Edit: modified to look up expected items in a dictionary.
public static class MyExtensions
{
public static IObservable<TSource> MatchByKeys<TSource, TKey>(this IObservable<TSource> source, IEnumerable<TKey> keys, Func<TSource, TKey> keySelector, IEqualityComparer<TKey> keyComparer = null)
{
if (source == null) throw new ArgumentNullException(nameof(source));
if (keys == null) throw new ArgumentNullException(nameof(keys));
if (keySelector == null) throw new ArgumentNullException(nameof(keySelector));
if (keyComparer == null) keyComparer = EqualityComparer<TKey>.Default;
return Observable.Create<TSource>(observer =>
{
var pattern = new LinkedList<SingleAssignment<TSource>>();
var matchesByKey = new Dictionary<TKey, LinkedList<SingleAssignment<TSource>>>(keyComparer);
foreach (var key in keys)
{
var match = new SingleAssignment<TSource>();
pattern.AddLast(match);
LinkedList<SingleAssignment<TSource>> matches;
if (!matchesByKey.TryGetValue(key, out matches))
{
matches = new LinkedList<SingleAssignment<TSource>>();
matchesByKey.Add(key, matches);
}
matches.AddLast(match);
}
if (pattern.First == null)
{
observer.OnCompleted();
return Disposable.Empty;
}
var sourceSubscription = new SingleAssignmentDisposable();
Action dispose = () =>
{
sourceSubscription.Dispose();
pattern.Clear();
matchesByKey.Clear();
};
sourceSubscription.Disposable = source.Subscribe(
value =>
{
try
{
var key = keySelector(value);
LinkedList<SingleAssignment<TSource>> matches;
if (!matchesByKey.TryGetValue(key, out matches)) return;
matches.First.Value.Value = value;
matches.RemoveFirst();
if (matches.First == null) matchesByKey.Remove(key);
while (pattern.First != null && pattern.First.Value.HasValue)
{
var match = pattern.First.Value;
pattern.RemoveFirst();
observer.OnNext(match.Value);
}
if (pattern.First != null) return;
dispose();
observer.OnCompleted();
}
catch (Exception ex)
{
dispose();
observer.OnError(ex);
}
},
error =>
{
dispose();
observer.OnError(error);
},
() =>
{
dispose();
observer.OnCompleted();
});
return Disposable.Create(dispose);
});
}
private sealed class SingleAssignment<T>
{
public bool HasValue { get; private set; }
private T _value;
public T Value
{
get
{
if (!HasValue) throw new InvalidOperationException("No value has been set.");
return _value;
}
set
{
if (HasValue) throw new InvalidOperationException("Value has alredy been set.");
HasValue = true;
_value = value;
}
}
}
}
Test code:
var src = new Subject<T>();
var ordered = src.MatchByKeys(new[] { 9, 3, 4, 4, 7 }, t => t.Identifier);
var result = new List<T>();
using (ordered.Subscribe(result.Add))
{
src.OnNext(new T { Identifier = 3, Character = 'e' });
src.OnNext(new T { Identifier = 9, Character = 'h' });
src.OnNext(new T { Identifier = 4, Character = 'l' });
src.OnNext(new T { Identifier = 4, Character = 'l' });
src.OnNext(new T { Identifier = 7, Character = 'o' });
src.OnCompleted();
}
Console.WriteLine(new string(result.Select(t => t.Character).ToArray()));
Related
This question already has answers here:
Get List<> element position in c# using LINQ
(11 answers)
How to get the index of an element in an IEnumerable?
(12 answers)
Closed 8 years ago.
Given a datasource like that:
var c = new Car[]
{
new Car{ Color="Blue", Price=28000},
new Car{ Color="Red", Price=54000},
new Car{ Color="Pink", Price=9999},
// ..
};
How can I find the index of the first car satisfying a certain condition with LINQ?
EDIT:
I could think of something like this but it looks horrible:
int firstItem = someItems.Select((item, index) => new
{
ItemName = item.Color,
Position = index
}).Where(i => i.ItemName == "purple")
.First()
.Position;
Will it be the best to solve this with a plain old loop?
myCars.Select((v, i) => new {car = v, index = i}).First(myCondition).index;
or the slightly shorter
myCars.Select((car, index) => new {car, index}).First(myCondition).index;
or the slightly shorter shorter
myCars.Select((car, index) => (car, index)).First(myCondition).index;
Simply do :
int index = List.FindIndex(your condition);
E.g.
int index = cars.FindIndex(c => c.ID == 150);
An IEnumerable is not an ordered set.
Although most IEnumerables are ordered, some (such as Dictionary or HashSet) are not.
Therefore, LINQ does not have an IndexOf method.
However, you can write one yourself:
///<summary>Finds the index of the first item matching an expression in an enumerable.</summary>
///<param name="items">The enumerable to search.</param>
///<param name="predicate">The expression to test the items against.</param>
///<returns>The index of the first matching item, or -1 if no items match.</returns>
public static int FindIndex<T>(this IEnumerable<T> items, Func<T, bool> predicate) {
if (items == null) throw new ArgumentNullException("items");
if (predicate == null) throw new ArgumentNullException("predicate");
int retVal = 0;
foreach (var item in items) {
if (predicate(item)) return retVal;
retVal++;
}
return -1;
}
///<summary>Finds the index of the first occurrence of an item in an enumerable.</summary>
///<param name="items">The enumerable to search.</param>
///<param name="item">The item to find.</param>
///<returns>The index of the first matching item, or -1 if the item was not found.</returns>
public static int IndexOf<T>(this IEnumerable<T> items, T item) { return items.FindIndex(i => EqualityComparer<T>.Default.Equals(item, i)); }
myCars.TakeWhile(car => !myCondition(car)).Count();
It works! Think about it. The index of the first matching item equals the number of (not matching) item before it.
Story time
I too dislike the horrible standard solution you already suggested in your question. Like the accepted answer I went for a plain old loop although with a slight modification:
public static int FindIndex<T>(this IEnumerable<T> items, Predicate<T> predicate) {
int index = 0;
foreach (var item in items) {
if (predicate(item)) break;
index++;
}
return index;
}
Note that it will return the number of items instead of -1 when there is no match. But let's ignore this minor annoyance for now. In fact the horrible standard solution crashes in that case and I consider returning an index that is out-of-bounds superior.
What happens now is ReSharper telling me Loop can be converted into LINQ-expression. While most of the time the feature worsens readability, this time the result was awe-inspiring. So Kudos to the JetBrains.
Analysis
Pros
Concise
Combinable with other LINQ
Avoids newing anonymous objects
Only evaluates the enumerable until the predicate matches for the first time
Therefore I consider it optimal in time and space while remaining readable.
Cons
Not quite obvious at first
Does not return -1 when there is no match
Of course you can always hide it behind an extension method. And what to do best when there is no match heavily depends on the context.
I will make my contribution here... why? just because :p Its a different implementation, based on the Any LINQ extension, and a delegate. Here it is:
public static class Extensions
{
public static int IndexOf<T>(
this IEnumerable<T> list,
Predicate<T> condition) {
int i = -1;
return list.Any(x => { i++; return condition(x); }) ? i : -1;
}
}
void Main()
{
TestGetsFirstItem();
TestGetsLastItem();
TestGetsMinusOneOnNotFound();
TestGetsMiddleItem();
TestGetsMinusOneOnEmptyList();
}
void TestGetsFirstItem()
{
// Arrange
var list = new string[] { "a", "b", "c", "d" };
// Act
int index = list.IndexOf(item => item.Equals("a"));
// Assert
if(index != 0)
{
throw new Exception("Index should be 0 but is: " + index);
}
"Test Successful".Dump();
}
void TestGetsLastItem()
{
// Arrange
var list = new string[] { "a", "b", "c", "d" };
// Act
int index = list.IndexOf(item => item.Equals("d"));
// Assert
if(index != 3)
{
throw new Exception("Index should be 3 but is: " + index);
}
"Test Successful".Dump();
}
void TestGetsMinusOneOnNotFound()
{
// Arrange
var list = new string[] { "a", "b", "c", "d" };
// Act
int index = list.IndexOf(item => item.Equals("e"));
// Assert
if(index != -1)
{
throw new Exception("Index should be -1 but is: " + index);
}
"Test Successful".Dump();
}
void TestGetsMinusOneOnEmptyList()
{
// Arrange
var list = new string[] { };
// Act
int index = list.IndexOf(item => item.Equals("e"));
// Assert
if(index != -1)
{
throw new Exception("Index should be -1 but is: " + index);
}
"Test Successful".Dump();
}
void TestGetsMiddleItem()
{
// Arrange
var list = new string[] { "a", "b", "c", "d", "e" };
// Act
int index = list.IndexOf(item => item.Equals("c"));
// Assert
if(index != 2)
{
throw new Exception("Index should be 2 but is: " + index);
}
"Test Successful".Dump();
}
Here is a little extension I just put together.
public static class PositionsExtension
{
public static Int32 Position<TSource>(this IEnumerable<TSource> source,
Func<TSource, bool> predicate)
{
return Positions<TSource>(source, predicate).FirstOrDefault();
}
public static IEnumerable<Int32> Positions<TSource>(this IEnumerable<TSource> source,
Func<TSource, bool> predicate)
{
if (typeof(TSource) is IDictionary)
{
throw new Exception("Dictionaries aren't supported");
}
if (source == null)
{
throw new ArgumentOutOfRangeException("source is null");
}
if (predicate == null)
{
throw new ArgumentOutOfRangeException("predicate is null");
}
var found = source.Where(predicate).First();
var query = source.Select((item, index) => new
{
Found = ReferenceEquals(item, found),
Index = index
}).Where( it => it.Found).Select( it => it.Index);
return query;
}
}
Then you can call it like this.
IEnumerable<Int32> indicesWhereConditionIsMet =
ListItems.Positions(item => item == this);
Int32 firstWelcomeMessage ListItems.Position(msg =>
msg.WelcomeMessage.Contains("Hello"));
Here's an implementation of the highest-voted answer that returns -1 when the item is not found:
public static int FindIndex<T>(this IEnumerable<T> items, Func<T, bool> predicate)
{
var itemsWithIndices = items.Select((item, index) => new { Item = item, Index = index });
var matchingIndices =
from itemWithIndex in itemsWithIndices
where predicate(itemWithIndex.Item)
select (int?)itemWithIndex.Index;
return matchingIndices.FirstOrDefault() ?? -1;
}
I have some data like
ID Sequence customIndex
1 1 0
2 2 0
3 3 2
4 4 1
5 5 0
I need to use sequence in order by when customIndex is zero other wise use customIndex.
So result should be ID in order of 1,2,4,3,5.
I need LINQ implementation using Lambda. I tried some solution but could not implement.
Posting duplicate and deleting previous one, because of wrong formatting the meaning of question got changed and I received bunch of negative votes.
Added code at dotnet fiddle:
https://stable.dotnetfiddle.net/fChl40
The answer is based on assumption, that CustomIndex is greater or equals to zero:
var result =
data.OrderBy(x => x.CustomIndex==0 ? x.Sequence :
data.Where(y => y.CustomIndex==0 && y.Sequence < x.Sequence)
.Max(y => (int?)y.Sequence))
.ThenBy(x => x.CustomIndex);
This is working for provided test data:
l.OrderBy(a => a.customIndex != 0 ?
list.Where(b => b.Sequence < a.Sequence && b.customIndex == 0)
.OrderByDescending(c => c.Sequence)
.FirstOrDefault()
.Sequence : a.Sequence)
.ThenBy(c=>c.customIndex )
.ToList();
The idea is to order non zero values by first preceding zero valued rows, and then by non zero values itself.
This is something I wanted:
public static void Main()
{
List<Data> data = new List<Data>();
data.Add(new Data{ Id=1, Sequence=1, CustomIndex=0});
data.Add(new Data{ Id=5, Sequence=5, CustomIndex=0});
data.Add(new Data{ Id=6, Sequence=6, CustomIndex=2});
data.Add(new Data{ Id=2, Sequence=2, CustomIndex=0});
data.Add(new Data{ Id=3, Sequence=3, CustomIndex=2});
data.Add(new Data{ Id=4, Sequence=4, CustomIndex=1});
data.Add(new Data{ Id=7, Sequence=7, CustomIndex=1});
int o = 0;
var result = data
.OrderBy(x=>x.Sequence).ToList()
.OrderBy((x)=> myCustomSort(x, ref o) )
;
result.ToList().ForEach(x=> Console.WriteLine(x.Id));
}
public static float myCustomSort(Data x, ref int o){
if(x.CustomIndex==0){
o = x.Sequence;
return x.Sequence ;
}
else
return float.Parse(o + "."+ x.CustomIndex);
}
Sample code: https://stable.dotnetfiddle.net/fChl40
I will refine it further
Based on your question and reply to my comment, I understand you need to clusterize the items' collection, then consider Sequence and CustomIndex on all items of each cluster.
Once clustered (split into blocks depending on a specific criterion) you can merge them back into a unique collection, but while doing that you can manipulate each cluster independently the way you need.
public static class extCluster
{
public static IEnumerable<KeyValuePair<bool, T[]>> Clusterize<T>(this IEnumerable<T> self, Func<T, bool> clusterizer)
{
// Prepare temporary data
var bLastCluster = false;
var cluster = new List<T>();
// loop all items
foreach (var item in self)
{
// Compute cluster kind
var bItemCluster = clusterizer(item);
// If last cluster kind is different from current
if (bItemCluster != bLastCluster)
{
// If previous cluster was not empty, return its items
if (cluster.Count > 0)
yield return new KeyValuePair<bool, T[]>(bLastCluster, cluster.ToArray());
// Store new cluster kind and reset items
bLastCluster = bItemCluster;
cluster.Clear();
}
// Add current item to cluster
cluster.Add(item);
}
// If previous cluster was not empty, return its items
if (cluster.Count > 0)
yield return new KeyValuePair<bool, T[]>(bLastCluster, cluster.ToArray());
}
}
// sample
static class Program
{
public class Item
{
public Item(int id, int sequence, int _customIndex)
{
ID = id; Sequence = sequence; customIndex = _customIndex;
}
public int ID, Sequence, customIndex;
}
[STAThread]
static void Main(string[] args)
{
var aItems = new[]
{
new Item(1, 1, 0),
new Item(2, 2, 0),
new Item(3, 3, 2),
new Item(4, 4, 1),
new Item(5, 5, 0)
};
// Split items into clusters
var aClusters = aItems.Clusterize(item => item.customIndex != 0);
// Explode clusters and sort their items
var result = aClusters
.SelectMany(cluster => cluster.Key
? cluster.Value.OrderBy(item => item.customIndex)
: cluster.Value.OrderBy(item => item.Sequence));
}
}
It ain't pretty, but it exemplifies what you were asking for, I think:
using System;
using System.Collections.Generic;
using System.Linq;
public class Program
{
public static void Main()
{
List<Data> data = new List<Data>();
data.Add(new Data { Id = 1, Sequence = 1, CustomIndex = 0 });
data.Add(new Data { Id = 2, Sequence = 2, CustomIndex = 0 });
data.Add(new Data { Id = 3, Sequence = 3, CustomIndex = 2 });
data.Add(new Data { Id = 4, Sequence = 4, CustomIndex = 1 });
data.Add(new Data { Id = 5, Sequence = 5, CustomIndex = 0 });
//List of items where the sequence is what counts
var itemsToPlaceBySequence = data.Where(x => x.CustomIndex == 0).OrderBy(x => x.Sequence).ToList();
//List of items where the custom index counts
var itemsToPlaceByCustomIndex = data.Where(x => x.CustomIndex != 0).OrderBy(x => x.CustomIndex).ToList();
//Array to hold items
var dataSlots = new Data[data.Count];
//Place items by sequence
foreach(var dataBySequence in itemsToPlaceBySequence) {
dataSlots[dataBySequence.Sequence - 1] = dataBySequence ;
}
//Find empty data slots and place remaining items in CustomIndex order
foreach (var dataByCustom in itemsToPlaceByCustomIndex) {
var index = dataSlots.ToList().IndexOf(null);
dataSlots[index] = dataByCustom ;
}
var result = dataSlots.ToList();
result.ForEach(x => Console.WriteLine(x.Id));
var discard = Console.ReadKey();
}
public class Data
{
public int Id { get; set; }
public int Sequence { get; set; }
public int CustomIndex { get; set; }
}
}
The ordering you want to do (order partly on CustomIndex and partly on Sequence) doesn't work like that. But this should be close to what you want. Order first by CustomIndex, and then by Sequence.
var result = data.OrderBy(x => x.CustomIndex).ThenBy(x => x.Sequence);
Is there an elegant way (with LINQ, for example) to partition a list into a list of sublists by delimiter? Partioning { 1, 2, delim, delim, 3, delim, 4, 5 } by delim might produce { { 1, 2 }, { 3 }, { 4, 5 } }....
I don't think there is an easy and elegant way of doing so with default LINQ methods. But you can create your own extension method:
public static class MyExtensions
{
public static IEnumerable<IEnumerable<TElement>> SplitBy<TElement>(
this IEnumerable<TElement> source,
TElement split,
bool skipEmptyGroups = true)
where TElement : IEquatable<TElement>
{
var group = new List<TElement>();
foreach (var item in source)
{
if (split.Equals(item))
{
if (group.Count > 0 || !skipEmptyGroups)
{
yield return group;
group = new List<TElement>();
}
}
else
{
group.Add(item);
}
}
if (group.Count > 0 || !skipEmptyGroups)
yield return group;
}
}
Usage is really simple:
var source = new List<int> { 1, 2, 3, 3, 4, 3, 5, 3, 6, 7, 8 };
var result = source.SplitBy(3);
If you want to return empty groups you can pass additional bool parameter:
var resultWithEmptyGroups = source.SplitBy(3, false);
Directly in Linq, I think it will be hard, but you can create a custom operator. Maybe something like this :
List<String> test = new List<String>() { "1", "8", ";", "2", "7", "42", ";", "3" };
var restul = test.StrangePartition(";");
with :
public static class Helper
{
public static IEnumerable<IEnumerable<T>> StrangePartition<T>(this IEnumerable<T> source, T partitionKey)
{
List<List<T>> partitions = new List<List<T>>();
List<T> partition = new List<T>();
foreach (T item in source)
{
if (item.Equals(partitionKey))
{
partitions.Add(partition);
partition = new List<T>();
}
else
{
partition.Add(item);
}
}
partitions.Add(partition);
return partitions;
}
}
No, there isn't anything similar, but it's quite easy to implement them:
public static class LinqEx
{
public static IEnumerable<List<TSource>> Split<TSource>(this IEnumerable<TSource> enu, TSource delimiter, IEqualityComparer<TSource> comparer = null)
{
// list == null handles the case where enu is empty
List<TSource> list = null;
if (comparer == null)
{
// Note how the equality comparer is "selected".
// This is how LINQ methods do it
// (see Enumerable.SequenceEqual<TSource>)
comparer = EqualityComparer<TSource>.Default;
}
foreach (TSource el in enu)
{
if (comparer.Equals(el, delimiter))
{
if (list == null)
{
list = new List<TSource>();
}
yield return list;
// Note that we have to recreate the list every time!
// We can't simply do a list.Clear()
list = new List<TSource>();
continue;
}
if (list == null)
{
list = new List<TSource>();
}
list.Add(el);
}
if (list != null)
{
yield return list;
}
}
public static IEnumerable<List<TSource>> SplitRemoveEmpty<TSource>(this IEnumerable<TSource> enu, TSource delimiter, IEqualityComparer<TSource> comparer = null)
{
var list = new List<TSource>();
if (comparer == null)
{
// Note how the equality comparer is "selected".
// This is how LINQ methods do it
// (see Enumerable.SequenceEqual<TSource>)
comparer = EqualityComparer<TSource>.Default;
}
foreach (TSource el in enu)
{
if (comparer.Equals(el, delimiter))
{
if (list.Count != 0)
{
yield return list;
// Note that we have to recreate the list every time!
// We can't simply do a list.Clear()
list = new List<TSource>();
}
continue;
}
list.Add(el);
}
if (list.Count != 0)
{
yield return list;
}
}
}
There are two variants: the first one (Split) will return even the empty groups, the second one will strip empty groups.
For example:
{ delim, 1, 2, delim, delim, 3, delim, 4, 5, delim }
With Split you will receive
{ { }, {1, 2}, { }, {3}, {4, 5}, { } }
While with SplitRemoveEmpty
{ {1, 2}, {3}, {4, 5} }
Use them like:
var res = new[] { 1, 2, 0, 0, 3, 4, 0, 5 }.SplitRemoveEmpty(0);
foreach (List<int> group in res)
{
// Do something
}
Remember that these split methods are based on the correctness of the Equals method! But there is an optional last parameter where you can give an IEqualityComparer<TSource> that the methods will use instead of the default one.
Although others mentioned it would be easier with an imperative approach, it is of course do-able in LINQ. The Aggregate operator has a lot of uses, but usually a poor readability.
var testCase = new List<String> { "1", "8", ";", "2", "7", "42", ";", "3" };
var result = testCase.Aggregate(new List<List<String>>() { new List<String>() }, (l, s) => {
if (s == ";") {
l.Add(new List<String>());
} else {
l[l.Count - 1].Add(s);
}
return l;
}).ToList();
It will produce the desired output, but its not only hard to comprehend, but it also produces side-effects within the query (not as bad as producing side-effects outside the query, but still ill-advised).
You can put it into your own extension method if you wishes so:
public static IEnumerable<IEnumerable<T>> PartitionBy(this IEnumerable<T> source, T delimiter) {
return source.Aggregate(new List<List<T>>() { new List<T>() }, (l, elem) =>
if(elem.Equals(delimiter)) {
l.Add(new List<T>());
} else {
l[l.Count - 1].Add(elem);
}
return l;
});
}
Again, consider this answer only for the completeness of a LINQ answer.
I have several huge sorted enumerable sequences that I want to merge. Theses lists are manipulated as IEnumerable but are already sorted. Since input lists are sorted, it should be possible to merge them in one trip, without re-sorting anything.
I would like to keep the defered execution behavior.
I tried to write a naive algorithm which do that (see below). However, it looks pretty ugly and I'm sure it can be optimized. It may exist a more academical algorithm...
IEnumerable<T> MergeOrderedLists<T, TOrder>(IEnumerable<IEnumerable<T>> orderedlists,
Func<T, TOrder> orderBy)
{
var enumerators = orderedlists.ToDictionary(l => l.GetEnumerator(), l => default(T));
IEnumerator<T> tag = null;
var firstRun = true;
while (true)
{
var toRemove = new List<IEnumerator<T>>();
var toAdd = new List<KeyValuePair<IEnumerator<T>, T>>();
foreach (var pair in enumerators.Where(pair => firstRun || tag == pair.Key))
{
if (pair.Key.MoveNext())
toAdd.Add(pair);
else
toRemove.Add(pair.Key);
}
foreach (var enumerator in toRemove)
enumerators.Remove(enumerator);
foreach (var pair in toAdd)
enumerators[pair.Key] = pair.Key.Current;
if (enumerators.Count == 0)
yield break;
var min = enumerators.OrderBy(t => orderBy(t.Value)).FirstOrDefault();
tag = min.Key;
yield return min.Value;
firstRun = false;
}
}
The method can be used like that:
// Person lists are already sorted by age
MergeOrderedLists(orderedList, p => p.Age);
assuming the following Person class exists somewhere:
public class Person
{
public int Age { get; set; }
}
Duplicates should be conserved, we don't care about their order in the new sequence. Do you see any obvious optimization I could use?
Here is my fourth (thanks to #tanascius for pushing this along to something much more LINQ) cut at it:
public static IEnumerable<T> MergePreserveOrder3<T, TOrder>(
this IEnumerable<IEnumerable<T>> aa,
Func<T, TOrder> orderFunc)
where TOrder : IComparable<TOrder>
{
var items = aa.Select(xx => xx.GetEnumerator()).Where(ee => ee.MoveNext())
.OrderBy(ee => orderFunc(ee.Current)).ToList();
while (items.Count > 0)
{
yield return items[0].Current;
var next = items[0];
items.RemoveAt(0);
if (next.MoveNext())
{
// simple sorted linear insert
var value = orderFunc(next.Current);
var ii = 0;
for ( ; ii < items.Count; ++ii)
{
if (value.CompareTo(orderFunc(items[ii].Current)) <= 0)
{
items.Insert(ii, next);
break;
}
}
if (ii == items.Count) items.Add(next);
}
else next.Dispose(); // woops! can't forget IDisposable
}
}
Results:
for (int p = 0; p < people.Count; ++p)
{
Console.WriteLine("List {0}:", p + 1);
Console.WriteLine("\t{0}", String.Join(", ", people[p].Select(x => x.Name)));
}
Console.WriteLine("Merged:");
foreach (var person in people.MergePreserveOrder(pp => pp.Age))
{
Console.WriteLine("\t{0}", person.Name);
}
List 1:
8yo, 22yo, 47yo, 49yo
List 2:
35yo, 47yo, 60yo
List 3:
28yo, 55yo, 64yo
Merged:
8yo
22yo
28yo
35yo
47yo
47yo
49yo
55yo
60yo
64yo
Improved with .Net 4.0's Tuple support:
public static IEnumerable<T> MergePreserveOrder4<T, TOrder>(
this IEnumerable<IEnumerable<T>> aa,
Func<T, TOrder> orderFunc) where TOrder : IComparable<TOrder>
{
var items = aa.Select(xx => xx.GetEnumerator())
.Where(ee => ee.MoveNext())
.Select(ee => Tuple.Create(orderFunc(ee.Current), ee))
.OrderBy(ee => ee.Item1).ToList();
while (items.Count > 0)
{
yield return items[0].Item2.Current;
var next = items[0];
items.RemoveAt(0);
if (next.Item2.MoveNext())
{
var value = orderFunc(next.Item2.Current);
var ii = 0;
for (; ii < items.Count; ++ii)
{
if (value.CompareTo(items[ii].Item1) <= 0)
{ // NB: using a tuple to minimize calls to orderFunc
items.Insert(ii, Tuple.Create(value, next.Item2));
break;
}
}
if (ii == items.Count) items.Add(Tuple.Create(value, next.Item2));
}
else next.Item2.Dispose(); // woops! can't forget IDisposable
}
}
One guess I would make that might improve clarity and performance is this:
Create a priority queue over pairs of T, IEnumerable<T> ordered according to your comparison function on T
For each IEnumerable<T> being merged, add the item to the priority queue annotated with a reference to the IEnumerable<T> where it originated
While the priority queue is not empty
Extract the minimum element from the priority queue
Advance the IEnumerable<T> in its annotation to the next element
If MoveNext() returned true, add the next element to the priority queue annotated with a reference to the IEnumerable<T> you just advanced
If MoveNext() returned false, don't add anything to the priority queue
Yield the dequeued element
How many lists do you expect to need to merge? It looks like your algorithm will not be efficient if you have many different lists to merge. This line is the issue:
var min = enumerators.OrderBy(t => orderBy(t.Value)).FirstOrDefault();
This will be run once for each element in all the lists, so your runtime will be O(n * m), where n is the TOTAL number of elements in all the lists, and n is the number of lists. Expressed in terms of the average length of a list in the list of lists, the runtime is O(a * m^2).
If you are going to need to merge a lot of lists, I would suggest using a heap. Then each iteration you can remove the smallest value from the heap, and add the next element to the heap from the list that the smallest value came from.
Here's a solution with NO SORTING ... just the minimum number of comparisons. (I omitted the actual order func passing for simplicity). Updated to build a balanced tree:-
/// <summary>
/// Merge a pair of ordered lists
/// </summary>
public static IEnumerable<T> Merge<T>(IEnumerable<T> aList, IEnumerable<T> bList)
where T:IComparable<T>
{
var a = aList.GetEnumerator();
bool aOK = a.MoveNext();
foreach (var b in bList)
{
while (aOK && a.Current.CompareTo(b) <= 0) {yield return a.Current; aOK = a.MoveNext();}
yield return b;
}
// And anything left in a
while (aOK) { yield return a.Current; aOK = a.MoveNext(); }
}
/// <summary>
/// Merge lots of sorted lists
/// </summary>
public static IEnumerable<T> Merge<T>(IEnumerable<IEnumerable<T>> listOfLists)
where T : IComparable<T>
{
int n = listOfLists.Count();
if (n < 2)
return listOfLists.FirstOrDefault();
else
return Merge (Merge(listOfLists.Take(n/2)), Merge(listOfLists.Skip(n/2)));
}
public static void Main(string[] args)
{
var sample = Enumerable.Range(1, 5).Select((i) => Enumerable.Range(i, i+5).Select(j => string.Format("Test {0:00}", j)));
Console.WriteLine("Merged:");
foreach (var result in Merge(sample))
{
Console.WriteLine("\t{0}", result);
}
Here is a solution that has very good complexity analysis and that is considerably shorter than the other solutions proposed.
public static IEnumerable<T> Merge<T>(this IEnumerable<IEnumerable<T>> self)
where T : IComparable<T>
{
var es = self.Select(x => x.GetEnumerator()).Where(e => e.MoveNext());
var tmp = es.ToDictionary(e => e.Current);
var dict = new SortedDictionary<T, IEnumerator<T>>(tmp);
while (dict.Count > 0)
{
var key = dict.Keys.First();
var cur = dict[key];
dict.Remove(key);
yield return cur.Current;
if (cur.MoveNext())
dict.Add(cur.Current, cur);
}
}
Here is my solution:
The algorithm takes the first element of each list and puts them within a small helper class (a sorted list that accepts mutliple elements with the same value). This sorted list uses a binary insert.
So the first element in this list is the element we want to return next. After doing so we remove it from the sorted list and insert the next element from its original source list (at least as long as this list contains any more elements). Again, we can return the first element of our sorted list. When the sorted list is empty once, we used all element from all different source lists and are done.
This solution uses less foreach statements and no OrderBy in each step - which should improve the runtime behaviour. Only the binary insert has to be done again and again.
IEnumerable<T> MergeOrderedLists<T, TOrder>( IEnumerable<IEnumerable<T>> orderedlists, Func<T, TOrder> orderBy )
{
// Get an enumerator for each list, create a sortedList
var enumerators = orderedlists.Select( enumerable => enumerable.GetEnumerator() );
var sortedEnumerators = new SortedListAllowingDoublets<TOrder, IEnumerator<T>>();
// Point each enumerator onto the first element
foreach( var enumerator in enumerators )
{
// Missing: assert true as the return value
enumerator.MoveNext();
// Initially add the first value
sortedEnumerators.AddSorted( orderBy( enumerator.Current ), enumerator );
}
// Continue as long as we have elements to return
while( sortedEnumerators.Count != 0 )
{
// The first element of the sortedEnumerator list always
// holds the next element to return
var enumerator = sortedEnumerators[0].Value;
// Return this enumerators current value
yield return enumerator.Current;
// Remove the element we just returned
sortedEnumerators.RemoveAt( 0 );
// Check if there is another element in the list of the enumerator
if( enumerator.MoveNext() )
{
// Ok, so add it to the sorted list
sortedEnumerators.AddSorted( orderBy( enumerator.Current ), enumerator );
}
}
My helper class (using a simple binary insert):
private class SortedListAllowingDoublets<TOrder, T> : Collection<KeyValuePair<TOrder, T>> where T : IEnumerator
{
public void AddSorted( TOrder value, T enumerator )
{
Insert( GetSortedIndex( value, 0, Count - 1 ), new KeyValuePair<TOrder, T>( value, enumerator ) );
}
private int GetSortedIndex( TOrder item, int startIndex, int endIndex )
{
if( startIndex > endIndex )
{
return startIndex;
}
var midIndex = startIndex + ( endIndex - startIndex ) / 2;
return Comparer<TOrder>.Default.Compare( this[midIndex].Key, item ) < 0 ? GetSortedIndex( item, midIndex + 1, endIndex ) : GetSortedIndex( item, startIndex, midIndex - 1 );
}
}
What's not implemented right now: check for an empty list, which will cause problems.
And the SortedListAllowingDoublets class could be improved to take a comparer instead of using the Comparer<TOrder>.Default on its own.
Here is a Linq friendly solution based on the Wintellect's OrderedBag:
public static IEnumerable<T> MergeOrderedLists<T, TOrder>(this IEnumerable<IEnumerable<T>> orderedLists, Func<T, TOrder> orderBy)
where TOrder : IComparable<TOrder>
{
var enumerators = new OrderedBag<IEnumerator<T>>(orderedLists
.Select(enumerable => enumerable.GetEnumerator())
.Where(enumerator => enumerator.MoveNext()),
(x, y) => orderBy(x.Current).CompareTo(orderBy(y.Current)));
while (enumerators.Count > 0)
{
IEnumerator<T> minEnumerator = enumerators.RemoveFirst();
T minValue = minEnumerator.Current;
if (minEnumerator.MoveNext())
enumerators.Add(minEnumerator);
else
minEnumerator.Dispose();
yield return minValue;
}
}
If you use any Enumerator based solution, don't forget to call Dispose()
And here is a simple test:
[Test]
public void ShouldMergeInOrderMultipleOrderedListWithDuplicateValues()
{
// given
IEnumerable<IEnumerable<int>> orderedLists = new[]
{
new [] {1, 5, 7},
new [] {1, 2, 4, 6, 7}
};
// test
IEnumerable<int> merged = orderedLists.MergeOrderedLists(i => i);
// expect
merged.ShouldAllBeEquivalentTo(new [] { 1, 1, 2, 4, 5, 6, 7, 7 });
}
My version of sixlettervariables's answer. I reduced the number of calls to orderFunc (each element only passes through orderFunc once), and in the case of ties, sorting is skipped. This is optimized for small numbers of sources, larger numbers of elements within each source and possibly an expensive orderFunc.
public static IEnumerable<T> MergePreserveOrder<T, TOrder>(
this IEnumerable<IEnumerable<T>> sources,
Func<T, TOrder> orderFunc)
where TOrder : IComparable<TOrder>
{
Dictionary<TOrder, List<IEnumerable<T>>> keyedSources =
sources.Select(source => source.GetEnumerator())
.Where(e => e.MoveNext())
.GroupBy(e => orderFunc(e.Current))
.ToDictionary(g => g.Key, g => g.ToList());
while (keyedSources.Any())
{
//this is the expensive line
KeyValuePair<TOrder, List<IEnumerable<T>>> firstPair = keyedSources
.OrderBy(kvp => kvp.Key).First();
keyedSources.Remove(firstPair.Key);
foreach(IEnumerable<T> e in firstPair.Value)
{
yield return e.Current;
if (e.MoveNext())
{
TOrder newKey = orderFunc(e.Current);
if (!keyedSources.ContainsKey(newKey))
{
keyedSources[newKey] = new List<IEnumerable<T>>() {e};
}
else
{
keyedSources[newKey].Add(e);
}
}
}
}
}
I'm betting this could be further improved by a SortedDictionary, but am not brave enough to try a solution using one without an editor.
Here is a modern implementation that is based on the new and powerful PriorityQueue<TElement, TPriority> class (.NET 6). It combines the low overhead of user7116's solution, with the O(log n) complexity of tanascius's solution (where N is the number of sources). It outperforms most of the other implementations presented in this question (I didn't measure them all), either slightly for small N, or massively for large N.
public static IEnumerable<TSource> MergeSorted<TSource, TKey>(
this IEnumerable<IEnumerable<TSource>> sortedSources,
Func<TSource, TKey> keySelector,
IComparer<TKey> comparer = default)
{
List<IEnumerator<TSource>> enumerators = new();
try
{
foreach (var source in sortedSources)
enumerators.Add(source.GetEnumerator());
var queue = new PriorityQueue<IEnumerator<TSource>, TKey>(comparer);
foreach (var enumerator in enumerators)
{
if (enumerator.MoveNext())
queue.Enqueue(enumerator, keySelector(enumerator.Current));
}
while (queue.TryPeek(out var enumerator, out _))
{
yield return enumerator.Current;
if (enumerator.MoveNext())
queue.EnqueueDequeue(enumerator, keySelector(enumerator.Current));
else
queue.Dequeue();
}
}
finally
{
foreach (var enumerator in enumerators) enumerator.Dispose();
}
}
In order to keep the code simple, all enumerators are disposed at the end of the combined enumeration. A more sophisticated implementation would dispose each enumerator immediately after its completion.
This looks like a terribly useful function to have around so i decided to take a stab at it. My approach is a lot like heightechrider in that it breaks the problem down into merging two sorted IEnumerables into one, then taking that one and merging it with the next in the list. There is most likely some optimization you can do but it works with my simple testcase:
public static IEnumerable<T> mergeSortedEnumerables<T>(
this IEnumerable<IEnumerable<T>> listOfLists,
Func<T, T, Boolean> func)
{
IEnumerable<T> l1 = new List<T>{};
foreach (var l in listOfLists)
{
l1 = l1.mergeTwoSorted(l, func);
}
foreach (var t in l1)
{
yield return t;
}
}
public static IEnumerable<T> mergeTwoSorted<T>(
this IEnumerable<T> l1,
IEnumerable<T> l2,
Func<T, T, Boolean> func)
{
using (var enumerator1 = l1.GetEnumerator())
using (var enumerator2 = l2.GetEnumerator())
{
bool enum1 = enumerator1.MoveNext();
bool enum2 = enumerator2.MoveNext();
while (enum1 || enum2)
{
T t1 = enumerator1.Current;
T t2 = enumerator2.Current;
//if they are both false
if (!enum1 && !enum2)
{
break;
}
//if enum1 is false
else if (!enum1)
{
enum2 = enumerator2.MoveNext();
yield return t2;
}
//if enum2 is false
else if (!enum2)
{
enum1 = enumerator1.MoveNext();
yield return t1;
}
//they are both true
else
{
//if func returns true then t1 < t2
if (func(t1, t2))
{
enum1 = enumerator1.MoveNext();
yield return t1;
}
else
{
enum2 = enumerator2.MoveNext();
yield return t2;
}
}
}
}
}
Then to test it:
List<int> ws = new List<int>() { 1, 8, 9, 16, 17, 21 };
List<int> xs = new List<int>() { 2, 7, 10, 15, 18 };
List<int> ys = new List<int>() { 3, 6, 11, 14 };
List<int> zs = new List<int>() { 4, 5, 12, 13, 19, 20 };
List<IEnumerable<int>> lss = new List<IEnumerable<int>> { ws, xs, ys, zs };
foreach (var v in lss.mergeSortedEnumerables(compareInts))
{
Console.WriteLine(v);
}
I was asked this question as an interview question this evening and did not have a great answer in the 20 or so minutes allotted. So I forced myself to write an algorithm without doing any searches. The constraint was that the inputs were already sorted. Here's my code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace Merger
{
class Program
{
static void Main(string[] args)
{
int[] a = { 1, 3, 6, 102, 105, 230 };
int[] b = { 101, 103, 112, 155, 231 };
var mm = new MergeMania();
foreach(var val in mm.Merge<int>(a, b))
{
Console.WriteLine(val);
}
Console.ReadLine();
}
}
public class MergeMania
{
public IEnumerable<T> Merge<T>(params IEnumerable<T>[] sortedSources)
where T : IComparable
{
if (sortedSources == null || sortedSources.Length == 0)
throw new ArgumentNullException("sortedSources");
//1. fetch enumerators for each sourc
var enums = (from n in sortedSources
select n.GetEnumerator()).ToArray();
//2. fetch enumerators that have at least one value
var enumsWithValues = (from n in enums
where n.MoveNext()
select n).ToArray();
if (enumsWithValues.Length == 0) yield break; //nothing to iterate over
//3. sort by current value in List<IEnumerator<T>>
var enumsByCurrent = (from n in enumsWithValues
orderby n.Current
select n).ToList();
//4. loop through
while (true)
{
//yield up the lowest value
yield return enumsByCurrent[0].Current;
//move the pointer on the enumerator with that lowest value
if (!enumsByCurrent[0].MoveNext())
{
//remove the first item in the list
enumsByCurrent.RemoveAt(0);
//check for empty
if (enumsByCurrent.Count == 0) break; //we're done
}
enumsByCurrent = enumsByCurrent.OrderBy(x => x.Current).ToList();
}
}
}
}
Hope it helps.
An attempt to improve on #cdiggins's answer.
This implementation works correctly if two elements that compare as equal are contained in two different sequences (i. e. doesn't have the flaw mentioned by #ChadHenderson).
The algorithm is described in Wikipedia, the complexity is O(m log n), where n is the number of lists being merged and m is the sum of the lengths of the lists.
The OrderedBag<T> from Wintellect.PowerCollections is used instead of a heap-based priority queue, but it doesn't change the complexity.
public static IEnumerable<T> Merge<T>(
IEnumerable<IEnumerable<T>> listOfLists,
Func<T, T, int> comparison = null)
{
IComparer<T> cmp = comparison != null
? Comparer<T>.Create(new Comparison<T>(comparison))
: Comparer<T>.Default;
List<IEnumerator<T>> es = listOfLists
.Select(l => l.GetEnumerator())
.Where(e => e.MoveNext())
.ToList();
var bag = new OrderedBag<IEnumerator<T>>(
(e1, e2) => cmp.Compare(e1.Current, e2.Current));
es.ForEach(e => bag.Add(e));
while (bag.Count > 0)
{
IEnumerator<T> e = bag.RemoveFirst();
yield return e.Current;
if (e.MoveNext())
{
bag.Add(e);
}
}
}
Each list being merged should be already sorted. This method will locate the equal elements with respect to the order of their lists. For example, if elements Ti == Tj, and they are respectively from list i and list j (i < j), then Ti will be in front of Tj in the merged result.
The complexity is O(mn), where n is the number of lists being merged and m is the sum of the lengths of the lists.
public static IEnumerable<T> Merge<T, TOrder>(this IEnumerable<IEnumerable<T>> TEnumerable_2, Func<T, TOrder> orderFunc, IComparer<TOrder> cmp=null)
{
if (cmp == null)
{
cmp = Comparer<TOrder>.Default;
}
List<IEnumerator<T>> TEnumeratorLt = TEnumerable_2
.Select(l => l.GetEnumerator())
.Where(e => e.MoveNext())
.ToList();
while (TEnumeratorLt.Count > 0)
{
int intMinIndex;
IEnumerator<T> TSmallest = TEnumeratorLt.GetMin(TElement => orderFunc(TElement.Current), out intMinIndex, cmp);
yield return TSmallest.Current;
if (TSmallest.MoveNext() == false)
{
TEnumeratorLt.RemoveAt(intMinIndex);
}
}
}
/// <summary>
/// Get the first min item in an IEnumerable, and return the index of it by minIndex
/// </summary>
public static T GetMin<T, TOrder>(this IEnumerable<T> self, Func<T, TOrder> orderFunc, out int minIndex, IComparer<TOrder> cmp = null)
{
if (self == null) throw new ArgumentNullException("self");
IEnumerator<T> selfEnumerator = self.GetEnumerator();
if (!selfEnumerator.MoveNext()) throw new ArgumentException("List is empty.", "self");
if (cmp == null) cmp = Comparer<TOrder>.Default;
T min = selfEnumerator.Current;
minIndex = 0;
int intCount = 1;
while (selfEnumerator.MoveNext ())
{
if (cmp.Compare(orderFunc(selfEnumerator.Current), orderFunc(min)) < 0)
{
min = selfEnumerator.Current;
minIndex = intCount;
}
intCount++;
}
return min;
}
I've took a more functional approach, hope this reads well.
First of all here is the merge method itself:
public static IEnumerable<T> MergeSorted<T>(IEnumerable<IEnumerable<T>> xss) where T :IComparable
{
var stacks = xss.Select(xs => new EnumerableStack<T>(xs)).ToList();
while (true)
{
if (stacks.All(x => x.IsEmpty)) yield break;
yield return
stacks
.Where(x => !x.IsEmpty)
.Select(x => new { peek = x.Peek(), x })
.MinBy(x => x.peek)
.x.Pop();
}
}
The idea is that we turn each IEnumerable into EnumerableStack that has Peek(), Pop() and IsEmpty members.
It works just like a regular stack. Note that calling IsEmpty might enumerate wrapped IEnumerable.
Here is the code:
/// <summary>
/// Wraps IEnumerable in Stack like wrapper
/// </summary>
public class EnumerableStack<T>
{
private enum StackState
{
Pending,
HasItem,
Empty
}
private readonly IEnumerator<T> _enumerator;
private StackState _state = StackState.Pending;
public EnumerableStack(IEnumerable<T> xs)
{
_enumerator = xs.GetEnumerator();
}
public T Pop()
{
var res = Peek(isEmptyMessage: "Cannot Pop from empty EnumerableStack");
_state = StackState.Pending;
return res;
}
public T Peek()
{
return Peek(isEmptyMessage: "Cannot Peek from empty EnumerableStack");
}
public bool IsEmpty
{
get
{
if (_state == StackState.Empty) return true;
if (_state == StackState.HasItem) return false;
ReadNext();
return _state == StackState.Empty;
}
}
private T Peek(string isEmptyMessage)
{
if (_state != StackState.HasItem)
{
if (_state == StackState.Empty) throw new InvalidOperationException(isEmptyMessage);
ReadNext();
if (_state == StackState.Empty) throw new InvalidOperationException(isEmptyMessage);
}
return _enumerator.Current;
}
private void ReadNext()
{
_state = _enumerator.MoveNext() ? StackState.HasItem : StackState.Empty;
}
}
Finally, here is the MinBy extension method in case haven't written one on your own already:
public static T MinBy<T, TS>(this IEnumerable<T> xs, Func<T, TS> selector) where TS : IComparable
{
var en = xs.GetEnumerator();
if (!en.MoveNext()) throw new Exception();
T max = en.Current;
TS maxVal = selector(max);
while(en.MoveNext())
{
var x = en.Current;
var val = selector(x);
if (val.CompareTo(maxVal) < 0)
{
max = x;
maxVal = val;
}
}
return max;
}
This is an alternate solution:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Reflection;
using System.Data;
using System.Text.RegularExpressions;
namespace ConsoleApplication1
{
class Person
{
public string Name
{
get;
set;
}
public int Age
{
get;
set;
}
}
public class Program
{
public static void Main()
{
Person[] persons1 = new Person[] { new Person() { Name = "Ahmed", Age = 20 }, new Person() { Name = "Ali", Age = 40 } };
Person[] persons2 = new Person[] { new Person() { Name = "Zaid", Age = 21 }, new Person() { Name = "Hussain", Age = 22 } };
Person[] persons3 = new Person[] { new Person() { Name = "Linda", Age = 19 }, new Person() { Name = "Souad", Age = 60 } };
Person[][] personArrays = new Person[][] { persons1, persons2, persons3 };
foreach(Person person in MergeOrderedLists<Person, int>(personArrays, person => person.Age))
{
Console.WriteLine("{0} {1}", person.Name, person.Age);
}
Console.ReadLine();
}
static IEnumerable<T> MergeOrderedLists<T, TOrder>(IEnumerable<IEnumerable<T>> orderedlists, Func<T, TOrder> orderBy)
{
List<IEnumerator<T>> enumeratorsWithData = orderedlists.Select(enumerable => enumerable.GetEnumerator())
.Where(enumerator => enumerator.MoveNext()).ToList();
while (enumeratorsWithData.Count > 0)
{
IEnumerator<T> minEnumerator = enumeratorsWithData[0];
for (int i = 1; i < enumeratorsWithData.Count; i++)
if (((IComparable<TOrder>)orderBy(minEnumerator.Current)).CompareTo(orderBy(enumeratorsWithData[i].Current)) >= 0)
minEnumerator = enumeratorsWithData[i];
yield return minEnumerator.Current;
if (!minEnumerator.MoveNext())
enumeratorsWithData.Remove(minEnumerator);
}
}
}
}
I'm suspicious LINQ is smart enough to take advantage of the prior existing sort order:
IEnumerable<string> BiggerSortedList = BigListOne.Union(BigListTwo).OrderBy(s => s);
This question already has answers here:
Get List<> element position in c# using LINQ
(11 answers)
How to get the index of an element in an IEnumerable?
(12 answers)
Closed 8 years ago.
Given a datasource like that:
var c = new Car[]
{
new Car{ Color="Blue", Price=28000},
new Car{ Color="Red", Price=54000},
new Car{ Color="Pink", Price=9999},
// ..
};
How can I find the index of the first car satisfying a certain condition with LINQ?
EDIT:
I could think of something like this but it looks horrible:
int firstItem = someItems.Select((item, index) => new
{
ItemName = item.Color,
Position = index
}).Where(i => i.ItemName == "purple")
.First()
.Position;
Will it be the best to solve this with a plain old loop?
myCars.Select((v, i) => new {car = v, index = i}).First(myCondition).index;
or the slightly shorter
myCars.Select((car, index) => new {car, index}).First(myCondition).index;
or the slightly shorter shorter
myCars.Select((car, index) => (car, index)).First(myCondition).index;
Simply do :
int index = List.FindIndex(your condition);
E.g.
int index = cars.FindIndex(c => c.ID == 150);
An IEnumerable is not an ordered set.
Although most IEnumerables are ordered, some (such as Dictionary or HashSet) are not.
Therefore, LINQ does not have an IndexOf method.
However, you can write one yourself:
///<summary>Finds the index of the first item matching an expression in an enumerable.</summary>
///<param name="items">The enumerable to search.</param>
///<param name="predicate">The expression to test the items against.</param>
///<returns>The index of the first matching item, or -1 if no items match.</returns>
public static int FindIndex<T>(this IEnumerable<T> items, Func<T, bool> predicate) {
if (items == null) throw new ArgumentNullException("items");
if (predicate == null) throw new ArgumentNullException("predicate");
int retVal = 0;
foreach (var item in items) {
if (predicate(item)) return retVal;
retVal++;
}
return -1;
}
///<summary>Finds the index of the first occurrence of an item in an enumerable.</summary>
///<param name="items">The enumerable to search.</param>
///<param name="item">The item to find.</param>
///<returns>The index of the first matching item, or -1 if the item was not found.</returns>
public static int IndexOf<T>(this IEnumerable<T> items, T item) { return items.FindIndex(i => EqualityComparer<T>.Default.Equals(item, i)); }
myCars.TakeWhile(car => !myCondition(car)).Count();
It works! Think about it. The index of the first matching item equals the number of (not matching) item before it.
Story time
I too dislike the horrible standard solution you already suggested in your question. Like the accepted answer I went for a plain old loop although with a slight modification:
public static int FindIndex<T>(this IEnumerable<T> items, Predicate<T> predicate) {
int index = 0;
foreach (var item in items) {
if (predicate(item)) break;
index++;
}
return index;
}
Note that it will return the number of items instead of -1 when there is no match. But let's ignore this minor annoyance for now. In fact the horrible standard solution crashes in that case and I consider returning an index that is out-of-bounds superior.
What happens now is ReSharper telling me Loop can be converted into LINQ-expression. While most of the time the feature worsens readability, this time the result was awe-inspiring. So Kudos to the JetBrains.
Analysis
Pros
Concise
Combinable with other LINQ
Avoids newing anonymous objects
Only evaluates the enumerable until the predicate matches for the first time
Therefore I consider it optimal in time and space while remaining readable.
Cons
Not quite obvious at first
Does not return -1 when there is no match
Of course you can always hide it behind an extension method. And what to do best when there is no match heavily depends on the context.
I will make my contribution here... why? just because :p Its a different implementation, based on the Any LINQ extension, and a delegate. Here it is:
public static class Extensions
{
public static int IndexOf<T>(
this IEnumerable<T> list,
Predicate<T> condition) {
int i = -1;
return list.Any(x => { i++; return condition(x); }) ? i : -1;
}
}
void Main()
{
TestGetsFirstItem();
TestGetsLastItem();
TestGetsMinusOneOnNotFound();
TestGetsMiddleItem();
TestGetsMinusOneOnEmptyList();
}
void TestGetsFirstItem()
{
// Arrange
var list = new string[] { "a", "b", "c", "d" };
// Act
int index = list.IndexOf(item => item.Equals("a"));
// Assert
if(index != 0)
{
throw new Exception("Index should be 0 but is: " + index);
}
"Test Successful".Dump();
}
void TestGetsLastItem()
{
// Arrange
var list = new string[] { "a", "b", "c", "d" };
// Act
int index = list.IndexOf(item => item.Equals("d"));
// Assert
if(index != 3)
{
throw new Exception("Index should be 3 but is: " + index);
}
"Test Successful".Dump();
}
void TestGetsMinusOneOnNotFound()
{
// Arrange
var list = new string[] { "a", "b", "c", "d" };
// Act
int index = list.IndexOf(item => item.Equals("e"));
// Assert
if(index != -1)
{
throw new Exception("Index should be -1 but is: " + index);
}
"Test Successful".Dump();
}
void TestGetsMinusOneOnEmptyList()
{
// Arrange
var list = new string[] { };
// Act
int index = list.IndexOf(item => item.Equals("e"));
// Assert
if(index != -1)
{
throw new Exception("Index should be -1 but is: " + index);
}
"Test Successful".Dump();
}
void TestGetsMiddleItem()
{
// Arrange
var list = new string[] { "a", "b", "c", "d", "e" };
// Act
int index = list.IndexOf(item => item.Equals("c"));
// Assert
if(index != 2)
{
throw new Exception("Index should be 2 but is: " + index);
}
"Test Successful".Dump();
}
Here is a little extension I just put together.
public static class PositionsExtension
{
public static Int32 Position<TSource>(this IEnumerable<TSource> source,
Func<TSource, bool> predicate)
{
return Positions<TSource>(source, predicate).FirstOrDefault();
}
public static IEnumerable<Int32> Positions<TSource>(this IEnumerable<TSource> source,
Func<TSource, bool> predicate)
{
if (typeof(TSource) is IDictionary)
{
throw new Exception("Dictionaries aren't supported");
}
if (source == null)
{
throw new ArgumentOutOfRangeException("source is null");
}
if (predicate == null)
{
throw new ArgumentOutOfRangeException("predicate is null");
}
var found = source.Where(predicate).First();
var query = source.Select((item, index) => new
{
Found = ReferenceEquals(item, found),
Index = index
}).Where( it => it.Found).Select( it => it.Index);
return query;
}
}
Then you can call it like this.
IEnumerable<Int32> indicesWhereConditionIsMet =
ListItems.Positions(item => item == this);
Int32 firstWelcomeMessage ListItems.Position(msg =>
msg.WelcomeMessage.Contains("Hello"));
Here's an implementation of the highest-voted answer that returns -1 when the item is not found:
public static int FindIndex<T>(this IEnumerable<T> items, Func<T, bool> predicate)
{
var itemsWithIndices = items.Select((item, index) => new { Item = item, Index = index });
var matchingIndices =
from itemWithIndex in itemsWithIndices
where predicate(itemWithIndex.Item)
select (int?)itemWithIndex.Index;
return matchingIndices.FirstOrDefault() ?? -1;
}