Develop Reference

Get the first item of an IEnumerable and return the rest as IEnumerable, iterating through only once

I've got an enumerable that contains responses from a service call that come in gradually.
I can't do ToList on the enumerable as that would block until all responses are received instead of listing them as they come.
I also can't iterate twice as that would trigger another service call.
How to get the first element in the enumerable and return the continuation of the enumerable? I can't use an iterator method as I get a compilation error:
Iterators cannot have ref, in or out parameters.
I've tried this code:
public IEnumerable<object> GetFirstAndRemainder(IEnumerable<object> enumerable, out object first)
{
first = enumerable.Take(1).FirstOrDefault();
return enumerable.Skip(1); // Second interation - unexceptable
}
// This one has a compilation error: Iterators cannot have ref, in or out parameters
public IEnumerable<object> GetFirstAndRemainder2(IEnumerable<object> enumerable, out object first)
{
var enumerator = enumerable.GetEnumerator();
enumerator.MoveNext();
first = enumerator.Current;
while (enumerator.MoveNext())
{
yield return enumerator.Current;
}
}

Instead of using an out parameter, you can use ValueTuple<T1, T2> (as of C# 7.0, documented here) to return two elements: the first item of the IEnumerable<T>, and the remainder as another IEnumerable<T>.
using System.Linq;
class Program {
static void Main(string[] args) {
(int first, IEnumerable<int> remainder) = GetFirstAndRemainder(Enumerable.Range(1, 5));
// first = 1
// remainder yields (2, 3, 4, 5)
}
// Returns the first item and the remainders as an IEnumerable
static (T, IEnumerable<T>) GetFirstAndRemainder<T>(IEnumerable<T> sequence) {
var enumerator = sequence.GetEnumerator();
enumerator.MoveNext();
return (enumerator.Current, enumerator.AsEnumerable());
}
}
You also need to convert from an IEnumerator to an IEnumerable which I did with an extension method:
static class Extensions {
public static IEnumerable<T> AsEnumerable<T>(this IEnumerator<T> enumerator) {
while (enumerator.MoveNext()) {
yield return enumerator.Current;
}
}
}
Note that due to your requirements, iterating once over the remainder will exhaust it even though it has the type IEnumerable<T>.

There is also the possibility to do cheeky deconstruction like so:
var (x, xs) = new int?[] { 1, 2, 3 };
// x = 1, xs = 2, 3
All you need is to implement Deconstruct() on IEnumerable<T>. Based on the implementations from previous answers:
public static class Ext {
public static void Deconstruct<T>(this IEnumerable<T> source, out T? first, out IEnumerable<T> tail) {
using var e = source.GetEnumerator();
(first, tail) = e.MoveNext()
? (e.Current, e.AsEnumerable())
: (default, Enumerable.Empty<T>());
}
public static IEnumerable<T> AsEnumerable<T>(this IEnumerator<T> enumerator) {
while (enumerator.MoveNext()) yield return enumerator.Current;
}
}
If you feel particularly funny you can extend that approach to do things like this:
var (fst, snd, trd, rest) = new int?[] { 1, 2, 3, 4, 5 };
public static class Ext {
// ...
public static void Deconstruct<T>(this IEnumerable<T> source, out T? first, out T? second, out IEnumerable<T> tail) {
using var e = source.GetEnumerator();
if (e.MoveNext())
(first, (second, tail)) = (e.Current, e.AsEnumerable());
else
(first, second, tail) = (default, default, Enumerable.Empty<T>());
}
public static void Deconstruct<T>(this IEnumerable<T> source, out T? first, out T? second, out T? third, out IEnumerable<T> tail) {
using var e = source.GetEnumerator();
if (e.MoveNext())
(first, (second, third, tail)) = (e.Current, e.AsEnumerable());
else
(first, second, third, tail) = (default, default, default, Enumerable.Empty<T>());
}
}

As the answer of #corentin-pane has too many pending edits, I will share the extension methods I ended up implementing as a full new answer
public static class ExtraLinqExtensions
{
public static (T, IEnumerable<T>) GetFirstAndRemainder<T>(this IEnumerable<T> sequence)
{
using var enumerator = sequence.GetEnumerator();
return enumerator.MoveNext()
? (enumerator.Current, enumerator.AsEnumerable())
: (default, Enumerable.Empty<T>());
}
public static IEnumerable<T> AsEnumerable<T>(this IEnumerator<T> enumerator)
{
while (enumerator.MoveNext())
{
yield return enumerator.Current;
}
}
}
Where the most notable change is the check for whether the enumerator succeeded in doing a MoveNext().

Chaining fluent LINQ-like queries together

I wanted to build a fluent api to iterate on an array where I filter values and continue processing the remaining (not the filtered ones) values. Something like this pseudo-code:
int[] input = { 5, 4, 1, 3, 9, 8, 6, 7, 2, 0 };
from a in Take(3) // a = {5,4,1}
from b in Skip(4) // b = null
from c in TakeWhile(x=> x != 0) // c = {7, 2}
select new Stuff(a, b, c)
I don't know where to start looking, what are the basis for something like this. So I wanted to ask for some help.
The system should not be restricted to int numbers.. another example:
string[] input = { "how", "are", "you", "doing", "?" };
from a in OneOf("how", "what", "where") // a = "how"
from b in Match("are") // b = "are"
from c in TakeWhile(x=> x != "?") // c = { "you", "doing" }
select new Stuff(a, b, c)

The following code will allow you to do input.FirstTake(3).ThenSkip(4).ThenTakeWhile(x => x != 0); to get the sequence 5, 4, 1, 7, 2. The main idea is that you need to keep track of the takes and skips you want to do so they can be applied when you iterate. This is similar to how OrderBy and ThenBy work. Note that you cannot do other Linq operations in between. This build up one enumeration of consecutive skips and takes, then that sequence will be fed through any Linq operations you tack on.
public interface ITakeAndSkip<out T> : IEnumerable<T>
{
ITakeAndSkip<T> ThenSkip(int number);
ITakeAndSkip<T> ThenTake(int number);
ITakeAndSkip<T> ThenTakeWhile(Func<T, bool> predicate);
ITakeAndSkip<T> ThenSkipWhile(Func<T, bool> predicate);
}
public class TakeAndSkip<T> : ITakeAndSkip<T>
{
private readonly IEnumerable<T> _source;
private class TakeOrSkipOperation
{
public bool IsSkip { get; private set; }
public Func<T, bool> Predicate { get; private set; }
public int Number { get; private set; }
private TakeOrSkipOperation()
{
}
public static TakeOrSkipOperation Skip(int number)
{
return new TakeOrSkipOperation
{
IsSkip = true,
Number = number
};
}
public static TakeOrSkipOperation Take(int number)
{
return new TakeOrSkipOperation
{
Number = number
};
}
public static TakeOrSkipOperation SkipWhile(Func<T, bool> predicate)
{
return new TakeOrSkipOperation
{
IsSkip = true,
Predicate = predicate
};
}
public static TakeOrSkipOperation TakeWhile(Func<T, bool> predicate)
{
return new TakeOrSkipOperation
{
Predicate = predicate
};
}
}
private readonly List<TakeOrSkipOperation> _operations = new List<TakeOrSkipOperation>();
public TakeAndSkip(IEnumerable<T> source)
{
_source = source;
}
public IEnumerator<T> GetEnumerator()
{
using (var enumerator = _source.GetEnumerator())
{
// move to the first item and if there are none just return
if (!enumerator.MoveNext()) yield break;
// Then apply all the skip and take operations
foreach (var operation in _operations)
{
int n = operation.Number;
// If we are not dealing with a while then make the predicate count
// down the number to zero.
var predicate = operation.Predicate ?? (x => n-- > 0);
// Iterate the items until there are no more or the predicate is false
bool more = true;
while (more && predicate(enumerator.Current))
{
// If this is a Take then yield the current item.
if (!operation.IsSkip) yield return enumerator.Current;
more = enumerator.MoveNext();
}
// If there are no more items return
if (!more) yield break;
}
// Now we need to decide what to do with the rest of the items.
// If there are no operations or the last one was a skip then
// return the remaining items
if (_operations.Count == 0 || _operations.Last().IsSkip)
{
do
{
yield return enumerator.Current;
} while (enumerator.MoveNext());
}
// Otherwise the last operation was a take and we're done.
}
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
public ITakeAndSkip<T> ThenSkip(int number)
{
_operations.Add(TakeOrSkipOperation.Skip(number));
return this;
}
public ITakeAndSkip<T> ThenSkipWhile(Func<T, bool> predicate)
{
_operations.Add(TakeOrSkipOperation.SkipWhile(predicate));
return this;
}
public ITakeAndSkip<T> ThenTake(int number)
{
_operations.Add(TakeOrSkipOperation.Take(number));
return this;
}
public ITakeAndSkip<T> ThenTakeWhile(Func<T, bool> predicate)
{
_operations.Add(TakeOrSkipOperation.TakeWhile(predicate));
return this;
}
}
public static class TakeAndSkipExtensions
{
public static ITakeAndSkip<T> FirstTake<T>(this IEnumerable<T> source, int number)
{
return new TakeAndSkip<T>(source).ThenTake(number);
}
public static ITakeAndSkip<T> FirstSkip<T>(this IEnumerable<T> source, int number)
{
return new TakeAndSkip<T>(source).ThenSkip(number);
}
public static ITakeAndSkip<T> FirstTakeWhile<T>(this IEnumerable<T> source, Func<T, bool> predicate)
{
return new TakeAndSkip<T>(source).ThenTakeWhile(predicate);
}
public static ITakeAndSkip<T> FirstSkipWhile<T>(this IEnumerable<T> source, Func<T, bool> predicate)
{
return new TakeAndSkip<T>(source).ThenSkipWhile(predicate);
}
}

What is the std::partial_sum equivalent in C#?

In C#, what is the best equivalent of the C++ std::partial_sum?

The operation appears to be a weak version of combining map (aka Select) and reduce (aka Aggregate) operations, restricted to a binary operation. We can do better!
public static IEnumerable<R> MyAggregate<T, R>(
this IEnumerable<T> items,
R seed,
Func<T, R, R> mapper)
{
R current = seed;
foreach(T item in items)
{
current = mapper(item, current);
yield return current;
}
}
And now your desired function is just a special case:
static IEnumerable<int> PartialSum(this IEnumerable<int> items) =>
items.MyAggregate(0, (i, s) => s + i);
Commenter Tom Blodget points out that this requires that the sum operation have an identity; what if you don't have one? In that case you have to give up the ability to make the sum type different than the summand type:
public static IEnumerable<T> PartialSum<T>(
this IEnumerable<T> items,
Func<T, T, T> sum)
{
bool first = true;
T current = default(T);
foreach(T item in items)
{
current = first ? item : sum(current, item);
first = false;
yield return current;
}
}
And you can use it
myints.PartialSum((i, s) => i + s);

Try this:
int[] values = new[] { 1, 2, 3, 4 };
var result = values.Select ((x, index) => values.Take (index + 1).Sum ());
Result: 1, 3, 6, 10
But it is better to write own method if you care about performance.
Edit:
public static IEnumerable<T> MyAggregate<T> (this IEnumerable<T> items, Func<T, T, T> mapper)
{
bool processingFirstElement = true;
T accumulator = default (T);
foreach (T item in items)
{
if (processingFirstElement)
{
processingFirstElement = false;
accumulator = item;
}
else
{
accumulator = mapper (accumulator, item);
}
yield return accumulator;
}
}
And now:
int[] values = new int[] { 1, 2, 3, 4 };
var result = values.MyAggregate ((accumulator, item) => accumulator + item).ToList ();

How to build a sequence using a fluent interface?

I'm trying to using a fluent interface to build a collection, similar to this (simplified) example:
var a = StartWith(1).Add(2).Add(3).Add(4).ToArray();
/* a = int[] {1,2,3,4}; */
The best solution I can come up with add Add() as:
IEnumerable<T> Add<T>(this IEnumerable<T> coll, T item)
{
foreach(var t in coll) yield return t;
yield return item;
}
Which seems to add a lot of overhead that going to be repeated in each call.
IS there a better way?
UPDATE:
in my rush, I over-simplified the example, and left out an important requirement. The last item in the existing coll influences the next item. So, a slightly less simplified example:
var a = StartWith(1).Times10Plus(2).Times10Plus(3).Times10Plus(4).ToArray();
/* a = int[] {1,12,123,1234}; */
public static IEnumerable<T> StartWith<T>(T x)
{
yield return x;
}
static public IEnumerable<int> Times10Plus(this IEnumerable<int> coll, int item)
{
int last = 0;
foreach (var t in coll)
{
last = t;
yield return t;
}
yield return last * 10 + item;
}

A bit late to this party, but here are a couple ideas.
First, consider solving the more general problem:
public static IEnumerable<A> AggregateSequence<S, A>(
this IEnumerable<S> items,
A initial,
Func<A, R, A> f)
{
A accumulator = initial;
yield return accumulator;
foreach(S item in items)
{
accumulator = f(accumulator, item);
yield return accumulator;
}
}
And now your program is just new[]{2, 3, 4}.AggregateSequence(1,
(a, s) => a * 10 + s).ToArray()
However that lacks the "fluency" you want and it assumes that the same operation is applied to every element in the sequence.
You are right to note that deeply nested iterator blocks are problematic; they have quadratic performance in time and linear consumption of stack, both of which are bad.
Here's an entertaining way to implement your solution efficiently.
The problem is that you need both cheap access to the "right" end of the sequence, in order to do an operation on the most recently added element, but you also need cheap access to the left end of the sequence to enumerate it. Normal queues and stacks only have cheap access to one end.
Therefore: start by implementing an efficient immutable double-ended queue. This is a fascinating datatype; I have an implementation here using finger trees:
https://blogs.msdn.microsoft.com/ericlippert/2008/01/22/immutability-in-c-part-10-a-double-ended-queue/
https://blogs.msdn.microsoft.com/ericlippert/2008/02/12/immutability-in-c-part-eleven-a-working-double-ended-queue/
Once you have that, your operations are one-liners:
static IDeque<T> StartWith<T>(T t) => Deque<T>.Empty.EnqueueRight(t);
static IDeque<T> Op<T>(this IDeque<T> d, Func<T, T> f) => d.EnqueueRight(f(d.PeekRight()));
static IDeque<int> Times10Plus(this IDeque<int> d, int j) => d.Op(i => i * 10 + j);
Modify IDeque<T> and Deque<T> to implement IEnumerable<T> in the obvious way and you then get ToArray for free. Or do it as an extension method:
static IEnumerable<T> EnumerateFromLeft(this IDeque<T> d)
{
var c = d;
while (!c.IsEmpty)
{
yield return c.PeekLeft();
c = c.DequeueLeft();
}
}

You could do the following:
public static class MySequenceExtensions
{
public static IReadOnlyList<int> Times10Plus(
this IReadOnlyList<int> sequence,
int value) => Add(sequence,
value,
v => sequence[sequence.Count - 1] * 10 + v);
public static IReadOnlyList<T> Starts<T>(this T first)
=> new MySequence<T>(first);
public static IReadOnlyList<T> Add<T>(
this IReadOnlyList<T> sequence,
T item,
Func<T, T> func)
{
var mySequence = sequence as MySequence<T> ??
new MySequence<T>(sequence);
return mySequence.AddItem(item, func);
}
private class MySequence<T>: IReadOnlyList<T>
{
private readonly List<T> innerList;
public MySequence(T item)
{
innerList = new List<T>();
innerList.Add(item);
}
public MySequence(IEnumerable<T> items)
{
innerList = new List<T>(items);
}
public T this[int index] => innerList[index];
public int Count => innerList.Count;
public MySequence<T> AddItem(T item, Func<T, T> func)
{
Debug.Assert(innerList.Count > 0);
innerList.Add(func(item));
return this;
}
public IEnumerator<T> GetEnumerator() => innerList.GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
}
Note that I'm using IReadOnlyList to make it possible to index into the list in a performant way and be able to get the last element if needed. If you need a lazy enumeration then I think you are stuck with your original idea.
And sure enough, the following:
var a = 1.Starts().Times10Plus(2).Times10Plus(3).Times10Plus(4).ToArray();
Produces the expected result ({1, 12, 123, 1234}) with, what I think is, reasonable performance.

You can do like this:
public interface ISequence
{
ISequenceOp StartWith(int i);
}
public interface ISequenceOp
{
ISequenceOp Times10Plus(int i);
int[] ToArray();
}
public class Sequence : ISequence
{
public ISequenceOp StartWith(int i)
{
return new SequenceOp(i);
}
}
public class SequenceOp : ISequenceOp
{
public List<int> Sequence { get; set; }
public SequenceOp(int startValue)
{
Sequence = new List<int> { startValue };
}
public ISequenceOp Times10Plus(int i)
{
Sequence.Add(Sequence.Last() * 10 + i);
return this;
}
public int[] ToArray()
{
return Sequence.ToArray();
}
}
An then just:
var x = new Sequence();
var a = x.StartWith(1).Times10Plus(2).Times10Plus(3).Times10Plus(4).ToArray();

Replace, Insert, Delete operations on IEnumerable

I have a library that only accepts a proprietary immutable collection type. I would like to have a function that accepts one of these collections and performs some changes to this collection by returning a new collection that contains the changes made.
I would like to use a LINQ syntax instead of copying this collection to a List and back.
Add operations is easy for me: concat the enumerable with another one.
But what about Replace (at the given index, return the value given instead of that IEnumerable's value), Insert (at given index, return the given value and then continue iterating over the IEnumerable) or Delete (at given index, skip the IEnumerable's value)?
Are there functions like this available in the .NET framework or in another library? If not, how would I go about implementing these functions?

You can make your own extensions for these operations:
Add
public static IEnumerable<T> Add<T>(this IEnumerable<T> enumerable, T value)
{
foreach (var item in enumerable)
yield return item;
yield return value;
}
or:
public static IEnumerable<T> Add<T>(this IEnumerable<T> enumerable, T value)
{
return enumerable.Concat(new T[] { value });
}
Insert
public static IEnumerable<T> Insert<T>(this IEnumerable<T> enumerable, int index, T value)
{
int current = 0;
foreach (var item in enumerable)
{
if (current == index)
yield return value;
yield return item;
current++;
}
}
or
public static IEnumerable<T> Insert<T>(this IEnumerable<T> enumerable, int index, T value)
{
return enumerable.SelectMany((x, i) => index == i ? new T[] { value, x } : new T[] { x });
}
Replace
public static IEnumerable<T> Replace<T>(this IEnumerable<T> enumerable, int index, T value)
{
int current = 0;
foreach (var item in enumerable)
{
yield return current == index ? value : item;
current++;
}
}
or
public static IEnumerable<T> Replace<T>(this IEnumerable<T> enumerable, int index, T value)
{
return enumerable.Select((x, i) => index == i ? value : x);
}
Remove
public static IEnumerable<T> Remove<T>(this IEnumerable<T> enumerable, int index)
{
int current = 0;
foreach (var item in enumerable)
{
if (current != index)
yield return item;
current++;
}
}
or
public static IEnumerable<T> Remove<T>(this IEnumerable<T> enumerable, int index)
{
return enumerable.Where((x, i) => index != i);
}
Then you can make calls like this:
IEnumerable<int> collection = new int[] { 1, 2, 3, 4, 5 };
var added = collection.Add(6); // 1, 2, 3, 4, 5, 6
var inserted = collection.Insert(0, 0); // 0, 1, 2, 3, 4, 5
var replaced = collection.Replace(1, 22); // 1, 22, 3, 4, 5
var removed = collection.Remove(2); // 1, 2, 4, 5

The question is a little broad, so I'll demonstrate a possibility for a Replace method. There are no methods for that in the framework that replace something in an IEnumerable, as IEnumerable should represent an immutable sequence.
So a naive way to return a new IEnumerable with replaced elements:
public static class Extensions
{
public static IEnumerable<T> Replace<T>(this IEnumerable<T> source, T oldValue, T newValue)
{
return source.Select(element => element == oldValue ? newValue : element);
}
}
This will iterate through the source sequence and return the source elements except for those who Equal the oldValue. Note that this uses the == operator and how this works depends on the type argument for T.
Also note that this uses deferred execution. The source sequence is only enumerated when you start to enumerate the resulting IEnumerable. So if you change the source sequence after a call to Replace, the resulting sequence will yield this change, too.
Implementations for Insert and Delete are straight forward, too, though you'll need to count an index in the source sequence.

IEnumerable is by definition a immutable enumerable collection of elements of a given type. Immutable means that you cannot modify it directly and you always have to create a new instance.
You can however use the yield keyword to implement this behavior, preferably using extension methods.
For example replace could look like this:
public static IEnumerable<T> ReplaceAt<T>(this IEnumerable<T> collection, int index, T item)
{
var currentIndex = 0;
foreach (var originalItem in collection)
{
if (currentIndex != index)
{
//keep the original item in place
yield return originalItem;
}
else
{
//we reached the index where we want to replace
yield return item;
}
currentIndex++;
}
}

For a functional, more LINQ-ey feeling solution, inspired by Arturo's answer:
public static IEnumerable<T> Replace<T>(this IEnumerable<T> enumerable, Func<T, bool> selector, T value)
{
foreach (var item in enumerable)
{
yield return selector(item) ? value : item;
}
}
Used like this: var newEnumerable = oldEnumerable.Replace(x => x.Id == 1, newItem)