Develop Reference

IEnumerable<T>.Count() returns 0

I have a variable of type IEnumerable and if I call Count() method on it before foreach loop then it returns the correct count but after the foreach loop it returns 0. Why is that?
[UPDATE]
According to the given answers, I found out that my IEnumerable is kind of one-shot thing. So I've attached my code because I already converted it to a list and returned as IEnumerable. So where I'm doing it wrong?
public async Task<IEnumerable<WorkItem>> Get(int[] workItemIds)
{
return await context.WorkItems
.Where(it => workItemIds.Contains(it.Id))
.ToListAsync();
}
private async Task<int> ApproveOrRejectWorkItems(IEnumerable<WorkItem> workItems, int status)
{
// var workItemsToBeUpdated = workItems.Count();
workItems = workItems.Where(it => it.StatusId == (int)WorkItemStatus.Submitted);
foreach (var workItem in workItems)
{
workItem.StatusId = status;
}
// here value becomes 0
await _unitOfWork.WorkItemRepository.Update(workItems);
return workItems.Count();
}

Thank you for clarifying that you were filtering the list after materializing it in the Get() method.
Your issue is that LINQ is a view. So if you iterate an IEnumerable twice, it will go through the source items twice, applying any filters, projections, etc. to the source items. This means that by changing the source items, the enumerable will yield different items the second time you iterate through it because those items no longer match the filter.
I would suggest you modify the method to be like this:
private async Task<int> ApproveOrRejectWorkItems(IEnumerable<WorkItem> workItems, int status)
{
var workItemsToBeUpdated = workItems.Count();
foreach (var workItem in workItems.Where(it => it.StatusId == (int)WorkItemStatus.Submitted))
{
workItem.StatusId = status;
}
await _unitOfWork.WorkItemRepository.Update(workItems);
return workItems.Count();
}

You're most likely dealing with one of two scenarios:
Outside circumstances change between enumerations
Please consider the following scenario:
var badRecords = repository.GetBadRecords(); //returns IEnumerable<T>
if(badRecords.Count() > n)
{
repository.DeleteBadRecords();
}
foreach( var badRecords in badRecords )
// This enumeration goes to the db again and selects 0 records because we just deleted them.
{
Log(badRecord);
};
The solution is to .ToList() early;
Single use generator
This is similar but slightly different, we're dealing with code that by design allows just one iteration.
public static IEnumerable<int> GetAllNumbers(List<int> availableNumbers) {
while(availableNumbers.Count > 0)
{
var x = availableNumbers[0];
availableNumbers.RemoveAt(0);
yield return x;
}
}
static void Main(string[] args)
{
var numbers = GetAllNumbers(new List<int>{1,2,3});
Console.WriteLine(numbers.Count()); // 3
Console.WriteLine(numbers.Count()); // 0
numbers = GetAllNumbers(new List<int>{1,2,3}).ToList();
Console.WriteLine(numbers.Count()); // 3
Console.WriteLine(numbers.Count()); // 3
}
Resharper's Possible multiple enumeration of IEnumerable warning
BTW. Resharper has a dedicated warning for this: Code Inspection: Possible multiple enumeration of IEnumerable

EntityFramework : does IQueryable or IEnumerable get all the results at the first place?

I learned that IQueryable or IEnumerable datatypes do not return the results at the first place and only return them when needed. However when I open that object in the watch inspector I saw all the objects are there.
Is there anything wrong in my code or it just showing because I had call it on the watch ?
[When I view the pendings object in the watch dialogbox I saw all the list items but it shouldn't load at the first place. Is there anything wrong in my approaching or is it just showing because I had call it on the watch.]
public IQueryable<PurchasePendingView> PurchasePendings() {
var pendings = db.PurchasePendingViews
.Where(m => m.AccountStatusID != StructAccountStatus.Succeed); // when I view it in the watch dialougebox I saw all the list items but it shouldn't load at the first place. Is there anything wrong in my approaching or is it just showing because I had call it on the watch.
if (ApplicationEnvironment.DEBUGGING) {
return pendings;
} else if (IsMobileAccount()) {
var showroom = db.ShowRooms.FirstOrDefault(m=> m.MemberID == employee.MemberID);
if (showroom != null) {
return pendings.Where(m => m.ShowRoomID == showroom.ShowRoomID);
} else {
return pendings.Where(m => m.CountryDivisionID == employee.CountryDivisionID);
}
} else {
//normal salary employee can see every detail
return pendings;
}
}
Note: Currently my lazy loading is off.

The collections are evaluated the first time you iterate through the results.
Since you're iterating through the results in the watch inspector, they are evalauated then.

This is easier to demonstrate than to explain:
public class MeanException : Exception
{
public MeanException() : base() { }
public MeanException(string message) : base(message) { }
}
public static IEnumerable<T> Break<T>(this IEnumerable<T> source)
where T : new()
{
if (source != null)
{
throw new MeanException("Sequence was evaluated");
}
if (source == null)
{
throw new MeanException("Sequence was evaluated");
}
//unreachable
//this will make this an iterator block so that it will have differed execution,
//just like most other LINQ extension methods
yield return new T();
}
public static IEnumerable<int> getQuery()
{
var list = new List<int> { 1, 2, 3, 4, 5 };
var query = list.Select(n => n + 1)
.Break()
.Where(n => n % 2 == 0);
return query;
}
So, what do we have here. We have a custom exception so we can catch it independently. We have an extension method for IEnumerable<T> that will always throw an exception as soon as the sequence is evaluated, but it uses deferred execution, just like Select and Where. Finally we have a method to get a query. We can see a LINQ method both before and after the Break call, and we can see that a List is used as the underlying data source. (In your example it could be either some collection in memory, or an object that will go query a database and then iterate over the results when iterated.)
Now let's use this query and see what happens:
try
{
Console.WriteLine("Before fetching query");
IEnumerable<int> query = getQuery();
Console.WriteLine("After fetching query");
foreach (var number in query)
{
Console.WriteLine("Inside foreach loop");
}
Console.WriteLine("After foreach loop");
}
catch (MeanException ex)
{
Console.WriteLine("Exception thrown: \n{0}", ex.ToString());
}
What you'll see if you run this code is the print before the query (obviously) the print after the query (meaning we just returned the query from a method and the mean exception was never thrown) and next the message that the exception was thrown (meaning we never got inside of, or past the end of, the foreach loop.
This is obviously a bit of a contrived example to demonstrate a concept, but this is something you'll actually see in practice often enough. For example, if you lose your connection with the database after creating your data context you won't actually get an exception until you iterate the query, or if your data holder objects are out of date and don't match the DB any longer you'll get an exception at the same point. In a less obvious example, if you hold onto the query for an extended period of time you will end up with data at the time you fetch the results of the query, not when you build it. Here is another demonstration of that:
var list = new List<int> { 1, 2, 3, 4, 5 };
var query = list.Where(num => num < 5);
Console.WriteLine(query.Count());
list.RemoveAll(num => num < 4);
Console.WriteLine(query.Count()
Here we have a list of data, and count the number of items less than 5 (it's 4). Then we modify the list (without changing query at all). We re-query query and end up with a count of 1.

How to yield return inside anonymous methods?

Basically I have an anonymous method that I use for my BackgroundWorker:
worker.DoWork += ( sender, e ) =>
{
foreach ( var effect in GlobalGraph.Effects )
{
// Returns EffectResult
yield return image.Apply (effect);
}
};
When I do this the compiler tells me:
"The yield statement cannot be used
inside an anonymous method or lambda
expression"
So in this case, what's the most elegant way to do this? Btw this DoWork method is inside a static method, in case that matters for the solution.

Unfortunately you can't.
The compiler does not allow you to combine the two "magic" pieces of code. Both involve rewriting your code to support what you want to do:
An anonymous method is done by moving the code to a proper method, and lifting local variables to fields on the class with that method
An iterator method is rewritten as a state machine
You can, however, rewrite the code to return the collection, so in your particular case I would do this:
worker.DoWork += ( sender, e ) =>
{
return GlobalGraph.Effects
.Select(effect => image.Apply(effect));
};
though it looks odd for an event (sender, e) to return anything at all. Are you sure you're showing a real scenario for us?
Edit Ok, I think I see what you're trying to do here.
You have a static method call, and then you want to execute code in the background, and then return data from that static method once the background call completes.
This is, while possible, not a good solution since you're effectively pausing one thread to wait for another, that was started directly before you paused the thread. In other words, all you're doing is adding overhead of context switching.
Instead you need to just kick off the background work, and then when that work is completed, process the resulting data.

Perhaps just return the linq expression and defer execution like yield:
return GlobalGraph.Effects.Select(x => image.Apply(x));

Unless I'm missing something, you can't do what you're asking.
(I do have an answer for you, so please read past my explanation of why you can't do what you're doing first, and then read on.)
You full method would look something like this:
public static IEnumerable<EffectResult> GetSomeValues()
{
// code to set up worker etc
worker.DoWork += ( sender, e ) =>
{
foreach ( var effect in GlobalGraph.Effects )
{
// Returns EffectResult
yield return image.Apply (effect);
}
};
}
If we assume that your code was "legal" then when GetSomeValues is called, even though the DoWork handler is added to worker, the lambda expression isn't executed until the DoWork event is fired. So the call to GetSomeValues completes without returning any results and the lamdba may or may not get called at a later stage - which is then too late for the caller of the GetSomeValues method anyway.
Your best answer is to the use Rx.
Rx turns IEnumerable<T> on its head. Instead of requesting values from an enumerable, Rx has values pushed to you from an IObservable<T>.
Since you're using a background worker and responding to an event you are effectively having the values pushed to you already. With Rx it becomes easy to do what you're trying to do.
You have a couple of options. Probably the simplest is to do this:
public static IObservable<IEnumerable<EffectResult>> GetSomeValues()
{
// code to set up worker etc
return from e in Observable.FromEvent<DoWorkEventArgs>(worker, "DoWork")
select (
from effect in GlobalGraph.Effects
select image.Apply(effect)
);
}
Now callers of your GetSomeValues method would do this:
GetSomeValues().Subscribe(ers =>
{
foreach (var er in ers)
{
// process each er
}
});
If you know that DoWork is only going to fire once, then this approach might be a little better:
public static IObservable<EffectResult> GetSomeValues()
{
// code to set up worker etc
return Observable
.FromEvent<DoWorkEventArgs>(worker, "DoWork")
.Take(1)
.Select(effect => from effect in GlobalGraph.Effects.ToObservable()
select image.Apply(effect))
.Switch();
}
This code looks a little more complicated, but it just turns a single do work event into a stream of EffectResult objects.
Then the calling code looks like this:
GetSomeValues().Subscribe(er =>
{
// process each er
});
Rx can even be used to replace the background worker. This might be the best option for you:
public static IObservable<EffectResult> GetSomeValues()
{
// set up code etc
return Observable
.Start(() => from effect in GlobalGraph.Effects.ToObservable()
select image.Apply(effect), Scheduler.ThreadPool)
.Switch();
}
The calling code is the same as the previous example. The Scheduler.ThreadPool tells Rx how to "schedule" the processing of subscriptions to the observer.
I hope this helps.

For new readers: the most elegant way to implement 'anonymous iterators' (i. e. nested in other methods) in C#5 is probably something like this cool trick with async/await (don't be confused by these keywords, the code below is computed absolutely synchronously - see details in the linked page):
public IEnumerable<int> Numbers()
{
return EnumeratorMonad.Build<int>(async Yield =>
{
await Yield(11);
await Yield(22);
await Yield(33);
});
}
[Microsoft.VisualStudio.TestTools.UnitTesting.TestMethod]
public void TestEnum()
{
var v = Numbers();
var e = v.GetEnumerator();
int[] expected = { 11, 22, 33 };
Numbers().Should().ContainInOrder(expected);
}
C#7 (available now in Visual Studio 15 Preview) supports local functions, which allow yield return:
public IEnumerable<T> Filter<T>(IEnumerable<T> source, Func<T, bool> filter)
{
if (source == null) throw new ArgumentNullException(nameof(source));
if (filter == null) throw new ArgumentNullException(nameof(filter));
return Iterator();
IEnumerable<T> Iterator()
{
foreach (var element in source)
{
if (filter(element)) { yield return element; }
}
}
}

DoWork is of type DoWorkEventHandler which returns nothing (void),
so it's not possible at all in your case.

The worker should set the Result property of DoWorkEventArgs.
worker.DoWork += (s, e) => e.Result = GlobalGraph.Effects.Select(x => image.Apply(x));

Ok so I did something like this which does what I wanted (some variables omitted):
public static void Run ( Action<float, EffectResult> action )
{
worker.DoWork += ( sender, e ) =>
{
foreach ( var effect in GlobalGraph.Effects )
{
var result = image.Apply (effect);
action (100 * ( index / count ), result );
}
}
};
and then in the call site:
GlobalGraph.Run ( ( p, r ) =>
{
this.Progress = p;
this.EffectResults.Add ( r );
} );

I wanted to supplement user1414213562's answer with an implementation of the ForEachMonad.
static class ForEachMonad
{
public static IEnumerable<A> Lift<A>(A a) { yield return a; }
// Unfortunately, this doesn't compile
// public static Func<IEnumerable<A>, IEnumerable<B>> Lift<A, B>(Func<A, IEnumerable<B>> f) =>
// (IEnumerable<A> ea) => { foreach (var a in ea) { foreach (var b in f(a)) { yield return b; } } }
// Fortunately, this does :)
public static Func<IEnumerable<A>, IEnumerable<B>> Lift<A, B>(Func<A, IEnumerable<B>> f)
{
IEnumerable<B> lift(IEnumerable<A> ea)
{
foreach (var a in ea) { foreach (var b in f(a)) { yield return b; } }
}
return lift;
}
public static void Demo()
{
var f = (int x) => (IEnumerable<int>)new int[] { x + 1, x + 2, x + 3 };
var g = (int x) => (IEnumerable<double>)new double[] { Math.Sqrt(x), x*x };
var log = (double d) => { Console.WriteLine(d); return Lift(d); };
var e1 = Lift(0);
var e2 = Lift(f)(e1);
var e3 = Lift(g)(e2);
// we call ToArray in order to materialize the IEnumerable
Lift(log)(e3).ToArray();
}
}
Running ForEachMonad.Demo() produces the following output:
1
1
1,4142135623730951
4
1,7320508075688772
9

Elegantly refactoring code like this (to avoid a flag)

I have a function running over an enumerable, but the function should be a little bit different for the first item, for example:
void start() {
List<string> a = ...
a.ForEach(DoWork);
}
bool isFirst = true;
private void DoWork(string s) {
// do something
if(isFirst)
isFirst = false;
else
print("first stuff");
// do something
}
How would you refactor this to avoid that ugly flag?

Expounding on Jimmy Hoffa's answer if you actually want to do something with the first item you could do this.
DoFirstWork(a[0])
a.Skip(1).ForEach(DoWork)
If the point is that it is separate in logic from the rest of the list then you should use a separate function.

It might be a bit heavy handed, but I pulled this from another SO question a while back.
public static void IterateWithSpecialFirst<T>(this IEnumerable<T> source,
Action<T> firstAction,
Action<T> subsequentActions)
{
using (IEnumerator<T> iterator = source.GetEnumerator())
{
if (iterator.MoveNext())
{
firstAction(iterator.Current);
}
while (iterator.MoveNext())
{
subsequentActions(iterator.Current);
}
}
}

Check out Jon Skeet's smart enumerations.
They are part of his Miscellaneous Utility Library

EDIT: added usage example, added a ForFirst method, reordered my paragraphs.
Below is a complete solution.
Usage is either of the following:
list.ForFirst(DoWorkForFirst).ForRemainder(DoWork);
// or
list.ForNext(1, DoWorkForFirst).ForRemainder(DoWork);
The crux is the ForNext method, which performs an action for the specified next set of items from the collection and returns the remaining items. I've also implemented a ForFirst method that simply calls ForNext with count: 1.
class Program
{
static void Main(string[] args)
{
List<string> list = new List<string>();
// ...
list.ForNext(1, DoWorkForFirst).ForRemainder(DoWork);
}
static void DoWorkForFirst(string s)
{
// do work for first item
}
static void DoWork(string s)
{
// do work for remaining items
}
}
public static class EnumerableExtensions
{
public static IEnumerable<T> ForFirst<T>(this IEnumerable<T> enumerable, Action<T> action)
{
return enumerable.ForNext(1, action);
}
public static IEnumerable<T> ForNext<T>(this IEnumerable<T> enumerable, int count, Action<T> action)
{
if (enumerable == null)
throw new ArgumentNullException("enumerable");
using (var enumerator = enumerable.GetEnumerator())
{
// perform the action for the first <count> items of the collection
while (count > 0)
{
if (!enumerator.MoveNext())
throw new ArgumentOutOfRangeException(string.Format(System.Globalization.CultureInfo.InvariantCulture, "Unexpected end of collection reached. Expected {0} more items in the collection.", count));
action(enumerator.Current);
count--;
}
// return the remainder of the collection via an iterator
while (enumerator.MoveNext())
{
yield return enumerator.Current;
}
}
}
public static void ForRemainder<T>(this IEnumerable<T> enumerable, Action<T> action)
{
if (enumerable == null)
throw new ArgumentNullException("enumerable");
foreach (var item in enumerable)
{
action(item);
}
}
}
I felt a bit ridiculous making the ForRemainder method; I could swear that I was re-implementing a built-in function with that, but it wasn't coming to mind and I couldn't find an equivalent after glancing around a bit. UPDATE: After reading the other answers, I see there apparently isn't an equivalent built into Linq. I don't feel so bad now.

using System.Linq; // reference to System.Core.dll
List<string> list = ..
list.Skip(1).ForEach(DoWork) // if you use List<T>.ForEeach()
but I recommend you to write your one:
public static void ForEach(this IEnumerable<T> collection, Action<T> action)
{
foreach(T item in collection)
action(item);
}
So you could do just next:
list.Skip(1).ForEach(DoWork)

It's hard to say what the "best" way to handle the first element differently is without knowing why it needs to be handled differently.
If you're feeding the elements of the sequence into the framework's ForEach method, you can't elegantly provide the Action delegate the information necessary for it to determine the element parameter's position in the source sequence, so I think an extra step is necessary. If you don't need to do anything with the sequence after you loop through it, you could always use a Queue (or Stack), pass the first element to whatever handler you're using through a Dequeue() (or Pop()) method call, and then you have the leftover "homogeneous" sequence.

It might seem rudimentary with all the shiny Linq stuff available, but there's always the old fashion for loop.
var yourList = new List<int>{1,1,2,3,5,8,13,21};
for(int i = 0; i < yourList.Count; i++)
{
if(i == 0)
DoFirstElementStuff(yourList[i]);
else
DoNonFirstElementStuff(yourList[i]);
}
This would be fine if you don't want to alter yourList inside the loop. Else, you'll probably need to use the iterator explicitly. At that point, you have to wonder if that's really worth it just to get rid of an IsFirst flag.

Depends on how you're "handling it differently". If you need to do something completely different, then I'd recommend handling the first element outside the loop. If you need to do something in addition to the regular element processing, then consider having a check for the result of the additional processing. It's probably easier to understand in code, so here's some:
string randomState = null; // My alma mater!
foreach(var ele in someEnumerable) {
if(randomState == null) randomState = setState(ele);
// handle additional processing here.
}
This way, your "flag" is really an external variable you (presumably) need anyway, so you're not creating a dedicated variable. You can also wrap it in an if/else if you don't want to process the first element like the rest of the enumeration.

First Time Calling Extension Methods is Slower Than Subsequent Calls

I have a class that modifies data via some extension methods. In order to debug performance, I have created some rough debug code to make multiple calls to the same methods using the same data a number of times. I am finding that it consistently takes a significantly longer time to do the calculations the first time through the loop than subsequent calls.
For example, for a small set of data, it appears to take about 5 seconds to do the calculations, while each subsequent call is a second or so.
Thanks,
wTs
The code looks something like this:
Test Code
void TestCode()
{
for (int i = 0; i < iterationsPerLoop; i++)
{
DateTime startTime = DateTime.Now;
// The test is actually being done in a BackgroundWorker
dispatcher.Invoke(DispatcherPriority.Normal,
(Action)(() => this.PropertyCausingCodeToRun = "Run";
while (this.WaitForSomeCondition)
Thread.Sleep(125);
DateTime endTime = DateTime.Now;
double result = endTime.Subtract(startTime).TotalSeconds;
}
}
Method where extension methods called
private static List<ObservableItem> GetAvailableItems(MyObject myObject)
{
var items = new List<ObservableItem>(myObject.Items.ToList());
var selectedItems = items.OrderByDescending(item => item.ASortableProperty)
.SetItemIsAvailable(false)
.SetItemPriority()
.OrderByDescending(item => item.Priority)
.Where(item => item.Priority > 0)
.SetItemIsAvailable(true)
.OrderBy(item => item.Date);
return selectedItems.ToList();
}
Extension Methods (ObservableItems all created on different thread)
static class MyExtensionMethods
{
public static IEnumerable<T> SetItemIsAvailable<T>(this IEnumerable<T> sourceList,
Boolean isAvailable) where T : ObservableItem
{
Action<T> setAvailable = i => i.IsAvailable = isAvailable;
List<DispatcherOperation> invokeResults = new List<DispatcherOperation>();
foreach (var item in sourceList)
{
invokeResults.Add(
item.ItemDispatcher.BeginInvoke(setAvailable , new object[] { item }));
}
invokeResults.ForEach(ir => ir.Wait());
return sourceList;
}
public static IEnumerable<T> SetItemPriority<T>(this IEnumerable<T> sourceList) where T : ObservableItem
{
Action<T, double> setPriority = new Action<T, double>((item, priority) =>
{
item.Priority = priority;
});
List<DispatcherOperation> invokeResults = new List<DispatcherOperation>();
foreach (var item in sourceList)
{
double priority = ......; // Some set of calculations
invokeResults.Add(
item.ItemDispatcher.BeginInvoke(setPriority,
new object[] { asset, priority }));
}
invokeResults.ForEach(ir => ir.Wait());
return sourceList;
}
}

Most often, the first time methods are called, there is some overhead associated with the JIT compilation time. This will have an effect (though most likely not that much).
However, looking at your code, you're spending a huge amount of time waiting on asynchronous calls marshalling to the UI via the dispatcher. This is going to put a large hit on your overall performance, and slow this way down.
I'd recommend doing all of your operations in one dispatch call, and using Invoke instead of BeginInvoke. Instead of marshalling one message per item, just marshal a single delegate that includes the foreach loop through your items.
This will be significantly faster.

The real issue, as I've figured out, was caused by the property initially being called to sort the items (before the extension methods are even called).
The property is of the form:
public Double ASortableProperty
{
get
{
if (mASortableProperty.HasValue)
{
return mASortableProperty.Value;
}
mASortableProperty = this.TryGetDoubleValue(...);
return (mASortableProperty.HasValue ? mASortableProperty.Value : 0);
}
}
Therefore, the first time through the loop, the values were not initialized from the database, and the cost was in retrieving these values, before the sort can take place.