Develop Reference

LINQ Select only throws IOException when the Enumerable is looked at

I'm currently using LINQ to load a list of files into XDocuments, like so:
var fileNames = new List<string>(){ "C:\file.xml" };
var xDocs = fileNames.Select(XDocument.Load);
var xDocs2 = xDocs.ToList(); // Crashes here
If I deliberately 'lock' one of the files with a different process, the IOException is only thrown when I actually start to look at the XDocuments I've been generating, ie when ToList() is called.
Can anyone explain why this is, and how best to handle this error? I'd like to have access to the working XDocuments still, if possible.

Can anyone explain why this is
As many pointed out, this is because of the so called deferred execution of many LINQ methods. For instanse, the Enumerable.Select method documentation states
This method is implemented by using deferred execution. The immediate return value is an object that stores all the information that is required to perform the action. The query represented by this method is not executed until the object is enumerated either by calling its GetEnumerator method directly or by using foreach in Visual C# or For Each in Visual Basic.
while the Enumerable.ToList documentation contains
The ToList<TSource>(IEnumerable<TSource>) method forces immediate query evaluation and returns a List that contains the query results. You can append this method to your query in order to obtain a cached copy of the query results.
So the XDocument.Load is really executed for each file name during the ToList call. I guess that covers the why part.
and how best to handle this error? I'd like to have access to the working XDocuments still, if possible.
I don't know what does "best" mean in this context, but if you want to ignore the errors and include the "working XDocuments", then you can use something like this
var xDocs = fileNames.Select(fileName =>
{
try { return XDocument.Load(fileName); }
catch { return null; }
});
and then either append .Where(doc => doc != null), or account for null documents when processing the list.

This is why the linq .Select is an IEnumerable and the elements are first called if you make your IEnumerable to an List. Then you go through all of your elements.

Using LINQ First() method shows "Possible multiple iteration of IEnumerable" warning by Resharper

I have code like this:
Person firstPerson = personsEnumerable.First();
, where personsEnumerable is IEnumerable<Person>
Now, Resharper underlines the personsEnumerable variable and says "Possible iteration of IEnumerable". I understand what this warning means from other questions here on SO, but I'm wondering why it's showing it in my example? I think First() returns the first element and there's no need to iterate the collection at all ?
Is this a 'general' warning message which is not applicable in my case (and I can ignore it) or I don't understand how First() actually works ?

Yes, it returns the first element but it still creates an enumerator to get that first element.You could add ToArray or ToList after Where to prevent this. However if you want to just get the first element, you could use the overloaded version of First which takes a Func delegate and you can remove the Where.

Consider this code:
var personsEnumerable = peopleList.OrderBy(p => p.Age);
var firstPerson = personsEnumerable.First();
foreach (var p in personsEnumerable)
{
// whatever
}
The first line of code sets things up to sort the list and create the results, but it doesn't actually do anything until you try to enumerate it. The second line of code, then would end up doing the sort. So although you're not enumerating the entire result, you're executing code to produce the result. And for a sort, that would entail doing all the heavy lifting.
The foreach loop would do the sort again before actually enumerating the results.

Using Linq for copying/filtering produces a dynamic result in foreach loops?

So I create this projection of a dictionary of items I would like to remove.
var toRemoveList =
this.outputDic.Keys.Where(key =>
this.removeDic.ContainsKey(key));
Then I iterate through the result removing from the actual dictionary
foreach(var key in toRemoveList)
this.outputDic.Remove(key);
However during that foreach an exception is thrown saying that the list was modified during the loop. But, how so? is the linq query somewhat dynamic and gets re evaluated every time the dictionary changes? A simple .ToArray() call on the end of the query solves the issues, but imo, it shouldn't even occur in the first place.

So I create this projection of a dictionary of items I would like to remove.
var toRemoveList =
this.outputDic.Keys.Where(key =>
this.removeDic.ContainsKey(key));
As I have often said, if I can teach people one thing about LINQ it is this: the result of a query expression is a query, not the results of executing the query. You now have an object that means "the keys of a dictionary such that the key is... something". It is not the results of that query, it is that query. The query is an object unto itself; it does not give you a result set until you ask for one.
Then you do this:
foreach(var key in toRemoveList)
this.outputDic.Remove(key);
So what are you doing? You are iterating over the query. Iterating over the query executes the query, so the query is iterating over the original dictionary. But you then remove an item from the dictionary, while you are iterating over it, which is illegal.
imo, it shouldn't even occur in the first place.
Your opinion about how the world should be is a common one, but doing it your way leads to deep inefficiencies. Let us suppose that creating a query executes the query immediately rather than creates a query object. What does this do?
var query = expensiveRemoteDatabase
.Where(somefilter)
.Where(someotherfilter)
.OrderBy(something);
The first call to Where produces a query, which in your world is then executed, pulling down from the remote database all records which match that query. The second call to Where then says "oh, sorry, I meant to also apply this filter here as well, can we do that whole query again, this time with the second filter?" and so then that whole record set is computed, and then we say "oh, no, wait, I forgot to tell you when you built that last query object, we're going to need to sort it, so database, can you run this query for me a third time?"
Now perhaps do you see why queries produce a query that then does not execute until it needs to?

The reason is that toRemoveList does not contain a list of things to be removed, it contains a description of how to get a list of things that can be removed.
If you step through this in a debugger using F11 you can see this quite clearly for yourself. The first point it stops is with the cursor on foreach which is what you would expect.
Next you stop at toRemoveList (the one in foreach(var key in toRemoveList)). This is where it is setting up the iterator.
When you step through var key (with F11) however it now jumps into the original definition of toRemoveList, specifically the this.removeDic.ContainsKey(key) part. Now you get an idea of what is really happening.
The foreach is calling the iterators Next method to move to the next point in the dictionary's keys and holds onto the list. When you call into this.outputDic.Remove(key); this detects that the iterator hasn't finished and therefore stops you with this error.
As everybody is saying on here, the correct way to solve this is to use ToArray()/ToList() as what these do is to give you another copy of the list. So the you have one to step through, and one to remove from.

The .ToArray() solves the issues because it forces you to evaluate the entire enumeration and cache the local values. Without doing so, when you enumerate through it the enumerable attempts to calculate the first index, return that, then return to the collection and calculate the next index. If the underlying collection you're iterating over changes, you can no longer guarantee that the enumeration will return the appropriate value.
In short: just force the evaluation with .ToArray() (or .ToList(), or whatever).

The LINQ query uses deferred execution. It streams the items one by one, retruning them as they're requested. So yes, every time you try to remove a key it changes the result which is why it throws an exception. When you invoke ToArray() it forces execution of the query which is why it works.
EDIT: This is somewhat in response to your comments. Check out iterator blocks on msdn this is the mechanism being used when your for each executes. Your query just gets turned into an expression tree and the filter, projects, operation ect is applied to the elements one by one when they're retrieved unless it is not possible.

The reason you are getting this error is because of deferred execution of linq. To explain it fully when your loop runs is actually when the data is fetch from the dictionary. Thus modification in outputdic takes place at this point of time and it is not allowed to modify the collection you are looping upon. This is why you get this error. You can get rid of this error by asking the compiler to execute it before you run the loop.
var toRemoveList =
this.outputDic.Keys.Where(key =>
this.removeDic.ContainsKey(key)).ToList();
Notice the ToList() in the above statement. It will make sure that your query has been executed and you have your list in toRemoveList.

Count of System.Collections.Generic.List

ViewBag.EquipmentList = myInventoryEntities.p_Configurable_Equipment_Request_Select(Address_ID, false).Select(c => new { Value = c.Quantity + " " + c.Device_Name + " (s)", ID = c.Device_ID.ToString() }).ToList();
In Razor i want to do the following
#ViewBag.EquipmentList.Count
But Count is always == 1
I know i can iterate in a foreach but would rather a more direct approach.
Perhaps I am conceptually off?

EDIT: Okay, so now it seems like you've got past Count not executing:
But Count is always == 1
Sounds like you're always fetching a list with exactly one entry in. If that's unexpected, you should look at why you're only getting a single entry - start with debugging into the code... this doesn't sound like a Razor problem at all.
If your value were really a List<T> for some T, I believe it should be fine. The expression ViewBag.EquipmentList.Count would evaluate dynamically at every point.
In other words, if you're really, really using the assignment code shown, it should be okay.
If, however, the value is just some implementation of IEnumerable<T> which doesn't expose a Count property, then you'd need to use the Enumerable.Count() extension method - and you can't use normal "extension method syntax" with dynamic typing. One simple fix would be to use:
Enumerable.Count(ViewBag.EquipmentList)
... which will still use dynamic typing for the argument to Enumerable.Count, but it won't have to find the Count() method as an extension method.
Alternatively, make sure that your value really is a List<T>. If it's actually an array, you should be able to change the code to use the Length property instead.

#ViewBag.EquipmentListNI.Count()

What does baking a method means?

I was reading this post here on micro ORM used on SO.
The author showed this stack-trace:
System.Reflection.Emit.DynamicMethod.CreateDelegate
System.Data.Linq.SqlClient.ObjectReaderCompiler.Compile
System.Data.Linq.SqlClient.SqlProvider.GetReaderFactory
System.Data.Linq.SqlClient.SqlProvider.System.Data.Linq.Provider.IProvider.Compile
System.Data.Linq.CommonDataServices+DeferredSourceFactory`1.ExecuteKeyQuery
System.Data.Linq.CommonDataServices+DeferredSourceFactory`1.Execute
System.Linq.Enumerable.SingleOrDefault
System.Data.Linq.EntityRef`1.get_Entity
Then said:
In the trace above you can see that 'EntityRef' is baking a method, which is not a problem, unless it is happening 100s of times a second.
Could someone explain the stack-trace in relation to what he meant by "baking a method" and why it would be a performance problem?

When you do something like:
int[] items = whatever;
IEnumerable<int> query = from item in items where item % 2 == 0 select item;
then the compiler turns that into something like:
static bool Predicate(int item) { return item % 2 == 0; }
...
IEnumerable<int> query = Enumerable.Where<int>(items, new Func<int, bool> (Predicate));
That is, the compiler generates the IL for the method. When you build the query object at runtime, the object returned by Where holds on to a delegate to the predicate, and executes the predicate when necessary.
But it is possible to build the IL for the delegate at runtime if you want to. What you do is persist the body of the predicate as an expression tree. At runtime the expression tree can dynamically compile itself into brand-new IL; basically we start up a very simplified compiler that knows how to generate IL for expression trees. That way you can change the details of the predicate at runtime and recompile the predicate without having to recompile the whole program.
The author of that comment is using "baking" as a slang for "dynamic lightweight code generation".

It refers to creating a method dynamically at run time, for example by using an expression tree. Compiling the expression tree to a method and returning a delegate for the compiled method could be called "baking" the method.

It's a fancy way of saying that they're emitting dynamic code using Refrection.Emit. A notoriously slow process.