Develop Reference

Linq .Where() function always fail

I am having an issue with Linq at the moment. We are using the Telerik controls in a WPF application in C#. I have am "array" of clients. The type of array is a QueryableEntityCoreCollectionView, which is from Telerik. This is the source of a gridview.
I now need to make a property which will return a subset of these clients. No matter what conditions I put in my Where clause, the Where fails and gives me the classic Object is not set to an Instance blahblahblah. (the grid is able to get a IEnumerable, so I don't have to convert back into a QueryableEntityCoreCollectionView at the end.
Here is a bit of code I am trying to play around with:
public IEnumerable<Client> FilteredClients {
get {
IEnumerable<Client> res1 = clients.AsEnumerable<Client>();
IEnumerable<Client> res2 = res1.Where(T => 1 == 1);
return res2;
}
}
In the above example, res1 contains 11 elements. res2 does not work. It goes over it, I don't receive an actual exception, but if I put a breakpoint on the return, and check out both res1 and res2, I can see res1 is okay, res2 when I open it's result view, I get the message "Object reference not set to an instance of an object." with the stack trace:
at System.Linq.Enumerable.WhereEnumerableIterator`1.ToArray()
at System.Linq.SystemCore_EnumerableDebugView`1.get_Items()
Given the search criteria "1==1", I would expect everything to come out of it... But it fails. Even with a real search, like T => T.City == T.City, it also fails the same way. No matter what I give to Where(), it fails. If I simply return res1 here, my full list passes back to my datagrid as an IEnumerable, and everything works fine. As soon as I try to filter on anything, it fails.
Any ideas?
edit:
After Hazrelle comment, I tried calling .ToList() ... But everytime I try to do that, I get a Null Exception.
clients.ToList<Client>(); // fails
clients.AsEnumerable<Client>().ToList<Client>(); // fails
Everything I try fails, and I don't know why...
Edit 2: Still not able to filter or anything... However I got the following line of code to pass:
var query = clients.SourceCollection.ToEnumerable().First();
However, query is not a Client, but rather a Telerik.Windows.Data.QueryableProxy , and is still seen as a collection with 11 items. It is NOT just the first Client.

Seems like this QueryableEntityCoreCollectionView object is either bugged, or we are not supposed to play with it, and should only be handled internally by Telerik stuff, but I have resolved the problem by never creating this object, I create a good ol' List of Client directly and give this to the grid instead. Linq works like a charm then...
For some reason, no amount of AsEnumerable, ToList, AsWhatever I did to a QueryableEntityCoreCollectionView(of Client), it always became a QueryableEntityView(of Client) at best, never actually became a simple List(of Client).
Strange.
It is very possible I am loosing the ability to update the database by doing that, However, at this point, this is not data that is meant to be updateable, at least not from that window.

How does this NHibernate code influence the commit?

I ran into an issue where the following code results in an exception.
public IList<Persoon> GetPersonenWithCurrentWorkScheme(int clientId)
{
//SELECT N+1
Session.QueryOver<Werkschema>()
.Fetch(ws => ws.Labels).Eager
.JoinQueryOver<WerkschemaHistory>(p => p.Histories)
.JoinQueryOver<ArbeidsContract>(a => a.ArbeidsContract)
.JoinQueryOver<Persoon>(p => p.Persoon)
.Where(p => p.Klant.ID == clientId)
.Future();
var result = Session.QueryOver<Persoon>()
.Fetch(p => p.ArbeidsContracten).Eager
.Where(p => p.Klant.ID == clientId)
.Future();
return result.ToList();
}
When I comment the first part of the method (Session.QueryOver WerkSchema...) the code runs fine. When it is not commented, the first NHibernate.Commit() that occurs throws an exception. (Something with a date-time conversion but that's not really what I'm worried about).
My question: Is the first bit of code useful? Does it do anything? The result is not stored in a variable which is used later, so to me it looks like dead code. Or is this some NHibernate dark magic which actually does something useful?

The Future is an optimization over existing API provided by NHibernate.
One of the nicest new features in NHibernate 2.1 is the Future() and FutureValue() functions. They essentially function as a way to defer query execution to a later date, at which point NHibernate will have more information about what the application is supposed to do, and optimize for it accordingly. This build on an existing feature of NHibernate, Multi Queries, but does so in a way that is easy to use and almost seamless.
This will execute multiple queries in single round trip to database. If it is not possible to get all the needed data in single round trip, multiple calls will be executed as expected; but it still helps in many cases.
The database call is triggered when first round trip is requested.
In your case, first round trip is requested by calling result.ToList(), which also including your first part of code.
As you suspect, output of first part; though retrieved is never being used. So in my view, that part can be safely commented. But, this is based on code you post in question only.
It may be the case that the data loaded while this call is saving round-trips in other part of code. But in that case, code should be optimized and should be moved to proper location.

When is data really retrieved from database using EF Core?

To retrieve data, first I wrote LINQ query, that I expect not to executed on database until I call the FirstAsync() method.
var query =
from tkn in Db.Set<TableA>()
where tkn.IsActive == true
where tkn.Token == token
select tkn.RelatedObjectMappedToTableB;
var retrievedObject = await clientQuery.FirstAsync();
The problem is while debugging I can see the values of the related object in the Watch of visual studio, before reaching the call to FirstAsync()
This means to me that database is already queried and EF has not waited until I ask it to do so.
Why is it doing so? Is my opinion about when a linq query is executed wrong?

This means to me that database is already queried and ef has not waited until I ask it to do so.
No, the database is being queried exactly because you asked it to do so.
Is my opinion about when a linq query is executed wrong?
No, it is not, however,
Why is it doing so?
LINQ retrieves the values in a lazy way, which means that it waits until you perform some enumeration on it. Using Visual Studio's Watch Window is asking the query to be evaluated so you can see the actual details.

LINQ support two execution behaviors deferred execution and immediate execution. Deferred execution means that the evaluation of an expression is delayed until its realized value is actually required. It greatly improves performance by avoiding unnecessary execution.
Deferred execution is applicable to any in-memory collection as well as remote LINQ providers like LINQ-to-SQL, LINQ-to-Entities, LINQ-to-XML, etc.
In the provided example in question, it seems that query executed at where but that's not happening actually. The query executed when you call
.FirstAsync(), It means you really need data and C# execute the query, extract data and store data into memory.
If you want immediate execution behavior you can use .ToList() it will extract the complete projected data and save into memory.
var result =(from tkn in Db.Set<TableA>()
where tkn.IsActive == true
& tkn.Token == token
select tkn.RelatedObjectMappedToTableB).ToList();
For more details see article

Using Linq for copying/filtering produces a dynamic result in foreach loops?

So I create this projection of a dictionary of items I would like to remove.
var toRemoveList =
this.outputDic.Keys.Where(key =>
this.removeDic.ContainsKey(key));
Then I iterate through the result removing from the actual dictionary
foreach(var key in toRemoveList)
this.outputDic.Remove(key);
However during that foreach an exception is thrown saying that the list was modified during the loop. But, how so? is the linq query somewhat dynamic and gets re evaluated every time the dictionary changes? A simple .ToArray() call on the end of the query solves the issues, but imo, it shouldn't even occur in the first place.

So I create this projection of a dictionary of items I would like to remove.
var toRemoveList =
this.outputDic.Keys.Where(key =>
this.removeDic.ContainsKey(key));
As I have often said, if I can teach people one thing about LINQ it is this: the result of a query expression is a query, not the results of executing the query. You now have an object that means "the keys of a dictionary such that the key is... something". It is not the results of that query, it is that query. The query is an object unto itself; it does not give you a result set until you ask for one.
Then you do this:
foreach(var key in toRemoveList)
this.outputDic.Remove(key);
So what are you doing? You are iterating over the query. Iterating over the query executes the query, so the query is iterating over the original dictionary. But you then remove an item from the dictionary, while you are iterating over it, which is illegal.
imo, it shouldn't even occur in the first place.
Your opinion about how the world should be is a common one, but doing it your way leads to deep inefficiencies. Let us suppose that creating a query executes the query immediately rather than creates a query object. What does this do?
var query = expensiveRemoteDatabase
.Where(somefilter)
.Where(someotherfilter)
.OrderBy(something);
The first call to Where produces a query, which in your world is then executed, pulling down from the remote database all records which match that query. The second call to Where then says "oh, sorry, I meant to also apply this filter here as well, can we do that whole query again, this time with the second filter?" and so then that whole record set is computed, and then we say "oh, no, wait, I forgot to tell you when you built that last query object, we're going to need to sort it, so database, can you run this query for me a third time?"
Now perhaps do you see why queries produce a query that then does not execute until it needs to?

The reason is that toRemoveList does not contain a list of things to be removed, it contains a description of how to get a list of things that can be removed.
If you step through this in a debugger using F11 you can see this quite clearly for yourself. The first point it stops is with the cursor on foreach which is what you would expect.
Next you stop at toRemoveList (the one in foreach(var key in toRemoveList)). This is where it is setting up the iterator.
When you step through var key (with F11) however it now jumps into the original definition of toRemoveList, specifically the this.removeDic.ContainsKey(key) part. Now you get an idea of what is really happening.
The foreach is calling the iterators Next method to move to the next point in the dictionary's keys and holds onto the list. When you call into this.outputDic.Remove(key); this detects that the iterator hasn't finished and therefore stops you with this error.
As everybody is saying on here, the correct way to solve this is to use ToArray()/ToList() as what these do is to give you another copy of the list. So the you have one to step through, and one to remove from.

The .ToArray() solves the issues because it forces you to evaluate the entire enumeration and cache the local values. Without doing so, when you enumerate through it the enumerable attempts to calculate the first index, return that, then return to the collection and calculate the next index. If the underlying collection you're iterating over changes, you can no longer guarantee that the enumeration will return the appropriate value.
In short: just force the evaluation with .ToArray() (or .ToList(), or whatever).

The LINQ query uses deferred execution. It streams the items one by one, retruning them as they're requested. So yes, every time you try to remove a key it changes the result which is why it throws an exception. When you invoke ToArray() it forces execution of the query which is why it works.
EDIT: This is somewhat in response to your comments. Check out iterator blocks on msdn this is the mechanism being used when your for each executes. Your query just gets turned into an expression tree and the filter, projects, operation ect is applied to the elements one by one when they're retrieved unless it is not possible.

The reason you are getting this error is because of deferred execution of linq. To explain it fully when your loop runs is actually when the data is fetch from the dictionary. Thus modification in outputdic takes place at this point of time and it is not allowed to modify the collection you are looping upon. This is why you get this error. You can get rid of this error by asking the compiler to execute it before you run the loop.
var toRemoveList =
this.outputDic.Keys.Where(key =>
this.removeDic.ContainsKey(key)).ToList();
Notice the ToList() in the above statement. It will make sure that your query has been executed and you have your list in toRemoveList.

Debugging LINQ - Children could not Be evaluated

I'm trying to understand why I can't inspect a returned value from a LINQ query when on a breakpoint. Expanding the results view simply says "Children Could Not Be Evaluated".
On the other hand enumerating with a foreach in code or using a ToList does let the collection be inspected in the debugger.
I would have thought, as it does it most other scenarios, expanding results in the debugger is equivalent to a ToList on the collection which is why I'm expecting it to work. The only thing that is a little different is that I'm calling from an EXE into a DLL, the DLL being where the objects are defined and the initial query built and returned. But I can't see it being that.
var timeboxes = assetRepo.ActiveTimeboxes();
// This can't be evaluated in the debugger
var stories = timeboxes.SelectMany(c => assetRepo.AllStories(c));
// This can be inspected in the debugger
var executedStories = stories.ToList();

It's not possible to debug from VS, but I there is LINQPad that could sometimes help you.
Your best option is to split your query into small statements.