Develop Reference

Use ToList() in Linq and Exe Time

Below is an example from my textbook:
var allGenres = from genre in myEntities.Genres.Include("Reviews")
orderby genre.Name
select new { genre.Name, genre.Reviews };
Repeater1.DataSource = allGenres.ToList();
Repeater1.DataBind();
and the book says:
as soon as you call ToList(), the query is executed and the relevant genres and reviews are retrieved from the database and assigned to the DataSource property
so my question is, if I get rid of Repeater1.DataSource = allGenres.ToList();, what does var allGenres contain? since the query hasn't been executed?

There are three stages to understand.
First, when the query is created.
Second, when the query variable is iterated over (deferred execution).
Third, forcing a query for immediate results.
var allGenres = from genre in myEntities.Genres.Include("Reviews")
orderby genre.Name
select new { genre.Name, genre.Reviews };
In this code, the query is only created. Its dead as a person in a cemetery.
If you need deferred execution, then you can iterate over the results using for loop etc.
To force immediate execution, you can use conversion operators like
ToList, ToArray, ToLookup, and ToDictionary.
Hope it helps.
You can put a breakpoint on this line:
var allGenres = from genre in myEntities.Genres.Include("Reviews")
orderby genre.Name
select new { genre.Name, genre.Reviews };
and notice nothing happens. After the following line, voila SQL Profiler will display the sql query happening:
Repeater1.DataSource = allGenres.ToList();

Your object allGenres is an object that implements IQueryable<...>. This means that it represents a sequence, and it has functions to get the first element of the sequence, and once you've got the element you can get the next one, until there are no more elements.
The <...> part defines what kind of elements are in the sequence. So IQueryable<Book> says that you can query for a sequence of Books, which you can enumerate one after another. Every element of the sequence will be an object of class Book
This enumeration is provided using a base interface of IQueryable, namely IEnumerable<Book>.
At the lowest level, this Enumeration is done as follows:
IEnumerable<Book> books = ... // for example new List<Book>(), or new Book[10]
IEnumerator<Book> bookEnumerator = books.Getenumerator();
// as long as there are books, print the title:
// the first MoveNext() moves to the first element
// every other MoveNext() move to the next element
// it returns false if there is no such element (no first, or no next)
while (bookEnumerator.MoveNext())
{
// the enumerator points to the next element. Property Current contains this element
Book book = bookEnumerator.Current;
Console.WriteLine(book.Title);
}
Normally we won't use this low level functionality. You'll see the foreach var more often:
foreach (Book book in books)
{
Console.WriteLine(book.Title);
}
foreach will do the GetEnumerator() / MoveNext() / Current for you
Note that the IEnumerable does not represent the enumeration itself, it represents the ability to enumerate. Quite often people are not that precise, and call the IEnumerable the sequence itself. But remember: to access the elements of the sequence you need to Enumerate over them (either by using MoveNext, or by calling foreach).
An IQueryable<...> seems very similar as an IEnumerable<...>. The difference is that it is usually meant to be processed by a different process, like a database management system, or a server on a different computer, it can also represent the lines in a CSV file, or whatever. The purpose of the IQueryable is to separate how the data is fetched from the manipulation of the fetched data.
Just like an IEnumerable holds the ability to enumerate, the IQueryable hold the ability to query data.
For this, the IQueryable has an Expression and a Provider. The Expression defines in a generic way what data must be fetched. The Provider knows who must provide the data (the database), and what language this data provider needs (SQL).
Perhaps you have noticed there are two kinds of LINQ statements. The ones that return an IQueryable and the ones that don't. The first group are functions like Where, Select, Join, GroupBy. They all return an IQueryable of some kind.
As long as you concatenate functions of this group, the Expression is changed. The query is not executed yet. The return value is still an object that represents the ability to query. These functions use deferred execution (or lazy execution), meaning that the query is not executed yet. You'll recognize these functions because they return IQueryable<...>. The remarks section of the description of these function also mentions that execution is deferred.
Only after you call GetEnumerator() / MoveNext(), either directly, or indirectly using foreach the query is executed.
If you start enumerating, the Expression is sent to the Provider, who will translate the Expression into the language that the executor of the query understands (SQL) and will order the executor to execute the query. The fetched data is converted to an IEnumerable<...> which is then enumerated as if the data was local.
It depends a bit on who made the Provider, but sometimes the Provider is really smart. it doesn't fetch all millions of Products from your products database, but it fetches a Page of Products. While you enumerate over this page the Provider fetches the next page. This will improve processing speed because there won't be fetched much more products than you actually will use, besides you can start enumerating before all Products are fetched.
I mentioned the LINQ functions that use deferred execution (= return IQueryable). The other group of functions will execute the query. This group of functions contain functions like ToList, ToDictionary, Max, FirstOrDefault, Count They don't return IQueryable<...>, but some TResult. If you look at the source code (google for "reference source queryable tolist"), you will see that they will do this by calling foreach or GetEnumerator.
Because the Expression must be translated to SQL, the query has less possibilities than an IEnumerable<...>. For instance, you can't use any locally defined methods in your query. If you do that, you will get a run-time exception as soon as the query is executed, telling you that the expression can't be translated into SQL.
What kind of expressions can be executed depends on who must execute your query. There is a list of Supported and Unsupported LINQ methods for ling-to-entities.

If you are using Entity Framework the allGeneres will be an instance of IQueryable<Genre> interface. The behavior is called deferred execution. For More reference https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/ef/language-reference/query-execution#deferred-query-execution

"Not to be used directly - use inside QueryOver expression" when using a generic list

What I'm trying to achieve is a search query modification based on some additional parameters. I have a list of upper limits for products. If a limit parameter is set for a product it should use it as the upper limit. If it isn't set it will use the product current value as the upper limit (so that it will allways pass).
The code looks like this:
subquery.WhereRestrictionOn(p => p.SomeNumberValue).IsNotNull().IsBetween(0).And(
Projections.Conditional(
Restrictions.On(() => alias.ProductId).IsIn(_limits),
Projections.Property<SearchItems>(x => _limits.Where(y => y.ProductID == x.ProductId).FirstOrDefault().NewUpperLimit),
Projections.Property<SearchItems>(p => p.SomeNumberValue)
)
);
Where:
_limits is a generic list of product limitations, taken from a database view
SearchItems is a view that contains products, also taken from a database view
When I launch this it throws an Exception: Not to be used directly - use inside QueryOver expression.
I'm not sure I quite understand the problem, but...
My question is: How to point nHibernate classes like Projections to a value stored somewhere in a generic List of CustomObjects?

Projections.Property expects a property being chosen in a lambda expression, like p => p.SomeNumberValue.
When _limits comes from a view, you should probably join the view in the query. Lists as arguments in queries could be a problem anyway, because the items are passed to the database as separate parameters, and there is a limit for the number of parameters.

Understanding .AsEnumerable() in LINQ to SQL

Given the following LINQ to SQL query:
var test = from i in Imports
where i.IsActive
select i;
The interpreted SQL statement is:
SELECT [t0].[id] AS [Id] .... FROM [Imports] AS [t0] WHERE [t0].[isActive] = 1
Say I wanted to perform some action in the select that cannot be converted to SQL. Its my understanding that the conventional way to accomplish this is to do AsEnumerable() thus converting it to a workable object.
Given this updated code:
var test = from i in Imports.AsEnumerable()
where i.IsActive
select new
{
// Make some method call
};
And updated SQL:
SELECT [t0].[id] AS [Id] ... FROM [Imports] AS [t0]
Notice the lack of a where clause in the executed SQL statement.
Does this mean the entire "Imports" table is cached into memory?
Would this slow performance at all if the table contained a large amount of records?
Help me to understand what is actually happening behind the scenes here.

The reason for AsEnumerable is to
AsEnumerable(TSource)(IEnumerable(TSource))
can be used to choose between query
implementations when a sequence
implements IEnumerable(T) but also has
a different set of public query
methods available
So when you were calling the Where method before, you were calling a different Where method from the IEnumerable.Where. That Where statement was for LINQ to convert to SQL, the new Where is the IEnumerable one that takes an IEnumerable, enumerates it and yields the matching items. Which explains why you see the different SQL being generated. The table will be taken in full from the database before the Where extension will be applied in your second version of the code. This could create a serious bottle neck, because the entire table has to be in memory, or worse the entire table would have to travel between servers. Allow SQL server to execute the Where and do what it does best.

At the point where the enumeration is enumerated through, the database will then be queried, and the entire resultset retrieved.
A part-and-part solution can be the way. Consider
var res = (
from result in SomeSource
where DatabaseConvertableCriterion(result)
&& NonDatabaseConvertableCriterion(result)
select new {result.A, result.B}
);
Let's say also that NonDatabaseConvertableCriterion requires field C from result. Because NonDatabaseConvertableCriterion does what its name suggests, this has to be performed as an enumeration. However, consider:
var partWay =
(
from result in SomeSource
where DatabaseConvertableCriterion(result)
select new {result.A, result.B, result.C}
);
var res =
(
from result in partWay.AsEnumerable()
where NonDatabaseConvertableCriterion select new {result.A, result.B}
);
In this case, when res is enumerated, queried or otherwise used, as much work as possible will be passed to the database, which will return enough to continue the job. Assuming that it is indeed really impossible to rewrite so that all the work can be sent to the database, this may be a suitable compromise.

There are three implementations of AsEnumerable.
DataTableExtensions.AsEnumerable
Extends a DataTable to give it an IEnumerable interface so you can use Linq against the DataTable.
Enumerable.AsEnumerable<TSource> and ParallelEnumerable.AsEnumerable<TSource>
The AsEnumerable<TSource>(IEnumerable<TSource>) method has no effect
other than to change the compile-time type of source from a type that
implements IEnumerable<T> to IEnumerable<T> itself.
AsEnumerable<TSource>(IEnumerable<TSource>) can be used to choose
between query implementations when a sequence implements
IEnumerable<T> but also has a different set of public query methods
available. For example, given a generic class Table that implements
IEnumerable<T> and has its own methods such as Where, Select, and
SelectMany, a call to Where would invoke the public Where method of
Table. A Table type that represents a database table could have a
Where method that takes the predicate argument as an expression tree
and converts the tree to SQL for remote execution. If remote execution
is not desired, for example because the predicate invokes a local
method, the AsEnumerable<TSource> method can be used to hide the
custom methods and instead make the standard query operators
available.
In other words.
If I have an
IQueryable<X> sequence = ...;
from a LinqProvider, like Entity Framework, and I do,
sequence.Where(x => SomeUnusualPredicate(x));
that query will be composed and run on the server. This will fail at runtime because the EntityFramework doesn't know how to convert SomeUnusualPredicate into SQL.
If I want that to run the statement with Linq to Objects instead, I do,
sequence.AsEnumerable().Where(x => SomeUnusualPredicate(x));
now the server will return all the data and the Enumerable.Where from Linq to Objects will be used instead of the Query Provider's implementation.
It won't matter that Entity Framework doesn't know how to interpret SomeUnusualPredicate, my function will be used directly. (However, this may be an inefficient approach since all rows will be returned from the server.)

I believe the AsEnumerable just tells the compiler which extension methods to use (in this case the ones defined for IEnumerable instead of those for IQueryable).
The execution of the query is still deferred until you call ToArray or enumerate on it.

How do I sort a gridview of Linq objects based on a derived field?

I have written a page which uses Linq to query the database and bind the resulting IQueryable to a datagrid. I have a partial class which contains extra properties which derive their values based on other values brought in from the database.
Sorting works fine on fields that are actually in the database but not for the derived fields. When I attempt to sort on such a field I get an error saying "The member 'Trip.Difference' has no supported translation to SQL.
Any idea how to allow sorting on these derived fields?

The problem is that you are binding to an IQueryable, so every time you enumerate it, you are translating the LINQ expression on the IQueryable to a SQL statement and going back to the database to execute it.
If you are trying to sort on properties that are not bound to the database model then you will get the error mentioned, as those properties only exist once an object has been created from a data row.
The simplest solution is to call ToList() on the IQueryable before using it for sorting and data-binding, so that you sort on the in-memory objects where the properties are actually available. ToList() converts your IQueryable into an IEnumerable (via List<T>), and stops it from going to the database again via LINQ to SQL.
This is generally a good design pattern to follow - the last thing you want is consumers of your business layer being able to unwittingly execute arbitrary queries against your database, simply because you returned IQueryable where you should have returned IEnumerable.

Call ToEnumerable() first, and then add the OrderBy:
var q = (from a in mDataContext.Somethings()
select a).ToEnumerable().OrderBy...

ASP.NET MVC View Model with LINQ To Entities

Let's say I create a query result
var query = from a in tblXYZ join c in tblABC on a.id = b.id select new {a.x, b.x};
What's the best way to pass that into a view? Should I create a new object and copy the query result into it?

I think it's almost always the preferred mechanism to create a view-specific model. I would also second #Marc's recommendation to materialize the query in the controller and pass the view to the list. If you have an issue with a query it's much easier to diagnose if the query is executed in the controller rather than the view. The stack trace is actually useful at that point.

As presented, that will be an anonymous type, which can be accessed (even easier in 4.0 via dynamic) but it is ugly to do so. It also currently suffers from ambiguity over when the data access happens, as the LINQ is deferred, meaning that if you pass that query into the view, you are really doing you data access during the view (and not inside the controller).
I would be tempted to create a class to represent the data (essentially a view-model), and return a list of the entities (not a deferred query).

Put the query in repository. In controller pass the query result to the specified model-view for that view.
Model-View - a specified class for transferring data from controller to the view.