Develop Reference

Queryable Linq Query Differences In Entity Framework

I have a very simple many to many table in entity framework connecting my approvals to my transactions (shown below).
I am trying to do a query inside the approval object to count the amount of transactions on the approval, which should be relatively easy.
If I do something like this then it works super fast.
int count;
EntitiesContainer dbContext = new EntitiesContainer ();
var aCnt = from a in dbContext.Approvals
where a.id == id
select a.Transactions.Count;
count = aCnt.First();
However when I do this
count = Transactions.Count;
or this
count = Transactions.AsQueryable<Transaction>().Count();
its exceedingly slow. I have traced the sql running on the server and it does indeed seem to be trying to load in all the transactions instead of just doing the COUNT query on the collection of Transactions.
Can anyone explain to me why?
Additional :
Here is how the EF model looks in regards to these two classes
UPDATE :
Thanks for all the responses, I believe where I was going wrong was to believe that the collections attached to the Approval object would execute as IQueryable. I'm going to have to execute the count against the dbContext object.
Thanks everyone.

var aCnt = from a in dbContext.Approvals
where a.id == id
select a.Transactions.Count;
EF compiles query by itself, the above query will be compiled as select count transactions
Unlike,
count = Transactions.AsQueryable<Transaction>().Count();
count = Transactions.Count;
these will select all the records from transaction and then computes the count

When you access the a.Transactions property, then you load the list of transactions (lazy loading). If you want to get the Count only, then use something like this:
dbContext.Transactions.Where(t => t.Approvals.Any(ap => ap.Id == a.Id)).Count();
where a is given Approval.

Your first method allows the counting to take place on the database server level. It will ask the database not to return the records, but to return the amount of records found. This is the most efficient method.
This is not to say that other methods can't work as efficiently, but with the other two lines, you are not making it clear in the first place that you are retrieving transactions from a join on Approvals. Instead, in the other two lines, you take the Transactions collection just by itself and do a count on that, basically forcing the collection to be filled so it can be counted.

Your first snippet causes a query to be executed on the database server. It works that because the IQueryable instance is of type ObjectQuery provided by the Entity Framework which performs the necessary translation to SQL and then execution.
The second snippet illustrates working with IEnumerable instances. Count() works on them by, in worst case, enumerating the entire collection.
In the third snippet you attempt to make the IEnumerable an IQueryable again. But the Enumerable.AsQueryable method has no way of knowing that the IEnumerable it is getting "came" from Entity Framework. The best it can do is to wrap the IEnumerable in a EnumerableQuery instance which simply dynamically compiles the expression trees given to all LINQ query operators and executes them in memory.
If you need the count to be calculated by the database server, you can either formulate the requisite query manually (that is, write what you already did in snippet one), or use the method CreateSourceQuery available to you if you're not using Code First. Note that it will really be executed on the database server, so if you have modified the collection and have not yet saved changes, the result will be different to what would be returned by calling Count directly.

Getting weird behavior when retrieving data from Microsoft CRM using LINQ

I'm having this problem accessing the Contact entity using LINQ.
I have the 2 functions below.
If I ran the 1st function and then call the 2nd one, I seemed to be missing a lot of fields in the 2nd query. Like firstname and lastname are not showing up. They just shows up as null values. If I ran the 2nd function on its own, I am getting the right data. The only fields that shows up correctly in both runs are Id, ContactId and new_username.
If I ran the 2nd function on its own, I am getting the right data.
Any ideas what am I doing wrong?
Thanks a lot
Here are the 2 functions
public List<String> GetContactsUsernameOnly()
{
IQueryable<String> _records = from _contactSet in _flinsafeContext.ContactSet
where
_contactSet.new_FAN == "username"
orderby _contactSet.new_username
select _contactSet.new_username;
return _records.ToList();
}
public List<Contact> GetContacts()
{
IQueryable<Contact> _records = from _contactSet in _flinsafeContext.ContactSet
where
_contactSet.new_FAN == "my-username-here"
orderby _contactSet.new_username
select _contactSet;
return _records.ToList();
}

It is because you are reusing the same CRM context when you call both methods (in your case _flinsafeContext)
What the context does is cache records, so the first method is returning your contact but only bringing back the new_username field.
The second method wants to return the whole record, but when it is called after the first one the record already exists in the context so it just returns that, despite only having the one field populated. It is not clever enough to lazy load the fields that have not been populated. If this method was called first, it doesn't exist in the context so will return the whole record.
There are 2 ways to get around this:
1) Don't reuse CRMContexts. Instead create a new one in each method based on a singleton IOrganizationService.
2) There is a ClearChanges() method on your context that will mean the next time you do a query it will go back to CRM and get the fields you have selected. This will also clear any unsaved Created/Updates/Deletes etc so you have to be careful around what state the context is in.
As an aside, creating a new CRM Context isn't an intensive operation so it's not often worthwhile passing contexts around and reusing them. It is creating the underlying OrganisationService that is the slowest bit.
This behaviour can be so painful, because it is horribly inefficient and slow to return the entire record so you WANT to be selecting only the fields you want for each query.

And here's how you return just the fields you want:
IEnumerable<ptl_billpayerapportionment> bpas = context.ptl_billpayerapportionmentSet
.Where(bm => bm.ptl_bill.Id == billId)
.Select(bm => new ptl_billpayerapportionment()
{
Id = bm.Id,
ptl_contact = bm.ptl_contact
})
This will ensure a much smaller sql statement will be executed against the context as the Id and ptl_contact are the only two fields being returned. But as Ben says above, further retrievals against the same entity in the same context will return nulls for fields not included in the initial select (as per the OP's question).
For bonus points, using IEnumerable and creating a new, lightweight, entity gives you access to the usual LINQ methods, e.g. .Any(), .Sum() etc. The CRM SDK doesn't like using them against var datasets, apparently.

C# linq to sql - selecting tables dynamically

I have the following scenario: there are a database that generates a new logTable every year. It started on 2001 and now has 11 tables. They all have the same structure, thus the same fields, indexes,pk's, etc.
I have some classes called managers that - as the name says - manages every operation on this DB. For each different table i have a manager, except for this logTable which i have only one manager.
I've read a lot and tried different things like using ITable to get tables dynamically or an interface that all my tables implements. Unfortunately, i lose strong-typed properties and with that i can't do any searches or updates or anything, since i can't use logTable.Where(q=> q.ID == paramId).
Considering that those tables have the same structure, a query that searches logs from 2010 can be the exact one that searches logs from 2011 and on.
I'm only asking this because i wouldn't like to rewrite the same code for each table, since they are equal on it's structure.
EDIT
I'm using Linq to SQL as my ORM. And these tables uses all DB operations, not just select.

Consider putting all your logs in one table and using partitioning to maintain performance. If that is not feasible you could create a view that unions all the log tables together and use that when selecting log data. That way when you added a new log table you just update the view to include the new table.
EDIT Further to the most recent comment:
Sounds like you need a new DBA if he won't let you create new SPs. Yes I think could define an ILogTable interface and then make your log table classes implement it, but that would not allow you do GetTable<ILogTable>(). You would have to have some kind of DAL class with a method that created a union query, e.g.
public IEnumerable<ILogTable> GetLogs()
{
var Log2010 = from log in DBContext.2010Logs
select (ILogTable)log;
var Log2011 = from log in DBContext.2011Logs
select (ILogTable)log;
return Log2010.Concat(Log2011);
}
Above code is completely untested and may fail horribly ;-)
Edited to keep #AS-CII happy ;-)

You might want to look into the Codeplex Fluent Linq to SQL project. I've never used it, but I'm familiar with the ideas from using similar mapping techniques in EF4. YOu could create a single object and map it dynamically to different tables using syntax such as:
public class LogMapping : Mapping<Log> {
public LogMapping(int year) {
Named("Logs" + year);
//Column mappings...
}
}

As long as each of your queries return the same shape, you can use ExecuteQuery<Log>("Select cols From LogTable" + instance). Just be aware that ExecuteQuery is one case where LINQ to SQL allows for SQL Injection. I discuss how to parameterize ExecuteQuery at http://www.thinqlinq.com/Post.aspx/Title/Does-LINQ-to-SQL-eliminate-the-possibility-of-SQL-Injection.

Understanding .AsEnumerable() in LINQ to SQL

Given the following LINQ to SQL query:
var test = from i in Imports
where i.IsActive
select i;
The interpreted SQL statement is:
SELECT [t0].[id] AS [Id] .... FROM [Imports] AS [t0] WHERE [t0].[isActive] = 1
Say I wanted to perform some action in the select that cannot be converted to SQL. Its my understanding that the conventional way to accomplish this is to do AsEnumerable() thus converting it to a workable object.
Given this updated code:
var test = from i in Imports.AsEnumerable()
where i.IsActive
select new
{
// Make some method call
};
And updated SQL:
SELECT [t0].[id] AS [Id] ... FROM [Imports] AS [t0]
Notice the lack of a where clause in the executed SQL statement.
Does this mean the entire "Imports" table is cached into memory?
Would this slow performance at all if the table contained a large amount of records?
Help me to understand what is actually happening behind the scenes here.

The reason for AsEnumerable is to
AsEnumerable(TSource)(IEnumerable(TSource))
can be used to choose between query
implementations when a sequence
implements IEnumerable(T) but also has
a different set of public query
methods available
So when you were calling the Where method before, you were calling a different Where method from the IEnumerable.Where. That Where statement was for LINQ to convert to SQL, the new Where is the IEnumerable one that takes an IEnumerable, enumerates it and yields the matching items. Which explains why you see the different SQL being generated. The table will be taken in full from the database before the Where extension will be applied in your second version of the code. This could create a serious bottle neck, because the entire table has to be in memory, or worse the entire table would have to travel between servers. Allow SQL server to execute the Where and do what it does best.

At the point where the enumeration is enumerated through, the database will then be queried, and the entire resultset retrieved.
A part-and-part solution can be the way. Consider
var res = (
from result in SomeSource
where DatabaseConvertableCriterion(result)
&& NonDatabaseConvertableCriterion(result)
select new {result.A, result.B}
);
Let's say also that NonDatabaseConvertableCriterion requires field C from result. Because NonDatabaseConvertableCriterion does what its name suggests, this has to be performed as an enumeration. However, consider:
var partWay =
(
from result in SomeSource
where DatabaseConvertableCriterion(result)
select new {result.A, result.B, result.C}
);
var res =
(
from result in partWay.AsEnumerable()
where NonDatabaseConvertableCriterion select new {result.A, result.B}
);
In this case, when res is enumerated, queried or otherwise used, as much work as possible will be passed to the database, which will return enough to continue the job. Assuming that it is indeed really impossible to rewrite so that all the work can be sent to the database, this may be a suitable compromise.

There are three implementations of AsEnumerable.
DataTableExtensions.AsEnumerable
Extends a DataTable to give it an IEnumerable interface so you can use Linq against the DataTable.
Enumerable.AsEnumerable<TSource> and ParallelEnumerable.AsEnumerable<TSource>
The AsEnumerable<TSource>(IEnumerable<TSource>) method has no effect
other than to change the compile-time type of source from a type that
implements IEnumerable<T> to IEnumerable<T> itself.
AsEnumerable<TSource>(IEnumerable<TSource>) can be used to choose
between query implementations when a sequence implements
IEnumerable<T> but also has a different set of public query methods
available. For example, given a generic class Table that implements
IEnumerable<T> and has its own methods such as Where, Select, and
SelectMany, a call to Where would invoke the public Where method of
Table. A Table type that represents a database table could have a
Where method that takes the predicate argument as an expression tree
and converts the tree to SQL for remote execution. If remote execution
is not desired, for example because the predicate invokes a local
method, the AsEnumerable<TSource> method can be used to hide the
custom methods and instead make the standard query operators
available.
In other words.
If I have an
IQueryable<X> sequence = ...;
from a LinqProvider, like Entity Framework, and I do,
sequence.Where(x => SomeUnusualPredicate(x));
that query will be composed and run on the server. This will fail at runtime because the EntityFramework doesn't know how to convert SomeUnusualPredicate into SQL.
If I want that to run the statement with Linq to Objects instead, I do,
sequence.AsEnumerable().Where(x => SomeUnusualPredicate(x));
now the server will return all the data and the Enumerable.Where from Linq to Objects will be used instead of the Query Provider's implementation.
It won't matter that Entity Framework doesn't know how to interpret SomeUnusualPredicate, my function will be used directly. (However, this may be an inefficient approach since all rows will be returned from the server.)

I believe the AsEnumerable just tells the compiler which extension methods to use (in this case the ones defined for IEnumerable instead of those for IQueryable).
The execution of the query is still deferred until you call ToArray or enumerate on it.

ASP.NET MVC View Model with LINQ To Entities

Let's say I create a query result
var query = from a in tblXYZ join c in tblABC on a.id = b.id select new {a.x, b.x};
What's the best way to pass that into a view? Should I create a new object and copy the query result into it?

I think it's almost always the preferred mechanism to create a view-specific model. I would also second #Marc's recommendation to materialize the query in the controller and pass the view to the list. If you have an issue with a query it's much easier to diagnose if the query is executed in the controller rather than the view. The stack trace is actually useful at that point.

As presented, that will be an anonymous type, which can be accessed (even easier in 4.0 via dynamic) but it is ugly to do so. It also currently suffers from ambiguity over when the data access happens, as the LINQ is deferred, meaning that if you pass that query into the view, you are really doing you data access during the view (and not inside the controller).
I would be tempted to create a class to represent the data (essentially a view-model), and return a list of the entities (not a deferred query).

Put the query in repository. In controller pass the query result to the specified model-view for that view.
Model-View - a specified class for transferring data from controller to the view.