Develop Reference

Entity Framework code-first IQueryable navigation property

Most of the examples I see on the internet show the navigation properties as either ICollection or straight List implementation. They are usually virtual, to enable lazy-loading.
However, when you access such property, it will load the entire collection in memory and if you have a subquery after it (i.e. object.MyListProperty.Where(...)) I have noticed that an SQL query will be issued for each item in the MyListProperty.
How do I avoid this? I want the where clause after the list property to execute on the SQL server, if possible. Can I use an IQueryable navigation property? Is there any best-practice for such case?

My advice for best practise is to disable Lazy loading altogether. Instead force the caller to eagerly load navigation properties through include statements or by using projections.
There are 3rd party products that support include with filters, as described in this post: How to filter include entities in entity framework, but in my experience this further complicates down-stream processing of the objects that are retrieved. If the entity object is loaded outside of method X, because method X can't know for sure if the navigation properties have been loaded with the correct filters, method X starts off by re-querying for the precise rows that it knows it needs.
using (var context = new MyDbContext())
{
context.Configuration.LazyLoadingEnabled = false;
// TODO: load your data
...
}
In this way the records will only be loaded when they are explicitly requested.
When you want access to an IQueryable so you can defer the loading of the data, then make those queries against the DbContext instance and not from the object.
In this example assume that a Customer has many thousands of transactions, so we don't want them to be eagerly or lazy loaded at all.
using (var context = new MyDbContext())
{
context.Configuration.LazyLoadingEnabled = false;
var customer = context.Customers.First(x => x.Id == 123);
...
// count the transactions in the last 30 days for this customer
int customerId = customer.Id;
DateTime dateFrom = DateTime.Today.AddDays(-30)
// different variations on the same query
int transactionCount1 = context.Customers.Where(x => x.Id == customerId)
.SelectMany(x => x.Transactions.Where(x => x.TransactionDate >= dateFrom))
.Count();
int transactionCount2 = context.Customers.Where(x => x.Id == customerId)
.SelectMany(x => x.Transactions)
.Where(x => x.TransactionDate >= dateFrom)
.Count();
int transactionCount3 = context.Transactions.Where(x => x.CustomerId == customerId)
.Where(x => x.TransactionDate >= dateFrom)
.Count();
}
It is good that you have identified that you want to use an IQueryable<T> we access them from the DbContext directly, not from the instances that were previously retrieved.

Lazy Loading, Collections and Single or First

I have the following model:
A User has a collection of Photos. In the Photo model, there is a property called IsProfilePhoto.
When I do the following, the results are not as expected.
var user = dbContext.Users.SingleOrDefault(u => u.Id == 1);
var profilePhoto = user.Photos.SingleOrDefault(p => p.IsProfilePhoto);
With lazy loading on, this performs two queries.
The first one gets the user by id as expected.
The second one however, gets the collection of photos by user id and then in memory does the match on IsProfilePhoto.
I was hoping that with lazy loading on it would add the SingleOrDefault to the query as well.
Is this just not possible and I must always do the inverse? E.g.
var profilePhoto = dbContext.Photos.SingleOrDefault(p => p.UserId == 1 && p.IsProfilePhoto);
var user = profilePhoto.User;
I get the reasoning, there are just certain reasons why it's more convenient to go from the User to get the profile photo.

You can get the result with a single database query by using a projection:
var userWithProfilePhoto = dbContext.Users
.Where(u => u.Id == 1)
.Select(u => new
{
User = u,
ProfilePhoto = u.Photos.Where(p => p.IsProfilePhoto).FirstOrDefault()
})
.SingleOrDefault();
userWithProfilePhoto.User and userWithProfilePhoto.ProfilePhoto are the two entities you are looking for.

You have to use Eagerly loading to load multiple levels. Lazy load, loads the level when you access this.
var user = dbContext.Users.Include(u => u.Photos).SingleOrDefault(u => u.Id == 1);
var profilePhoto = user.Photos.SingleOrDefault(p => p.IsProfilePhoto);

This is the subtle difference in LINQ methods.
You can do the filtering as part of the query as:
var profilePhoto = user.Photos.Where(p => p.IsProfilePhoto).SingleOrDefault();
Due to this behavior in LINQ to Entities, I try to always use the Where method for the condition and the parameterless overloads for First, Single, FirstOrDefault, SingleOrDefault, Any, and Count.
Edit:
My bad, MSDN mentions it directly, but any reference to a navigation property (when lazy loading is enabled) loads all of the related records.
My best suggestion then is to a) accept extra database access, or b) query as you did in your alternative example, with the first table being the 'many' of the 'one-to-many' relationship.

First, and FirstOrDefault are lazy loaded, Single, and SingleOrDefault eagerly loaded. If you don't need a an exception thrown in case of several items returned by the query, you can change it to FirstOrDefault.

Merging two iqueryables that are projected to one entity from different tables

I have tried This answer, This one and this one to merge two iqueryables. But I always receive the following error:
The type 'Estudio' appears in two structurally incompatible initializations within a single LINQ to Entities query. A type can be initialized in two places in the same query, but only if the same properties are set in both places and those properties are set in the same order.
I'm mapping from two different but similar Entity Framework Entities (EXAMEN and EXPLORACION) to my domain entity Estudio, with the following code.
IQueryable<Estudio> listExamen = context.Set<EXAMEN>().Project().To<Estudio>();
IQueryable<Estudio> listExploracion = context.Set<EXPLORACION>().Project().To<Estudio>();
var listCombined = listExamen.Concat(listExploracion);
Is there anyway of generate a IQueryable (not enumerable) with the merging of both list? If AsEnumerable() is used, then the following filters (Order, Take, etc) are executed on memory. So I need to merge the list but still be able to apply filter to the merged list wihtout execute the queries.
//This will force the next condition is executed on memory
var listCombined = listExamen.AsEnumerable().Concat(listExploracion);
Is that possible?

I would try to select your data into an anonymous type in your linq query, perform the union, and add your criteria.
var listExamen = context.Examen
.Select(x => new { x.Prop1, x.Prop2, ... }); // Add properties
var listExploracion = context.Exploraction
.Select(x => new { x.Prop1, x.Prop2, ... }); // Add identical properties
var listCombined = listExamen.Concat(listExploracion);
var whereAdded = listCombines
.Where(x => x.Prop1 == someValue);
var result = whereAdded
.Skip(skipCount)
.Take(takeCount)
.ToList();
Note: I have no idea if you can use Common Table Expressions (the SQL necessity for skip/take) in combination with a Union-query
Note: I've changed the methods used to create the expressions, since I do not know your methods (Project, To)
So I think the solution is not to cast to a specific type, but to an anonymous type, since that probably can be translated to SQL.
Warning: didn't test it

My solution was to revise my mapping code. Instead of using individual property-based mappers, I had to project the entire entity at once, making sure that all of the properties were given in the same order.
So, instead of the ForMember syntax:
Mapper.CreateMap<Client, PersonResult>()
.ForMember(p => p.Name, cfg => cfg.MapFrom(c => c.Person.FirstName + " " + c.Person.LastName))
...
I used the ProjectUsing syntax:
Mapper.CreateMap<Client, PersonResult>()
.ProjectUsing(c => new PersonResult()
{
Name = c.Person.FirstName + " " + c.Person.LastName
...
});
This must be because of the way AutoMapper constructs its projections.

One way to work around this is to add dummy types:
class Estudio<T> : Estudio { }
And new mapping:
Mapper.CreateMap<Estudio , Estudio>();
Mapper.CreateMap<EXAMEN , Estudio<EXAMEN>>();
Mapper.CreateMap<EXPLORACION, Estudio<EXPLORACION>>();
One caveat is that all fields in Estudio need some value in mapping.
You can't use ignore. Returning 0 or "" is fine.
Now we can do:
var a = context.Set<EXAMEN>().ProjectTo<Estudio<EXAMEN>>();
var b = context.Set<EXPLORACION>().ProjectTo<Estudio<EXPLORACION>>();
return a.ProjectTo<Estudio>().Concat(b.ProjectTo<Estudio>());

Linq To Entities Query

Consider the following Query :
var profilelst =
(
from i in dbContext.ProspectProfiles
where i.CreateId == currentUser
select new ProspectProfile
{
ProspectId = i.ProspectId,
Live = i.Live,
Name = i.Name,
ServiceETA = i.Opportunities.OrderByDescending(t => t.FollowUpDate)
.FirstOrDefault()
.ServiceETA.ToString(),
FollowUpDate = i.Opportunities.OrderByDescending(t => t.FollowUpDate)
.FirstOrDefault()
.FollowUpDate
}
)
.ToList();
return profilelst.OrderByDescending(c=>c.FollowUpDate)
.Skip(0).Take(endIndex)
.ToList();
Here in this query please take a look at FollowUpDate and ServiceType, these both i have fetched from Opportunity table, is there any other work around to get these both..
One to Many Relationship in tables is like: ProspectProfile -> Opportunities
Whether the query i have written is ok or is there any another work around that can be done in easier way.

The only thing you can improve is to avoid ordering twice by changing your code to this:
var profilelst
= dbContext.ProspectProfiles
.Where(i => i.CreateId == currentUser)
.Select(i =>
{
var opportunity
= i.Opportunities
.OrderByDescending(t => t.FollowUpDate)
.First();
return new ProspectProfile
{
ProspectId = i.ProspectId,
Live = i.Live,
Name = i.Name,
ServiceETA = opportunity.ServiceETA.ToString(),
FollowUpDate = opportunity.FollowUpDate
}
}).ToList();
return profilelst.OrderByDescending(c => c.FollowUpDate).Take(endIndex).ToList();
I made several changes to your original query:
I changed it to use method chains syntax. It is just so much easier to read in my opinion.
I removed the unnecessary Skip(0).
The biggest change is in the Select part:
I changed FirstOrDefault to First, because you are accessing the properties of the return value anyway. This will throw a descriptive exception if no opportunity exists. That's better than what you had: In your case it would throw a NullReferenceException. That's bad, NullReferenceExceptions always indicate a bug in your program and are not descriptive at all.
I moved the part that selects the opportunity out of the initializer, so we need to do the sorting only once instead of twice.

There are quite a few problems in your query:
You cannot project into an entity (select new ProspectProfile). LINQ to Entities only supports projections into anonymous types (select new) or other types which are not part of your entity data model (select new MySpecialType)
ToString() for a numeric or DateTime type is not supported in LINQ to Entities (ServiceETA.ToString())
FirstOrDefault().ServiceETA (or FollowUpdate) will throw an exception if the Opportunities collection is empty and ServiceETA is a non-nullable value type (such as DateTime) because EF cannot materialize any value into such a variable.
Using .ToList() after your first query will execute the query in the database and load the full result. Your later Take happens in memory on the full list, not in the database. (You effectively load the whole result list from the database into memory and then throw away all objects except the first you have Takeen.
To resolve all four problems you can try the following:
var profilelst = dbContext.ProspectProfiles
.Where(p => p.CreateId == currentUser)
.Select(p => new
{
ProspectId = p.ProspectId,
Live = p.Live,
Name = p.Name,
LastOpportunity = p.Opportunities
.OrderByDescending(o => o.FollowUpDate)
.Select(o => new
{
ServiceETA = o.ServiceETA,
FollowUpDate = o.FollowUpDate
})
.FirstOrDefault()
})
.OrderByDescending(x => x.LastOpportunity.FollowUpDate)
.Skip(startIndex) // can be removed if startIndex is 0
.Take(endIndex)
.ToList();
This will give you a list of anonymous objects. If you need the result in a list of your entity ProspectProfile you must copy the values after this query. Note that LastOpportunity can be null in the result if a ProspectProfile has no Opportunities.

Join between in memory collection and EntityFramework

Is there any mechanism for doing a JOIN between an in-memory collection and entity framework while preserving the order.
What I am trying is
var itemsToAdd =
myInMemoryList.Join(efRepo.All(), listitem => listitem.RECORD_NUMBER,
efRepoItem => efRepoItem.RECORD_NUMBER, (left, right) => right);
which gives me the rather curiously titled "This method supports the LINQ to Entities infrastructure and is not intended to be used directly from your code." error.
Now of course I can do this iteratively with something like
foreach (var item in myInMemoryList)
{
var ho = efRepo.Where(h => h.RECORD_NUMBER == item.RECORD_NUMBER).FirstOrDefault();
tmp.Add(ho);
}
but this is an N+1 query. Which is nasty as myInMemoryList might be quite large!
Resharper can refactor that for me to
tmp = (from TypeOfItemInTheList item in myInMemoryList
select efRepo.Where(h => h.RECORD_NUMBER == item.RECORD_NUMBER)
.FirstOrDefault());
which I suspect is still doing N+1 queries. So any ideas for a better approach to getting ef entities that match (on key field) with an in-memory collection. The resulting set must be in the same order as the in-memory collection was.

No you cannot join in-memory collection with database result set without loading whole result set to the memory and performing the join with linq-to-objects. Try using contains instead of join:
var myNumbers = myInMemoryList.Select(i => i.RECORD_NUMBER);
var itemsToAdd = efRepo.Where(e => myNumbers.Contains(e.RECORD_NUMBER));
This will generate query with IN operator

You can read how you can do this with the PredicateBuilder from the LINQKit or Stored Procedures in my blog post.
http://kalcik.net/2014/01/05/joining-data-in-memory-with-data-in-database-table/

try this:
var list = (from n in efRepo
where myInMemoryList.Select(m=>m.RECORD_NUMBER).Contains(n.RECORD_NUMBER)
select n).ToList();
Contains will be translated to IN operator in SQL (only if your RECORD_NUMBER member is a primitive type like int, string, Guid, etc)

What about loading the whole efRepo? I mean something like this (ToArray()):
var itemsToAdd = myInMemoryList.Join(
efRepo.ToArray(),
listitem => listitem.RECORD_NUMBER, efRepoItem => efRepoItem.RECORD_NUMBER, (left, right) => right);