Develop Reference

EF6 eager loading inside POCO

Lets say, I have such structure:
Branch: Id
Group: Id
BranchGroup: BranchId, GroupId, Since, Till
Relations are Branch 1..N BranchGroup 1..1 Group, so that generated Branch class has an ICollection<BranchGroup> BranchGroupsnavigation property.
And there is a property inside Branch POCO, returning current groups (simplified a bit):
public List<Group> Groups => BranchGroups.Where(bg => bg.Since <= DateHelper.Current &&
bg.Till >= DateHelper.Current)
.Select(bg => bg.Group).ToList();
I call it and db is exploded with thousands of queries, each for one group. Is it possible to somehow optimize it without having access to dbContext (or at least reach current context from inside entity)?
UPD: simplest real query looks like this:
public List<Group> Groups => BranchGroups.Where(bg => bg.Since <= DateHelper.Current &&
bg.Till >= DateHelper.Current)
.Select(bg => bg.Group)
.Where(g => !g.Deleted).ToList();

That seems to be not possible - whenever you access navigation property (BranchGroups in this case), whole related set is pulled from database, so the rest of your query (Where, Select, Where) is performed over data set in memory, so it not translated to database query. Then, bg.Group is also navigation property, so to get one, separate database query is performed, just as you observe.
Real POCO objects should not have any relation to DbContext, they are just plain objects. So you cannot (and should not be able to) reach to the context from inside POCO (or that is not POCO any more).
So your options are:
Fill Groups property when constructing Branch object (not good if you don't always need to load those Groups).
Fetch Groups with the help of external context reference (not from inside POCO).
Maybe, when constructing Branch object, set Groups property like this:
branch.Groups = context.BranchGroups.Where(c => c.BranchID = branch.ID && ...); // the rest of your query
Then it will still be POCO (Groups are just IEnumerable) but contains query you need which is lazy loaded on access (and no explicit references to context).

Manually assign existing object to Entity Framework navigation property

Is it possible to manually assign an existing object to an Entity Framework (db first) object's navigation property?
The context to the question is that I have a problem in trying to bring back a (heavily filtered) list of objects with all the children and descendants attached so that the full graph is available in memory after the context is disposed.
I tried doing this via .Include() statements using something like this:
using (var ctx = new MyEntities())
{
myParents = ctx.Parents
.Where(p => MyFilter(p))
.Include(p => p.Children)
.Include(p => p.Children.Select(c=>c.Grandchildren))
.Include(p => p.Children.Select(c=>c.Grandchildren.Select(g=>g.GreatGrandChildren)));
}
but the generated query runs too slowly because of the known performance problems with using nested include statements (as explained in lots of places including this blog).
I can pull back the parents, children and Grandchildren without performance issues - I only hit the troubles when I include the very last .Include() statement for the greatgrandchildren.
I can easily get the GreatGrandChildren objects back from the database with a second separate query by building a list of GrandChildrenIds from the GrandChildren already retrieved and doing something like:
greatGrandKids = ctx.GreatGrandChildren.Where(g=>ids.Contains(g.GrandChildId)).ToList();
but now, once I dispose of the context, I cannot do something like grandChildA.GreatGrandChildren without hitting the object context disposed exception.
I could have as many as a few thousand GrandChildren objects so I really want to avoid a round trip to the database to fetch the GreatGrandChildren for each one which rules out simply using .Load() on each GrandChild object, right?
I could feasibly work around this by either just looking up the required greatgrandchildren from greatGrandKids each time I needed them in my subsequent code or even by adding a new (non-mapped) Property such as .GreatGrandChildrenLocal to the GrandChild class and assigning them all up front but these both feel very kludgy & ugly. I'd MUCH prefer to find a way to just be able to access the existing .GreatGrandChildren navigation property on each GrandChild object.
Trying the obvious of assigning to the navigation property with something like this:
grandchild.GreatGrandChildren = greatGrandKids
.Where(g=>g.GrandChildId == grandChild.Id)
.ToList();
fails too when I then try to access grandchild.GreatGrandChildren (still giving the object disposed exception).
So my question is:
Is there a way I can assign the existing GreatGrandChdildren objects I have already retrieved from the database to the .GreatGrandChdildren navigation property on the GrandChild object in such a way as to make them available (only needed for read operations) after the context is disposed?
(Or indeed is there a different solution to the problem?)

If you disable proxy creation with:
ctx.Configuration.ProxyCreationEnabled = false;
then reading and writing from/to the navigation property works exactly as expected without trying to lazily load the entities and throwing the object disposed exception.
So we have something like:
using (var ctx = new MyEntities())
{
myParents = ctx.Parents
.Where(p => MyFilter(p))
.Include(p => p.Children)
.Include(p => p.Children.Select(c=>c.Grandchildren));
//skip the final GreatGrandChildren include statement
//get the associated grandchildren & their ids:
var grandKids = myParents.SelectMany(p=>p.Children)
.SelectMany(c=>c.Grandchildren)
.ToList();
var ids = grandKids.Select(g=>g.Id)).ToList();
//Get the great grandkids:
var greatGrandKids = ctx.GreatGrandChildren
.Where(g=>ids.Contains(g.GrandChildId)).ToList();
//Assign the greatgrandchildren to the grandchildren:
foreach (grandChild in grandKids)
{
grandChild.GreatGrandChildren = greatGrandKids
.Where(g=>g.GrandChildId == grandChild.Id)
.ToList();
}
}
and now we can access the the .GreatGrandChildren property outside the context without hitting the context disposed exception. Whilst this still feels a little messy, it works out MUCH cheaper than either using the original Include() statement or calling .Load() on each GrandChild.
N.B. As these objects are only used in read operations and I don't need Lazy Loading then there are no negative implications to turning off proxy creation in my circumstances. If write operations and/or lazy loading were also necessary then we would also need to consider the implications of turning this off for the given EF context.

How to merge entity graphs in LLBLGen?

Let's assume I have 2 tables, A and B with 1-0..1 relation. I use the Adapter approach. I need to load a collection of A entities in one place, and then load all related B entities for all A entities later. The reason to not use Prefetch is that in most cases I will not need to load B entities.
I use LINQ everywhere, so I would like to do it the same way.
The code I am trying to write looks like this:
var linqMetadata = new LinqMetaData(adapter)
{
ContextToUse = new Context()
};
var aCollection = linqMetadata.A.Where(/*some filter*/).ToList();
// ...
var bIds = aCollection.Select(x => x.BId);
var bCollection = linqMetadata.B.Where(x => bIds.Contains(x.BId)).ToList();
The problem is that bCollection and aCollection stay unlinked, i.e. all A entities have B = null and vice versa. I want these references to be set, and therefore the 2 graphs to be united into a single one.
I can join 2 collections using LINQ to Objects, but that's not elegant, and besides this might get much more complicated if both collections contain complex graph with interconnections that need to be established.
I can write a prefetch from B to A, but that's one extra DB query that is completely unnecessary.
Is there an easy way to get these 2 graphs merged?

Object relation graph is built automatically if you add related entities using assignment for master entity and AddRange for detail entities.
Here's a pseudo-code sample:
foreach aEntity in aCollection
aEntity.BEntity = bCollection.First(x=>x.Id == aEntity.bId);
OR
foreach bEntity in bCollection
bEntity.AEntity.AddRange(aCollection.Where(x=>x.bId == bEntity.Id));
After doing this your object graph is complete.

Since you are already using a Context, you can use PrefetchPath and context to load the PrefetchPath later, that way your entities are always linked. Then you can run a Linq2Objects query to load in-memory all B's.
var bs = aCollection.Select(x => x.B).ToList();

EF: Include with where clause [duplicate]

This question already has answers here:
Filtering on Include in EF Core
(9 answers)
Closed 2 years ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
As the title suggest I am looking for a way to do a where clause in combination with an include.
Here is my situations:
I am responsible for the support of a large application full of code smells.
Changing too much code causes bugs everywhere so I am looking for the safest solution.
Let's say I have an object Bus and an object People(Bus has a navigation prop Collection of People).
In my Query I need to select all the Busses with only the Passengers that are awake. This is a simplistic dummy example
In the current code:
var busses = Context.Busses.Where(b=>b.IsDriving == true);
foreach(var bus in busses)
{
var passengers = Context.People.Where(p=>p.BusId == bus.Id && p.Awake == true);
foreach(var person in passengers)
{
bus.Passengers.Add(person);
}
}
After this code the Context is disposed and in the calling method the resulting Bus entities are Mapped to a DTO class (100% copy of Entity).
This code causes multiple calls to DB which is a No-Go, so I found this solution ON MSDN Blogs
This worked great when debugging the result but when the entities are mapped to the DTO (Using AutoMapper) I get an exception that the Context/Connection has been closed and that the object can't be loaded. (Context is always closed can’t change this :( )
So I need to make sure that the Selected Passengers are already loaded (IsLoaded on navigation property is also False). If I inspect the Passengers collection The Count also throws the Exception but there is also a collection on the Collection of Passegers called “wrapped related entities” which contain my filtered objects.
Is there a way to load these wrapped related entities into the whole collection?
(I can't change the automapper mapping config because this is used in the whole application).
Is there another way to Get the Active Passengers?
Any hint is welcome...
Edit
Answer of Gert Arnold doesn't work because the data isn't loaded eagerly.
But when I simplify it and delete the where it is loaded. This is realy strange since the execute sql returns all the passengers in both cases. So there must be a problem when putting the results back into the entity.
Context.Configuration.LazyLoadingEnabled = false;
var buses = Context.Busses.Where(b => b.IsDriving)
.Select(b => new
{
b,
Passengers = b.Passengers
})
.ToList()
.Select(x => x.b)
.ToList();
Edit2
After a lot of struggle the answer of Gert Arnold work!
As Gert Arnold suggested you need to disable Lazy Loading and Keep it OFF.
This will ask for some extra changes to the appliaction since the prev developer loved Lazy Loading -_-

This feature has now been added to Entity Framework core 5. For earlier versions you need a work-around (note that EF6 is an earlier version).
Entity Framework 6 work-around
In EF6, a work-around is to first query the required objects in a projection (new) and let relationship fixup do its job.
You can query the required objects by
Context.Configuration.LazyLoadingEnabled = false;
// Or: Context.Configuration.ProxyCreationEnabled = false;
var buses = Context.Busses.Where(b => b.IsDriving)
.Select(b => new
{
b,
Passengers = b.Passengers
.Where(p => p.Awake)
})
.AsEnumerable()
.Select(x => x.b)
.ToList();
What happens here is that you first fetch the driving buses and awake passengers from the database. Then, AsEnumerable() switches from LINQ to Entities to LINQ to objects, which means that the buses and passengers will be materialized and then processed in memory. This is important because without it EF will only materialize the final projection, Select(x => x.b), not the passengers.
Now EF has this feature relationship fixup that takes care of setting all associations between objects that are materialized in the context. This means that for each Bus now only its awake passengers are loaded.
When you get the collection of buses by ToList you have the buses with the passengers you want and you can map them with AutoMapper.
This only works when lazy loading is disabled. Otherwise EF will lazy load all passengers for each bus when the passengers are accessed during the conversion to DTOs.
There are two ways to disable lazy loading. Disabling LazyLoadingEnabled will re-activate lazy loading when it is enabled again. Disabling ProxyCreationEnabled will create entities that aren't capable of lazy loading themselves, so they won't start lazy loading after ProxyCreationEnabled is enabled again. This may be the best choice when the context lives longer than just this single query.
But... many-to-many
As said, this work-around relies on relationship fixup. However, as explained here by Slauma, relationship fixup doesn't work with many-to-many associations. If Bus-Passenger is many-to-many, the only thing you can do is fix it yourself:
Context.Configuration.LazyLoadingEnabled = false;
// Or: Context.Configuration.ProxyCreationEnabled = false;
var bTemp = Context.Busses.Where(b => b.IsDriving)
.Select(b => new
{
b,
Passengers = b.Passengers
.Where(p => p.Awake)
})
.ToList();
foreach(x in bTemp)
{
x.b.Pasengers = x.Passengers;
}
var busses = bTemp.Select(x => x.b).ToList();
...and the whole thing becomes even less appealing.
Third-party tools
There is a library, EntityFramework.DynamicFilters that makes this a lot easier. It allows you to define global filters for entities, that will subsequently be applied any time the entity is queried. In your case this could look like:
modelBuilder.Filter("Awake", (Person p) => p.Awake, true);
Now if you do...
Context.Busses.Where(b => b.IsDriving)
.Include(b => b.People)
...you'll see that the filter is applied to the included collection.
You can also enable/disable filters, so you have control over when they are applied. I think this is a very neat library.
There is a similar library from the maker of AutoMapper: EntityFramework.Filters
Entity Framework core work-around
Since version 2.0.0, EF-core has global query filters. These can be used to set predefined filter on entities that are to be included. Of course that doesn't offer the same flexibility as filtering Include on the fly.
Although global query filters are a great feature, so far the limitation is that a filter can't contain references to navigation properties, only to the root entity of a query. Hopefully in later version these filters will attain wider usage.

Now EF Core 5.0's Filter Include method now supports filtering of the entities included
var busses = _Context.Busses
.Include(b => b.Passengers
.Where(p => p.Awake))
.Where(b => b.IsDriving);

Disclaimer: I'm the owner of the project Entity Framework Plus
EF+ Query IncludeFilter feature allows filtering related entities.
var buses = Context.Busses
.Where(b => b.IsDriving)
.IncludeFilter(x => x.Passengers.Where(p => p.Awake))
.ToList();
Wiki: EF+ Query IncludeFilter

In my case the Include was an ICollection, and also did not want to return them, I just needed to get the main entities but filtered by the referenced entity. (in other words, Included entity), what I ended up doing is this. This will return list of Initiatives but filtered by InitiativeYears
return await _context.Initiatives
.Where(x => x.InitiativeYears
.Any(y => y.Year == 2020 && y.InitiativeId == x.Id))
.ToListAsync();
Here the Initiatives and the InitiativeYears has following relationship.
public class Initiative
{
public int Id { get; set; }
public string Name { get; set; }
public ICollection<InitiativeYear> InitiativeYears { get; set; }
}
public class InitiativeYear
{
public int Year { get; set; }
public int InitiativeId { get; set; }
public Initiative Initiative { get; set; }
}

For any one still curious about this. there builtin functionality for doing this in EF Core. using .Any inside of a where clause so the code would like similar to something like this
_ctx.Parent
.Include(t => t.Children)
.Where(t => t.Children.Any(t => /* Expression here */))

Entity Framework Code First - select based on type of child TPH class

I am currently working with Entity Framework 4 on a project that is using Table Per Hierarchy to represent one set of classes using a single table. The way this works is that this table represents states and different states are associated with different other classes.
So you might imagine it to look like this, ignoring the common fields all states share:
InactiveState
has a -> StateStore
ActiveState
has a -> ActivityLocation
CompletedState
has a -> CompletionDate
Each state has a collection of all the InventoryItems that belong to it.
Now each item in my inventory has many states, where the last one in the history is the current state. To save on lists I have a shortcut field that points to the current state of my Inventory:
public class InventoryItem : Entity
{
// whole bunch of other properties
public virtual ICollection<InventoryState> StateHistory { get; set; }
public virtual InventoryState LastState { get; set; }
}
The first problem I am having is when I want to find, for example, all the InventoryItems which are in the Active state.
It turns out Linq-To-Entities doesn't support GetType() so I can't use a statement like InventoryRepository.Get().Where( x => x.LastState.GetType() == someType ). I can use the is operator, but that requires a fixed type so rather than being able to have a method like this:
public ICollection<InventoryItem> GetInventoryItemsByState( Type state )
{
return inventoryRepository.Get().Where( x => x.LastState is state );
}
I have to run some kind of if statement based on the type before I make the Linq query, which feels wrong. The InventoryItem list is likely to get large, so I need to do this at the EF level, I can't pull the whole list into memory and use GetType(), for example.
I have two questions in this situation, connected closely enough that I think they can be combined as they probably reflect a lack of understanding on my part:
Is it possible to find a list of items that share a child table type using Linq To Entities?
Given that I am not using Lazy Loading, is it possible to Include related items for child table types using TPH so that, for example, if I have an InactiveState as the child of my InventoryItem I can preload the StateStore for that InactiveState?

Is it possible to find a list of items that share a child table type
using Linq To Entities?
I don't think it's possible in another way than using an if/switch that checks for the type and builds a filter expression using is T or OfType<T>. You could encapsulate this logic into an extension method for example to have a single place to maintain and a reusable method:
public static class Extensions
{
public static IQueryable<InventoryItem> WhereIsLastState(
this IQueryable<InventoryItem> query, Type state)
{
if (state == typeof(InactiveState))
return query.Where(i => i.LastState is InactiveState);
if (state == typeof(ActiveState))
return query.Where(i => i.LastState is ActiveState);
if (state == typeof(CompletedState))
return query.Where(i => i.LastState is CompletedState);
throw new InvalidOperationException("Unsupported type...");
}
}
To be used like this:
public ICollection<InventoryItem> GetInventoryItemsByState(Type state)
{
return inventoryRepository.Get().WhereIsLastState(state).ToList();
}
I don't know if it would be possible to build the i => i.LastState is XXX expression manually using the .NET Expression API and based on the Type passed into the method. (Would interest me too, to be honest, but I have almost no clue about expression manipulation to answer that myself.)
Given that I am not using Lazy Loading, is it possible to Include
related items for child table types using TPH so that, for example, if
I have an InactiveState as the child of my InventoryItem I can preload
the StateStore for that InactiveState?
I am not sure if I understand that correctly but generally eager loading with Include does not support any filtering or additional operations on specific included children.
One way to circumvent this limitation and still get the result in a single database roundtrip is using a projection which would look like this:
var result = context.InventoryItems
.Select(i => new
{
InventoryItem = i,
LastState = i.LastState,
StateStore = (i.LastState is InactiveState)
? (i.LastState as InactiveState).StateStore
: null
})
.AsEnumerable()
.Select(x => x.InventoryItem)
.ToList();
If the query is a tracked query (which it is in the example above) and the relationships are not many-to-many (they are not in your example) the context will fixup the relationships when the entities are loaded into the context, that is InventoryItem.LastState and InventoryItem.LastState.StateStore (if LastState is of type InactiveState) will be set to the loaded entities (as if they had been loaded with eager loading).