Disclaimer: these requirements are not set by me, unless this is an impossible task I cannot convince my boss otherwise.
Let's say we have two entities: Item and ItemTranslation.
public class Item
{
public int Id { get; set; }
public string Description { get; set; }
public virtual ICollection<Item> Children { get; set; }
public virtual Item Parent { get; set; }
public virtual ICollection<ItemTranslation> Translations { get; set; }
}
public class ItemTranslation
{
public int Id { get; set; }
public string CultureId { get; set; }
public string Description { get; set; }
public virtual Item Item { get; set; }
}
The requirement is that Item.Description should be filled in based on a language selected by default, but also allowing it to be specified based on what the user wants. The Item.Description column doesn't actually exist in the database.
In SQL this would be easy: all you have to do is query both tables like so
SELECT [Item].[Id], [ItemTranslation].[Description], [Item].[ParentId]
FROM [Item]
LEFT JOIN [ItemTranslation] ON [Item].[Id] = [ItemTranslation].[ItemId]
WHERE [CultureId] = {cultureId}
Or use an OUTER APPLY depending on your implementation. I have added this query to the .FromSql() function built in Entity Framework.
Put this all together in an OData API and this all works fine for one Item. However as soon as you start using $expand (which behind the scenes is a sort of .Include()) it no longer works. The query being sent to the database for the related entities no longer holds the SQL which I specified in .FromSql(). Only the first query does. On top of this when you would query an Item from a different controller e.g. ItemTranslation this would also no longer work since .FromSql() is only applied in the other controller.
I could write a query interceptor which simply replaces the generated SQL by Entity Framework and replaces FROM [Item] with FROM [Item] LEFT JOIN [ItemTranslation] ON [Item].[Id] = [ItemTranslation].[ItemId] WHERE [CultureId] = {cultureId} but I wonder if there is a better implementation than that. Perhaps even a redesign in models. I'm open to suggestions.
FromSql has some limitations. I suspect this is the reason why Include won't work.
But once you use EF, why are you messing with SQL? What difficulties does that query have which prevents you from doing it in LINQ? Left join maybe?
from item in ctx.Items
from itemTranslation in ctx.ItemTranslations.Where(it => it.Item.Id == item.Id).DefaultIfEmpty()
where itemTranslation.CultureId == cultureId
select new { item.Id, itemTranslation.Description, ParentId = item.Parent.Id };
Update
Going over the issue again, I see a further problem. Include will only work on an IQueryable<T> where T is an entity whose navigation properties are mapped properly. Now, from this perspective, it doesn't matter if you use FromSql or LINQ if it produces an IQueryable of some projection instead of an entity, Include won't work for obvious reasons.
To be able to include ItemTranslation entities, your action method should look something like this:
[Queryable]
public IQueryable<Item> GetItems()
{
return db.Items;
}
So the framework can perform $expand on the IQueryable<Item> you return. However, this will include all item translations, not just the ones with the desired culture. If I get it correctly, this is your core issue.
It's obvious as well that you cannot apply this culture filter to an IQueryable<Item>. But you shouldn't do that as this is achieved by $filter in OData:
GET https://.../Items/$expand=Translations&$filter=Translations/CultureId eq culture
Related
I use EF Core 5 in a project, one of my entities contains a NotMapped property that mixed two properties of the entity, I expect in the select statement only properties that contain in the select statement load from the database but after profiling, I have seen that all properties were loaded.
As a sample, the Contact entity contains one NotMapped property as follows.
public class Contact
{
public int ContactId { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
[NotMapped]
public string FullName => $"{FirstName} {LastName}";
public string Email { get; set; }
public string Phone { get; set; }
public string Address { get; set; }
}
public class SampleContext : DbContext
{
public DbSet<Contact> Contacts { get; set; }
}
In the following query, I need only ContactId and FullName, I expect only ContactId, FirstName, LastName to load in the TSQL query, but all properties are loaded.
var list = dbContext.Contacts.Select( e => new
{
e.ContactId,
e.FullName
}).ToList();
If you just want to load some columns you could:
var list = dbContext.Contacts
.Where(...)
.Select( e => new
{
e.ContactId,
LastName = e.FirstName + " " + e.LastName
})
.ToList() // hit the database
I'm actually surprised this runs at all. The whole point of NotMapped is to indicate that the column is not bound and shouldn't even be used in a Select or any other Linq expression. With EF 6 trying that would have resulted in an error. Your choices were:
A) Select the entity and use the client-side computed property as you had declared. (Essentially what your code is doing behind the scenes)
or
B) as tymtam suggested, compute the property in the anonymous type.
var list = dbContext.Contacts
.Where(...)
.Select( e => new
{
e.ContactId,
LastName = e.FirstName + " " + e.LastName
}).ToList() // hit the database
EF Core had introduced client side evaluation which was enabled by default in earlier versions but I believe disabled by default since EF Core 3... This issue smells either of that you have client side evaluation enabled, or a bug in EF Core 5 that is still reverting to a client-side evaluation for an unmapped property. Either way, I am not aware of marking a property as "client side computed" as this would only possibly work if the property was declared as an expression (rather than effectively a string.Format) so your options are either of the two above, or relying on what looks to be client side evaluation which is either something your project has configured (and potentially will bite you in the butt down the road) or a bug that may be "fixed" and stop working at some point in the future.
EFCore doesn't (can't) decompile [NotMapped] to calculate the dependencies (that's a pretty hard thing to do), it defaults back to the next best thing it can do which is load all properties so it can calculate it client side.
If you do want to be able to push this calculation to SQL you need to use an expression tree. I would suggest using a package like NeinLinq or EFCore Projectables that allows you to do this in a fairly easy way.
(there's quite a few libraries that allow this, this was just the first two i knew off the top of my head)
I wrote a query which is pretty simple:
var locations = await _context.Locations
.Include(x => x.LocationsOfTheUsers)
.Include(x => x.Address)
.ThenInclude(x => x.County)
.Where(CalculateFilters(searchObj))
.ToListAsync(cancellationToken);
And everytime LocationsOfTheUsers were null so I decided to .Include(x => x.LocationsOfTheUsers) and I received results as expected but I'm not sure why do I have to include this collections since it's defined like this:
public class Location
{
public string Title { get; set; }
public long? RegionId { get; set; }
public Region Region { get; set; }
public long? AddressId { get; set; }
public Address Address { get; set; }
public long? CountyId { get; set; }
public County County { get; set; }
public ICollection<LocationsOfTheUsers> LocationsOfTheUsers { get; set; }
}
I thought this will be automatically included since it exist as ICollection in Location class.
So why is .Include() on LocationsOfTheUsers needed here?
Thanks guys
Cheers
In entity framework the non-virtual properties represent the columns of the tables, the virtual properties represent the relations between the tables (one-to-many, many-to-many, ...)
So your property should have been defined as:
public virtual ICollection<LocationsOfTheUsers> LocationsOfTheUsers { get; set; }
One of the slower parts of a database query is the transfer of the selected data from the database management system to your local process. Hence it is wise to limit the selected data to the values you actually plan to use.
If you have a one-to-many relation between Schools and Students, and you ask for School [10] you don't want automatically to fetch its 2000 Students.
Even if you would like to have "School [10] with all its Students" it would not be efficient to use Include to also fetch the Students. Every Student will have a foreign key SchoolId with a Value of [10]. If you would use Include you would transfer this foreign key 2000 times. What a waste!
When using entity framework always use Select to fetch data and select only the properties that you actually plan to use. Only use Include if you plan to change the included items.
This way you can separate your database table structure from the actual query. If your database structure changes, only the query changes, users of your query don't notice the internal changes.
Apart from better performance and more robustness against changes, readers of your code can more easily see what values are in their query.
Certainly don't use Include to save you some typing. Having to debug one error after future changes will take way more time than you will ever save by typeing include instead of Select
Finally: limit your data early in your process, so put the Where in front.
So your query should be:
var predicate = CalculateFilters(searchObj)
var queryLocations = dbContext.Locations
.Where(predicate)
.Select(location => new
{
// Select only the location properties that you plan to use
Id = location.Id,
Name = location.Name,
// Locations Of the users:
UserLocations = location.LocationsOfTheUsers
.Select(userLocation => new
{
// again: only the properties that you plan to use
Id = userLocation.Id,
...
// Not needed, you already know the value
// LocationId = userLocation.LocationId
})
.ToList(),
Address = new
{
Street = location.Address.Street,
PostCode = location.Addrress.PostCode,
...
County = location.Address.County.Name // if you only want one property
// or if you want more properties:
County = new
{
Name = location.Address.County.Name,
Abbr = location.Address.Count.Abbr,
...
}),
},
});
I thought this will be automatically included since it exist as ICollection in Location class.
Well, it's not automatically included, probably for performance reasons as the graph of related entities and their recursive child entities may be rather deep.
That's why you use eager loading to explicitly include the related entities that you want using the Include method.
The other option is to use lazy loading which means that the related entities are loaded as soon as you access the navigation property in your code, assuming some prerequisites are fulfilled and that the context is still around when this happens.
Please refer to the docs for more information.
I believe you are using EntityFrameworkCore. In EntityFramework (EF6), lazy loading is enabled by default, However, in EntityFrameworkCore, lazy loading related entities is handled by a separate package Microsoft.EntityFrameworkCore.Proxies.
To enable the behaviour you are seeking, install the above package and add the following code
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseLazyLoadingProxies();
}
After this, the related entities will be loaded without the Include call.
Below is a class I have used to generate a table in my database using Entity Framework. I'd like to be able to link this table to another table, Property. However, the way my code is set up there is not an Id column in the Instruction table, there is a Property property within the class, which then generates a PropertyId column in the actual database, but since the Property property is not an Id I am unable to using Linq to join these tables.
Instruction table
[Table("Instruction")]
public class Instruction
{
[Key]
public int Id { get; set; }
public InstructionTypes InstructionType { get; set; }
public Property Property { get; set; } //Generates the EF property FK, but is not an ID so therefore cannot be used in linq.
}
Property table
[Table("Property")]
public partial class Property
{
[Key]
public int Id { get; set; }
public Address Correspondence { get; set; }
}
Join Query
var instruction =
from instructions in _context.Instructions
join properties in _context.Properties on instructions.Property equals properties.Id
where ...
The above query gives a compiler error of: `The type of one of the expressions in the join clause is incorrect.
This error is being generated as I'm attempting to use a property object to join with a propertyId.
How can I alter this query so that I am able to join these two tables?
In 99% of all cases, you do not want to use the join operator. Entity Framework automatically generates SQL JOINS for you when you are using Navigation Properties.
var instruction = await _context.Instructions.Where(i => i.Property...).FirstOrDefaultAsync().ConfigureAwait(false);
Note, that depending on whether you are using EF6 or EF Core or with different configuration, Lazy Loading may be disabled (if not, I strongly encourage you to disable it as it is a massive performance bottleneck).
So you have to use the Include Method to eagerly load the related entity.
var instruction = await _context.Instructions.Include(i => i.Property).Where(i => i.Property...).FirstOrDefaultAsync().ConfigureAwait(false);
But before doing this, think if you really need the Instruction. If not, your code could become:
var property = await _context.Properties.Where(p => p.Instructions.Any(i => ...)).FirstOrDefaultAsync().ConfigureAwait(false);
Please note that you have to extend your Property class for this to work to have a back-reference
public partial class Property
{
// No need for the Key attribute, as this is convention
public int Id { get; set; }
public Address Correspondence { get; set; }
public int CorrespondenceId { get; set; } // Not needed in this scenario, but good practice
public ICollection<Instruction> Instructions { get; } = new HashSet<Instruction>();
}
You seems to be a newcomer to linq. As such you are still thinking as if you still are in an sql world.
With linq to entities, the use of join is the exception. SQL join are generated silently by EF using the navigation properties.
So your query can be:
var instruction =
from instruction in _context.Instructions
where instruction.Porperty.Correspondence.Contains("abc");
then you can access
instruction.First().Property.Correspondence
As a good practice you can delclare the foreign keys as class members and use the fluent API to bind them.
To test you can use the following code,
//assuming that Instructions is a DbSet<Instruction>
using (var context = new MyContext() ) {
context.Instructions.Add(
new instruction {
Property = new Property {
Correspondence = new Address {}
}
});
}
using (var context = new MyContext() ) {
var c = context.Instructions.First();
console.WriteLine($"{c.Id}, {c?.Property.Id}, {c?.Property?.Correspondence.Id}");
});
I wonder if anyone can shed some light on what may be happening here. I'm using C#, MVC, with entity framework.
So I run these two lines of code:
var booboo = _context.AppItems.Where(ai => ai.id == 101);
var sql = booboo.ToString();
And I get some strange behavior. The booboo.ToString() method hangs, thus failing. Nothing about the booboo DbQuery object works properly in fact.
I'm having similar problems all over the code with my AppItem entity (AppItems is DbSet as you might guess). Entity Framework appears to be unable to construct a query for the AppItem entity.
Edit:
I wasn't patient enough! After leaving it for a very long time, I do get the following exception:
"Message=Internal error: An expression services limit has been reached. Please look for potentially complex expressions in your query, and try to simplify them."
Interestingly that's a Sql.Client exception, which I wasn't expecting.
Here's what the AppItem class looks like:
public class AppItem : Domain.Item
{
public int? UserProfileId { get; set; }
public virtual UserProfile UpdatedByUser { get; set; }
[MaxLength(50)]
public String Type { get; set;}
public DateTime UpdatedDate { get; set;}
// flags
public virtual ICollection<ItemFlag> Flags { get; set; }
// actions
public virtual ICollection<ItemAction> Actions { get; set; }
// notes
public virtual ICollection<Note> Notes { get; set; }
}
Domain Item contains a primary key field (id) and a few other fields.
The Note / ItemAction / ItemFlag Classes there all inherit from AppItem, so perhaps some sort of circular referencing is to blame?
Other items can be queried just fine. For example, I have numerous classes that inherit from AppItem (like ItemFlag, ItemAction and Note) and I can query all of these just fine.
So, where Members is DbSet and Member inherits from AppItem:
var foofoo = _context.Members.Where(ai => ai.id = 101);
var sql = foofoo.ToString();
This Works fine; foofoo.ToString() returns the constructed SQL and everything appears to be in order.
It seems really bizarre to me, there's no error message or anything, the application just hangs when it tries to query AppItems. The table exists in the database, but that doesn't matter because we aren't getting as far as querying the database, we are failing to construct a query in the first place.
Thanks in advance for any help.
I found what the problem was.
I'm using Table-per-Type for inheritance. With AppItem being a base type, the SQL query it generates for querying it is huge (several thousand lines long in this case) and causes problems.
So basically, you need to avoid querying on base types that have more than a few types inheriting from them when using Table-per-Type.
I am assuming that your query is meant to return 1 item.
Add .FirstOrDefault(); onto the end of your query to only return one item (your current query returns an IQueriable of type AppItems)
The Entity Framework does not execute the query until needed, so in your code it will execute the query when the .ToString() method is called.
Try this:
var booboo = _context.AppItems.Where(ai => ai.id == 101).FirstOrDefault();
if (booboo != null)
{
var sql = booboo.ToString();
//etc
}
I am new to EF. I am trying to get Entity Framework 4.2 to do a sort by a calculated property (not mapped).
Here is what my entity look like:
public class Site : Entity
{
public Site()
{
Equipments = new HashSet<Equipment>();
Forecasts = new HashSet<Forecast>();
}
[StringLength(8)]
public string Number { get; set; }
[StringLength(50)]
public string EquipmentShortCLLI { get; set; }
[StringLength(50)]
public string Location { get; set; }
public virtual Central Central { get; set; }
public virtual ICollection<Equipment> Equipments { get; set; }
public virtual ICollection<Forecast> Forecasts { get; set; }
#region Calculated Items
public bool IsEmbargo {
get { return Equipments.Count > 0 && Equipments.SelectMany(x => x.EquipmentDetails).Any(e => e.IsEmbargo); }
}
//...
public int PortsCapacity
{
get
{
return Equipments.Count > 0
? Equipments.SelectMany(x => x.Slots).Sum(x => x.PortsCapacity)
: 0;
}
}
#endregion
//...
By trying to order using any of my readonly properties I am getting the exception:
The specified type member 'PortsCapacity' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.
Which makes sense because EF is trying to build an sql orderby with a field that does not exist in the database (my understanding..).
Now, by using some dynamic linq code I was able to make this work for my many-to-one columns by passing "Central.SomeField" (as opposed to making a ReadOnly Property that returns Central.SomeField).
I.E.:
query.OrderBy("Central.SomeField");
However, I still face the same issue when it comes to a collection of items (Equipments). I am trying to make this as dynamic as possible by using a string coming from the client side and avoiding a long switch case, but at this point I will accept any ideas, so long as the sorting happens on the database side.
Edit 1:
Following what Ladislav Mrnka says, how would one execute an OrderBy clause on one-to-many child items using lambdas or expression?
I don't think that Dynamic Linq is capable of this. You need a real Linq subquery to compute aggregations on Equipements so it will simply not work. If the user selects ordering by IsEmbargo or PortsCapacity you must have some switch / if block to handle this case by appending special part of the query - no other way.