Develop Reference

Entity Framework Core 3.1.5: How to take a table collection logic out side of a linq query to avoid client-side evaluation error?

I've a query like this
public IEnumerable<ContractInquiry> ContractInquiry(Guid itemSoldID)
{
IEnumerable<ContractInquiry> result;
using (var context = new ContractDbContext(_ctxOptions))
{
var qry = from con in context.Contracts
join product in context.ContractProducts on con.ID equals product.ContractID
join service in context.ServiceDetails on con.ID equals service.ContractID into tmpService
from service in tmpService.DefaultIfEmpty()
where product.ItemSoldID == itemSoldID
&& product.ProductStatus != ProductStatus.Deleted.ToString()
&& con.Status != Status.Deleted.ToString()
select new ContractInquiry
{
ServiceID = con.ID,
ServiceType = con.ServiceType,
ServiceDate = service.ServiceDate,
ServiceNumber = service.ServiceNumber,
ServiceManager = con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault()
};
result = qry.ToList();
}
return result;
}
This query was working fine. But when we upgraded to .NET Core 3.1.5 and Entity Framework Core 3.1.5, it started throwing a client-side evaluation error:
"could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()."
So I had to take the following line out from the query:
ServiceManager = con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault()
So re-wrote the query like this:
public IEnumerable<ContractInquiry> ContractInquiry(Guid itemSoldID)
{
List<ContractInquiry> result;
using (var context = new ContractDbContext(_ctxOptions))
{
var result = (from con in context.Contracts
join product in context.ContractProducts on con.ID equals product.ContractID
join service in context.ServiceDetails on con.ID equals service.ContractID into tmpService
from service in tmpService.DefaultIfEmpty()
where product.ItemSoldID == itemSoldID
&& product.ProductStatus != ProductStatus.Deleted.ToString()
&& con.Status != Status.Deleted.ToString()
select new ContractInquiry
{
ServiceID = con.ID,
ServiceType = con.ServiceType,
ServiceDate = service.ServiceDate,
ServiceNumber = service.ServiceNumber,
Contacts = con.Contacts
}).ToList();
}
result.ForEach(con => con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault();
return result;
}
Here
con.Contacts
is a table collection in Contract.cs class
I've added a property like this in ContractInquiry.cs class:
[JsonIgnore]
public IEnumerable<Contact> Contacts { set; get; }
This is working fine as well.
Question:
Doing like this will work fine but at run time, the table collection "con.Contacts" will be in memory right? And that will impact the performance of the query right if the table is a huge collection? So is there a work around for this instead of using a memory table? How can I take out the "ServiceManager = .." from the select clause in my first query?
UPDATE: Can someone answer my question?

To answer your question:
No the whole Contacts table won't be loaded into memory.
It will be slower than using a database query but unless you have a crazy amount of records you won't be able to 'humanly' measure it (obv. a stress test will point out that this may be slower by 150ms on 10000 records).
Why this is:
EF Core only loads related data and when it is needed. For example you have 1000 of these ContractInquiry records when calling .ToList(). Every one of these records contain ten contacts. Then EF Core will load only 1000*10 contacts. Due to references if any of these overlap they will share memory location and only a reference to it will be saved.
Some changes you can do to make this even faster:
Change .ToList() to .AsEnumerable(). You can do this because you only iterate over that list once, so you save a whole iteration using .AsEnumerable(). (to create a list the program must iterate over it and then you iterate over it again). Also you are returning an IEnumerable so creating a list is pointless (if you are iterating over it once, which is the case here) unless you later cast it back, which I do not recommend.
Add .AsNoTracking() in the query. I don't know how you can achieve the same thing with this type of querying (I only use Linq). This will save a lot of time because EF Core will not have to create tracking (and will also save memory).
If you would change the query to a Linq query I would be happy to have a look at it and help you optimise it.

Return result of primary table queried then append result of join to a property

I'm trying to tell .NET to create a new object, take properties from the first data set and append the result of a join to one of the properties of that newly created object
var projects = from p in projectSet //projectSet is DbSet<Project>....
join lmp in LMProjects on p.ProjectID equals lmp.lmp_ProjectID
where p.UserID == value
select new Project { LMProject = lmp };
Here's the rub, I can tell it to create a new object, assign the result of the join (lmp) to one of it's property, but how to I tell it to use the result of p to initiate all the other values of the newly created object ? Is that even possible without manually assigning values (there are about 30 in Project class...)
I thought of doing this, but that's not exactly what I'm looking for.
var projects = from p in projectSet
join lmp in LMProjects on p.ProjectID equals lmp.lmp_ProjectID
where p.UserID == value
select new { Project = p, LMProject = lmp };
The reason I'm doing this is I've got a database with no foreign keys and I'm trying to emulate navigation properties. (I can't change the model)

Even if you want to manually list all the fields in the select clause, you cannot do that directly because Entity Framework does not allow projection to entity types.
The only possible way is to combine your second approach to execute LINQ to Entities query agains the database with switching to LINQ to Objects context and doing the "fixup" and final projection like this:
var query = from p in projectSet
join lmp in LMProjects on p.ProjectID equals lmp.lmp_ProjectID
where p.UserID == value
select new { Project = p, LMProject = lmp };
var projects = query.AsEnumerable().Select(r =>
{
r.Project.LMProject = r.LMProject;
return r.Project;
};
But note that this would change the context and the type of the projects to IEnumerable<Project>. Of course you can turn it back to IQueryable<Project> by using AsQueryable method, but it will be no more a real database query, so if you apply additional filters, ordering, paging etc., all they will happen in memory (only the query part will execute against the database).

LINQ: Is there a way to combine these queries into one?

I have a database that contains 3 tables:
Phones
PhoneListings
PhoneConditions
PhoneListings has a FK from the Phones table(PhoneID), and a FK from the Phone Conditions table(conditionID)
I am working on a function that adds a Phone Listing to the user's cart, and returns all of the necessary information for the user. The phone make and model are contained in the PHONES table, and the details about the Condition are contained in the PhoneConditions table.
Currently I am using 3 queries to obtain all the neccesary information. Is there a way to combine all of this into one query?
public ActionResult phoneAdd(int listingID, int qty)
{
ShoppingBasket myBasket = new ShoppingBasket();
string BasketID = myBasket.GetBasketID(this.HttpContext);
var PhoneListingQuery = (from x in myDB.phoneListings
where x.phonelistingID == listingID
select x).Single();
var PhoneCondition = myDB.phoneConditions
.Where(x => x.conditionID == PhoneListingQuery.phonelistingID).Single();
var PhoneDataQuery = (from ph in myDB.Phones
where ph.PhoneID == PhoneListingQuery.phonePageID
select ph).SingleOrDefault();
}

You could project the result into an anonymous class, or a Tuple, or even a custom shaped entity in a single line, however the overall database performance might not be any better:
var phoneObjects = myDB.phoneListings
.Where(pl => pl.phonelistingID == listingID)
.Select(pl => new
{
PhoneListingQuery = pl,
PhoneCondition = myDB.phoneConditions
.Single(pc => pc.conditionID == pl.phonelistingID),
PhoneDataQuery = myDB.Phones
.SingleOrDefault(ph => ph.PhoneID == pl.phonePageID)
})
.Single();
// Access phoneObjects.PhoneListingQuery / PhoneCondition / PhoneDataQuery as needed
There are also slightly more compact overloads of the LINQ Single and SingleOrDefault extensions which take a predicate as a parameter, which will help reduce the code slightly.
Edit
As an alternative to multiple retrievals from the ORM DbContext, or doing explicit manual Joins, if you set up navigation relationships between entities in your model via the navigable join keys (usually the Foreign Keys in the underlying tables), you can specify the depth of fetch with an eager load, using Include:
var phoneListingWithAssociations = myDB.phoneListings
.Include(pl => pl.PhoneConditions)
.Include(pl => pl.Phones)
.Single(pl => pl.phonelistingID == listingID);
Which will return the entity graph in phoneListingWithAssociations
(Assuming foreign keys PhoneListing.phonePageID => Phones.phoneId and
PhoneCondition.conditionID => PhoneListing.phonelistingID)

You should be able to pull it all in one query with join, I think.
But as pointed out you might not achieve alot of speed from this, as you are just picking the first match and then moving on, not really doing any inner comparisons.
If you know there exist atleast one data point in each table then you might aswell pull all at the same time. if not then waiting with the "sub queries" is nice as done by StuartLC.
var Phone = (from a in myDB.phoneListings
join b in myDB.phoneConditions on a.phonelistingID equals b.conditionID
join c in ph in myDB.Phones on a.phonePageID equals c.PhoneID
where
a.phonelistingID == listingID
select new {
Listing = a,
Condition = b,
Data = c
}).FirstOrDefault();
FirstOrDefault because single throws error if there exists more than one element.

Retrieving entities from a one to many relationship (Odata)

Summarizing, I have two main tables: Company and Employees, with a one-to-many relationship between them: employees belongs to a company.
The Company entity has a property called Employees, which allows to get the employees who belongs to the specific Company.
If I type in the browser this URL, it works and I get an employees list:
http://domain.com/DynamicsNAV80/OData/Company('whatever')/Employees
Now, I want to retrieve the employees using a Linq query, how can I do it?
I have tried this:
var dataServiceQuery = (DataServiceQuery<Company>)from comp in _context.Company.Expand(comp => comp.WhseEmployee)
where comp.Name == "whatever"
select comp.WhseEmployee;
But this is not working for me.

What does that query return, an error or just not the data your looking for? Im not sure if the syntax for querying Odata is different but this is how i would do it any other time.
var dataServiceQuery = from comp in _context.Company.Expand("WhseEmployees")
where comp.Name == "whatever"
select comp;

Which version of OData are you using?
If it is V4. You can try following code.
var employees = _context.Company.ByKey("whatever").WhseEmployee;
Please refer to Client Delayed Query
If it is V3. You need to query the company first, and then use LoadProperty to send request to /Company('whatever')/WsheEmployee.
var company = _context.Company.Where(c=>c.Name="whatever").First();
dsc.LoadProperty(company, "WsheEmployee");

Finally I could do with this query:
var dataServiceQuery = (DataServiceQuery<WhseEmployee>)_context.Company.Where(c => c.Name == companyName)
.SelectMany(c => c.WhseEmployee);

Getting Entity Framework to eager load on Group By

I know that changing the shape of a query causes Entity Framework to ignore the include calls but is there a way I can get it to load the sub properties when I do a select many and a group by. In the following example I want to notify all the employees who have a job booked in a certain time period. Calling .ToArray() after the where only hits the database once but I am doing the SelectMany and GroupBy in memory. Is there a way I can get the SelectMany and the GroupBy to happen on the SQL server and still include the ServiceType and Ship and the Employee details?
I am looking for a way to make one SQL call to the database and end up with a list of Employees who have a job in the time period and the jobs they are assigned to.
var employeeJobs = DataContext.Jobs.
Include("ServiceType").
Include("Ship").
Include("JobEmployees.Employee").
Where(j => j.Start >= now && j.Start <= finish).
OrderBy(j => j.Start).
ToArray().
SelectMany(j => j.JobEmployees, (j, je) => new {
Job = j,
Employee = je.Employee
}).GroupBy(j => j.Employee);

The following should work:
var q = from e in DataContext.Employees
where e.Job.Start > ....
order by e.Job.Start
select new {Employee = e, Job = e.Job, Ship = e.Job.Ship, ServiceType = e.Job.ServiceType}; // No db hit yet.
var l = q.GroupBy(item=>item.Employee) // no db hit yet.
.ToList(); // This one causes a db hit.

why don't you create a view and then reference this from the EF?
, this also has the added benefit of the database server doing the work, rather than the app server.

Trying move the Include()'s to the very end after your groupby.