I know that changing the shape of a query causes Entity Framework to ignore the include calls but is there a way I can get it to load the sub properties when I do a select many and a group by. In the following example I want to notify all the employees who have a job booked in a certain time period. Calling .ToArray() after the where only hits the database once but I am doing the SelectMany and GroupBy in memory. Is there a way I can get the SelectMany and the GroupBy to happen on the SQL server and still include the ServiceType and Ship and the Employee details?
I am looking for a way to make one SQL call to the database and end up with a list of Employees who have a job in the time period and the jobs they are assigned to.
var employeeJobs = DataContext.Jobs.
Include("ServiceType").
Include("Ship").
Include("JobEmployees.Employee").
Where(j => j.Start >= now && j.Start <= finish).
OrderBy(j => j.Start).
ToArray().
SelectMany(j => j.JobEmployees, (j, je) => new {
Job = j,
Employee = je.Employee
}).GroupBy(j => j.Employee);
The following should work:
var q = from e in DataContext.Employees
where e.Job.Start > ....
order by e.Job.Start
select new {Employee = e, Job = e.Job, Ship = e.Job.Ship, ServiceType = e.Job.ServiceType}; // No db hit yet.
var l = q.GroupBy(item=>item.Employee) // no db hit yet.
.ToList(); // This one causes a db hit.
why don't you create a view and then reference this from the EF?
, this also has the added benefit of the database server doing the work, rather than the app server.
Trying move the Include()'s to the very end after your groupby.
Related
I've a query like this
public IEnumerable<ContractInquiry> ContractInquiry(Guid itemSoldID)
{
IEnumerable<ContractInquiry> result;
using (var context = new ContractDbContext(_ctxOptions))
{
var qry = from con in context.Contracts
join product in context.ContractProducts on con.ID equals product.ContractID
join service in context.ServiceDetails on con.ID equals service.ContractID into tmpService
from service in tmpService.DefaultIfEmpty()
where product.ItemSoldID == itemSoldID
&& product.ProductStatus != ProductStatus.Deleted.ToString()
&& con.Status != Status.Deleted.ToString()
select new ContractInquiry
{
ServiceID = con.ID,
ServiceType = con.ServiceType,
ServiceDate = service.ServiceDate,
ServiceNumber = service.ServiceNumber,
ServiceManager = con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault()
};
result = qry.ToList();
}
return result;
}
This query was working fine. But when we upgraded to .NET Core 3.1.5 and Entity Framework Core 3.1.5, it started throwing a client-side evaluation error:
"could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()."
So I had to take the following line out from the query:
ServiceManager = con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault()
So re-wrote the query like this:
public IEnumerable<ContractInquiry> ContractInquiry(Guid itemSoldID)
{
List<ContractInquiry> result;
using (var context = new ContractDbContext(_ctxOptions))
{
var result = (from con in context.Contracts
join product in context.ContractProducts on con.ID equals product.ContractID
join service in context.ServiceDetails on con.ID equals service.ContractID into tmpService
from service in tmpService.DefaultIfEmpty()
where product.ItemSoldID == itemSoldID
&& product.ProductStatus != ProductStatus.Deleted.ToString()
&& con.Status != Status.Deleted.ToString()
select new ContractInquiry
{
ServiceID = con.ID,
ServiceType = con.ServiceType,
ServiceDate = service.ServiceDate,
ServiceNumber = service.ServiceNumber,
Contacts = con.Contacts
}).ToList();
}
result.ForEach(con => con.Contacts.Where(q => q.Role.Contains(ContractRole.ServiceManager.ToString()))
.OrderBy(o => o.ID).FirstOrDefault();
return result;
}
Here
con.Contacts
is a table collection in Contract.cs class
I've added a property like this in ContractInquiry.cs class:
[JsonIgnore]
public IEnumerable<Contact> Contacts { set; get; }
This is working fine as well.
Question:
Doing like this will work fine but at run time, the table collection "con.Contacts" will be in memory right? And that will impact the performance of the query right if the table is a huge collection? So is there a work around for this instead of using a memory table? How can I take out the "ServiceManager = .." from the select clause in my first query?
UPDATE: Can someone answer my question?
To answer your question:
No the whole Contacts table won't be loaded into memory.
It will be slower than using a database query but unless you have a crazy amount of records you won't be able to 'humanly' measure it (obv. a stress test will point out that this may be slower by 150ms on 10000 records).
Why this is:
EF Core only loads related data and when it is needed. For example you have 1000 of these ContractInquiry records when calling .ToList(). Every one of these records contain ten contacts. Then EF Core will load only 1000*10 contacts. Due to references if any of these overlap they will share memory location and only a reference to it will be saved.
Some changes you can do to make this even faster:
Change .ToList() to .AsEnumerable(). You can do this because you only iterate over that list once, so you save a whole iteration using .AsEnumerable(). (to create a list the program must iterate over it and then you iterate over it again). Also you are returning an IEnumerable so creating a list is pointless (if you are iterating over it once, which is the case here) unless you later cast it back, which I do not recommend.
Add .AsNoTracking() in the query. I don't know how you can achieve the same thing with this type of querying (I only use Linq). This will save a lot of time because EF Core will not have to create tracking (and will also save memory).
If you would change the query to a Linq query I would be happy to have a look at it and help you optimise it.
I'm working on Schools System, and the System have many Users(teachers And Managers).
(Tools)
Asp.net 5 (MVC With C#) VS 2015
SQL SERVER (Azure)
Code-First Approach
Each user Assigned to One Or More Of Classes.
(The User Should only See His Classes And Students inside of Classes)
in the other side
to catch Each User Classes
I've write this Method
public static IEnumerable<Class> GetClasses()
{
string UserId = HttpContext.Current.User.Identity.GetUserId();
ApplicationDbContext db = new ApplicationDbContext();
var Job = db.Jobs.Where(j => j.UserId == UserId).FirstOrDefault();
if (Job.EntityLevelID == 1)
{
var x = db.Classes.Where(a => a.Levels.SupervisionCenter.Organization.Id == Job.EntityID) as IEnumerable<Class>;
return x;
}
return null;
}
in The Students List Controller I just want the User to get his Students which are Only in his Classes
to get all the Students i write this Code
var items = db.Students.AsEnumerable().Where(o => o.Name.ToLower().Contains(search)).AsQueryable();
This Code will Give all the Students, it takes around 10 Sec to load and finish all the data (But that's is not needed Bcs, i need to Filter on ClassID Field).
when i need to filter on user Permissions based-classes
I've edit the code to be
var items = db.Students.AsEnumerable().Where(o => o.Name.ToLower().Contains(search))
.Where(s => UserDB.GetClasses().Select(c => c.Id).Contains(s.ClassID.Value))
.AsQueryable();
in the Previous case, when i add the where query it takes more than 80 Sec
EDIT (1)
The Organization Chart Will Be
Organizations
SuperVisionCenter
Schools
Classes
So I don't know what is the Problem I've made it here, can you please advise me for this
Thanks And Regards.
The very first thing that is slowing you down is AsEnumerable()
See here the effects of using AsEnumerable()
Basically you are getting all students in memory then running your where clause in app memory instead of just getting selected students.
Edit 1:
o.Name.ToLower().Contains(search)
I dont think you need to do ToLower() as MSSQL is case insensitive by default. This should remove the need for db.Students.AsEnumerable()
Edit 2:
Why don't you use join ? That would be a good optimisation to begin with.
something like
List<Student> data = (from student in db.Students
join clss in db.Classes on student.Class equals clss
where student.Name.Contains(search)).ToList()
This solved My Problem, and it takes around 3 to 4 seconds only.
int[] Classes = UserDB.GetClasses().Select(c => c.Id).ToArray();
var items = db.Students.Where(o => o.Name.ToLower().Contains(search))
.Where(r => Classes.Contains(r.ClassID.Value)).AsQueryable();
I'm querying my local list against database and would like to know what's the best way to do so.
Currently I'm loading my database to memory where "options" in local List
tmp.AddRange(from course in cs.studhists.Where(x => x.year == year).AsEnumerable()
from option in options.Where(x => x.type.Equals("course"))
join stud in cs.sstudents on course.studid equals stud.studid
where
course.csid.Contains(option.identifier) && course.crsinst.Contains(option.extra_identifier)
select stud);
"studhists" table has quite a few rows and if I didn't have option to select only the ones with current year or if year was inside option object it would take a while to load to memory.
Alternatively I could just loop through every option (which worked a lot faster before I figured out to load only data with current year. I haven't timed it, but I think it still does.
foreach (OptionListItem opt in options.Where(x => x.type.Equals("course")))
{
tmp.AddRange(from course in cs.studhists
join stud in cs.sstudents on course.studid equals stud.studid
where course.year == year
&& course.csid.Contains(opt.identifier) && course.crsinst.Contains(opt.extra_identifier)
select stud);
}
Is there a way to maybe create temp table to hold "options" and query it?
Or am I completely missing some other way to do this?
Because you're using "Contains", it's difficult, and you'd probably have build a query dynamically, using PredicateBuilder:
var predicate = PredicateBuilder.False<Course>();
foreach(var opt in options.Where(x => x.type.Equals("course"))
{
predicate = predicate.Or(course.csid.Contains(opt.identifier) && course.crsinst.Contains(opt.extra_identifier));
}
(the first example on the Predicate builder page shows a similar example)
I have a database that contains 3 tables:
Phones
PhoneListings
PhoneConditions
PhoneListings has a FK from the Phones table(PhoneID), and a FK from the Phone Conditions table(conditionID)
I am working on a function that adds a Phone Listing to the user's cart, and returns all of the necessary information for the user. The phone make and model are contained in the PHONES table, and the details about the Condition are contained in the PhoneConditions table.
Currently I am using 3 queries to obtain all the neccesary information. Is there a way to combine all of this into one query?
public ActionResult phoneAdd(int listingID, int qty)
{
ShoppingBasket myBasket = new ShoppingBasket();
string BasketID = myBasket.GetBasketID(this.HttpContext);
var PhoneListingQuery = (from x in myDB.phoneListings
where x.phonelistingID == listingID
select x).Single();
var PhoneCondition = myDB.phoneConditions
.Where(x => x.conditionID == PhoneListingQuery.phonelistingID).Single();
var PhoneDataQuery = (from ph in myDB.Phones
where ph.PhoneID == PhoneListingQuery.phonePageID
select ph).SingleOrDefault();
}
You could project the result into an anonymous class, or a Tuple, or even a custom shaped entity in a single line, however the overall database performance might not be any better:
var phoneObjects = myDB.phoneListings
.Where(pl => pl.phonelistingID == listingID)
.Select(pl => new
{
PhoneListingQuery = pl,
PhoneCondition = myDB.phoneConditions
.Single(pc => pc.conditionID == pl.phonelistingID),
PhoneDataQuery = myDB.Phones
.SingleOrDefault(ph => ph.PhoneID == pl.phonePageID)
})
.Single();
// Access phoneObjects.PhoneListingQuery / PhoneCondition / PhoneDataQuery as needed
There are also slightly more compact overloads of the LINQ Single and SingleOrDefault extensions which take a predicate as a parameter, which will help reduce the code slightly.
Edit
As an alternative to multiple retrievals from the ORM DbContext, or doing explicit manual Joins, if you set up navigation relationships between entities in your model via the navigable join keys (usually the Foreign Keys in the underlying tables), you can specify the depth of fetch with an eager load, using Include:
var phoneListingWithAssociations = myDB.phoneListings
.Include(pl => pl.PhoneConditions)
.Include(pl => pl.Phones)
.Single(pl => pl.phonelistingID == listingID);
Which will return the entity graph in phoneListingWithAssociations
(Assuming foreign keys PhoneListing.phonePageID => Phones.phoneId and
PhoneCondition.conditionID => PhoneListing.phonelistingID)
You should be able to pull it all in one query with join, I think.
But as pointed out you might not achieve alot of speed from this, as you are just picking the first match and then moving on, not really doing any inner comparisons.
If you know there exist atleast one data point in each table then you might aswell pull all at the same time. if not then waiting with the "sub queries" is nice as done by StuartLC.
var Phone = (from a in myDB.phoneListings
join b in myDB.phoneConditions on a.phonelistingID equals b.conditionID
join c in ph in myDB.Phones on a.phonePageID equals c.PhoneID
where
a.phonelistingID == listingID
select new {
Listing = a,
Condition = b,
Data = c
}).FirstOrDefault();
FirstOrDefault because single throws error if there exists more than one element.
I have an Entity Framework entity Provider, with a list of rating votes for that provider. My current queries look something like this:
int previousVote = provider.ProviderRankings.FirstOrDefault(r => r.UserId == CurrUserId);
double averageVote = provider.ProviderRankings.Average(r => r.Rating);
int totalVotes = provider.ProviderRankings.Count();
This seems functionally correct. However, I believe this will result in three additional trips to the database. Is there anyway to have these requests combined into a single query such that only one SQL query will be sent, and all results can be returned with only one additional trip to the server?
You could combine the two aggregates fairly easily using a Group By:
Multiple SQL aggregate functions in a single Linq-to-Entities query
I am pretty sure the FirstOrDefault will work if you choose a suitably vague key for the grouping (for example key = 0) i.e:
from t in ProviderRankings
group t by key = 0
into g
select new {
previousVote = g.FirstOrDefault(r => r.UserId == CurrUserId),
totalVotes = g.Count(),
averageVote = g.Average(x => x.Rating)
}