Entity LINQ query to slow - c#

I'm new to the C#, I have a database that someone else designed, query works great, but compared with SQL, it's 10 times slower.
I made mistakes here for sure, anybody have tips to speed this up a little bit. This model is for displaying in table, and I converting int to ENUM and calculating discount for display.
Code is:
var results = from w in db.Washes.AsEnumerable()
join t in db.Wash_Types.AsEnumerable() on w.WashTypeId equals t.Id
join a in db.Accounts.AsEnumerable() on w.AccountId equals a.Id
orderby w.Id descending
select new AllWashesTable
{
Id = w.Id,
WashTime = w.WashTime,
WashTimeEnd = w.WashTimeEnd,
Name = a.Name,
Client = (w.Client != null ? w.Client.Naziv : ""),
MobileNumber = a.MobileNumber,
Identification = w.Identification,
WashType = WashTypeShowEnum.WashTypeShowEnumToString((WashTypeShowEnum.WashType) w.WashTypeId),
Price = int.Parse(t.WashPrice) * (1 - w.Discount) + "",
Discount = w.Discount
};
return results.ToList();
Seems all my entity queries are at least 5+ times slower than SQL. Somewhere I am making some mistake.

Your problem is the use of AsEnumerable. When the query gets executed (in your case the results.ToList()), Everything that appears after it, will be evaluated using linq-to-objects. It means that your joins will not be handled by DB. You will fetch all the records from the tables.
However, your function WashTypeShowEnum.WashTypeShowEnumToString will not be recognized by entity framework.
You might want to move the AsEnumerable to the end and then select the results.
var results = (from w in db.Washes
join t in db.Wash_Types on w.WashTypeId equals t.Id
join a in db.Accounts on w.AccountId equals a.Id
orderby w.Id descending
select new {w, a, t}).AsEnumerable().Select(arg=> new AllWashesTable
{
Id = arg.w.Id,
WashTime = arg.w.WashTime,
WashTimeEnd = arg.w.WashTimeEnd,
Name = arg.a.Name,
Client = (arg.w.Client != null ? arg.w.Client.Naziv : ""),
MobileNumber = arg.a.MobileNumber,
Identification = arg.w.Identification,
WashType = WashTypeShowEnum.WashTypeShowEnumToString((WashTypeShowEnum.WashType) arg.w.WashTypeId),
Price = int.Parse(arg.t.WashPrice) * (1 - arg.w.Discount) + "",
Discount = arg.w.Discount
};
return results.ToList();

Related

How to handle null values in linq expression with joins? [duplicate]

This question already has answers here:
Sequence contains no elements?
(7 answers)
Closed 10 months ago.
I try to get some data from few table with joining. This query is give an error as System.InvalidOperationException , Message=" Sequence contains no elements ". How can I avoid this error.
var assignments = (from s in _db.SubmissionLinks
join a in _db.Assignments on s.AssignmentID equals a.AssignmentID
join p in _db.Projects on a.ProjectID equals p.ID
join s2 in _db.SystemUsers on p.SystemUserFK equals s2.ID
select new AssignmentViewModel()
{
SubmissionlinkID = s.SubmissionLinkID,
SubmissionlinkName = s.SubmissionName,
ProjectID = p.ID,
ProjectName = p.ProjectName,
Deadline = s.Deadline,
Userid = s2.ID,
Assignmentid = a.AssignmentID,
IsActive = s.ActiveStatus
}).Where(s =>s.Userid == ID && s.IsActive == 0).Distinct().First();
I try to avoid that by checking returning objects inside of the where clause but it didn't work
var assignments = (from s in _db.SubmissionLinks
join a in _db.Assignments on s.AssignmentID equals a.AssignmentID
join p in _db.Projects on a.ProjectID equals p.ID
join s2 in _db.SystemUsers on p.SystemUserFK equals s2.ID
select new AssignmentViewModel()
{
SubmissionlinkID = s.SubmissionLinkID,
SubmissionlinkName = s.SubmissionName,
ProjectID = p.ID,
ProjectName = p.ProjectName,
Deadline = s.Deadline,
Userid = s2.ID,
Assignmentid = a.AssignmentID,
IsActive = s.ActiveStatus
}).Where(s => s != null && s.Userid == ID && s.IsActive == 0).Distinct().First();
Agree with Panagiotis; its supposed to be that SubmissionLink has a .Assignment property that returns the parent Assignment, Assignment has a .Project prop that returns the Project etc, and then you just do:
_db.SubmissionLinks
.Where(s =>s.Userid == ID && s.IsActive == 0)
.Select(s => s.Select(s => new AssignmentViewModel()
{
SubmissionlinkID = s.SubmissionLinkID,
SubmissionlinkName = s.SubmissionName,
ProjectID = s.Assignment.Project.ID,
ProjectName = s.Assignment.Project.ProjectName,
Deadline = s.Deadline,
Userid = s.Assignment.Project.SystemUser.ID,
Assignmentid = s.Assignment.AssignmentID,
IsActive = s.ActiveStatus
})
.SomethingThatRunsTheQueryLike_First_FirstOrDefault_ToList_etc
EF works out the rest from how it seens you use the navigations (s.Assignment.Project.ProjectName etc). Starting from the many end and going up to the one end is quite easy, because it's just a single object at the end of the nav. Going down through collections is a bit more of a nuisance, so think carefully about where you start your query from for the easiest life. For example if you wanted the Project and its most recent submission, you might:
db.Projects.Select(p => new Something{
p.ProjectName,
MostRecentLinkId = p.SubmissionLinks.OrderByDescending(sl => sl.CreatedDate).First().Id
})
That p.SubmissionLinks.OrderByDescending(sl => sl.CreatedDate).First().Id would be translated (by recent EFC) to something like ... ROW_NUMBER() OVER(PARTITION BY projectid ORDER BY createddate DESC) rn ... WHERE rn=1 in a subquery, but working with collections is usually that little more tricky because you're performing aggregations to make them make sense in the context of a single parent. If you can start from a different place so youre not digging into collections it's simpler

Linq Join query returning empty dataset

I am using below code to join two tables based on officeId field. Its retuning 0 records.
IQueryable<Usage> usages = this.context.Usage;
usages = usages.Where(usage => usage.OfficeId == officeId);
var agencyList = this.context.Agencies.ToList();
var usage = usages.ToList();
var query = usage.Join(agencyList,
r => r.OfficeId,
a => a.OfficeId,
(r, a) => new UsageAgencyApiModel () {
Id = r.Id,
Product = r.Product,
Chain = a.Chain,
Name = a.Name
}).ToList();
I have 1000+ records in agencies table and 26 records in usage table.
I am expecting 26 records as a result with chain and name colums attached to result from agency table.
Its not returning anything. I am new to .net please guide me if I am missing anything
EDIT
#Tim Schmelter's solution works fine if I get both table context while executing join. But I need to add filter on top of usage table before applying join
IQueryable<Usage> usages = this.context.Usage;
usages = usages.Where(usage => usage.OfficeId == officeId);
var query = from a in usages
// works with this.context.usages instead of usages
join u in this.context.Agencies on a.OfficeId equals u.OfficeId
select new
{
Id = a.Id,
Product = a.Product,
Chain = u.Chain,
Name = u.Name
};
return query.ToList();
Attaching screenshot here
same join query works fine with in memory data as you see below
Both ways works fine if I add in memory datasource or both datasource directly. But not working if I add filter on usages based on officeId before applying join query
One problem ist that you load all into memory first(ToList()).
With joins i prefer query syntax, it is less verbose:
var query = from a in this.context.Agencies
join u in this.context.Usage on a.OfficeId equals u.OfficeId
select new UsageAgencyApiModel()
{
Id = u.Id,
Product = u.Product,
Chain = a.Chain,
Name = a.Name
};
List<UsageAgencyApiModel> resultList = query.ToList();
Edit: You should be able to apply the Where after the Join. If you still don't get records there are no matching:
var query = from a in this.context.Agencies
join u in this.context.Usage on a.OfficeId equals u.OfficeId
where u.OfficeId == officeId
select new UsageAgencyApiModel{ ... };
The following code can help to get the output based on the ID value.
Of course, I wrote with Lambda.
var officeId = 1;
var query = context.Agencies // your starting point - table in the "from" statement
.Join(database.context.Usage, // the source table of the inner join
agency => agency.OfficeId, // Select the primary key (the first part of the "on" clause in an sql "join" statement)
usage => usage.OfficeId , // Select the foreign key (the second part of the "on" clause)
(agency, usage) => new {Agency = agency, Usage = usage }) // selection
.Where(x => x.Agency.OfficeId == id); // where statement

Entity Framework Core 2.2 orderby evaluated locally

I have this semi-complex query that counts the most voted user post of the last 7 days:
var fromDate = dateTimeService.Now.Add(-interval);
var votedPosts =
from vote in dbContext.Votes
join post in dbContext.Posts on vote.PostId equals post.PostId
group new {vote.Sign, post.PostId} by post.PostId
into postVotes
select new {
PostId = postVotes.Key,
TotalRating = postVotes.Sum(pv => pv.Sign)
};
var bestPost = (
from post in dbContext.Posts
join votedPost in votedPosts on post.PostId equals votedPost.PostId
join room in dbContext.Rooms on post.RoomId equals room.RoomId
join game in dbContext.Modules on room.ModuleId equals game.ModuleId
where room.RoomAccess == RoomAccessType.Open && post.CreateDate > fromDate
orderby votedPost.TotalRating descending,
post.CreateDate descending
select new BestPost
{
UserId = post.UserId,
ModuleId = game.ModuleId,
ModuleTitle = game.Title,
PostId = post.PostId,
PostText = post.Text,
PostCommentary = post.Commentary,
PostCreateDate = post.CreateDate,
TotalRating = bestPost.TotalRating
}).FirstOrDefault();
What I try to do here is to group user votes by PostId, sum the evaluations of their votes by field Sign (can be -1, 0 or 1), then join it with some additional data like game Id/Title and post texts, filter non-public or too old posts, then order it by rank and then by create date, then map it onto DTO and return the very first result if present.
All the fields here are simple basic types: the Vote.Sign is int, Post.CreateDate is DateTime, all the *Id are Guid and Text/Title/Commentary are string.
I get the warning:
warn: Microsoft.EntityFrameworkCore.Query[20500]
The LINQ expression 'orderby [bestPost].TotalRating desc' could not be translated and will be evaluated locally.
warn: Microsoft.EntityFrameworkCore.Query[20500]
The LINQ expression 'FirstOrDefault()' could not be translated and will be evaluated locally.
If I remove the sort by TotalRating and only leave the CreateDate sorting, it works fine, creates proper LIMIT request. But with TotalRating the query looks like this:
SELECT
t."PostId", t."TotalRating", post."CreateDate" AS "PostCreateDate",
post."UserId", game."ModuleId", game."Title" AS "ModuleTitle",
post."PostId" AS "PostId0", post."Text" AS "PostText",
post."Commentary" AS "PostCommentary"
FROM
"Posts" AS post
INNER JOIN
(SELECT
post0."PostId", SUM(vote."Sign")::INT AS "TotalRating"
FROM
"Votes" AS vote
INNER JOIN
"Posts" AS post0 ON vote."PostId" = post0."PostId"
GROUP BY
post0."PostId") AS t ON post."PostId" = t."PostId"
INNER JOIN
"Rooms" AS room ON post."RoomId" = room."RoomId"
INNER JOIN
"Modules" AS game ON room."ModuleId" = game."ModuleId"
WHERE
(room."RoomAccess" = 0) AND (post."CreateDate" > #__fromDate_0)
And it looks pretty bad to be calculated in dotnet runtime.
I tried to wrap the result in another select from, but it didn't help. I also cannot do the group by on all the columns because then I won't be able to aggregate things like ModuleId because EF Core 2.2 does not support the group.FirstOrDefault things and PostgreSQL does not support max(uuid) (otherwise I could use group.Max(g => g.ModuleId)).
What am I doing wrong?
What happens if you combine the gist of votedPosts into the query so you don't duplicate the join on posts?
var bestPost = (
from post in dbContext.Posts
join vote in dbContext.Votes on post.PostId equals vote.PostId into votej
let voteTotalRating = votej.Sum(pv => pv.Sign)
join room in dbContext.Rooms on post.RoomId equals room.RoomId
join game in dbContext.Modules on room.ModuleId equals game.ModuleId
where room.RoomAccess == RoomAccessType.Open && post.CreateDate > fromDate
orderby voteTotalRating descending,
post.CreateDate descending
select new BestPost {
UserId = post.UserId,
ModuleId = game.ModuleId,
ModuleTitle = game.Title,
PostId = post.PostId,
PostText = post.Text,
PostCommentary = post.Commentary,
PostCreateDate = post.CreateDate,
TotalRating = voteTotalRating
}).FirstOrDefault();

LINQ query ideas for this table structure

I know it's a rather unusual question, but I've been spending a bit of time trying to figure out the fastest way to retrieve data from an unusually structured database. Basically, the database looks like this:
'Trip' - nice and simply structured table, no issues here
'Consignments' - each consignment links to a trip above. The interesting, and annoying, thing here is that each consignment has 2 records in the table one for the 'Sales' value, and one for the 'Costs' value - these are differentiated by a Char field ('S' or 'C').
My query basically needs to select all consignments between a date range (the date is stored on the Trip table), with the sales and costs value in the row. Here's my LINQ query now:
var cons = from cS in dc.Consignments.Where(a => a.BreakdownType == 'S')
join cC in dc.Consignments on new { NTConsignment = cS.NTConsignment, LegacyID = cS.LegacyID, TripDate = cS.TripDate, Depot = cS.Depot, TripNumber = cS.TripNumber, BreakdownType = 'C' } equals new { NTConsingment = cC.NTConsignment, LegacyID = cC.LegacyID, TripDate = cC.TripDate, Depot = cC.Depot, TripNumber = cC.TripNumber, BreakdownType = cC.BreakdownType }
join sl in dc.SageAccounts on new { LegacyID = cS.Customer, Customer = true } equals new { LegacyID = sl.LegacyID, Customer = sl.Customer }
join ss in dc.SageAccounts on sl.ParentAccount equals ss.ID
join vt in dc.VehicleTypes on cS.Trip.VehicleType.Trim() equals vt.ID.ToString() into vtg
where cS.Trip.DeliveryDate >= dates.FromDate && cS.DeliveryDate <= dates.ToDate
orderby cS.Depot, cS.TripNumber
select new GMConsignment
{
NTConsignment = cS.NTConsignment,
Trip = cS.Trip,
LegacyID = cS.LegacyID,
Costs = cC.Value ?? 0.00m,
Sales = cS.Value ?? 0.00m,
Margin = (cS.Value ?? 0.00m) - (cC.Value ?? 0.00m),
Customer = cmS.Customer,
SageID = ss.SageID,
CustomerName = ss.ShortName,
FullCustomerName = ss.Name,
Vehicle = cS.Trip.Vehicle ?? "None",
VehicleType = vtg.SingleOrDefault().VehicleTypeDescription ?? "Subcontractor",
VehicleGroup = vtg.FirstOrDefault().VehicleGroup1
};
The problem is, the above is horrifically slow. I have profiled the SQL, and there are no obvious mistakes I'm making such as missing indexes. I know there are a million possibilities why the query is slow, and it's very difficult without knowing more about it, but I'm just wondering if there are any different, and potentially faster ways of doing what I'm doing in LINQ?
I've also tried selecting the 'Cost' in the 'Select' statement (i.e Costs = dc.Costs.Where...blah blah) but it's even slower.

Linq to Entities Left Outer Join with different types

I've probably spent 40 hours on this problem so far, I've tried every solution on this site and on google, and I still can't make this work.
I need to left join a table to the results of a previous query, stored in a var. The joining field is a varchar in the table queried for the result in the var, and a bigint (long) in the table being joined. Here is the current attempt, which tells me "Object reference not set to an instance of an object." All Entities errors seem like nonsense and lies to me, I assume it's trying to tell me nothing matched, but who knows.
List<reportUser> ru = leaders
.GroupJoin(db.sweeps,
a => a.FBID.ToString(),
s => s.userFBID.First().ToString(),
(a, matching) => new reportUser
{
FBID = a.FBID,
CurrentPoints = a.CurrentPoints,
Name = matching.FirstOrDefault().Name,
Email = matching.FirstOrDefault().email
}
?? new reportUser
{
FBID = 0,
CurrentPoints = 0,
Name = "",
Email = ""
})
.Select(a => a)
.ToList();
Here's the SQL requested below. I've included the SQL to build the Leaders object as well, all the above is really meant to represent is the last line, which is simply a left join.
select s.name, s.email, b.score, c.score overall
from (
select a.userfbid, sum(a.pointvalue) score
from (
select userfbid, pointvalue
from l
left join qa on qa.id = l.qaid
left join q on q.id = qa.qid
left join qz on qz.id = q.qzid
where qa.pointvalue > 0 and qz.cid = 12
union all
select fbid userfbid, pointvalue
from bn
where date >= '5/9/2011 04:00' and
date <= '5/16/2011 04:00'
) a
group by a.userfbid
) b
left join (
select a.userfbid, sum(a.pointvalue) score
from (
select userfbid, pointvalue
from l
left join qa on qa.id = l.qaid
left join q on q.id = qa.qid
left join qz on qz.id = q.qzid
where qa.pointvalue > 0
union all
select fbid userfbid, pointvalue
from bn
) a
group by a.userfbid
) c on c.userfbid=b.userfbid
left join s on s.userfbid=b.userfbid
order by score desc
I'm assuming that in your database s.userFBID.First() is never null?
If that's right, then your problem could be in the FirstOrDefault().Name type statements - when FirstOrDefault() evaluates to null then obviously you will get a nullreferenceexception :/
To get around this, try something like:
List<reportUser> ru = leaders
.GroupJoin(db.sweeps,
a => a.FBID.ToString(),
s => s.userFBID.First().ToString(),
(a, matching) =>
{
var match = matching.FirstOrDefault();
return match != null ?
new reportUser
{
FBID = a.FBID,
CurrentPoints = a.CurrentPoints,
Name = match.Name,
Email = match.email
}
: new reportUser
{
FBID = 0, // a.FBID ?
CurrentPoints = 0, // a.CurrentPoints ?
Name = "",
Email = ""
}})
.Select(a => a)
.ToList();
However, I find it a bit hard to do this without seeing the structure of the database... or some sample data
Once you've got something working... then I highly recommend you try breaking this down into something more easily understandable - I'm really not sure what's going on here!
Here's a simple left outer join for you:
var query = from l leaders
join s in db.sweeps on l.FBID equals s.userFBID.First() into joined
from j in joined.FirstOrDefault()
select new reportUser
{
FBID = l.FBID,
CurrentPoints = l.CurrentPoints,
Name = j == null ? string.Empty : j.Name,
Email = j == null ? string.Empty : j.email
}
If this isn't quite what you are looking for... maybe try posting the SQL for what you actually want.

Categories

Resources