Develop Reference

How to write this linq query so it isn't so slow

I have this SQL statement which is pretty instantaneous when running it:
select Distinct statuses.Description, count(*) as count
from referrals
inner join statuses on referrals.StatusId = statuses.id
group by statuses.Description
But when I run the below linq code with Entity Framework Core, it takes almost 5 minutes to run and there are only 680 rows in the database.
var data = context.Referrals
.Include(s => s.Status).AsEnumerable()
.GroupBy(r => r.Status)
.Select(g => new StatusCountItem
{
Status = g.Key.Description,
Count = g.Select(r => r).Count()
}).ToList();
Is there a way to write a similar Linq statement that won't take forever to run or do I need to figure out a different way to do what I want?
EDIT: when I don't have the AsEnumerable I get this error message which is why I added it:
The LINQ expression 'DbSet().Join(inner: DbSet(),
outerKeySelector: r => EF.Property<int?>(r, "StatusId"),
innerKeySelector: s => EF.Property<int?>(s, "Id"),
resultSelector: (o, i) => new TransparentIdentifier<Referral, Status>(Outer = o, Inner = i))
.GroupBy(r => r.Inner)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync

Try this:
var data = context.Referrals
.GroupBy(r => r.StatusId) // notice the change here, you need to group by the id
.Select(g => new StatusCountItem()
{
Status = g.First().Status.Description,
Count = g.Count()
}).ToList();

Your Sql query is built based on context.Referrals.Include(s => s.Status).AsEnumerable(), which is equivalent to:
select *
from referrals
inner join statuses on referrals.StatusId = statuses.id
Note the star, you're querying every column. In other words, remove the random AsEnumerable() in the middle of your query.

Use this one, it is simple and will improve query performance.
from r in context.Referrals
join s in context.statuses on r.StatusId equals s.Id
select new { s.Description, r.StatusId , S.Id) into result
group result by new { s.Description } into g
select new {
CompanyName = g.Key.Description,
Count = g.Count()
}

Write multiple Sum() SQL query using Lambda in Entity Framework C#

Below is my SQL query that I'm looking to convert to Lambda
SELECT
SUM("Rating" * "Rating") / SUM("Rating")
FROM
public."CustomerRating"
WHERE
"DriverId" = '232'
This is my C# code:
public double GetDriverAvgRating(string id)
{
var driver = _context.CustomerRating
.AsQueryable()
.Where(d => d.DriverId == id);
var avgDriverRating = // i need to perform that query here
return avgDriverRating;
}

I believe EF will be able to correctly translate it as a grouping, something like
var driver = _context.CustomerRating
.Where(cr => cr.DriverId == id)
.GroupBy(cr => cr.DriverId, cr => cr.Rating)
.Select(g => g.Select(r => r * r).Sum() / g.Sum())
.First();
It might have some extra fluff in the query (eg a group by that produces only one group) but I don't expect it'll make any significant difference to the overall performance/planning of the query

Why AsEnumerable also execute filter on server

I'm using Linq to Entities in my program, and i have the following two simple queries:
var result = dbContext.Customers.Where(c => c.Name == "Mostafa").ToList();
var result2 = dbContext.Customers.Where(c => c.Name == "Mostafa").AsEnumerable().ToList();
when i run the SQL Profiler, i found that the generated SQL Query is the same for both of the queries !
the questions is why in the second query, although i'm using asEnumerable, executes the filtering on the server side ?
Update:
The filtering was applied on memory instead of the server when changing the second query to be like this:
var result2 = dbContext.Customers.AsEnumerable().Where(c => c.Name == "Mostafa").ToList();
Thanks to #Willem Van Onsem

AsEnumerable() makes the remainder of the query execute locally. Anything earlier than the AsEnumerable() is still part of the IQueryable execution flow. For example, think about this (imagining an Age property):
var result = dbContext.Customers
.Where(c => c.Name == "Mostafa")
.Where(c => c.Age == 18)
.ToList();
That would result in SQL which filtered by name and age. Compare that with this:
var result = dbContext.Customers
.Where(c => c.Name == "Mostafa")
.AsEnumerable()
.Where(c => c.Age == 18)
.ToList();
That would filter by name in the SQL, but it would filter by age locally (in memory).

LINQ Multiple GroupBy Query Performing several times slower than T-SQL

I'm totally new to LINQ.
I have an SQL GroupBy which runs in barely a few milliseconds. But when I try to achieve the same thing via LINQ, it just seems awfully slow.
What I'm trying to achieve is fetch an average monthly duration of a ceratin database update.
In SQL =>
select SUBSTRING(yyyyMMdd, 0,7),
AVG (duration)
from (select (CONVERT(CHAR(8), mmud.logDateTime, 112)) as yyyyMMdd,
DateDIFF(ms, min(mmud.logDateTime), max(mmud.logDateTime)) as duration
from mydb.mydbo.updateData mmud
left
join mydb.mydbo.updateDataKeyValue mmudkv
on mmud.updateDataid = mmudkv.updateDataId
left
join mydb.mydbo.updateDataDetailKey mmuddk
on mmudkv.updateDataDetailKeyid = mmuddk.Id
where dbname = 'MY_NEW_DB'
and mmudkv.value in ('start', 'finish')
group
by (CONVERT(CHAR(8), mmud.logDateTime, 112))
) as resultSet
group
by substring(yyyyMMdd, 0,7)
order
by substring(yyyyMMdd, 0,7)
in LINQ => I first fetch the record from a table that links information of the Database Name and UpdateData and then do filtering and groupby on the related information.
entry.updatedata.Where(
ue => ue.updatedataKeyValue.Any(
uedkv =>
uedkv.Value.ToLower() == "starting update" ||
uedkv.Value.ToLower() == "client release"))
.Select(
ue =>
new
{
logDateTimeyyyyMMdd = ue.logDateTime.Date,
logDateTime = ue.logDateTime
})
.GroupBy(
updateDataDetail => updateDataDetail.logDateTimeyyyyMMdd)
.Select(
groupedupdatedata => new
{
UpdateDateyyyyMM = groupedupdatedata.Key.ToString("yyyyMMdd"),
Duration =
(groupedupdatedata.Max(groupMember => groupMember.logDateTime) -
groupedupdatedata.Min(groupMember => groupMember.logDateTime)
)
.TotalMilliseconds
}
).
ToList();
var updatedataMonthlyDurations =
updatedataInDateRangeWithDescriptions.GroupBy(ue => ue.UpdateDateyyyyMM.Substring(0,6))
.Select(
group =>
new updatedataMonthlyAverageDuration
{
DbName = entry.DbName,
UpdateDateyyyyMM = group.Key.Substring(0,6),
Duration =
group.Average(
gmember =>
(gmember.Duration))
}
).ToList();
I know that GroupBy in LINQ isn't the same as GroupBy in T-SQL, but not sure what happens behind the scenes. Could anyone explain the difference and what happens in memory when I run the LINQ version? After I did the .ToList() after the first GroupBy things got a little faster. But even then this way of finding average duration is really slow.
What would be the best alternative and are there ways of improving a slow LINQ statement using Visual Studio 2012?

Your linq query is doing most of its work in linq-to-objects. You should be constructing a linq-to-entities/sql query that generates the complete query in one shot.
Your query seems to have a redundant group by clause, and I am not sure which table dbname comes from, but the following query should get you on the right track.
var query = from mmud in context.updateData
from mmudkv in context.updateDataKeyValue
.Where(x => mmud.updateDataid == x.updateDataId)
.DefaultIfEmpty()
from mmuddk in context.updateDataDetailKey
.Where(x => mmudkv.updateDataDetailKeyid == x.Id)
.DefaultIfEmpty()
where mmud.dbname == "MY_NEW_DB"
where mmudkv.value == "start" || mmudkv.value == "finish"
group mmud by mmud.logDateTime.Date into g
select new
{
Date = g.Key,
Average = EntityFunctions.DiffMilliseconds(g.Max(x => x.logDateTime), g.Min(x => x.logDateTime)),
};
var queryByMonth = from x in query
group x by new { x.Date.Year, x.Date.Month } into x
select new
{
Year = x.Key.Year,
Month = x.Key.Month,
Average = x.Average(y => y.Average)
};
// Single sql statement is to sent to your database
var result = queryByMonth.ToList();
If you are still having problems, we will need to know if you are using entityframework or linq-to-sql. And you will need to provide your context/model information

how to convert T-SQL Query to linq

i have 4 table in SQL: DocumentType,ClearanceDocument,Request, RequestDocument.
i want when page load and user select one request, show all Document Based on clearanceType in RequestTable and check in RequestDocument and when exist set is_exist=true
I have written this query with SqlServer Query Editor for get result this Scenario but i can't convert this Query to Linq
select *,
is_Orginal=
(select is_orginal from CLEARANCE_REQUEST_DOCUMENT
where
DOCUMENT_ID=a.DOCUMENT_ID and REQUEST_ID=3)
from
DOCUMENT_TYPES a
where
DOCUMENT_ID in
(select DOCUMENT_ID from CLEARANCE_DOCUMENTS dt
where
dt.CLEARANCE_ID=
(SELECT R.CLEARANCE_TYPE FROM CLEARANCE_REQUEST R
WHERE
R.REQUEST_ID=3))
i write this Query in linq but not work
var list = (from r in context.CLEARANCE_REQUEST
where r.REQUEST_ID == 3
join cd in context.CLEARANCE_DOCUMENTS on r.CLEARANCE_TYPE equals cd.CLEARANCE_ID
join dt in context.DOCUMENT_TYPES on cd.DOCUMENT_ID equals dt.DOCUMENT_ID into outer
from t in outer.DefaultIfEmpty()
select new
{
r.REQUEST_ID,
cd.CLEARANCE_ID,
t.DOCUMENT_ID,
t.DOCUMENT_NAME,
is_set=(from b in context.CLEARANCE_REQUEST_DOCUMENT where
b.REQUEST_ID==r.REQUEST_ID && b.DOCUMENT_ID==t.DOCUMENT_ID
select new{b.IS_ORGINAL})
}
).ToList();
I want convert this Query to LINQ. Please help me. Thanks.

There is no need to manually join objects returned from an Entity Framework context.
See Why use LINQ Join on a simple one-many relationship?
If you use the framework as intended your job will be much easier.
var result = var clearanceTypes = context.CLEARANCE_REQUEST
.Single(r => r.REQUEST_ID == 3)
.CLEARANCE_DOCUMENTS
.SelectMany(dt => dt.DOCUMENT_TYPES)
.Select(a => new
{
DocumentType = a,
IsOriginal = a.CLEARANCE_REQUEST_DOCUMENT.is_original
});

Since your query won't be executed untill you iterate over the data, you can split your query in several subqueries to help you obtain the results like this:
var clearanceIds = context.CLEARANCE_REQUEST
.Where(r => r.REQUEST_ID == 3)
.Select(r => r.CLEARANCE_TYPE);
var documentIds = context.CLEARANCE_DOCUMENTS
.Where(dt => clearanceIds.Contains(dt.CLEARANCE_ID))
.Select(dt => dt.DOCUMENT_ID);
var result = context.DOCUMENT_TYPES
.Where(a => documentIds.Contains(a.DOCUMENT_ID))
.Select(a => new
{
// Populate properties here
IsOriginal = context.CLEARANCE_REQUEST_DOCUMENT
.Single(item => item.DOCUMENT_ID == a.DOCUMENT_ID &&
item.REQUEST_ID == 3)
.IS_ORIGINAL
})
.ToList();