I have an entity framework query that I inherited which includes several Sums, cutdown example:-
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = c.ClientTransactions.Select(ct => ct.Amount)
.DefaultIfEmpty(0m).Sum(),
}).ToList();
As the number of clients and the number of transactions has grown, this query has obviously become slower and slower.
Ideally I'd want to store balances rather than calculate them every time, but currently the system doesn't do that, and it would be a very large change to implement, so for now I'm just attempting a band-aid fix.
The fix I'm attempting to implement is to simply not do the Sum calculations (there are several, example above just has one) for people that aren't interested in them.
My first attempt was simply to use ternary conditional operators to determine whether or not to do the calculation:-
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = ClientSearchExcludeCurrentBalance ? 0m :
c.ClientTransactions.Select(ct => ct.fAmount).DefaultIfEmpty(0m).Sum(),
}).ToList();
The problem with this, it turns out, is that regardless of the value of the condition (ClientSearchExcludeCurrentBalance) both sides are still calculated, and then the ternary decides which one to use. So even setting the condition to false, the Sum still gets processed and the query takes too long.
Commenting out the sum, as below...
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = ClientSearchExcludeCurrentBalance ? 0m : 0m,
//c.ClientTransactions.Select(ct => ct.fAmount).DefaultIfEmpty(0m).Sum(),
}).ToList();
... is now nice and fast, so the ternary is definitely running it even when it's not used.
So, with that idea out the window, I tried using an expression instead:-
Expression<Func<Client, Decimal>> currentBalance = c => 0m;
if (!ClientSearchExcludeCurrentBalance)
{
currentBalance = c => c.ClientTransactions
.Select(ct => ct.Amount).DefaultIfEmpty(0m).Sum();
}
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = currentBalance.Invoke(c),
}).ToList();
This fell over with an unknown expression error:-
LINQ to Entities does not recognize the method 'System.Decimal Invoke[Client,Decimal](System.Linq.Expressions.Expression`1[System.Func`2[RPM.DAO.UI.Client,System.Decimal]], RPM.DAO.UI.Client)' method, and this method cannot be translated into a store expression
I also tried using Expand()
CurrentBalance = currentBalance.Expand().Invoke(c)
but still got the unknown expression error.
Just to see, I tried it with defaulting the Sum values to 0m, and then in the loop that assigns the results to the DTO Collection doing the sum there if needed
foreach (var client in Clients)
{
if (!ClientSearchExcludeCurrentBalance) {
var c = db.Clients.FirstOrDefault(cl => cl.ClientID == client.ClientID);
client.CurrentBalance = c.ClientTransactions.Select(ct => ct.fAmount)
.DefaultIfEmpty(0m).Sum();
}
}
This works, in that it only does the sum if told to, but doing it outside the main select means the entire query now takes twice as long as it used to, so is clearly not viable.
So, my questions are:-
Does anyone know if it's possible to make Entity Framework only run the parts of a ternary conditional operator that will be used?
Does anyone know if it's possible to use an Expression to return a value in Entity Framework?
Or, alternatively, how to add an IF statement into an Entity Framework query?
For (non-working) example:-
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = if (ClientSearchExcludeCurrentBalance)
return 0m;
else
return c.ClientTransactions.Select(tf => tf.fAmount)
.DefaultIfEmpty(0m).Sum(),
}).ToList();
Thanks!
Edit:
I tried Barr J's solution:-
from c in db.Clients
let currentBalance = ClientSearchExcludeCurrentBalance ? 0m :
c.ClientTransactions.Select(ct => ct.Amount).DefaultIfEmpty(0m).Sum()
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = currentBalance
}).ToList();
I get a null reference exception:
System.NullReferenceException: 'Object reference not set to an instance of an object.'
Edit #2: The cut-down version above doesn't give the null exception error, but the full version (with identical code) does... weird!
Anyway, with the working cut-down version above I tried it with the setting set to true and fall, and both took the same time, so it still does the Sum evaluation either way
Linq will evaluate the operand from both sides regardless of the ternary operator, because it is being evaluated at run-time.
You will have to evaluate the operands outside of your linq statement and then use it.
for example:
var tst = from p in products join i in info on p.id equals i.pid
let status = p.type = "home" ? homestatus.Select(s=>s.status) :
p.type = "offshore" ? offshorestatus.Select(s=>s.status) :
p.type = "internal" ? internalestatus.Select(s=>s.status) : null
select new {
name = p.name,
status = status != null ? status.StatusText : string.Empty;
}
or:
var tst = from p in products join i in info on p.id equals i.pid
let status = (p.type = "home" ? homestatus.Select(s=>s.status.StatusText) :
p.type = "offshore" ? offshorestatus.Select(s=>s.status.StatusText) :
p.type = "internal" ? internalestatus.Select(s=>s.status.StatusText) : null) ?? string.Empty
select new {
name = p.name,
status = status;
}
first of all: #Barr 's answer ist not correct. Not the evaluation at runtime is the problem (in the end, it is not evaluated at all for Linq To Entities!), but what the underlying provider of Linq2Entities tries to do:
Run though the whole expresion tree and build some valid SQL out of it. And of course, Find a SQL equivalent of "Invoke". Well there is nothing it can use so it throws the exception LINQ to Entities does not recognize the method
You have to avoid everything within such linq2entity statements that MUST be evaluated at runtime. E.g. access to DateTimeOffset.Now will also not work.
Currently I am not able to test your query so I can not tell you why the ternary operator does not work as expected. It may depend on how the SQL looks like.
I can give you two advices:
take a look at the query outcome. To do this, install SQL profiler (distributed with SQL Server installation), debug into your application until your linq2entities statement is executed. Hopefully you know that this will not happen unless you call ToList(), Any(), First() or something else onto the query.
If you have no profiler, you should also be able to store the whole linq query in a variable (without calling toList()) and call ToString() onto it. This should you also give the query.
Have you thought about checking the execution plan of the query? This sounds like a missing index on ClientIdof the table Transaction. Maybe you can provide us with the SQL statement and/or the execution plan, so we will be able to help you more.
Additional hint: After retriving the query, you can execute it in SQL Management Studio. Please let you show the real execution plan. If you do this and there are some indices missing and SQL Server detects this missing index, it will suggest you what you can do to speed up the query.
Related
I have the following code. I'm getting error:
"The cast to value type 'Int32' failed because the materialized value is null. Either the result type's generic parameter or the query must use a nullable type."
when CreditHistory table has no records.
var creditsSum = (from u in context.User
join ch in context.CreditHistory on u.ID equals ch.UserID
where u.ID == userID
select ch.Amount).Sum();
How can I modify the query to accept null values?
A linq-to-sql query isn't executed as code, but rather translated into SQL. Sometimes this is a "leaky abstraction" that yields unexpected behaviour.
One such case is null handling, where there can be unexpected nulls in different places. ...DefaultIfEmpty(0).Sum(0) can help in this (quite simple) case, where there might be no elements and sql's SUM returns null whereas c# expect 0.
A more general approach is to use ?? which will be translated to COALESCE whenever there is a risk that the generated SQL returns an unexpected null:
var creditsSum = (from u in context.User
join ch in context.CreditHistory on u.ID equals ch.UserID
where u.ID == userID
select (int?)ch.Amount).Sum() ?? 0;
This first casts to int? to tell the C# compiler that this expression can indeed return null, even though Sum() returns an int. Then we use the normal ?? operator to handle the null case.
Based on this answer, I wrote a blog post with details for both LINQ to SQL and LINQ to Entities.
To allow a nullable Amount field, just use the null coalescing operator to convert nulls to 0.
var creditsSum = (from u in context.User
join ch in context.CreditHistory on u.ID equals ch.UserID
where u.ID == userID
select ch.Amount ?? 0).Sum();
Had this error message when I was trying to select from a view.
The problem was the view recently had gained some new null rows (in SubscriberId column), and it had not been updated in EDMX (EF database first).
The column had to be Nullable type for it to work.
var dealer = Context.Dealers.Where(x => x.dealerCode == dealerCode).FirstOrDefault();
Before view refresh:
public int SubscriberId { get; set; }
After view refresh:
public Nullable<int> SubscriberId { get; set; }
Deleting and adding the view back in EDMX worked.
Hope it helps someone.
I have used this code and it responds correctly, only the output value is nullable.
var packesCount = await botContext.Sales.Where(s => s.CustomerId == cust.CustomerId && s.Validated)
.SumAsync(s => (int?)s.PackesCount);
if(packesCount != null)
{
// your code
}
else
{
// your code
}
You are using aggregate function which not getting the items to perform action , you must verify linq query is giving some result as below:
var maxOrderLevel =sdv.Any()? sdv.Max(s => s.nOrderLevel):0
I see that this question is already answered. But if you want it to be split into two statements, following may be considered.
var credits = from u in context.User
join ch in context.CreditHistory
on u.ID equals ch.UserID
where u.ID == userID
select ch;
var creditSum= credits.Sum(x => (int?)x.Amount) ?? 0;
Got this error in Entity Framework 6 with this code at runtime:
var fileEventsSum = db.ImportInformations.Sum(x => x.FileEvents)
Update from LeandroSoares:
Use this for single execution:
var fileEventsSum = db.ImportInformations.Sum(x => (int?)x.FileEvents) ?? 0
Original:
Changed to this and then it worked:
var fileEventsSum = db.ImportInformations.Any() ? db.ImportInformations.Sum(x => x.FileEvents) : 0;
I was also facing the same problem and solved through making column as nullable using "?" operator.
Sequnce = db.mstquestionbanks.Where(x => x.IsDeleted == false && x.OrignalFormID == OriginalFormIDint).Select(x=><b>(int?)x.Sequence</b>).Max().ToString();
Sometimes null is returned.
Suppose I have a list of {City, State}. It originally came from the database, and I have LocationID, but by now I loaded it into memory. Suppose I also have a table of fast food restaurants that has City and State as part of the record. I need to get a list of establishments that match city and state.
NOTE: I try to describe a simplified scenario; my business domain is completely different.
I came up with the following LINQ solution:
var establishments = from r in restaurants
from l in locations
where l.LocationId == id &&
l.City == r.City &&
l.State == r.State
select r
and I feel there must be something better. For starters, I already have City/State in memory - so to go back to the database only to have a join seems very inefficient. I am looking for some way to say {r.City, r.State} match Any(MyList) where MyList is my collection of City/State.
UPDATE
I tried to update based on suggestion below:
List<CityState> myCityStates = ...;
var establishments =
from r in restaurants
join l in myCityStates
on new { r.City, r.State } equals new { l.City, l.State } into gls
select r;
and I got the following compile error:
Error CS1941 The type of one of the expressions in the join clause is incorrect. Type inference failed in the call to 'Join'.
UPDATE 2
Compiler didn't like anonymous class in the join. I made it explicit and it stopped complaining. I'll see if it actually works in the morning...
It seems to me that you need this:
var establishments =
from r in restaurants
join l in locations.Where(x => x.LocationId == id)
on new { r.City, r.State } equals new { l.City, l.State } into gls
select r;
Well, there isn't a lot more that you can do, as long as you rely on a table lookup, the only thing you can do to speed up things is to put an index on City and State.
The linq statement has to translate into a valid SQL Statement, where "Any" would translate to something like :
SELECT * FROM Restaurants where City in ('...all cities')
I dont know if other ORM's give better performance for these types of scenarios that EF, but it might be worth investigating. EF has never had a rumor for being fast on reads.
Edit: You can also do this:
List<string> names = new List { "John", "Max", "Pete" };
bool has = customers.Any(cus => names.Contains(cus.FirstName));
this will produce the necessary IN('value1', 'value2' ...) functionality that you were looking for
I'm working on a linq to sql project using Visual C# 2008 and dot Net framework 4.5.
my query is as follows
var q =
from a in dc.GetTable<invoice_in>()
join b in dc.GetTable<supplier>()
on a.supplier_id equals b.id
where a.invoice_date >= date_from
select new Invoice_in(a.id, a.amount ?? 0, a.invoice_number ,
a.supplier_id ?? 0, a.supplier.s_name,
a.invoice_date ?? System.DateTime.Today);
invoce_in is a linq class while Invoice_in is a class I defined with a similar structure.
When I put the date comparison inside where clause within the last query, everything is OK. But I need to use a conditional where, as the query parameters goes after the main query clause
I added the following lines to the previous code
if (date_from != null)
{
q = q.Where(w => w.invoice_date >= date_from);
}
Where w.invoice_date is of DateTime type and it is data member of the class Invoice_in (defined by me).
Adding that last lines of code causes the following runtime error:
"has no supported translation to SQL"
I've tried dozens of methods on the web such as using SQLMethods for comparing dates and such stuff, nothing works
Please Help... Thanks in advance...
This should work for you:
var q =
from a in dc.GetTable<invoice_in>()
join b in dc.GetTable<supplier>()
on a.supplier_id equals b.id
select a;
//since you compared date_from against null I assume it is Nullable<DateTime>
if (date_from.HasValue)
{
q = q.Where(a => a.invoice_date >= date_from.Value);
}
var result =
q.Select(a => new Invoice_in(a.id, a.amount ?? 0,
a.invoice_number ,
a.supplier_id ?? 0,
a.supplier.s_name,
a.invoice_date ?? System.DateTime.Today))
.ToList();
Using EF 4.5, I want to convert a zero-count sub-query (relatedDrivers) to null in the following statement:
var query = from car in context.tblCar
let relatedDrivers = (from driver in context.tblDriver
where driver.CarId == car.CarId
select driver)
select new
{
CarId = car.CarId,
Drivers = relatedDrivers.Count() == 0 ? null : relatedDrivers
};
But I get the 'NotSupportedException' stating that it is impossible create a null constant value of type 'System.Linq.IQueryable`1' !
I wonder why this is so hard for Entity Framework to translate this query to T-SQL. Examining a sub-query and returning NULL if the result count is zero doesn't seem to be that much complicated.
Any solution and explanation is highly appreciated.
The solution is IQueriable.DefaultIfEmpty(). So the query will be changed to:
var query = from car in context.tblCar
let relatedDrivers = (from driver in context.tblDriver
where driver.CarId == car.CarId
select driver).DefaultIfEmpty()
select new
{
CarId = car.CarId,
Drivers = relatedDrivers
};
I've been searching for possible solutions and attempting this for several hours without luck. Any help will be greatly appreciated.
I've got a Sql statement which I'm trying to put together as a C# LINQ query.
Here is the working SQL:
SELECT up.UserProfileID
,up.FirstName
,up.LastName
,SUM(CASE WHEN ul.CompletionDate IS NULL THEN 0
ELSE ISNULL(ul.Score, 0)
END) AS TotalScore
FROM dbo.UserProfile up
LEFT OUTER JOIN dbo.UserLearning ul ON up.UserProfileID = ul.UserProfileID
WHERE up.ManagerUserProfileID IS NULL
GROUP BY up.UserProfileID, up.FirstName, up.LastName
I've tried several different ways but seem to end up with either a statement that doesn't return what I want or doesn't execute successfully
My current (non-working) code looks something like this:
var pd = from up in db.UserProfiles
join ul in db.UserLearnings on up.UserProfileID equals ul.UserProfileID into temp
from upJOINul in temp.DefaultIfEmpty(new UserLearning() { Score = 0 })
where up.ManagerUserProfileID.Equals(null)
group up by new
{
UserProfileID = up.UserProfileID,
FirstName = up.FirstName,
LastName = up.LastName,
TotalScore = up.UserLearnings.Sum(u => u.Score)
};
Thank you for any help
After several more attempts and further use of google I finally managed to get a working solution. I hope it'll be of use to someone else.
var pd = db.UserProfiles.AsEnumerable()
.Where(up => up.ManagerUserProfileID.Equals(null))
.Select(up => new
{
UserProfileID = up.UserProfileID,
FirstName = up.FirstName,
LastName = up.LastName,
TotalScore = up.UserLearnings
.Where(ul => ul.CompletionDate.HasValue && ul.Score.HasValue)
.DefaultIfEmpty()
.Sum(ul => ul != null && ul.Score.HasValue ? ul.Score : 0)
});
Not what you asked for, but if you have a working complex SQL query, that is fairly static, put it in a stored proc, and drag that SP to your LINQ DataContext.
The LINQ provider has to compile your query to sql every time it's called, and that takes time, and server CPU cycles. If it's a complex query, it can eat up significant resources. Also may miss some optimizations you can do with straight SQL.
Unless of course there is a purpose to it.
If you have ORM problem, grap the actual SQL commands, take a look at it, and compare with what you want to achieve. Can you show the generated SQL as well, so we can find the difference easier?