Develop Reference

Convert C# Entity Framework linq query to SQL

I am converting the C# linq query to SQL, LINQ never return the values.
But the same query I wrote in SQL returns some values. Can anyone help me find out what the issue is for my SQL query?
LINQ query:
var query = from l in _DbContext.Licenses
join lp in _DbContext.LicenseParts on l.PartNumber equals lp.PartNumber
join lpc in _DbContext.LicensePartConfigurations on lp.Id equals lpc.LicensePartId
join p in _DbContext.Products on lp.ProductId equals p.Id
join lsn in _DbContext.LicenseSerialNumbers on l.Id equals lsn.LicenseId
join lact in _DbContext.LicenseActivations on new { a = lsn.Id, b = lp.ProductId } equals new { a = lact.LicenseSerialNumberId, b = lact.ProductId }
where lact.AccountId == AccountId && JsonExtensions.JsonValue(lact.ActivationInfoJSON, "$.SubscriptionKey") !=
" " && (JsonExtensions.JsonValue(lact.ActivationInfoJSON, "$.IsConverted") == null || JsonExtensions.JsonValue(lact.ActivationInfoJSON, "$.IsConverted") == "0" || JsonExtensions.JsonValue(lact.ActivationInfoJSON, "$.IsConverted") == "false") && p.Name == "ClearPass Legacy"
select new SubscriptionKeys { SubscriptionKey = JsonExtensions.JsonValue(lact.ActivationInfoJSON, "$.SubscriptionKey"), CustomerMail = JsonExtensions.JsonValue(lact.ActivationInfoJSON, "$.CustomerMail"), CustomerName = JsonExtensions.JsonValue(lact.ActivationInfoJSON, "$.CustomerName") };
response.PageSize = pageSize;
response.PageNumber = pageNumber;
response.Model = await query.Distinct().ToListAsync();
response.ItemsCount = response.Model.Count();
SQL query:
SELECT
l.AccountId,CustomerMail,
JSON_VALUE(ActivationInfo, '$.SubscriptionKey')
FROM
Licenses l
JOIN
LicenseParts lp ON l.PartNumber = lp.PartNumber
JOIN
LicensePartConfigurations lpc ON lp.Id = lpc.LicensePartId
JOIN
Products p ON lp.ProductId = p.Id
JOIN
LicenseSerialNumbers lsn ON l.Id = lsn.LicenseId
JOIN
LicenseActivations lact ON lsn.Id = lact.LicenseSerialNumberId
AND lp.ProductId = lact.ProductId
WHERE
lact.AccountId = 'QWNjb3VudDoxNTMwNDAzMi00MWM2LTExZTktOWYzMy1kMzQxZjE5OWZlYjM='
AND JSON_VALUE(lact.ActivationInfo, '$.SubscriptionKey') != ' '
AND (JSON_VALUE(lact.ActivationInfo, '$.IsConverted') = NULL OR
JSON_VALUE(lact.ActivationInfo, '$.IsConverted') = 0 OR
JSON_VALUE(lact.ActivationInfo, '$.IsConverted') = 'false')
AND p.Name = 'ClearPass Legacy'

To start from a valid point, execute the code where it fires this linq query and use sql profiler to catch it up. That is a good way to find the exact equivalent sql statement it finally produces and executes. You need to set up a trace to sql profiler prior to execute the linq. Get the statement and then you can compare with the sql you already have.
this sql:
(JSON_VALUE(lact.ActivationInfo, '$.IsConverted') = NULL
is not equal to:
(JsonExtensions.JsonValue(lact.ActivationInfoJSON, "$.IsConverted") == null
as in first case you compare to database NULL value using '=' and this requires ansi_nulls off to work properly and it is not a good practice.

ORMs like EF Core are meant to Map Objects to Relational constructs. Instead of trying to write SQL in C# through LINQ, you should try to Map the attributes you need to entity properties.
In this case the SubscriptionKey and IsConverted fields should appear in the table itself, either as proper fields or computed columns. If that's not possible, you could use computed columns to map the SubscriptionKey and IsConverted attributes to entity properties so you can use them in queries.
In your LicenseActivation class add these properties:
public bool? IsConverted {get;set;}
public string? SubscriptionKey {get;set;}
public string? CustomerEmail {get;set;}
In your DbContext, you can specify computed columns with HasComputedColumnSql:
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<LicenseActivation>()
.Property(b => b.SubscriptionKey)
.HasComputedColumnSql("JSON_VALUE(ActivationInfo, '$.SubscriptionKey')");
modelBuilder.Entity< LicenseActivations >()
.Property(b => b.IsConverted)
.HasComputedColumnSql("JSON_VALUE(ActivationInfo, '$.IsConverted')");
modelBuilder.Entity< LicenseActivations >()
.Property(b => b.CustomerEmail)
.HasComputedColumnSql("JSON_VALUE(ActivationInfo, '$.CustomerEmail')");
}
This will allow you to use the properties in LINQ queries.
You shouldn't have to use all those JOINs either. It's the ORM's job to generate JOINs from the relations between objects. If you add proper relations between the entities the query could be simplified to :
var binValue='QWNjb3VudDoxNTMwNDAzMi00MWM2LTExZTktOWYzMy1kMzQxZjE5OWZlYjM=';
var query=_dbContext.LicenseActivations
.Where(lact=>
lact.AccountId == binValue
&& (lact.IsConverted==null || !lact.IsConverted))
.Select(lact=>new {
lact.AccountId,
lact.SubscriptionKey,
lact.CustomerEmail});
or, if the AccountId fields don't hold the same data :
.Select(lact=>new {
AccountId =lact.LicensePart.License.AccountId,
lact.SubscriptionKey,
lact.CustomerEmail
});
EF Core will generate the appropriate SQL and JOIN clauses to get from LicenseActivation to License based on the relations between the entities

Using ternary conditional operator or expression in Entity Framework

I have an entity framework query that I inherited which includes several Sums, cutdown example:-
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = c.ClientTransactions.Select(ct => ct.Amount)
.DefaultIfEmpty(0m).Sum(),
}).ToList();
As the number of clients and the number of transactions has grown, this query has obviously become slower and slower.
Ideally I'd want to store balances rather than calculate them every time, but currently the system doesn't do that, and it would be a very large change to implement, so for now I'm just attempting a band-aid fix.
The fix I'm attempting to implement is to simply not do the Sum calculations (there are several, example above just has one) for people that aren't interested in them.
My first attempt was simply to use ternary conditional operators to determine whether or not to do the calculation:-
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = ClientSearchExcludeCurrentBalance ? 0m :
c.ClientTransactions.Select(ct => ct.fAmount).DefaultIfEmpty(0m).Sum(),
}).ToList();
The problem with this, it turns out, is that regardless of the value of the condition (ClientSearchExcludeCurrentBalance) both sides are still calculated, and then the ternary decides which one to use. So even setting the condition to false, the Sum still gets processed and the query takes too long.
Commenting out the sum, as below...
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = ClientSearchExcludeCurrentBalance ? 0m : 0m,
//c.ClientTransactions.Select(ct => ct.fAmount).DefaultIfEmpty(0m).Sum(),
}).ToList();
... is now nice and fast, so the ternary is definitely running it even when it's not used.
So, with that idea out the window, I tried using an expression instead:-
Expression<Func<Client, Decimal>> currentBalance = c => 0m;
if (!ClientSearchExcludeCurrentBalance)
{
currentBalance = c => c.ClientTransactions
.Select(ct => ct.Amount).DefaultIfEmpty(0m).Sum();
}
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = currentBalance.Invoke(c),
}).ToList();
This fell over with an unknown expression error:-
LINQ to Entities does not recognize the method 'System.Decimal Invoke[Client,Decimal](System.Linq.Expressions.Expression`1[System.Func`2[RPM.DAO.UI.Client,System.Decimal]], RPM.DAO.UI.Client)' method, and this method cannot be translated into a store expression
I also tried using Expand()
CurrentBalance = currentBalance.Expand().Invoke(c)
but still got the unknown expression error.
Just to see, I tried it with defaulting the Sum values to 0m, and then in the loop that assigns the results to the DTO Collection doing the sum there if needed
foreach (var client in Clients)
{
if (!ClientSearchExcludeCurrentBalance) {
var c = db.Clients.FirstOrDefault(cl => cl.ClientID == client.ClientID);
client.CurrentBalance = c.ClientTransactions.Select(ct => ct.fAmount)
.DefaultIfEmpty(0m).Sum();
}
}
This works, in that it only does the sum if told to, but doing it outside the main select means the entire query now takes twice as long as it used to, so is clearly not viable.
So, my questions are:-
Does anyone know if it's possible to make Entity Framework only run the parts of a ternary conditional operator that will be used?
Does anyone know if it's possible to use an Expression to return a value in Entity Framework?
Or, alternatively, how to add an IF statement into an Entity Framework query?
For (non-working) example:-
from c in db.Clients
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = if (ClientSearchExcludeCurrentBalance)
return 0m;
else
return c.ClientTransactions.Select(tf => tf.fAmount)
.DefaultIfEmpty(0m).Sum(),
}).ToList();
Thanks!
Edit:
I tried Barr J's solution:-
from c in db.Clients
let currentBalance = ClientSearchExcludeCurrentBalance ? 0m :
c.ClientTransactions.Select(ct => ct.Amount).DefaultIfEmpty(0m).Sum()
where {where clauses}
select new
{
ClientID = c.ClientID,
ClientName = c.ClientName,
CurrentBalance = currentBalance
}).ToList();
I get a null reference exception:
System.NullReferenceException: 'Object reference not set to an instance of an object.'
Edit #2: The cut-down version above doesn't give the null exception error, but the full version (with identical code) does... weird!
Anyway, with the working cut-down version above I tried it with the setting set to true and fall, and both took the same time, so it still does the Sum evaluation either way

Linq will evaluate the operand from both sides regardless of the ternary operator, because it is being evaluated at run-time.
You will have to evaluate the operands outside of your linq statement and then use it.
for example:
var tst = from p in products join i in info on p.id equals i.pid
let status = p.type = "home" ? homestatus.Select(s=>s.status) :
p.type = "offshore" ? offshorestatus.Select(s=>s.status) :
p.type = "internal" ? internalestatus.Select(s=>s.status) : null
select new {
name = p.name,
status = status != null ? status.StatusText : string.Empty;
}
or:
var tst = from p in products join i in info on p.id equals i.pid
let status = (p.type = "home" ? homestatus.Select(s=>s.status.StatusText) :
p.type = "offshore" ? offshorestatus.Select(s=>s.status.StatusText) :
p.type = "internal" ? internalestatus.Select(s=>s.status.StatusText) : null) ?? string.Empty
select new {
name = p.name,
status = status;
}

first of all: #Barr 's answer ist not correct. Not the evaluation at runtime is the problem (in the end, it is not evaluated at all for Linq To Entities!), but what the underlying provider of Linq2Entities tries to do:
Run though the whole expresion tree and build some valid SQL out of it. And of course, Find a SQL equivalent of "Invoke". Well there is nothing it can use so it throws the exception LINQ to Entities does not recognize the method
You have to avoid everything within such linq2entity statements that MUST be evaluated at runtime. E.g. access to DateTimeOffset.Now will also not work.
Currently I am not able to test your query so I can not tell you why the ternary operator does not work as expected. It may depend on how the SQL looks like.
I can give you two advices:
take a look at the query outcome. To do this, install SQL profiler (distributed with SQL Server installation), debug into your application until your linq2entities statement is executed. Hopefully you know that this will not happen unless you call ToList(), Any(), First() or something else onto the query.
If you have no profiler, you should also be able to store the whole linq query in a variable (without calling toList()) and call ToString() onto it. This should you also give the query.
Have you thought about checking the execution plan of the query? This sounds like a missing index on ClientIdof the table Transaction. Maybe you can provide us with the SQL statement and/or the execution plan, so we will be able to help you more.
Additional hint: After retriving the query, you can execute it in SQL Management Studio. Please let you show the real execution plan. If you do this and there are some indices missing and SQL Server detects this missing index, it will suggest you what you can do to speed up the query.

Error converting SQL with join to LINQ

I have a SQL query that I'm trying to convert to LINQ and am having trouble understanding the obscure error messages when the query is enumerated.
The SQL query (which works as intended), is:
select a.TestGuid, MIN(a.StartTime) as StartTime, COUNT(b.TestCaseId) as NumTests, COUNT(DINSTINCT a.Id) as NumScenarios
from LoadTestSummary as a
join LoadTestTestSummaryData as b
on a.LoadTestRunid = b.LoadTestRunId
where
a.TargetStack = env and
a.TestGuid IS NOT NULL AND
a.StartTime IS NOT NULL AND
a.LoadTestRunId IS NOT NULL
group by a.TestGuid
Converting to LINQ, I get the following:
var q = from a in _context.LoadTestSummary
where
a.TargetStack == env &&
a.TestGuid != null &&
a.StartTime != null &&
a.LoadTestRunId != null
join b in _context.LoadTestTestSummaryData on new
{
LoadTestRunId = Convert.ToInt32(a.LoadTestRunId)
} equals new
{
LoadTestRunId = b.LoadTestRunId
}
group new { a, b } by new
{
a.TestGuid
}
into g
select new
{
DateCreated = g.Min(p => p.a.StartTime),
NumScenarios = g.Count(),
TestGuid = g.Key.TestGuid
NumTests = // ???
};
Two problems I have:
1) When the query is enumerated I get a run-time error that I'm having trouble deciphering. The query works fine in Linqpad, but gives me a run-time error in my program. I am not sure what would cause this. Just staring at this makes my head hurt:
ArgumentException: Expression of type 'System.Func``2[Microsoft.Data.Entity.Query.EntityQueryModelVisitor+TransparentIdentifier``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData],<>f__AnonymousType7``1[System.String]]' cannot be used for parameter of type 'System.Func``2[<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData],<>f__AnonymousType7``1[System.String]]' of method 'System.Collections.Generic.IEnumerable``1[System.Linq.IGrouping``2[<>f__AnonymousType7``1[System.String],<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData]]] _GroupBy[<>f__AnonymousType5``2,<>f__AnonymousType7``1,<>f__AnonymousType5``2](System.Collections.Generic.IEnumerable``1[<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData]], System.Func``2[<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData],<>f__AnonymousType7``1[System.String]], System.Func``2[<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData],<>f__AnonymousType5``2[PerfPortal.Models.LoadTestSummary,PerfPortal.Models.LoadTestTestSummaryData]])'
2) I am not quite sure how to get the COUNT(DISTINCT a.Id) into the NumTests field. It looks like this isn't supported in LINQ but it looks like other people have asked this question to so I may be able to figure it out once #1 is resolved.
Any thoughts on what's wrong here? I am not even sure exactly what the error is telling me.
All help is appreciated!

Looking just at the SQL query and your LINQ code, I came up with something like this:
from a in LoadTestSummary
join b in LoadTestTestSummaryData
on a.LoadTestRunId equals b.LoadTestRunId
where
a.TargetStack == env &&
a.TestGuid != null &&
a.StartTime != null &&
a.LoadTestRunId != null
group new { a, b } by a.TestGuid into g
select new
{
TestGuid = g.Key,
DateCreated = g.Min(el => el.a.StartTime),
NumTests = g.Select(el => el.b.TestCaseId).Count(),
NumScenarios = g.Select(el => el.a.Id).Distinct().Count()
};
Note, that you don't need to convert LoadTestRunId to int, you may just use standard string comparision.
That horrendous error is most likely caused by grouping and comparing using anonimous objects, thou I prefer not to read that error too much as it's an eldritch abomination not ment to be seen nor comprehend by mere mortals, it seems.

Conditional where clause with date parameters raises a has no supported translation to SQL. error in Linq to SQL with C#

I'm working on a linq to sql project using Visual C# 2008 and dot Net framework 4.5.
my query is as follows
var q =
from a in dc.GetTable<invoice_in>()
join b in dc.GetTable<supplier>()
on a.supplier_id equals b.id
where a.invoice_date >= date_from
select new Invoice_in(a.id, a.amount ?? 0, a.invoice_number ,
a.supplier_id ?? 0, a.supplier.s_name,
a.invoice_date ?? System.DateTime.Today);
invoce_in is a linq class while Invoice_in is a class I defined with a similar structure.
When I put the date comparison inside where clause within the last query, everything is OK. But I need to use a conditional where, as the query parameters goes after the main query clause
I added the following lines to the previous code
if (date_from != null)
{
q = q.Where(w => w.invoice_date >= date_from);
}
Where w.invoice_date is of DateTime type and it is data member of the class Invoice_in (defined by me).
Adding that last lines of code causes the following runtime error:
"has no supported translation to SQL"
I've tried dozens of methods on the web such as using SQLMethods for comparing dates and such stuff, nothing works
Please Help... Thanks in advance...

This should work for you:
var q =
from a in dc.GetTable<invoice_in>()
join b in dc.GetTable<supplier>()
on a.supplier_id equals b.id
select a;
//since you compared date_from against null I assume it is Nullable<DateTime>
if (date_from.HasValue)
{
q = q.Where(a => a.invoice_date >= date_from.Value);
}
var result =
q.Select(a => new Invoice_in(a.id, a.amount ?? 0,
a.invoice_number ,
a.supplier_id ?? 0,
a.supplier.s_name,
a.invoice_date ?? System.DateTime.Today))
.ToList();

How to get SQL query into LINQ form in C# code

How can I convert the following SQL queries into LINQ query form in C#, .NET 3.5 code:
1)
select COUNT(distinct Skill_Name)
from Table1
where Department = 'ABC' and Skill_Name is not null
2)
select distinct location, country from Customer where Customer_Code ='1001';

I suspect you want:
var query = from entry in dbContext.Table1
where entry.Department == "ABC" && entry.SkillName != null
select entry.SkillName;
var count = query.Distinct().Count();
Or using extension method syntax, in one go:
var count = dbContext.Table1
.Where(entry => entry.Department == "ABC" &&
entry.SkillName != null)
.Select(entry => entry.SkillName)
.Distinct()
.Count();
As shown by mesiesta, you can combine query expressions with calls not supported within query expressions, but I tend to assign the query expression to an intermediate variable... I personally find it clearer, but use whichever you (and your team) prefer.

Something like this
int count = (from p in Table1
where p.Department == "ABC" && p.Skill_Name != null
select p.Skill_Name).Distinct().Count();
For second query you can use this
var query= (from p in Customer
where p.Customer_Code=="1001"
select new { Location=p.location ,Country=p.country}).Distinct();

you can use linqpad to convert to linq and lambda expressions