I'm trying to recreate this SQL query in LINQ:
SELECT *
FROM Policies
WHERE PolicyID IN(SELECT PolicyID
FROM PolicyRegister
WHERE PolicyRegister.StaffNumber = #CurrentUserStaffNo
AND ( PolicyRegister.IsPolicyAccepted = 0
OR PolicyRegister.IsPolicyAccepted IS NULL ))
Relationship Diagram for the two tables:
Here is my attempt so far:
var staffNumber = GetStaffNumber();
var policyRegisterIds = db.PolicyRegisters
.Where(pr => pr.StaffNumber == staffNumber && (pr.IsPolicyAccepted == false || pr.IsPolicyAccepted == null))
.Select(pr => pr.PolicyID)
.ToList();
var policies = db.Policies.Where(p => p.PolicyID.//Appears in PolicyRegisterIdsList)
I think I'm close, will probably make two lists and use Intersect() somehow but I looked at my code this morning and thought there has to be an easier way to do this,. LINQ is supposed to be a more readble database language right?
Any help provided is greatly appreciated.
Just use Contains:
var policies = db.Policies.Where(p => policyRegisterIds.Contains(p.PolicyID));
Also better store policyRegisterIds as a HashSet<T> instead of a list for search in O(1) instead of O(n) of List<T>:
var policyRegisterIds = new HashSet<IdType>(db.PolicyRegisters......);
But better still is to remove the ToList() and let it all happen as one query in database:
var policyRegisterIds = db.PolicyRegisters.Where(pr => pr.StaffNumber == staffNumber &&
(pr.IsPolicyAccepted == false || pr.IsPolicyAccepted == null));
var policies = db.Policies.Where(p => policyRegisterIds.Any(pr => pr.PolicyID == p.PolicyID));
Related
I have the following queries:
var truckCount = await DbContext.Trucks
.Where(t => t.Departure == null)
.CountAsync();
var firstTruck = await DbContext.Trucks
.Where(t => t.Departure == null)
.MinAsync(t => t.Arrival);
var railcarCount = await DbContext.Railcars
.Where(r => r.Departure == null)
.CountAsync();
var firstRailcar = await DbContext.Railcars
.Where(t => t.Departure == null)
.MinAsync(t => t.Arrival);
Can anyone tell me if it's possible to combine these queries into one so that there is only one round trip to the database?
I'd be looking to generate a query something like this.
select
(select count(*) from Trucks where Departure is null) as TruckCount,
(select min(Arrival) from Trucks where Departure is null) as FirstTruck,
(select count(*) from Railcars where Departure is null) as RailcarCount,
(select min(Arrival) from Railcars where Departure is null) as FirstRailcar
My backend is SQL Server.
You need to use a third party library which enables to execute multiple queries in a single roundtrip to the database. Maybe this extension with it's future queries works for you.
Otherwise you could implement a stored-procedure which encapsulates the queries (as subqueries) and returns the desired information.
Another option might be to just use 2 queries instead of 4:
var truckInfo = await DbContext.Trucks
.GroupBy(t => t.Departure == null)
.Where(g => g.Key == true)
.Select(g => new { Count = g.Count(), FirstTruck = g.Min(t => t.Arrival) })
.FirstOrDefaultAsync() ?? new { Count = 0, FirstTruck = DateTime.MinValue };
// same for Railcars
Not with linq, no. Why? Because of two reasons:
Query syntax has no way to get count and use union to get from one query
Method count, is immediate execution and not deferred, so you can't chain into one query
To be honest, that would be difficult to achieve even with a sql query as the data has different data types and columns.
Just in case, there is EF Core extension linq2db.EntityFrameworkCore (note that I'm one of the creators) which can run this query and almost any SQL ANSI query via LINQ
using var db = DbContext.CreateLinqToDBConnection();
var trucks = DbContext.Trucks
.Where(t => t.Departure == null);
var railcars = DbContext.Railcars
.Where(r => r.Departure == null);
var result = await db.SelectAsync(() => new
{
TruckCount = trucks.Ccount(),
FirstTruck = trucks.Min(t => t.Arrival),
RailcarCount = railcars.Count(),
FirstRailcar = railcars.Min(t => t.Arrival)
});
I'm getting some Entities from EF, which I iterate and creating Anonymous type objects such as:
var payments = ctx.Payments.ToList();
var data = ctx.Activities.OrderBy(p => p.ID).ToList().Select(p => new
{
ID = p.ID,
Date = p.Date?.ToString("dd/MM/yyyy"),
PaymentMethod = p.PaymentMethods != null ? p.PaymentMethods.Description : "",
ActivityStatusID = payments.Where(q => q.ActivityID == p.ID && !q.Paid).Count() == 0 ? 1 : .. // I need some other check
}).ToList();
Now, I'd like to check the payments.Where(q => q.ActivityID == p.ID && !q.Paid).Count() several times before set a value.
Such as:
if .Count() == 0, value 1
if .Count() == 1, value 8
if .Count() > 1, value 10
else 87
and so on. I won't do somethings like this:
ActivityStatusID = payments.Where(q => q.ActivityID == p.ID && !q.Paid).Count() == 0 ? 1 : payments.Where(q => q.ActivityID == p.ID && !q.Paid).Count() == 1 ? 8 : payments.Where(q => q.ActivityID == p.ID && !q.Paid).Count() > 1 ? 10 : 87
Is there any way to do payments.Where(q => q.ActivityID == p.ID && !q.Paid).Count() once for each p, and than evalutate the conditions?
People suggested to convert your query into query syntax, which would enable your to use a let statement to create a variable in which you could save count.
If you investigate what let does, it adds one column value to your result. Every row in your result has the same let value in this column. See this answer on stackoverflow
Keep your query in Method Syntax
If you want to keep your query in method syntax, simply add a Count to your anonymous type:
var result = ctx.Activities
.OrderBy(p => p.ID)
.Select(p => new
{
Id = p.Id,
Date = p.Date?.ToString("dd/MM/yyyy"),
PaymentMethod = p.PaymentMethods != null ? p.PaymentMethods.Description : "",
PaidCount = payments
.Where(q => q.ActivityID == p.ID && !q.Paid)
.Count();
})
.Select(p => new
{
Id = p.Id,
Date = p.Date,
ActivityStatusId =
{
// count == 0 => 1
// count == 1 => 8
// count > 1 => 10
// count < 0 => 87
if (p.PaidCount < 0) return 87;
switch (p.PaidCount)
{
case 0:
return 0;
case 1:
return 8;
default:
return 10;
}
},
});
Note, the switch is only possible because you brought your complete ctx.Activities to local memory using the ToList.
Improved efficiency
You do a ToList before your select. This means that your complete ctx.payments are materialized in local memory into a List, before you start selecting and counting items in your sequence.
If ctx.Payments is from an external source, like a database, or a file, then ctx.Payments is an IQueryable instead of an IEnumerable. Fetching your complete Payments to local memory is not an efficient approach.
Advise: Whenever you have an IQueryable, try to keep it IQueryable as long as possible. Your source data provider can process your queries much more efficiently than your local processor. Only materialize it to local memory if your source data provider can't process it anymore, for instance because you need to call local procedures, or because there is nothing to process anymore.
Furthermore, don't move values to local memory that you don't plan to use. Only Select the properties that you actually will use in your local memory.
One improvement would be:
var result = ctx.Payments.Select(payment => new
{ // select only the properties you plan to use locally:
Id = payment.Id,
Date = payment.Date,
PaymentMethod = payment.PaymentMethods?.Description,
PaidCount = ctx.Payments
.Where(q => q.ActivityID == p.ID && !q.Paid)
.Count(),
})
.OrderBy(fetchedPaymentData => fetchedPaymentData.Id)
// from here you need to move it to local memory
// Use AsEnumerable instead of ToList
.AsEnumerable()
.Select(fetchedPaymentData => new
{
Id = fetchedPaymentData.Id,
PaymentMethod = fetchedPaymentData.PaymentMethod ?? String.Empty,
ActivityStatusId = {...}
});
AsEnumerable is more efficient than ToList, especially if you don't need all items at once. For instance if you would end with FirstOrDefault, or only Take(5), then it would be a waste to move all items to local memory.
Finally: with some trying you can get rid of the switch statement, thus allowing your DBMS to calculate the ActivityStatusId. But as the selecting of your source data and the transport of the selected data to local memory is the slower part of your complete query, I doubt whether this would lead to shorter execution time. The switch surely makes your requirement better readable, especially if your numbers 1 / 8 / 87 are put into enums.
I would change the query from a lambda expression to an ordinary query. Then use the let syntax to set a variable for each iteration. Something like this:
var payments = ctx.Payments.ToList();
var data = (
from p in ctx.Activities.ToList()
orderby p.ID
let paymentCount = payments.Where(q => q.ActivityID == p.ID && !q.Paid).Count()
select new
{
ID = p.ID,
Date = p.Date?.ToString("dd/MM/yyyy"),
PaymentMethod = p.PaymentMethods != null ? p.PaymentMethods.Description : "",
ActivityStatusID = paymentCount == 0 ? 1 : .. // I need some other check
}
).ToList();
And btw you can do this part diffrently as well. This:
payments.Where(q => q.ActivityID == p.ID && !q.Paid).Count()
You can write like this:
payments.Count(q => q.ActivityID == p.ID && !q.Paid)
I have the following Entity Framework function that it joining a table to a list. Each item in serviceSuburbList contains two ints, ServiceId and SuburbId.
public List<SearchResults> GetSearchResultsList(List<ServiceSuburbPair> serviceSuburbList)
{
var srtList = new List<SearchResults>();
srtList = DataContext.Set<SearchResults>()
.AsEnumerable()
.Where(x => serviceSuburbList.Any(m => m.ServiceId == x.ServiceId &&
m.SuburbId == x.SuburbId))
.ToList();
return srtList;
}
Obviously that AsEnumerable is killing my performance. I'm unsure of another way to do this. Basically, I have my SearchResults table and I want to find records that match serviceSuburbList.
If serviceSuburbList's length is not big, you can make several Unions:
var table = DataContext.Set<SearchResults>();
IQuerable<SearchResults> query = null;
foreach(var y in serviceSuburbList)
{
var temp = table.Where(x => x.ServiceId == y.ServiceId && x.SuburbId == y.SuburbId);
query = query == null ? temp : query.Union(temp);
}
var srtList = query.ToList();
Another solution - to use Z.EntityFramework.Plus.EF6 library:
var srtList = serviceSuburbList.Select(y =>
ctx.Customer.DeferredFirstOrDefault(
x => x.ServiceId == y.ServiceId && x.SuburbId == y.SuburbId
).FutureValue()
).ToList().Select(x => x.Value).Where(x => x != null).ToList();
//all queries together as a batch will be sent to database
//when first time .Value property will be requested
I have a somewhat complex LINQ to SQL query that I'm trying to optimise (no, not prematurely, things are slow), that goes a little bit like this;
IQueryable<SearchListItem> query = DbContext.EquipmentLives
.Where(...)
.Select(e => new SearchListItem {
EquipmentStatusId = e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null).Id,
StatusStartDate = e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null).DateFrom,
...
});
The where clauses aren't important, they don't filter EquipmentStatuses, happy to include if someone thinks they're required.
This is on quite a large set of tables and returns a fairly details object, there's more references to EquipmentStatuses, but I'm sure you get the idea. The problem is that there's quite obviously two sub-queries and I'm sure that (among some other things) is not ideal, especially since they are exactly the same sub-query each time.
Is it possible to flatten this out a bit? Perhaps it's easier to do a few smaller queries to the database and create the SearchListItem in a foreach loop?
Here's my take given your comments, and with some assumptions I've made
It may look scary, but give it a try, with and without the ToList() before the GroupBy()
If you have LinqPad, check the SQL produced, and the number of queries, or just plug in the SQL Server Profiler
With LinqPad you could even put a Stopwatch to measure things precisely
Enjoy ;)
var query = DbContext.EquipmentLives
.AsNoTracking() // Notice this!!!
.Where(...)
// WARNING: SelectMany is an INNER JOIN
// You won't get EquipmentLive records that don't have EquipmentStatuses
// But your original code would break if such a case existed
.SelectMany(e => e.EquipmentStatuses, (live, status) => new
{
EquipmentLiveId = live.Id, // We'll need this one for grouping
EquipmentStatusId = status.Id,
EquipmentStatusDateTo = status.DateTo,
StatusStartDate = status.DateFrom
//...
})
// WARNING: Again, you won't get EquipmentLive records for which none of their EquipmentStatuses have a DateTo == null
// But your original code would break if such a case existed
.Where(x => x.EquipmentStatusDateTo == null)
// Now You can do a ToList() before the following GroupBy(). It depends on a lot of factors...
// If you only expect one or two EquipmentStatus.DateTo == null per EquipmentLive, doing ToList() before GroupBy may give you a performance boost
// Why? GroupBy sometimes confuses the EF SQL generator and the SQL Optimizer
.GroupBy(x => x.EquipmentLiveId, x => new SearchListItem
{
EquipmentLiveId = x.EquipmentLiveId, // You may or may not need this?
EquipmentStatusId = x.EquipmentStatusId,
StatusStartDate = x.StatusStartDate,
//...
})
// Now you have one group of SearchListItem per EquipmentLive
// Each group has a list of EquipmenStatuses with DateTo == null
// Just select the first one (you could do g.OrderBy... as well)
.Select(g => g.FirstOrDefault())
// Materialize
.ToList();
You don't need to repeat the FirstOrDefault. You can add an intermediate Select to select it once and then reuse it:
IQueryable<SearchListItem> query = DbContext.EquipmentLives
.Where(...)
.Select(e => e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null))
.Select(s => new SearchListItem {
EquipmentStatusId = s.Id,
StatusStartDate = s.DateFrom,
...
});
In query syntax (which I find more readable) it would look like this:
var query =
from e in DbContext.EquipmentLives
where ...
let s = e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null)
select new SearchListItem {
EquipmentStatusId = s.Id,
StatusStartDate = s.DateFrom,
...
});
There is another problem in your query though. If there is no matching EquipmentStatus in your EquipmentLive, FirstOrDefault will return null, which will cause an exception in the last select. So you might need an additional Where:
IQueryable<SearchListItem> query = DbContext.EquipmentLives
.Where(...)
.Select(e => e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null))
.Where(s => s != null)
.Select(s => new SearchListItem {
EquipmentStatusId = s.Id,
StatusStartDate = s.DateFrom,
...
});
or
var query =
from e in DbContext.EquipmentLives
where ...
let s = e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null)
where s != null
select new SearchListItem {
EquipmentStatusId = s.Id,
StatusStartDate = s.DateFrom,
...
});
Given that you don't test for null after calling FirstOrDefault(s => s.DateTo == null) I assume that:
either for each device there is always a status with DateTo == null or
you need to see only devices which have such status
In order to do so you need to join EquipmentLives with EquipmentStatuses to avoid subqueries:
var query = DbContext.EquipmentLives
.Where(l => true)
.Join(DbContext.EquipmentStatuses.Where(s => s.DateTo == null),
eq => eq.Id,
status => status.EquipmentId,
(eq, status) => new SelectListItem
{
EquipmentStatusId = status.Id,
StatusStartDate = status.DateFrom
});
However, if you do want to perform a left join replace DbContext.EquipmentStatuses.Where(s => s.DateTo == null) with DbContext.EquipmentStatuses.Where(s => s.DateTo == null).DefaultIfEmpty().
I have an object "IdentityProvider" and "IdentityProvider" has child Domains.
class IdentityProvider
{
...
public virtual ICollection<Domain> Domains { get; set; }
...
}
class Domain
{
...
public string Name { get; set; }
...
}
There is a catch all domain called "*"
Using Linq Extensions, I need to find all the IdentityProviders that have either the specified domain, or IdentityProviders that have the catch all, but nor both.
How would I form my query?
Something like this should make it:
from i in identityProviders
let hasDomain = i.Domains.Any(d => d.Name == domainName)
let hasCatchAll = i.Domains.Any(d => d.Name == "*")
where (hasDomain && !hasCatchAll) || (!hasDomain && hasCatchAll)
select i;
You could try using XOR (^) instead in where clause:
from i in identityProviders
let hasDomain = i.Domains.Any(d => d.Name == domainName)
let hasCatchAll = i.Domains.Any(d => d.Name == "*")
where hasDomain ^ !hasCatchAll
select i;
but I'm not sure if it get's translated into SQL by your provider (you didn't specify what kind of LINQ source you're dealing with...).
Your condition can't be checked with standard LINQ functions, without iterating the Domains collection twice which is needlessly inefficient. I would use a custom filter function like this, iterating once and failing early if both are found:
identityProviders.Where(identityProvider => {
bool hasDomain = false, hasCatchAll = false;
foreach (var domain in identityProvider.Domains) {
hasDomain = hasDomain || domain.Name == domainName;
hasCatchAll = hasCatchAll || domain.Name == "*";
if (hasDomain && hasCatchAll) return false;
}
return hasDomain || hasCatchAll;
})
If you group your data by domains which have the catch all such as:
var grouped = ipProviders.Domains
.GroupBy (dm => dm.Name == "*");
Then you can either return all the catch alls at once or extract the target domains with the exact name such as
var targetDomain = "Jabberwocky";
var targets = grouped.Where (gr => gr.Key == (targetDomain == "*"))
.Select (gr => gr.Where (dm => dm.Name == targetDomain));
The grouping looks like this with data of Jabberwocky, OmegaMan and two domains with *
Thanks to those that answered, you helped me in other areas, but for this question I ended up doing the following, probably not the best way, but it works:
See if the domain exists:
var domExists = db.Domains.Any(d => d.Name == domain);
Find all the identity providers where domain exists AND domExists OR find wildcard and not domExists.
IdentityProviders.Where(d =>
d.Domains.Any(n => n.Name == domain && domExists) ||
d.Domains.Any(n => n.Name == "*" && !domExists)
).Any()
The answer you gave doesn't give you what you asked for in the question. You said you wanted providers that had one or the other but not both.
First, if a provider has both, it will be selected by this code because the first condition is true:
d.Domains.Any(n => n.Name == domain && domExists)
Second, if a provider has the catch-all but NOT the specified domain, it will not be selected if the domain does exist in a different provider. This is because domExists will be true, so the second check will fail:
d.Domains.Any(n => n.Name == "*" && !domExists)
I don't see how capturing the domExists flag can really help you. However, I think starting by searching the entire collection of domains is the right idea. You could try this:
First, collect all the IDs of providers for domains that match either "*" or the name (I assume Domain must have a foreign key to IdentityProvider):
var providerIds =
db.Domains.Where(d => d.Name == domain || d.Name == "*")
.Select(d => d.IdentityProviderID)
.ToList();
This narrows it down quite a lot, and we have a way of filtering it again: Any providers that have both will have been added to the list twice, so we just need to select all the IDs that only appear once:
var uniqueProviderIds =
providerIds.GroupBy(id => id)
.Where(g => g.Count() == 1)
.Select(g => g.Key)
.ToList();
Now uniqueProviderIds.Any() will give you your answer. You can also use this list to build another SQL query to get the actual IdentityProvider objects if you need them:
db.IdentityProviders.Where(ip => uniqueProviderIds.Contains(ip.ID)).ToList()
please try
identityProviders.Where(ip=>ip.Domains.Any(d=>d.Name=="SearchDomain" || d.Name=="*"))