Develop Reference

C# db query where conditions are met, orderby date and then get the first result

While evaluating some queries we found some possible optimization. The ideia is shown below but I currently don't know how to solve this.
Current query:
public static List<Object> SampleQuerySales(int store_id)
{
var query = (from clients in db.table1.Where(p => p.store_id == store_id)
from sales in db.table2.Where(q => q.customer_id == clients.customer_id))
select new Object {
...
}).ToList();
return query;
}
This returns all sales made, but its required only the latest sale (OperationDate) from a datetime reference. As obvious this became a bottleneck.
My ideia was to make it similar to query below, which is incorrect (doesn't compile). How can I achieve this dataset?
var query = (from clients in db.table1.Where(p => p.store_id == store_id)
from sales in db.table2.Where(q => q.customer_id == clients.customer_id
&& q.OperationDate <= dateReference)
.OrderByDescending(s => s.OperationDate).FirstOrDefault() //error
select new Object {
...
}).Tolist();

Since you only want one value from table2, use let instead of from:
var query = (from client in db.table1.Where(p => p.store_id == store_id)
let mostRecentSaleAfterDateReference = db.table2
.Where(q => q.customer_id == client.customer_id
&& q.OperationDate <= dateReference)
.OrderByDescending(s => s.OperationDate)
.FirstOrDefault()
select new Object {
...
}).Tolist();

Is it possible to combine small queries into single query?

I have the following queries:
var truckCount = await DbContext.Trucks
.Where(t => t.Departure == null)
.CountAsync();
var firstTruck = await DbContext.Trucks
.Where(t => t.Departure == null)
.MinAsync(t => t.Arrival);
var railcarCount = await DbContext.Railcars
.Where(r => r.Departure == null)
.CountAsync();
var firstRailcar = await DbContext.Railcars
.Where(t => t.Departure == null)
.MinAsync(t => t.Arrival);
Can anyone tell me if it's possible to combine these queries into one so that there is only one round trip to the database?
I'd be looking to generate a query something like this.
select
(select count(*) from Trucks where Departure is null) as TruckCount,
(select min(Arrival) from Trucks where Departure is null) as FirstTruck,
(select count(*) from Railcars where Departure is null) as RailcarCount,
(select min(Arrival) from Railcars where Departure is null) as FirstRailcar
My backend is SQL Server.

You need to use a third party library which enables to execute multiple queries in a single roundtrip to the database. Maybe this extension with it's future queries works for you.
Otherwise you could implement a stored-procedure which encapsulates the queries (as subqueries) and returns the desired information.
Another option might be to just use 2 queries instead of 4:
var truckInfo = await DbContext.Trucks
.GroupBy(t => t.Departure == null)
.Where(g => g.Key == true)
.Select(g => new { Count = g.Count(), FirstTruck = g.Min(t => t.Arrival) })
.FirstOrDefaultAsync() ?? new { Count = 0, FirstTruck = DateTime.MinValue };
// same for Railcars

Not with linq, no. Why? Because of two reasons:
Query syntax has no way to get count and use union to get from one query
Method count, is immediate execution and not deferred, so you can't chain into one query
To be honest, that would be difficult to achieve even with a sql query as the data has different data types and columns.

Just in case, there is EF Core extension linq2db.EntityFrameworkCore (note that I'm one of the creators) which can run this query and almost any SQL ANSI query via LINQ
using var db = DbContext.CreateLinqToDBConnection();
var trucks = DbContext.Trucks
.Where(t => t.Departure == null);
var railcars = DbContext.Railcars
.Where(r => r.Departure == null);
var result = await db.SelectAsync(() => new
{
TruckCount = trucks.Ccount(),
FirstTruck = trucks.Min(t => t.Arrival),
RailcarCount = railcars.Count(),
FirstRailcar = railcars.Min(t => t.Arrival)
});

Joining table to a list using Entity Framework

I have the following Entity Framework function that it joining a table to a list. Each item in serviceSuburbList contains two ints, ServiceId and SuburbId.
public List<SearchResults> GetSearchResultsList(List<ServiceSuburbPair> serviceSuburbList)
{
var srtList = new List<SearchResults>();
srtList = DataContext.Set<SearchResults>()
.AsEnumerable()
.Where(x => serviceSuburbList.Any(m => m.ServiceId == x.ServiceId &&
m.SuburbId == x.SuburbId))
.ToList();
return srtList;
}
Obviously that AsEnumerable is killing my performance. I'm unsure of another way to do this. Basically, I have my SearchResults table and I want to find records that match serviceSuburbList.

If serviceSuburbList's length is not big, you can make several Unions:
var table = DataContext.Set<SearchResults>();
IQuerable<SearchResults> query = null;
foreach(var y in serviceSuburbList)
{
var temp = table.Where(x => x.ServiceId == y.ServiceId && x.SuburbId == y.SuburbId);
query = query == null ? temp : query.Union(temp);
}
var srtList = query.ToList();
Another solution - to use Z.EntityFramework.Plus.EF6 library:
var srtList = serviceSuburbList.Select(y =>
ctx.Customer.DeferredFirstOrDefault(
x => x.ServiceId == y.ServiceId && x.SuburbId == y.SuburbId
).FutureValue()
).ToList().Select(x => x.Value).Where(x => x != null).ToList();
//all queries together as a batch will be sent to database
//when first time .Value property will be requested

LINQ subquery with multiple columns

I'm trying to recreate this SQL query in LINQ:
SELECT *
FROM Policies
WHERE PolicyID IN(SELECT PolicyID
FROM PolicyRegister
WHERE PolicyRegister.StaffNumber = #CurrentUserStaffNo
AND ( PolicyRegister.IsPolicyAccepted = 0
OR PolicyRegister.IsPolicyAccepted IS NULL ))
Relationship Diagram for the two tables:
Here is my attempt so far:
var staffNumber = GetStaffNumber();
var policyRegisterIds = db.PolicyRegisters
.Where(pr => pr.StaffNumber == staffNumber && (pr.IsPolicyAccepted == false || pr.IsPolicyAccepted == null))
.Select(pr => pr.PolicyID)
.ToList();
var policies = db.Policies.Where(p => p.PolicyID.//Appears in PolicyRegisterIdsList)
I think I'm close, will probably make two lists and use Intersect() somehow but I looked at my code this morning and thought there has to be an easier way to do this,. LINQ is supposed to be a more readble database language right?
Any help provided is greatly appreciated.

Just use Contains:
var policies = db.Policies.Where(p => policyRegisterIds.Contains(p.PolicyID));
Also better store policyRegisterIds as a HashSet<T> instead of a list for search in O(1) instead of O(n) of List<T>:
var policyRegisterIds = new HashSet<IdType>(db.PolicyRegisters......);
But better still is to remove the ToList() and let it all happen as one query in database:
var policyRegisterIds = db.PolicyRegisters.Where(pr => pr.StaffNumber == staffNumber &&
(pr.IsPolicyAccepted == false || pr.IsPolicyAccepted == null));
var policies = db.Policies.Where(p => policyRegisterIds.Any(pr => pr.PolicyID == p.PolicyID));

Flattening Complex LINQ to SQL

I have a somewhat complex LINQ to SQL query that I'm trying to optimise (no, not prematurely, things are slow), that goes a little bit like this;
IQueryable<SearchListItem> query = DbContext.EquipmentLives
.Where(...)
.Select(e => new SearchListItem {
EquipmentStatusId = e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null).Id,
StatusStartDate = e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null).DateFrom,
...
});
The where clauses aren't important, they don't filter EquipmentStatuses, happy to include if someone thinks they're required.
This is on quite a large set of tables and returns a fairly details object, there's more references to EquipmentStatuses, but I'm sure you get the idea. The problem is that there's quite obviously two sub-queries and I'm sure that (among some other things) is not ideal, especially since they are exactly the same sub-query each time.
Is it possible to flatten this out a bit? Perhaps it's easier to do a few smaller queries to the database and create the SearchListItem in a foreach loop?

Here's my take given your comments, and with some assumptions I've made
It may look scary, but give it a try, with and without the ToList() before the GroupBy()
If you have LinqPad, check the SQL produced, and the number of queries, or just plug in the SQL Server Profiler
With LinqPad you could even put a Stopwatch to measure things precisely
Enjoy ;)
var query = DbContext.EquipmentLives
.AsNoTracking() // Notice this!!!
.Where(...)
// WARNING: SelectMany is an INNER JOIN
// You won't get EquipmentLive records that don't have EquipmentStatuses
// But your original code would break if such a case existed
.SelectMany(e => e.EquipmentStatuses, (live, status) => new
{
EquipmentLiveId = live.Id, // We'll need this one for grouping
EquipmentStatusId = status.Id,
EquipmentStatusDateTo = status.DateTo,
StatusStartDate = status.DateFrom
//...
})
// WARNING: Again, you won't get EquipmentLive records for which none of their EquipmentStatuses have a DateTo == null
// But your original code would break if such a case existed
.Where(x => x.EquipmentStatusDateTo == null)
// Now You can do a ToList() before the following GroupBy(). It depends on a lot of factors...
// If you only expect one or two EquipmentStatus.DateTo == null per EquipmentLive, doing ToList() before GroupBy may give you a performance boost
// Why? GroupBy sometimes confuses the EF SQL generator and the SQL Optimizer
.GroupBy(x => x.EquipmentLiveId, x => new SearchListItem
{
EquipmentLiveId = x.EquipmentLiveId, // You may or may not need this?
EquipmentStatusId = x.EquipmentStatusId,
StatusStartDate = x.StatusStartDate,
//...
})
// Now you have one group of SearchListItem per EquipmentLive
// Each group has a list of EquipmenStatuses with DateTo == null
// Just select the first one (you could do g.OrderBy... as well)
.Select(g => g.FirstOrDefault())
// Materialize
.ToList();

You don't need to repeat the FirstOrDefault. You can add an intermediate Select to select it once and then reuse it:
IQueryable<SearchListItem> query = DbContext.EquipmentLives
.Where(...)
.Select(e => e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null))
.Select(s => new SearchListItem {
EquipmentStatusId = s.Id,
StatusStartDate = s.DateFrom,
...
});
In query syntax (which I find more readable) it would look like this:
var query =
from e in DbContext.EquipmentLives
where ...
let s = e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null)
select new SearchListItem {
EquipmentStatusId = s.Id,
StatusStartDate = s.DateFrom,
...
});
There is another problem in your query though. If there is no matching EquipmentStatus in your EquipmentLive, FirstOrDefault will return null, which will cause an exception in the last select. So you might need an additional Where:
IQueryable<SearchListItem> query = DbContext.EquipmentLives
.Where(...)
.Select(e => e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null))
.Where(s => s != null)
.Select(s => new SearchListItem {
EquipmentStatusId = s.Id,
StatusStartDate = s.DateFrom,
...
});
or
var query =
from e in DbContext.EquipmentLives
where ...
let s = e.EquipmentStatuses.FirstOrDefault(s => s.DateTo == null)
where s != null
select new SearchListItem {
EquipmentStatusId = s.Id,
StatusStartDate = s.DateFrom,
...
});

Given that you don't test for null after calling FirstOrDefault(s => s.DateTo == null) I assume that:
either for each device there is always a status with DateTo == null or
you need to see only devices which have such status
In order to do so you need to join EquipmentLives with EquipmentStatuses to avoid subqueries:
var query = DbContext.EquipmentLives
.Where(l => true)
.Join(DbContext.EquipmentStatuses.Where(s => s.DateTo == null),
eq => eq.Id,
status => status.EquipmentId,
(eq, status) => new SelectListItem
{
EquipmentStatusId = status.Id,
StatusStartDate = status.DateFrom
});
However, if you do want to perform a left join replace DbContext.EquipmentStatuses.Where(s => s.DateTo == null) with DbContext.EquipmentStatuses.Where(s => s.DateTo == null).DefaultIfEmpty().