Develop Reference

Use a filter to query a list with only child entity conditions EF Core 5

This works fine. But how do I customize the conditions for adding filters?input is the value entered by the user
var resultList = dbContext.BuyerBill
.Include(x=>x.BuyerBillItems.Where(x=>x.Status == input.Status && x.BuildTime > input.BeginTime && x.BuildTime < input.EndTime))
.ToList();
the way I want:
var query = WhereIf(input.Status!=null,x=>x.Status == input.Status);
query = WhereIf(input.BeginTime!=null,x=>x.BuildTime > input.BeginTime);
query = WhereIf(input.EndTime!=null,x=>x.BuildTime > input.EndTime);
this is my entity
public class BuyerBill
{
public BuyerBill()
{
BuyerBillItems = new List<BuyerBillItems>();
}
public int Id {get;set;}
public int BuyUserId {get;set;}
public int OrderId {get;set;}
public List<BuyerBillItems> BuyerBillItems { get; set; }
....
}
public class BuyerBillItems
{
public int Id {get;set;}
public int BuyerBillId {get;set;}
public decimal Fee {get;set;}
public int Status {get;set;}
public dateTime CreateTime {get;set;}
public BuyerBill BuyerBill {get;set;}
....
}
1、If the user does not select the time query
Select * from BuyerBill as buy inner join BuyerBillItems As item On buy.Id=item.BuyerBillId
where item.Status=1
2、If the user selects the time query
Select * from BuyerBill as buy inner join BuyerBillItems as item on buy.Id=item.BuyerBillId
where item.Status=1 and item.BuildTime > '2022-7-19' and item.BuildTime < '2022-7-19'
How to use efcore to implement the SQL conditions I described?
Mainly, I want to filter sub entities according to conditions. If there is only one entity, I know Where() method can be used for filtering, but I don't know how to use conditional filtering for sub entities
I solved the above problem with LinqKit, but now I have a new problem。
var predicatess = PredicateBuilder.New<BuyerBillItems>(true);
predicatess = predicatess.And(x => x.CreateTime > StringHelper.AddDateTime("2022-07-16"));
predicatess = predicatess.And(x => x.Status == 2);
//I'm dumb and this line of code seems redundant. But I don't know how to convert implicitly
var convertPredicate = (Expression<Func<BuyerBillItems, bool>>)predicatess;
var query = dbContext.BuyerBill.AsExpandable().Include(x => x.BuyerBillItems.Where(x => convertPredicate.Invoke(x)))
.Where(x => x.BuyerBillItems.Any(s => convertPredicate.Invoke(s)))
.Where(x => x.BuyUserId == 4);
//If you don't use Select, everything is normal
var result1 = query.ToList();
//BuyerBillItemsDto result is incorrect after using Select
var result2 = query.Select(x => new BuyerBillDto
{
Id = x.Id,
BuyUserId = x.BuyUserId,
OrderId = x.OrderId,
BuyerBillItemsDto = mapper.Map<List<BuyerBillItems>, List<BuyerBillItemsDto>>(x.BuyerBillItems)
}).ToList();
I have to use select to filter the columns to avoid performance loss

Curently you cannot use own extensions in Include body. So, consider to write query in the following way:
var resultList = dbContext.BuyerBill
.Include(x => x.BuyerBillItems.Where(x =>
(input.Status == null || x.Status == input.Status && x.BuildTime > input.BeginTime && x.BuildTime < input.EndTime)) &&
(input.BeginTime == null || x.BuildTime > input.BeginTime) &&
(input.EndTime == null || x.BuildTime > input.EndTime)
)
.ToList();
Update based on updated reuiqrements
For fully custom projection, there is not needed to build Include, because Select specifies which data is needed to retrieve from database.
Query can be rewritten using WhereIf extension:
var result = dbContext.BuyerBill
.Select(b => new BuyerBillDto
{
Id = b.Id,
BuyUserId = b.BuyUserId,
OrderId = b.OrderId,
BuyerBillItemsDto = b.BuyerBillItems
.AsQueryable()
.WhereIf(input.Status != null, x => x.Status == input.Status)
.WhereIf(input.BeginTime != null, x => x.BuildTime > input.BeginTime)
.WhereIf(input.EndTime != null, x => x.BuildTime > input.EndTime)
.Select(item => new BuyerBillItemsDto
{
Id = item.Id,
// other properties
})
.ToList()
}).ToList();

Linq query where Contains() checks for list of strings

Consider the Following Query:
var query = from o in this.OrderManager.LoadOrders()
join s in this.SkuManager.LoadSkus() on o.SKU equals s.SKU
where o.ORDER_ID == orderId
let parcelItem = o.RPU != "Y" && o.DROP_SHIP != "Y" && s.TRUCK_SHIP != "T" && o.SKU != "ABC-123" && o.SKU != "XYZ-789" && o.SKU != "JKL-456"
select new OrderMailLineItem
{
OrderId = o.ORDER_ID,
Sku = s.SKU,
WarehouseId = s.SITE_ID,
QualifyingItem = qualifyingItem,
OversizedItem = parcelItem && (s.DROP_SHIP == null)
};
I would like to be able to write the line let parcelItem = ... to be more like !o.SKU.Contains(skuList) where:
List<string> skuList = new List<string> { "ABC-123", "XYZ-789", "JKL-456"};

You should check whether SKU is not in list instead of checking whether list is in SKU:
let parcelItem = !skuList.Contains(o.SKU)

!skuList.Contains(o.SKU) is exactly how you'd usually do it.
But you could write an In operator, if you like:
public static class ExtensionMethods
{
public static bool In<T>(this T t, params T[] values)
=> values.Contains(t);
}
...
let parcelItem = o.RPU != "Y" && o.DROP_SHIP != "Y" && s.TRUCK_SHIP != "T" &&
!o.SKU.In("ABC-123", "XYZ-789", "JKL-456")
I doubt that'll work with Linq to SQL though.

I don't see why this wouldn't work. You would just need to check those three Yes/No flags in addition to your SKU list.
var skuList = new[] { "ABC-123", "XYZ-789", "JKL-456"};
var query = from o in this.OrderManager.LoadOrders()
join s in this.SkuManager.LoadSkus() on o.SKU equals s.SKU
where o.ORDER_ID == orderId
let parcelItem = o.RPU != "Y" && o.DROP_SHIP != "Y" && s.TRUCK_SHIP != "T" && skuList.Contains(o.SKU)
select new OrderMailLineItem
{
OrderId = o.ORDER_ID,
Sku = s.SKU,
WarehouseId = s.SITE_ID,
QualifyingItem = qualifyingItem,
OversizedItem = parcelItem && (s.DROP_SHIP == null)
};

What is the appropriate LINQ query to this specific case?

Given the following two classes:
public class Apple
{
public int Id { get; set; }
public string Name { get; set; }
}
public class Worm
{
public int AppleId { get; set; }
public int WormType { get; set; }
public int HungerValue { get; set; }
}
All instances of Worm are given an AppleId equal to a randomly existing Apple.Id
public void DoLINQ(List<Apple> apples, List<Worm> worms, string targetAppleName, List<int> wormTypes )
{
// Write LINQ Query here
}
How can we write a Linq query which
finds all the elements in 'apples', whose 'Name' matches the 'targetAppleName'
AND
(does not "contain" the any worm with Wormtype given in Wormtypes
OR
only contains worms with Hungervalue equal to 500)?
Note that an instance of Apple does not actually 'contain' any elements of Worm, since the relation is the other way around. This is also what complicates things and why it is more difficult to figure out.
--Update 1--
My attempt which selects multiple apples with the same Id:
var query =
from a in apples
join w in worms
on a.Id equals w.AppleId
where (a.Name == targetAppleName) && (!wormTypes.Any(p => p == w.WormType) || w.HungerValue == 500)
select a;
--Update 2--
This is closer to a solution. Here we use two queries and then merge the results:
var query =
from a in apples
join w in worms
on a.Id equals w.AppleId
where (a.Name == targetAppleName) && !wormTypes.Any(p => p == w.WormType)
group a by a.Id into q
select q;
var query2 =
from a in apples
join w in worms
on a.Id equals w.AppleId
where (a.Name == targetAppleName) && wormTypes.Any(p => p == w.WormType) && w.HungerValue == 500
group a by a.Id into q
select q;
var merged = query.Concat(query2).Distinct();
--Update 3--
For the input we expect the LINQ query to use the parameters in the method, and those only.
For the output we want all apples which satisfy the condition described above.

You can use a let construct to find the worms of a given apple if you want to use query syntax:
var q =
from a in apples
let ws = from w in worms where w.AppleId == a.Id select w
where
(ws.All(w => w.HungerValue == 500)
|| ws.All(w => !wormTypes.Any(wt => wt == w.WormType)))
&& a.Name == targetAppleName
select a;
In method chain syntax this is equivalent to introducing an intermediary anonymous object using Select:
var q =
apples.Select(a => new {a, ws = worms.Where(w => w.AppleId == a.Id)})
.Where(t => (t.ws.All(w => w.HungerValue == 500)
|| t.ws.All(w => wormTypes.All(wt => wt != w.WormType)))
&& t.a.Name == targetAppleName).Select(t => t.a);
I wouldn't exactly call this more readable, though :-)

var result = apples.Where(apple =>
{
var wormsInApple = worms.Where(worm => worm.AppleId == apple.Id);
return apple.Name == targetAppleName
&& (wormsInApple.Any(worm => wormTypes.Contains(worm.WormType)) == false
|| wormsInApple.All(worm => worm.HungerValue == 500));
});
For each apple, create a collection of worms in that apple. Return only apples that match the required name AND (contain no worms that are in WormType OR only contain worms with a HungerValue of 500).

You were so close in your first attempt. But instead of a Join which multiplies the apples you really need GroupJoin which "Correlates the elements of two sequences based on key equality and groups the results". In query syntax it's represented by the join .. into clause.
var query =
from apple in apples
join worm in worms on apple.Id equals worm.AppleId into appleWorms
where apple.Name == targetAppleName
&& (!appleWorms.Any(worm => wormTypes.Contains(worm.WormType))
|| appleWorms.All(worm => worm.HungerValue == 500))
select apple;

Using lambda would look like this:
var result = apples.Where(a =>
a.Name == targetAppleName &&
(worms.Any(w => w.AppleId == a.Id && w.HungerValue >= 500)) ||
worms.All(w => w.AppleId != a.Id));
I think the lambda makes the code look a bit cleaner/easier to read, plus, the usage of.Any() and .All() is more efficient than a full on join IMHO... I haven't tested it with any heavy data so hard to speak with authority here (plus, there can't be that many apples...!)
BTW, this is the entire body of code. Kind of surprised it doesn't work for you. Maybe you missed something...?
public class Apple
{
public int Id { get; set; }
public string Name { get; set; }
}
public class Worm
{
public int AppleId { get; set; }
public int WormType { get; set; }
public int HungerValue { get; set; }
}
void Main()
{
var apples = Enumerable.Range(1, 9).Select(e => new Apple { Id = e, Name = "Apple_" + e}).ToList();
var worms = Enumerable.Range(1, 9).SelectMany(a =>
Enumerable.Range(1, 5).Select((e, i) => new Worm { AppleId = a, WormType = e, HungerValue = i %2 == 0 ? a * e * 20 : 100 })).ToList();
DoLINQ(apples, worms, "Apple_4", new[] {4, 5});
}
public void DoLINQ(IList apples, IList worms, string targetAppleName, IList wormTypes)
{
// Write LINQ Query here
var result = apples.Where(a =>
a.Name == targetAppleName &&
(worms.All(w => w.AppleId != a.Id) || worms.Any(w => w.AppleId == a.Id && w.HungerValue >= 500)));
result.Dump(); // remark this out if you're not using LINQPad
apples.Dump(); // remark this out if you're not using LINQPad
worms.Dump(); // remark this out if you're not using LINQPad
}

I have modify your query but didn't tested yet lets have a look and try it. Hopefully it will solve your problem.
var query =
from a in apples
join w in worms
on a.Id equals w.AppleId into pt
from w in pt.DefaultIfEmpty()
where (a.Name == targetAppleName) && (!wormTypes.Any(p => p == w.WormType) || (w.HungerValue == 500))
select a;
Thanks.

Project attribute values

Not sure if this is possible. I have a subset of 'MarketInventory' nodes:
<MARKET_INVENTORY _Type="TotalSales" _MonthRangeType="Prior7To12Months" _Count="18"/>
<MARKET_INVENTORY _Type="TotalSales" _MonthRangeType="Prior4To6Months" _Count="6"/>
<MARKET_INVENTORY _Type="TotalSales" _MonthRangeType="Last3Months" _Count="11"/>
<MARKET_INVENTORY _Type="TotalSales" _TrendType="Stable"/>
filtered on the _Type="TotalSales" node.
I'm wondering if it's possible to project the _Count value attributes into this class:
public class MarketInventoryListing
{
public string Prior7To12Months { get; set; }
public string Prior4To6Months { get; set; }
public string LastThreeMonths { get; set; }
}
This is as far as I got:
var marketInventoryTotalListings = from totalListings in xe.Descendants("MARKET_INVENTORY")
where (string) totalListings.Attribute("_Type") == "TotalSales"
select new MarketInventoryListing()
{
Prior7To12Months =
(
from thing in totalListings.Descendants()
where (string)totalListings.Attribute("_MonthRangeType") == "Prior7To12Months"
select thing.Attribute("_Count").Value
)
};

It's not working for 2 reasons:
Select is returning IEnumerable where as you need a single string
All MARKET_INVENTORY elements are on the same level so "from thing in totalListings.Descendants()" is not returning anything. All these elements are siblings at this point.
I changed your code to address those issues and it works as below:
var marketInventoryTotalListings = (from totalListings in xe.Descendants("MARKET_INVENTORY")
where (string)totalListings.Attribute("_Type") == "TotalSales"
select new MarketInventoryListing()
{
Prior7To12Months =
(
from thing in totalListings.Parent.Descendants()
where (string)totalListings.Attribute("_MonthRangeType") == "Prior7To12Months"
select thing.Attribute("_Count").Value
).FirstOrDefault(),
}).FirstOrDefault();

If you are sure that there will be just one node for each, then you can use FirstOrDefault like this:-
var marketInventoryTotalListings = from totalListings in xe.Descendants("MARKET_INVENTORY")
where (string)totalListings.Attribute("_Type") == "TotalSales"
let prior712 = xe.Descendants("MARKET_INVENTORY")
.FirstOrDefault(x => (string)x.Attribute("_MonthRangeType") == "Prior7To12Months")
let prior46 = xe.Descendants("MARKET_INVENTORY")
.FirstOrDefault(x => (string)x.Attribute("_MonthRangeType") == "Prior4To6Months")
let last3 = xe.Descendants("MARKET_INVENTORY")
.FirstOrDefault(x => (string)x.Attribute("_MonthRangeType") == "Last3Months")
select new MarketInventoryListing
{
Prior7To12Months = prior712 != null ? (string)prior712.Attribute("_Count") : "",
Prior4To6Months = prior712 != null ? (string)prior46.Attribute("_Count") : "",
LastThreeMonths = last3 != null ? (string)last3.Attribute("_Count") : "",
};
Otherwise if they are multiple then you should have IEnumerable<string> as the datatype in the properties of MarketInventoryListing instead of string.

Entity Framework - Results from subquery, speed issue?

I've made a query to get a list of articles, each bound to a "header",
so i retrieve the header plus the related articles and their properties.
The query works, however in its current style it is
Somewhat messy ( in my opinion)
The .ToList() takes way longer than i would expect.
Does anyone see any obvious reason for the speed-issue?
var offerheaders =
from o in dbcontext.F_CAB_OFFER_HEADERS
where
o.OFHD_FK_BUYER == userinfo.orgaTypeSequence
&& o.OFHD_VALID_FROM <= userinfo.selectedDate
&& o.OFHD_VALID_TO >= userinfo.selectedDate
&& o.OFHD_DELETED_YN == 0
&& o.OFHD_DELETED_BY_OWNER_YN == false
&& o.OFHD_OFFER_TYPE == userinfo.offerType
orderby o.OFHD_NO ascending
select o;
var offerlist =
from ofhd in offerheaders
select new {
ofhd = new {
OfferNo = ofhd.OFHD_NO,
OfferSequence = ofhd.OFHD_SEQUENCE,
ValidFrom = ofhd.OFHD_VALID_FROM,
ValidTo = ofhd.OFHD_VALID_TO,
OfferType = ofhd.OFHD_OFFER_TYPE,
Maingroup = new { cdmg_seq = ofhd.F_CAB_CD_MAIN_GROUP_TYPE.CDMG_SEQUENCE, Desc = ofhd.F_CAB_CD_MAIN_GROUP_TYPE.CDMG_DESC },
Supplier = new {
Name = ofhd.F_CAB_GROWER.F_CAB_ORGANISATION.ORGA_NAME,
Pic = ofhd.F_CAB_GROWER.F_CAB_ORGANISATION.ORGA_FK_PICTURE,
Seq = ofhd.F_CAB_GROWER.GROW_SEQUENCE
},
Caption = ofhd.OFHD_CAPTION,
Seperate = ofhd.OFHD_SHOW_SEPARATE_YN,
//ofdts = (from ofdt in dbcontext.F_CAB_OFFER_DETAILS.Where(x => x.OFDT_FK_OFFER_HEADER == ofhd.OFHD_SEQUENCE && x.OFDT_NUM_OF_ITEMS > 0 && x.OFDT_LATEST_DELIVERY_DATE_TIME > compareDateTime && x.OFDT_LATEST_ORDER_DATE_TIME > compareDateTime)
ofdts = from ofdt in dbcontext.F_CAB_OFFER_DETAILS
join props in dbcontext.F_CAB_CAB_PROP on ofdt.OFDT_FK_CAB_CODE equals props.PROP_FK_CABC_SEQ
join cabcode in dbcontext.F_CAB_CD_CAB_CODE on ofdt.OFDT_FK_CAB_CODE equals cabcode.CABC_SEQUENCE
join cabgroup in dbcontext.F_CAB_CD_CAB_GROUP on cabcode.CABC_FK_CAB_GROUP equals cabgroup.CDGR_SEQUENCE
join grouptype in dbcontext.F_CAB_CD_GROUP_TYPE on cabgroup.CDGR_FK_GROUP_TYPE equals grouptype.CDGT_SEQUENCE
join maingrouptype in dbcontext.F_CAB_CD_MAIN_GROUP_TYPE on grouptype.CDGT_FK_MAIN_GROUP equals maingrouptype.CDMG_SEQUENCE
join caca in dbcontext.F_CAB_CAB_CASK_MATRIX on ofdt.OFDT_FK_CACA_SEQ equals caca.CACA_SEQUENCE
join cask in dbcontext.F_CAB_CD_CASK on caca.CACA_FK_CASK equals cask.CDCA_SEQUENCE
join vbncode in dbcontext.F_CAB_CAB_VBN_MATRIX on cabcode.CABC_SEQUENCE equals vbncode.CVMA_FK_CAB_CODE
join grel in dbcontext.F_CAB_GENERAL_RELATIONS on ofdt.OFDT_FK_GREL_SEQ equals grel.GREL_SEQUENCE into greltable
from g_loj in greltable.DefaultIfEmpty()
where
ofdt.OFDT_FK_OFFER_HEADER == ofhd.OFHD_SEQUENCE
&& ofdt.OFDT_NUM_OF_ITEMS > 0
&& props.PROP_FK_CDLA_SEQ == userinfo.lang.CDLA_SEQUENCE
orderby props.PROP_CAB_DESC ascending
select new {
Desc = props.PROP_CAB_DESC,
Group = new { cdgr_seq = cabgroup.CDGR_SEQUENCE, Desc = cabgroup.CDGR_DESC },
Grouptype = new { grouptype.CDGT_SEQUENCE, Desc = grouptype.CDGT_DESC },
Properties = new CABProperties { props = props },
Price = ofdt.OFDT_ITEM_PRICE,
PIC_SEQ = ofdt.OFDT_FK_PICTURE ?? ((cabcode.CABC_FK_PICTURE ?? cabcode.CABC_SEQUENCE)),
PIC_URL = ofdt.OFDT_EXT_PICTURE_REF ?? "",
Seq = ofdt.OFDT_SEQUENCE,
Available = ofdt.OFDT_NUM_OF_ITEMS,
CabCode = ofdt.F_CAB_CD_CAB_CODE.CABC_CAB_CODE,
VBNCode = vbncode.CVMA_FK_VBN_CODE,
Remark = ofdt.OFDT_REMARK,
IsSpecial = ofdt.OFDT_SPECIAL_YN,
Arrived = inTransit ? ofdt.OFDT_ARRIVAL_DATE < DateTime.Now : true,
Cask = new CABCask { cask = cask, caca = caca },
Supplier = g_loj == null ? (ofdt.OFDT_SUPPLIER ?? "") : g_loj.GREL_NAME,
SupplierWeb = g_loj == null ? "" : g_loj.GREL_WEBSITE_URL,
SupplierLogo = g_loj == null ? ofhd.F_CAB_GROWER.F_CAB_ORGANISATION.ORGA_FK_PICTURE : g_loj.GREL_FK_PICT_SEQ,
SupplierSeq = g_loj == null ? -1 : g_loj.GREL_SEQUENCE,
}
}
};
userinfo.mainofferlist = offerlist.ToList();

As Daniel Kelly also mentioned the ToList function is where your query is executed, because these LinqToEntities queries are executed at the point where they are first enumerated, and ToList does that to be able to create a list.
Basically the reason why your querying takes so much time can be separated into two different reasons:
you are using too much projections and I thine (the parts with new {
})
your query has an incredible amount of join clauses
I would recommend to separate your query into subqueries, and run them separately like the first part in
...
select o
use
...
select o).ToList()
by breaking down the main query where you have a lot of subqueries it will be faster and much more readable, so you have less "messiness".
And last but not least you should create mapping for the anonymous objects, and use those classes other than projection that should speed up your query.