Develop Reference

Filter LINQ To Entities (EF Core) query by List of HashSet of String and Enum

I have this Linq to Entities (EF Core) query which looks like below
var query = (from p in db.Samples
join q in db.Items on p.Id equals q.SampleId
Where p.active = IsActive and p.Id = GivenId
group new
{
p.Name,
p.Address,
p.Marks,
p.LocationId,
q.EmailId,
q.Grade
}
by new
{ q.Grade }
into data
select new DataSummary()
{
UserName = data.Name,
Grade = data.Min(x => x.Grade),
Email = data.Min(x => x.Email,
Total = data.Sum(x => x.Marks)
}.ToList()
Now I have a constant List of Hashset of Grades and Location that looks like this:
public List<(HashSet<string> Grades, HashSet<Location> Loctions)> LocationGrades => new()
{
(new() { "A", "B" }, new()), // Includes all location
(new() { "C"}, new(){
Location.Boston, //Location is Enum
Location.Maine
}
}
I want to get the data where if the student has grade A or B include all location and if the student has grade C only include Boston and Maine.
Is it possible to integrate this within the LINQ to Entities query?
Sample Table
| ID | Name | Address | Marks | LocationId |
|-----|-------|---------|-------|-------------|
| 234 | Test | 123 St | 240 | 3 (Maine) |
| 122 | Test1 | 234 St | 300 | 5 (Texas) |
| 142 | Test1 | 234 St | 390 | 1 (Boston) |
Items Table
| ID | SampelId | Grade | Email |
|----|----------|-------|-------|
| 12 | 234 | A | a.com |
| 13 | 122 | C | b.com |
| 14 | 142 | C | c.com |
So, In the table above I shouldn't get Texas row but get Boston row as they both have Grade C but Texas does not exist in the HashSet combo.

Okay, now I got it. You have to add dynamic ORed constraints to the query based on a given list of elements. This is a little tricky, because AND can be done with using multiple .Where() statements, but OR not. I did something similar recently against CosmosDB by using LinqKit and the same should also work against EF.
In your case you probably of to do something like this:
...
into data
.WhereAny(grades, (item, grade) => item.Grade == grade)
select new DataSummary()
...
I think the given example doesn't match your exact case, but it allows you to define multiple ORed constraints from a given list and I think this is the missing part you're searching. Take care to use within the lambda method only definitions which are also supported by EF core. The given inner enumeration (in this example grades) will be iterated on the client side and can be dynamically build with everything available in C#.

How to merge multiple list by id and get specific data?

i have 3 lists with common IDs. I need to group by object in one list, and extract data from other two. Will give example for more understanding
table for groupNames:
| Id | Name |
|--------------|
| 1 | Hello |
| 2 | Hello |
| 3 | Hey |
| 4 | Dude |
| 5 | Dude |
table for countId:
| Id | whatever |
|---------------|
| 1 | test0 |
| 1 | test1 |
| 2 | test2 |
| 3 | test3 |
| 3 | test4 |
table for lastTime:
| Id | timestamp |
|-----------------|
| 1 | 1636585230 |
| 1 | 1636585250 |
| 2 | 1636585240 |
| 3 | 1636585231 |
| 3 | 1636585230 |
| 5 | 1636585330 |
and I'm expecting result in list like this
| Name | whateverCnt | lastTimestamp |
|---------------------------------------|
| Hello | 3 | 1636585250 |
| Hey | 2 | 1636585231 |
| Dude | 0 | 1636585330 |
for now i had something like this, but it doesnt work
return groupNames
.GroupBy(x => x.Name)
.Select(x =>
{
return new myElem
{
Name = x.Name,
lastTimestamp = new DateTimeOffset(lastTime.Where(a => groupNames.Where(d => d.Name == x.Key).Select(d => d.Id).Contains(a.Id)).Max(m => m.timestamp)).ToUnixTimeMilliseconds(),
whateverCnt = countId.Where(q => (groupNames.Where(d => d.Name == x.Key).Select(d => d.Id)).ToList().Contains(q.Id)).Count()
};
})
.ToList();
Many thanks for any advice.

I think I'd mostly skip LINQ for this
class Thing{
public string Name {get;set;}
public int Count {get;set;}
public long LastTimestamp {get;set;}
}
...
var ids = new Dictionary<int, string>();
var result = new Dictionary<string, Thing>();
foreach(var g in groupNames) {
ids[g.Id] = g.Name;
result[g.Name] = new Whatever { Name = n };
}
foreach(var c in counts)
result[ids[c.Id]].Count++;
foreach(var l in lastTime){
var t = result[ids[l.Id]];
if(t.LastTimeStamp < l.Timestamp) t.LastTimeStamp = l.TimeStamp;
}
We start off making two dictionaries (you could ToDictionary this).. If groupNames is already a dictionary that maps id:name then you can skip making the ids dictionary and just use groupNames directly. This gives us fast lookup from ID to Name, but we actually want to colelct results into a name:something mapping, so we make one of those too. doing result[name] = thing always succeeds, even if we've seen name before. We could skip on some object creation with a ContainsKey check here if you want
Then all we need to do is enumerate our other N collections, building the result. The result we want is accessed from result[ids[some_id_value_here]] and it always exists if groupnames id space is complete (we will never have an id in the counts that we do not have in groupNames)
For counts, we don't care for any of the other data; just the presence of the id is enough to increment the count
For dates, it's a simple max algorithm of "if known max is less than new max make known max = new max". If you know your dates list is sorted ascending you can skip that if too..

In your example, the safest would be a list of the last specified object and just LINQ query the other arrays of objects for the same id.
So something like
public IEnumerable<SomeObject> MergeListsById(
IEnumerable<GroupNames> groupNames,
IEnumerable<CountId> countIds,
IEnumerable<LastTime> lastTimes)
{
IEnumerable<SomeObject> mergedList = new List<SomeObject>();
groupNames.ForEach(gn => {
mergedList.Add(new SomeObject {
Name = gn.Name,
whateverCnt = countIds.FirstOrDefault(ci => ci.Id == gn.Id)?.whatever,
lastTimeStamp = lastTimes.LastOrDefault(lt => lt.Id == gn.Id)?.timestamp
});
});
return mergedList;
}
Try it in a Fiddle or throwaway project and tweak it to your needs. A solution in pure LINQ is probably not desired here, for readability and maintainability sake.
And yes, as the comments say do carefully consider whether LINQ is your best option here. While it works, it does not always do better in performance than a "simple" foreach. LINQ's main selling point is and always has been short, one-line querying statements which maintain readability.

Well, having
List<(int id, string name)> groupNames = new List<(int id, string name)>() {
( 1, "Hello"),
( 2, "Hello"),
( 3, "Hey"),
( 4, "Dude"),
( 5, "Dude"),
};
List<(int id, string comments)> countId = new List<(int id, string comments)>() {
( 1 , "test0"),
( 1 , "test1"),
( 2 , "test2"),
( 3 , "test3"),
( 3 , "test4"),
};
List<(int id, int time)> lastTime = new List<(int id, int time)>() {
( 1 , 1636585230 ),
( 1 , 1636585250 ),
( 2 , 1636585240 ),
( 3 , 1636585231 ),
( 3 , 1636585230 ),
( 5 , 1636585330 ),
};
you can, technically, use the Linq below:
var result = groupNames
.GroupBy(item => item.name, item => item.id)
.Select(group => (Name : group.Key,
whateverCnt : group
.Sum(id => countId.Count(item => item.id == id)),
lastTimestamp : lastTime
.Where(item => group.Any(g => g == item.id))
.Max(item => item.time)));
Let's have a look:
Console.Write(string.Join(Environment.NewLine, result));
Outcome:
(Hello, 3, 1636585250)
(Hey, 2, 1636585231)
(Dude, 0, 1636585330)
But be careful: List<T> (I mean countId and lastTime) are not efficient data structures here. In the Linq query we have to scan them in order to get Sum and Max. If countId and lastTime are long, turn them (by grouping) into Dictionary<int, T> with id being Key

How to increate the performance when using group in LINQ

Database
Cars | CarDetails | Owners
----------------------------------
Id | CarDetailId | Id
Name | CarId | CarId
Type | CarId | OwnerName
| | PhoneNumber
LINQ Code
var intiQuery = from c in Cars
join cd in CarDetails
join o in Owners
select new { c,cd,o}
var results = from qry in intiQuery
group new { qry.c, qry.cd, qry.o} by qry.c.Id into g
select new
select new { CarId= g.Key,
Name = g.Select(g=>g.c.Name).FirstOrDefault(),
Type = g.Select(g=>g.c.Type).FirstOrDefault(),
Price= g.Select(g=>g.cd.Price).FirstOrDefault(),
OwnerName= g.Select(g=>g.o.OwnerName).FirstOrDefault(),
PhoneNumber= g.Select(g=>g.o.PhoneNumber).FirstOrDefault(),
}
My question is simply how to increase the performance when calling this query as you can see for each field, I need to.Select().FirstOrDefault() to get the corresponding data. If let's says I got 100 data I will need to get the data one by one 500 times it will take ages to display the data.
Extra Info in case someone not clear.
Cars
Id |Name |Type
-----------------------------------
1 |Toyota |Family
CarDetails
CarDetailId | CarId | Price
-----------------------------------
1 | 1 | 200000
Owners
Id| CarId | OwnerName | PhoneNumber
-----------------------------------
1 | 1 | Mitch | 48774800
2 | 1 | Camilo | 87404078
The result I wanted to get is something like this hope some of you can have a clearer picture
CarId| Name | Type | Price |OwnerName |PhoneNumber
----------------------------------------------------------------------
1 | Toyota | Family | 200000 | Mitch,Camilo | 48774800,87404078

This is absolutely not the best solution, but it will certainly be easy to understand and if you are not filtering the data then it may be good enough.
As you state, the performance problem is because your hitting your database hundreds of time, and so we can easily avoid that by simply pulling the information from the database in three simple queries.
var allCars = Cars.ToList();
var allCarDetails = CarDetails.ToList();
var allOwners= Owners.ToList();
Once you have all this information in memory, you can manipulate the in-memory objects to produce the results you need.
var results = (from car in allCars
let owners = allOwners.Where(a => a.CarID == car.Id)
select new
{
CarID = car.Id,
car.Name,
car.Type,
Price = allCarDetails.Where(a => a.CarID == car.Id).Select(a => a.Price).SingleOrDefault(),
OwnerName = String.Join(',', owners.Select(a => a.Name)),
PhoneNumber = String.Join(',', owners.Select(a => a.PhoneNumber))
}
);
If your tables have a lot of extra fields which you have not mentioned, then we may want to change the initial three queries to just pull the information required.

How to select 'product' rows instead of 'sale' rows

In my LINQ query below I want to select the 'product' rows and add the 'sale' rows data into it but the opposite is happening, it's selecting the 'sale' rows and adding the 'product' rows
var query = (from product in SampleProductTable
from sale in SampleSalesTable
where (sale.ProductId == product.Id)
select new
{
Id = product.Id,
TotalSales = product.TotalSales + ((product.Id == sale.ProductId) ? sale.Amount : 0)
})
Sample Product Table
+-------+------------+---------+-----------------+-------+------------+
| Id | CategoryId | BrandId | Name | Price | TotalSales |
+-------+------------+---------+-----------------+-------+------------+
| mlk3 | MLK | BRND1 | Creamy Milk | 5 | 10 |
| snck2 | SNCK | BRND2 | Chocolate Snack | 2 | 24 |
+-------+------------+---------+-----------------+-------+------------+
Sample Sales Table
+-----+-----------+--------+
| Id | ProductId | Amount |
+-----+-----------+--------+
| 120 | mlk3 | 55 |
| 121 | mlk3 | 15 |
| 122 | snck2 | 12 |
| 123 | mlk3 | 5 |
| 124 | mlk3 | 10 |
| 125 | snck2 | 2 |
| 126 | mlk3 | 115 |
| 127 | snck2 | 6 |
| 128 | snck2 | 34 |
+-----+-----------+--------+
Desired Output
+-------+------------+
| Id | TotalSales |
+-------+------------+
| mlk3 | 210 |
| snck2 | 78 |
+-------+------------+

var answer = (from product in SampleProductTable
join sale in SampleSalesTable on product.Id == sale.ProductId into subSales
from subSale in subSales.DefaultIfEmpty()
group subSale by new { product.Id, product.TotalSales } into gr
select new
{
gr.Key.Id,
TotalSales = gr.Sum(x => x == null ? 0 : x.Amount) + gr.Key.TotalSales
}).ToList();
Approximate T-SQL:
select p.Id, p.TotalSales + sum(coalesce(s.Amount, 0)) TotalSales
from SampleProductTable p
left outer join SampleSalesTable s on p.Id = s.ProductId
group by p.Id, p.TotalSales

In your example you join two collections the way that the result will have as many rows as there are distinct child records (sales in this case) and create a new object for each record (similar to INNER JOIN). That's why the result is "sales-based".
If I understand your intent correctly, I would approach it like:
SampleProductTable.Select(p => new
{
Id = p.Id,
TotalSales = p.Sales.Sum(s => s.Amount)
}
please note that for this approach you will need to map "Sales" collection on a product.

First of all it would be better to use the join statement instead, and then, it seems you need to group your Sales tables based on the ProductId:
var query = (from product in SampleProductTable
join sale in SampleSalesTable.GroupBy(c => c.ProductId)
on product.Id equals sale.Key
select new
{
Id = product.Id,
TotalSales = product.TotalSales + sale.Sum(c=>c.Amount)
}).ToList();
Also please note: since you used a where statement in your code, you don't need to use this condition (product.Id == sale.ProductId) ? in your select anymore. Same as mine, because I used the join statement with on keyword, there is no need to use the condition in the select area.
You can see your desired result in the following link:
https://dotnetfiddle.net/RFTtrv

In general LINQ terms the query shape you are looking for is called grouped join:
The group join is useful for producing hierarchical data structures. It pairs each element from the first collection with a set of correlated elements from the second collection.
In your case, it will produce a collection of correlated Sales for each Product. Then all you need is to apply aggregate (Sum) inside the final projection (select):
var query =
from product in SampleProductTable
join sale in SampleSalesTable on product.Id equals sale.ProductId into productSales
select new
{
Id = product.Id,
TotalSales = product.TotalSales + productSales.Sum(sale => sale.Amount)
};
But since in some of the comments you mentioned converting to SQL, most likely you are using some ORM like LinqToSQL, EF or EF Core. In such case the things are even simpler. These ORMs support a so called navigation properties which represent the relationships, and when used inside queries are translated to SQL with all the necessary joins, so you don't need to be bothered with such details and can concentrate on the logic needed to produce the desired result.
If that's the case, the Product class would normally have something like
public ICollection<Sale> Sales { get; set; }
and the query in question would be simple Select like this:
var query = db.Products
.Select(product => new
{
Id = product.Id,
TotalSales = product.TotalSales + product.Sales.Sum(sale => sale.Amount)
});

LEFT JOIN with grouping looks like
var query =
from product in SampleProductTable
join sale in SampleSalesTable.GroupBy(c => c.ProductId)
on product.Id equals sale.Key into join1
from lj in join1.DefaultIfEmpty() // left join
select new
{
Id = product.Id,
TotalSales = product.TotalSales + (lj == null ? 0 : lj.Sum(c => c.Amount))
};
Left join may return null, so check the potential group, lj before trying to sum it. For later c# versions null check could be abbreviated to
TotalSales = product.TotalSales + (lj?.Sum(c => c.Amount) ?? 0)
Fiddle

In query syntax, Slava's solution should return with the result you're looking for i.e.
var querySyntax = (from product in SampleProductTable
join sale in SampleSalesTable on product.Id equals sale.ProductId into sales
from subSales in sales.DefaultIfEmpty()
group subSales by new { product.Id, product.TotalSales }
into grp
select new
{
grp.Key.Id,
TotalSales = grp.Sum(s => s.Amount) + grp.Key.TotalSales
}).ToList();
If you have a burning desire to use method syntax for whatever reason, this equivalent LINQ query will also work:
var methodSyntax = (SampleProductTable
.GroupJoin(SampleSalesTable, product => product.Id, sale => sale.ProductId,
(product, sales) => new {product, sales})
.SelectMany(s => s.sales.DefaultIfEmpty(), (s, subSales) => new {s, subSales})
.GroupBy(ss => new {ss.s.product.Id, ss.s.product.TotalSales}, ss => ss.subSales)
.Select(grp => new {grp.Key.Id, TotalSales = grp.Sum(s => s.Amount) + grp.Key.TotalSales})).ToList();

Linq select result with duplicating rows

My select returns joined data from db where for every object there are several attributes. For every attribute of the object I have a new row.
I use Entity Framework to retrieve the data.
var products = _ctx.ExecuteFunction<GetProducts_Result>("GetProducts");
GetProducts is a stored proc that returns following result.
Code | Name | Term | Rate
--------+------+------+-----
111 | Foo | 12 | 10
111 | Foo | 24 | 12
111 | Foo | 36 | 16
222 | Boo | 12 | 8
222 | Boo | 24 | 9
How I can efficiently query data with linq and map to a class like
class Product
{
int Code
string Name
List<Term> terms;
}
Where Term is a class with parameters (Term, Rate)

var products = from p in _ctx.ExecuteFunction<GetProducts_Result>("GetProducts")
group p by new {p.Code, p.Name} into g
select new Product
{
Code = g.Key.Code,
Name = g.Key.Name,
terms = g.Select(x => new Terms {
Term = x.Term,
Rate = x.Rate }).ToList()
};