I have a header-detail relation in my SQL Server database. I have around 10k headers, each of them having 1-1k details. And the number of unique elements is about 1k.
Elements [id]
1
2
3
Headers [id]
1
2
3
Details [id, header_id, element_id]
1 1 1
2 1 2
3 1 3
4 2 1
5 3 1
It's very easy to query a list of headers with their details with such structure:
var searchHeaderIds = new List<int>{1,2,3};
var headers = context.Headers
.Where(h => searchHeaderIds.Contains(h.Id))
.Include(h => h.Details)
.ToList();
But what I want to query is a list of elements (1-200) where every element has a list of headers it belongs to (something like an inversion). I can write it in C# as below:
var searchElementIds = new List<int>{1,2,3};
var headers = context.Details
.Where(d => searchElementIds.Contains(d.element_id))
.GroupBy(d => d.element_id)
.Select(g => new {
id = g.Key,
header_ids = g.Select(x => x.header_id) })
.ToList();
But I wonder, what will be the fastest way to do it using the power of SQL/EF?
UPD: I'm ready to use extra data structures, preprocess the data in the database, or do anything else to improve performance.
what about:
var searchElementIds = new List<int>{1,2,3};
var headers = (
from header in context.Headers
join detail in context.Details on header.id equals detail.header_id
where searchElementIds.Contains(detail.element_id)
select header).Distinct();
If you want instances of the Element class:
var headers =
context.Details
.Where(d => searchElementIds.Contains(d.element_id))
.GroupBy(d => d.element_id)
.Select(g => new Element
{
id = g.Key,
header_ids = g.Select(x => x.header_id
})
.ToList();
Don't cal ToList() in the middle of your query.
This is most optimal query in your case. It is closer to original post, but reduced number of retrieved fields for intermediate result:
var headers = context.Details
.Where(d => searchElementIds.Contains(d.element_id))
.Select(d => new { d.element_id, d.header_id })
.ToList() // we need this, EF do not support retrieving grouping detals
.GroupBy(d => d.element_id)
.Select(g => new Element
{
id = g.Key,
header_ids = g.Select(x => x.header_id).ToList()
})
.ToList();
Related
I have a DataTable (dtResult) with 4 fields, id, store, sku and qty. Unfortunately there are a lot of duplicates in the DataTable that I want to remove (qtys are diff, but other fields are the same).
I want to sort the DataTable by id asc, store asc, sku asc, and group by id, store and sku so I would have a list of unique records.
IDEALLY I would like to overwrite the existing DataTable with the results of the query, and qty can just be 0 for everything. I have the sort, and currently I'm putting it into a new DataTable:
var dtUniqueResults = dtResult.AsEnumerable()
.OrderBy(r => r.Field<string>("id"))
.ThenBy(r => r.Field<string>("store"))
.ThenBy(r => r.Field<string>("sku"))
.CopyToDataTable();
I don't understand how to group with LINQ. I think I need to add something like this, but it's not working.
var dtUniqueResults = dtResult.AsEnumerable()
.GroupBy(n => n.Field<string>("id"),
n => n.Field<string>("store"),
n => n.Field<string>("sku")
)
.OrderBy(r => r.Field<string>("id"))
.ThenBy(r => r.Field<string>("store"))
.ThenBy(r => r.Field<string>("sku"))
.CopyToDataTable();
I've read a lot of posts, and I see several ways of doing it. However it seems the two that are suggested the most are these, but they seem so different it just confuses me more.
GroupBy( x => new { x.Column1, x.Column2 })
AND
GroupBy(x=> new { x.Column1, x.Column2 }, (key, group) => new
{
Key1 = key.Column1,
Key2 = key.Column2,
Result = group.ToList()
});
If you need to filter out duplicates, try the following query:
var dtUniqueResults = dtResult.AsEnumerable()
.GroupBy(n => new
{
Id = n.Field<string>("id"),
Store = n.Field<string>("store"),
Sku = n.Field<string>("sku")
}
)
.SelectMany(g => g.Take(1)) // get from group only one record
.CopyToDataTable();
I have a DB table [Table1] with a one to many relationship.
This related table [Table2] has a type field which is a string.
Table 1 Table 2
Field | Type Field | Type
------|----- ---------|-----
Id | int Id | int
Table1Id | int
Type | string
I am trying to create a summary of how often each combination of types occurs,
and am attempting to do as much work on the DB as possible as it is too slow to bring it all into memory.
The code I have works but seems repetitive to me.
items
.Where(x => x.Table2s.Count > 1)
.Select(x => new
{
Type = x.Table2s.Select(y => y.Type)
})
.ToList() // into memory as string.join has no translation into sql
.Select(x => new
{
Type = string.Join(",", x.Type) // no sql translation
})
.GroupBy(x => x.Type) // can't work on IEnumerable[string]
.Select(x => new Result()
{
Type = x.Key,
Count = x.Count()
})
.OrderByDescending(x => x.Count)
.ToList();
Is there a way to group by this list of strings so that I can do the grouping on the DB and also reduce the number of select statements in my code
Linq to SQL doesn't support Aggregate or String.Join or corresponding SQL tricks, so unless you want to use a stored procedure, some of the work has to happen on the client side.
One alternative would be to create the groupings first and then send them back to the server to count the matches, but that doesn't seem like a gain.
I think the best you can do is something like
var sqlans = items.Where(x => x.Table2s.Count > 0).AsEnumerable();
var ans = sqlans.Select(x => String.Join(",", x.Table2s.Select(t2 => t2.Type).OrderBy(ty => ty).ToArray())).GroupBy(ty => ty, ty => ty, (key, g) => new { key, Count = g.Count() });
I ordered the Types belonging to an item so they would match up when grouped.
The sqlans portion would be executed by the server, but the rest has to execute on the client to process the String.Join.
Instead of using EF I would be tempted to work on Table2 (doing left semi join if you might have orphan Table2 entries) directly
var sqlans = Table2.GroupBy(t2 => t2.Table1Id, t2 => t2.Type, (key, g) => g).AsEnumerable();
var ans = sqlans.Select(x => String.Join(",", x.OrderBy(ty => ty).ToArray())).GroupBy(ty => ty, ty => ty, (key, g) => new { key, Count = g.Count() });
I am new to LINQ and trying to build a website with MVC and LINQ. I want to display top 5 products with the most views. I have 2 tables like below, I am explaining as simple as possible.
PRODUCT
-------
ID
NAME
PRODUCT_VIEWS_TABLE
-------
ID
PRODUCT_ID
Everytime a product is viewed, I insert a new row to the PRODUCT_VIEWS_TABLE. How can I write the LINQ query for this?
(from c in db.PRODUCT select c).Take(5)
How about this:
var top5 = productViews
.GroupBy(view => view.ProductId) // group views by product
.OrderByDescending(g => g.Count()) // order from most- to least-viewed
.Take(5) // take the top 5
.Select(g => g.First().Product); // fetch the corresponding product
var topProductsIds = db.PRODUCT_VIEWS_TABLE // table with a row for each view of a product
.GroupBy(x => x.PRODUCT_ID) //group all rows with same product id together
.OrderByDescending(g => g.Count()) // move products with highest views to the top
.Take(5) // take top 5
.Select(x => x.Key) // get id of products
.ToList(); // execute query and convert it to a list
var topProducts = db.PRODUCTS // table with products information
.Where(x=> topProductsIds.Contains(x.ID)); // get info of products that their Ids are retrieved in previous query
Try this:
var topProducts = m_context.PRODUCTS
.Join(m_context.PRODUCT_VIEW, product=> product.Id, view => view.ProductId,
(product, view ) => new { ProductId = product.Id, ViewId = view.Id})
.GroupBy(g => g.ProductId)
.Select(s => new {
Id = s.Key,
ViewCount= s.Count()
})
.OrderByDescending(o => o.ViewCount)
.Take(5).ToList();
My data structure:
BrowserName(Name) Count(Y)
MSIE9 7
MSIE10 8
Chrome 10
Safari 11
-- and so on------
What I'm trying to do is get the top 10 and then get the sum of rest and call it 'others'.
I'm trying to get the others as below but geting error..
Data.OrderBy(o => o.count).Skip(10)
.Select(r => new downModel { modelname = "Others", count = r.Sum(w => w.count) }).ToList();
The error is at 'r.Sum(w => w.count)' and it says
downModel does not contain a definition of Sum
The downModel just has string 'modelname' and int 'count'.
Any help is sincerely appreciated.
Thanks
It should be possible to get the whole result - the top ten and the accumulated "others" - in a single database query like so:
var downModelList = context.Data
.OrderByDescending(d => d.Count)
.Take(10)
.Select(d => new
{
Name = d.Name,
Count = d.Count
})
.Concat(context.Data
.OrderByDescending(d => d.Count)
.Skip(10)
.Select(d => new
{
Name = "Others",
Count = d.Count
}))
.GroupBy(x => x.Name)
.Select(g => new downModel
{
modelName = g.Key,
count = g.Sum(x => x.Count)
})
.ToList();
If you want to create just one model, then get the sum first and create your object:
var count = Data.OrderBy(o => o.count).Skip(10).Sum(x => x.count);
var model = new downModel { modelname = "Others", count = count };
Btw, OrderBy performs a sort in ascending order. If you want to get (or Skip) top results you need to use OrderByDescending.
If I have a set of entities with 3 properties (Id, Type, Size) all of which are strings.
Is there a way using Linq to Entities where I can do a group query which gives me the Size + Type as the key and then a list of the related Id's for that Size + Type?
Example below of getting the count:
Items.GroupBy(x => new { x.Size, x.Type})
.Select(x => new { Key = x.Key, Count = x.Count() })
but I am looking to get a list of the Ids for each grouping?
I am looking to see if it is possible using Linq-to-EF before I decide to iterate through this in code and build up the result instead.
If you want to get List of Ids for each group then you have to select x.Select(r => r.Id) like:
var result = Items.GroupBy(x => new { x.Size, x.Type })
.Select(x => new
{
Key = x.Key,
Ids = x.Select(r => r.Id)
});
Another way to build up a Dictionary<string, IEnumerable<string?>> in dotnet 6.0 according to the docs;
where we have the dictionary Key as {Size, Type} and Value the list of Ids, you can write:
Dictionary<string, IEnumerable<string?>> result = Items.GroupBy(item => new { item.Size, item.Type }
item => item.Id),
(itemKey, itemIds) =>
{
Key = itemKey,
Ids = itemIds
})
.ToDictionary(x => x.Key, x=> x.Ids);