Develop Reference

How to GroupBy and Order by multiple fields with LINQ

I have a DataTable (dtResult) with 4 fields, id, store, sku and qty. Unfortunately there are a lot of duplicates in the DataTable that I want to remove (qtys are diff, but other fields are the same).
I want to sort the DataTable by id asc, store asc, sku asc, and group by id, store and sku so I would have a list of unique records.
IDEALLY I would like to overwrite the existing DataTable with the results of the query, and qty can just be 0 for everything. I have the sort, and currently I'm putting it into a new DataTable:
var dtUniqueResults = dtResult.AsEnumerable()
.OrderBy(r => r.Field<string>("id"))
.ThenBy(r => r.Field<string>("store"))
.ThenBy(r => r.Field<string>("sku"))
.CopyToDataTable();
I don't understand how to group with LINQ. I think I need to add something like this, but it's not working.
var dtUniqueResults = dtResult.AsEnumerable()
.GroupBy(n => n.Field<string>("id"),
n => n.Field<string>("store"),
n => n.Field<string>("sku")
)
.OrderBy(r => r.Field<string>("id"))
.ThenBy(r => r.Field<string>("store"))
.ThenBy(r => r.Field<string>("sku"))
.CopyToDataTable();
I've read a lot of posts, and I see several ways of doing it. However it seems the two that are suggested the most are these, but they seem so different it just confuses me more.
GroupBy( x => new { x.Column1, x.Column2 })
AND
GroupBy(x=> new { x.Column1, x.Column2 }, (key, group) => new
{
Key1 = key.Column1,
Key2 = key.Column2,
Result = group.ToList()
});

If you need to filter out duplicates, try the following query:
var dtUniqueResults = dtResult.AsEnumerable()
.GroupBy(n => new
{
Id = n.Field<string>("id"),
Store = n.Field<string>("store"),
Sku = n.Field<string>("sku")
}
)
.SelectMany(g => g.Take(1)) // get from group only one record
.CopyToDataTable();

EF Core header-detail query optimization

I have a header-detail relation in my SQL Server database. I have around 10k headers, each of them having 1-1k details. And the number of unique elements is about 1k.
Elements [id]
1
2
3
Headers [id]
1
2
3
Details [id, header_id, element_id]
1 1 1
2 1 2
3 1 3
4 2 1
5 3 1
It's very easy to query a list of headers with their details with such structure:
var searchHeaderIds = new List<int>{1,2,3};
var headers = context.Headers
.Where(h => searchHeaderIds.Contains(h.Id))
.Include(h => h.Details)
.ToList();
But what I want to query is a list of elements (1-200) where every element has a list of headers it belongs to (something like an inversion). I can write it in C# as below:
var searchElementIds = new List<int>{1,2,3};
var headers = context.Details
.Where(d => searchElementIds.Contains(d.element_id))
.GroupBy(d => d.element_id)
.Select(g => new {
id = g.Key,
header_ids = g.Select(x => x.header_id) })
.ToList();
But I wonder, what will be the fastest way to do it using the power of SQL/EF?
UPD: I'm ready to use extra data structures, preprocess the data in the database, or do anything else to improve performance.

what about:
var searchElementIds = new List<int>{1,2,3};
var headers = (
from header in context.Headers
join detail in context.Details on header.id equals detail.header_id
where searchElementIds.Contains(detail.element_id)
select header).Distinct();
If you want instances of the Element class:
var headers =
context.Details
.Where(d => searchElementIds.Contains(d.element_id))
.GroupBy(d => d.element_id)
.Select(g => new Element
{
id = g.Key,
header_ids = g.Select(x => x.header_id
})
.ToList();
Don't cal ToList() in the middle of your query.

This is most optimal query in your case. It is closer to original post, but reduced number of retrieved fields for intermediate result:
var headers = context.Details
.Where(d => searchElementIds.Contains(d.element_id))
.Select(d => new { d.element_id, d.header_id })
.ToList() // we need this, EF do not support retrieving grouping detals
.GroupBy(d => d.element_id)
.Select(g => new Element
{
id = g.Key,
header_ids = g.Select(x => x.header_id).ToList()
})
.ToList();

How to merge duplicate rows (Based on column) of data table & put it in separate data table?

I have a data table which returns result something like following :
What i wanted to do is group by them based on column : cfg with the respective count.
The desired output should be like :
i tried with following LINQ syntax to get this done :
DataTable objDataTable = DLService.GetData();
var distinctTable = from row in objDataTable.AsEnumerable()
group row by row.Field<string>("cfg") into newProps
orderby newProps.Key
select new
{
PropName = newProps.Key,
CountOfProps = newProps.Count()
};
i am able to group by cfg using this but not able to get count (sum) based on cfg.
as there are 4 entries for 'Apartment', what i am getting is : 4 Apartment (its returning count of entries). it should return : 74 Apartment
Can anyone provide solution to get this done? any improvements in the syntax or any other method?
Please note that i don't want to group by using sql query.

In your case Group by returns an IEnumerable<IGrouping<object, DataRow>>. What you need to to is sum up all the values in the grouping:
Dictionary<string, int> counts = dt.AsEnumerable()
.GroupBy(x => x["cfg"])
.OrderBy(x => x.Key)
.ToDictionary(
x => (string) x.Key,
x => x.Select(row => (int) row["count"]).Sum()
);

Try following. :
var distinctTable = objDataTable.AsEnumerable()
.GroupBy(x => row.Field<string>("cfg"))
.OrderBy(x => x.Key)
.Select( x => new { PropName = x.Key, CountofProps = x.Count}).ToList();

LINQ: how to get a group of a table ordering with a related table?

I have a doubt about the object IGrouping that results from a linq where I use a "group by" sentence.
I have two tables in the database, Products and Responses they have a relationship 1 to *. In the Responses table we have a column called FinalRate which is the rate of the product. The products can have n responses or rates.
I want to get the Products order by the sum of the FinalRate divided by the number of rates done. That is to say, order by the average rate descending from higher to lower marks.
As it can be read in the code (at the end of the question), I try to get the responses first. To sum all the finalrates and divide them by the count I use a group.
There are 2 problems with the code, even if the current code works:
1.-I tried to get the Products in a single query but it is impossible because I can not use the products table in the group and then use the Response table in the "orderby". One more thing LINQ only gives you the possibility to group one table, it is imposible to have "group prod, response".
I couldn't get this sql sentence in LINQ:
select prod.ProductID,prod.Commercial_Product_Name,prod.Manufacturer_Name,
prod.ProductImageUrl
from rev_product prod
inner join rev_response res on res.AtProductid=prod.ProductID
group by prod.ProductID,prod.Commercial_Product_Name,prod.Manufacturer_Name
,prod.ProductImageUrl
order by (sum(res.FinalRate)/count(res.AtProductid))
I tried this:
var gruposproductos = (from prod in ctx.Products
join res in ctx.Responses on prod.ProductID equals res.AtProductId
group prod by prod.ProductID into g
orderby (g.Sum(ra =>ra.FinalRate)/g.Count())
descending select g).Take(2);
But as I say, the "orderby (g.Sum..." gives an error, because "into g" groups the Product table, not the Response Table.
So this is why in my final code I don't get the products in the same LINQ sentence.
2.-Once accepted this fact, the problem is that I get an IGrouping, but I don't obtain a list of Responses that I can iterate without doing the two foreach in the code. I wanted only one loop, as one would do if you had a "List" object.
It is not really a cool method but it works. Moreover, I have to control that in the second loop there is only added 1 time.
Any better code?
var groupproducts = (from res in ctx.Responses
group res by res.AtProductId into g
orderby (g.Sum(ra =>ra.FinalRate)/g.Count())
descending select g).Take(2).ToList();
List<Product> theproducts = new List<Product>();
foreach (var groupresponse in groupproducts)
{
foreach (var response in groupresponse)
{
var producttemp= (from prod in ctx.Products
where prod.ProductID == response.AtProductId
select prod).First();
theproducts.Add(producttemp);
}
}
}
FINAL SOLUTION (thx a lot #Daniel)
var productsanonymtype = ctx.Products.Select(x => new
{
Product = x,
AverageRating = x.Responses.Count() == 0 ? 0 : x.Responses.Select(r => (double)r.FinalRate).Sum() / x.Responses.Count()
}).OrderByDescending(x => x.AverageRating);
List<Product> products = new List<Product>();
foreach (var prod in productsanonymtype)
{
products.Add(prod.Product);
}

Try this:
products.Select(x => new
{
Product = x,
AverageRating = x.Responses.Sum(x => x.FinalRate) /
x.Responses.Count()
});
The Sum overload I am using is not implemented in all providers. If that's a problem for you, you can use this alternate version:
products.Select(x => new
{
Product = x,
AverageRating = x.Responses.Select(x => x.FinalRate)
.Sum() /
x.Responses.Count()
});
If there is no navigation property from product to its responses you should first try to fix that. If you can't you can use this version:
products.Join(responses, x => x.Id, x => x.ProductId,
(p, r) => new { Product = p, Response = r })
.GroupBy(x => x.Product)
.Select(g => new { Product = g.Key,
AverageRating = g.Select(x => x.Response.FinalRate)
.Sum() /
g.Count()
});
Assuming FinalRate is an int, both methods will calculate the average rating with an int, i.e. there will be no 4.5 rating. And there will be no rounding, i.e. an actual average rating of 4.9 will result in 4. You can fix that by casting one of the operands of the division to double.
Another problem is the case with no ratings so far. The code above will result in an exception in this case. If that's a problem for you, you can change the calculation to this:
AverageRating = g.Count() == 0
? 0
: g.Select(x => (double)x.Response.FinalRate).Sum() / g.Count()

ctx.Products.GroupBy(x => new {
ProductId = x.ProductId,
FinalRate = x.Responses.Sum(y => y.FinalRate),
CountProductId = x.Responses.Count
})
.OrderBy(x => x.Key.FinalRate / x.Key.CountProductId);
And here with the projection.....
ctx.Products.Select(x => new {
ProductID = x.ProductID,
Commercial_Product_Name = x.Commercial_Product_Name,
Manufacturer_Name = x.Manufacturer_Name,
ProductImageUrl = x.ProductImageUrl,
FinalRate = x.Responses.Sum(y => y.FinalRate),
CountProductId = x.Responses.Count
})
.GroupBy(x => new {
ProductId = x.ProductId,
FinalRate = x.FinalRate,
CountProductId = x.CountProductId
})
.OrderBy(x => x.Key.FinalRate / x.Key.CountProductId);

Find duplicate and merge record into single datatable c#

I am able to find the duplicates out of DataTable rows. Like following:
var groups = table.AsEnumerable()
.GroupBy(r => new
{
c1 = r.Field<String>("Version"),
});
var tblDuplicates = groups
.Where(grp => grp.Count() > 1)
.SelectMany(grp => grp)
.CopyToDataTable();
Now, I want to merge all the duplicate records in to single and sum it's Value column value.
Pretty much like following:
DataTable with Duplicates:
Version Value
1 2
2 2
2 1
1 3
2 1
3 2
DataTable with no duplicates and Value summed.:
Version Value
1 5
2 4
3 2
I am aware about this link which does this with the help of reflection.
http://forums.asp.net/t/1570562.aspx/1
Anyother way to do it?
Edit:
However, if I have more than two columns, like five columns and I still want to do the sum on Value column and also need other columns data in resulatant summed datatable. How to do it? Here I get the Version and Value in my result DataTable. I want other columns with values also. Like following:
Version col1 col2 Value
1 A A 2
2 B B 2
2 B B 1
1 A A 3
2 B B 1
3 C C 2

var result = table.AsEnumerable()
.GroupBy(r => r.Field<string>("Version"))
.Select(g =>
{
var row = table.NewRow();
row.ItemArray = new object[]
{
g.Key,
g.Sum(r => r.Field<int>("Value"))
};
return row;
}).CopyToDataTable();
Edit:
If you want to keep other field, try below:
var result = table.AsEnumerable()
.GroupBy(r => new
{
Version = r.Field<String>("Version"),
Col1 = r.Field<String>("Col1"),
Col2 = r.Field<String>("Col2")
})
.Select(g =>
{
var row = g.First();
row.SetField("Value", g.Sum(r => r.Field<int>("Value")));
return row;
}).CopyToDataTable();