In T-SQL you can use CROSS APPLY to get all possible variations between the table left and right from the statement. Now I have the following situation in C# and I hope there is a way to solve my problem using LINQ-to-Objects.
I have a list with TestData objects (like below) which is similar to the KeyValuePair<string, object> object (Just a Key and a Value property):
The key can be everything and there can be multiple objects with the same key.
IList<KeyValuePair<String, Object>> objects;
// Content of list
// # | Key | Value
// 1 | "A" | 1
// 2 | "A" | 2
// 3 | "A" | 3
// 4 | "B" | 4
// 5 | "B" | 5
// 6 | "C" | 6
// 7 | "D" | 7
// 8 | "D" | 8
I have also a list of requested keys:
IList<String> requestedKeys = new List<string>() { "A", "D" };
Now I want to have all possible combinations of KeyValuePair objects between the keys in the requestedKeys list.
IList<IList<KeyValuePair<String, Object>>> result = ...
// Content of 'result' will be in this example 6 lists with each 2 KeyValuePair objects
// # | "A" | "D" | (If there are more in the requestedKey list then there are more KeyValuePair items in the innerlist.)
// 1 | 1 | 7 |
// 2 | 2 | 7 |
// 3 | 3 | 7 |
// 4 | 1 | 8 |
// 5 | 2 | 8 |
// 6 | 3 | 8 |
Is it possible to solve my problem using LINQ-to-Objects. If not can you tell me the most efficient way to build it anyway.
EDIT 1:
To make more clear what the result should be:
I want to have a LINQ-to-Objects query something like this:
#Joanna thanks for the tip about multiple froms but the problem is: With this syntax you cannot have a dynamic amount of froms. In my case I need as many froms as items in the requestedKeys list
var result =
from listA in objects.Where(m => m.Key == "A")
from listD in objects.Where(m => m.Key == "D")
// from .....
// from .....
// overhere as many froms as items in 'requestedKeys' list
select new [] { listA, listD /*, All other lists */ }
Something along these lines should work:
var filtered = objects
.Where(o => requestedKeys.Contains(o.Key));
var crossJoined = from el1 in filtered
from el2 in filtered
select new [] {el1, el2};
The cross join is achieved by chaining multiple from clauses.
EDIT:
In this case I can't think of an easier way of doing this than what you started in your edit. The only missing thing is to select the values:
var result =
from listA in objects.Where(m => m.Key == "A").Select(m => m.Value)
from listD in objects.Where(m => m.Key == "D").Select(m => m.Value)
// from .....
// from .....
// overhere as many froms as items in 'requestedKeys' list
select new [] { listA, listD /*, All other lists */ }
I found the solution myself:
It is a very complex join in LINQ because each item in the requestKeys list requires an extra cross join. Regarding to the given example list, the result should be objects.Count(m => m.Key == "A") * objects.Count(m => m.Key == "D") (result is 3 * 2 = 6). Each extra item in the list causes an extra multiply of the whole result set.
So this is the result:
// The result list
IEnumerable<IList<KeyValuePair<char, int>>> result;
// If there are no requestedKeys there is no result expected
if(requestedKeys.Count() > 0)
{
// Loop through all request keys to cross join them together
foreach (var key in requestedKeys)
{
if (result == null)
{
// First time the innerlist List<KeyValuePair<char, int>> will contain 1 item
// Don't forget to use ToList() otherwise the expression will be executed to late.
result = objects.Where(m => m.Key == key).Select(m => new List<KeyValuePair<char, int>>() { m }).ToList();
}
else
{
// Except for the first time the next subresult will be cross joined
var subresult = objects.Where(m => m.Key == key).Select(m => new List<KeyValuePair<char, int>>() { m });
result = result.Join(
subresult,
l1 => 0, // This and the next parameter does the cross join trick
l2 => 0, // This and the previous parameter does the cross join trick
(l1, l2) => l1.Concat(l2).ToList() // Concat both lists which causes previous list plus one new added item
).ToList(); // Again don't forget to 'materialize' (I don't know it is called materialization in LINQ-to-Objects
// but it has simular behaviors because the expression needs to be executed right away)
}
}
}
return result;
Unfortunately it is not completely LINQ so if someone know an better solution. Please comment me or answer my question :)
user this way can genreate sql cross apply:
var comments = AppCommentRepository.Where(com => com.iAction > -1 && productIds.Contains(com.sProductId))
.GroupBy(c => c.sProductId)
.SelectMany(p => p.OrderByDescending(cc => cc.dAddTime).Take(commentNum)).ToList();
finally,the sql is :
SELECT [t3].[iCommentId], .....FROM (
SELECT [t0].[sProductId]
FROM [dbo].[App_Comment] AS [t0]
WHERE ([t0].[iAction] > -1) --AND ([t0].[sProductId] IN (#p1))
GROUP BY [t0].[sProductId]
) AS [t1]
CROSS APPLY (
SELECT TOP (2) [t2].[iCommentId],......
FROM [dbo].[App_Comment] AS [t2]
WHERE ([t1].[sProductId] = [t2].[sProductId]) AND ([t2].[iAction] > -1)
-- AND ([t2].sProductId] IN (#p1))
ORDER BY [t2].[dAddTime] DESC
) AS [t3]
ORDER BY [t3].sProductId DESC
objects
.Join(requestedKeys, o => o.Key, rk => rk, (o, rk) => o)
.SelectMany(o => requestedKeys.Select(k => new {Key = k, Value = o.Value}));
Related
i have 3 lists with common IDs. I need to group by object in one list, and extract data from other two. Will give example for more understanding
table for groupNames:
| Id | Name |
|--------------|
| 1 | Hello |
| 2 | Hello |
| 3 | Hey |
| 4 | Dude |
| 5 | Dude |
table for countId:
| Id | whatever |
|---------------|
| 1 | test0 |
| 1 | test1 |
| 2 | test2 |
| 3 | test3 |
| 3 | test4 |
table for lastTime:
| Id | timestamp |
|-----------------|
| 1 | 1636585230 |
| 1 | 1636585250 |
| 2 | 1636585240 |
| 3 | 1636585231 |
| 3 | 1636585230 |
| 5 | 1636585330 |
and I'm expecting result in list like this
| Name | whateverCnt | lastTimestamp |
|---------------------------------------|
| Hello | 3 | 1636585250 |
| Hey | 2 | 1636585231 |
| Dude | 0 | 1636585330 |
for now i had something like this, but it doesnt work
return groupNames
.GroupBy(x => x.Name)
.Select(x =>
{
return new myElem
{
Name = x.Name,
lastTimestamp = new DateTimeOffset(lastTime.Where(a => groupNames.Where(d => d.Name == x.Key).Select(d => d.Id).Contains(a.Id)).Max(m => m.timestamp)).ToUnixTimeMilliseconds(),
whateverCnt = countId.Where(q => (groupNames.Where(d => d.Name == x.Key).Select(d => d.Id)).ToList().Contains(q.Id)).Count()
};
})
.ToList();
Many thanks for any advice.
I think I'd mostly skip LINQ for this
class Thing{
public string Name {get;set;}
public int Count {get;set;}
public long LastTimestamp {get;set;}
}
...
var ids = new Dictionary<int, string>();
var result = new Dictionary<string, Thing>();
foreach(var g in groupNames) {
ids[g.Id] = g.Name;
result[g.Name] = new Whatever { Name = n };
}
foreach(var c in counts)
result[ids[c.Id]].Count++;
foreach(var l in lastTime){
var t = result[ids[l.Id]];
if(t.LastTimeStamp < l.Timestamp) t.LastTimeStamp = l.TimeStamp;
}
We start off making two dictionaries (you could ToDictionary this).. If groupNames is already a dictionary that maps id:name then you can skip making the ids dictionary and just use groupNames directly. This gives us fast lookup from ID to Name, but we actually want to colelct results into a name:something mapping, so we make one of those too. doing result[name] = thing always succeeds, even if we've seen name before. We could skip on some object creation with a ContainsKey check here if you want
Then all we need to do is enumerate our other N collections, building the result. The result we want is accessed from result[ids[some_id_value_here]] and it always exists if groupnames id space is complete (we will never have an id in the counts that we do not have in groupNames)
For counts, we don't care for any of the other data; just the presence of the id is enough to increment the count
For dates, it's a simple max algorithm of "if known max is less than new max make known max = new max". If you know your dates list is sorted ascending you can skip that if too..
In your example, the safest would be a list of the last specified object and just LINQ query the other arrays of objects for the same id.
So something like
public IEnumerable<SomeObject> MergeListsById(
IEnumerable<GroupNames> groupNames,
IEnumerable<CountId> countIds,
IEnumerable<LastTime> lastTimes)
{
IEnumerable<SomeObject> mergedList = new List<SomeObject>();
groupNames.ForEach(gn => {
mergedList.Add(new SomeObject {
Name = gn.Name,
whateverCnt = countIds.FirstOrDefault(ci => ci.Id == gn.Id)?.whatever,
lastTimeStamp = lastTimes.LastOrDefault(lt => lt.Id == gn.Id)?.timestamp
});
});
return mergedList;
}
Try it in a Fiddle or throwaway project and tweak it to your needs. A solution in pure LINQ is probably not desired here, for readability and maintainability sake.
And yes, as the comments say do carefully consider whether LINQ is your best option here. While it works, it does not always do better in performance than a "simple" foreach. LINQ's main selling point is and always has been short, one-line querying statements which maintain readability.
Well, having
List<(int id, string name)> groupNames = new List<(int id, string name)>() {
( 1, "Hello"),
( 2, "Hello"),
( 3, "Hey"),
( 4, "Dude"),
( 5, "Dude"),
};
List<(int id, string comments)> countId = new List<(int id, string comments)>() {
( 1 , "test0"),
( 1 , "test1"),
( 2 , "test2"),
( 3 , "test3"),
( 3 , "test4"),
};
List<(int id, int time)> lastTime = new List<(int id, int time)>() {
( 1 , 1636585230 ),
( 1 , 1636585250 ),
( 2 , 1636585240 ),
( 3 , 1636585231 ),
( 3 , 1636585230 ),
( 5 , 1636585330 ),
};
you can, technically, use the Linq below:
var result = groupNames
.GroupBy(item => item.name, item => item.id)
.Select(group => (Name : group.Key,
whateverCnt : group
.Sum(id => countId.Count(item => item.id == id)),
lastTimestamp : lastTime
.Where(item => group.Any(g => g == item.id))
.Max(item => item.time)));
Let's have a look:
Console.Write(string.Join(Environment.NewLine, result));
Outcome:
(Hello, 3, 1636585250)
(Hey, 2, 1636585231)
(Dude, 0, 1636585330)
But be careful: List<T> (I mean countId and lastTime) are not efficient data structures here. In the Linq query we have to scan them in order to get Sum and Max. If countId and lastTime are long, turn them (by grouping) into Dictionary<int, T> with id being Key
I asked this question previously, but missed a vital part of my problem.
Return certain record based on criteria
Take this list of results
Client | Date | YESorNO
-------------------------------
A1 | 01/01/2001 | NO
A1 | 01/01/2002 | NO
A1 | 01/01/2003 | YES
A1 | 01/01/2004 | NO
A1 | 01/01/2005 | NO
A1 | 01/01/2006 | NO
A1 | 01/01/2007 | YES
A1 | 01/01/2008 | YES
A1 | 01/01/2009 | YES
A2 | 01/01/2001 | NO
A2 | 01/01/2002 | NO
A2 | 01/01/2003 | YES
A2 | 01/01/2004 | NO
A2 | 01/01/2005 | YES
A2 | 01/01/2006 | YES
A3 | 01/01/2001 | NO
...etc...
The list is ordered chronologically and I cannot sort this is any other way other than descending / ascending.
I cannot sort for Yes | NO and find the First() or Last() as this won't give me the required value.
I want to be able to return the first 'YES' after all 'NO's have been accounted for, per Client.
In the above example for Client[A1] row 7 is the record I want returned (on 01/01/2007).
Client[A2] - row 5 (01/01/2005) ..etc
My code is as follows
var query =
(
from m in db.MyTable
where m.Criteria == XYZ
select new
{
Client = m.Client,
Date = m.Date,
YESorNO = m.YESorNO
}
).OrderBy(x => x.Date);
Using .FirstOrDefault(x => x.YesOrNO == "YES") returns the 3rd record.
User #RenéVogt advised that
var result = query.AsEnumerable()
.TakeWhile(x => x.YESorNO == "YES")
.LastOrDefault();
would get the job done and it does, but I forgot to add that the query will be returning many Clients and I need the first 'YES' for each Client, therefore the above code won't suffice.
Iterating over my results would be hugely time consuming and whilst that is a solution I would prefer this logic to be within the database query itself (if possible)
Many thanks
What you have to do is grouping by client,and then find the last YES of each one starting from the end. Something like this (ClientList is a List<>, you may have to change it depending on where is your data):
var query = ClientList.OrderBy(x => x.client).ThenBy(x => x.date).GroupBy(x => x.client);
foreach (var client in query)
{
var lastYES=client.Reverse().TakeWhile(x => x.YESorNO == "YES")
.LastOrDefault();
Console.WriteLine(String.Format("{0} {1}",client.Key,lastYES.date));
}
//Output: A1 01/01/2007 0:00:00
// A2 01/01/2005 0:00:00
Edit
Mansur Anorboev rightly suggested ordering by descending date, thus eliminating the need of Reverse, so the code would be:
var query = ClientList.OrderBy(x => x.client).ThenByDescending(x => x.date).GroupBy(x => x.client);
foreach (var client in query)
{
var lastYES=client.TakeWhile(x => x.YESorNO == "YES")
.LastOrDefault();
Console.WriteLine(String.Format("{0} {1}",client.Key,lastYES.date));
}
Edit 2
I still was not completly happy with my solution, as it is using a foreach. This does everything in one Linq command:
var query = ClientList.OrderBy(x => x.client)
.ThenByDescending(x => x.date)
.GroupBy(x => x.client, (key, g) => g.TakeWhile(x => x.YESorNO == "YES").LastOrDefault())
.ToList();
This returns a list with one element per client and with the correct date.
I can provide a little sql query
;WITH cte AS (
SELECT *, ROW_NUMBER() OVER (ORDER BY Client DESC) AS rn
FROM [dbo].[tblSkaterhaz]
)
,gte AS (
SELECT Client,max(rn) mx FROM cte
WHERE YesOrNo = 'NO'
GROUP BY Client
)
SELECT cte.* FROM gte
INNER JOIN cte on cte.Client = gte.Client and cte.rn = gte.mx + 1
Although it is not the required solution, but it yields the required result. You can create a stored proc and use it in your code.
NOTE: This is tested against the same table (and data) mentioned in question above
I hope this will be helpful for you.
This question already has answers here:
Is it possible to Pivot data using LINQ?
(7 answers)
Closed 7 years ago.
I have the following data returned from a query:
+-------------------------------------+
| **SUM** **Account** |
+-------------------------------------+
| A & E FEE 182396 |
| CM FEE 108569 |
| CONTRACT VALUE 2256044 |
| OVERHEAD 197397 |
+-------------------------------------+
I need to select into object Properties AeFee, CmFee, ContractValue, and Overhead which would essentially make the properties the column headings so I need the data to look like:
+--------+--------+---------------+----------+
| AeFee | CmFee | ContractValue | Overhead |
+--------+--------+---------------+----------+
| 182396 | 108569 | 2256044 | 197397 |
+--------+--------+---------------+----------+
I tried using a subselect to select something like this (pseudo code)
Select(s => new
{
AeFee = Sum if [Account] == "A & E FEE"
}
This link show exactly what I'm trying to do with a SQL pivot. Converts rows into columns. SQL Pivot
Any ideas?
I always love a question where the answer is Aggregate.
var result = data.Aggregate(
// base object to store
new { A = 0, CM = 0, CO = 0, O = 0},
// add in an element
(a,d) => {
switch(d.sum)
{
case "A & E FEE":
a.A += d.Account;
break;
case "CM FEE":
a.CM += d.Account;
break;
//etc
}
return a;
});
Note you can also - not know the possible values for d.sum in this way -- you would need to use and expando object. That would look something like this (NOT TESTED)
var result = data.Aggregate(
// base object to store
new ExpandoObject()
// add in an element
(a,d.sum.Replace(" ","_") => {
a[d.sum.Replace(" ","_")] += d.Account;
return a;
});
This won't work if your sum strings have values which are not valid in property identifiers.
This produced the results I was looking for
var accounts = new List<string>()
{
"A & E FEE",
"cm fee",
"contract value",
"overhead"
};
var commission = PJPTDSUMs
.Where(p => p.Project.StartsWith("b29317")
&& accounts.Contains(p.Acct)
&& !p.Pjt_entity.StartsWith("05"))
.GroupBy(c => c.Project)
.Select(g => new
{
AeFee = g.Where(p => p.Acct == "A & E FEE").Sum(s => s.Eac_amount),
CmFee = g.Where(p => p.Acct == "cm fee").Sum(s => s.Eac_amount),
ContractValue = g.Where(p => p.Acct == "contract value").Sum(s => s.Eac_amount),
Overhead = g.Where(p => p.Acct == "overhead").Sum(s => s.Eac_amount),
});
+--------+--------+---------------+----------+
| AeFee | CmFee | ContractValue | Overhead |
+--------+--------+---------------+----------+
| 182396 | 108569 | 2256044 | 197397 |
+--------+--------+---------------+----------+
My database has a sales table with entries like so:
_____________________________________
| id | title_id | qty |
-------------------------------------
| 0 | 6 | 10 |
-------------------------------------
| 1 | 5 | 5 |
-------------------------------------
| 2 | 6 | 2 |
-------------------------------------
Title_id is Foreign key pointing to Titles table which is as follows:
_____________________________________
| id | title_id | title |
-------------------------------------
| 0 | 5 | Soda |
-------------------------------------
| 1 | 6 | Coffee |
-------------------------------------
I want to find top 5 sold products wich means i need to calculate the qty value for each product for all it's entried in sales table then order the result by qty in descending order and limit the select to 5.
However I'm new to C# ASP.NET and somewhat new to SQL. I dont know how to do this with LINQ.
This is my code so far:
var getIds = (from sale in db.sales
join tit in db.titles on sale.title_id equals tit.title_id
group sale by sale.qty into result
orderby result.Sum(i => i.qty) descending
select new Publication
{
PubID = sales.title_id, Title = tit.title
}
).Take(5);
Assuming you have a navigation property Sale.Title, something like this should do:
var tops =
db.Sales
.GroupBy( o => o.Title )
.Select( o => new { Title = o.Key, Sum = o.Sum( x => x.Quantity ) } )
.OrderByDescending( o => o.Sum )
.Take( 5 )
.ToList();
tops is then a list of an anonymous type with two properties: the Title object and the sum of the quantities.
You can then get the values like this:
foreach( var top in tops )
{
int titleId = top.Title.title_id;
string title = top.Title.title;
int sumOfQuantities = top.Sum;
...
If you just want the top Title objects, can can select them like this:
List<Title> topTitles = tops.Select( o => o.Title ).ToList();
var result= (from p in sales
let k = new
{
Name = p.Name
}
group p by k into t
orderby Name descending
select new
{
Name = t.Name,
Qty = t.Sum(p => p.Qty)
}).Take(5);
If the entries in the Sales table are more than one per item (ie: in your example you have 'Soda' 10 + 'Soda' 2, then you need to GroupBy(), using the name as the key (or it's related id if it's in another table), but not the qty.
var topSales = db.sales.GroupBy(x => x.title)
.Select(g => new
{
Title = g.Key,
Qty = g.Sum(x => x.qty)
})
.OrderByDescending(x => x.Qty)
.Select(x => new Publication
{
PubID = x.Title.title_id,
Title = x.Title.title1
})
.Take(5)
.ToList();
Note that I've omitted the join statement assuming that you have a foreign key between sales.title_id -> title.id, and you are using LINQ to SQL. Also note that I've avoided using the query syntax in favor of the chained method syntax, I think it's much clear in this use case (although not always true, ie: cross-joins).
Also, SQL and LINQ have some similarities but don't let the names of clauses/methods fool you, LINQ is not SQL, IMHO, Microsoft just tried to make people comfortable by making it look similar ;)
EDIT: fixed GroupBy()
var result= (from p in sales
let k = new
{
Name = p.Name
}
group p by k into t
select new
{
Name = t.Name,
Qty = t.Sum(p => p.Qty)
}).OrderByDescending(i => i.Qty).Take(5);
You need to look at GroupBy; this will give you what you need
http://code.msdn.microsoft.com/101-LINQ-Samples-3fb9811b
Sql:
SELECT date,total_usage_T1 as TotalUsageValue,'T1' as UsageType FROM TblSayacOkumalari
UNION ALL
SELECT date,total_usage_T2 as TotalUsageValue,'T2' as UsageType FROM TblSayacOkumalari
And I try to do to convert it to linq
IEnumerable<TblSayacOkumalari> sayac_okumalari = entity.TblSayacOkumalari
.Select(x => new
{ x.date, x.total_usage_T1 })
.Union(entity.TblSayacOkumalari.Select(x => new
{ x.date, x.total_usage_T2 }));
But I dont know how to convert 'T1' as UsageType to linq. Also my union using is incorrect too.
My table fields like this:
| date | total_usage_T1 | total_usage_T2 |
| 2010 | 30 | 40 |
| 2011 | 40 | 45 |
| 2012 | 35 | 50 |
I want like this
| date | TotalUsageValue | UsageType |
| 2010 | 30 | T1 |
| 2011 | 40 | T1 |
| 2012 | 35 | T1 |
| 2010 | 40 | T2 |
| 2011 | 45 | T2 |
| 2012 | 50 | T2 |
I tried very hard, but could not. Please help.
EDIT
Def. from MSDN
Enumerable.Concat - Concatenates two sequences.
Enumerable.Union - Produces the set union of two sequences by using the default equality comparer.
My post : Concat() vs Union()
IEnumerable<TblSayacOkumalari> sayac_okumalari =
entity.TblSayacOkumalari
.Select(x => new
{
date= x.date,
TotalUsageValue = x.total_usage_T1,
UsageType = "T1"
})
.Concat(entity.TblSayacOkumalari
.Select(x => new
{
date= x.date,
TotalUsageValue = x.total_usage_T2,
UsageType = "T2" }
));
for usage type you juse need to add UsageType = "T2" in your new anonymous type as i did above this will do the task for you
Than you should go for Concat method rather than Union method ..
Example
int[] ints1 = { 1, 2, 3 }; int[] ints2 = { 3, 4, 5 };
IEnumerable<INT> union = ints1.Union(ints2);
Console.WriteLine("Union");
foreach (int num in union)
{
Console.Write("{0} ", num);
}
Console.WriteLine();
IEnumerable<INT> concat = ints1.Concat(ints2);
Console.WriteLine("Concat");
foreach (int num in concat)
{
Console.Write("{0} ", num);
}
output
Fact about Union and Concat
The output shows that Concat() method just combine two enumerable collection to single one but doesn't perform any operation/ process any element just return single enumerable collection with all element of two enumerable collections.
Union() method return the enumerable collection by eliminating the duplicate i.e just return single element if the same element exists in both enumerable collection on which union is performed.
Important point to Note
By this fact we can say that Concat() is faster than Union() because it doesn't do any processing.
But if after combining two collection using Concat() having single collection with too many number of duplicate element and if you want to perform further operation on that created collection takes longer time than collection created using Union() method, because Union() eliminate duplicate and create collection with less elements.
Use this:
var result = entity.TblSayacOkumalari
.Select(x => new
{
Date = x.date,
TotalUsage = x.total_usage_T1,
UsageType = "T1"
})
.Union(entity.TblSayacOkumalari.Select(x => new
{
Date = x.date,
TotalUsage = x.total_usage_T2,
UsageType = "T2"
}));
In order to get the expected property names on the anonymous type you probably want to do something like:
new { x.date, TotalUsage = x.total_usage_T1, UsageType="T1" }
and also
new { x.date, TotalUsage = x.total_usage_T2, UsageType="T2" }