Speed up LINQ Object Query - c#

I am attempting to speed up the following LINQ Object query:
var setOfCodes = codeList1
.SelectMany(q => q.Codes)
.Union(codeList2.SelectMany(q => q.Codes)
.Union(codeList3.SelectMany(q => q.Codes)
.ToList();
Where
codeListX is a List<Item>
and
public Item {
public List<int> Codes = new List<int>();
}
Example:
var codeList1 = new List<Item> {
new Item {
Codes = new List<int> {
100,
105,
110
}
},
new Item {
Codes = new List<int> {
100,
110,
115
}
},
};
var codeList2 = new List<Item> {
new Item {
Codes = new List<int> {
150,
155,
160
}
},
new Item {
Codes = new List<int> {
150,
155,
170
}
},
};
And the output should be (Not fussed about order, I can sort later):
100, 105, 110, 115, 150, 155, 160, 170
IE: Outputs a list containing all the codes that appear within the codeListX's.
Is there a faster way to do this?

You can write it like this :
var setOfCodes = new[] { codeList1, codeList2, codeList3 }
.SelectMany(x => x)
.SelectMany(x => x.Codes)
.Distinct()
.ToList();

Related

Is it possible to group make this with LINQ?

I have the following method which returns an IList<IGrouping<bool, int>>, I was wondering if it was possible to do this using LINQ.(Test case beneath I wrote to see what should happen, small numbers (<=100) should have a 'yes' boolean and big numbers (>100) should have a 'no' boolean.
public IList<IGrouping<bool, int>> GroupSmallAndBigNumbers(int[] numbers)
{
}
[Test]
public void NumbersSmallerThanOrEqualTo100AndBiggerNumbersCanBeGroupedUsingGroupBy()
{
//Arrange
int[] numbers = { 5, 700, 15, 108, 25, 28, 100 };
int[] expectedSmallNumbers = { 5, 15, 25, 28, 100};
int[] expectedBigNumbers = { 700, 108 };
//Act
var results = _examples.GroupSmallAndBigNumbers(numbers);
//Assert
Assert.That(results.Count, Is.EqualTo(2));
var smallNumbers = results[0];
Assert.That(smallNumbers, Is.EquivalentTo(expectedSmallNumbers));
var bigNumbers = results[1];
Assert.That(bigNumbers, Is.EquivalentTo(expectedBigNumbers));
}
You could group by checking if each number is greater than 100.
public IList<IGrouping<bool, int>> GroupSmallAndBigNumbers(int[] numbers)
=> numbers.GroupBy(x => x >= 100).ToList();
I think a Dictionary would work better (and easier) for you. Using that, you can easily do something like this:
int[] numbers = { 5, 700, 15, 108, 25, 28, 100 };
Dictionary<int, bool> dict = new Dictionary<int, bool>();
foreach (int number in numbers)
{
if (number > 100)
dict.Add(number, true);
else
dict.Add(number, false);
}

Group By Five Minutes TimeSpan

I have the following code
TimeSpan[] hours = new[] {
new TimeSpan(10,35,50),
new TimeSpan(10,36,48),
new TimeSpan(10,41,48),
new TimeSpan(10,47,58),
new TimeSpan(10,49,14),
new TimeSpan(11,22,15),
new TimeSpan(11,24,18),
new TimeSpan(11,25,25),
};
I want to group the hours and minutes by 5 minutes.
I want to get the result below
1st Group
**10:35:50
10:36:48
10:41:48**
2nd Group
**10:47:58
10:49:14**
3rd Group
**11:22:15
11:24:18
11:25:25**
I have tried the following code but I cannot get the exact result that I need
var query = (from x in hours select x)
.GroupBy(r=> r.Ticks / TimeSpan.FromMinutes(5).Ticks));
You can try to implement a simple loop instead of Linq:
Code:
private static IEnumerable<TimeSpan[]> MyGrouping(IEnumerable<TimeSpan> source,
double minutes) {
List<TimeSpan> list = new List<TimeSpan>();
foreach (TimeSpan item in source.OrderBy(x => x)) {
// shall we start a new group?
if (list.Any() && (item - list.Last()).TotalMinutes > minutes) {
// if yes, return the previous
yield return list.ToArray();
list.Clear();
}
list.Add(item);
}
if (list.Any())
yield return list.ToArray();
}
Demo:
var result = MyGrouping(hours, 5)
.Select((list, index) => $"Group {index + 1}: [{string.Join(", ", list)}]");
Console.Write(string.Join(Environment.NewLine, result));
Outcome:
Group 1: [10:35:50, 10:36:48, 10:41:48]
Group 2: [10:47:58, 10:49:14]
Group 3: [11:22:15, 11:24:18, 11:25:25]
You could do this by using Aggregate to determine which "group" each timespan would belong to and then GroupBy that group
TimeSpan[] hours = new[] {
new TimeSpan(10,35,50),
new TimeSpan(10,36,48),
new TimeSpan(10,41,48),
new TimeSpan(10,47,58),
new TimeSpan(10,49,14),
new TimeSpan(11,22,15),
new TimeSpan(11,24,18),
new TimeSpan(11,25,25),
};
var query = hours.Aggregate(new List<(int group, TimeSpan ts)>(), (acc,curr) => {
if(acc.Count == 0)
acc.Add((0,curr));
else
{
var (lastGroup,lastTs) = acc.Last();
if(curr.Subtract(lastTs).TotalMinutes <= 5)
acc.Add((lastGroup,curr));
else
acc.Add((lastGroup+1,curr));
}
return acc;
}).GroupBy(x => x.group, y => y.ts);
foreach(var item in query)
Console.WriteLine(String.Join(", ", item));
Output is
10:35:50, 10:36:48, 10:41:48
10:47:58, 10:49:14
11:22:15, 11:24:18, 11:25:25
Live example: https://dotnetfiddle.net/zXhw0g
Good question here on Aggregate explained to help you understand it.
You can write a method to group the items like so:
public static IEnumerable<List<TimeSpan>> GroupItemsWithin(IEnumerable<TimeSpan> times, TimeSpan maxDelta)
{
var previous = TimeSpan.MinValue;
var spans = new List<TimeSpan>();
foreach (var span in times)
{
if (previous == TimeSpan.MinValue || (span - previous) <= maxDelta)
{
spans.Add(span);
}
else if (spans.Count > 0)
{
yield return spans;
spans = new List<TimeSpan>{ span };
}
previous = span;
}
if (spans.Count > 0)
yield return spans;
}
Then you can use it like this:
public static void Main()
{
TimeSpan[] hours = new[] {
new TimeSpan(10, 35, 50),
new TimeSpan(10, 36, 48),
new TimeSpan(10, 41, 48),
new TimeSpan(10, 47, 58),
new TimeSpan(10, 49, 14),
new TimeSpan(11, 22, 15),
new TimeSpan(11, 24, 18),
new TimeSpan(11, 25, 25),
};
foreach (var group in GroupItemsWithin(hours, TimeSpan.FromMinutes(5)))
{
Console.WriteLine(string.Join(", ", group));
}
}
Output:
10:35:50, 10:36:48, 10:41:48
10:47:58, 10:49:14
11:22:15, 11:24:18, 11:25:25
If you want to use Linq. Then this is the simplest solution Demo
TimeSpan[] hours = new[]
{ new TimeSpan(10, 35, 50),
new TimeSpan(10, 36, 48),
new TimeSpan(10, 41, 48),
new TimeSpan(10, 47, 58),
new TimeSpan(10, 49, 14),
new TimeSpan(11, 22, 15),
new TimeSpan(11, 24, 18),
new TimeSpan(11, 25, 25), };
int groupID = -1;
var result = hours.OrderBy(h => h).Select((item, index) =>
{
if (index == 0 || (!(Math.Abs(hours[index - 1].Subtract(item).TotalMinutes) <= 5)))
++groupID;
return new { group = groupID, item = item };
}).GroupBy(item => item.group);

Linq group by columns in a nested collection

IEnumerable<IEnumerable<object>> rows = new List<List<object>
{
//Columns
new List<object> { 10, 20, 30, 40, 50 },
new List<object> { 10, 20, 30, 40, 50 },
new List<object> { 10, 20, 30, 40, 50 },
new List<object> { 10, 20, 30, 40, 50 }
}
I basically want to group all the 10's, 20's, 30's, 40's, 50's.
I want to do it using Linq and in the best possible way. Please take into consideration that the size of my collection can go upto 20 million or more.
I was able to achieve it by doing the following however it gives me out of memory exception when I do a for each on the group collection.
IEnumerable<IEnumerable<object>> rows = Enumerable.Repeat(new List<object> { 10, 20, 30, 40, 50 }, 20000000);
var group = rows.SelectMany(x => x.Select((InnerValue, Index) => new { Index, InnerValue })).GroupBy(x => x.Index);
EXPECTED RESULT BELOW:
var expectedResult = new List<List<object>
{
new List<object> { 10, 10, 10, 10, 10 },
new List<object> { 20, 20, 20, 20, 20 },
new List<object> { 30, 30, 30, 30, 30 },
new List<object> { 40, 40, 40, 40, 40 }
}
Thanks.
If you want to use linq :), you can do it with the following query:
List<List<object>> expectedResult =
rows.SelectMany(r => r)
.ToLookup(k => k, v => v)
.Select(kv => kv.ToList())
.ToList();

Sort a list of objects into groups in C#

I have a list of 'tickets' and each 'ticket' contains three numbers. I would to sort all the tickets into groups so each group contains tickets which share at least one number in common. How do I sort this data into a final list of grouped tickets?
In short here is the initial list of tickets:
ticketA = { 1, 2, 3 }
ticketB = { 3, 4, 1 }
ticketC = { 5, 6, 7 }
ticketD = { 7, 8, 5 }
ticketE = { 9, 10, 11 }
ticketF = { 11, 1, 9 }
The resulting output would be (broken into seperate lines for ease of reading visually:
GroupedTickets = {
<List>( ticketA, ticketB, ticketF ticketE )
<List>( ticketC, ticketD )
}
Below is the snippet of code I've been using to figure out a solution for...
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace CrowdTool
{
class Ticket
{
public string Name { get; set; }
public List<int> Numbers { get; set; }
}
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
Sort();
}
public void Sort()
{
List<Ticket> allTickets = new List<Ticket>();
Ticket ticketA = new Ticket();
ticketA.Numbers = new List<int> { 1, 2, 3 };
allTickets.Add(ticketA);
Ticket ticketB = new Ticket();
ticketB.Numbers = new List<int> { 3, 4, 1 };
allTickets.Add(ticketB);
Ticket ticketC = new Ticket();
ticketC.Numbers = new List<int> { 5, 6, 7 };
allTickets.Add(ticketC);
Ticket ticketD = new Ticket();
ticketD.Numbers = new List<int> { 7, 8, 5 };
allTickets.Add(ticketD);
Ticket ticketE = new Ticket();
ticketE.Numbers = new List<int> { 9, 10, 11 };
allTickets.Add(ticketE);
Ticket ticketF = new Ticket();
ticketF.Numbers = new List<int> { 11, 1, 9 };
allTickets.Add(ticketF);
// variable to store groups of tickets
List <List<Ticket>> GroupedTickets = new List<List<Ticket>>();
foreach (var ticket in allTickets)
{
Console.WriteLine(ticket);
}
}
}
}
So, I've taken the approach of making all the groups - for all ticket numbers. The final results can then be queried to get you what you want.
I had to change the data into a form that suited the processing. I started with this:
var tickets = new Dictionary<string, int[]>()
{
{ "TicketA", new [] { 1, 2, 3 } },
{ "TicketB", new [] { 3, 4, 1 } },
{ "TicketC", new [] { 5, 6, 7 } },
{ "TicketD", new [] { 7, 8, 5 } },
{ "TicketE", new [] { 9, 10, 11 } },
{ "TicketF", new [] { 11, 1, 9 } },
};
Now I can do this query:
var groupedTickets =
tickets
.SelectMany(t => t.Value, (t, n) => new { t, n })
.ToLookup(x => x.n, x => x.t)
.OrderBy(x => x.Key)
.Select(x => new
{
number = x.Key,
tickets = x.Select(y => new
{
ticket = y.Key,
numbers = y.Value
}).ToList()
})
.ToList();
Now that gave me the results like this:
But that's not terribly easy to see the whole thing, so I reformatted like this:
1: TicketA = {1, 2, 3}, TicketB = {3, 4, 1}, TicketF = {11, 1, 9}
2: TicketA = {1, 2, 3}
3: TicketA = {1, 2, 3}, TicketB = {3, 4, 1}
4: TicketB = {3, 4, 1}
5: TicketC = {5, 6, 7}, TicketD = {7, 8, 5}
6: TicketC = {5, 6, 7}
7: TicketC = {5, 6, 7}, TicketD = {7, 8, 5}
8: TicketD = {7, 8, 5}
9: TicketE = {9, 10, 11}, TicketF = {11, 1, 9}
10: TicketE = {9, 10, 11}
11: TicketE = {9, 10, 11}, TicketF = {11, 1, 9}
You should be able to query against groupedTickets to get precisely what you want.
For example, you could do this:
var output =
groupedTickets
.Where(x => x.tickets.Skip(1).Any())
.Select(x => String.Join(", ", x.tickets.Select(y => y.ticket)))
.OrderBy(x => x)
.Distinct();
Which will give you this output:
TicketA, TicketB
TicketA, TicketB, TicketF
TicketC, TicketD
TicketE, TicketF
And this is quite similar to the output requested, but formatted for display purposes.
Based on the question edit and the comments below here is an updated solution.
var lookup =
tickets
.SelectMany(t => t.Value, (t, n) => new { t, n })
.ToLookup(x => x.n, x => x.t.Value);
var groupedTickets =
tickets
.SelectMany(t => t.Value, (t, n) => new { t, n })
.OrderBy(x => x.n)
.ToLookup(x => x.n, x => x.t)
.SelectMany(
x => x.SelectMany(y => y.Value),
(x, y) => new []
{
Tuple.Create(x.Key, y),
Tuple.Create(y, x.Key)
})
.SelectMany(t => t)
.Where(t => t.Item1 != t.Item2)
.Distinct();
Func<
IEnumerable<Tuple<int, int>>,
IEnumerable<Tuple<int, int>>,
int,
IEnumerable<Tuple<int, int>>> fold = null;
fold = (ts0, ts1, n) =>
n == 0
? ts0
: ts0
.Concat(fold(
ts0.Join(
ts1,
t0 => t0.Item2,
t1 => t1.Item1,
(t0, t1) => Tuple.Create(t0.Item1, t1.Item2)),
ts1,
n - 1))
.Distinct()
.ToArray();
var pairs = tickets.SelectMany(t => t.Value).Distinct().Count();
var final =
fold(groupedTickets, groupedTickets, pairs)
.OrderBy(x => x.Item1)
.ThenBy(x => x.Item2)
.GroupBy(x => x.Item1, x => x.Item2)
.GroupBy(x => String.Join(",", x), x => x.Key)
.Select(x => x.SelectMany(y => lookup[y]).Distinct());
This produces the two distinct sets:
{ { 1, 2, 3 }, { 3, 4, 1 }, { 11, 1, 9 }, { 9, 10, 11 } }
{ { 5, 6, 7 }, { 7, 8, 5 } }
Not very optimized, but this will do the job and you can improve upon it for efficiency (most obviously with the .Clear() and .AddRange()).
var tickets = new Dictionary<string, List<int>>()
{
{ "TicketA", new List<int> { 1, 2, 3 } },
{ "TicketB", new List<int> { 3, 4, 1 } },
{ "TicketC", new List<int> { 5, 6, 7 } },
{ "TicketD", new List<int> { 7, 8, 5 } },
{ "TicketE", new List<int> { 9, 10, 11 } },
{ "TicketF", new List<int> { 11, 1, 9 } },
};
var newDict = new Dictionary<string, List<int>>(tickets);
foreach(var ticket in newDict)
{
bool madeChange = true;
while(madeChange)
{
var groupTickets = newDict.Where(t => t.Key != ticket.Key && t.Value.Intersect(ticket.Value).Any() && t.Value.Except(ticket.Value).Any()).ToList();
madeChange = false;
if (groupTickets.Any())
{
var newSet = groupTickets.SelectMany (t => t.Value).Union(ticket.Value).Distinct().ToList();
ticket.Value.Clear();
ticket.Value.AddRange(newSet);
foreach(var groupTicket in groupTickets)
{
groupTicket.Value.Clear();
groupTicket.Value.AddRange(newSet);
}
madeChange = true;
}
}
}
newDict.GroupBy (t => String.Join(",", t.Value)).Dump();
Essentially, it will look for all tickets with a matching number. It will then insert the numbers into all tickets which match. It repeats this until no new numbers are found.

C# Split time List into time ranges

I am looking to extract ranges from an list of integers using linq:
for example I am looking to split the following list:
List<int> numberList = new List<int>() { 30, 60, 90, 120, 150, 180, 270, 300, 330 };
into a list of integer ranges that will look like:
{ 30, 180 }
{ 270, 330 }
ie: where the next seq is greater than 30
another example :
List<int> numberList = new List<int>() { 30, 60, 120, 150, 270, 300, 330 };
into a list of integer ranges that will look like:
{ 30, 60 }
{ 120, 150 }
{ 270, 330 }
I have tried with for loops to find the best way possible however I don't
know where to start trying to use a linq query to do this.
You could write a method to handle the split:
IEnumerable<IList<int>> SplitValues(IList<int> input, int difference = 30)
{
List<int> results = new List<int>();
int last = input.First();
foreach(var value in input)
{
if (value - last > difference)
{
yield return new[] {results.First(), results.Last()};
results = new List<int>();
}
results.Add(value);
last = value;
}
yield return new[] {results.First(), results.Last()};
}
This matches your specifications as described, returning:
{ 30, 60 }
{ 120, 150 }
{ 270, 330 }
Note that a single value within the collection without a range will be duplicated. For example, { 30, 120, 150 } will return:
{ 30, 30 }
{ 120, 150 }
You can do this in one linq statement:
var numberList = new List<int>() { 30, 60, 120, 150, 270, 300, 330 };
var section = 0;
var result = numberList
.Select( (x, i) => new {value = x, section = (i == 0 ? 0 : ((x - numberList[i - 1]) > 30 ? ++section : section))})
.GroupBy(x => x.section)
.Select(x => x.Select(v => v.value).ToList()).ToList();
Well. There are many ways to do so and all have their pros and cons.
So here's yet another solution, hope it will be helpful to someone.
public static IEnumerable<TSource[]> ToRanges<TSource>(
this IEnumerable<TSource> source, Func<TSource, TSource, TSource, bool> isNear)
{
List<TSource[]> result = source./*OrderBy(value => value).*/Aggregate(
new List<TSource[]> { new[] { source.First(), source.First() } },
(ranges, currentValue) => {
TSource[] currentRange = ranges.Last();
TSource previousValue = currentRange[1];
if (isNear(currentRange[0], previousValue, currentValue))
currentRange[1] = currentValue;
else
ranges.Add(new[] { currentValue, currentValue});
return ranges;
}
);
return result;
}
Example usage:
List<int> numbers = new List<int>() { 30, 60, 90, 120, 150, 180, 270, 300, 330 };
// split by max difference
numberList.ToRanges(
(first, previous, current) => current - previous <= 30).ToArray();
// { 30, 180 }
// { 270, 330 }
// split by max range
numberList.ToRanges(
(first, previous, current) => current - first <= 90).ToArray();
// { 30, 120 }
// { 150, 180 }
// { 270, 330 }
In addition, you can split not only integers but also, for example, words by their first letter. Or DateTime/TimeSpan. Or whatever you want.
You must use LINQ? If not, what about:
List<int> numberList = new List<int>() { 30, 60, 120, 150, 270, 300, 330 };
Dictionary<int, int> result = new Dictionary<int, int>();
int lastStart = numberList.First();
for(int i=1; i < numberList.Count; i++)
{
if(numberList[i] >= lastStart + 30)
{
result.Add(lastStart, numberList[i]);
if (i == numberList.Count - 1) break;
lastStart = numberList[i + 1];
i++;
}
}
foreach (var item in result)
{
Console.WriteLine("{{{0}, {1}}}", item.Key, item.Value);
}
You can use TakeWhile and add the result to another list
void SplitByRange()
{
List<int> numberList = new List<int>() { 30, 60, 120, 150, 270, 300, 330 };
IEnumerable<int> aux = new List<int>();
int n = numberList.First();
int skip = 0;
List<List<int>> output = new List<List<int>>();
while ((aux = numberList.Skip(skip).TakeWhile(o => { bool r = (o - n) <= 30; n = o; return r; })).Count() > 0)
{
output.Add(aux.ToList());
skip += aux.Count();
}
}
At the end numberList will be empty and output will be a list of lists.
output[0] // { 30, 60 }
...
The current code will required at least 1 element on the list, and if you have
{ 30, 100 }
It will return as two lists with 1 element in each
{ 30 }
{ 100 }
Try this:
private static List<int[]> GetGroups(List<int> numberList)
{
List<List<int>> groups = new List<List<int>>();
numberList.Zip(numberList.Skip(1), (a, b) =>
{
if ((b - a) == 30)
{
if (groups.Count == 0)
groups.Add(new List<int>());
groups[groups.Count - 1].Add(a);
}
else if (a == b)
{
groups[groups.Count - 1].Add(a);
}
else
{
groups[groups.Count - 1].Add(a);
groups.Add(new List<int>());
}
return a;
}).ToList();
groups[groups.Count - 1].Add(numberList.Last());
return groups.Select(g => new[] { g.First(), g.Last() }).ToList();
}
Sample usage:
//List<int> numberList = new List<int>() { 30, 60, 90, 120, 150, 180, 270, 300, 330 };
List<int> numberList = new List<int>() { 30, 60, 120, 150, 270, 300, 330 };
var result = GetGroups(numberList);

Categories

Resources