I have a list of dates organized like this:
(From, To)
(From, To)
...
(From, To)
I am trying to find how to consolidate ranges in an efficient way (it has to be quite fast because it is to consolidate financial data streams in realtime).
Dates do NOT overlap.
what I was thinking about is:
Sort everything by From time
and then iterate through pairs to see if Pair1.To == Pair2.From to merge them, but this means several iterations.
Is there a better way to do this, like in a single pass
Here are some examples
(2019-1-10, 2019-1-12)
(2019-3-10, 2019-3-14)
(2019-1-12, 2019-1-13)
expected output:
(2019-1-10, 2019-1-12) + (2019-1-12, 2019-1-13) -> (2019-1-10, 2019-1-13)
(2019-3-10, 2019-3-14) -> (2019-3-10, 2019-3-14)
In practice, it's really about seconds and not dates, but the idea is the same.
You mention that dates never overlap but I think it is slightly simpler to write code that just merges overlapping dates. First step is to define the date range type:
class Interval
{
public DateTime From { get; set; }
public DateTime To { get; set; }
}
You can then define an extension method that checks if two intervals overlap:
static class IntervalExtensions
{
public static bool Overlaps(this Interval interval1, Interval interval2)
=> interval1.From <= interval2.From
? interval1.To >= interval2.From : interval2.To >= interval1.From;
}
Notice that this code assumes that From <= To so you might want to change Interval into an immutable type and verify this in the constructor.
You also need a way to merge two intervals:
public static Interval MergeWith(this Interval interval1, Interval interval2)
=> new Interval
{
From = new DateTime(Math.Min(interval1.From.Ticks, interval2.From.Ticks)),
To = new DateTime(Math.Max(interval1.To.Ticks, interval2.To.Ticks))
};
Next step is define another extension method that iterates a sequence of intervals and tries to merge consecutive overlapping intervals. This is best done using an iterator block:
public static IEnumerable<Interval> MergeOverlapping(this IEnumerable<Interval> source)
{
using (var enumerator = source.GetEnumerator())
{
if (!enumerator.MoveNext())
yield break;
var previousInterval = enumerator.Current;
while (enumerator.MoveNext())
{
var nextInterval = enumerator.Current;
if (!previousInterval.Overlaps(nextInterval))
{
yield return previousInterval;
previousInterval = nextInterval;
}
else
{
previousInterval = previousInterval.MergeWith(nextInterval);
}
}
yield return previousInterval;
}
}
If two consecutive intervals don't overlap it yields the previous interval. However, if they overlap it instead updates the previous interval by merging the two intervals and keep the merged interval as the previous interval for the next iteration.
Your sample data is not sorted so before merging the intervals you have to sort them:
var mergedIntervals = intervals.OrderBy(interval => interval.From).MergeOverlapping();
However, if the real data is sorted which you have indicated in a comment you can skip the sorting. The algorithm will do a single pass over the data and thus is O(n).
Give this a go:
var source = new[]
{
new { from = new DateTime(2019, 1, 10), to = new DateTime(2019, 1, 12) },
new { from = new DateTime(2019, 3, 10), to = new DateTime(2019, 3, 14) },
new { from = new DateTime(2019, 1, 12), to = new DateTime(2019, 1, 13) },
};
var data =
source
.OrderBy(x => x.from)
.ThenBy(x => x.to)
.ToArray();
var results =
data
.Skip(1)
.Aggregate(
data.Take(1).ToList(),
(a, x) =>
{
if (a.Last().to >= x.from)
{
a[a.Count - 1] = new { from = a.Last().from, to = x.to };
}
else
{
a.Add(x);
}
return a;
});
It's a nice query and it gives the output that you want.
Create two Dictionaries (i.e. hash maps), one using the To date as the key and the From-To date as the value, the other with the From date as the key.
Iterate over your date ranges and for each range check if the From date exists as a key in the To-date-keyed Dictionary, and vice versa.
If not a match in either then add the range to both the Dictionaries.
If there is a match in one but not the other then remove the matching range from both Dictionaries (using the appropriate key), merge the new range with the existing range and add the result to both.
If there is a match in both Dictionaries (the range being added fills a hole) then remove both matches from both Dictionaries, merge the three ranges (two existing and one new) and add the result to both Dictionaries.
At the end your Dictionaries contain an unsorted set of all date ranges, which you can extract by iterating over the keys of one of the Dictionaries.
Here is a 'two-dictionaries' implementation, that consolidates the ranges without sorting them first. The assumptions are that there is no overlapping, and no duplicate properties. A duplicate property will cause an exception to be thrown.
public static IEnumerable<TSource> Consolidate<TSource, TProperty>(
this IEnumerable<TSource> source,
Func<TSource, TProperty> property1Selector,
Func<TSource, TProperty> property2Selector,
Func<TSource, TSource, TSource> combine)
{
var dict1 = source.ToDictionary(property1Selector);
var dict2 = source.ToDictionary(property2Selector);
if (dict1.Keys.Count == 0) yield break;
var first = dict2.Values.First(); // Start with a random element
var last = first;
var current = first;
while (true) // Searching backward
{
dict1.Remove(property1Selector(first));
dict2.Remove(property2Selector(first));
if (dict2.TryGetValue(property1Selector(first), out current))
{
first = current; // Continue searching backward
}
else
{
while (true) // Searching forward
{
if (dict1.TryGetValue(property2Selector(last), out current))
{
last = current; // Continue searching forward
dict1.Remove(property1Selector(last));
dict2.Remove(property2Selector(last));
}
else
{
yield return combine(first, last);
break;
}
}
if (dict1.Keys.Count == 0) break;
first = dict1.Values.First(); // Continue with a random element
last = first;
}
}
}
Usage example:
var source = new List<(DateTime From, DateTime To)>()
{
(new DateTime(2019, 1, 10), new DateTime(2019, 1, 12)),
(new DateTime(2019, 3, 10), new DateTime(2019, 3, 14)),
(new DateTime(2019, 1, 12), new DateTime(2019, 1, 13)),
(new DateTime(2019, 3, 5), new DateTime(2019, 3, 10)),
};
var consolidated = source
.Consolidate(r => r.From, r => r.To, (r1, r2) => (r1.From, r2.To))
.OrderBy(r => r.From)
.ToList();
foreach (var range in consolidated)
{
Console.WriteLine($"{range.From:yyyy-MM-dd} => {range.To:yyyy-MM-dd}");
}
Output:
2019-01-10 => 2019-01-13
2019-03-05 => 2019-03-14
My take using MoreLinq and functional style. IMO, easy to understand. Most lines here are sample data, logic is only few lines (GetAsDays method and all.Segment call)
How it is done: we transform date ranges into collection of days, union these collections and split them into separate ranges (where more then 1 day is between end and start of the next).
void Main()
{
var baseD = new DateTime(01, 01, 01);
var from = DateTime.Today.Dump("from");
var to = from.AddDays(20).Dump("to");
var range1 = GetAsDays(from, to);
var from2 = DateTime.Today.AddDays(10).Dump("from2");
var to2 = from2.AddDays(20).Dump("to2");
var from3 = DateTime.Today.AddDays(50).Dump("from2");
var to3 = from3.AddDays(10).Dump("to2");
var range2 = GetAsDays(from2, to2);
var range3 = GetAsDays(from3, to3);
var all = range3
.Union(range1)
.Union(range2)
.OrderBy(e=>e);
var split=all.Segment((iPlus1, i, a) => (iPlus1 - i) > 1);
split.Select(s=>(baseD.AddDays(s.First()),baseD.AddDays(s.Last()))).Dump();
}
public IList<int> GetAsDays(DateTime from, DateTime to)
{
var baseD = new DateTime(01, 01, 01);
var fromSpan = from - baseD;
var toSpan = to - baseD;
var set1 = Enumerable.Range((int)fromSpan.TotalDays, (int)(toSpan - fromSpan).TotalDays);
return new List<int>(set1);
}
Related
So I have a list of prices from a database. I would like to sort it so that the first entry in a list is the entry with the lowest number. And then all other entry are order by input date.
How can this be done?
This is my code, which is a mess, sorry I'm trying stuff :)
var itemPriceDate = itemPrice.OrderBy(d => d.Invoice.DateInvoice).ToList();
var itemPriceDateLow= itemPriceDate.OrderBy(c => c.qtPrice).ThenBy(d => d.Invoice.DateInvoice);
ViewBag.ItemPrice = itemPriceDateLow; ```
First find out the lowest value from the List(itemPrice).
double lowest_price = itemPrice.Min(c => c.qtPrice);
Next, remove the lowest element from the list.
var itemToRemove = itemPrice.Single(c => c.qtPrice == lowest_price);
itemPrice.Remove(itemToRemove);
Next, sort the remaining list based on input Date.
var newList = itemPrice.OrderByDescending(d => d.Invoice.DateInvoice).ToList();
Finally, add lowest element at first index
newList.Insert(0, lowest_price);
LINQ is great when it works, but it sometimes does unexpected things. Depending on how large your dataset is, you may be better off doing it as a stored procedure that returns the data already ordered.
If the dataset is small or you're cornered into using C# to do it there is the option of using a custom sort function. Without knowing the exact structure of your data, this is more intended as a blanket example that will need tweaking accordingly.
Let's say your list is stored in the itemPrice variable, if you do something along the lines of:
itemPrice.Sort((a, b) => {
int retVal = a.qtPrice < b.qtPrice;
return ret != 0 ? ret : a.Invoice.DateInvoice < b.Invoice.DateInvoice;
});
Will sort by qtPrice and then fall back to the DateInvoice field; you may need to swap the less than to a greater than to get your desired order.
One sort is enough. What I think it should be is:
var itemPriceDateLow= itemPriceDate.OrderBy(c => c.qtPrice).ThenBy(d => d.Invoice.DateInvoice);
This will obviously give you whole collection. You might want to use .First() if you want to get top most element.
One thing to remember - ThenBy, OrderBy are ascending by default.
Take a look at this example:
class Program
{
static void Main(string[] args)
{
List<ItemPrice> items = new List<ItemPrice>();
items.Add(new ItemPrice() { Date = DateTime.Now, QtyPrice = 1});
items.Add(new ItemPrice() { Date = DateTime.Now.AddDays(-1), QtyPrice = 1});
items.Add(new ItemPrice() { Date = DateTime.Now, QtyPrice = 2});
var sortedItem = items.OrderBy(p => p.QtyPrice).ThenBy(p => p.Date).First();
Console.WriteLine($"Default Ascending sort {sortedItem.Date}, {sortedItem.QtyPrice}");
var sortedItemWithReverseDate = items.OrderBy(p => p.QtyPrice).ThenByDescending(p => p.Date).First();
Console.WriteLine($"Descending sort on date {sortedItemWithReverseDate.Date}, {sortedItemWithReverseDate.QtyPrice}");
}
}
class ItemPrice {
public DateTime Date { get; set; }
public decimal QtyPrice { get; set; }
}
It will give you:
Default Ascending sort 16/08/2021 12:47:34, 1
Descending sort on date 17/08/2021 12:47:34, 1
You would need to iterate the collection twice in this case, since you would first need to know the Aggregate Value (Min). Then, you could use a Custom Comparer as the following.
public class CustomComparer : IComparer<Item>
{
private int _minValue;
public CustomComparer(int minValue)
{
_minValue= minValue;
}
public int Compare(Item instanceA, Item instanceB)
{
if(instanceA.Price == _minValue) return -1;
if(instanceB.Price == _minValue) return 1;
return instanceA.InputDate.CompareTo(instanceB.InputDate);
}
}
Now you can fetch the result as
var min = list.Min(x=>x.Price);
var result = list.OrderBy(x=>x,new CustomComparer(min));
Example,
public class Item
{
public int Price{get;set;}
public DateTime InputDate{get;set;}
}
var list = new List<Item>
{
new Item{Price = 2, InputDate=new DateTime(2021,3,1)},
new Item{Price = 12, InputDate=new DateTime(2021,7,1)},
new Item{Price = 12, InputDate=new DateTime(2021,9,1)},
new Item{Price = 42, InputDate=new DateTime(2021,1,1)},
new Item{Price = 32, InputDate=new DateTime(2021,6,1)},
new Item{Price = 22, InputDate=new DateTime(2021,4,1)},
new Item{Price = 2, InputDate=new DateTime(2021,3,2)},
new Item{Price = 12, InputDate=new DateTime(2021,2,1)}
};
var min = list.Min(x=>x.Price);
var result = list.OrderBy(x=>x,new CustomComparer(min));
Output
Thx for all your inputs.
For me the right way to go was.
Order my "itemPrice" list by "OrderByDescending(by date)"
Then find out the lowest value from the List(itemPrice).
double lowest_price = itemPrice.Min(c => c.qtPrice);
Then declare a new List
List<qtInvoice> newItemPrice = new List<qtInvoice>();
First loop that adds all the "lowest_price" to "newItemPrice" list
foreach (var item in itemPriceDate)
{
if (item.qtPrice == lowest_price)
{
newItemPrice.Add(item);
}
}
Then in second loop you add all the rest of the prices to "newItemPrice" list
foreach (var item in itemPriceDate)
{
if (item.qtPrice != lowest_price)
{
newItemPrice.Add(item);
}
}
I have a list of records containing Id, DateFrom, DateTo. For the sake of this question we can use this one:
List<(int, DateTime, DateTime)> data = new List<(int, DateTime, DateTime)>
{
(1, new DateTime(2012, 5, 16), new DateTime(2018, 1, 25)),
(2, new DateTime(2009, 1, 1), new DateTime(2011, 4, 27)),
(3, new DateTime(2014, 1, 1), new DateTime(2016, 4, 27)),
(4, new DateTime(2015, 1, 1), new DateTime(2015, 1, 3)),
(2, new DateTime(2013, 5, 10), new DateTime(2017, 4, 27)),
(5, new DateTime(2013, 5, 16), new DateTime(2018, 1, 24)),
(2, new DateTime(2017, 4, 28), new DateTime(2018, 1, 24)),
};
In my real case the List could be a lot bigger. Initially I was working with the assumption that there can be only one record for a certain Id and I was able to come up with a pretty good solution but now, as you can see, the assumption is that you can have several periods for an Id and all periods should be taken into consideration when comparing the whole time.
The task is to find the two records that has the longest time overlap and to return the ids and the number of days overlapped.
Which in this sample case means that these should be records 1 and 2.
My implementation of this is the following:
public (int, int, int) GetLongestElapsedPeriodWithDuplications(List<(int, DateTime, DateTime)> periods)
{
Dictionary<int, List<(DateTime, DateTime)>> periodsByPeriodId = new Dictionary<int, List<(DateTime, DateTime)>>();
foreach (var period in periods)
{
if (periodsByPeriodId.ContainsKey(period.Item1))
{
periodsByPeriodId[period.Item1].Add((period.Item2, period.Item3));
}
else
{
periodsByPeriodId[period.Item1] = new List<(DateTime, DateTime)>();
periodsByPeriodId[period.Item1].Add((period.Item2, period.Item3));
}
}
int firstId = -1;
int secondId = -1;
int periodInDays = 0;
foreach (var period in periodsByPeriodId)
{
var Id = period.Key;
foreach (var currPeriod in periodsByPeriodId)
{
int currentPeriodInDays = 0;
if (Id != currPeriod.Key)
{
for (var i = 0; i < period.Value.Count; i++)
{
for (var j = 0; j < currPeriod.Value.Count; j++)
{
var firstPeriodDateFrom = period.Value[i].Item1;
var firstPeriodDateTo = period.Value[i].Item2;
var secondPeriodDateFrom = currPeriod.Value[j].Item1;
var secondPeriodDateTo = currPeriod.Value[j].Item2;
if (secondPeriodDateFrom < firstPeriodDateTo && secondPeriodDateTo > firstPeriodDateFrom)
{
DateTime commonStartingDate = secondPeriodDateFrom > firstPeriodDateFrom ? secondPeriodDateFrom : firstPeriodDateFrom;
DateTime commonEndDate = secondPeriodDateTo > firstPeriodDateTo ? firstPeriodDateTo : secondPeriodDateTo;
currentPeriodInDays += (int)(commonEndDate - commonStartingDate).TotalDays;
}
}
}
if (currentPeriodInDays > periodInDays)
{
periodInDays = currentPeriodInDays;
firstId = Id;
secondId = currPeriod.Key;
}
}
}
}
return (firstId, secondId, periodInDays);
}
As you can see the method is pretty big and in my opinion far from optimized in terms of execution speed. I know that those nested loops rise the complexity a lot, but this additional requirement to deal with more than one period for an Id really left me without ideas. How can I optimize this logic so in case of bigger input it would execute faster than now?
As in your original solution - you need to compare each interval with any other, except intervals with the same id, so I'd code this like this:
Supporting classes, just to simplify actual algorithm:
class Period {
public DateTime Start { get; }
public DateTime End { get; }
public Period(DateTime start, DateTime end) {
this.Start = start;
this.End = end;
}
public int Overlap(Period other) {
DateTime a = this.Start > other.Start ? this.Start : other.Start;
DateTime b = this.End < other.End ? this.End : other.End;
return (a < b) ? b.Subtract(a).Days : 0;
}
}
class IdData {
public IdData() {
this.Periods = new List<Period>();
this.Overlaps = new Dictionary<int, int>();
}
public List<Period> Periods { get; }
public Dictionary<int, int> Overlaps { get; }
}
Method to find max overlap:
static int GetLongestElapsedPeriod(List<(int, DateTime, DateTime)> periods) {
int maxOverlap = 0;
Dictionary<int, IdData> ids = new Dictionary<int, IdData>();
foreach (var period in periods) {
int id = period.Item1;
Period idPeriod = new Period(period.Item2, period.Item3);
// preserve interval for ID
var idData = ids.GetValueOrDefault(id, new IdData());
idData.Periods.Add(idPeriod);
ids[id] = idData;
foreach (var idObj in ids) {
if (idObj.Key != id) {
// here we calculate of new interval with all previously met
int o = idObj.Value.Overlaps.GetValueOrDefault(id, 0);
foreach (var otherPeriods in idObj.Value.Periods)
o += idPeriod.Overlap(otherPeriods);
idObj.Value.Overlaps[id] = o;
// check whether newly calculate overlapping is the maximal one, preserve Ids if needed too
if (o > maxOverlap)
maxOverlap = o;
}
}
}
return maxOverlap;
}
You can use TimePeriodLibrary.NET:
PM> Install-Package TimePeriodLibrary.NET
TimePeriodCollection timePeriods = new TimePeriodCollection(
data.Select(q => new TimeRange(q.Item2, q.Item3)));
var longestOverlap = timePeriods
.OverlapPeriods(new TimeRange(timePeriods.Start, timePeriods.End))
.OrderByDescending(q => q.Duration)
.FirstOrDefault();
With an extension method:
public static T MaxBy<T, TKey>(this IEnumerable<T> src, Func<T, TKey> key, Comparer<TKey> keyComparer = null) {
keyComparer = keyComparer ?? Comparer<TKey>.Default;
return src.Aggregate((a, b) => keyComparer.Compare(key(a), key(b)) > 0 ? a : b);
}
And some helper functions
DateTime Max(DateTime a, DateTime b) => (a > b) ? a : b;
DateTime Min(DateTime a, DateTime b) => (a < b) ? a : b;
int OverlappingDays((DateTime DateFrom, DateTime DateTo) span1, (DateTime DateFrom, DateTime DateTo) span2) {
var maxFrom = Max(span1.DateFrom, span2.DateFrom);
var minTo = Min(span1.DateTo, span2.DateTo);
return Math.Max((minTo - maxFrom).Days, 0);
}
You can group together the spans with matching Ids
var dg = data.GroupBy(d => d.Id);
Generate all pairs of Ids
var pdgs = from d1 in dg
from d2 in dg.Where(d => d.Key > d1.Key)
select new[] { d1, d2 };
Then compute the overlap in days between each pair of Ids and find the maximum:
var MaxOverlappingPair = pdgs.Select(pdg => new {
Id1 = pdg[0].Key,
Id2 = pdg[1].Key,
OverlapInDays = pdg[0].SelectMany(d1 => pdg[1].Select(d2 => OverlappingDays((d1.DateFrom, d1.DateTo), (d2.DateFrom, d2.DateTo)))).Sum()
}).MaxBy(TwoOverlap => TwoOverlap.OverlapInDays);
Since efficiency is mentioned, I should say that implementing some of these operations directly instead of using LINQ is more efficient, but you are using Tuples and in-memory structures so I don't think it will make much difference.
I ran some performance tests using a list of 24000 spans with 1249 unique IDs. The LINQ code took about 16 seconds. By inlining some of the LINQ and replacing anonymous objects with tuples, it came down to about 3.1 seconds. By adding a shortcut skipping any IDs whose cumulative days were shorter than the current max overlapping days and a few more optimizations, I got it down to less than 1 second.
var baseDate = new DateTime(1970, 1, 1);
int OverlappingDays(int DaysFrom1, int DaysTo1, int DaysFrom2, int DaysTo2) {
var maxFrom = DaysFrom1 > DaysFrom2 ? DaysFrom1 : DaysFrom2;
var minTo = DaysTo1 < DaysTo2 ? DaysTo1 : DaysTo2;
return (minTo > maxFrom) ? minTo - maxFrom : 0;
}
var dgs = data.Select(d => {
var DaysFrom = (d.DateFrom - baseDate).Days;
var DaysTo = (d.DateTo - baseDate).Days;
return (d.Id, DaysFrom, DaysTo, Dist: DaysTo - DaysFrom);
})
.GroupBy(d => d.Id)
.Select(dg => (Id: dg.Key, Group: dg, Dist: dg.Sum(d => d.Dist)))
.ToList();
var MaxOverlappingPair = (Id1: 0, Id2: 0, OverlapInDays: 0);
for (int j1 = 0; j1 < dgs.Count; ++j1) {
var dg1 = dgs[j1];
if (dg1.Dist > MaxOverlappingPair.OverlapInDays)
for (int j2 = j1 + 1; j2 < dgs.Count; ++j2) {
var dg2 = dgs[j2];
if (dg2.Dist > MaxOverlappingPair.OverlapInDays) {
var testOverlapInDays = 0;
foreach (var d1 in dg1.Group)
foreach (var d2 in dg2.Group)
testOverlapInDays += OverlappingDays(d1.DaysFrom, d1.DaysTo, d2.DaysFrom, d2.DaysTo);
if (testOverlapInDays > MaxOverlappingPair.OverlapInDays)
MaxOverlappingPair = (dg1.Id, dg2.Id, testOverlapInDays);
}
}
}
Optimizations applied:
Convert each spans DateTimes to # of days from an arbitrary baseDate to optimize overlapping days calculation by doing date conversion once.
Compute the total days for each span and skip any span pairs that can't exceed the current overlap
Replace SelectMany/Select with nested foreach to compute overlapping days.
Use ValueTuples instead of anonymous objects which are (slightly) faster for this problem.
Replace pair generation LINQ with nested for loops generating each possible pair directly
Pass individual from/to parameters instead of objects to OverlappingDays function
Note: I tried a smarter overlapping days calculation but when the number of spans per ID is small, the overhead took longer than just doing the calculation directly.
There are already few solutions
but
if you want to improve the efficiency then you don't have to compare every objects/value with everyother value or object. You can use Interval Search Tree for this problem and it can be solved in RlogN where R are number of intersections between intervals.
I recommend you to watch this video of Robert Sedgwick and also that book is online available.
Your basic problem here is how to identify a unique set of time periods. Give each one its own unique ID yourself.
When you write your final answer, include the additional details in the output so the user can understand which (original) IDs and original time periods resulted in the final answer.
Remember - the problem is still the same as in the original post (https://codereview.stackexchange.com/questions/186014/finding-the-longest-overlapping-period/186031?noredirect=1#comment354707_186031) and you still have the same information to work with. Don't get too hung up on the "ID"s as provided in the original list - you are still iterating through a list of time periods.
I have a simple table that keeps track of the entry date. I would like to select records that are X minutes apart.
IMAGE_LOCATION IMAGE DATE
============== =============
2227.jpg 08/03/2014 22:27:47
2228.jpg 08/03/2014 22:28:48
2229.jpg 08/03/2014 22:59:49
2230.jpg 08/03/2014 23:12:50
2231.jpg 08/03/2014 23:29:49
From the sample above i would like the query to return items that are at least X minutes apart, lets say 30 min. so from the list above 2227.jpg, 2229.jpg and 2231.jpg would be returned only.
This is what i have so far that just returns the latest images, however i need the latest ones but separated by at least 30 minutes between records.
using (var db = new GibFrontierEntities())
{
var result = (from u in db.CCTV_IMAGES.OrderByDescending(u => u.ImageDate)
select u).Take(rows);
return result.ToList();
}
This is a quick attempt to achieve exactly what you asked for, a LINQ solution (tested and working in .NET 4):
var list = db.CCTV_IMAGES.OrderByDescending(u => u.ImageDate);
return list.Where((d, i) =>
{
//Look ahead to compare against the next if it exists.
if (list.ElementAtOrDefault(i + 1) != null)
{
return d.ImageDate.Subtract(list.ElementAtOrDefault(i + 1).ImageDate).TotalMinutes > 30;
}
//Look behind to compare against the previous if this is the last item in the list.
if (list.ElementAtOrDefault(i - 1) != null)
{
return list.ElementAtOrDefault(i - 1).ImageDate.Subtract(d.ImageDate).TotalMinutes > 30;
}
return false;
}).ToList();
Per comments and a clearer definition of the requirement:
Because you stated in the comments below that you will have 1 item a minute and you previously stated that you need them separated by at least 30 minutes, would you consider simplifying the logic to grab every 30th item from the list?
return list.Where((d, i) => i % 30 == 0);
You can use SelectMany to achieve what you want:
using (var db = new GibFrontierEntities())
{
var images = db.CCTV_IMAGES;
var result = images
.SelectMany(i => images,
(first, second) => new { First = first, Second = second })
.Where(i => i.First != i.Second)
.Where(i => Math.Abs(
EntityFunctions
.DiffMinutes(i.First.ImageDate, i.Second.ImageDate)) >= 30)
.Select(i => i.First)
.Distinct()
.OrderByDescending(i => i.ImageDate)
.Take(rows)
.ToList();
return result;
}
As mentioned already, this could be easily achievable through iteration. However, it you really have to deal with LINQ expression, here's a quick and dirty sample that will return dates that are 30 minutes apart:
List<DateTime> dateLlist = new List<DateTime>();
dateLlist.Add(new DateTime(2014, 1, 1, 1, 0, 0, 0));
dateLlist.Add(new DateTime(2014, 1, 1, 1, 10, 0, 0));
dateLlist.Add(new DateTime(2014, 1, 1, 1, 45, 0, 0));
DateTime previousTime = new DateTime();
bool shouldAdd = false;
List<DateTime> newList = dateLlist.Where(x =>
{
shouldAdd = (previousTime == DateTime.MinValue || previousTime.AddMinutes(30) < x);
previousTime = x;
return shouldAdd;
}).ToList();
I need a list with some objects for calculation.
my current code looks like this
private class HelperClass
{
public DateTime TheDate {get;set;}
public TimeSpan TheDuration {get;set;}
public bool Enabled {get;set;}
}
private TimeSpan TheMethod()
{
// create entries for every date
var items = new List<HelperClass>();
foreach(DateTime d in GetAllDatesOrdered())
{
items.Add(new HelperClass { TheDate = d, Enabled = GetEnabled(d), });
}
// calculate the duration for every entry
for (int i = 0; i < items.Count; i++)
{
var item = items[i];
if (i == items.Count -1) // the last one
item.TheDuration = DateTime.Now - item.TheDate;
else
item.TheDuration = items[i+1].TheDate - item.TheDate;
}
// calculate the total duration and return the result
var result = TimeSpan.Zero;
foreach(var item in items.Where(x => x.Enabled))
result = result.Add(item.TheDuration);
return result;
}
Now I find it a bit ugly just to introduce a type for my calculation (HelperClass).
My first approach was to use Tuple<DateTime, TimeSpan, bool> like I usually do this but since I need to modify the TimeSpan after creating the instance I can't use Tuple since Tuple.ItemX is readonly.
I thought about an anonymous type, but I can't figure out how to init my List
var item1 = new { TheDate = DateTime.Now,
TheDuration = TimeSpan.Zero, Enabled = true };
var items = new List<?>(); // How to declare this ???
items.Add(item1);
Using a projection looks like the way forward to me - but you can compute the durations as you go, by "zipping" your collection with itself, offset by one. You can then do the whole method in one query:
// Materialize the result to avoid computing possibly different sequences
var allDatesAndNow = GetDatesOrdered().Concat(new[] { DateTime.Now })
.ToList();
return allDatesNow.Zip(allDatesNow.Skip(1),
(x, y) => new { Enabled = GetEnabled(x),
Duration = y - x })
.Where(x => x.Enabled)
.Aggregate(TimeSpan.Zero, (t, pair) => t + pair.Duration);
The Zip call pairs up each date with its subsequent one, converting each pair of values into a duration and an enabled flag. The Where call filters out disabled pairs. The Aggregate call sums the durations from the resulting pairs.
You could do it with LINQ like:
var itemsWithoutDuration = GetAllDatesOrdered()
.Select(d => new { TheDate = d, Enabled = GetEnabled(d) })
.ToList();
var items = itemsWithoutDuration
.Select((it, k) => new { TheDate = it.d, Enabled = it.Enabled,
TheDuration = (k == (itemsWithoutDuration.Count - 1) ? DateTime.Now : itemsWithoutDuration[k+1].TheDate) - it.TheDate })
.ToList();
But by that point the Tuple is both more readable and more concise!
I want to display a customer's accounting history in a DataGridView and I want to have a column that displays the running total for their balance. The old way I did this was by getting the data, looping through the data, and adding rows to the DataGridView one-by-one and calculating the running total at that time. Lame. I would much rather use LINQ to SQL, or LINQ if not possible with LINQ to SQL, to figure out the running totals so I can just set DataGridView.DataSource to my data.
This is a super-simplified example of what I'm shooting for. Say I have the following class.
class Item
{
public DateTime Date { get; set; }
public decimal Amount { get; set; }
public decimal RunningTotal { get; set; }
}
I would like a L2S, or LINQ, statement that could generate results that look like this:
Date Amount RunningTotal
12-01-2009 5 5
12-02-2009 -5 0
12-02-2009 10 10
12-03-2009 5 15
12-04-2009 -15 0
Notice that there can be multiple items with the same date (12-02-2009). The results should be sorted by date before the running totals are calculated. I'm guessing this means I'll need two statements, one to get the data and sort it and a second to perform the running total calculation.
I was hoping Aggregate would do the trick, but it doesn't work like I was hoping. Or maybe I just couldn't figure it out.
This question seemed to be going after the same thing I wanted, but I don't see how the accepted/only answer solves my problem.
Any ideas on how to pull this off?
Edit
Combing the answers from Alex and DOK, this is what I ended up with:
decimal runningTotal = 0;
var results = FetchDataFromDatabase()
.OrderBy(item => item.Date)
.Select(item => new Item
{
Amount = item.Amount,
Date = item.Date,
RunningTotal = runningTotal += item.Amount
});
Using closures and anonymous method:
List<Item> myList = FetchDataFromDatabase();
decimal currentTotal = 0;
var query = myList
.OrderBy(i => i.Date)
.Select(i =>
{
currentTotal += i.Amount;
return new {
Date = i.Date,
Amount = i.Amount,
RunningTotal = currentTotal
};
}
);
foreach (var item in query)
{
//do with item
}
How about this: (credit goes to this source)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ConsoleApplication1
{
class Program
{
delegate string CreateGroupingDelegate(int i);
static void Main(string[] args)
{
List<int> list = new List<int>() { 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 69, 2007};
int running_total = 0;
var result_set =
from x in list
select new
{
num = x,
running_total = (running_total = running_total + x)
};
foreach (var v in result_set)
{
Console.WriteLine( "list element: {0}, total so far: {1}",
v.num,
v.running_total);
}
Console.ReadLine();
}
}
}
In case this hasn't been answered yet, I have a solution that I have been using in my projects. This is pretty similar to an Oracle partitioned group. The key is to have the where clause in the running total match the orig list, then group it by date.
var itemList = GetItemsFromDBYadaYadaYada();
var withRuningTotals = from i in itemList
select new {i.Date, i.Amount,
RunningTotal = itemList.Where( x=> x.Date == i.Date).
GroupBy(x=> x.Date).
Select(DateGroup=> DateGroup.Sum(x=> x.Amount)).Single()};
Aggregate can be used to obtain a running total as well:
var src = new [] { 1, 4, 3, 2 };
var running = src.Aggregate(new List<int>(), (a, i) => {
a.Add(a.Count == 0 ? i : a.Last() + i);
return a;
});
Most of the other answers to this, which properly set the running totals within the objects, rely on a side-effect variable, which is not in the spirit of functional coding and the likes of .Aggregate(). This solution eliminates the side-effect variable.
(NB - This solution will run on the client as with other answers, and so may not be optimal for what you require.)
var results = FetchDataFromDatabase()
.OrderBy(item => item.Date)
.Aggregate(new List<Item>(), (list, i) =>
{
var item = new Item
{
Amount = i.Amount,
Date = i.Date,
RunningTotal = i.Amount + (list.LastOrDefault()?.RunningTotal ?? 0)
};
return list.Append(item).ToList();
// Or, possibly more efficient:
// list.Add(item);
// return list;
});
using System;
using System.Linq;
using System.Collections.Generic;
public class Program
{
public static void Main()
{
var list = new List<int>{1, 5, 4, 6, 8, 11, 3, 12};
int running_total = 0;
list.ForEach(x=> Console.WriteLine(running_total = x+running_total));
}
}