Develop Reference

Increase performance of timeinterval calculation

I have the code fragment below (short version first, compete version after) which loops over lots of records which are in chronological order. The number of records ranges from 100's of thousands to millions. I need to compare the time interval between successive records and determine the difference in minutes to decide on some action and set a value. This is the performance bottleneck of the whole application so I need to do something. The profiler clearly shows that
(DayList[nextIndex].ThisDate - entry.ThisDate).Minutes
is the bottleneck of the bottleneck. When this is solved, the next bottleneck will be the date call in the DayList creation:
List<MonthfileValue> DayList = thisList.Where(x => x.ThisDate.Date == i.Date).ToList();
Those two lines roughly take 60% - 70% of all CPU.
So the question is: how can I increase performance (dramatically) or should I abandon this road completely (because this performance is unacceptable)?
for ( DateTime i=startdate; i<=enddate; i=i.AddDays(1) )
{
int nextIndex = 0;
List<MonthfileValue> DayList = thisList.Where(x => x.ThisDate.Date == i.Date).ToList();
foreach (MonthfileValue entry in DayList)
{
if (++nextIndex < DayList.Count - 1)
{
IntervalInMinutes = (DayList[nextIndex].ThisDate - entry.ThisDate).Minutes;
}
// do some calculations
}
// do some calculations
}
The complete version is below:
for ( DateTime i=startdate; i<=enddate; i=i.AddDays(1) )
{
int nextIndex = 0;
DaySolarValues tmp = new DaySolarValues();
List<MonthfileValue> DayList = thisList.Where(x => x.ThisDate.Date == i.Date).ToList();
foreach (MonthfileValue entry in DayList)
{
if (++nextIndex < DayList.Count - 1)
{
OldIntervalInMinutes = IntervalInMinutes;
IntervalInMinutes = (DayList[nextIndex].ThisDate - entry.ThisDate).Minutes;
if (IntervalInMinutes > 30)
{
IntervalInMinutes = OldIntervalInMinutes; //reset the value and try again
continue; // If more than 30 minutes, then skip this data
}
else if (IntervalInMinutes != OldIntervalInMinutes)
{
// Log some message and continue
}
}
tmp.SolarHours += entry.SolarRad / entry.SolarTheoreticalMax >= SunThreshold ? IntervalInMinutes : 0;
tmp.SolarEnergy += entry.SolarRad * IntervalInMinutes * 60;
tmp.SunUpTimeInMinutes += IntervalInMinutes;
}
tmp.SolarHours /= 60;
tmp.SolarEnergy /= 3600;
tmp.ThisDate = i;
DailySolarValuesList.Add(tmp);
}

I can clearly see that the Where(...) call steals performance.
For me it would be the first step to try this:
var dayLookup = thisList.ToLookup(x => x.ThisDate.Date);
for ( DateTime currentDate =startdate; currentDate <=enddate; currentDate = currentDate.AddDays(1) )
{
int nextIndex = 0;
List<MonthfileValue> DayList = dayLookup[currentDate];
...
}
This way you create a hash lookup before the loop, so getting the DayList will be a less expensive operation

How to implement clock() of c++ in c#?

I want to implement clock method in c# and get difference time of two clock as describe in below code. I use timers ,stopwatch and DateTime in c# but i didn't get correct time as i get in c++.
C++ code is describe below :
clock_t start_time, diff_time;
start_time = clock();
int i = 0;
for(i=0;i<10;i++)
{
.....
}
diff_time = clock();
diff_time -= start_time;
C# code is :
var start_time = Stopwatch.StartNew();
for (int i = 0; i <= 10; i++)
i++;
var diff_time = Stopwatch.StartNew();
var diff_times = (start_time.ElapsedTicks - diff_time.ElapsedTicks);

I think StopWatch should be used to measure code performance and other similar comparisons. If is the case, ok.
Or else, maybe you should consider DateTime.Now
The C function clock() returns a clock_t value that is the number of clock ticks elapsed since the program started. The number of seconds used by the CPU is obtained dividing this result by CLOCKS_PER_SEC.
For DateTime, check:
https://learn.microsoft.com/pt-br/dotnet/api/system.datetime?view=netframework-4.8
-> Some parts below:
The DateTime value type represents dates and times measured in 100-nanosecond units called ticks. Example: in the gregorian calendar a ticks value of 31241376000000000L represents the date Friday, January 01, 0100 12:00:00 midnight.
...
If you are working with a ticks value that you want to convert to some other time interval, such as minutes or seconds, you should use the TimeSpan.TicksPerDay, TimeSpan.TicksPerHour, TimeSpan.TicksPerMinute, TimeSpan.TicksPerSecond, or TimeSpan.TicksPerMillisecond constant to perform the conversion. For example, to add the number of seconds represented by a specified number of ticks to the Second component of a DateTime value, you can use the expression
dateValue.Second + nTicks/Timespan.TicksPerSecond.
...

Here is a simple implementation of the Clock() function in C#
static void Main(string[] args)
{
long CLOCKS_PER_SEC = 10000000;
long lngElapsedTime_t = Clock();
long lngElapsedTime;
int i, j, k, m;
for (i = 0; i < 5000000; i++)
{
j = i / 50;
k = j * 50;
m = i - k;
if (m == 1) { Console.WriteLine(i); }
}
lngElapsedTime = (Clock() - lngElapsedTime_t) / CLOCKS_PER_SEC;
Console.WriteLine("<<{0}>> The time in Ticks", lngElapsedTime_t);
Console.WriteLine("<<{0}>> The elapsed time in seconds", lngElapsedTime);
Console.ReadKey();
}
public static long Clock()
{
long clock_t = (long)DateTime.Now.Ticks;
return clock_t;
}

Storing DateTimes in 30 min intervals to array

My objective is to populate a combo box with time intervals of 30 min for 24 hours. I.E - 12.00am, 12.30am, 1.00am, 1.30am and so on. I need to know how to put these details into array. Thank you

Perhaps:
string[] comboboxDataSource = Enumerable.Range(0, 2 * 24)
.Select(min => DateTime.Today.AddMinutes(30 * min).ToString("h.mmtt", CultureInfo.InvariantCulture))
.ToArray();

One way is to iterate 30 minutes in a day and add this DateTime values with a specific string representation to your list. Like;
List<string> list = new List<string>();
DateTime start = DateTime.Today;
DateTime end = DateTime.Today.AddDays(1);
while (end > start)
{
list.Add(start.ToString("h.mmtt", CultureInfo.InvariantCulture));
start = start.AddMinutes(30);
}
If you wanna get them as an array, just use list.ToArray() to get it. Also time designators are in .NET Framework are mostly (I haven't check all of them) upper case. That means, you will get AM or PM when you use tt specifier, not am or pm. In such a case, you need to replace these values with their lower cases.

Don't know exactly what you mean. I would start with something like this:
private IEnumerable<Timespan> Get30MinuteIntervalls()
{
var currentValue = new Timespan(0);
while (currentValue <= Timespan.FromHours(24)
{
yield return currentValue;
currentValue = currentValue.Add(Timespan.FromMinutes(30));
}
}
var values = Get30MinuteIntervalls().ToArray();

Try:
var d = new DateTime();
d = d.Date.AddHours("0").AddMinutes("0");
for (int i = 0; i < 48; i++)
{
d.AddMinutes(30);
cbo.AddItem(d.TimeOfDay.ToString());
}

Data Aggregations over time, when time is variable

I am in the process of developing an application which calculates the shared acquired in a product over a specified time period (Term).
After the calculations have been performed, it is necessary for me to aggregate the data into groups based on a predefined review period (for example if the time required to gain 100% ownership of the product is 25 years, and the review period value is 5 years, I would have 5 sets of data aggregations for the agreement).
I perform the aggregations as shown by looping through my calculation result set:
if (Year% ReviewPeriod == 0)
{
// Perform Aggregations
}
This works fine in most scenarios.
However I do have a number of scenarios where the product reaches 100% ownership before the end of term.
What I need to be able to do is aggregate the calculations performed based on the ReviewPeriod variable, but if the final number of values in the calculations is not equal to the review period, aggregate the items based on the number of items remaining.
For example, given a 22 year term, data would be aggregated based on the Review Period variable, however if there is a remainder, then the remainder should be aggregated based on the value of the remainder.
Worked Example
Year 0 - 5 = 5 Aggregations
Year 6 - 10 = 5 Aggregations
Year 11 - 15 = 5 Aggregations
Year 16 - 20 = 5 Aggregations
Year 21 - 22 = 2 Aggregations
Could anyone help me with the logic to aggregate the data as I have described.

Probably the simplest way would be something like:
for ( int year = 0; year <= max_year; year++ ) {
if ( year % reviewPeriod == 0 ) {
// start a new aggregation
}
// add year to current aggregation
}
You could keep a list of aggregations and add a new one at the start of each period.
Here is a working example that just groups years in lists:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace Aggregations
{
class Program
{
static void Main(string[] args)
{
int maxYear = 22;
int period = 5;
int year = 1985;
List<List<int>> aggregations = new List<List<int>>();
int i = -1;
for (int y = 0; y <= maxYear; y++)
{
if (y % period == 0)
{
aggregations.Add(new List<int>());
i++;
}
aggregations.ElementAt(i).Add(year);
year++;
}
foreach ( List<int> l in aggregations )
{
foreach (int yy in l)
{
Console.Write(yy + " ");
}
Console.WriteLine();
}
}
}
}

You've not really given enough of your code to go on. Hopefully you should be able to use this however your loop is currently set up. It "leaks" the mod value to the outside of the loop; after the loop is over, you can check the final mod value to see how many aggregations are left.
int modValue = 0;
for //foreach/while/... - your loop here
{
...
modValue = Year % ReviewPeriod;
if (modValue == 0)
{
// Perform Aggregations
}
...
} // end of your loop
if (modValue != 0)
{
// Perform final aggregation. There are modValue items to aggregate.
}

I think my suggestion is not worth 300rep bounty, and either I misunderstood your problem, or you've overshot the bounty..
Do your existing code that calculates the final aggregations works well? If so, then to determine the ranges yo umay just use modulo (%) and simple math:
int minYear = ...the first year // inclusive, i.e. 1970
int maxYear = ...the last year // inclusive, i.e. 2012
int span = maxYear - minYear + 1; // 1970..2012->43, 2001..2006->6
int fullFives = span / 5; // 1970..2012->8, 2001..2006->1
int remainder = span % 5; // 2001..2006->3, 2001..2006->1
for(int i=0; i<fullFives; ++i)
{
int yearFrom = minYear + 5*i
int yearTo = minYear + 5*(i+1) - 1
// 1970..2012 -> 1970-1974, 1975-1979,1980-1984,1985-1989,1990-1994,1995-1999,2000-2004,2005-2009
// 2001..2006 -> 2001-2005
aggregate(yearFrom, yearTo);
}
if(remainder > 0)
{
int yearFrom = minYear + 5*fullFives
int yearTo = minYear + maxYear
// 1970..2012 -> 2010-2012
// 2001..2006 -> 2006-2006
aggregate(yearFrom, yearTo);
}
This is written "out of thin air", I've not checked/compiled it - it is just to sketch the idea.
Note: you've said that everything works but sometimes "a number of scenarios where the product reaches 100% ownership before the end of term." - that would suggest that you rather have an error in the calculations, not in the looping. If the error were in the loop or year boundary detection, then probably almost all would be off. It's hard to say without more of the calculating code is revealed.

The code sample will fire on years 0, 5, 10 etc rather than for every year.
If you just need the number of years to aggregate when that code fires, and the term can be set in advance when a product reaches 100% ownership early, I think this would work:
int term = 22;
int reviewperiod = 5;
for (int year = 0; year < term; year++)
{
if (year % reviewperiod == 0)
{
var endyear = Math.Min(year + reviewperiod, term);
Console.WriteLine("Aggregate years {0} to {1}, {2} Aggregations ", year, endyear, endyear - year);
}
}

Do you think of something like
private int reviewPeriod = 5;
public void Aggregate(int term)
{
Enumerable.Range(0, term)
.ToList()
.Foreach(this.AggregateYear);
}
when this.AggregateYear is defined as follows
public void AggregateYear(int year)
{
var currentRemainder = year % this.reviewPeriod;
var aggregatePeriod = (currentRemainder == 0)
? this.reviewPeriod
: currentRemainder;
this.PerformAggregation(aggregatePeriod);
}
and this.PerformAggregation is defined as follows
private void PerformAggregation(int aggregatePeriod)
{
//...
}

Assuming this data is in memory (since you have not specified otherwise), then you can just use the GroupBy function from Linq:
struct YearValue
{
public int Year, Value;
}
static void Main()
{
// Create some data, hopefully representative of what you are dealing with...
Random r = new Random();
YearValue[] dataValues = new YearValue[22];
for (int i = 0; i < dataValues.Length; i++)
dataValues[i] = new YearValue {Year = i, Value = r.Next(200)};
// Average of values across 'ReviewPeriod' of five:
foreach (var item in dataValues.AsEnumerable().GroupBy(i => i.Year / 5))
{
YearValue[] items = item.ToArray();
Console.WriteLine("Group {0} had {1} item(s) averaging {2}",
item.Key,
items.Length,
items.Average(i => i.Value)
);
}
}
This program then outputs the following text:
Group 0 had 5 item(s) averaging 143.6
Group 1 had 5 item(s) averaging 120.4
Group 2 had 5 item(s) averaging 83
Group 3 had 5 item(s) averaging 145.2
Group 4 had 2 item(s) averaging 98.5

DateTime.AddDays or new DateTime

I'm creating a list of a month's worth of dates. I'm wondering what will be more efficient
List<DateTime> GetDates(DateTime StartDay) {
List<DateTime> dates = new List<DateTime>();
int TotalDays=StartDay.AddMonths(1).AddDays(-1).Day;
for (int i=1; i<TotalDays; i++) {
dates.Add(new DateTime(StartDay.Year, StartDay.Month, i));
}
return dates;
}
or
List<DateTime> GetDates(DateTime StartDay) {
List<DateTime> dates = new List<DateTime>();
DateTime NextMonth = StartDay.AddMonths(1);
for (DateTime curr=StartDay; !curr.Equals(NextMonth); curr=curr.AddDays(1)) {
dates.Add(curr);
}
return dates;
}
basically, is new DateTime() or DateTime.addDays more efficient.
UPDATE:
static void Main(string[] args) {
System.Diagnostics.Stopwatch sw=new System.Diagnostics.Stopwatch();
long t1, t2, total;
List<DateTime> l;
DateTime begin = DateTime.Now;
total = 0L;
for (int i=0; i<10; i++) {
sw.Start();
l = GetDates(begin);
sw.Stop();
sw.Stop();
t1 = sw.ElapsedTicks;
sw.Reset();
sw.Start();
l = GetDates2(begin);
sw.Stop();
t2=sw.ElapsedTicks;
total += t1- t2;
Console.WriteLine("Test {0} : {1} {2} : {3}", i,t1,t2, t1- t2);
}
Console.WriteLine("Total: {0}", total);
Console.WriteLine("\n\nDone");
Console.ReadLine();
}
static List<DateTime> GetDates(DateTime StartDay) {
List<DateTime> dates = new List<DateTime>();
int TotalDays=StartDay.AddMonths(10000).AddDays(-1).Day;
for (int i=1; i<TotalDays; i++) {
dates.Add(new DateTime(StartDay.Year, StartDay.Month, i));
}
return dates;
}
static List<DateTime> GetDates2(DateTime StartDay) {
List<DateTime> dates = new List<DateTime>();
DateTime NextMonth = StartDay.AddMonths(10000);
for (DateTime curr=StartDay; !curr.Equals(NextMonth); curr=curr.AddDays(1)) {
dates.Add(curr);
}
return dates;
}
Test 0 : 2203229 63086205 : -60882976
Test 1 : 63126483 102969090 : -39842607
Test 2 : 102991588 93487982 : 9503606
Test 3 : 93510942 69439034 : 24071908
Test 4 : 69465137 70660555 : -1195418
Test 5 : 70695702 68224849 : 2470853
Test 6 : 68248593 63555492 : 4693101
Test 7 : 63578536 65086357 : -1507821
Test 8 : 65108190 64035573 : 1072617
Test 9 : 64066128 64933449 : -867321
Total: -62484058
Done
results are consistently negative... way negative, so, looks like the constructor and integer test is the more efficient method.

Measure it - write a test program and see which one takes less time.

I believe datetime operations return new datetime structures so you will be creating new instances either way.
http://msdn.microsoft.com/en-us/library/system.datetime.aspx

Unless you are doing some financial processing then I would worry more about readability than performance here. Only start worrying about performance somewhere like here if it's a proven bottleneck.

Since they both do the same thing in the end, there isn't much of a difference.
If you're looking for efficiency, just use ticks. All (that I've seen) calls in DateTime are eventually converted into ticks before any math gets done.

It's really hard to imagine a case in which this would make a significant difference, but Reflector shows that the AddDays technique should be more efficient.
Compare the core logic of AddDays (from Add(Double, Int32))
long num = (long) ((value * scale) + ((value >= 0.0) ? 0.5 : -0.5));
if ((num <= -315537897600000L) || (num >= 0x11efae44cb400L)) {
// Throw omitted
}
return this.AddTicks(num * 0x2710L);
To the core logic of the DateTime(int, int, int) constructor (from DateToTicks):
if (((year >= 1) && (year <= 0x270f)) && ((month >= 1) && (month <= 12)))
{
int[] numArray = IsLeapYear(year) ? DaysToMonth366 : DaysToMonth365;
if ((day >= 1) && (day <= (numArray[month] - numArray[month - 1])))
{
int num = year - 1;
int num2 = ((((((num * 0x16d) + (num / 4)) - (num / 100)) + (num / 400)) + numArray[month - 1]) + day) - 1;
return (num2 * 0xc92a69c000L);
}
}
// Throw omitted
AddDays just converts the specified number of days to the equivalent number of ticks (a long) and adds it to the existing ticks.
Creating a new DateTime using the year/month/day constructor requires many more calculations. That constructor has to check whether the specified year is a leap year, allocate an array of days in each month, perform a bunch of extra operations, just to finally get the number of ticks those three numbers represent.
Edit: DateTime.AddDays(int) is faster than new DateTime(int, int, int), but your first algorithm is faster than the second algorithm. This is probably because the iteration costs are much higher in the second algorithm. As you observed in your edit, this might well be because DateTime.Equals is more expensive than comparing integers.

Here is a working test program, with the algorithms implemented so that they can actually be compared (they still need work, though):
class Program
{
static void Main(string[] args)
{
IList<DateTime> l1, l2;
DateTime begin = new DateTime(2000, 1, 1);
Stopwatch timer1 = Stopwatch.StartNew();
for (int i = 0; i < 10000; i++)
l1 = GetDates(begin);
timer1.Stop();
Stopwatch timer2 = Stopwatch.StartNew();
for (int i = 0; i < 10000; i++)
l2 = GetDates2(begin);
timer2.Stop();
Console.WriteLine("new DateTime: {0}\n.AddDays: {1}",
timer1.ElapsedTicks, timer2.ElapsedTicks);
Console.ReadLine();
}
static IList<DateTime> GetDates(DateTime StartDay)
{
IList<DateTime> dates = new List<DateTime>();
int TotalDays = DateTime.DaysInMonth(StartDay.Year, StartDay.Month);
for (int i = 0; i < TotalDays; i++)
dates.Add(new DateTime(StartDay.Year, StartDay.Month, i + 1));
return dates;
}
static IList<DateTime> GetDates2(DateTime StartDay)
{
IList<DateTime> dates = new List<DateTime>();
DateTime NextMonth = StartDay.AddMonths(1);
for (DateTime curr = StartDay; !curr.Equals(NextMonth); curr = curr.AddDays(1))
dates.Add(curr);
return dates;
}
} // class
Output (I added the commas):
new DateTime: 545,307,375
.AddDays: 180,071,512
These results seem pretty clear to me, though honestly I thought they'd be a lot closer.

I agree with Mark. Test both methods yourself and see which one is faster. Use the Stopwatch class to get accurate timings of how long each method takes to run. My first guess is that since both end up creating new structures anyway, that any speed difference will be negligible. Also, with only generating a month's worth of dates (31 days maximum), I don't think either method will be that much slower than the other. Perhaps is you you were generating thousands or millions of dates, it would make a difference, but for 31 dates, it's probably premature optimization.