Develop Reference

or-tools - Compute the stdev from a SumArray()

I need to generate plannings for employees using Google's Optimization Tools.
One of the constraints would be that every employee has approximately the same amount of working hours.
Thus, I want to aggregate in a list how many hours each employee is working, and then minimize the standard deviation of this list.
var workingTimes = new List<SumArray>();
foreach (var employee in employees) {
// Gather the duration of each task the employee is
// assigned to in a list
// o.IsAssign is an IntVar and task.Duration is an int
var allDurations = shifts.Where(o => o.Employee == employee.Name)
.Select(o => o.IsAssigned * task[o.Task].Duration);
// Total time the employee is working
var workTime = new SumArray(allDurations);
workingTimes.Add(workTime);
}
Now I want to minimize the stdev of workingTimes. I tried the following:
IntegerExpression workingTimesMean = new SumArray(workingTimes) * (1/workingTimes.Count);
var gaps = workingTimes.Select(o => (o - workingTimesMean)*(o - workingTimesMean));
var stdev = new SumArray(gaps) * (1/gaps.Count());
model.Minimize(stdev);
But the LINQ query at the 2nd line of the last code snippet is throwing me an error:
Can't apply operator * to IntegerExpression and IntegerExpression
How can I compute the standard deviation of a Google.OrTools.Sat.SumArray?

The 'natural' API only supports linear expressions.
You need to use the AddProductEquality() API.
Please note that 1 / Gaps.Count() will always return 0 (we are in integer arithmetic).
So you need to scale everything up.
Personally, I would just minimize the unscaled sum of abs(val - average). No need to divide by the number of elements.
Just check that the computation of the average has the right precision (once again, we are in integer arithmetic).
You could also consider just minimize the max(abs(val - average)). This is simpler and may be good enough.

Calculating time with hours and minutes only

I am attempting to create a timesheet calculator which takes calculates the time an employee works and I am close, with one problem.
As I perform the calculation, I only want hours and minutes to display. I am able to get that done, but that causes an issue. If the employee punches out before a full minute is elapsed, that minute is not included in the calculation.
For example, if an emp punches in at 12:00:30 and punches out at 5:00:29, that last minute is not counted in the calculation, so the time shows as 4:59 instead of 5:00.
How do I get the calculation to be based on the hours and minutes and exclude seconds completely?
This is the code I have:
private void btnPunchOut_Click(object sender, EventArgs e)
{
DateTime stopTime = DateTime.Now;
lblPunchOutTime.Text = stopTime.ToShortTimeString();
TimeSpan timeWorked = new TimeSpan();
timeWorked = stopTime - startTime;
lblTimeWorked.Text = timeWorked.ToString(#"hh\:mm");
}

Use TimeSpan.TotalSeconds perhaps...And then add 30 seconds or more, before you convert it to hours by dividing by 3600.
As in
lblTimeWorked.Text = ((timeWorked.TotalSeconds+30)/3600).ToString("0.00") + " hours";
Use Timespan.TotalHours if you want the hours.
But if you want to be accurate, you should create a separate class dedicated to calculating the hours worked by a staff member. Then you can encapsulate lots of business rules in the dedicated class. Staff have entitlements and overtime, expenses or penalty rates - so this can get complex if done properly.

If you want a calculation that really ignores the seconds, the clearest way to accomplish that is to get rid of the seconds on both the start time and the end time. It might not seem accurate because it allows a difference of one second to become a difference of one minute. But that could still be a valid business rule, that you want to subtract according the the minutes that appeared on the clock rather than the actual elapsed seconds.
In other words,
1:00:01 is adjusted to 1:00:00.
1:00:59 is adjusted to 1:00:00.
1:01:00 is "adjusted" to 1:01:00.
1:01:01 is adjusted to 1:01:00.
You can accomplish that with an extension like this:
public static class TimespanExtensions
{
public static TimeSpan TrimToMinutes(this TimeSpan input)
{
return TimeSpan.FromMinutes(Math.Truncate(input.TotalMinutes));
}
}
(I'm sure there's a more efficient way of truncating the seconds, but at least this is clear.)
Now instead of having to figure out how to calculate the difference while rounding seconds or adding seconds, you just trim the seconds before calculating the difference. Here's a unit test:
[TestMethod]
public void NumberOfMinutesIgnoresSeconds()
{
var startTime = TimeSpan.FromSeconds(59).TrimToMinutes();
var endTime = TimeSpan.FromSeconds(60).TrimToMinutes();
Assert.AreEqual(1, (endTime - startTime).TotalMinutes);
}
One Timespan represents 59 seconds, and the next one is 60, or the first second of the next minute. But if you trim the seconds and then calculate the difference you get exactly one minute.
In the context of your code,
DateTime stopTime = DateTime.Now;
lblPunchOutTime.Text = stopTime.ToShortTimeString();
var timeWorked = stopTime.TrimToMinutes() - startTime.TrimToMinutes();
lblTimeWorked.Text = timeWorked.ToString(#"hh\:mm");

How to divide time in equal slots in C#

I want to divide time in equal intervals in C#. Like from 3:00pm to 6:00pm create time intervals with a gap of 45 minutes (e.g, 3:00pm, 3:45pm, 4:30pm .... 6:00pm.
How can I acheive this in C# ?

You can use the DateTime.Ticks property to define your intervals, and then create a series of DateTime objects based on your defined interval. The example below can be run in LINQpad. Per the documentation, there are 10000000 ticks in one second. With that in mind:
var startTS = Convert.ToDateTime("6/17/2018 15:00:00");
var endTS = Convert.ToDateTime("6/17/2018 18:00:00");
long ticksPerSecond = 10000000;
long ticksPerMinute = ticksPerSecond * 60;
long ticksPer45Min = ticksPerMinute * 45;
long startTSInTicks = startTS.Ticks;
long endTsInTicks = endTS.Ticks;
for(long i = startTSInTicks; i <= endTsInTicks; i+=ticksPer45Min)
{
new DateTime(i).Dump();
}
In LINQpad, the output looks like this:
6/17/2018 15:00:00
6/17/2018 15:45:00
6/17/2018 16:30:00
6/17/2018 17:15:00

Try this
DateTime StartTime = DateTime.Parse("3:0:0");//If pm it should be 15
DateTime EndTime = DateTime.Parse("6:0:0");//If pm it should be 18
while (StartTime!=EndTime)
{
double minuts = +45;
StartTime = StartTime.AddMinutes(minuts);
}
Hope this helps

Datetime.AddMinutes(double value) should do what you are looking for. Just keep on adding until the result of the addition goes over the maximum date/time you have.
NOTE: This assumes you know your interval. If, on the other hand, you require to split a time span in a equal n parts you would require a different approach, as shown here.

Resample, aggregate, and interpolate of TimeSeries trend data

In analysis of energy demand and consumption data, I'm having issue re-sampling and interpolating time series trended data.
Data set example:
timestamp value kWh
------------------ ---------
12/19/2011 5:43:21 PM 79178
12/19/2011 5:58:21 PM 79179.88
12/19/2011 6:13:21 PM 79182.13
12/19/2011 6:28:21 PM 79183.88
12/19/2011 6:43:21 PM 79185.63
Based upon these observations, I'd like some aggregation to roll-up values based upon a period of time, with that frequency set to a unit of time.
As in, intervals on the hour filling any gaps of missing data
timestamp value (approx)
------------------ ---------
12/19/2011 5:00:00 PM 79173
12/19/2011 6:00:00 PM 79179
12/19/2011 7:00:00 PM 79186
For a linear algorithm, it seems I would take the difference in time and multiply the value against that factor.
TimeSpan ts = current - previous;
Double factor = ts.TotalMinutes / period;
Value and timestamp could be calculated based upon the factor.
With such quantity of available information, I'm unsure why it's difficult to find the most elegant approach to this.
Perhaps first, are there open source analysis libraries that could be recommended?
Any recommendations for a programmatic approach? Ideally C#, or possibly with SQL?
Or, any similar questions (with answers) I could be pointed to?

By using the time-ticks that are used internally to represent DateTimes, you get the most accurate values that are possible. Since these time ticks do not restart at zero at midnight, you will not have problems at day boundaries.
// Sample times and full hour
DateTime lastSampleTimeBeforeFullHour = new DateTime(2011, 12, 19, 17, 58, 21);
DateTime firstSampleTimeAfterFullHour = new DateTime(2011, 12, 19, 18, 13, 21);
DateTime fullHour = new DateTime(2011, 12, 19, 18, 00, 00);
// Times as ticks (most accurate time unit)
long t0 = lastSampleTimeBeforeFullHour.Ticks;
long t1 = firstSampleTimeAfterFullHour.Ticks;
long tf = fullHour.Ticks;
// Energy samples
double e0 = 79179.88; // kWh before full hour
double e1 = 79182.13; // kWh after full hour
double ef; // interpolated energy at full hour
ef = e0 + (tf - t0) * (e1 - e0) / (t1 - t0); // ==> 79180.1275 kWh
Explanation of the formula
In geometry, similar triangles are triangles that have the same shape but different sizes. The formula above is based on the fact that the ratios of any two sides in one triangle are the same for the corresponding sides of a similar triangle.
If you have a triangle A B C and a similar triangle a b c, then A : B = a : b. The equality of two ratios is called a proportion.
We can apply this proportionality rule to our problem:
(e1 – e0) / (t1 – t0) = (ef – e0) / (tf – t0)
--- large triangle -- --- small triangle --

I have written a LINQ function to interpolate and normalize time series data so that it can be aggregated/merged.
The Resample function is as follows. I have written a short article about this technique at the Code Project.
// The function is an extension method, so it must be defined in a static class.
public static class ResampleExt
{
// Resample an input time series and create a new time series between two
// particular dates sampled at a specified time interval.
public static IEnumerable<OutputDataT> Resample<InputValueT, OutputDataT>(
// Input time series to be resampled.
this IEnumerable<InputValueT> source,
// Start date of the new time series.
DateTime startDate,
// Date at which the new time series will have ended.
DateTime endDate,
// The time interval between samples.
TimeSpan resampleInterval,
// Function that selects a date/time value from an input data point.
Func<InputValueT, DateTime> dateSelector,
// Interpolation function that produces a new interpolated data point
// at a particular time between two input data points.
Func<DateTime, InputValueT, InputValueT, double, OutputDataT> interpolator
)
{
// ... argument checking omitted ...
//
// Manually enumerate the input time series...
// This is manual because the first data point must be treated specially.
//
var e = source.GetEnumerator();
if (e.MoveNext())
{
// Initialize working date to the start date, this variable will be used to
// walk forward in time towards the end date.
var workingDate = startDate;
// Extract the first data point from the input time series.
var firstDataPoint = e.Current;
// Extract the first data point's date using the date selector.
var firstDate = dateSelector(firstDataPoint);
// Loop forward in time until we reach either the date of the first
// data point or the end date, which ever comes first.
while (workingDate < endDate && workingDate <= firstDate)
{
// Until we reach the date of the first data point,
// use the interpolation function to generate an output
// data point from the first data point.
yield return interpolator(workingDate, firstDataPoint, firstDataPoint, 0);
// Walk forward in time by the specified time period.
workingDate += resampleInterval;
}
//
// Setup current data point... we will now loop over input data points and
// interpolate between the current and next data points.
//
var curDataPoint = firstDataPoint;
var curDate = firstDate;
//
// After we have reached the first data point, loop over remaining input data points until
// either the input data points have been exhausted or we have reached the end date.
//
while (workingDate < endDate && e.MoveNext())
{
// Extract the next data point from the input time series.
var nextDataPoint = e.Current;
// Extract the next data point's date using the data selector.
var nextDate = dateSelector(nextDataPoint);
// Calculate the time span between the dates of the current and next data points.
var timeSpan = nextDate - firstDate;
// Loop forward in time until wwe have moved beyond the date of the next data point.
while (workingDate <= endDate && workingDate < nextDate)
{
// The time span from the current date to the working date.
var curTimeSpan = workingDate - curDate;
// The time between the dates as a percentage (a 0-1 value).
var timePct = curTimeSpan.TotalSeconds / timeSpan.TotalSeconds;
// Interpolate an output data point at the particular time between
// the current and next data points.
yield return interpolator(workingDate, curDataPoint, nextDataPoint, timePct);
// Walk forward in time by the specified time period.
workingDate += resampleInterval;
}
// Swap the next data point into the current data point so we can move on and continue
// the interpolation with each subsqeuent data point assuming the role of
// 'next data point' in the next iteration of this loop.
curDataPoint = nextDataPoint;
curDate = nextDate;
}
// Finally loop forward in time until we reach the end date.
while (workingDate < endDate)
{
// Interpolate an output data point generated from the last data point.
yield return interpolator(workingDate, curDataPoint, curDataPoint, 1);
// Walk forward in time by the specified time period.
workingDate += resampleInterval;
}
}
}
}

Maby something like this:
SELECT DATE_FORMAT('%Y-%m-%d %H', timestamp) as day_hour, AVG(value) as aprox FROM table GROUP BY day_hour
What database engine you use?

for what you are doing it appears that you are declaring the TimeSpan incorrectly for starters ts = (TimeSpan)(current- previous); also make sure that current and previous are of DateTime type.
if you want to look at calculating or rolling up I would look at TotalHours() here is an example that you can look at for an idea if you like
here is check if a LastWrite / Modified time is within a 24 hour period
if (((TimeSpan)(DateTime.Now - fiUpdateFileFile.LastWriteTime)).TotalHours < 24){}
I know that this is different that your case but you get the drift on how to use TotalHours

please help optimize this loop

i need to calculate the number of workdays between two dates. a workday is any day between Monday through Friday except for holidays. the code below does this, but it uses a loop. does anyone see a way to get rid of the loop or at least optimize it?
thanks
konstantin
using System;
using System.Linq;
namespace consapp
{
static class Program
{
static void Main(string[] args)
{
var holidays = new DateTime[] { new DateTime(2010, 11, 23), new DateTime(2010, 11, 30) };
var date_start = new DateTime(2010, 12, 3);
var date_end = date_start.AddDays(-9.9);
var duration = (date_end - date_start).Duration();
for (var d = date_end; d < date_start; d = d.Date.AddDays(1))
{
if (holidays.Contains(d.Date) || d.DayOfWeek == DayOfWeek.Saturday || d.DayOfWeek == DayOfWeek.Sunday)
{
duration -= TimeSpan.FromDays(1) - d.TimeOfDay;
}
}
Console.WriteLine(duration);
}
}
}

I would investigate an algorithm like the following. (Sorry no code provided.)
Count the number of full 7-day weeks between your start and end date. You should be able to do this with .NET DateTime and TimeSpan objects.
For each full 7-day week, add 5 days to your result.
Figure out the partial weeks including your start and end dates.
Loop through your holidays and reduce your result by 1 for each holiday between your start and end date. Looping is better here because there are likely far fewer holidays to loop over than days between your start and end date.
Have fun!
EDIT: For source code, check out this answer: Calculate the number of business days between two dates?

Is performance really problematic here? Unless profiling suggests otherwise I'd guess this code doesn't really slow down your application. It's performance should be fine unless you calculate the workdays for thousands of long intervals per second.
If your holiday list is much larger than just two dates then convert it into a HashSet<T> which has O(1) lookup time.
And of course you can turn around the code. So you don't loop over the days in the interval, but over the holidays. Then you just calculate the number of week-days in the interval(should be simple math) and subtract the number of holidays that fall on a week-day.
If it's really necessary you can pre-calculate the workdays since some fixed date, and then subtract the lookup result from the beginning of the period from the lookup result from the end of the period.

if you want faster code, don't loop over each day in the range:
remove from your list of holidays all holidays that fall on sunday or saturday, then use the timespan methods to give you the number of days between the two dates. With a little math (think about integer division by 7) you can get the number of mon-thursday days in that range, subtract the number of holidays that don't fall on the weekend from that number and you are done.

Just roll with it as is. This is not going to waste much time since the bounds are small. When you have some working code, move on. No need to mercilessly optimise code for no reason.

just because I started this as a fun puzzle, here's the code:
[Test]
public void TestDateTime() {
var start = DateTime.Now.Date;
var end = DateTime.Now.Date.AddDays(35);
var workdays = (end - start).Days - ((end - start).Days/7)*2
- (((end - start).Days%7==0)?0:(((int)start.DayOfWeek==0)?1:Math.Max(Math.Min((int)start.DayOfWeek + (end - start).Days%7 - 6, 2), 0)));
new []{DateTime.Now.AddDays(19), DateTime.Now.AddDays(20)}.ToList().ForEach(
x => { if (x.DayOfWeek != DayOfWeek.Saturday && x.DayOfWeek != DayOfWeek.Sunday) workdays--; });
Console.Out.WriteLine("workdays = {0}", workdays);
}
Christmas day and Boxing day are included as holidays.