Develop Reference

Adding an ETA to an embedded loop sequence

UPDATE 1:
Some of the solutions offered below seem good. However, I only know the amount of times a loop will iterate after it's parent loop's iterations have been determined. So I can't count all the iterations beforehand.
ORIGINAL QUESTION:
I have embedded loops in a program similar to this:
Prog1:
using System;
using System.Threading;
namespace time_remaining_loop_strip
{
class Program
{
static void Main(string[] args)
{
var random = new Random();
Console.Clear();
// Simulate initiation delay
Console.WriteLine("initiate");
Thread.Sleep(random.Next(100, 1000));
int intCount = random.Next(1, 10);
for (int loop1 = 0; loop1 <= intCount; loop1++)
{
// Simulate loop1 delay
Console.WriteLine("\tloop1");
Thread.Sleep(random.Next(100, 1000));
for (int loop2 = 0; loop2 <= random.Next(1, 10); loop2++)
{
// Simulate loop2 delay
Console.WriteLine("\t\tloop2");
Thread.Sleep(random.Next(100, 1000));
for (int loop3 = 0; loop3 <= random.Next(1, 10); loop3++)
{
// Simulate loop3 delay
Console.WriteLine("\t\t\tloop3");
Thread.Sleep(random.Next(100, 1000));
for (int loop4 = 0; loop4 <= random.Next(1, 10); loop4++)
{
// Simulate loop4 delay
Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(random.Next(100, 1000));
}
}
}
}
}
}
}
I am trying to display Processing Time Remaining (ETA), so I can see a rough estimate of the amount of time remaining before the loop sequence above completes
I now have another bit of code which does display an ETA which works fine when the loop is very simplistic:
Prog2:
using System;
using System.Threading;
namespace time_remaining
{
class Program
{
public static TimeSpan ComputeRemaining((int count, DateTime time) start, (int count, DateTime time) current, int end) =>
current.count - start.count == 0
? TimeSpan.MaxValue
: TimeSpan.FromSeconds((end - current.count) * current.time.Subtract(start.time).TotalSeconds / (current.count - start.count));
static void Main(string[] args)
{
Console.Clear();
var random = new Random();
int Count = random.Next(10, 60);
DateTime startTime = DateTime.Now;
for (int i = 0; i <= Count; i++)
{
Thread.Sleep(random.Next(100, 2000));
TimeSpan timeRemaining = ComputeRemaining((0, startTime), (i, DateTime.Now), Count);
Console.SetCursorPosition(0,0);
Console.Write("ETA: ");
Console.Write(String.Format("{0} Days, {1} Hours, {2} Minutes, {3} Seconds", timeRemaining.Days.ToString().PadLeft(3,'0'), timeRemaining.Hours.ToString().PadLeft(2,'0'), timeRemaining.Minutes.ToString().PadLeft(2,'0'), timeRemaining.Seconds.ToString().PadLeft(2,'0')));
}
}
}
}
When I try to combine the ETA aspect of Prog1 into Prog2, it does not seem to work well:
Prog3 = Prog1+Prog2:
using System;
using System.Threading;
namespace time_remaining_loop_strip
{
class Program
{
public static TimeSpan ComputeRemaining((int count, DateTime time) start, (int count, DateTime time) current, int end) =>
current.count - start.count == 0
? TimeSpan.MaxValue
: TimeSpan.FromSeconds((end - current.count) * current.time.Subtract(start.time).TotalSeconds / (current.count - start.count));
static void Main(string[] args)
{
DateTime startTime = DateTime.Now;
var random = new Random();
Console.Clear();
// Simulate initiation delay
//Console.WriteLine("initiate");
Thread.Sleep(random.Next(100, 1000));
int intCount = random.Next(1, 10);
for (int loop1 = 0; loop1 <= intCount; loop1++)
{
// Simulate loop1 delay
//Console.WriteLine("\tloop1");
Thread.Sleep(random.Next(100, 1000));
for (int loop2 = 0; loop2 <= random.Next(1, 10); loop2++)
{
// Simulate loop2 delay
//Console.WriteLine("\t\tloop2");
Thread.Sleep(random.Next(100, 1000));
for (int loop3 = 0; loop3 <= random.Next(1, 10); loop3++)
{
// Simulate loop3 delay
//Console.WriteLine("\t\t\tloop3");
Thread.Sleep(random.Next(100, 1000));
for (int loop4 = 0; loop4 <= random.Next(1, 10); loop4++)
{
// Simulate loop4 delay
//Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(random.Next(100, 1000));
}
}
}
TimeSpan timeRemaining = ComputeRemaining((0, startTime), (loop1, DateTime.Now), intCount);
Console.SetCursorPosition(0,0);
Console.Write("ETA: ");
Console.Write(String.Format("{0} Days, {1} Hours, {2} Minutes, {3} Seconds", timeRemaining.Days.ToString().PadLeft(3,'0'), timeRemaining.Hours.ToString().PadLeft(2,'0'), timeRemaining.Minutes.ToString().PadLeft(2,'0'), timeRemaining.Seconds.ToString().PadLeft(2,'0')));
}
}
}
}
This doesn't seem to work very well at all. It does display an ETA, but it has long delays before it shows anything, because of the way the loop is structured.
How can I update this so the ETA code displays an ETA more accurately and at more predictive intervals, as in every second?

With the concession that you've just built a simple model of what's actually happening (you have some series of nested process of variable delay and count, which cannot even be determined until runtime), what you're asking for currently is a random number predictor.
There will be between n & m cycles (1 & 10 in your example) of between o & p duration (100 & 1000ms), but still random.. well, random quantized into bands. The way you've written it, this is dice-roll random (a dice has no memory), although in practice it seems more likely the duration of one cycle must somewhat imply the duration of the next, (which is how you've written ComputeRemaining) and within one band the count of one loop must help with the count of the next.
So despite its apparent simplicity Prog2 covers our examples.. given a known loop count, where each cycle takes a random duration (which is in fact pick(n,m)^3*pick(o,p).. but this is still just a random number) - predict the end. For reporting purposes, you'll want to refactor this to consider the inner loops as well too, but it's effectively the same process. (^3 is a simplification, it's actually a series of independent picks multiplied)
Ok, so we don't need to talk about time/delays (I mean.. you clearly want that, but it's just some number that represents the future - a TimeSpan is an long count of x ticks since some time... a finish time is just Now + x*tick). So we can simplify this into a long predictor.
Setup
interface IAvger
{
public double Avg { get; }
}
interface IAdder
{
void Add(long value);
}
class Mean
{
private int _count = 0;
public double Total { get; private set; } = 0;
public double? Avg => _count == 0 ? null : (double?)(Total / _count);
public void Add(double val)
{
Total += val;
_count++;
}
}
You can ignore the interfaces (I used them when switching out potential solutions). Class Mean should be familiar... it calculates the mean average of a number of values, scaling/adapting as more values are found.
/// <summary>
/// Equivalent to your ComputeRemaining
/// </summary>
class RunningAvg : IAvger, IAdder
{
private Mean _mean = new Mean();
private readonly double _guess;
public RunningAvg(double guess)
{
_guess = guess;
}
public double Avg => _mean.Avg ?? _guess;
public void Add(long value) => _mean.Add(value);
}
Here's an equivalent to your ComputeRemaining. The value of guess helps an early prediction when nothing else is known (vs the equivalent of TimeSpan.Max)
/// <summary>
/// Drop the lowest and highest value
/// - Fairly typical in stats, however note this is only one biggest/smallest
/// (will work best when the standard devation is low, and outliers
/// are rare)
/// </summary>
class IgnoreExtremes : IAvger, IAdder
{
private long? _worst;
private long? _best;
private Mean _mean = new Mean();
private readonly int _wt;
private readonly double _guess;
public IgnoreExtremes(double guess, int weight = 4)
{
_wt = weight;
_guess = guess;
}
public long Best => _best ?? (long)Math.Round(_guess);
public long Worst => _worst ?? (long)Math.Round(_guess);
public double Avg
{
get
{
var avg = _mean.Avg;
if (!avg.HasValue) return _guess;
return (Best + _wt * avg.Value + Worst) / (2 + _wt);
}
}
public void Add(long value)
{
if (!_best.HasValue)
{
_best = value;
}
else if (value < _best)
{
_mean.Add(_best.Value);
_best = value;
}
else if (!_worst.HasValue)
{
_worst = value;
}
else if (value > _worst)
{
_mean.Add(_worst.Value);
_worst = value;
}
else
{
_mean.Add(value);
}
}
}
Finally some stats! IgnoreExtremes suppresses the highest and lowest (single) value. In scientific sampling it's fairly typical to ignore these, however with a real random distribution of numbers (eg dice roll, or random.Next) only one extreme will be discarded. This should predict better numbers than RunningAvg. Note this is a form of weighted average, you can tune it (slightly) by supplying a weight value at construction (wt=4 is fairly common), or tying _wt to _mean.count (some code changes required)
class IgnoreStdDevOutlier : IAvger, IAdder
{
private const int AT_LEAST = 5;
private Mean _mean = new Mean();
private readonly List<long> _vals = new List<long>();
private readonly double _guess;
//private long _tot;
private readonly double _outlierStdDevMulti;
public IgnoreStdDevOutlier(double guess, double outlierStdDevMulti = 2)
{
_guess = guess;
_outlierStdDevMulti = outlierStdDevMulti;
}
private double StdDev()
{
var avg = Avg;
double tot = 0;
foreach (var item in _vals)
tot += (item - avg) * (item - avg);
return Math.Sqrt(tot / (_vals.Count - 1));
}
public void Add(long value)
{
_vals.Add(value);
if (_vals.Count > AT_LEAST)
{
var avg = Avg;
var sd = StdDev();
var min = avg - _outlierStdDevMulti * sd;
var max = avg + _outlierStdDevMulti * sd;
//Ignore outliers
if (value < min || value > max) return;
}
_mean.Add(value);
}
public double Avg => _mean.Avg ?? 0;
}
Another statistics approach is to ignore values more than n*StandardDeviation from average, where n is often 2 or 3 (you'll find conflicting opinions). All values seen are part of the standard deviation, but only those that aren't outliers are considered part of the average. This ends up acting like a suppression factor, preventing the estimate swinging too much.
Alright, so to run a test we need some sort of measuring class:
class Performance
{
private readonly List<long> _set = new List<long>();
private long _actual;
public void Add(long item) => _set.Add(item);
public void SetFinal(long final) => _actual = final;
public void Report()
{
foreach (var item in _set)
{
Console.WriteLine("{0} {1} = {2}", item, _actual, (item / (double)_actual - 1) * 100);
}
}
}
A real guess can't know the final (_actual) value, but this class allows us to see how the guess is doing so far.
Finally the program class:
class Program
{
const int MIN_MSEC = 100;
const int MAX_MSEC = 1000;
const int MIN_LOOP = 10;
const int MAX_LOOP = 50;
static void C(Random random)
{
int nm = random.Next(MAX_LOOP, MAX_LOOP);
var guess = (double)((MAX_LOOP + MIN_LOOP) / 2 * (MAX_MSEC + MIN_MSEC) / 2);
var predict = new RunningAvg(guess);
//var predict = new IgnoreExtremes(guess);
//var predict = new IgnoreStdDevOutlier(guess,3);
var per = new Performance();
long tot = 0;
for (int i = 0; i <= nm; i++)
{
var op = random.Next(MIN_MSEC, MAX_MSEC);
predict.Add(op);
per.Add((long)Math.Round(predict.Avg * nm));
tot += op;
}
per.SetFinal(tot);
per.Report();
}
static void Main(string[] args)
{
var random = new Random();
C(random);
}
}
You can ignore the work is done in a method called C (just a side effect of the code - A was your Prog1, while B was Prog2). Within C try changing which of RunningAvg, IgnoreExtremes or IgnoreStdDevOutlier is uncommented. Because, again, what's written is dice-roll random, you cannot take a single run as a good benchmark. The next stage is to wrap this in repeat-runs and take the average of the standard deviation of the predictions (or perhaps only the late predictions - a user probably doesn't mind if an early estimate is far off, as long as by the end it isn't jumping around) - but I ran out of time. I find IgnoreStdDevOutlier, on average, converges on the right answer fairly swiftly being off by 0-1% by the end. IgnoreExtremes suffers from only ignoring one extreme (in each direction), so it's sort of a light version of IgnoreStdDevOutlier. If your data isn't random, and there are only occasionally extreme cases - it'll do just fine. RunningAvg doesn't actually perform terribly some of the time, other times it's off by double digit percentages all the way through. Random numbers, if only they were easy to predict.
Note on usage
Timespan.Ticks is a long. All of this is written to predict a long, which can be considered the difference between then and now. To directly switch use new Timespan(long ticks) to build the spans, and DateTime.Now.Subtract(startTime).Ticks to get a long from a span. Obviously all these classes could be re-written for TimeSpan instead of long.. unfortunately there isn't an easy generic where constraint that includes both long and TimeSpan

You can know how many times(t) loop 4 (L4) will execute multiplying L1tL2tL3t*L4t = total. Now at the very beginning you declare
dt = DateTime.Now;
count = 0L;
Now inside L4 you increase count and calculates how much time is elapsed
et= DateTime.Now - dt;
count++;
And there using rule of three you calculate how many seconds are left to reach the total iterations.
remainingSeconds = (total*et.TotalSeconds/count) - et.TotalSeconds;
Now the ETA is
DateTime.Now.AddSeconds(remainingSeconds);

I used below logic to solve this.
Before first foreach each loop I calculated the all different loops count.
After that I created 4 different variables to hold sleep time for each loop which is settings inside them.
Now before first for loop I calculated the total time in milliseconds that will be taken be all loops using below logic:
var totalTime = (firstLoopCount * firstLoopSleepTime) +
(firstLoopCount * secondLoopCount * secondLoopSleepTime) +
(firstLoopCount * secondLoopCount * thirdLoopCount * thirdLoopSleepTime) +
(firstLoopCount * secondLoopCount * thirdLoopCount * fourthLoopCount * fourthLoopSleepTime);
Note: I added 1 in function call (GetTotoalTimeinMilliSecond) where loops count is passing because loop is starting from 0 and ending with different loops count including itself.
Now before loop start, printed total time taken
Inside each loop, after thread sleep, subtract thread sleep time from total time and print that. And set new calculated time as total time.
Repeat this inside every loop.
Below is the code:
class Program
{
static void Main(string[] args)
{
DateTime startTime = DateTime.Now;
var random = new Random();
Console.Clear();
// Simulate initiation delay
//Console.WriteLine("initiate");
Thread.Sleep(random.Next(100, 1000));
int firstLoopCount = random.Next(1, 10);
int secondLoopCount = random.Next(1, 10);
int thirdLoopCount = random.Next(1, 10);
int fourthLoopCount = random.Next(1, 10);
int firstLoopSleepTime = random.Next(100, 1000);
int secondLoopSleepTime =random.Next(100, 1000);
int thirdLoopSleepTime = random.Next(100, 1000);
int fourthLoopSleepTime = random.Next(100, 1000);
//**************Added 1 because loop is starting from 0 and ending with different loops count including itself.
var totalTimeinMillSec = GetTotoalTimeinMilliSecond(firstLoopCount + 1, secondLoopCount + 1, thirdLoopCount + 1, fourthLoopCount + 1, firstLoopSleepTime, secondLoopSleepTime, thirdLoopSleepTime, fourthLoopSleepTime);
PrintAndGetTimeRemaining(totalTimeinMillSec);
for (int loop1 = 0; loop1 <= firstLoopCount; loop1++)
{
// Simulate loop1 delay
//Console.WriteLine("\tloop1");
Thread.Sleep(firstLoopSleepTime);
totalTimeinMillSec = PrintAndGetTimeRemaining(totalTimeinMillSec - firstLoopSleepTime);
for (int loop2 = 0; loop2 <= secondLoopCount; loop2++)
{
// Simulate loop2 delay
//Console.WriteLine("\t\tloop2");
Thread.Sleep(secondLoopSleepTime);
totalTimeinMillSec = PrintAndGetTimeRemaining(totalTimeinMillSec - secondLoopSleepTime);
for (int loop3 = 0; loop3 <= thirdLoopCount; loop3++)
{
// Simulate loop3 delay
//Console.WriteLine("\t\t\tloop3");
Thread.Sleep(thirdLoopSleepTime);
totalTimeinMillSec = PrintAndGetTimeRemaining(totalTimeinMillSec - thirdLoopSleepTime);
for (int loop4 = 0; loop4 <= fourthLoopCount; loop4++)
{
// Simulate loop4 delay
//Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(fourthLoopSleepTime);
totalTimeinMillSec = PrintAndGetTimeRemaining(totalTimeinMillSec - fourthLoopSleepTime);
}
}
}
}
}
private static int PrintAndGetTimeRemaining(int totalTimeinMillSec)
{
TimeSpan timeRemaining = TimeSpan.FromMilliseconds(totalTimeinMillSec);
Console.SetCursorPosition(0, 0);
Console.WriteLine("ETA: ");
Console.WriteLine(String.Format("{0} Days, {1} Hours, {2} Minutes, {3} Seconds, {4} Milliseconds", timeRemaining.Days.ToString().PadLeft(3, '0'), timeRemaining.Hours.ToString().PadLeft(2, '0'), timeRemaining.Minutes.ToString().PadLeft(2, '0'), timeRemaining.Seconds.ToString().PadLeft(2, '0'), timeRemaining.Milliseconds.ToString().PadLeft(2, '0')));
return totalTimeinMillSec;
}
private static int GetTotoalTimeinMilliSecond(int firstLoopCount, int secondLoopCount, int thirdLoopCount, int fourthLoopCount, int firstLoopSleepTime, int secondLoopSleepTime, int thirdLoopSleepTime, int fourthLoopSleepTime)
{
var totalTime = (firstLoopCount * firstLoopSleepTime +
firstLoopCount * secondLoopCount * secondLoopSleepTime +
firstLoopCount * secondLoopCount * thirdLoopCount * thirdLoopSleepTime +
firstLoopCount * secondLoopCount * thirdLoopCount * fourthLoopCount * fourthLoopSleepTime);
return totalTime;
}
}
}
Please let me know If I missed something.

I think this may work for you; in order to accomplish the solution I created a few classs to help.
The code itself has some comments.
First an enum, to know in which loop we are; it is not fully necessary, but it can be worth for a refactor later on.
public enum LoopEnum
{
loop1,
loop2,
loop3,
loop4
}
Then i created a class called EtaLoop which will contain the entire loop information/logic to know how long a single iteration of a loop takes, I use StopWatch:
public class EtaLoop
{
public readonly int TotalIterations;
private readonly List<long> _loopsTimesInMiliseconds;
private readonly Stopwatch _stopwatch;
public EtaLoop(int totalIterations)
{
//+1 as the loops starts on 0
TotalIterations = totalIterations+1;
_loopsTimesInMiliseconds = new List<long>();
_stopwatch = new Stopwatch();
}
public double AvgExecution()
{
return _loopsTimesInMiliseconds.Any() ? _loopsTimesInMiliseconds.Average(a => a) : 0;
}
public void Start()
{
if(!_stopwatch.IsRunning)
_stopwatch.Start();
}
public void Stop()
{
_stopwatch.Stop();
_loopsTimesInMiliseconds.Add(_stopwatch.ElapsedMilliseconds);
ResetStopWatch();
}
public int CurrentIteration()
{
return _loopsTimesInMiliseconds.Count();
}
public long EstimatedCurrentIteration()
{
return Convert.ToInt64(_loopsTimesInMiliseconds.Average(a => a) * TotalIterations);
}
private void ResetStopWatch()
{
_stopwatch.Reset();
}
}
the methods I think are clear enough without more explanation.
another class called EtaTime which will contain more logic, this class is a wrapper for EtaLoop for example, if a for has 5 iterations, it will contain 5 elements on the list.
each element (EtaLoop) is added to the list, once we finish (all the iterations are finished) It has some more methods, one of them, the "hard one" it is explained
public class EtaTime
{
public readonly List<EtaLoop> Loops;
public readonly LoopEnum Loop;
private EtaLoop _currentLoop;
public EtaTime(LoopEnum loop)
{
Loops = new List<EtaLoop>();
Loop = loop;
}
public void SetUpTotal(int totalIterations)
{
_currentLoop = new EtaLoop(totalIterations);
}
public void StartLoop()
{
_currentLoop.Start();
}
public void EndLoop()
{
_currentLoop.Stop();
}
public void RegisterLoop()
{
Loops.Add(_currentLoop);
}
/// <summary>
/// Get the average time per execution, and the average number of loops per parent loop.
/// The idea is to know how many times (and how long) the loop x is executed per each x-1
/// </summary>
/// <returns></returns>
public (double, double) GetAvgTimeAndAvgExTimes()
{
double avgTimePerLoop = Loops.Any() ? Loops.Average(a => a.AvgExecution()) : _currentLoop.AvgExecution();
double avgTotalLoopsIteration = Loops.Any() ? Loops.Average(a => a.TotalIterations) : _currentLoop.TotalIterations;
return (avgTimePerLoop, avgTotalLoopsIteration);
}
public int GetCurrentIteration()
{
return _currentLoop.CurrentIteration();
}
public int TotalIterations()
{
return _currentLoop.TotalIterations;
}
}
Finally the wrapper for the EtaTimeHelper which will contain all the EtaTimes; originally i was thinking to make a list, thats why the enum was for, but, i think like this is more clear.
Note: this class can be split/moved to extension methods.
The main points here are RegisterLoop every time we finish a loop we have to call that method; and it needs to be here because i need the information from the other loops.
public class EtaTimeHelper
{
//This part can be done in a list, but i think it is easier to see like this.
public readonly EtaTime Loop1;
public readonly EtaTime Loop2;
public readonly EtaTime Loop3;
public readonly EtaTime Loop4;
public readonly DateTime StartTime;
private DateTime _lastPrintTime;
private const int TimeBetweenPrintsInSeconds = 10;
public EtaTimeHelper()
{
Loop1 = new EtaTime(LoopEnum.loop1);
Loop2 = new EtaTime(LoopEnum.loop2);
Loop3 = new EtaTime(LoopEnum.loop3);
Loop4 = new EtaTime(LoopEnum.loop4);
StartTime = DateTime.Now;
_lastPrintTime = DateTime.MinValue;
}
public void RegisterLoop(LoopEnum loopNumber)
{
switch (loopNumber)
{
case LoopEnum.loop1:
Loop1.RegisterLoop();
break;
case LoopEnum.loop2:
Loop2.RegisterLoop();
break;
case LoopEnum.loop3:
Loop3.RegisterLoop();
break;
case LoopEnum.loop4:
Loop4.RegisterLoop();
break;
default:
throw new NotImplementedException("please register the loop");
}
PrintCompletionTime(DateTime.Now, loopNumber);
}
public void PrintCompletionTime(DateTime printTime, LoopEnum loopNumber)
{
if(_lastPrintTime.AddSeconds(TimeBetweenPrintsInSeconds) < printTime)
{
var time = CalculatePredictionTime(loopNumber);
Print(time);
_lastPrintTime = printTime;
}
}
private long CalculatePredictionTime(LoopEnum loopNumber)
{
switch (loopNumber)
{
case LoopEnum.loop1:
return LoopPrediction(Loop1.GetAvgTimeAndAvgExTimes());
case LoopEnum.loop2:
return Loop2Prediction(Loop1, Loop2);
case LoopEnum.loop3:
return Loop3Prediction(Loop1, Loop2, Loop3);
case LoopEnum.loop4:
return Loop4Prediction(Loop1, Loop2, Loop3, Loop4);
default:
throw new NotImplementedException("please register the loop");
}
//If all loops in #1 are finished, all sub loops are also finished. which means, it is the "end of the loop"
long LoopPrediction((double, double) avgTimeAndAvgExTimes)
{
double avgTimePerLoop = avgTimeAndAvgExTimes.Item1;
double avgIterations = avgTimeAndAvgExTimes.Item2;
return Convert.ToInt64(avgTimePerLoop * avgIterations);
}
long Loop2Prediction(EtaTime loop1, EtaTime loop2)
{
var loop1Prediction = LoopPrediction(loop1.GetAvgTimeAndAvgExTimes());
var loop2Values = loop2.GetAvgTimeAndAvgExTimes();
long avgPerLoop = LoopPrediction(loop2Values);
var loop1AvgIterations = loop1.GetAvgTimeAndAvgExTimes().Item2;
var expectedLoop2Iterations = loop1AvgIterations;
double loop2Predictions = avgPerLoop * expectedLoop2Iterations;
if (loop1Prediction == 0)
{
return Convert.ToInt64(loop2Predictions);
}
else
{
//1+current iteration
return loop1Prediction + loop2.GetCurrentIteration();
}
}
long Loop3Prediction(EtaTime loop1, EtaTime loop2, EtaTime loop3)
{
var loop1_2Prediction = Loop2Prediction(loop1, loop2);
var loop3Values = loop3.GetAvgTimeAndAvgExTimes();
long avgPerLoop = LoopPrediction(loop3Values);
var loop2AvgIterations = loop2.GetAvgTimeAndAvgExTimes().Item2;
var loop1AvgIterations = loop1.GetAvgTimeAndAvgExTimes().Item2;
var expectedLoop3Iterations = loop2AvgIterations * loop1AvgIterations;
double loop3Predictions = avgPerLoop * expectedLoop3Iterations;
if (loop1_2Prediction == 0)
{
return Convert.ToInt64(loop3Predictions);
}
else
{
//1-2+current iteration
return loop1_2Prediction+ loop3.GetCurrentIteration();
}
}
long Loop4Prediction(EtaTime loop1, EtaTime loop2, EtaTime loop3, EtaTime loop4)
{
var loop1_2_3Prediction = Loop3Prediction(loop1, loop2, loop3);
var loop4Values = loop4.GetAvgTimeAndAvgExTimes();
long avgPerLoop = LoopPrediction(loop4Values);
var loop2AvgIterations = loop2.GetAvgTimeAndAvgExTimes().Item2;
var loop1AvgIterations = loop1.GetAvgTimeAndAvgExTimes().Item2;
var loop3AvgIterations = loop3.GetAvgTimeAndAvgExTimes().Item2;
var expectedLoop4Iterations = loop2AvgIterations * loop1AvgIterations* loop3AvgIterations;
double loop4Predictions = avgPerLoop * expectedLoop4Iterations;
if (loop1_2_3Prediction == 0)
{
return Convert.ToInt64(loop4Predictions);
}
else
{
//1-2-3+current iteration
return loop1_2_3Prediction + loop4.GetCurrentIteration();
}
}
}
private void Print(long ms)
{
DateTime estimatedCompletionTime = StartTime.AddMilliseconds(ms);
TimeSpan leftTime = (estimatedCompletionTime - DateTime.Now);
Console.WriteLine("ETA: ");
Console.WriteLine($"{leftTime.Days} Days, {leftTime.Hours} Hours, {leftTime.Minutes} Minutes, {leftTime.Seconds} Seconds");//, leftTime.Days.ToString().PadLeft(3, '0'), leftTime.Hours.ToString().PadLeft(2, '0'), leftTime.Minutes.ToString().PadLeft(2, '0'), leftTime.Seconds.ToString().PadLeft(2, '0')));
Console.WriteLine($"on {estimatedCompletionTime.ToString("yyyy/MM/dd HH:mm:ss")}");
Console.WriteLine($"Current Time: {DateTime.Now.ToString("yyyy/MM/dd HH:mm:ss")}");
}
}
and the print. as you mention you want some mecanisim to print, but you did not specify if it is a consol app a web or a winforms app. Probably it is winforms if it is a desktop app you probably can do something like myLabel.SetPropertyThreadSafe to modify the label you want to update.
The algorithim which calculates the time it is on CalculatePredictionTime I tried to make it a single method for n number of loops but I couldnt.
Now the main.
I did a few changes on the Thread.Sleep as it is easier for me to figure it out if makes sense while i was testing. also, a range 10-100 (10x) I thought it was very big, i reduced it for testing reasons.
the code follows a pattern, first you need to instanciate the EtaTimeHelper class.
before each loop you have to setUpTotal which means you have to pass the number of iterations are going to be.
The first line of the loop will be .StartLoop() this will start the StopWatch and the last line of the loop (before the }) will be a .EndLoop();
after the closing bracket } you have to indicate the etaTimeHelper.RegisterLoop(loopNumber); passing as parameter the loop number this will print if necessary the time.
static void Main(string[] args)
{
var random = new Random();
Console.Clear();
// Simulate initiation delay
//Console.WriteLine("initiate");
EtaTimeHelper etaTimeHelper = new EtaTimeHelper();
int intCount = random.Next(1, 10);
etaTimeHelper.Loop1.SetUpTotal(intCount);
for (int loop1 = 0; loop1 <= intCount; loop1++)
{
etaTimeHelper.Loop1.StartLoop();
// Simulate loop1 delay
Console.WriteLine("\tloop1");
Thread.Sleep(random.Next(40, 50));
//number of times the loop 2 is going to execute inside this loop 1;
int loop2times = random.Next(1, 10);
etaTimeHelper.Loop2.SetUpTotal(loop2times);
for (int loop2 = 0; loop2 <= loop2times; loop2++)
{
etaTimeHelper.Loop2.StartLoop();
// Simulate loop2 delay
//Console.WriteLine("\t\tloop2");
Thread.Sleep(random.Next(30, 40));
//number of times the loop 3 is going to execute inside this loop 3;
int loop3times = random.Next(1, 10);
etaTimeHelper.Loop3.SetUpTotal(loop3times);
for (int loop3 = 0; loop3 <= loop3times; loop3++)
{
etaTimeHelper.Loop3.StartLoop();
// Simulate loop3 delay
//Console.WriteLine("\t\t\tloop3");
Thread.Sleep(random.Next(10, 20));
var loop4Times = random.Next(1, 10);
etaTimeHelper.Loop4.SetUpTotal(loop4Times);
for (int loop4 = 0; loop4 <= loop4Times; loop4++)
{
etaTimeHelper.Loop4.StartLoop();
// Simulate loop4 delay
//Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(random.Next(20, 30));
etaTimeHelper.Loop4.EndLoop();
}
etaTimeHelper.RegisterLoop(LoopEnum.loop4);
etaTimeHelper.Loop3.EndLoop();
}
etaTimeHelper.RegisterLoop(LoopEnum.loop3);
etaTimeHelper.Loop2.EndLoop();
}
etaTimeHelper.RegisterLoop(LoopEnum.loop2);
etaTimeHelper.Loop1.EndLoop();
}
etaTimeHelper.RegisterLoop(LoopEnum.loop1);
}
Here is a "working" fiddle https://dotnetfiddle.net/Z06W4g
note: it times out after a few seconds, in your local machine works fine.

I think you will have to benchmark it a few times for the operation that you need and then extrapolate. There are variables here. The compute power and storage type (if involved) and network (if involved) of the target will all contribute to the ETA and every execution will have a different execution time. But ETA can be closely predicted after some benchmarking. And if you want to be really clever, you can show a warning that it may take more time than the displayed ETA because of x, y and z e.g. CPU utilisation by other processes running on the target.
Jon Skeet is an authority on this topic and he has a good resource here:
link
Also read here for how the types in your operation and bitness of the CPU will affect your ETA:
link

In order to assert the ETA every second , what I have done is attaching an event to a System.Timers.Timer object. The event will be triggered every second in order to continuously providing the user with the ETA feedback.
System.Timers.Timer aTimer = new System.Timers.Timer(1000);
aTimer.Elapsed += (sender, e) => ATimer_Elapsed(sender, e, new MyEventArguments
{
Loop1TotalIterations = intCount,
CurrentIndex = loop1,
Start = startTime
});
aTimer.AutoReset = true;
aTimer.Enabled = true;
In your example we are using the Thread.Sleep in order simulate a delay.
Using Thread.Sleep will put everything to sleep - therefor the time logic will be inaccurate.
What I have done is creating a Multi-thread application - in order to run the logic on one thread and the time on another. Therefor when we pause the time checking and ETA will continue to run.
Putting it all together we have the following:
namespace ConsoleApp10
{
#region Usings
using System;
using System.Threading;
using System.Threading.Tasks;
using System.Timers;
#endregion
/// <summary>
/// Event class to be passed to ElapsedEventHandler
/// </summary>
public class MyEventArguments
{
public int Loop1TotalIterations { get; set; }
public DateTime Start { get; set; }
public DateTime CurrentDate => DateTime.Now;
public int CurrentIndex { get; set; }
}
class Program
{
/// <summary>
/// Compute the remaining time
/// </summary>
public static void ComputeRemaining((int count, DateTime time) start, (int count, DateTime time) current, int end)
{
var T = current.count - start.count == 0
? TimeSpan.MaxValue
: TimeSpan.FromSeconds((end - current.count) * current.time.Subtract(start.time).TotalSeconds / (current.count - start.count));
Console.Clear();
Console.SetCursorPosition(0, 0);
Console.ForegroundColor = (ConsoleColor)15;
Console.WriteLine(String.Format("ETA: {0} Days, {1} Hours, {2} Minutes, {3} Seconds",
T.Days.ToString().PadLeft(3, '0'),
T.Hours.ToString().PadLeft(2, '0'),
T.Minutes.ToString().PadLeft(2, '0'),
T.Seconds.ToString().PadLeft(2, '0')));
Console.WriteLine();
}
static void Main(string[] args)
{
DateTime startTime = DateTime.Now;
Random random = new Random();
int intCount = random.Next(1, 10);
int loop1 = 0;
var time = Task.Factory.StartNew(() =>
{
System.Timers.Timer aTimer = new System.Timers.Timer(1000);
aTimer.Elapsed += (sender, e) => ATimer_Elapsed(sender, e, new MyEventArguments
{
Loop1TotalIterations = intCount,
CurrentIndex = loop1,
Start = startTime
});
aTimer.AutoReset = true;
aTimer.Enabled = true;
});
var logic = Task.Factory.StartNew(() =>
{
PrintAndSimulate("Initiate");
for (loop1 = 0; loop1 <= intCount; loop1++)
{
PrintAndSimulate("\tloop1");
for (int loop2 = 0; loop2 <= random.Next(1, 10); loop2++)
{
PrintAndSimulate("\t\tloop2");
for (int loop3 = 0; loop3 <= random.Next(1, 10); loop3++)
{
PrintAndSimulate("\t\t\tloop3");
for (int loop4 = 0; loop4 <= random.Next(1, 10); loop4++)
{
PrintAndSimulate("\t\t\t\tloop4");
}
}
}
}
});
Task.WaitAll(new[] { time, logic });
}
/// <summary>
/// Display the info string and simulates a delay
/// </summary>
private static void PrintAndSimulate(string info)
{
int time = new Random().Next(100, 1000);
Console.SetCursorPosition(5, 5);
Console.ForegroundColor = (ConsoleColor)new Random().Next(15);
Console.WriteLine(info);
Thread.Sleep(time);
}
/// <summary>
/// ElapsedEventHandler
/// </summary>
private static void ATimer_Elapsed(object sender, ElapsedEventArgs e, MyEventArguments myEventArguments)
{
ComputeRemaining((0, myEventArguments.Start), (myEventArguments.CurrentIndex, myEventArguments.CurrentDate), myEventArguments.Loop1TotalIterations);
}
}
}
I have added some colours to the console feedback - making a change more evident.

This may not be ideal solution but has potential to solve the issue.
First you build a Tree of Action containing what you are going to execute and then execute the Actions by iterating the tree.
The tree acts as index of what you are going to execute. Even you can track what is being executed. Whats done and whats pending.
I have nested the actions in single level in the tree for demo purpose but you can nest the tree as much as possible, inside loops etc.(pseudo code ahead but poc builds and executes well)
class Program
{
static void Main(string[] args)
{
var random = new Random();
TreeNode<Action> root = new TreeNode<Action>(() => { });
var loop1 = root.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop2 = loop1.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop3 = loop2.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop4 = loop3.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop5 = loop4.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop6 = loop5.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
root.Execute(DateTime.Now);
Console.ReadLine();
}
}
public static class Extensions
{
public static string Humanize(this TimeSpan timeRemaining)
{
return String.Format("{0} Days, {1} Hours, {2} Minutes, {3} Seconds", timeRemaining.Days.ToString().PadLeft(3, '0'), timeRemaining.Hours.ToString().PadLeft(2, '0'), timeRemaining.Minutes.ToString().PadLeft(2, '0'), timeRemaining.Seconds.ToString().PadLeft(2, '0'));
}
public static void PrintETA(this TimeSpan timeRemaining)
{
//Console.SetCursorPosition(0, 0);
Console.Write("ETA: ");
Console.Write(timeRemaining.Humanize());
}
public static TimeSpan ComputeRemaining((int count, DateTime time) start, (int count, DateTime time) current, int end) =>
current.count - start.count == 0
? TimeSpan.MaxValue
: TimeSpan.FromSeconds((end - current.count) * current.time.Subtract(start.time).TotalSeconds / (current.count - start.count));
public static void Execute(this TreeNode<Action> root, DateTime startTime)
{
var current = root;
var end = current.Count();
var currentCount = 1;
Iterate:
var Count = current.Children.Count();
for (int i = 0; i < Count; i++)
{
TreeNode<Action> node = current.Children.ElementAt(i);
node.Data();
TimeSpan timeRemaining = ComputeRemaining((0, startTime), (currentCount++, DateTime.Now), end-1);
timeRemaining.PrintETA();
Console.WriteLine();
Console.WriteLine("Processed {0} of {1}", currentCount - 1, end-1);
if (node.Children.Count() > 0)
{
current = node;
goto Iterate;
}
}
}
}
Ref:
TreeNode.cs
Yet another tree structure: https://github.com/gt4dev/yet-another-tree-structure

Updated.
The first version of the answer calculated the values of RemainingTime and TotalTime at the end of every iteration. Given the way the for loops are nested, it could cause long delays between updates. In order to read those values at given intervals, some changes are made.
Let's start with the Loop class. It is used to keep track of the details of every for loop, like the total amount of iterations, the current iteration and the time consumed in every iteration. To acomplish the later, two System.Diagnostic Stopwatch are used. One is kept running free, not reseted, to ease the calculation of the average time for an iteration. The other clock is reseted on every iteration to provide a value for LoopTime, used in calculations on the fly of RemainingTime and TotalTime, when accesed via the property getters. When the iteration ends, signaled by the method StopClock(), the average loop time and the related properties are updated. The values obtained here are more accurate than the ones calculated on the fly.
Its parent, LoopTimer class, is in charge of creating and storing references of Loop instances, start and stop clocks and calculate the global ETA. The method EnterLoop() is used at the beginning of the for loop. It creates a new Loop. The overload EnterLoop() with a single parameter is used in the rest of iterations to retrieve Loop instances stored in a Dictionary. The method ExitLoop(), at the end of the loop, stops the clock and updates calculations.
Functions ExitLoopRetETA() and ExitLoopRetTotalEta() are provided as replacements of ExitLoop(), to print data at the end of the loop, for testing or debugging. Method Bonus() shows how to use it. Method ShowStatus() and function GetStatus provide internal information on the Loop objects.
To show the values periodically, a Task is used, to run DoUpdate() (or DoUpdateTotal() or DoUpdateStatus())in a separate thread.
The target framework is .Net 4.0
The working classes:
using System;
using System.Collections.Generic;
using System.Diagnostics;
namespace LoopTimer
{
public struct TotalEta
{
public TimeSpan Eta;
public TimeSpan Total;
}
internal class LoopTimer
{
// Private helper class
private class Loop
{
// Declarations
private Stopwatch _clock;
private Stopwatch _interval_clock;
// Constructor
public Loop(int index, int count)
{
Index = index;
Count = count;
_clock = new Stopwatch();
_interval_clock = new Stopwatch();
}
// Properties
internal int Index { get; set; }
internal int Count { get; private set; }
private double _loopTimeMs;
internal double LoopTimeMs
{
get
{
if (_clock.IsRunning)
{
return _interval_clock.Elapsed.TotalMilliseconds;
}
else
return _loopTimeMs; ;
}
}
private double _remainingTimeMs;
internal double RemainingTimeMs
{
get
{
if (_clock.IsRunning)
return CalculateRemainingTime();
else
return _remainingTimeMs;
}
}
private double _totalTimeMs;
internal double TotalTimeMs
{
get
{
if (_clock.IsRunning)
return CalculateTotalTime();
else
return _totalTimeMs;
}
}
internal TimeSpan LoopTime
{
get { return TimeSpan.FromMilliseconds(LoopTimeMs); }
}
internal TimeSpan TotalTime
{
get { return TimeSpan.FromMilliseconds(TotalTimeMs); }
}
internal TimeSpan RemainingTime
{
get { return TimeSpan.FromMilliseconds(RemainingTimeMs); }
}
// Methods
internal void StartClock()
{
_clock.Start();
_interval_clock.Start();
}
internal void StopClock()
{
_clock.Stop();
_interval_clock.Stop();
UpdateTimes();
_interval_clock.Reset();
}
private void UpdateTimes()
{
// reading clock
double elapsed = _clock.Elapsed.TotalMilliseconds;
// Calculating average loop time. The Stopwatch is not reseted between iterations.
_loopTimeMs = elapsed / (Index + 1);
// Calculating estimated remaining time = average loop time * remaining iterations.
_remainingTimeMs = CalculateRemainingTime();
// Calculating estimated total time = average loop time * iterations.
_totalTimeMs = CalculateTotalTime();
}
private double CalculateRemainingTime()
{
// Calculating estimated remaining time = average loop time * remaining iterations.
double time;
int countt = Count - Index;
if (countt > 1)
time = LoopTimeMs * countt;
else if (countt == 1)
time = LoopTimeMs;
else
time = 0;
return time;
}
private double CalculateTotalTime()
{
return LoopTimeMs * Count;
}
}
// End Private helper class
// Declarations
private Dictionary<int, Loop> _loopDict;
private int _loopIndex;
// Constructor
public LoopTimer()
{
_loopDict = new Dictionary<int, Loop>();
_loopIndex = -1;
}
// Properties
internal TimeSpan TotalTime
{
get { return TimeSpan.FromMilliseconds(TotalTimeMs); }
}
internal TimeSpan RemainingTime
{
get { return TimeSpan.FromMilliseconds(RemainingTimeMs); }
}
private double TotalTimeMs
{ get { return CalculateTotalTime(); } }
private double RemainingTimeMs
{ get { return CalculateRemainingTime(); } }
// Methods
internal void EnterLoop(int index, int count)
{
Loop newLoop;
// increase index
_loopIndex++;
if (!_loopDict.ContainsKey(_loopIndex))
{
// create new Loop
newLoop = new Loop(index, count);
_loopDict[_loopIndex] = newLoop;
}
else
{ // retrieve Loop from Dict
newLoop = _loopDict[_loopIndex];
}
newLoop.StartClock();
}
internal void EnterLoop(int index)
{
// increase index
_loopIndex++;
// retrive loop & start clock
_loopDict[_loopIndex].Index = index;
_loopDict[_loopIndex].StartClock();
}
internal void ExitLoop()
{ // retrive loop & stop clock
_loopDict[_loopIndex].StopClock();
// decrease index
_loopIndex--;
}
// bonus method
internal TimeSpan ExitLoopRetETA()
{ // retrive loop & stop clock
_loopDict[_loopIndex].StopClock();
// decrease index
_loopIndex--;
return RemainingTime;
}
// bonus method
internal TotalEta ExitLoopRetTotalEta()
{
TotalEta retval;
retval.Eta = ExitLoopRetETA();
retval.Total = TotalTime;
return retval;
}
// debug method
internal void ShowStatus()
{
Console.WriteLine("Status:");
Console.WriteLine(" RemainingTime:");
for (int i = 0; i < _loopDict.Count; i++)
{
TimeSpan time = _loopDict[i].RemainingTime;
Console.WriteLine(string.Format(" Loop: {0} Value: {1}", i, time.ToString()));
}
Console.WriteLine();
}
// debug method
internal TotalEta[] GetStatus()
{
TotalEta[] retArr = new TotalEta[_loopDict.Count];
TotalEta retval;
for (int i = 0; i < _loopDict.Count; i++)
{
retval = new TotalEta();
retval.Eta = _loopDict[i].RemainingTime;
retval.Total = _loopDict[i].TotalTime;
retArr[i] = retval;
}
return retArr;
}
private double CalculateRemainingTime()
{
double max, time;
max = 0;
// Remaining Time, the greater of all
for (int i = 0; i < _loopDict.Count; i++)
{
time = _loopDict[i].RemainingTimeMs;
if (time > max)
max = time;
}
return max;
}
// Total Time, bonus
private double CalculateTotalTime()
{
double max, time;
max = 0;
// Total Time, the greater of all
for (int i = 0; i < _loopDict.Count; i++)
{
time = _loopDict[i].TotalTimeMs;
if (time > max)
max = time;
}
return max;
}
}
}
The sample program:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace LoopTimer
{
class Solution
{
static CancellationTokenSource ts;
static void Main(string[] args)
{
Console.Clear();
LoopTimer lm = new LoopTimer();
var random = new Random();
// For easy change test parameters
int minRndCount = 1;
int maxRndCount = 10;
int minRndSleep = 100;
int maxRndSleep = 1000;
// A task to update console, with cancellation token
ts = new CancellationTokenSource();
Task updater = new Task(() => DoUpdate(lm), ts.Token);
// Uncomment to show estimated total time.
//Task updater = new Task(() => DoUpdateTotal(lm), ts.Token);
// Uncomment to show estimated total time and internal values of every loop.
//Task updater = new Task(() => DoUpdateStatus(lm), ts.Token);
// Simulate initiation delay
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
// Console.WriteLine("initiate");
updater.Start();
int intCountL1 = random.Next(minRndCount, maxRndCount);
for (int loop1 = 0; loop1 <= intCountL1; loop1++)
{
// Entering loop1
if (loop1 == 0)
lm.EnterLoop(loop1, intCountL1);
else
lm.EnterLoop(loop1);
// Simulate loop1 delay
//Console.WriteLine("\tloop1");
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
int intCountL2 = random.Next(minRndCount, maxRndCount);
for (int loop2 = 0; loop2 <= intCountL2; loop2++)
{
// Entering loop2
if (loop2 == 0)
lm.EnterLoop(loop2, intCountL2);
else
lm.EnterLoop(loop2);
// Simulate loop2 delay
//Console.WriteLine("\t\tloop2");
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
int intCountL3 = random.Next(minRndCount, maxRndCount);
for (int loop3 = 0; loop3 <= intCountL3; loop3++)
{
// Entering loop3
if (loop3 == 0)
lm.EnterLoop(loop3, intCountL3);
else
lm.EnterLoop(loop3);
// Simulate loop3 delay
//Console.WriteLine("\t\t\tloop3");
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
int intCountL4 = random.Next(minRndCount, maxRndCount);
for (int loop4 = 0; loop4 <= intCountL4; loop4++)
{
// Entering loop4
if (loop4 == 0)
lm.EnterLoop(loop4, intCountL4);
else
lm.EnterLoop(loop4);
// Simulate loop4 delay
//Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
// Exiting loop4
lm.ExitLoop();
}
// Exiting loop3
lm.ExitLoop();
}
// Exiting loop2
lm.ExitLoop();
}
// Exiting loop1
lm.ExitLoop();
}
ts.Cancel();
}
static private void DoUpdate(LoopTimer lm)
{
char[] animchar = { '|', '/', '-', '\\' };
int index = 0;
Thread.Sleep(100);
while (true)
{
TimeSpan eta = lm.RemainingTime;
Console.SetCursorPosition(0, 0);
Console.Write(string.Format(" {4} ETA: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", eta.Days.ToString().PadLeft(3, '0'), eta.Hours.ToString().PadLeft(2, '0'), eta.Minutes.ToString().PadLeft(2, '0'), eta.Seconds.ToString().PadLeft(2, '0'), animchar[index].ToString()));
if (++index > 3)
index = 0;
Thread.Sleep(1000);
ts.Token.ThrowIfCancellationRequested();
}
}
/*
This method is provided as a sample on displaying the estimated total time.
*/
static private void DoUpdateTotal(LoopTimer lm)
{
char[] animchar = { '|', '/', '-', '\\' };
int index = 0;
Thread.Sleep(100);
while (true)
{
TimeSpan eta = lm.RemainingTime;
TimeSpan total = lm.TotalTime;
Console.SetCursorPosition(0, 0);
Console.Write(string.Format(" {4} ETA: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", eta.Days.ToString().PadLeft(3, '0'), eta.Hours.ToString().PadLeft(2, '0'), eta.Minutes.ToString().PadLeft(2, '0'), eta.Seconds.ToString().PadLeft(2, '0'), animchar[index].ToString()));
Console.Write(string.Format("\n Total: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", total.Days.ToString().PadLeft(3, '0'), total.Hours.ToString().PadLeft(2, '0'), total.Minutes.ToString().PadLeft(2, '0'), total.Seconds.ToString().PadLeft(2, '0')));
if (++index > 3)
index = 0;
Thread.Sleep(1000);
ts.Token.ThrowIfCancellationRequested();
}
}
/*
This method is provided as a sample on displaying the estimated total time, and
the internal values of every loop.
*/
static private void DoUpdateStatus(LoopTimer lm)
{
char[] animchar = { '|', '/', '-', '\\' };
int index = 0;
Thread.Sleep(100);
while (true)
{
TimeSpan eta = lm.RemainingTime;
TimeSpan total = lm.TotalTime;
TotalEta[] status = lm.GetStatus();
Console.SetCursorPosition(0, 0);
Console.Write(string.Format(" {4} ETA: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", eta.Days.ToString().PadLeft(3, '0'), eta.Hours.ToString().PadLeft(2, '0'), eta.Minutes.ToString().PadLeft(2, '0'), eta.Seconds.ToString().PadLeft(2, '0'), animchar[index].ToString()));
Console.Write(string.Format("\n Total: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", total.Days.ToString().PadLeft(3, '0'), total.Hours.ToString().PadLeft(2, '0'), total.Minutes.ToString().PadLeft(2, '0'), total.Seconds.ToString().PadLeft(2, '0')));
Console.WriteLine();
int loop = 0;
foreach (var item in status)
{
Console.Write(string.Format("\n Loop: {0} ETA: {1} \tTotal: {2}", loop, item.Eta.ToString(#"hh\:mm\:ss\.FFFF"), item.Total.ToString(#"hh\:mm\:ss\.FFFF")));
loop++;
}
if (++index > 3)
index = 0;
Thread.Sleep(1000);
ts.Token.ThrowIfCancellationRequested();
}
}
/*
This method is provided as a sample for variations on
the ExitLoopRet method. Uses in-place calls.
*/
static internal void Bonus()
{
TotalEta remVal;
TimeSpan remTime;
LoopTimer lm = new LoopTimer();
Console.Clear();
// easy change test parameters
var random = new Random();
int minRndCount = 1;
int maxRndCount = 5;
int maxRndSleep = 1000;
// First, outer loop
int intCountL1 = random.Next(minRndCount, maxRndCount);
for (int i = 0; i < intCountL1; i++)
{
if (i == 0)
lm.EnterLoop(i, intCountL1);
else
lm.EnterLoop(i);
Console.WriteLine(string.Format("\nLoop1 begin iteration: {0} of {1}. Will work(sleep) for {2} ms.", i, intCountL1 - 1, maxRndSleep));
Thread.Sleep(maxRndSleep);
// Second, middle loop
int intCountL2 = random.Next(minRndCount, maxRndCount);
for (int j = 0; j < intCountL2; j++)
{
if (j == 0)
lm.EnterLoop(j, intCountL2);
else
lm.EnterLoop(j);
Console.WriteLine(string.Format("\n\tLoop2 begin iteration: {0} of {1}. Will work(sleep) for {2} ms.", j, intCountL2 - 1, maxRndSleep));
Thread.Sleep(maxRndSleep);
// Third, inner loop
int intCountL3 = random.Next(minRndCount, maxRndCount);
for (int k = 0; k < intCountL3; k++)
{
if (k == 0)
lm.EnterLoop(k, intCountL3);
else
lm.EnterLoop(k);
Console.WriteLine(string.Format("\n\t\tLoop3 begin iteration: {0} of {1}. Will work(sleep) for {2} ms.", k, intCountL3 - 1, maxRndSleep));
Thread.Sleep(maxRndSleep);
lm.ExitLoop();
Console.WriteLine(string.Format("\n\t\tLoop3 end iteration: {0} of {1}", k, intCountL3 - 1));
lm.ShowStatus();
}
remTime = lm.ExitLoopRetETA();
Console.WriteLine(string.Format("\n\tLoop2 end iteration: {0} of {1}", j, intCountL2 - 1));
Console.WriteLine("\t\tRem: " + remTime.ToString());
}
remVal = lm.ExitLoopRetTotalEta();
Console.WriteLine(string.Format("\nLoop1 end iteration: {0} of {1}", i, intCountL1 - 1));
Console.WriteLine("\t\tTot: " + remVal.Total.ToString());
Console.WriteLine("\t\tRem: " + remVal.Eta.ToString());
}
}
}
}

Comparison time- Single Threads vs Multiple Threads differing in results

I wrote this code in purpose to test multi and single threading speeds. Thanks for all the feedback! I rewrote most of it based on the great comments I received. This now functions properly (maybe has a bug here or there), tests multi threads first, and takes an average to find a more accurate speed: (Scroll to bottom for cont.)
Main method Class
using System;
namespace SingleAndMultiThreading
{
internal class Threads
{
private static void Main(string[] args)
{
long numOfObjCreated;
int numberOfTests;
while (true)
{
try
{
Console.Write("Number of objects to create: ");
numOfObjCreated = Convert.ToInt64(Console.ReadLine());
break;
}
catch (Exception)
{
Console.WriteLine("Invalid input.");
}
}
while (true)
{
try
{
Console.Write("Number of tests to run: ");
numberOfTests = Convert.ToInt32(Console.ReadLine());
break;
}
catch (Exception)
{
Console.WriteLine("Invalid input.");
}
}
CalculateResults(numOfObjCreated, numberOfTests);
Console.ReadKey();
}
private static void CalculateResults(long numOfObjCreated, int numberOfTests)
{
double totalPercentages = 0;
for (var i = 0; i < numberOfTests; i++)
{
totalPercentages += CompleteTests(numOfObjCreated);
}
var accuracy = totalPercentages / numberOfTests;
if ((int)accuracy == 0)
{
Console.WriteLine("\nIn this case, neither single threading or multithreading is faster.\n" +
"They both run equally well under these conditions.\n");
return;
}
if (accuracy < 0)
{
Console.WriteLine("\nIn this case with {0} objects being created, single threading is faster!\n",
string.Format("{0:#,###0}", numOfObjCreated));
return;
}
Console.WriteLine("\nFrom {0} test(s), {1}% was the average percentage of increased speed in multithreading.\n",
string.Format("{0:#,###0}", numberOfTests), string.Format("{0:#,###0}", accuracy));
}
private static double CompleteTests(long numOfObjCreated)
{
Console.WriteLine("Computing...");
var numOfCores = Environment.ProcessorCount;
var timeForMultiThread = MultiThread.Run(numOfObjCreated, numOfCores);
var timeForSingleThread = SingleThread.Run(numOfObjCreated);
var percentFaster = ((timeForSingleThread / timeForMultiThread) * 100) - 100;
//note: .NET does its part in assigning a certian thread to its own core
Console.WriteLine("Using all {0} cores, creating {1} objects is {2}% faster.",
numOfCores, string.Format("{0:#,###0}", numOfObjCreated), string.Format("{0:#,###0}", percentFaster));
return percentFaster;
}
}
}
Single Threading Class
using System;
using System.Diagnostics;
namespace SingleAndMultiThreading
{
internal class SingleThread
{
public static double Run(long numOfObjCreated)
{
var watch = new Stopwatch();
watch.Start();
for (long i = 0; i < numOfObjCreated; i++)
{
new object();
}
watch.Stop();
var totalTime = watch.ElapsedTicks;
Console.WriteLine("The time to create {0} objects with 1 thread is: {1} ticks.",
string.Format("{0:#,###0}", numOfObjCreated), string.Format("{0:#,###0}", totalTime));
return totalTime;
}
}
}
Multi Threading Class
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
namespace SingleAndMultiThreading
{
internal class MultiThread
{
public static double Run(long numOfObjCreated, int numOfCores)
{
var watch = new Stopwatch();
var workerObject = new Worker(numOfObjCreated / numOfCores);
var listOfThreads = new List<Thread>();
for (long k = 0; k < numOfCores; k++)
{
var workerThread = new Thread(workerObject.DoWork);
listOfThreads.Add(workerThread);
}
watch.Start();
foreach (var thread in listOfThreads)
{
thread.Start();
}
byte countOfCompletedThreads = 0;
while (true)
{
foreach (var thread in listOfThreads)
if (!thread.IsAlive)
countOfCompletedThreads++;
if (countOfCompletedThreads == numOfCores)
break;
countOfCompletedThreads = 0;
}
watch.Stop();
var totalTime = watch.ElapsedTicks;
Console.WriteLine("The time to create {0} objects utilizing all {1} cores is: {2} ticks.",
string.Format("{0:#,###0}", numOfObjCreated), numOfCores, string.Format("{0:#,###0}", totalTime));
return totalTime;
}
}
}
Worker Class
namespace SingleAndMultiThreading
{
public class Worker
{
private readonly long _numOfObjToCreate;
public bool IsDone;
public Worker(long numOfObjToCreate)
{
_numOfObjToCreate = numOfObjToCreate;
}
public void DoWork()
{
for (long i = 0; i < _numOfObjToCreate; i++)
{
new object();
}
IsDone = true;
}
}
}
The output of this code is a bit too long to post (I urge you to copy and paste into your own IDE, its really fascinating). I guess the accepted answer that this doesn't give the same result per every test is due to CPU scheduling, other or minor issues like ASLR and such. More than one thing is happening aside from visual studio running this program, and priorities differently. Also thank you for pointing out that running multi threading first helps because of the already-done memory allocation!
Another thing to point out, I found this while running:
The spikes are when the process of multi threading takes place.

Parallel.Foreach loop gets different result than For loop?

I've
written simple for loop iterating through array and Parallel.ForEach loop doing the same thing. However, resuls I've get are different so I want to ask what the heck is going on? :D
class Program
{
static void Main(string[] args)
{
long creating = 0;
long reading = 0;
long readingParallel = 0;
for (int j = 0; j < 10; j++)
{
Stopwatch timer1 = new Stopwatch();
Random rnd = new Random();
int[] array = new int[100000000];
timer1.Start();
for (int i = 0; i < 100000000; i++)
{
array[i] = rnd.Next(5);
}
timer1.Stop();
long result = 0;
Stopwatch timer2 = new Stopwatch();
timer2.Start();
for (int i = 0; i < 100000000; i++)
{
result += array[i];
}
timer2.Stop();
Stopwatch timer3 = new Stopwatch();
long result2 = 0;
timer3.Start();
Parallel.ForEach(array, (item) =>
{
result2 += item;
});
if (result != result2)
{
Console.WriteLine(result + " - " + result2);
}
timer3.Stop();
creating += timer1.ElapsedMilliseconds;
reading += timer2.ElapsedMilliseconds;
readingParallel += timer3.ElapsedMilliseconds;
}
Console.WriteLine("Create : \t" + creating / 100);
Console.WriteLine("Read: \t\t" + reading / 100);
Console.WriteLine("ReadP: \t\t" + readingParallel / 100);
Console.ReadKey();
}
}
So in the condition I get results:
result = 200009295;
result2 = 35163054;
Is there anything wrong?

The += operator is non-atomic and actually performs multiple operations:
load value at location that result is pointing to, into memory
add array[i] to the in-memory value (I'm simplifying here)
write the result back to result
Since a lot of these add operations will be running in parallel it is not just possible, but likely that there will be races between some of these operations where one thread reads a result value and performs the addition, but before it has the chance to write it back, another thread grabs the old result value (which hasn't yet been updated) and also performs the addition. Then both threads write their respective values to result. Regardless of which one wins the race, you end up with a smaller number than expected.
This is why the Interlocked class exists.
Your code could very easily be fixed:
Parallel.ForEach(array, (item) =>
{
Interlocked.Add(ref result2, item);
});
Don't be surprised if Parallel.ForEach ends up slower than the fully synchronous version in this case though. This is due to the fact that
the amount of work inside the delegate you pass to Parallel.ForEach is very small
Interlocked methods incur a slight but non-negligible overhead, which will be quite noticeable in this particular case

Duplicate math result in Bakery Algorithm (C# code)

Index out of bounds when create new thread with parameters? - Continue to my previous topic , now i got a new problem with my my Bakery Algorithm code !
Here's my code
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace BakeryAlgorithm
{
class Program
{
static int threads = 10;
static string x = "";
static int count = 0;
static int[] ticket = new int[threads];
static bool[] entering = new bool[threads];
public static void doLock(int pid)
{
for (int i = 0; i < threads; i++)
{
ticket[i] = 0;
entering[i] = false;
}
entering[pid] = true;
int max = 0;
for (int i = 0; i < threads; i++)
{
if (ticket[i] > ticket[max]) { max = i; }
}
ticket[pid] = 1+max;
entering[pid] = false;
for (int i = 0; i < threads; ++i)
{
if (i != pid)
{
while (entering[i])
{
Thread.Yield();
}
while (ticket[i] != 0 && (ticket[pid] > ticket[i] ||
(ticket[pid] == ticket[i] && pid > i)))
{
Thread.Yield();
}
}
}
if (x == "C" || x == "c")
Console.WriteLine("[System] PID " + pid.ToString() + " get into critical section");
}
public static void unlock(int pid)
{
ticket[pid] = 0;
count++;
Console.WriteLine("[Thread] PID " + pid.ToString() + " complete.");
}
public static void arrayInit()
{
for (int i = 0; i < threads; i++)
{
ticket[i] = 0;
entering[i] = false;
}
}
public static void simThread(int i)
{
doLock(i);
if (x == "C" || x=="c")
Console.WriteLine("[Thread] PID " + i.ToString() + " begin to process...");
//Do some thing ????
Random rnd = new Random((int)DateTime.Now.Ticks & 0x0000FFFF);
int a = rnd.Next(1,99);
int b = rnd.Next(1,99);
int c = rnd.Next(1,4);
int d = 0;
string o="";
if (c == 1)
{
d = a + b;
o="+";
}
else if (c == 2)
{
d = a * b;
o="*";
}
else if (c == 3)
{
d = a / b;
o="/";
}
else
{
d = a - b;
o="-";
}
if (x == "C" || x == "c")
Console.WriteLine("Math Result : " + a.ToString() + o + b.ToString() + "=" + d.ToString());
unlock(i);
}
[STAThread]
static void Main(string[] args)
{
arrayInit();
string choice="C";
while (choice == "C" || x == "c")
{
Console.WriteLine("Process log (C=Yes,K=No) : ");
x = Console.ReadLine();
if (x == "")
x = "C";
Console.Clear();
Console.WriteLine("----------------------------------");
Console.WriteLine("Bakery Algorithm C#");
Console.WriteLine("Number of threads : " + threads.ToString());
Console.WriteLine("Process Log...");
Console.WriteLine("----------------------------------");
Thread[] threadArray = new Thread[threads];
for (int i = 0; i < 10; i++)
{
int copy = i;
threadArray[i] = new Thread(() => simThread(copy));
if (x == "C" || x == "c")
Console.WriteLine("[System] PID " + i.ToString() + " created");
threadArray[i].Start();
}
Console.ReadLine();
Console.WriteLine("----------------------------------");
Console.WriteLine("Complete processed " + count.ToString() + " threads !");
count = 0;
Console.WriteLine("----------------------------------");
Console.WriteLine("You want to restart (Yes=C or No=K)");
choice = Console.ReadLine();
if (choice == "")
choice = "C";
}
}
}
}
The result are here :
2*2=4
2*2=4 << duplicated
3*2=6
4*2=8
4*6=24
4*2=8 << duplicated
.... and continue with duplicate values ( random position ) !
Hope somebody here can help !

There's many things wrong with your code, but the most important part is that you didn't read the requirements that make Lamport's bakery work:
Lamport's bakery algorithm assumes a sequential consistency memory model.
You will be hard-pressed to find a modern computer that has sequential consistency.
So even if your implementation was correct with respect to those constraints, it would still be wrong on pretty much any computer that runs .NET. To make this work on a modern CPU and in .NET, you'll need to insert memory barriers to prevent instruction reordering and introduce cache refreshing to make sure each CPU core sees the same values... and by then you're probably better off using different synchronization primitives altogether.
Now, fixing these kinds of algorithms tends to be rather hard - multi-threading is hard on its own, doing lock-less multi-threading just pushes this to absurd territory. So let me just address some points:
1) You can't just use new Random() and expect statistically random numbers from that. Random has an internal state that's by default initialized to the current OS tick - that means that creating 10 Randoms in a row and then doing Next on each of those is pretty likely to produce exactly the same "random" numbers.
One way of handling that gracefully would be to have a thread-local field:
ThreadLocal<Random> rnd
= new ThreadLocal<Random>(() => new Random(Guid.NewGuid().GetHashCode()));
Each of your threads can then safely do rnd.Value.Next(...) and get reliable numbers without locking.
However, since the whole point of this excercise is to allow shared access to mutable state, a solution more in line with the task would be to use a single shared Random field instead (created only once, before starting the threads). Since the Bakery algorithm is supposed to make sure you can safely use shared stuff in the critical section, this should be safe, if implemented correctly :)
2) To actually make the Bakery part work, you need to enforce the only proper instruction ordering.
This is hard. Seriously.
I'm not actually sure how to do this safely.
The best way to start is to insert an explicit memory barrier before and after each read and write of shared state. Then you can go one by one and remove those that aren't necessary. Of course, you should only need this in the doLock and unlock methods - the rest of simThread should be single-threaded.
For a short sample:
Thread.MemoryBarrier();
entering[pid] = true;
Thread.MemoryBarrier();
int max = 0;
for (int i = 0; i < threads; i++)
{
if (ticket[i] > ticket[max]) { max = i; }
}
Thread.MemoryBarrier();
ticket[pid] = 1+max;
Thread.MemoryBarrier();
entering[pid] = false;
Thread.MemoryBarrier();
So, which one of those is it safe to remove? I have no idea. I'd have to use a lot of mental power to make sure this is safe. Heck, I'm not sure if it's safe as is - do I need to rewrite the for cycle too? Are ticket[i] and ticket[max] going to be fresh enough for the algorithm to work? I know some are definitely needed, but I'm not sure which can safely be left out.
I'm pretty sure this will be slower than using a simple lock, though. For any production code, steer clear away from code like this - "smart" code usually gets you in trouble, even if everyone in your team understands it well. It's kind of hard finding those kinds of experts, and most of those wouldn't touch lock-less code like that with a meter-long stick :)

You must create a different random number for each thread (more details)
so try this code in your main method:
for (int i = 0; i < 10; i++)
{
int temp = i;
threadArray[i] = new Thread(() => simThread(temp));
Console.WriteLine("[He Thong] PID " + i.ToString() + " duoc khoi tao");
threadArray[i].Start();
Thread.Sleep(20);
}
and the following code in you threads:
Random rand = new Random((int) DateTime.Now.Ticks & 0x0000FFFF);
now you can ensure you produce different random number for each thread.

Try:
Random rnd = new Random(Environment.TickCount / (i + 1));
This will give different seeds to each RNG.

Multithreaded code to do work using configured number of thread

I want to create a multithreaded application code. I want to execute configured no of threads and each thread do the work. I want to know is this the write approach or do we have better approach. All the threads needs to be executed asynchronously.
public static bool keepThreadsAlive = false;
static void Main(string[] args)
{
Program pgm = new Program();
int noOfThreads = 4;
keepThreadsAlive = true;
for (int i = 1; i <= noOfThreads; i++)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(DoWork), (object)i);
}
System.Console.ReadLine();
StopAllThreads();
System.Console.ReadLine();
}
private static void DoWork(object threadNumber)
{
int threadNum = (int)threadNumber;
int counter = 1;
while (keepThreadsAlive)
{
counter = ProcessACK(threadNum, counter);
}
}
private static int ProcessACK(int threadNum, int counter)
{
System.Console.WriteLine("Thread {0} count {1}", threadNum, counter++);
Random ran = new Random();
int randomNumber = ran.Next(5000, 100000);
for (int i = 0; i < randomNumber; i++) ;
Thread.Sleep(2000);
return counter;
}

As others have pointed out, the methods you are using are dated and not as elegant as the more modern C# approach to accomplishing the same tasks.
Have a look at System.Threading.Tasks for an overview of what is available to you these days. There is even a way to set the maximum threads used in a parallel operation. Here is a simple (pseudocode) example:
Parallel.ForEach(someListOfItems, new ParallelOptions { MaxDegreeOfParallelism = 8 }, item =>
{
//do stuff for each item in "someListOfItems" using a maximum of 8 threads.
});
Hope this helps.