Design a thread-safe Hit Counter - c#

I have designed a Hit Counter which can conveniently be used to design a Rate Limiter
Original question: https://leetcode.com/problems/design-hit-counter/
To make it thread safe, I used lock statement.
public class HitCounter {
private readonly int DURATION = 300;
private readonly int[] hits, times;
//https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/lock
private readonly Object _lockObj = new Object();
public HitCounter() {
times = new int[DURATION];
hits = new int[DURATION];
}
public void Hit(int timestamp) {
var idx = timestamp % DURATION;
lock (_lockObj){
if(times[idx] != timestamp){
times[idx] = timestamp;
hits[idx] = 1;
}else{
hits[idx]++;
}
}
}
public int GetHits(int timestamp) {
var res = 0;
lock (_lockObj){
for(var i = 0; i < DURATION; ++i){
if(timestamp - times[i] < DURATION){
res += hits[i];
}
}
}
return res;
}
}
My questions are:
Is it just about surrounding the part which the two arrays are accessed with locks
Isn't it better to use ConcurrentDictionary as an index-array?
Any insight is appreciated.

Related

Adding an ETA to an embedded loop sequence

UPDATE 1:
Some of the solutions offered below seem good. However, I only know the amount of times a loop will iterate after it's parent loop's iterations have been determined. So I can't count all the iterations beforehand.
ORIGINAL QUESTION:
I have embedded loops in a program similar to this:
Prog1:
using System;
using System.Threading;
namespace time_remaining_loop_strip
{
class Program
{
static void Main(string[] args)
{
var random = new Random();
Console.Clear();
// Simulate initiation delay
Console.WriteLine("initiate");
Thread.Sleep(random.Next(100, 1000));
int intCount = random.Next(1, 10);
for (int loop1 = 0; loop1 <= intCount; loop1++)
{
// Simulate loop1 delay
Console.WriteLine("\tloop1");
Thread.Sleep(random.Next(100, 1000));
for (int loop2 = 0; loop2 <= random.Next(1, 10); loop2++)
{
// Simulate loop2 delay
Console.WriteLine("\t\tloop2");
Thread.Sleep(random.Next(100, 1000));
for (int loop3 = 0; loop3 <= random.Next(1, 10); loop3++)
{
// Simulate loop3 delay
Console.WriteLine("\t\t\tloop3");
Thread.Sleep(random.Next(100, 1000));
for (int loop4 = 0; loop4 <= random.Next(1, 10); loop4++)
{
// Simulate loop4 delay
Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(random.Next(100, 1000));
}
}
}
}
}
}
}
I am trying to display Processing Time Remaining (ETA), so I can see a rough estimate of the amount of time remaining before the loop sequence above completes
I now have another bit of code which does display an ETA which works fine when the loop is very simplistic:
Prog2:
using System;
using System.Threading;
namespace time_remaining
{
class Program
{
public static TimeSpan ComputeRemaining((int count, DateTime time) start, (int count, DateTime time) current, int end) =>
current.count - start.count == 0
? TimeSpan.MaxValue
: TimeSpan.FromSeconds((end - current.count) * current.time.Subtract(start.time).TotalSeconds / (current.count - start.count));
static void Main(string[] args)
{
Console.Clear();
var random = new Random();
int Count = random.Next(10, 60);
DateTime startTime = DateTime.Now;
for (int i = 0; i <= Count; i++)
{
Thread.Sleep(random.Next(100, 2000));
TimeSpan timeRemaining = ComputeRemaining((0, startTime), (i, DateTime.Now), Count);
Console.SetCursorPosition(0,0);
Console.Write("ETA: ");
Console.Write(String.Format("{0} Days, {1} Hours, {2} Minutes, {3} Seconds", timeRemaining.Days.ToString().PadLeft(3,'0'), timeRemaining.Hours.ToString().PadLeft(2,'0'), timeRemaining.Minutes.ToString().PadLeft(2,'0'), timeRemaining.Seconds.ToString().PadLeft(2,'0')));
}
}
}
}
When I try to combine the ETA aspect of Prog1 into Prog2, it does not seem to work well:
Prog3 = Prog1+Prog2:
using System;
using System.Threading;
namespace time_remaining_loop_strip
{
class Program
{
public static TimeSpan ComputeRemaining((int count, DateTime time) start, (int count, DateTime time) current, int end) =>
current.count - start.count == 0
? TimeSpan.MaxValue
: TimeSpan.FromSeconds((end - current.count) * current.time.Subtract(start.time).TotalSeconds / (current.count - start.count));
static void Main(string[] args)
{
DateTime startTime = DateTime.Now;
var random = new Random();
Console.Clear();
// Simulate initiation delay
//Console.WriteLine("initiate");
Thread.Sleep(random.Next(100, 1000));
int intCount = random.Next(1, 10);
for (int loop1 = 0; loop1 <= intCount; loop1++)
{
// Simulate loop1 delay
//Console.WriteLine("\tloop1");
Thread.Sleep(random.Next(100, 1000));
for (int loop2 = 0; loop2 <= random.Next(1, 10); loop2++)
{
// Simulate loop2 delay
//Console.WriteLine("\t\tloop2");
Thread.Sleep(random.Next(100, 1000));
for (int loop3 = 0; loop3 <= random.Next(1, 10); loop3++)
{
// Simulate loop3 delay
//Console.WriteLine("\t\t\tloop3");
Thread.Sleep(random.Next(100, 1000));
for (int loop4 = 0; loop4 <= random.Next(1, 10); loop4++)
{
// Simulate loop4 delay
//Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(random.Next(100, 1000));
}
}
}
TimeSpan timeRemaining = ComputeRemaining((0, startTime), (loop1, DateTime.Now), intCount);
Console.SetCursorPosition(0,0);
Console.Write("ETA: ");
Console.Write(String.Format("{0} Days, {1} Hours, {2} Minutes, {3} Seconds", timeRemaining.Days.ToString().PadLeft(3,'0'), timeRemaining.Hours.ToString().PadLeft(2,'0'), timeRemaining.Minutes.ToString().PadLeft(2,'0'), timeRemaining.Seconds.ToString().PadLeft(2,'0')));
}
}
}
}
This doesn't seem to work very well at all. It does display an ETA, but it has long delays before it shows anything, because of the way the loop is structured.
How can I update this so the ETA code displays an ETA more accurately and at more predictive intervals, as in every second?
With the concession that you've just built a simple model of what's actually happening (you have some series of nested process of variable delay and count, which cannot even be determined until runtime), what you're asking for currently is a random number predictor.
There will be between n & m cycles (1 & 10 in your example) of between o & p duration (100 & 1000ms), but still random.. well, random quantized into bands. The way you've written it, this is dice-roll random (a dice has no memory), although in practice it seems more likely the duration of one cycle must somewhat imply the duration of the next, (which is how you've written ComputeRemaining) and within one band the count of one loop must help with the count of the next.
So despite its apparent simplicity Prog2 covers our examples.. given a known loop count, where each cycle takes a random duration (which is in fact pick(n,m)^3*pick(o,p).. but this is still just a random number) - predict the end. For reporting purposes, you'll want to refactor this to consider the inner loops as well too, but it's effectively the same process. (^3 is a simplification, it's actually a series of independent picks multiplied)
Ok, so we don't need to talk about time/delays (I mean.. you clearly want that, but it's just some number that represents the future - a TimeSpan is an long count of x ticks since some time... a finish time is just Now + x*tick). So we can simplify this into a long predictor.
Setup
interface IAvger
{
public double Avg { get; }
}
interface IAdder
{
void Add(long value);
}
class Mean
{
private int _count = 0;
public double Total { get; private set; } = 0;
public double? Avg => _count == 0 ? null : (double?)(Total / _count);
public void Add(double val)
{
Total += val;
_count++;
}
}
You can ignore the interfaces (I used them when switching out potential solutions). Class Mean should be familiar... it calculates the mean average of a number of values, scaling/adapting as more values are found.
/// <summary>
/// Equivalent to your ComputeRemaining
/// </summary>
class RunningAvg : IAvger, IAdder
{
private Mean _mean = new Mean();
private readonly double _guess;
public RunningAvg(double guess)
{
_guess = guess;
}
public double Avg => _mean.Avg ?? _guess;
public void Add(long value) => _mean.Add(value);
}
Here's an equivalent to your ComputeRemaining. The value of guess helps an early prediction when nothing else is known (vs the equivalent of TimeSpan.Max)
/// <summary>
/// Drop the lowest and highest value
/// - Fairly typical in stats, however note this is only one biggest/smallest
/// (will work best when the standard devation is low, and outliers
/// are rare)
/// </summary>
class IgnoreExtremes : IAvger, IAdder
{
private long? _worst;
private long? _best;
private Mean _mean = new Mean();
private readonly int _wt;
private readonly double _guess;
public IgnoreExtremes(double guess, int weight = 4)
{
_wt = weight;
_guess = guess;
}
public long Best => _best ?? (long)Math.Round(_guess);
public long Worst => _worst ?? (long)Math.Round(_guess);
public double Avg
{
get
{
var avg = _mean.Avg;
if (!avg.HasValue) return _guess;
return (Best + _wt * avg.Value + Worst) / (2 + _wt);
}
}
public void Add(long value)
{
if (!_best.HasValue)
{
_best = value;
}
else if (value < _best)
{
_mean.Add(_best.Value);
_best = value;
}
else if (!_worst.HasValue)
{
_worst = value;
}
else if (value > _worst)
{
_mean.Add(_worst.Value);
_worst = value;
}
else
{
_mean.Add(value);
}
}
}
Finally some stats! IgnoreExtremes suppresses the highest and lowest (single) value. In scientific sampling it's fairly typical to ignore these, however with a real random distribution of numbers (eg dice roll, or random.Next) only one extreme will be discarded. This should predict better numbers than RunningAvg. Note this is a form of weighted average, you can tune it (slightly) by supplying a weight value at construction (wt=4 is fairly common), or tying _wt to _mean.count (some code changes required)
class IgnoreStdDevOutlier : IAvger, IAdder
{
private const int AT_LEAST = 5;
private Mean _mean = new Mean();
private readonly List<long> _vals = new List<long>();
private readonly double _guess;
//private long _tot;
private readonly double _outlierStdDevMulti;
public IgnoreStdDevOutlier(double guess, double outlierStdDevMulti = 2)
{
_guess = guess;
_outlierStdDevMulti = outlierStdDevMulti;
}
private double StdDev()
{
var avg = Avg;
double tot = 0;
foreach (var item in _vals)
tot += (item - avg) * (item - avg);
return Math.Sqrt(tot / (_vals.Count - 1));
}
public void Add(long value)
{
_vals.Add(value);
if (_vals.Count > AT_LEAST)
{
var avg = Avg;
var sd = StdDev();
var min = avg - _outlierStdDevMulti * sd;
var max = avg + _outlierStdDevMulti * sd;
//Ignore outliers
if (value < min || value > max) return;
}
_mean.Add(value);
}
public double Avg => _mean.Avg ?? 0;
}
Another statistics approach is to ignore values more than n*StandardDeviation from average, where n is often 2 or 3 (you'll find conflicting opinions). All values seen are part of the standard deviation, but only those that aren't outliers are considered part of the average. This ends up acting like a suppression factor, preventing the estimate swinging too much.
Alright, so to run a test we need some sort of measuring class:
class Performance
{
private readonly List<long> _set = new List<long>();
private long _actual;
public void Add(long item) => _set.Add(item);
public void SetFinal(long final) => _actual = final;
public void Report()
{
foreach (var item in _set)
{
Console.WriteLine("{0} {1} = {2}", item, _actual, (item / (double)_actual - 1) * 100);
}
}
}
A real guess can't know the final (_actual) value, but this class allows us to see how the guess is doing so far.
Finally the program class:
class Program
{
const int MIN_MSEC = 100;
const int MAX_MSEC = 1000;
const int MIN_LOOP = 10;
const int MAX_LOOP = 50;
static void C(Random random)
{
int nm = random.Next(MAX_LOOP, MAX_LOOP);
var guess = (double)((MAX_LOOP + MIN_LOOP) / 2 * (MAX_MSEC + MIN_MSEC) / 2);
var predict = new RunningAvg(guess);
//var predict = new IgnoreExtremes(guess);
//var predict = new IgnoreStdDevOutlier(guess,3);
var per = new Performance();
long tot = 0;
for (int i = 0; i <= nm; i++)
{
var op = random.Next(MIN_MSEC, MAX_MSEC);
predict.Add(op);
per.Add((long)Math.Round(predict.Avg * nm));
tot += op;
}
per.SetFinal(tot);
per.Report();
}
static void Main(string[] args)
{
var random = new Random();
C(random);
}
}
You can ignore the work is done in a method called C (just a side effect of the code - A was your Prog1, while B was Prog2). Within C try changing which of RunningAvg, IgnoreExtremes or IgnoreStdDevOutlier is uncommented. Because, again, what's written is dice-roll random, you cannot take a single run as a good benchmark. The next stage is to wrap this in repeat-runs and take the average of the standard deviation of the predictions (or perhaps only the late predictions - a user probably doesn't mind if an early estimate is far off, as long as by the end it isn't jumping around) - but I ran out of time. I find IgnoreStdDevOutlier, on average, converges on the right answer fairly swiftly being off by 0-1% by the end. IgnoreExtremes suffers from only ignoring one extreme (in each direction), so it's sort of a light version of IgnoreStdDevOutlier. If your data isn't random, and there are only occasionally extreme cases - it'll do just fine. RunningAvg doesn't actually perform terribly some of the time, other times it's off by double digit percentages all the way through. Random numbers, if only they were easy to predict.
Note on usage
Timespan.Ticks is a long. All of this is written to predict a long, which can be considered the difference between then and now. To directly switch use new Timespan(long ticks) to build the spans, and DateTime.Now.Subtract(startTime).Ticks to get a long from a span. Obviously all these classes could be re-written for TimeSpan instead of long.. unfortunately there isn't an easy generic where constraint that includes both long and TimeSpan
You can know how many times(t) loop 4 (L4) will execute multiplying L1tL2tL3t*L4t = total. Now at the very beginning you declare
dt = DateTime.Now;
count = 0L;
Now inside L4 you increase count and calculates how much time is elapsed
et= DateTime.Now - dt;
count++;
And there using rule of three you calculate how many seconds are left to reach the total iterations.
remainingSeconds = (total*et.TotalSeconds/count) - et.TotalSeconds;
Now the ETA is
DateTime.Now.AddSeconds(remainingSeconds);
I used below logic to solve this.
Before first foreach each loop I calculated the all different loops count.
After that I created 4 different variables to hold sleep time for each loop which is settings inside them.
Now before first for loop I calculated the total time in milliseconds that will be taken be all loops using below logic:
var totalTime = (firstLoopCount * firstLoopSleepTime) +
(firstLoopCount * secondLoopCount * secondLoopSleepTime) +
(firstLoopCount * secondLoopCount * thirdLoopCount * thirdLoopSleepTime) +
(firstLoopCount * secondLoopCount * thirdLoopCount * fourthLoopCount * fourthLoopSleepTime);
Note: I added 1 in function call (GetTotoalTimeinMilliSecond) where loops count is passing because loop is starting from 0 and ending with different loops count including itself.
Now before loop start, printed total time taken
Inside each loop, after thread sleep, subtract thread sleep time from total time and print that. And set new calculated time as total time.
Repeat this inside every loop.
Below is the code:
class Program
{
static void Main(string[] args)
{
DateTime startTime = DateTime.Now;
var random = new Random();
Console.Clear();
// Simulate initiation delay
//Console.WriteLine("initiate");
Thread.Sleep(random.Next(100, 1000));
int firstLoopCount = random.Next(1, 10);
int secondLoopCount = random.Next(1, 10);
int thirdLoopCount = random.Next(1, 10);
int fourthLoopCount = random.Next(1, 10);
int firstLoopSleepTime = random.Next(100, 1000);
int secondLoopSleepTime =random.Next(100, 1000);
int thirdLoopSleepTime = random.Next(100, 1000);
int fourthLoopSleepTime = random.Next(100, 1000);
//**************Added 1 because loop is starting from 0 and ending with different loops count including itself.
var totalTimeinMillSec = GetTotoalTimeinMilliSecond(firstLoopCount + 1, secondLoopCount + 1, thirdLoopCount + 1, fourthLoopCount + 1, firstLoopSleepTime, secondLoopSleepTime, thirdLoopSleepTime, fourthLoopSleepTime);
PrintAndGetTimeRemaining(totalTimeinMillSec);
for (int loop1 = 0; loop1 <= firstLoopCount; loop1++)
{
// Simulate loop1 delay
//Console.WriteLine("\tloop1");
Thread.Sleep(firstLoopSleepTime);
totalTimeinMillSec = PrintAndGetTimeRemaining(totalTimeinMillSec - firstLoopSleepTime);
for (int loop2 = 0; loop2 <= secondLoopCount; loop2++)
{
// Simulate loop2 delay
//Console.WriteLine("\t\tloop2");
Thread.Sleep(secondLoopSleepTime);
totalTimeinMillSec = PrintAndGetTimeRemaining(totalTimeinMillSec - secondLoopSleepTime);
for (int loop3 = 0; loop3 <= thirdLoopCount; loop3++)
{
// Simulate loop3 delay
//Console.WriteLine("\t\t\tloop3");
Thread.Sleep(thirdLoopSleepTime);
totalTimeinMillSec = PrintAndGetTimeRemaining(totalTimeinMillSec - thirdLoopSleepTime);
for (int loop4 = 0; loop4 <= fourthLoopCount; loop4++)
{
// Simulate loop4 delay
//Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(fourthLoopSleepTime);
totalTimeinMillSec = PrintAndGetTimeRemaining(totalTimeinMillSec - fourthLoopSleepTime);
}
}
}
}
}
private static int PrintAndGetTimeRemaining(int totalTimeinMillSec)
{
TimeSpan timeRemaining = TimeSpan.FromMilliseconds(totalTimeinMillSec);
Console.SetCursorPosition(0, 0);
Console.WriteLine("ETA: ");
Console.WriteLine(String.Format("{0} Days, {1} Hours, {2} Minutes, {3} Seconds, {4} Milliseconds", timeRemaining.Days.ToString().PadLeft(3, '0'), timeRemaining.Hours.ToString().PadLeft(2, '0'), timeRemaining.Minutes.ToString().PadLeft(2, '0'), timeRemaining.Seconds.ToString().PadLeft(2, '0'), timeRemaining.Milliseconds.ToString().PadLeft(2, '0')));
return totalTimeinMillSec;
}
private static int GetTotoalTimeinMilliSecond(int firstLoopCount, int secondLoopCount, int thirdLoopCount, int fourthLoopCount, int firstLoopSleepTime, int secondLoopSleepTime, int thirdLoopSleepTime, int fourthLoopSleepTime)
{
var totalTime = (firstLoopCount * firstLoopSleepTime +
firstLoopCount * secondLoopCount * secondLoopSleepTime +
firstLoopCount * secondLoopCount * thirdLoopCount * thirdLoopSleepTime +
firstLoopCount * secondLoopCount * thirdLoopCount * fourthLoopCount * fourthLoopSleepTime);
return totalTime;
}
}
}
Please let me know If I missed something.
I think this may work for you; in order to accomplish the solution I created a few classs to help.
The code itself has some comments.
First an enum, to know in which loop we are; it is not fully necessary, but it can be worth for a refactor later on.
public enum LoopEnum
{
loop1,
loop2,
loop3,
loop4
}
Then i created a class called EtaLoop which will contain the entire loop information/logic to know how long a single iteration of a loop takes, I use StopWatch:
public class EtaLoop
{
public readonly int TotalIterations;
private readonly List<long> _loopsTimesInMiliseconds;
private readonly Stopwatch _stopwatch;
public EtaLoop(int totalIterations)
{
//+1 as the loops starts on 0
TotalIterations = totalIterations+1;
_loopsTimesInMiliseconds = new List<long>();
_stopwatch = new Stopwatch();
}
public double AvgExecution()
{
return _loopsTimesInMiliseconds.Any() ? _loopsTimesInMiliseconds.Average(a => a) : 0;
}
public void Start()
{
if(!_stopwatch.IsRunning)
_stopwatch.Start();
}
public void Stop()
{
_stopwatch.Stop();
_loopsTimesInMiliseconds.Add(_stopwatch.ElapsedMilliseconds);
ResetStopWatch();
}
public int CurrentIteration()
{
return _loopsTimesInMiliseconds.Count();
}
public long EstimatedCurrentIteration()
{
return Convert.ToInt64(_loopsTimesInMiliseconds.Average(a => a) * TotalIterations);
}
private void ResetStopWatch()
{
_stopwatch.Reset();
}
}
the methods I think are clear enough without more explanation.
another class called EtaTime which will contain more logic, this class is a wrapper for EtaLoop for example, if a for has 5 iterations, it will contain 5 elements on the list.
each element (EtaLoop) is added to the list, once we finish (all the iterations are finished) It has some more methods, one of them, the "hard one" it is explained
public class EtaTime
{
public readonly List<EtaLoop> Loops;
public readonly LoopEnum Loop;
private EtaLoop _currentLoop;
public EtaTime(LoopEnum loop)
{
Loops = new List<EtaLoop>();
Loop = loop;
}
public void SetUpTotal(int totalIterations)
{
_currentLoop = new EtaLoop(totalIterations);
}
public void StartLoop()
{
_currentLoop.Start();
}
public void EndLoop()
{
_currentLoop.Stop();
}
public void RegisterLoop()
{
Loops.Add(_currentLoop);
}
/// <summary>
/// Get the average time per execution, and the average number of loops per parent loop.
/// The idea is to know how many times (and how long) the loop x is executed per each x-1
/// </summary>
/// <returns></returns>
public (double, double) GetAvgTimeAndAvgExTimes()
{
double avgTimePerLoop = Loops.Any() ? Loops.Average(a => a.AvgExecution()) : _currentLoop.AvgExecution();
double avgTotalLoopsIteration = Loops.Any() ? Loops.Average(a => a.TotalIterations) : _currentLoop.TotalIterations;
return (avgTimePerLoop, avgTotalLoopsIteration);
}
public int GetCurrentIteration()
{
return _currentLoop.CurrentIteration();
}
public int TotalIterations()
{
return _currentLoop.TotalIterations;
}
}
Finally the wrapper for the EtaTimeHelper which will contain all the EtaTimes; originally i was thinking to make a list, thats why the enum was for, but, i think like this is more clear.
Note: this class can be split/moved to extension methods.
The main points here are RegisterLoop every time we finish a loop we have to call that method; and it needs to be here because i need the information from the other loops.
public class EtaTimeHelper
{
//This part can be done in a list, but i think it is easier to see like this.
public readonly EtaTime Loop1;
public readonly EtaTime Loop2;
public readonly EtaTime Loop3;
public readonly EtaTime Loop4;
public readonly DateTime StartTime;
private DateTime _lastPrintTime;
private const int TimeBetweenPrintsInSeconds = 10;
public EtaTimeHelper()
{
Loop1 = new EtaTime(LoopEnum.loop1);
Loop2 = new EtaTime(LoopEnum.loop2);
Loop3 = new EtaTime(LoopEnum.loop3);
Loop4 = new EtaTime(LoopEnum.loop4);
StartTime = DateTime.Now;
_lastPrintTime = DateTime.MinValue;
}
public void RegisterLoop(LoopEnum loopNumber)
{
switch (loopNumber)
{
case LoopEnum.loop1:
Loop1.RegisterLoop();
break;
case LoopEnum.loop2:
Loop2.RegisterLoop();
break;
case LoopEnum.loop3:
Loop3.RegisterLoop();
break;
case LoopEnum.loop4:
Loop4.RegisterLoop();
break;
default:
throw new NotImplementedException("please register the loop");
}
PrintCompletionTime(DateTime.Now, loopNumber);
}
public void PrintCompletionTime(DateTime printTime, LoopEnum loopNumber)
{
if(_lastPrintTime.AddSeconds(TimeBetweenPrintsInSeconds) < printTime)
{
var time = CalculatePredictionTime(loopNumber);
Print(time);
_lastPrintTime = printTime;
}
}
private long CalculatePredictionTime(LoopEnum loopNumber)
{
switch (loopNumber)
{
case LoopEnum.loop1:
return LoopPrediction(Loop1.GetAvgTimeAndAvgExTimes());
case LoopEnum.loop2:
return Loop2Prediction(Loop1, Loop2);
case LoopEnum.loop3:
return Loop3Prediction(Loop1, Loop2, Loop3);
case LoopEnum.loop4:
return Loop4Prediction(Loop1, Loop2, Loop3, Loop4);
default:
throw new NotImplementedException("please register the loop");
}
//If all loops in #1 are finished, all sub loops are also finished. which means, it is the "end of the loop"
long LoopPrediction((double, double) avgTimeAndAvgExTimes)
{
double avgTimePerLoop = avgTimeAndAvgExTimes.Item1;
double avgIterations = avgTimeAndAvgExTimes.Item2;
return Convert.ToInt64(avgTimePerLoop * avgIterations);
}
long Loop2Prediction(EtaTime loop1, EtaTime loop2)
{
var loop1Prediction = LoopPrediction(loop1.GetAvgTimeAndAvgExTimes());
var loop2Values = loop2.GetAvgTimeAndAvgExTimes();
long avgPerLoop = LoopPrediction(loop2Values);
var loop1AvgIterations = loop1.GetAvgTimeAndAvgExTimes().Item2;
var expectedLoop2Iterations = loop1AvgIterations;
double loop2Predictions = avgPerLoop * expectedLoop2Iterations;
if (loop1Prediction == 0)
{
return Convert.ToInt64(loop2Predictions);
}
else
{
//1+current iteration
return loop1Prediction + loop2.GetCurrentIteration();
}
}
long Loop3Prediction(EtaTime loop1, EtaTime loop2, EtaTime loop3)
{
var loop1_2Prediction = Loop2Prediction(loop1, loop2);
var loop3Values = loop3.GetAvgTimeAndAvgExTimes();
long avgPerLoop = LoopPrediction(loop3Values);
var loop2AvgIterations = loop2.GetAvgTimeAndAvgExTimes().Item2;
var loop1AvgIterations = loop1.GetAvgTimeAndAvgExTimes().Item2;
var expectedLoop3Iterations = loop2AvgIterations * loop1AvgIterations;
double loop3Predictions = avgPerLoop * expectedLoop3Iterations;
if (loop1_2Prediction == 0)
{
return Convert.ToInt64(loop3Predictions);
}
else
{
//1-2+current iteration
return loop1_2Prediction+ loop3.GetCurrentIteration();
}
}
long Loop4Prediction(EtaTime loop1, EtaTime loop2, EtaTime loop3, EtaTime loop4)
{
var loop1_2_3Prediction = Loop3Prediction(loop1, loop2, loop3);
var loop4Values = loop4.GetAvgTimeAndAvgExTimes();
long avgPerLoop = LoopPrediction(loop4Values);
var loop2AvgIterations = loop2.GetAvgTimeAndAvgExTimes().Item2;
var loop1AvgIterations = loop1.GetAvgTimeAndAvgExTimes().Item2;
var loop3AvgIterations = loop3.GetAvgTimeAndAvgExTimes().Item2;
var expectedLoop4Iterations = loop2AvgIterations * loop1AvgIterations* loop3AvgIterations;
double loop4Predictions = avgPerLoop * expectedLoop4Iterations;
if (loop1_2_3Prediction == 0)
{
return Convert.ToInt64(loop4Predictions);
}
else
{
//1-2-3+current iteration
return loop1_2_3Prediction + loop4.GetCurrentIteration();
}
}
}
private void Print(long ms)
{
DateTime estimatedCompletionTime = StartTime.AddMilliseconds(ms);
TimeSpan leftTime = (estimatedCompletionTime - DateTime.Now);
Console.WriteLine("ETA: ");
Console.WriteLine($"{leftTime.Days} Days, {leftTime.Hours} Hours, {leftTime.Minutes} Minutes, {leftTime.Seconds} Seconds");//, leftTime.Days.ToString().PadLeft(3, '0'), leftTime.Hours.ToString().PadLeft(2, '0'), leftTime.Minutes.ToString().PadLeft(2, '0'), leftTime.Seconds.ToString().PadLeft(2, '0')));
Console.WriteLine($"on {estimatedCompletionTime.ToString("yyyy/MM/dd HH:mm:ss")}");
Console.WriteLine($"Current Time: {DateTime.Now.ToString("yyyy/MM/dd HH:mm:ss")}");
}
}
and the print. as you mention you want some mecanisim to print, but you did not specify if it is a consol app a web or a winforms app. Probably it is winforms if it is a desktop app you probably can do something like myLabel.SetPropertyThreadSafe to modify the label you want to update.
The algorithim which calculates the time it is on CalculatePredictionTime I tried to make it a single method for n number of loops but I couldnt.
Now the main.
I did a few changes on the Thread.Sleep as it is easier for me to figure it out if makes sense while i was testing. also, a range 10-100 (10x) I thought it was very big, i reduced it for testing reasons.
the code follows a pattern, first you need to instanciate the EtaTimeHelper class.
before each loop you have to setUpTotal which means you have to pass the number of iterations are going to be.
The first line of the loop will be .StartLoop() this will start the StopWatch and the last line of the loop (before the }) will be a .EndLoop();
after the closing bracket } you have to indicate the etaTimeHelper.RegisterLoop(loopNumber); passing as parameter the loop number this will print if necessary the time.
static void Main(string[] args)
{
var random = new Random();
Console.Clear();
// Simulate initiation delay
//Console.WriteLine("initiate");
EtaTimeHelper etaTimeHelper = new EtaTimeHelper();
int intCount = random.Next(1, 10);
etaTimeHelper.Loop1.SetUpTotal(intCount);
for (int loop1 = 0; loop1 <= intCount; loop1++)
{
etaTimeHelper.Loop1.StartLoop();
// Simulate loop1 delay
Console.WriteLine("\tloop1");
Thread.Sleep(random.Next(40, 50));
//number of times the loop 2 is going to execute inside this loop 1;
int loop2times = random.Next(1, 10);
etaTimeHelper.Loop2.SetUpTotal(loop2times);
for (int loop2 = 0; loop2 <= loop2times; loop2++)
{
etaTimeHelper.Loop2.StartLoop();
// Simulate loop2 delay
//Console.WriteLine("\t\tloop2");
Thread.Sleep(random.Next(30, 40));
//number of times the loop 3 is going to execute inside this loop 3;
int loop3times = random.Next(1, 10);
etaTimeHelper.Loop3.SetUpTotal(loop3times);
for (int loop3 = 0; loop3 <= loop3times; loop3++)
{
etaTimeHelper.Loop3.StartLoop();
// Simulate loop3 delay
//Console.WriteLine("\t\t\tloop3");
Thread.Sleep(random.Next(10, 20));
var loop4Times = random.Next(1, 10);
etaTimeHelper.Loop4.SetUpTotal(loop4Times);
for (int loop4 = 0; loop4 <= loop4Times; loop4++)
{
etaTimeHelper.Loop4.StartLoop();
// Simulate loop4 delay
//Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(random.Next(20, 30));
etaTimeHelper.Loop4.EndLoop();
}
etaTimeHelper.RegisterLoop(LoopEnum.loop4);
etaTimeHelper.Loop3.EndLoop();
}
etaTimeHelper.RegisterLoop(LoopEnum.loop3);
etaTimeHelper.Loop2.EndLoop();
}
etaTimeHelper.RegisterLoop(LoopEnum.loop2);
etaTimeHelper.Loop1.EndLoop();
}
etaTimeHelper.RegisterLoop(LoopEnum.loop1);
}
Here is a "working" fiddle https://dotnetfiddle.net/Z06W4g
note: it times out after a few seconds, in your local machine works fine.
I think you will have to benchmark it a few times for the operation that you need and then extrapolate. There are variables here. The compute power and storage type (if involved) and network (if involved) of the target will all contribute to the ETA and every execution will have a different execution time. But ETA can be closely predicted after some benchmarking. And if you want to be really clever, you can show a warning that it may take more time than the displayed ETA because of x, y and z e.g. CPU utilisation by other processes running on the target.
Jon Skeet is an authority on this topic and he has a good resource here:
link
Also read here for how the types in your operation and bitness of the CPU will affect your ETA:
link
In order to assert the ETA every second , what I have done is attaching an event to a System.Timers.Timer object. The event will be triggered every second in order to continuously providing the user with the ETA feedback.
System.Timers.Timer aTimer = new System.Timers.Timer(1000);
aTimer.Elapsed += (sender, e) => ATimer_Elapsed(sender, e, new MyEventArguments
{
Loop1TotalIterations = intCount,
CurrentIndex = loop1,
Start = startTime
});
aTimer.AutoReset = true;
aTimer.Enabled = true;
In your example we are using the Thread.Sleep in order simulate a delay.
Using Thread.Sleep will put everything to sleep - therefor the time logic will be inaccurate.
What I have done is creating a Multi-thread application - in order to run the logic on one thread and the time on another. Therefor when we pause the time checking and ETA will continue to run.
Putting it all together we have the following:
namespace ConsoleApp10
{
#region Usings
using System;
using System.Threading;
using System.Threading.Tasks;
using System.Timers;
#endregion
/// <summary>
/// Event class to be passed to ElapsedEventHandler
/// </summary>
public class MyEventArguments
{
public int Loop1TotalIterations { get; set; }
public DateTime Start { get; set; }
public DateTime CurrentDate => DateTime.Now;
public int CurrentIndex { get; set; }
}
class Program
{
/// <summary>
/// Compute the remaining time
/// </summary>
public static void ComputeRemaining((int count, DateTime time) start, (int count, DateTime time) current, int end)
{
var T = current.count - start.count == 0
? TimeSpan.MaxValue
: TimeSpan.FromSeconds((end - current.count) * current.time.Subtract(start.time).TotalSeconds / (current.count - start.count));
Console.Clear();
Console.SetCursorPosition(0, 0);
Console.ForegroundColor = (ConsoleColor)15;
Console.WriteLine(String.Format("ETA: {0} Days, {1} Hours, {2} Minutes, {3} Seconds",
T.Days.ToString().PadLeft(3, '0'),
T.Hours.ToString().PadLeft(2, '0'),
T.Minutes.ToString().PadLeft(2, '0'),
T.Seconds.ToString().PadLeft(2, '0')));
Console.WriteLine();
}
static void Main(string[] args)
{
DateTime startTime = DateTime.Now;
Random random = new Random();
int intCount = random.Next(1, 10);
int loop1 = 0;
var time = Task.Factory.StartNew(() =>
{
System.Timers.Timer aTimer = new System.Timers.Timer(1000);
aTimer.Elapsed += (sender, e) => ATimer_Elapsed(sender, e, new MyEventArguments
{
Loop1TotalIterations = intCount,
CurrentIndex = loop1,
Start = startTime
});
aTimer.AutoReset = true;
aTimer.Enabled = true;
});
var logic = Task.Factory.StartNew(() =>
{
PrintAndSimulate("Initiate");
for (loop1 = 0; loop1 <= intCount; loop1++)
{
PrintAndSimulate("\tloop1");
for (int loop2 = 0; loop2 <= random.Next(1, 10); loop2++)
{
PrintAndSimulate("\t\tloop2");
for (int loop3 = 0; loop3 <= random.Next(1, 10); loop3++)
{
PrintAndSimulate("\t\t\tloop3");
for (int loop4 = 0; loop4 <= random.Next(1, 10); loop4++)
{
PrintAndSimulate("\t\t\t\tloop4");
}
}
}
}
});
Task.WaitAll(new[] { time, logic });
}
/// <summary>
/// Display the info string and simulates a delay
/// </summary>
private static void PrintAndSimulate(string info)
{
int time = new Random().Next(100, 1000);
Console.SetCursorPosition(5, 5);
Console.ForegroundColor = (ConsoleColor)new Random().Next(15);
Console.WriteLine(info);
Thread.Sleep(time);
}
/// <summary>
/// ElapsedEventHandler
/// </summary>
private static void ATimer_Elapsed(object sender, ElapsedEventArgs e, MyEventArguments myEventArguments)
{
ComputeRemaining((0, myEventArguments.Start), (myEventArguments.CurrentIndex, myEventArguments.CurrentDate), myEventArguments.Loop1TotalIterations);
}
}
}
I have added some colours to the console feedback - making a change more evident.
This may not be ideal solution but has potential to solve the issue.
First you build a Tree of Action containing what you are going to execute and then execute the Actions by iterating the tree.
The tree acts as index of what you are going to execute. Even you can track what is being executed. Whats done and whats pending.
I have nested the actions in single level in the tree for demo purpose but you can nest the tree as much as possible, inside loops etc.(pseudo code ahead but poc builds and executes well)
class Program
{
static void Main(string[] args)
{
var random = new Random();
TreeNode<Action> root = new TreeNode<Action>(() => { });
var loop1 = root.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop2 = loop1.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop3 = loop2.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop4 = loop3.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop5 = loop4.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
var loop6 = loop5.AddChild(() =>
{
int Count = random.Next(1, 3);
for (int i = 0; i <= Count; i++)
Thread.Sleep(random.Next(100, 2000));
});
root.Execute(DateTime.Now);
Console.ReadLine();
}
}
public static class Extensions
{
public static string Humanize(this TimeSpan timeRemaining)
{
return String.Format("{0} Days, {1} Hours, {2} Minutes, {3} Seconds", timeRemaining.Days.ToString().PadLeft(3, '0'), timeRemaining.Hours.ToString().PadLeft(2, '0'), timeRemaining.Minutes.ToString().PadLeft(2, '0'), timeRemaining.Seconds.ToString().PadLeft(2, '0'));
}
public static void PrintETA(this TimeSpan timeRemaining)
{
//Console.SetCursorPosition(0, 0);
Console.Write("ETA: ");
Console.Write(timeRemaining.Humanize());
}
public static TimeSpan ComputeRemaining((int count, DateTime time) start, (int count, DateTime time) current, int end) =>
current.count - start.count == 0
? TimeSpan.MaxValue
: TimeSpan.FromSeconds((end - current.count) * current.time.Subtract(start.time).TotalSeconds / (current.count - start.count));
public static void Execute(this TreeNode<Action> root, DateTime startTime)
{
var current = root;
var end = current.Count();
var currentCount = 1;
Iterate:
var Count = current.Children.Count();
for (int i = 0; i < Count; i++)
{
TreeNode<Action> node = current.Children.ElementAt(i);
node.Data();
TimeSpan timeRemaining = ComputeRemaining((0, startTime), (currentCount++, DateTime.Now), end-1);
timeRemaining.PrintETA();
Console.WriteLine();
Console.WriteLine("Processed {0} of {1}", currentCount - 1, end-1);
if (node.Children.Count() > 0)
{
current = node;
goto Iterate;
}
}
}
}
Ref:
TreeNode.cs
Yet another tree structure: https://github.com/gt4dev/yet-another-tree-structure
Updated.
The first version of the answer calculated the values of RemainingTime and TotalTime at the end of every iteration. Given the way the for loops are nested, it could cause long delays between updates. In order to read those values at given intervals, some changes are made.
Let's start with the Loop class. It is used to keep track of the details of every for loop, like the total amount of iterations, the current iteration and the time consumed in every iteration. To acomplish the later, two System.Diagnostic Stopwatch are used. One is kept running free, not reseted, to ease the calculation of the average time for an iteration. The other clock is reseted on every iteration to provide a value for LoopTime, used in calculations on the fly of RemainingTime and TotalTime, when accesed via the property getters. When the iteration ends, signaled by the method StopClock(), the average loop time and the related properties are updated. The values obtained here are more accurate than the ones calculated on the fly.
Its parent, LoopTimer class, is in charge of creating and storing references of Loop instances, start and stop clocks and calculate the global ETA. The method EnterLoop() is used at the beginning of the for loop. It creates a new Loop. The overload EnterLoop() with a single parameter is used in the rest of iterations to retrieve Loop instances stored in a Dictionary. The method ExitLoop(), at the end of the loop, stops the clock and updates calculations.
Functions ExitLoopRetETA() and ExitLoopRetTotalEta() are provided as replacements of ExitLoop(), to print data at the end of the loop, for testing or debugging. Method Bonus() shows how to use it. Method ShowStatus() and function GetStatus provide internal information on the Loop objects.
To show the values periodically, a Task is used, to run DoUpdate() (or DoUpdateTotal() or DoUpdateStatus())in a separate thread.
The target framework is .Net 4.0
The working classes:
using System;
using System.Collections.Generic;
using System.Diagnostics;
namespace LoopTimer
{
public struct TotalEta
{
public TimeSpan Eta;
public TimeSpan Total;
}
internal class LoopTimer
{
// Private helper class
private class Loop
{
// Declarations
private Stopwatch _clock;
private Stopwatch _interval_clock;
// Constructor
public Loop(int index, int count)
{
Index = index;
Count = count;
_clock = new Stopwatch();
_interval_clock = new Stopwatch();
}
// Properties
internal int Index { get; set; }
internal int Count { get; private set; }
private double _loopTimeMs;
internal double LoopTimeMs
{
get
{
if (_clock.IsRunning)
{
return _interval_clock.Elapsed.TotalMilliseconds;
}
else
return _loopTimeMs; ;
}
}
private double _remainingTimeMs;
internal double RemainingTimeMs
{
get
{
if (_clock.IsRunning)
return CalculateRemainingTime();
else
return _remainingTimeMs;
}
}
private double _totalTimeMs;
internal double TotalTimeMs
{
get
{
if (_clock.IsRunning)
return CalculateTotalTime();
else
return _totalTimeMs;
}
}
internal TimeSpan LoopTime
{
get { return TimeSpan.FromMilliseconds(LoopTimeMs); }
}
internal TimeSpan TotalTime
{
get { return TimeSpan.FromMilliseconds(TotalTimeMs); }
}
internal TimeSpan RemainingTime
{
get { return TimeSpan.FromMilliseconds(RemainingTimeMs); }
}
// Methods
internal void StartClock()
{
_clock.Start();
_interval_clock.Start();
}
internal void StopClock()
{
_clock.Stop();
_interval_clock.Stop();
UpdateTimes();
_interval_clock.Reset();
}
private void UpdateTimes()
{
// reading clock
double elapsed = _clock.Elapsed.TotalMilliseconds;
// Calculating average loop time. The Stopwatch is not reseted between iterations.
_loopTimeMs = elapsed / (Index + 1);
// Calculating estimated remaining time = average loop time * remaining iterations.
_remainingTimeMs = CalculateRemainingTime();
// Calculating estimated total time = average loop time * iterations.
_totalTimeMs = CalculateTotalTime();
}
private double CalculateRemainingTime()
{
// Calculating estimated remaining time = average loop time * remaining iterations.
double time;
int countt = Count - Index;
if (countt > 1)
time = LoopTimeMs * countt;
else if (countt == 1)
time = LoopTimeMs;
else
time = 0;
return time;
}
private double CalculateTotalTime()
{
return LoopTimeMs * Count;
}
}
// End Private helper class
// Declarations
private Dictionary<int, Loop> _loopDict;
private int _loopIndex;
// Constructor
public LoopTimer()
{
_loopDict = new Dictionary<int, Loop>();
_loopIndex = -1;
}
// Properties
internal TimeSpan TotalTime
{
get { return TimeSpan.FromMilliseconds(TotalTimeMs); }
}
internal TimeSpan RemainingTime
{
get { return TimeSpan.FromMilliseconds(RemainingTimeMs); }
}
private double TotalTimeMs
{ get { return CalculateTotalTime(); } }
private double RemainingTimeMs
{ get { return CalculateRemainingTime(); } }
// Methods
internal void EnterLoop(int index, int count)
{
Loop newLoop;
// increase index
_loopIndex++;
if (!_loopDict.ContainsKey(_loopIndex))
{
// create new Loop
newLoop = new Loop(index, count);
_loopDict[_loopIndex] = newLoop;
}
else
{ // retrieve Loop from Dict
newLoop = _loopDict[_loopIndex];
}
newLoop.StartClock();
}
internal void EnterLoop(int index)
{
// increase index
_loopIndex++;
// retrive loop & start clock
_loopDict[_loopIndex].Index = index;
_loopDict[_loopIndex].StartClock();
}
internal void ExitLoop()
{ // retrive loop & stop clock
_loopDict[_loopIndex].StopClock();
// decrease index
_loopIndex--;
}
// bonus method
internal TimeSpan ExitLoopRetETA()
{ // retrive loop & stop clock
_loopDict[_loopIndex].StopClock();
// decrease index
_loopIndex--;
return RemainingTime;
}
// bonus method
internal TotalEta ExitLoopRetTotalEta()
{
TotalEta retval;
retval.Eta = ExitLoopRetETA();
retval.Total = TotalTime;
return retval;
}
// debug method
internal void ShowStatus()
{
Console.WriteLine("Status:");
Console.WriteLine(" RemainingTime:");
for (int i = 0; i < _loopDict.Count; i++)
{
TimeSpan time = _loopDict[i].RemainingTime;
Console.WriteLine(string.Format(" Loop: {0} Value: {1}", i, time.ToString()));
}
Console.WriteLine();
}
// debug method
internal TotalEta[] GetStatus()
{
TotalEta[] retArr = new TotalEta[_loopDict.Count];
TotalEta retval;
for (int i = 0; i < _loopDict.Count; i++)
{
retval = new TotalEta();
retval.Eta = _loopDict[i].RemainingTime;
retval.Total = _loopDict[i].TotalTime;
retArr[i] = retval;
}
return retArr;
}
private double CalculateRemainingTime()
{
double max, time;
max = 0;
// Remaining Time, the greater of all
for (int i = 0; i < _loopDict.Count; i++)
{
time = _loopDict[i].RemainingTimeMs;
if (time > max)
max = time;
}
return max;
}
// Total Time, bonus
private double CalculateTotalTime()
{
double max, time;
max = 0;
// Total Time, the greater of all
for (int i = 0; i < _loopDict.Count; i++)
{
time = _loopDict[i].TotalTimeMs;
if (time > max)
max = time;
}
return max;
}
}
}
The sample program:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace LoopTimer
{
class Solution
{
static CancellationTokenSource ts;
static void Main(string[] args)
{
Console.Clear();
LoopTimer lm = new LoopTimer();
var random = new Random();
// For easy change test parameters
int minRndCount = 1;
int maxRndCount = 10;
int minRndSleep = 100;
int maxRndSleep = 1000;
// A task to update console, with cancellation token
ts = new CancellationTokenSource();
Task updater = new Task(() => DoUpdate(lm), ts.Token);
// Uncomment to show estimated total time.
//Task updater = new Task(() => DoUpdateTotal(lm), ts.Token);
// Uncomment to show estimated total time and internal values of every loop.
//Task updater = new Task(() => DoUpdateStatus(lm), ts.Token);
// Simulate initiation delay
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
// Console.WriteLine("initiate");
updater.Start();
int intCountL1 = random.Next(minRndCount, maxRndCount);
for (int loop1 = 0; loop1 <= intCountL1; loop1++)
{
// Entering loop1
if (loop1 == 0)
lm.EnterLoop(loop1, intCountL1);
else
lm.EnterLoop(loop1);
// Simulate loop1 delay
//Console.WriteLine("\tloop1");
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
int intCountL2 = random.Next(minRndCount, maxRndCount);
for (int loop2 = 0; loop2 <= intCountL2; loop2++)
{
// Entering loop2
if (loop2 == 0)
lm.EnterLoop(loop2, intCountL2);
else
lm.EnterLoop(loop2);
// Simulate loop2 delay
//Console.WriteLine("\t\tloop2");
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
int intCountL3 = random.Next(minRndCount, maxRndCount);
for (int loop3 = 0; loop3 <= intCountL3; loop3++)
{
// Entering loop3
if (loop3 == 0)
lm.EnterLoop(loop3, intCountL3);
else
lm.EnterLoop(loop3);
// Simulate loop3 delay
//Console.WriteLine("\t\t\tloop3");
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
int intCountL4 = random.Next(minRndCount, maxRndCount);
for (int loop4 = 0; loop4 <= intCountL4; loop4++)
{
// Entering loop4
if (loop4 == 0)
lm.EnterLoop(loop4, intCountL4);
else
lm.EnterLoop(loop4);
// Simulate loop4 delay
//Console.WriteLine("\t\t\t\tloop4");
Thread.Sleep(random.Next(minRndSleep, maxRndSleep));
// Exiting loop4
lm.ExitLoop();
}
// Exiting loop3
lm.ExitLoop();
}
// Exiting loop2
lm.ExitLoop();
}
// Exiting loop1
lm.ExitLoop();
}
ts.Cancel();
}
static private void DoUpdate(LoopTimer lm)
{
char[] animchar = { '|', '/', '-', '\\' };
int index = 0;
Thread.Sleep(100);
while (true)
{
TimeSpan eta = lm.RemainingTime;
Console.SetCursorPosition(0, 0);
Console.Write(string.Format(" {4} ETA: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", eta.Days.ToString().PadLeft(3, '0'), eta.Hours.ToString().PadLeft(2, '0'), eta.Minutes.ToString().PadLeft(2, '0'), eta.Seconds.ToString().PadLeft(2, '0'), animchar[index].ToString()));
if (++index > 3)
index = 0;
Thread.Sleep(1000);
ts.Token.ThrowIfCancellationRequested();
}
}
/*
This method is provided as a sample on displaying the estimated total time.
*/
static private void DoUpdateTotal(LoopTimer lm)
{
char[] animchar = { '|', '/', '-', '\\' };
int index = 0;
Thread.Sleep(100);
while (true)
{
TimeSpan eta = lm.RemainingTime;
TimeSpan total = lm.TotalTime;
Console.SetCursorPosition(0, 0);
Console.Write(string.Format(" {4} ETA: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", eta.Days.ToString().PadLeft(3, '0'), eta.Hours.ToString().PadLeft(2, '0'), eta.Minutes.ToString().PadLeft(2, '0'), eta.Seconds.ToString().PadLeft(2, '0'), animchar[index].ToString()));
Console.Write(string.Format("\n Total: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", total.Days.ToString().PadLeft(3, '0'), total.Hours.ToString().PadLeft(2, '0'), total.Minutes.ToString().PadLeft(2, '0'), total.Seconds.ToString().PadLeft(2, '0')));
if (++index > 3)
index = 0;
Thread.Sleep(1000);
ts.Token.ThrowIfCancellationRequested();
}
}
/*
This method is provided as a sample on displaying the estimated total time, and
the internal values of every loop.
*/
static private void DoUpdateStatus(LoopTimer lm)
{
char[] animchar = { '|', '/', '-', '\\' };
int index = 0;
Thread.Sleep(100);
while (true)
{
TimeSpan eta = lm.RemainingTime;
TimeSpan total = lm.TotalTime;
TotalEta[] status = lm.GetStatus();
Console.SetCursorPosition(0, 0);
Console.Write(string.Format(" {4} ETA: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", eta.Days.ToString().PadLeft(3, '0'), eta.Hours.ToString().PadLeft(2, '0'), eta.Minutes.ToString().PadLeft(2, '0'), eta.Seconds.ToString().PadLeft(2, '0'), animchar[index].ToString()));
Console.Write(string.Format("\n Total: {0} Days, {1} Hours, {2} Minutes, {3} Seconds", total.Days.ToString().PadLeft(3, '0'), total.Hours.ToString().PadLeft(2, '0'), total.Minutes.ToString().PadLeft(2, '0'), total.Seconds.ToString().PadLeft(2, '0')));
Console.WriteLine();
int loop = 0;
foreach (var item in status)
{
Console.Write(string.Format("\n Loop: {0} ETA: {1} \tTotal: {2}", loop, item.Eta.ToString(#"hh\:mm\:ss\.FFFF"), item.Total.ToString(#"hh\:mm\:ss\.FFFF")));
loop++;
}
if (++index > 3)
index = 0;
Thread.Sleep(1000);
ts.Token.ThrowIfCancellationRequested();
}
}
/*
This method is provided as a sample for variations on
the ExitLoopRet method. Uses in-place calls.
*/
static internal void Bonus()
{
TotalEta remVal;
TimeSpan remTime;
LoopTimer lm = new LoopTimer();
Console.Clear();
// easy change test parameters
var random = new Random();
int minRndCount = 1;
int maxRndCount = 5;
int maxRndSleep = 1000;
// First, outer loop
int intCountL1 = random.Next(minRndCount, maxRndCount);
for (int i = 0; i < intCountL1; i++)
{
if (i == 0)
lm.EnterLoop(i, intCountL1);
else
lm.EnterLoop(i);
Console.WriteLine(string.Format("\nLoop1 begin iteration: {0} of {1}. Will work(sleep) for {2} ms.", i, intCountL1 - 1, maxRndSleep));
Thread.Sleep(maxRndSleep);
// Second, middle loop
int intCountL2 = random.Next(minRndCount, maxRndCount);
for (int j = 0; j < intCountL2; j++)
{
if (j == 0)
lm.EnterLoop(j, intCountL2);
else
lm.EnterLoop(j);
Console.WriteLine(string.Format("\n\tLoop2 begin iteration: {0} of {1}. Will work(sleep) for {2} ms.", j, intCountL2 - 1, maxRndSleep));
Thread.Sleep(maxRndSleep);
// Third, inner loop
int intCountL3 = random.Next(minRndCount, maxRndCount);
for (int k = 0; k < intCountL3; k++)
{
if (k == 0)
lm.EnterLoop(k, intCountL3);
else
lm.EnterLoop(k);
Console.WriteLine(string.Format("\n\t\tLoop3 begin iteration: {0} of {1}. Will work(sleep) for {2} ms.", k, intCountL3 - 1, maxRndSleep));
Thread.Sleep(maxRndSleep);
lm.ExitLoop();
Console.WriteLine(string.Format("\n\t\tLoop3 end iteration: {0} of {1}", k, intCountL3 - 1));
lm.ShowStatus();
}
remTime = lm.ExitLoopRetETA();
Console.WriteLine(string.Format("\n\tLoop2 end iteration: {0} of {1}", j, intCountL2 - 1));
Console.WriteLine("\t\tRem: " + remTime.ToString());
}
remVal = lm.ExitLoopRetTotalEta();
Console.WriteLine(string.Format("\nLoop1 end iteration: {0} of {1}", i, intCountL1 - 1));
Console.WriteLine("\t\tTot: " + remVal.Total.ToString());
Console.WriteLine("\t\tRem: " + remVal.Eta.ToString());
}
}
}
}

c# 1D-byte array to 2D-double array

I'm dealing with c# concurrent-queue and multi-threading in socket-programming tcp/ip
First, I've already done with socket-programming itself. That means, I've already finished coding about client, server and stuffs about communication itself
basic structure is pipe-lined(producer-consumer problem) and now I'm doing with bit conversion
below is brief summary about my code
client-socket ->server-socket -> concurrent_queue_1(with type byte[65536],Thread_1 process this) -> concurrent_queue_2(with type double[40,3500], Thread_2 process this) -> display-data or other work(It can be gpu-work)
*(double[40,3500] can be changed to other size)
Till now,I've implemented putting_data into queue1(Thread1) and just dequeuing all(Thread2) and, its speed is about 700Mbps
The reason I used two concurrent_queue is, I want communication,and type conversion work to be processed in background regardless of main procedure about control things.
Here is the code about my own concurrent_queue with Blocking
public class BlockingConcurrentQueue<T> : IDisposable
{
private readonly ConcurrentQueue<T> _internalQueue;
private AutoResetEvent _autoResetEvent;
private long _consumed;
private long _isAddingCompleted = 0;
private long _produced;
private long _sleeping;
public BlockingConcurrentQueue()
{
_internalQueue = new ConcurrentQueue<T>();
_produced = 0;
_consumed = 0;
_sleeping = 0;
_autoResetEvent = new AutoResetEvent(false);
}
public bool IsAddingCompleted
{
get
{
return Interlocked.Read(ref _isAddingCompleted) == 1;
}
}
public bool IsCompleted
{
get
{
if (Interlocked.Read(ref _isAddingCompleted) == 1 && _internalQueue.IsEmpty)
return true;
else
return false;
}
}
public void CompleteAdding()
{
Interlocked.Exchange(ref _isAddingCompleted, 1);
}
public void Dispose()
{
_autoResetEvent.Dispose();
}
public void Enqueue(T item)
{
_internalQueue.Enqueue(item);
if (Interlocked.Read(ref _isAddingCompleted) == 1)
throw new InvalidOperationException("Adding Completed.");
Interlocked.Increment(ref _produced);
if (Interlocked.Read(ref _sleeping) == 1)
{
Interlocked.Exchange(ref _sleeping, 0);
_autoResetEvent.Set();
}
}
public bool TryDequeue(out T result)
{
if (Interlocked.Read(ref _consumed) == Interlocked.Read(ref _produced))
{
Interlocked.Exchange(ref _sleeping, 1);
_autoResetEvent.WaitOne();
}
if (_internalQueue.TryDequeue(out result))
{
Interlocked.Increment(ref _consumed);
return true;
}
return false;
}
}
My question is here
As I mentioned above, concurrent_queue1's type is byte[65536] and 65536 bytes = 8192 double data.
(40 * 3500=8192 * 17.08984375)
I want merge multiple 8192 double data into form of double[40,3500](size can be changed)and enqueue to concurrent_queue2 with Thread2
It's easy to do it with naive-approach(using many complex for loop) but it's slow cuz, It copys all the
data and expose to upper class or layer.
I'm searching method automatically enqueuing with matched size like foreach loop automatically iterates through 2D-array in row-major way, not yet found
Is there any fast way to merge 1D-byte array into form of 2D-double array and enqueue it?
Thanks for your help!
I try to understand your conversion rule, so I write this conversion code. Use Parallel to speed up the calculation.
int maxSize = 65536;
byte[] dim1Array = new byte[maxSize];
for (int i = 0; i < maxSize; ++i)
{
dim1Array[i] = byte.Parse((i % 256).ToString());
}
int dim2Row = 40;
int dim2Column = 3500;
int byteToDoubleRatio = 8;
int toDoubleSize = maxSize / byteToDoubleRatio;
double[,] dim2Array = new double[dim2Row, dim2Column];
Parallel.For(0, toDoubleSize, i =>
{
int row = i / dim2Column;
int col = i % dim2Column;
int originByteIndex = row * dim2Column * byteToDoubleRatio + col * byteToDoubleRatio;
dim2Array[row, col] = BitConverter.ToDouble(
dim1Array,
originByteIndex);
});

Task process ending before finishing all the work

I've been having trouble running multiple tasks with heavy operations.
It seems as if the task processes is killed before all the operations are complete.
The code here is an example code I used to replicate the issue. If I add something like Debug.Write(), the added wait for writing fixes the issue. The issue is gone if I test on a smaller sample size too. The reason there is a class in the example below is to create complexity for the test.
The real case where I encountered the issue first is too complicated to explain for a post here.
public static class StaticRandom
{
static int seed = Environment.TickCount;
static readonly ThreadLocal<Random> random =
new ThreadLocal<Random>(() => new Random(Interlocked.Increment(ref seed)));
public static int Next()
{
return random.Value.Next();
}
public static int Next(int maxValue)
{
return random.Value.Next(maxValue);
}
public static double NextDouble()
{
return random.Value.NextDouble();
}
}
// this is the test function I run to recreate the problem:
static void tasktest()
{
var testlist = new List<ExampleClass>();
for (var index = 0; index < 10000; ++index)
{
var newClass = new ExampleClass();
newClass.Populate(Enumerable.Range(0, 1000).ToList());
testlist.Add(newClass);
}
var anotherClassList = new List<ExampleClass>();
var threadNumber = 5;
if (threadNumber > testlist.Count)
{
threadNumber = testlist.Count;
}
var taskList = new List<Task>();
var tokenSource = new CancellationTokenSource();
CancellationToken cancellationToken = tokenSource.Token;
int stuffPerThread = testlist.Count / threadNumber;
var stuffCounter = 0;
for (var count = 1; count <= threadNumber; ++count)
{
var toSkip = stuffCounter;
var threadWorkLoad = stuffPerThread;
var currentIndex = count;
// these ifs make sure all the indexes are covered
if (stuffCounter + threadWorkLoad > testlist.Count)
{
threadWorkLoad = testlist.Count - stuffCounter;
}
else if (count == threadNumber && stuffCounter + threadWorkLoad < testlist.Count)
{
threadWorkLoad = testlist.Count - stuffCounter;
}
taskList.Add(Task.Factory.StartNew(() => taskfunc(testlist, anotherClassList, toSkip, threadWorkLoad),
cancellationToken, TaskCreationOptions.None, TaskScheduler.Default));
stuffCounter += stuffPerThread;
}
Task.WaitAll(taskList.ToArray());
}
public class ExampleClass
{
public ExampleClassInner[] Inners { get; set; }
public ExampleClass()
{
Inners = new ExampleClassInner[5];
for (var index = 0; index < Inners.Length; ++index)
{
Inners[index] = new ExampleClassInner();
}
}
public void Populate(List<int> intlist) {/*adds random ints to the inner class*/}
public ExampleClass(ExampleClass copyFrom)
{
Inners = new ExampleClassInner[5];
for (var index = 0; index < Inners.Length; ++index)
{
Inners[index] = new ExampleClassInner(copyFrom.Inners[index]);
}
}
public class ExampleClassInner
{
public bool SomeBool { get; set; } = false;
public int SomeInt { get; set; } = -1;
public ExampleClassInner()
{
}
public ExampleClassInner(ExampleClassInner copyFrom)
{
SomeBool = copyFrom.SomeBool;
SomeInt = copyFrom.SomeInt;
}
}
}
static int expensivefunc(int theint)
{
/*a lot of pointless arithmetic and loops done only on primitives and with primitives,
just to increase the complexity*/
theint *= theint + 1;
var anotherlist = Enumerable.Range(0, 10000).ToList();
for (var index = 0; index < anotherlist.Count; ++index)
{
theint += index;
if (theint % 5 == 0)
{
theint *= index / 2;
}
}
var yetanotherlist = Enumerable.Range(0, 50000).ToList();
for (var index = 0; index < yetanotherlist.Count; ++index)
{
theint += index;
if (theint % 7 == 0)
{
theint -= index / 3;
}
}
while (theint > 8)
{
theint /= 2;
}
return theint;
}
// this function is intentionally creating a lot of objects, to simulate complexity
static void taskfunc(List<ExampleClass> intlist, List<ExampleClass> anotherClassList, int skip, int take)
{
if (take == 0)
{
take = intlist.Count;
}
var partial = intlist.Skip(skip).Take(take).ToList();
for (var index = 0; index < partial.Count; ++index)
{
var testint = expensivefunc(index);
var newClass = new ExampleClass(partial[index]);
newDna.Inners[StaticRandom.Next(5)].SomeInt = testint;
anotherClassList.Add(new ExampleClass(newClass));
}
}
The expected result is that the list anotherClassList will be the same size as testlist and this happens when the lists are smaller or the complexity of the task operations is smaller. However, when I increase the volume of operations, the anotherClassList has a few indexes missing and sometimes some of the indexes in the list are null objects.
Example result:
Why does this happen, I have Task.WaitAll?
Your problem is it's just not thread-safe; you just can't add to a list<T> in a multi-threaded environment and expect it to play nice.
One way is to use lock or a thread safe collection, but I feel this all should be refactored (my OCD is going off all over the place).
private static object _sync = new object();
...
private static void TaskFunc(List<ExampleClass> intlist, List<ExampleClass> anotherClassList, int skip, int take)
{
...
var partial = intlist.Skip(skip).Take(take).ToList();
...
// note that locking here will likely drastically decrease any performance threading gain
lock (_sync)
{
for (var index = 0; index < partial.Count; ++index)
{
// this is your problem, you are adding to a list from multiple threads
anotherClassList.Add(...);
}
}
}
In short, I think you need to better thinking about the threading logic of your method, identify what you are trying to achieve, and how to make it conceptually thread safe (while keeping your performance gains).
After TheGeneral enlightened me that Lists are not thread safe, I changed the List to which I was adding in a thread, to an Array type and this fixed my issue.

Mutithreading in C# queries

I am new to multithreading in C# . I have a 3D array of size (x)(y)(z) and say i want to calculate the average of all the z samples for every (x,y) values. I wish to do that using multithreading (say 2 threads) where i will send half the array of size (x/2)*y*z for processing to thread1 and the other half to thread2.
How to do it? How do I pass and retrieve arguments from individual threads? A code example will be helpful.
Regards
I would recommend using PLINQ for this instead of threading this yourself.
It will let you run your query using LINQ syntax, but parallelize it (across all of your cores) automatically.
There are many reasons why it makes sense to use something PLINQ (as mentioned by Reed) or Parallel.For as implementing a low-overhead scheduler for distributing jobs over several cpus is a bit challenging.
So if I understood you correctly maybe this could get you started (on my 4 core machine the parallel version is 3 times faster than the single core version):
using System;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
class Program
{
static void AverageOfZ (
double[] input,
double[] result,
int x,
int y,
int z
)
{
Debug.Assert(input.Length == x*y*z);
Debug.Assert(result.Length == x*y);
//Replace Parallel with Sequential to compare with non-parallel loop
//Sequential.For(
Parallel.For(
0,
x*y,
i =>
{
var begin = i*z;
var end = begin + z;
var sum = 0.0;
for (var iter = begin; iter < end; ++iter)
{
sum += input[iter];
}
result[i] = sum/z;
});
}
static void Main(string[] args)
{
const int X = 64;
const int Y = 64;
const int Z = 64;
const int Repetitions = 40000;
var random = new Random(19740531);
var samples = Enumerable.Range(0, X*Y*Z).Select(x => random.NextDouble()).ToArray();
var result = new double[X*Y];
var then = DateTime.Now;
for (var iter = 0; iter < Repetitions; ++iter)
{
AverageOfZ(samples, result, X, Y, Z);
}
var diff = DateTime.Now - then;
Console.WriteLine(
"{0} samples processed {1} times in {2} seconds",
samples.Length,
Repetitions,
diff.TotalSeconds
);
}
}
static class Sequential
{
public static void For(int from, int to, Action<int> action)
{
for (var iter = from; iter < to; ++iter)
{
action(iter);
}
}
}
PS. When going for concurrent performance its important to consider how the different cores access memory as its very easy to get disappointing performance otherwise.
Dot Net 3.5 and onward introduced many shortcut keywords that abstract away the complexity of things like Parallel for multi threading or Async for Async IO. Unfortunately this also provides no opportunity for understanding whats involved in these tasks. For example a colleague of mine was recently trying to use Async for a login method which returned an authentication token.
Here is the full blown multi threaded sample code for your scenario. to make it more real the sample code pretends that:
X is Longitude
Y is Lattitude
and Z is Rainfall Samples at the coordinates
The sample code also follows the Unit of Work design pattern where Rainfall Samples at each coordinate becomes a work item. It also creates discrete foreground threads instead of using a background threadpool.
Due to the simplicity of the work item and short compute time involved I've split the thread synchronization locks into two locks. one for the work queue and one for the output data.
Note: I have not used any Dot net shortcuts such as Lync so this code should run on Dot Net 2.0 as well.
In real world app development something like whats below would only be needed in complex scenarios such as stream processing of a continuous stream of work items in which case you would also need to implement output data buffers cleared regularly as the threads would effectively run forever.
public static class MultiThreadSumRainFall
{
const int LongitudeSize = 64;
const int LattitudeSize = 64;
const int RainFallSamplesSize = 64;
const int SampleMinValue = 0;
const int SampleMaxValue = 1000;
const int ThreadCount = 4;
public static void SumRainfallAndOutputValues()
{
int[][][] SampleData;
SampleData = GenerateSampleRainfallData();
for (int Longitude = 0; Longitude < LongitudeSize; Longitude++)
{
for (int Lattitude = 0; Lattitude < LattitudeSize; Lattitude++)
{
QueueWork(new WorkItem(Longitude, Lattitude, SampleData[Longitude][Lattitude]));
}
}
System.Threading.ThreadStart WorkThreadStart;
System.Threading.Thread WorkThread;
List<System.Threading.Thread> RunningThreads;
WorkThreadStart = new System.Threading.ThreadStart(ParallelSum);
int NumThreads;
NumThreads = ThreadCount;
if (ThreadCount < 1)
{
NumThreads = 1;
}
else if (NumThreads > (Environment.ProcessorCount + 1))
{
NumThreads = Environment.ProcessorCount + 1;
}
OutputData = new int[LongitudeSize, LattitudeSize];
RunningThreads = new List<System.Threading.Thread>();
for (int I = 0; I < NumThreads; I++)
{
WorkThread = new System.Threading.Thread(WorkThreadStart);
WorkThread.Start();
RunningThreads.Add(WorkThread);
}
bool AllThreadsComplete;
AllThreadsComplete = false;
while (!AllThreadsComplete)
{
System.Threading.Thread.Sleep(100);
AllThreadsComplete = true;
foreach (System.Threading.Thread WorkerThread in RunningThreads)
{
if (WorkerThread.IsAlive)
{
AllThreadsComplete = false;
}
}
}
for (int Longitude = 0; Longitude < LongitudeSize; Longitude++)
{
for (int Lattitude = 0; Lattitude < LattitudeSize; Lattitude++)
{
Console.Write(string.Concat(OutputData[Longitude, Lattitude], #" "));
}
Console.WriteLine();
}
}
private class WorkItem
{
public WorkItem(int _Longitude, int _Lattitude, int[] _RainFallSamples)
{
Longitude = _Longitude;
Lattitude = _Lattitude;
RainFallSamples = _RainFallSamples;
}
public int Longitude { get; set; }
public int Lattitude { get; set; }
public int[] RainFallSamples { get; set; }
}
public static int[][][] GenerateSampleRainfallData()
{
int[][][] Result;
Random Rnd;
Rnd = new Random();
Result = new int[LongitudeSize][][];
for(int Longitude = 0; Longitude < LongitudeSize; Longitude++)
{
Result[Longitude] = new int[LattitudeSize][];
for (int Lattidude = 0; Lattidude < LattitudeSize; Lattidude++)
{
Result[Longitude][Lattidude] = new int[RainFallSamplesSize];
for (int Sample = 0; Sample < RainFallSamplesSize; Sample++)
{
Result[Longitude][Lattidude][Sample] = Rnd.Next(SampleMinValue, SampleMaxValue);
}
}
}
return Result;
}
private static object SyncRootWorkQueue = new object();
private static Queue<WorkItem> WorkQueue = new Queue<WorkItem>();
private static void QueueWork(WorkItem SamplesWorkItem)
{
lock(SyncRootWorkQueue)
{
WorkQueue.Enqueue(SamplesWorkItem);
}
}
private static WorkItem DeQueueWork()
{
WorkItem Samples;
Samples = null;
lock (SyncRootWorkQueue)
{
if (WorkQueue.Count > 0)
{
Samples = WorkQueue.Dequeue();
}
}
return Samples;
}
private static int QueueSize()
{
lock(SyncRootWorkQueue)
{
return WorkQueue.Count;
}
}
private static object SyncRootOutputData = new object();
private static int[,] OutputData;
private static void SetOutputData(int Longitude, int Lattitude, int SumSamples)
{
lock(SyncRootOutputData)
{
OutputData[Longitude, Lattitude] = SumSamples;
}
}
private static void ParallelSum()
{
WorkItem SamplesWorkItem;
int SummedResult;
SamplesWorkItem = DeQueueWork();
while (SamplesWorkItem != null)
{
SummedResult = 0;
foreach (int SampleValue in SamplesWorkItem.RainFallSamples)
{
SummedResult += SampleValue;
}
SetOutputData(SamplesWorkItem.Longitude, SamplesWorkItem.Lattitude, SummedResult);
SamplesWorkItem = DeQueueWork();
}
}
}

ThreadQueue problems in "Accelerated C# 2008"

Example for threading queue book "Accelerated C# 2008" (CrudeThreadPool class) not work correctly. If I insert long job in WorkFunction() on 2-processor machine executing for next task don't run before first is over. How to solve this problem? I want to load the processor to 100 percent
public class CrudeThreadPool
{
static readonly int MAX_WORK_THREADS = 4;
static readonly int WAIT_TIMEOUT = 2000;
public delegate void WorkDelegate();
public CrudeThreadPool()
{
stop = 0;
workLock = new Object();
workQueue = new Queue();
threads = new Thread[MAX_WORK_THREADS];
for (int i = 0; i < MAX_WORK_THREADS; ++i)
{
threads[i] = new Thread(new ThreadStart(this.ThreadFunc));
threads[i].Start();
}
}
private void ThreadFunc()
{
lock (workLock)
{
int shouldStop = 0;
do
{
shouldStop = Interlocked.Exchange(ref stop, stop);
if (shouldStop == 0)
{
WorkDelegate workItem = null;
if (Monitor.Wait(workLock, WAIT_TIMEOUT))
{
// Process the item on the front of the queue
lock (workQueue)
{
workItem = (WorkDelegate)workQueue.Dequeue();
}
workItem();
}
}
} while (shouldStop == 0);
}
}
public void SubmitWorkItem(WorkDelegate item)
{
lock (workLock)
{
lock (workQueue)
{
workQueue.Enqueue(item);
}
Monitor.Pulse(workLock);
}
}
public void Shutdown()
{
Interlocked.Exchange(ref stop, 1);
}
private Queue workQueue;
private Object workLock;
private Thread[] threads;
private int stop;
}
public class EntryPoint
{
static void WorkFunction()
{
Console.WriteLine("WorkFunction() called on Thread 0}", Thread.CurrentThread.GetHashCode());
//some long job
double s = 0;
for (int i = 0; i < 100000000; i++)
s += Math.Sin(i);
}
static void Main()
{
CrudeThreadPool pool = new CrudeThreadPool();
for (int i = 0; i < 10; ++i)
{
pool.SubmitWorkItem(
new CrudeThreadPool.WorkDelegate(EntryPoint.WorkFunction));
}
pool.Shutdown();
}
}
I can see 2 problems:
Inside ThreadFunc() you take a lock(workLock) for the duration of the method, meaning your threadpool is no longer async.
in the Main() method, you close down the threadpool w/o waiting for it to finish. Oddly enough that is why it is working now, stopping each ThreadFunc after 1 loop.
It's hard to tell because there's no indentation, but it looks to me like it's executing the work item while still holding workLock - which is basically going to serialize all the work.
If at all possible, I suggest you start using the Parallel Extensions framework in .NET 4, which has obviously had rather more time spent on it. Otherwise, there's the existing thread pool in the framework, and there are other implementations around if you're willing to have a look. I have one in MiscUtil although I haven't looked at the code for quite a while - it's pretty primitive.

Categories

Resources