Related
This is my first attempt at parallel programming.
I'm writing a test console app before using this in my real app and I can't seem to get it right. When I run this, the parallel search is always faster than the sequential one, but the parallel search never finds the correct value. What am I doing wrong?
I tried it without using a partitioner (just Parallel.For); it was slower than the sequential loop and gave the wrong number. I saw a Microsoft doc that said for simple computations, using Partitioner.Create can speed things up. So I tried that but still got the wrong values. Then I saw Interlocked, but I think I'm using it wrong.
Any help would be greatly appreciated
Random r = new Random();
Stopwatch timer = new Stopwatch();
do {
// Make and populate a list
List<short> test = new List<short>();
for (int x = 0; x <= 10000000; x++)
{
test.Add((short)(r.Next(short.MaxValue) * r.NextDouble()));
}
// Initialize result variables
short rMin = short.MaxValue;
short rMax = 0;
// Do min/max normal search
timer.Start();
foreach (var amp in test)
{
rMin = Math.Min(rMin, amp);
rMax = Math.Max(rMax, amp);
}
timer.Stop();
// Display results
Console.WriteLine($"rMin: {rMin} rMax: {rMax} Time: {timer.ElapsedMilliseconds}");
// Initialize parallel result variables
short pMin = short.MaxValue;
short pMax = 0;
// Create list partioner
var rangePortioner = Partitioner.Create(0, test.Count);
// Do min/max parallel search
timer.Restart();
Parallel.ForEach(rangePortioner, (range, loop) =>
{
short min = short.MaxValue;
short max = 0;
for (int i = range.Item1; i < range.Item2; i++)
{
min = Math.Min(min, test[i]);
max = Math.Max(max, test[i]);
}
_ = Interlocked.Exchange(ref Unsafe.As<short, int>(ref pMin), Math.Min(pMin, min));
_ = Interlocked.Exchange(ref Unsafe.As<short, int>(ref pMax), Math.Max(pMax, max));
});
timer.Stop();
// Display results
Console.WriteLine($"pMin: {pMin} pMax: {pMax} Time: {timer.ElapsedMilliseconds}");
Console.WriteLine("Press enter to run again; any other key to quit");
} while (Console.ReadKey().Key == ConsoleKey.Enter);
Sample output:
rMin: 0 rMax: 32746 Time: 106
pMin: 0 pMax: 32679 Time: 66
Press enter to run again; any other key to quit
The correct way to do a parallel search like this is to compute local values for each thread used, and then merge the values at the end. This ensures that synchronization is only needed at the final phase:
var items = Enumerable.Range(0, 10000).ToList();
int globalMin = int.MaxValue;
int globalMax = int.MinValue;
Parallel.ForEach<int, (int Min, int Max)>(
items,
() => (int.MaxValue, int.MinValue), // Create new min/max values for each thread used
(item, state, localMinMax) =>
{
var localMin = Math.Min(item, localMinMax.Min);
var localMax = Math.Max(item, localMinMax.Max);
return (localMin, localMax); // return the new min/max values for this thread
},
localMinMax => // called one last time for each thread used
{
lock(items) // Since this may run concurrently, synchronization is needed
{
globalMin = Math.Min(globalMin, localMinMax.Min);
globalMax = Math.Max(globalMax, localMinMax.Max);
}
});
As you can see this is quite a bit more complex than a regular loop, and this is not even doing anything fancy like partitioning. An optimized solution would work over larger blocks to reduce overhead, but this is omitted for simplicity, and it looks like the OP is aware such issues already.
Be aware that multi threaded programming is difficult. While it is a great idea to try out such techniques in a playground rather than a real program, I would still suggest that you should start by studying the potential dangers of thread safety, there is fairly easy to find good resources about this.
Not all problems will be as obviously wrong like this, and it is quite easy to cause issues that breaks once in a million, or only when the cpu load is high, or only on single CPU systems, or issues that are only detected long after the code is put into production. It is a good practice to be paranoid whenever multiple threads may read and write the same memory concurrently.
I would also recommend learning about immutable data types, and pure functions, since these are much safer and easier to reason about once multiple threads are involved.
Interlocked.Exchange is thread safe only for Exchange, every Math.Min and Math.Max can be with race condition. You should compute min/max for every batch separately and then join results.
Using low-lock techniques like the Interlocked class is tricky and advanced. Taking into consideration that your experience in multithreading is not excessive, I would say go with a simple and trusty lock:
object locker = new object();
//...
lock (locker)
{
pMin = Math.Min(pMin, min);
pMax = Math.Max(pMax, max);
}
For some testing I'm doing I need a C# function that takes around 10 seconds to execute. It will be called from an ASPX page, but I need the function to eat up CPU time on the server, not rendering time. A slow query into the Northwinds database would work, or some very slow calculations. Any ideas?
Try to calculate nth prime number to simulate CPU intensive work -
public void Slow()
{
long nthPrime = FindPrimeNumber(1000); //set higher value for more time
}
public long FindPrimeNumber(int n)
{
int count=0;
long a = 2;
while(count<n)
{
long b = 2;
int prime = 1;// to check if found a prime
while(b * b <= a)
{
if(a % b == 0)
{
prime = 0;
break;
}
b++;
}
if(prime > 0)
{
count++;
}
a++;
}
return (--a);
}
How much time it will take will depend on the hardware configuration of the system.
So try with input as 1000 then either increase input value or decrease it.
This function will simulate CPU intensive work.
Arguably the simplest such function is this:
public void Slow()
{
var end = DateTime.Now + TimeSpan.FromSeconds(10);
while (DateTime.Now < end)
/* nothing here */ ;
}
You can use a 'while' loop to make the CPU busy.
void CpuIntensive()
{
var startDt = DateTime.Now;
while (true)
{
if ((DateTime.Now - startDt).TotalSeconds >= 10)
break;
}
}
This method will stay in the while loop for 10 seconds. Also, if you run this method in multiple threads, you can make all CPU cores busy.
For maxing out multiple cores I adjusted #Motti's answer a bit, and got the following:
Enumerable
.Range(1, Environment.ProcessorCount) // replace with lesser number if 100% usage is not what you are after.
.AsParallel()
.Select(i => {
var end = DateTime.Now + TimeSpan.FromSeconds(10);
while (DateTime.Now < end)
/*nothing here */ ;
return i;
})
.ToList(); // ToList makes the query execute.
This is CPU intensive on a single thread/CPU, and lasts 10 seconds.
var endTime = DateTime.Now.AddSeconds(10);
while(true) {
if (DateTime.Now >= endTime)
break;
}
As a side note, you should not normally do this.
Just use
Thread.Sleep(number of milliseconds here);
You will have to add using System.Threading;
I'd like to write a class that will fire an event x a pre-defined numbers of times per second call it n.
However, I would like x NOT to fire evenly spaced within each second.
So, say n = 100, 25 might fire in the first 300ms then a further 50 over the next 600ms and the final 25 in the remaining 100ms.
Ideally, I'd like better granularity then that presented above and a greater range of spacing within each second.
I'm wondering if I would be able to create an array with defined millisecond and how many events to fire. Then use a loop and the Stopwatch class to determine whether the defined events for that millisecond waypoint should be fired.
The question is, could the array be calculated for each second fast enough, each second should have random spacing.
Obviously, the event would need to be Asynchronous to avoid getting delayed by what ever is wired up to it.
Anyone come across any similar need?
UPDATE
I thought I'd at least put my initial effort on here.
So I found that there is enough resolution to check which millisecond you are currently in and which second. Basically, each second I rebuild my spacing each millisecond gets an entry in the array and each entry says how many times to fire the event that millisecond.
My issue is the spacing...I needs a better way to try and clump the event count, currently this jsut seems to plonk them in evenly spaced in either 0, 1 or 2.
public delegate void EmptyEventDelegate();
public class RandEvent
{
public event EmptyEventDelegate OnEvent = delegate { };
private bool running = false;
Random r = new Random();
private int eventsPS;
public RandEvent(int eventsPS = 1)
{
this.eventsPS = eventsPS;
}
public void Start()
{
running = true;
Task.Factory.StartNew(() =>
{
Run();
});
}
private void Run()
{
var sw = new Stopwatch();
sw.Start();
int currentSecond = 0;
int[] eventCount = BuildEventSpacing();
while(running)
{
if (currentSecond != sw.Elapsed.Seconds)
{
currentSecond = sw.Elapsed.Seconds;
eventCount = BuildEventSpacing();
}
else
{
for(int i = 0; i < eventCount[sw.Elapsed.Milliseconds]; i++)
OnEvent();
}
}
sw.Stop();
}
private int[] BuildEventSpacing()
{
var array = new int[1000];
for (int i = 0; i < eventsPS; i++)
{
array[r.Next(0, 999)]++;
}
return array;
}
public void Stop()
{
running = false;
}
}
To create the array of random event times:
Choose a minimum time interval between events, and fill an array with all possible event times. For example, if you choose a granularity of 1ms, then the array will have 1000 values: 0, 0.001, 0.002, ..., 0.999.
Each second, shuffle the array to randomize the order of the elements.
Use the first n elements of the array as the firing times for the event.
Create a Timer and in the Tick event do whatever needs to be done. Additionally, use the following method (in an additional handler of the Tick event) to change the interval each time. It would be best to provide the same Random instance is passed to the method every time rather than providing new instances.
private static double millisecondsPerSecond = 1000.0;
/// <summary>
/// Method used to determine how long you would wait for the event to fire next
/// </summary>
/// <param name="averageFiresPerSecond">The approximate number of times the event should occur per second.</param>
/// <param name="variance">How much variance should be allowed, as a percentage. i.e. a variance of 0.1 would mean that
/// the delay will be +/- 10% of the exact rate.</param>
/// <param name="generator">A randon number generator.</param>
/// <returns>The number of milliseconds to wait for the next event to fire.</returns>
public double GetNextDelay(int averageFiresPerSecond, double variance, Random generator)
{
double randomFactor = ((generator.NextDouble() * 2) * variance);
return (millisecondsPerSecond / averageFiresPerSecond) * randomFactor;
}
Let's say you want 100 numbers whose sum is 1. You can do the following:
Generate 100 random numbers between 0 and 1 (i.e. Random.NextDouble) and store in a list.
Sum the numbers.
Divide each number in the list by the sum.
You now have a list of 100 numbers that sum to 1.0.
Now, create a one-shot timer whose interval is the first value in the list. When it fires, the handler does its job and then sets up the one-shot timer again with an interval equal to the next value in the list.
Except that's not going to work perfectly because it doesn't take into account the amount of time required to process the tick and setup the timer again. So you'll have to keep track of that latency and adjust the next tick accordingly. But even that's not going to be perfect.
As I said in my comment, you're going to have a difficult time getting any of the .NET timers to give you more than 60 ticks per second. At best, you can expect 15 ms between timer ticks. Even if you go with a custom timer implementation, you're going to have some trouble because timer ticks don't occur exactly on time. The problem gets worse, of course, as the timer interval decreases.
In any event, there's no way that you can get a Windows timer to give you better than 1 ms resolution, so you'll have to adjust the method I described above to give you numbers that are greater than or equal to 0.001.
Your idea of using a Stopwatch can work with my method of creating the event intervals. Understand, however, that the loop is going to consume close to 100% of CPU time on a single core. Even then, it might not give you events exactly on time, because other higher-priority tasks might cause your loop to be swapped out.
I would suggest to make a collection of TimeSpan objects initialized to be equal and have a sum of 1 second. The exact values would be determined by your n.
From there, you can offset pairs or groups by whatever values you like and your total will still be one second. For example...
var offset = new TimeSpan.FromMilliseconds(10); // 10ms offset
timeSpans[0] += offset;
timeSpans[1] -= offset;
In this way you can shift them around without affecting the overall sum. If you need to make them so that more come early and fewer come late, then you could do something like...
timeSpans[0] -= offset;
timeSpans[1] -= offset;
timeSpans[2] += offset;
timeSpans[2] += offset;
This would make indexes 0 and 1 have a shorter delay and index 2's delay will be twice as long, but keep the sum total unaffected.
The only thing to keep in mind is that none of the time spans should be less than 0s and your sum total should always be 1s, then you're golden. Once you have your TimeSpan objects distributed, you can use them in order to pause between firing event x. You're going to want to vary (randomize?) your offset a few times and each time you change it, choose new timeSpans (randomly?) to apply the offset to. At the end of the process, it should be pretty chaotically spaced out.
I can provide a more detailed code sample if this is too abstract. :)
Hope this helps!
using System;
using System.Text;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Diagnostics;
namespace TestProject1
{
[TestClass]
public class Test
{
[TestMethod]
public void TestSteps()
{
var random = new Random();
for (int i = 0; i < 20000; i++)
{
int numberOfEvents = random.Next(1,1000);
var generator = new RandomTimeStepGenerator(numberOfEvents);
var stopwatch = new Stopwatch();
stopwatch.Start();
var steps = generator.MillisecondDeltas;
Assert.AreEqual(numberOfEvents, steps.Count);
var sum = generator.MillisecondDeltas.Sum();
Assert.AreEqual(1000.0,sum,0.1);
Assert.IsTrue(stopwatch.ElapsedMilliseconds<10);
}
}
}
public class RandomTimeStepGenerator
{
private readonly int _numberOfEvents;
const int timeResolution = 10000;
public RandomTimeStepGenerator(int numberOfEvents)
{
_numberOfEvents = numberOfEvents;
}
public int NumberOfEvents
{
get { return _numberOfEvents; }
}
public List<double> MillisecondDeltas
{
get
{
var last=0;
var result = new List<double>();
var random = new Random();
for (var i = 0; i < timeResolution && result.Count < _numberOfEvents; i++)
{
var remainingEvents = _numberOfEvents - result.Count;
var remainingTime = timeResolution - i;
if(remainingEvents==1) // make sure the last event fires on the second
{
result.Add((timeResolution - last) / 10.0);
last = i;
}
else if (remainingTime <= remainingEvents) // hurry up and fire you lazy !!! your going to run out of time
{
result.Add(0.1);
}
else
{
double probability = remainingEvents / (double)remainingTime;
int next = random.Next(0,timeResolution);
if ((next*probability) > _numberOfEvents)
{
result.Add((i - last)/10.0);
last = i;
}
}
}
return result;
}
}
}
}
I am writing a test application for stress-testing a server. I have a list of messages which I need to send in a given time interval, but I am interested in not just sending them with a uniform gap, but "clumping" messages together to simulate bursts. This will be controlled by a slider in the UI - at one end is uniform, at the other is "clumpy". Is there a neat mathematical way to do this?
So lets say that you have slider, max, a time interval T that you can cut up in to N divisions, and a total number of message M you want to send. Then at time division N, I would send
(M/N)*(max(1-slider/max)+k*(slider/max)cosine(PI*N/T))
When slider=0, this will give total uniformity, with M/N messages arriving each interval. When slider=max, the uniformity will be completely damped, and you can adjust the parameter k to make the clumping even more extreme.
Here's what I think could work:
var rngClumpMax = new Random();
var clumpMin = 1;
var clumpMax = 1;
var clumpSize = 1;
clumpSize = rngClumpMax.Next(clumpMin, clumpMax);
while (true)
{
List<string> messages = FetchMessages(clumpSize);
if (messages.Count > 0)
{
SendMessages(messages);
}
else
{
break;
}
}
The scrollbar could set the value of clumpMax, so you can generate clumps between sizes of 1 and some N value from the scrollbar.
If you had a control with double scrolls, you could set even clumpMin and generate a random clumpSize between clumpMin and clumpMax.
What's a good algorithm for determining the remaining time for something to complete? I know how many total lines there are, and how many have completed already, how should I estimate the time remaining?
Why not?
(linesProcessed / TimeTaken) (timetaken / linesProcessed) * LinesLeft = TimeLeft
TimeLeft will then be expressed in whatever unit of time timeTaken is.
Edit:
Thanks for the comment you're right this should be:
(TimeTaken / linesProcessed) * linesLeft = timeLeft
so we have
(10 / 100) * 200 = 20 Seconds now 10 seconds go past
(20 / 100) * 200 = 40 Seconds left now 10 more seconds and we process 100 more lines
(30 / 200) * 100 = 15 Seconds and now we all see why the copy file dialog jumps from 3 hours to 30 minutes :-)
I'm surprised no one has answered this question with code!
The simple way to calculate time, as answered by #JoshBerke, can be coded as follows:
DateTime startTime = DateTime.Now;
for (int index = 0, count = lines.Count; index < count; index++) {
// Do the processing
...
// Calculate the time remaining:
TimeSpan timeRemaining = TimeSpan.FromTicks(DateTime.Now.Subtract(startTime).Ticks * (count - (index+1)) / (index+1));
// Display the progress to the user
...
}
This simple example works great for simple progress calculation.
However, for a more complicated task, there are many ways this calculation could be improved!
For example, when you're downloading a large file, the download speed could easily fluctuate. To calculate the most accurate "ETA", a good algorithm would be to only consider the past 10 seconds of progress. Check out ETACalculator.cs for an implementation of this algorithm!
ETACalculator.cs is from Progression -- an open source library that I wrote. It defines a very easy-to-use structure for all kinds of "progress calculation". It makes it easy to have nested steps that report different types of progress. If you're concerned about Perceived Performance (as #JoshBerke suggested), it will help you immensely.
Make sure to manage perceived performance.
Although all the progress bars took exactly the same amount of time in the test, two characteristics made users think the process was faster, even if it wasn't:
progress bars that moved smoothly towards completion
progress bars that sped up towards the end
Not to revive a dead question but I kept coming back to reference this page.
You could create an extension method on the Stopwatch class to get functionality that would get an estimated remaining time span.
static class StopWatchUtils
{
/// <summary>
/// Gets estimated time on compleation.
/// </summary>
/// <param name="sw"></param>
/// <param name="counter"></param>
/// <param name="counterGoal"></param>
/// <returns></returns>
public static TimeSpan GetEta(this Stopwatch sw, int counter, int counterGoal)
{
/* this is based off of:
* (TimeTaken / linesProcessed) * linesLeft=timeLeft
* so we have
* (10/100) * 200 = 20 Seconds now 10 seconds go past
* (20/100) * 200 = 40 Seconds left now 10 more seconds and we process 100 more lines
* (30/200) * 100 = 15 Seconds and now we all see why the copy file dialog jumps from 3 hours to 30 minutes :-)
*
* pulled from http://stackoverflow.com/questions/473355/calculate-time-remaining/473369#473369
*/
if (counter == 0) return TimeSpan.Zero;
float elapsedMin = ((float)sw.ElapsedMilliseconds / 1000) / 60;
float minLeft = (elapsedMin / counter) * (counterGoal - counter); //see comment a
TimeSpan ret = TimeSpan.FromMinutes(minLeft);
return ret;
}
}
Example:
int y = 500;
Stopwatch sw = new Stopwatch();
sw.Start();
for(int x = 0 ; x < y ; x++ )
{
//do something
Console.WriteLine("{0} time remaining",sw.GetEta(x,y).ToString());
}
Hopefully it will be of some use to somebody.
EDIT:
It should be noted this is most accurate when each loop takes the same amount of time.
Edit 2:
Instead of subclassing I created an extension method.
Generally, you know three things at any point in time while processing:
How many units/chunks/items have been processed up to that point in time (A).
How long it has taken to process those items (B).
The number of remaining items (C).
Given those items, the estimate (unless the time to process an item is constant) of the remaining time will be
B * C / A
I made this and it works quite good, feel free to change the method signature according to your variable types or also to the return type, probably you would like to get the TimeSpan object or just the seconds...
/// <summary>
/// Calculates the eta.
/// </summary>
/// <param name="processStarted">When the process started</param>
/// <param name="totalElements">How many items are being processed</param>
/// <param name="processedElements">How many items are done</param>
/// <returns>A string representing the time left</returns>
private string CalculateEta(DateTime processStarted, int totalElements, int processedElements)
{
int itemsPerSecond = processedElements / (int)(processStarted - DateTime.Now).TotalSeconds;
int secondsRemaining = (totalElements - processedElements) / itemsPerSecond;
return new TimeSpan(0, 0, secondsRemaining).ToString();
}
You will require to initialize a DateTime variable when the processing starts and send it to the method on each iteration.
Do not forget that probably your window will be locked if the process is quite long, so when you place the return value into a control, don't forget to use the .Refresh() method of it.
If you are using threads then you can attempt to set the text using the Invoke(Action) method, would be easier to use this extension method to archieve it easily.
If you use a console application, then you should not have problems displaying the output line by line.
Hope it helps someone.
It depends greatly on what the "something" is. If you can assume that the amount of time to process each line is similar, you can do a simple calculation:
TimePerLine = Elapsed / LinesProcessed
TotalTime = TimePerLine * TotalLines
TimeRemaining = TotalTime - LinesRemaining * TimePerLine
there is no standard algorithm i know of, my sugestion would be:
Create a variable to save the %
Calculate the complexity of the task you wish to track(or an estimative of it)
Put increments to the % from time to time as you would see fit given the complexity.
You probably seen programs where the load bar runs much faster in one point than in another. Well that's pretty much because this is how they do it. (though they probably just put increments at regular intervals in the main wrapper)
Where time$("ms") represents the current time in milliseconds since 00:00:00.00, and lof represents the total lines to process, and x represents the current line:
if Ln>0 then
Tn=Tn+time$("ms")-Ln 'grand total of all laps
Rn=Tn*(lof-x)/x^2 'estimated time remaining in seconds
end if
Ln=time$("ms") 'start lap time (current time)
That really depends on what is being done... lines are not enough unless each individual line takes the same amount of time.
The best way (if your lines are not similar) would probably be to look at logical sections of the code find out how long each section takes on average, then use those average timings to estimate progress.
If you know the percentage completed, and you can simply assume that the time scales linearly, something like
timeLeft = timeSoFar * (1/Percentage)
might work.
I already knew the percentage complete & time elapsed, so this helped me:
TimeElapsed * ((100 - %complete) / %complete) = TimeRemaining
I then updated this value every time %complete changed, giving me a constant varying ETA.
There is 2 ways of showing time
Time elapsed and Time Remaining overall:
so elapsed will increase but remaining will be likely stable total time needed (if per second is stable)
Time elapsed and Time Left:
so Time Left = Total Needed - Elapsed
My idea/formula is more likely like this:
Processed - updated from running thread from 0 to Total
I have timer with 1000ms interval that calculates processed per second:
processedPerSecond = Processed - lastTickProcessed;
lastTickProcessed = Processed; //store state from past call
processedPerSecond and lastTickProcessed are global variables out of timer method
Now if we would like to get how many seconds is required to complete the processing (in ideal constant assumption)
totalSecondsNeeded = TotalLines / PerSecond
but we want to show case 2. TimeLeft so
TimeLeftSeconds = (TotalLines - Processed) / PerSecond
TimeSpan remaining = new TimeSpan(0, 0, (transactions.Count - Processed) / processedPerSecond);
labelTimeRemaining.Text = remaining.ToString(#"hh\:mm\:ss");
Of course TimeLeftSeconds will "jump" if PerSecond jumps, so if past PerSecond was 10 then 30 then back to 10, the user will see it.
There is a way to calculate average, but this may not show real time left if process speeds up at the end
int perSecond = (int)Math.Ceiling((processed / (decimal)timeElapsed.TotalSeconds)); //average not in past second
So it may be the choice for a developer to "pick" a method that will be most accurate based on prediction of how "jumpy" the processing is
We could also calculate and save each PerSecond, then take last 10 second and made average, but in this case user will have to wait 10 seconds to see first calculation
or we could show time left starting from first per second and then progressively average summing up to 10 last PerSecond
I hope my "jumpy" thoughts will help someone to build something satisfying
How about this....
I used this to walk through a set of records (rows in an Excel file, in one case)
L is the current row number
X is the total number of rows
dat_Start is set to Now() when the routine begins
Debug.Print Format((L / X), "percent") & vbTab & "Time to go:" & vbTab & Format((DateDiff("n", dat_Start, Now) / L) * (X - L), "00") & ":" & Format(((DateDiff("s", dat_Start, Now) / L) * (X - L)) Mod 60, "00")
PowerShell function
function CalculateEta([datetime]$processStarted, [long]$totalElements, [long]$processedElements) {
$itemsPerSecond = $processedElements / [DateTime]::Now.Subtract($processStarted).TotalSeconds
$secondsRemaining = ($totalElements - $processedElements) / $itemsPerSecond
return [TimeSpan]::FromSeconds($secondsRemaining)
}
I prefer System.Threading.Timer rather than System.Diagnostics.Stopwatch.
System.Threading.Timer, which executes a single callback method on a
thread pool thread
The following code is an example of a calculating elapsed time with Threading.Timer.
public class ElapsedTimeCalculator : IDisposable
{
private const int ValueToInstantFire = 0;
private readonly Timer timer;
private readonly DateTime initialTime;
public ElapsedTimeCalculator(Action<TimeSpan> action)
{
timer = new Timer(new TimerCallback(_ => action(ElapsedTime)));
initialTime = DateTime.UtcNow;
}
// Use Timeout.Infinite if you don't want to set period time.
public void Fire() => timer.Change(ValueToInstantFire, Timeout.Infinite);
public void Dispose() => timer?.Dispose();
private TimeSpan ElapsedTime => DateTime.UtcNow - initialTime;
}
BTW You can use System.Reactive.Concurrency.IScheduler (scheduler.Now.UtcDateTime) instead of using DateTime directly, if you would like to mock and virtualize the datetime for unit tests.
public class PercentageViewModel : IDisposable
{
private readonly ElapsedTimeCalculator elapsedTimeCalculator;
public PercentageViewModel()
{
elapsedTimeCalculator = new ElapsedTimeCalculator(CalculateTimeRemaining))
}
// Use it where You would like to estimate time remaining.
public void UpdatePercentage(double percent)
{
Percent = percent;
elapsedTimeCalculator.Fire();
}
private void CalculateTimeRemaining(TimeSpan timeElapsed)
{
var timeRemainingInSecond = GetTimePerPercentage(timeElapsed.TotalSeconds) * GetRemainingPercentage;
//Work with calculated time...
}
public double Percent { get; set; }
public void Dispose() => elapsedTimeCalculator.Dispose();
private double GetTimePerPercentage(double elapsedTime) => elapsedTime / Percent;
private double GetRemainingPercentage => 100 - Percent;
}
In Python:
First create a array with the time taken per entry, then calculate the remaining elements and calculate average time taken
import datetime from datetime
import time
# create average function**
def average(total):
return float(sum(total)) / max(len(total), 1)
# create array
time_elapsed = []
# capture starting time
start_time = datetime.now()
# do stuff
# capture ending time
end_time = datetime.now()
# get the total seconds from the captured time (important between two days)
time_in_seconds = (end_time - start_time).total_seconds()
# append the time to a array
time_elapsed.append(time_in_seconds)
# calculate the remaining elements, then multiply it with the average time taken
est_time_left = (LastElement - Processed) * average(time_elapsed)
print(f"Estimated time left: {time.strftime('%H:%M:%S', time.gmtime(est_time_left))}")
** timeit() with k=5000 random elements and number=1000
def average2(total):
avg = 0
for e in total: avg += e
return avg / max(len(total),1)
>> timeit average 0.014467999999999925
>> timeit average2 0.08711790000000003
>> timeit numpy.mean: 0.16030989999999967
>> timeit numpy.average: 0.16210096000000003
>> timeit statistics.mean: 2.8182458