I've been given a task to write a program to count how many page views are requested from our site. My current approach is to get data from google analytics Real Time API, which works to my suprise.
My problem is that to get pageviews every minute I need to poll data from google API twice (cause it returns sum of last 29 minutes + a value from a timer that resets every minute). After I set up 'the point of reset', lets just say, on a 55th second every minute, I poll data on 56th and later on at 53th second, which gives me relatively good estimation of new users / page views requested.
So this is my current approach:
static System.Timers.Timer myTimer = new System.Timers.Timer();
myTimer.AutoReset = false;
myTimer.Interval = interval();
myTimer.Elapsed += myTimer_Elapsed2;
myTimer.Start();
static double interval()
{
return 1000 - DateTime.Now.Millisecond;
}
static void myTimer_Elapsed2(object sender, System.Timers.ElapsedEventArgs e)
{
if (DateTime.Now.Second == (resetPoint.Second - 1) % 60 && warden)
{
DoStuff(); //mostly inserting google API data to database
}
else if (DateTime.Now.Second == (resetPoint.Second + 1) % 60) //so we dont get riddiculous 60 and above
{
//I get some data here, to later use it in DoStuff - mostly to calculate the gap between later
}
myTimer.Interval = interval(); //Because DoStuff() takes about 0.5 sec to execute, i need to recalibrate
myTimer.Start();
}
And it works really well, until it stops after about 2 hours, for now I have no idea why (program runs, just timer doesn't do its work anymore).
How do I make it stable for long periods of time? Best case scenario would be to run it for months without intervention.
# I edited to give a better sense what I'm actually doing
#END CREDITS
I ended up using two timers, each running in a one minute circle. And a database writing sometimes crashed and I didn't handle the corresponding exception properly. Log told me that google API functions from time to time tend to retrieve data a bit longer, which led to multiple Threading.Event calls and made my database data handling throw an exception hence stopping the timer.
I tried to use Quartz approach but its lack of human-friendly howto made me abandon this library.
You should really look into using Quartz.net for scheduling events on a reliable basis. Using a timer for scheduling is asking for stuff like race conditions, event skips and database deadlocks.
http://www.quartz-scheduler.net/ allows you to schedule events at precise intervals, independant of when your code starts or stops.
An example on how you use it: This will build a trigger that will fire at the top of the next hour, then repeat every 2 hours, forever:
trigger = TriggerBuilder.Create()
.WithIdentity("trigger8") // because group is not specified, "trigger8" will be in the default group
.StartAt(DateBuilder.EvenHourDate(null)) // get the next even-hour (minutes and seconds zero ("00:00"))
.WithSimpleSchedule(x => x
.WithIntervalInHours(2)
.RepeatForever())
// note that in this example, 'forJob(..)' is not called
// - which is valid if the trigger is passed to the scheduler along with the job
.Build();
scheduler.scheduleJob(trigger, job);
http://www.quartz-scheduler.net/documentation/quartz-2.x/tutorial/simpletriggers.html has a few examples. I really URGE you to use it, since it will severely simplify development.
The .NET timer is reliable. That is, it won't just stop working randomly for no apparent reason.
Most likely, something in your timer event handler is throwing an exception, which is not surfaced because System.Timers.Timer squashes exceptions. As the documentation states:
The Timer component catches and suppresses all exceptions thrown by event handlers for the Elapsed event. This behavior is subject to change in future releases of the .NET Framework.
That bit about the behavior being "subject to change" has been there since at least .NET 2.0.
What I think is happening is that the timer calls your event handler. The event handler or one of the methods it calls throws an exception, and the timer just drops it on the floor because you don't handle it.
You need to put an exception handler in your myTimer_Elapsed2 method so that you can at least log any exceptions that crop up. With the information provided from the exception log, you can probably identify what the problem is.
Better yet, stop using System.Timers.Timer. Use System.Threading.Timer instead.
Finally, there's no way that your code as written will reliably give you a timer tick at exactly 55 seconds past the minute, every minute. The timer isn't exact. It will be off by a few milliseconds each minute. Over time, it's going to start ticking at 54 seconds (or maybe 56), and then 53 (or 57), etc. If you really need this to tick reliably at 55 seconds past the minute, then you'll need to reset the timer after every minute, taking into account the current time.
I suspect that your need to check every minute at exactly the 55 second mark is overkill. Just set your timer to tick every minute, and then determine the exact elapsed time since the last tick. So one "minute" might be 61 or 62 seconds, and another might be 58 or 59 seconds. If you store the number of requests and the elapsed time, subsequent processing can smooth the bumps and give you a reliable requests-per-minute number. Trying to gather the data on exact one-minute boundaries is going to be exceedingly difficult, if even possible with a non-real-time operating system like Windows.
Related
Is there a way to set hangfire recurring jobs every few seconds?
I do not seek a solution where fire and forget task creates another fire and forget task, and if not, what are suggested alternatives?
Not sure when this became supported but tried this in ASP.NET Core 2.0 with Hangfire 1.7.0. The following code schedules a job every 20 seconds:
RecurringJob.AddOrUpdate<SomeJob>(
x => x.DoWork(),
"*/20 * * * * *");
If I am not mistaken 6 tokens (as opposed to standard 5 tokens) is supported due to Hangfire use of NCrontab which allows cron expressions with 6 tokens (second granularity instead of minute granularity).
Hangfire dashboard also nicely shows the small time interval between runs:
I think anyone who is against allowing a recurring trigger of less than 1 min is short sighted. After all, is 55 secs any less efficient than 1 min ? It seems so arbitrary! As much as I love Hangfire, I've encountered situations where I've had to steer a client to Quartz.net simply because there was a business requirement for a job to run every 55 secs or alike.
Anyone who makes the counter argument that if it was configured to run every 1sec it would have a serious impact on performance is again taking a closed view on things. Of course a trigger with a 1 sec interval is probably not a good idea, but do we dissalow 55 sec or 45 sec for the unlikely situation where someone will choose 1 sec ?
In any case, performance is both subjective and dependent on the host platform and hardware. It really isn't up to the API to enforce opinion when it comes to performance. Just make the polling interval and trigger recurrence configurable. That way the user can determine the best result for themselves.
Although a background process which is orchestrating a job to run every 55 sec may be an option, it isn't very satisfactory. In this case, the process isn't visible via the Hangfire UI so it's hidden from the administrator. I feel this approach is circumventing one of the major benefits of Hangfire.
If Hangfire was a serious competitor to the likes of Quartz.net it would at least match their basic functionality. If Quartz can support triggers with an interval below 1 min than why can't Hangfire!
Although Hangfire doesn't allow you to schedule tasks for less than a minute, you can actually achieve this by having the function recursively scheduling itself; i.e. let's say you want some method to be hit every 2s you can schedule a background job that calls the method on Startup;
BackgroundJob.Schedule(() => PublishMessage(), TimeSpan.FromMilliseconds(2000));
And then in your PublishMessage method do your stuff and then schedule a job to call the same method;
public void PublishMessage()
{
/* do your stuff */
//then schedule a job to exec the same method
BackgroundJob.Schedule(() => PublishMessage(), TimeSpan.FromMilliseconds(2000));
}
The other thing you need to override is the default SchedulePollingInterval of 15s, otherwise your method will only be hit after every 15s. To do so just pass in an instance of BackgroundJobServerOptions to UseHangfireServer in your startup, like so;
var options = new BackgroundJobServerOptions
{
SchedulePollingInterval = TimeSpan.FromMilliseconds(2000)
};
app.UseHangfireServer(options);
I don't know how "foolproof" my solution is, but I managed to achieve my goal with it and everything is "happy" in production.
I had to do the same but with 5 seconds. The default schedule polling interval is set to 15s. So it requires 2 steps to achieve a 5s interval job.
in Startup.cs
var options = new BackgroundJobServerOptions
{
SchedulePollingInterval = TimeSpan.FromMilliseconds(5000)
};
app.UseHangfireDashboard();
app.UseHangfireServer(options);
Your job
RecurringJob.AddOrUpdate(() => YourJob(), "*/5 * * * * *");
Hangfire doesn't support intervals of less than a minute for recurring jobs.
Why? Imagine if they allowed less than a minute: let say 1 sec. How frequently would hangfire check recurring jobs in the database? This would cause a lot of database IO.
See this discussion on Hangfire for more information.
I faced with the same problem, and here it is my solution:
private void TimingExecuteWrapper(Action action, int sleepSeconds, int intervalSeconds)
{
DateTime beginTime = DateTime.UtcNow, endTime;
var interval = TimeSpan.FromSeconds(intervalSeconds);
while (true)
{
action();
Thread.Sleep(TimeSpan.FromSeconds(sleepSeconds));
endTime = DateTime.UtcNow;
if (endTime - beginTime >= interval)
break;
}
}
intervalSeconds is minimal NCRON interval. It is 1 minute.
action is our job code.
Also I suggest to use DisableConcurrentExecution to avoid some collisions of concurrency.
I had a similar requirement, in that I had a recurring job that needs running every 15 seconds.
What I did to try to get around this limitation was to just delay the creation of the scheduled jobs (set to 1 minute intervals), which seemed to do the trick.
However what I found was that, taking into account the polling intervals (set the schedulepolling interval to my frequency) and delays in picking up the new jobs this isn't always as accurate as it should be, but is doing the trick for the moment. However a better/proper solution would be good.
feel a bit dirty having to resolve to the below approach, but it helped me out...
so in essence I created 4 jobs doing the same thing, created 15 seconds apart.
along the lines of:
...
new Thread(() =>
{
//loop from {id=1 through 4}
// create job here with {id} in the name at interval of 1 minute
// sleep 15 seconds
//end loop
}).Start();
...
In my application, I have used the number of System.Threading.Timer and set this timer to fire every 1 second. My application execute the thread at every 1 second but it execution of the millisecond is different.
In my application i have used the OPC server & OPC group .one thread reading the data from the OPC server (like one variable changing it's value & i want to log this moment of the changes values into my application every 1 s)
then another thread to read this data read this data from the first thread every 1s & second thread used for store data into the MYSQL database .
in this process when i will read the data from the first thread then i will get the old data values like , read the data at 10:28:01.530 this second then i will get the information of 10:28:00.260 this second.so i want to mange these threads the first thread worked at 000 millisecond & second thread worked at 500 millisecond. using this first thread update the data at 000 second & second thread read the data at 500 millisecond.
My output is given below:
10:28:32.875
10:28:33.390
10:28:34.875
....
10:28:39.530
10:28:40.875
However, I want following results:
10:28:32.000
10:28:33.000
10:28:34.000
....
10:28:39.000
10:28:40.000
How can the timer be set so the callback is executed at "000 milliseconds"?
First of all, it's impossible. Even if you are to schedule your 'events' for a time that they are fired few milliseconds ahead of schedule, then compare millisecond component of the current time with zero in a loop, the flow control for your code could be taken away at the any given moment.
You will have to rethink your design a little, and not depend on when the event would fire, but think of the algorithm that will compensate for the milliseconds delayed.
Also, you won't have much help with the Threading.Timer, you would have better chance if you have used your own thread, periodically:
check for the current time, see what is the time until next full second
Sleep() for that amount minus the 'spice' factor
do the work you have to do.
You'll calculate your 'spice' factor depending on the results you are getting - does the sleep finishes ahead or behind the schedule.
If you are to give more information about your apparent need for having event at exactly zero ms, I could help you get rid of that requirement.
HTH
I would say that its impossible. You have to understand that switching context for cpu takes time (if other process is running you have to wait - cpu shelduler is working). Each CPU tick takes some time so synchronization to 0 milliseconds is impossible. Maybe with setting high priority of your process you can get closer to 0 but you won't achive it ever.
IMHO it will be impossible to really get a timer to fire exactly every 1sec (on the milisecond) - even in hardcore assembler this would be a very hard task on your normal windows-machine.
I think first what you need to do: is to set right dueTime for a timer. I do it so:
dueTime = 1000 - DateTime.Now.Milliseconds + X; where X - is serving for accuracy and you need select It by testing. Then Threading.Timer each time It ticks running on thread from CLR thread pool and, how tests show - this thread is different each time. Creating threads slows timer, because of this you can use WaitableTimer, which always will be running at the same thread. Instead of WaitableTimer you can using Thread.Sleep method in such way:
Thread.CurrentThread.Priority = Priority.High; //If time is really critical
Thread.Sleep (1000 - DateTime.Now + 50); //Make bound = 1s
while (SomeBoolCondition)
{
Thread.Sleep (980); //1000 ms = 1 second, but something ms will be spent on exit from Sleep
//Here you write your code
}
It will be work faster then a timer.
So a simple enough question really.
How exactly does the interval for System.Timers work?
Does it fire 1 second, each second, regardless of how long the timeout event takes or does it require the routine to finish first and then restarts the interval?
So either:
1 sec....1 sec....1 sec and so on
1 sec + process time....1 sec + process time....1 sec + process time and so on
The reason I ask this is I know my "processing" takes much less than 1 second but I would like to fire it every one second on the dot (or as close as).
I had been using a Thread.Sleep method like so:
Thread.Sleep(1000 - ((int)(DateTime.Now.Subtract(start).TotalMilliseconds) >= 1000 ? 0 : (int)(DateTime.Now.Subtract(start).TotalMilliseconds)));
Where start time is registered at start of the routine. The problem here is that Thread.Sleep only works in milliseconds. So my routine could restart at 1000ms or a fraction over like 1000.0234ms, which can happen as one of my routines takes 0ms according to "TimeSpan" but obviously it has used ticks/nanoseconds - which would then mean the timing is off and is no longer every second. If I could sleep by ticks or nanoseconds it would be bang on.
If number 1 applies to System.Timers then I guess I'm sorted. If not I need some way to "sleep" the thread to a higher resolution of time i.e ticks/nanoseconds.
You might ask why I do an inline IF statement, well sometimes the processing can go above 1000ms so we need to make sure we don't create a minus figure. Also, by the time we determine this, the ending time has changed slightly - not by much, but, it could make the thread delay slightly longer causing the entire subsequent sleeping off.
I know, I know, the time would be negligible... but what happens if the system suddenly stalled for a few ms... it would protect against that in this case.
Update 1
Ok. So I didn't realise you can put a TimeSpan in as the timing value. So I used the below code:
Thread.Sleep(TimeSpan.FromMilliseconds(1000) - ((DateTime.Now.Subtract(start).TotalMilliseconds >= 1000) ? TimeSpan.FromMilliseconds(0) : DateTime.Now.Subtract(start)));
If I am right, this should then allow me to repeat the thread at exactly 1 second - or as close as the system will allow.
IF you have set AutoReset = true; then your theory 1 is true, otherwise you would have to deal with it in code – see the docuementation for Timer on MSDN.
I have a service that is always running, it has a timer to perform a particular action every day at 2AM.
TimeSpan runTime = new TimeSpan(2, 0, 0); // 2 AM
TimeSpan timeToFirstRun = runTime - DateTime.Now.TimeOfDay;
if (timeToFirstRun.TotalHours < 0)
{
timeToFirstRun += TimeSpan.FromDays(1.0);
}
_dailyNodalRunTimer = new Timer(
RunNodalDailyBatch,
null,
timeToFirstRun,
TimeSpan.FromDays(1.0)); //repeat event daily
That initialization code is called once when the service first starts, over the past few days I have logged when the Timer has fired:
2011-05-21 02:00:01.580
2011-05-22 02:00:03.840
...
2011-05-31 02:00:25.227
2011-06-01 02:00:27.423
2011-06-02 02:00:29.847
As you can see its drifting by 2 seconds every day, getting farther and farther from when it was supposed to fire(at 2 AM).
Am I using it wrong or is this Timer not designed to be accurate? I could recreate the timer each day, or have it fire at some small interval and repeatedly check if I want to perform the action, but that seems kind of hacky.
EDIT
I tried using System.Timers.Timer and it appears to have the same issue. The reseting the Interval is because you cant schedule the initial time before the first tick in System.Timers.Timer like you can in System.Threading.Timer
int secondsInterval = 5;
double secondsUntilRunFirstRun = secondsInterval - (DateTime.Now.TimeOfDay.TotalSeconds % secondsInterval);
var timer = new System.Timers.Timer(secondsUntilRunFirstRun * 1000.0);
timer.AutoReset = true;
timer.Elapsed += (sender, e) =>
{
Console.WriteLine(DateTime.Now.ToString("hh:mm:ss.fff"));
if (timer.Interval != (secondsInterval * 1000.0))
timer.Interval = secondsInterval * 1000.0;
};
timer.Start();
Produce the following times, you can see how they are drifting slightly:
06:47:40.020
06:47:45.035
06:47:50.051
...
06:49:40.215
06:49:45.223
06:49:50.232
So I guess the best approach really is to just reschedule the timer in the tick handler? The following produces a tick at a regular interval within ~15 milliseconds
double secondsUntilRunFirstRun = secondsInterval - (DateTime.Now.TimeOfDay.TotalSeconds % secondsInterval);
var timer = new System.Timers.Timer(secondsUntilRunFirstRun * 1000.0);
timer.AutoReset = false;
timer.Elapsed += (sender, e) =>
{
Console.WriteLine(DateTime.Now.ToString("hh:mm:ss.fff"));
timer.Interval = (secondsInterval - (DateTime.Now.TimeOfDay.TotalSeconds % secondsInterval)) * 1000.0;
};
timer.Start();
06:51:45.009
06:51:50.001
...
06:52:50.011
06:52:55.013
06:53:00.001
Don't let timer inaccuracies accumulate. Use the RTC to calculate how many ms remain until the timeout time. Sleep/setInterval to half this time. When the timer fires/sleep returns, use the RTC again to recalculate the interval left and set interval/sleep again to half-life. Repeat this loop until the remaining interval is less than 50ms. Then CPU loop on the RTC until the desired time is exceeded. Fire the event.
Rgds,
Martin
None of the timers in the .NET Framework will be accurate. There are too many variables in play. If you want a more accurate timer then take a look at multimedia timers. I have never used them over longer durations, but I suspect they are still substantially more accurate than the BCL timers.
But, I see no reason that would prohibit you from using the System.Threading.Timer class. Instead of specifying TimeSpan.FromDays(1) use Timeout.Infinite to prevent periodic signaling. You will then have to restart the timer, but you can specify 23:59:58 or 1.00:00:05 for the dueTime parameter depending on what you calculate the next due time to be to have signal at 2:00a.
By the way, the System.Timers.Timer will do no better than System.Threading.Timer. The reason is because the former actually uses the later behind the scenes anyway. System.Timers.Timer just adds a few handy features like auto resetting and marshaling the execution of the Elapsed onto an ISynchronizeInvoke hosted thread (usually a UI thread).
I think you've already realized this but if you want something to fire at a certain time of day (2AM) you'd be better off with a dedicated thread that sleeps, periodically wakes up and looks to see if it's time to run yet. A sleep around 100 milliseconds would be appropriate and would burn virtually no CPU.
Another approach would be that after you've done your daily work, you compute when to next fire based on 2AM tomorrow - DateTime.Current, etc. This may still not be as accurate as you want (I'm not sure) but at least the drift won't get worse and worse and worse.
If you need accurate timing, you'll need System.Timers.Timer class.
Also see this question: .NET, event every minute (on the minute). Is a timer the best option?
From msdn:
System.Threading.Timer is a simple,
lightweight timer ... For server-based timer functionality, you might consider using System.Timers.Timer, which raises events and has additional features.
You can also move it to Windows Task Scheduler
I have a WPF app that uses DispatcherTimer to update a clock tick.
However, after my application has been running for approx 6 hours the clocks hands angles no longer change. I have verified that the DispatcherTimer is still firing with Debug and that the angle values are still updating, however the screen render does not reflect the change.
I have also verified using WPFPerf tools Visual Profiler that the Unlabeled Time, Tick (Time Manager) and AnimatedRenderMessageHandler(Media Content) are all gradually growing until they are consuming nearly 80% of the CPU, however Memory is running stable.
The hHandRT.Angle is a reference to a RotateTransform
hHandRT = new RotateTransform(_hAngle);
This code works perfectly for approx 5 hours of straight running but after that it delays and the angle change does not render to the screen. Any suggestions for how to troubleshoot this problem or any possible solutions you may know of.
.NET 3.5, Windows Vista SP1 or Windows XP SP3 (both show the same behavior)
EDIT: Adding Clock Tick Function
//In Constructor
...
_dt = new DispatcherTimer();
_dt.Interval = new TimeSpan(0, 0, 1);
_dt.Tick += new EventHandler(Clock_Tick);
...
private void Clock_Tick(object sender, EventArgs e)
{
DateTime startTime = DateTime.UtcNow;
TimeZoneInfo tst = TimeZoneInfo.FindSystemTimeZoneById(_timeZoneId);
_now = TimeZoneInfo.ConvertTime(startTime, TimeZoneInfo.Utc, tst);
int hoursInMinutes = _now.Hour * 60 + _now.Minute;
int minutesInSeconds = _now.Minute * 60 + _now.Second;
_hAngle = (double)hoursInMinutes * 360 / 720;
_mAngle = (double)minutesInSeconds * 360 / 3600;
_sAngle = (double)_now.Second * 360 / 60;
// Use _sAngle to showcase more movement during Testing.
//hHandRT.Angle = _sAngle;
hHandRT.Angle = _hAngle;
mHandRT.Angle = _mAngle;
sHandRT.Angle = _sAngle;
//DSEffect
// Add Shadows to Hands creating a UNIFORM light
//hands.Effect = textDropShadow;
}
Along the lines of too much happening in the clock tick, I'm currently trying this adjustment to see if it helps. Too bad it takes 5 hours for the bug to manifest itself :(
//DateTime startTime = DateTime.UtcNow;
//TimeZoneInfo tst = TimeZoneInfo.FindSystemTimeZoneById(_timeZoneId);
//_now = TimeZoneInfo.ConvertTime(startTime, TimeZoneInfo.Utc, tst);
_now = _now.AddSeconds(1);
You say you're creating an instance of the Clock class each time? Note that timers in .NET will root themselves to keep themselves from being garbage collected. They'll keep on firing until you stop them yourself, and they will keep your Clock objects alive because they are referenced in the timer tick event.
I think what's happening is that with each Clock you create you start another timer. At first you only fire 1 event per second, but then you get add on another timer and get 2 per second, and they continue to accumulate in this way. Eventually you see your Tick handler and AnimatedRenderMessageHandler rising in CPU until they bog down and are unable to update your screen. That would also explain why increasing the frequency of the timer firings made your symptoms appear sooner.
The fix should be simple: just stop or dispose the DispatcherTimer when you are done with your Clock object.
You're assuming it's the DispatcherTimer and focusing totally on that. I personally have a hard time believing it has anything to do with the timer itself, but rather think it has to do with whatever you're doing within the timer tick. Can you tell us more about exactly what is going on each time the timer ticks?
hHandRT.Angle = _hAngle;
mHandRT.Angle = _mAngle;
sHandRT.Angle = _sAngle;
I believe you have to look at your above code once again.
You are setting Angle property of your transform for all 3 transforms even if you dont need them to change every second. Your minute will change for every 60 changes and your hour will change for every 3600 seconds. However you can atleast reduce your changing hours angle for every second.
What is happening here is, whenever you request transform changes to WPF, WPF queues the request to priority dispatch queue and every second you are pushing more changes to be done then it can process. And this is the only reason your CPU usage keeps on increasing instead of memory.
Detailed Analysis:
After looking at your code, I feel your DispatcherTimer_Tick event does too much of calculation, remember Dispatcher thread is already overloaded with lots of things to do like managing event routing, visual update etc, if keep your cpu more busy to do custom task in dispatcher thread that too in every second event, it will definately keep on increasing the queue of pending tasks.
You might think its a small multiplication calculation but for Dispatcher thread it can be costly when it comes to loading timezones, converting time value etc. You should profile and see the tick execution time.
You should use System.Threading.Timer object, that will run on another thread, after every tick event, when you are done with your calculations of final angles required, you can then pass them on to Dispatcher thread.
like,
Dispatcher.BeginInvoke((Action)delegate(){
hHandRT.Angle = _hAngle;
mHandRT.Angle = _mAngle;
sHandRT.Angle = _sAngle;
});
By doing this, you will be reducing workload from dispatcher thread little bit.