public interface Event
{
Guid identifier;
Timestamp ts;
}
We're thinking of using Reactive Extensions for a rewrite of a problem at my financial firm.
The premise is that we get events identified by a Guid (stock symbol + uniqueness entropy embedded into it), a timestamp, and a Value field. These come at a high rate, and we cannot act on these objects until "at least" after X seconds (10 seconds), after which we have to act on them, and remove them from the system.
Think about it like two windows, an initial window of "10 seconds" (for example T0 to T10), where we identify all the unique events (basically, group by guid), then we look into the next "10 seconds", "secondary window" (T10-T20), to make sure we're implementing the policy of "at least" 10 seconds. From the "initial window", we remove all the events (because we've accounted for them), and then from the "secondary window", we remove the ones that occurred in the "initial window". And we keep on moving 10 second sliding windows, so now we're looking at window T20-T30, repeat and rinse.
How could I implement this in Rx, because it seems like the way to go.
If you can rely on the server clock and the timestamp in the message (that is, we're in 'real life' mode), and you're after a sliding 10 second delay as opposed to a jumping 10 second window, then you can just delay the events 10 seconds:
var events = new Subject<Event>();
var delayedEvents = events.Delay(TimeSpan.FromSeconds(10));
Checking for unique events etc is just a matter of adding them to a set of some sort:
var guidSet = new HashSet<Guid>();
delayedEvents.Do(e => guidSet.Add(e.identifier));
If you're problem is instead that you must wait 10 seconds and then process the last 10 seconds at once, then you just want to buffer for 10 seconds instead:
var bufferedEvents = events.Buffer(TimeSpan.FromSeconds(10));
bufferedEvents.Do(es => { foreach (var e in es) guidSet.Add(e.identifier); });
I haven't shown the example of a sliding 10 second window as I can't imagine that's what you want (events get processed more than once).
Now we get serious. Let's say you don't want to rely on wall time and instead want to use the time within your events to drive your logic. Assuming event is redefined as:
public class Event
{
public Guid identifier;
public DateTime ts;
}
Create the historical scheduler and feed the scheduled events from the original ones:
var scheduler = new HistoricalScheduler();
var driveSchedule = events.Subscribe(e => scheduler.AdvanceTo(e.ts));
var target = events.SelectMany(e => Observable.Timer(e.ts, scheduler).Select(_ => e));
Now you can simply use the regular Rx combinators on target instead of event, and just pass through the scheduler so they are triggered appropriately, for example:
var bufferedEvents = target.Buffer(TimeSpan.FromSeconds(10), scheduler);
Here's a simple test. Create a hundred events each 'virtually' 30 seconds apart but in real-time triggered every second:
var now = DateTime.Now;
var test = Enumerable.Range(0,99).Select(i =>
Scheduler.ThreadPool.Schedule(
TimeSpan.FromSeconds(i),
() => events.OnNext(new Event() {
identifier = Guid.NewGuid(),
ts = now.AddSeconds(i * 30)
})
)
).ToList();
Subscribe to it and ask for 60 seconds of buffered events - and actually receive 2 events every 2 'real' seconds (60 virtual seconds):
target.Select(e => String.Format("{0} {1}", e.identifier, e.ts.ToString()))
.Buffer(TimeSpan.FromSeconds(60), scheduler)
.Select(es => String.Join(" - ", es))
.DumpLive();
Related
I am developing an Android app using Xamarin. I am trying to figure out how to have it send notifications using AlarmManager with a variable interval. I have figured out how to trigger a notification and have it sent on a fixed interval (say every five minutes). But in some cases I would like to have the interval more dynamic. For instance, let's say I would like to get notified every day a sunset. Since the interval between sunsets are varying, how do I schedule a notification that has an interval that will change?
Here is my code that works for a fix interval:
public void SendNotification( int id
, string title
, string message
, DateTime notifyTime
, long interval)
{
if ( ! _channelInitialized)
{
CreateNotificationChannel();
}
var intent = new Intent(AndroidApp.Context
, typeof(AlarmHandler));
intent.PutExtra(TitleKey
, title);
intent.PutExtra(MessageKey
, message);
var pendingIntent = PendingIntent.GetBroadcast(AndroidApp.Context
, id
, intent
, PendingIntentFlags.UpdateCurrent);
var triggerTime = GetNotifyTime(notifyTime);
var alarmManager = AndroidApp.Context
.GetSystemService(Context.AlarmService) as AlarmManager;
alarmManager?.SetRepeating(AlarmType.RtcWakeup
, triggerTime
, interval
, pendingIntent);
}
It all depends on how exact your alarm should go off and if you know the intervals beforehand or if they are dynamically gotten.
So, if you know the intervals beforehand you can just set multiple alarms (by using different ids on the PendingIntent, docs) to go off on exactly the specified intervals using
setExact(Android.App.AlarmType type, long triggerAtMillis, Android.App.PendingIntent? operation)
setExactAndAllowWhileIdle(Android.App.AlarmType type, long triggerAtMillis, Android.App.PendingIntent? operation)
setAlarmClock(AlarmManager.AlarmClockInfo info, PendingIntent operation)
e.g. idk your specific scenario but here you have a generic example with two alarms:
var baseline = Java.Lang.JavaSystem.CurrentTimeMillis(); // Time from 1970
var time1 = baseline + TimeSpan.FromSeconds(5332).TotalMilliseconds;
var time2 = baseline + TimeSpan.FromSeconds(8900).TotalMilliseconds;
var id1 = DateTime.Now().TotalMilliseconds
var pendingIntentWithId1 = PendingIntent.GetBroadcast(AndroidApp.Context
, id1
, intent
, PendingIntentFlags.UpdateCurrent);
var pendingIntentWithId2 = PendingIntent.GetBroadcast(AndroidApp.Context
, id1 + 1
, intent
, PendingIntentFlags.UpdateCurrent);
var alarmManager = AndroidApp.Context
.GetSystemService(Context.AlarmService) as AlarmManager;
alarmManager?.SetExact(AlarmType.RtcWakeup,
time1,
pendingIntentWithId1);
alarmManager?.SetExact(AlarmType.RtcWakeup,
time2,
pendingIntentWithId2);
If the intervals/time to go off need to be dynamically updated you need to create a Service that runs on the background and updates the alarms on the AlarmManager, or every time you receive on the BroadcastReceiver you set the next alarm time to go off.
Be careful on setting exact alarms because they are resource consuming and if possible you should set inexact alarms.
Also if you're setting exact alarms remember to have the proper permission set if you are targeting Android 12+:
SCHEDULE_EXACT_ALARM (this is target Android 12+)
USE_EXACT_ALARM (this is target Android 13+)
Most of the times SCHEDULE_EXACT_ALARM is enough, the later one can be used in the next cases:
Your app is an alarm clock app or a timer app. Your app is a calendar
app that shows notifications for upcoming events
I have a relatively simple method to wait until an element exists and is displayed. The method handles the situation where more than a single element is returned for the given By (usually we only expect one of them to be displayed but in any case the method will return the first displayed element found).
The issue I'm having is that when there is no matching element on the page (at all), it is taking magnitudes of time more* than the TimeSpan specified, and I can't figure why.
*I just tested with a 30s timeout and it took a little over 5m
code:
/// <summary>
/// Returns the (first) element that is displayed when multiple elements are found on page for the same by
/// </summary>
public static IWebElement FindDisplayedElement(By by, int secondsToWait = 30)
{
WebDriver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(secondsToWait);
// Wait for an element to exist and also displayed
IWebElement element = null;
bool success = SpinWait.SpinUntil(() =>
{
var collection = WebDriver.FindElements(by);
if (collection.Count <= 0)
return false;
element = collection.ToList().FirstOrDefault(x => x.Displayed == true);
return element != null;
}
, TimeSpan.FromSeconds(secondsToWait));
if (success)
return element;
// if element still not found
throw new NoSuchElementException("Could not find visible element with by: " + by.ToString());
}
You would call it with something like this:
[Test]
public void FindDisplayedElement()
{
webDriver.Navigate().GoToUrl("https://stackoverflow.com/questions");
var nonExistenetElementBy = By.CssSelector("#custom-header99");
FindDisplayedElement(nonExistenetElementBy , 10);
}
If you run the test (with 10s timeout) you will find it takes about 100 seconds to actually exit.
It looks like it might have something to do with the mix of the inherit wait built into WebDriver.FindElements() wrapped inside a SpinWait.WaitUntil().
Would like to hear what you guys think about this conundrum.
Cheers!
That's because SpinWait.WaitUntil is implemented rougly as follows:
public static bool SpinUntil(Func<bool> condition, TimeSpan timeout) {
int millisecondsTimeout = (int) timeout.TotalMilliseconds;
long num = 0;
if (millisecondsTimeout != 0 && millisecondsTimeout != -1)
num = Environment.TickCount;
SpinWait spinWait = new SpinWait();
while (!condition())
{
if (millisecondsTimeout == 0)
return false;
spinWait.SpinOnce();
// HERE
if (millisecondsTimeout != -1 && spinWait.NextSpinWillYield && millisecondsTimeout <= (Environment.TickCount - num))
return false;
}
return true;
}
Note condition under "HERE" comment above. It only checks whether timeout has expired IF spinWait.NextSpinWillYield returns true. What that means is: if next spin will result in context switch and timeout is expired - give up and return. But otherwise - keep spinning without even checking a timeout.
NextSpinWillYield result depends on number of previous spins. Basically this construct spins X amount of times (10 I believe), then starts to yield (give up current thread time slice to other threads).
In your case, condition inside SpinUntil take VERY long time to evaluate, which is completely against design of SpinWait - it expects condition evaluation take no time at all (and where SpinWait is actually applicable - it's true). Say one evaluation of condition takes 5 seconds in your case. Then, even if timeout is 1 second - it will spin 10 times first (50 seconds total) before even checking the timeout. That's because SpinWait is not designed for the thing you are trying to use it for. From documentation:
System.Threading.SpinWait is a lightweight synchronization type that
you can use in low-level scenarios to avoid the expensive context
switches and kernel transitions that are required for kernel events.
On multicore computers, when a resource is not expected to be held for
long periods of time, it can be more efficient for a waiting thread to
spin in user mode for a few dozen or a few hundred cycles, and then
retry to acquire the resource. If the resource is available after
spinning, then you have saved several thousand cycles. If the resource
is still not available, then you have spent only a few cycles and can
still enter a kernel-based wait. This spinning-then-waiting
combination is sometimes referred to as a two-phase wait operation.
None of which is applicable to your situation, in my opinion. Another part of documentation states "SpinWait is not generally useful for ordinary applications".
In this case, with such a long condition evaluation time - you can just run it in a loop without additional waiting or spinning, and manually check if timeout has expired on each iteration.
Doing some further testing I found out that reducing the WebDriver Implicit Wait Timeout to a low number (e.g. 100ms) fixes the issue. This corresponds to the explanation #Evk provided to why using SpinUntil doesn't work.
I've changed the function to use WebDriverWait instead (as shown in this answer to a different question) and it now works correctly. This removed the need to use the implicit wait timeout at all.
/// <summary>
/// Returns the (first) element that is displayed when multiple elements are found on page for the same by
/// </summary>
/// <exception cref="NoSuchElementException">Thrown when either an element is not found or none of the found elements is displayed</exception>
public static IWebElement FindDisplayedElement(By by, int secondsToWait = DEFAULT_WAIT)
{
var wait = new WebDriverWait(WebDriver, TimeSpan.FromSeconds(secondsToWait));
try
{
return wait.Until(condition =>
{
return WebDriver.FindElements(by).ToList().FirstOrDefault(x => x.Displayed == true);
});
}
catch (WebDriverTimeoutException ex)
{
throw new NoSuchElementException("Could not find visible element with by: " + by.ToString(), ex);
}
}
We have a running service which processes messages from system x to system y.
It basically looks as follows:
aSystem.Messages.Subscribe(message =>
{
try
{
ProcessMessage(message);
}
catch(Exception ex)
{
_logger.LogFatal(ex.Message);
}
})
The problem is that we receive at least one message every second and our LogFatal message is configured to send an email. As a result, the mailbox exploded in a certain moment.
The code is "improved" by adding a custom Logging class which will hold the last timestamp. Based on that timestamp the message would log or not.
This looks cumbersome and I think this is the perfect scenario for Rx.NET. What we need is the following:
1) Log if the string changes
2) Log if a certain time passed
What I tried is the following:
var logSubject = new Subject<string>();
var logMessagesChangedStream = logSubject.DistinctUntilChanged(); // log on every message change
var logMessagesSampleStream = logSubject.Sample(TimeSpan.FromSeconds(10)); // log at least every 10 seconds
var subscription = logMessagesChangedStream.Merge(logMessagesSampleStream).Subscribe(result =>
{
_logger.LogFatal(result);
});
aSystem.Messages.Subscribe(message =>
{
try
{
ProcessMessage(message);
}
catch(Exception ex)
{
logSubject.OnNext(ex.Message);
}
})
It looks like it's working, but this will log the message twice, once for the DistinctUntilChanged and once for the Sample. So somehow I should reset the streams if one of them emitted a value. They work perfect independently, once merged they should listen to each other ;-)
There's the ambiguous operator Amb which races two sequences to see which wins first.
Observable.Amb(logMessagesChangedStream, logMessagesSampleStream)
The winning stream continues to propagate to the end - we don't want that. We're interested in starting the race again for the next value. Let's do that:
Observable.Amb(logMessagesChangedStream, logMessagesSampleStream)
.Take(1)
.Repeat()
Now the last problem is that DistinctUntilChanged loses its state every time we restart the race, and its behavior is to push the very first value it gets immediately. Let's fix that by turning it into a hot observable.
logSubject.DistinctUntilChanged().Publish();
Putting it all together:
var logSubject = new Subject<string>();
var logMessagesChangedStream = logSubject.DistinctUntilChanged().Publish(); // log on every message change
var logMessagesSampleStream = logSubject.Sample(TimeSpan.FromSeconds(5)); // log at least every 5 seconds
var oneof =
Observable
.Amb(logMessagesChangedStream, logMessagesSampleStream)
.Take(1)
.Repeat();
logMessagesChangedStream.Connect();
oneof.Timestamp().Subscribe(c => Console.WriteLine(c));
Changed the 10 seconds to 5, because.
Test
Action<string> onNext = logSubject.OnNext;
onNext("a");
onNext("b");
Delay(1000, () => { onNext("c"); onNext("c"); onNext("c"); onNext("d"); });
Delay(3000, () => { onNext("d"); onNext("e"); });
Delay(6000, () => { onNext("e"); });
Delay(10000, () => { onNext("e"); });
Output
a#1/30/2020 12:17:52 PM +00:00
b#1/30/2020 12:17:52 PM +00:00
c#1/30/2020 12:17:53 PM +00:00
d#1/30/2020 12:17:53 PM +00:00
e#1/30/2020 12:17:55 PM +00:00
e#1/30/2020 12:18:00 PM +00:00
e#1/30/2020 12:18:05 PM +00:00
I had a function which update the database by every second (as continuously data coming by some Network) I wanted to put delay on that updating function.. As it would update database table by every 5 minutes..
Here is my Code
if (ip==StrIp)
{
Task.Delay(300000).ContinueWith(_=>
{ //I'm Using Task.Delay to make delay
var res= from i in dc.Pins //LINQ Query
where i.ip== ip
select i;
for each (var p in res)
{
p.time= System.DateTime.Now,
p.temperature= temp,
.
. //some other values
.
};
datacontext.submitChanges();
});
}
It is working and updating data by every 5 minutes, Now I want that data should update immediately only first time when application start but after that It should update after every 5 minutes.. But Right now my code isn't doing that..
How can I make such delay which ignore the operation first time, but apply on upcoming data iterations..?
Thanks in Advance
You could use a flag to determine whether it is the first time your method is called, e.g.:
private uint _counter = 0;
public YourMethod()
{
if (ip == StrIp)
{
Action<Task> action = _ =>
{
var res = from i in dc.Pins //LINQ Query
where i.ip == ip
select i;
//...
datacontext.submitChanges();
};
if (_counter++ == 0)
action();
else
Task.Delay(300000).ContinueWith(action);
}
}
Extract the inner logic of the task into a function/method (refactoring of VS or R# can to this automatically) and call the new function/method at start and on the interval.
I personally would go into another direction:
Have a in-memory queue that gets filled with data as it comes into your app. Then I would have a thread/task etc. which checks the queue every 5 minutes and updates the database accordingly. Remember to lock the queue for updates (concurrency). The ConcurrentQueue of .Net is one way to do it.
FWIW - I'm scrapping the previous version of this question in favor of different one along the same way after asking for advice on meta
I have a webservice that contains configuration data. I would like to call it at regular intervals Tok in order to refresh the configuration data in the application that uses it. If the service is in error (timeout, down, etc) I want to keep the data from the previous call and call the service again after a different time interval Tnotok. Finally I want the behavior to be testable.
Since managing time sequences and testability seems like a strong point of the Reactive Extensions, I started using an Observable that will be fed by a generated sequence. Here is how I create the sequence:
Observable.Generate<DataProviderResult, DataProviderResult>(
// we start with some empty data
new DataProviderResult() {
Failures = 0
, Informations = new List<Information>()},
// never stop
(r) => true,
// there is no iteration
(r) => r,
// we get the next value from a call to the webservice
(r) => FetchNextResults(r),
// we select time for next msg depending on the current failures
(r) => r.Failures > 0 ? tnotok : tok,
// we pass a TestScheduler
scheduler)
.Suscribe(r => HandleResults(r));
I have two problems currently:
It looks like I am creating a hot observable. Even trying to use Publish/Connect I have the suscribed action missing the first event. How can I create it as a cold observable?
myObservable = myObservable.Publish();
myObservable.Suscribe(r => HandleResults(r));
myObservable.Connect() // doesn't call onNext for first element in sequence
When I suscribe, the order in which the suscription and the generation seems off, since for any frame the suscription method is fired before the FetchNextResults method. Is it normal? I would expect the sequence to call the method for frame f, not f+1.
Here is the code that I'm using for fetching and suscription:
private DataProviderResult FetchNextResults(DataProviderResult previousResult)
{
Console.WriteLine(string.Format("Fetching at {0:hh:mm:ss:fff}", scheduler.Now));
try
{
return new DataProviderResult() { Informations = dataProvider.GetInformation().ToList(), Failures = 0};
}
catch (Exception)
{}
previousResult.Failures++;
return previousResult;
}
private void HandleResults(DataProviderResult result)
{
Console.WriteLine(string.Format("Managing at {0:hh:mm:ss:fff}", scheduler.Now));
dataResult = result;
}
Here is what I'm seeing that prompted me articulating these questions:
Starting at 12:00:00:000
Fetching at 12:00:00:000 < no managing the result that has been fetched here
Managing at 12:00:01:000 < managing before fetching for frame f
Fetching at 12:00:01:000
Managing at 12:00:02:000
Fetching at 12:00:02:000
EDIT: Here is a bare bones copy-pastable program that illustrates the problem.
/*using System;
using System.Reactive.Concurrency;
using System.Reactive.Linq;
using Microsoft.Reactive.Testing;*/
private static int fetchData(int i, IScheduler scheduler)
{
writeTime("fetching " + (i+1).ToString(), scheduler);
return i+1;
}
private static void manageData(int i, IScheduler scheduler)
{
writeTime("managing " + i.ToString(), scheduler);
}
private static void writeTime(string msg, IScheduler scheduler)
{
Console.WriteLine(string.Format("{0:mm:ss:fff} {1}", scheduler.Now, msg));
}
private static void Main(string[] args)
{
var scheduler = new TestScheduler();
writeTime("start", scheduler);
var datas = Observable.Generate<int, int>(fetchData(0, scheduler),
(d) => true,
(d) => fetchData(d, scheduler),
(d) => d,
(d) => TimeSpan.FromMilliseconds(1000),
scheduler)
.Subscribe(i => manageData(i, scheduler));
scheduler.AdvanceBy(TimeSpan.FromMilliseconds(3000).Ticks);
}
This outputs the following:
00:00:000 start
00:00:000 fetching 1
00:01:000 managing 1
00:01:000 fetching 2
00:02:000 managing 2
00:02:000 fetching 3
I don't understand why the managing of the first element is not picked up immediately after its fetching. There is one second between the sequence effectively pulling the data and the data being handed to the observer. Am I missing something here or is it expected behavior? If so is there a way to have the observer react immediately to the new value?
You are misunderstanding the purpose of the timeSelector parameter. It is called each time a value is generated and it returns a time which indicates how long to delay before delivering that value to observers and then generating the next value.
Here's a non-Generate way to tackle your problem.
private DataProviderResult FetchNextResult()
{
// let exceptions throw
return dataProvider.GetInformation().ToList();
}
private IObservable<DataProviderResult> CreateObservable(IScheduler scheduler)
{
// an observable that produces a single result then completes
var fetch = Observable.Defer(
() => Observable.Return(FetchNextResult));
// concatenate this observable with one that will pause
// for "tok" time before completing.
// This observable will send the result
// then pause before completing.
var fetchThenPause = fetch.Concat(Observable
.Empty<DataProviderResult>()
.Delay(tok, scheduler));
// Now, if fetchThenPause fails, we want to consume/ignore the exception
// and then pause for tnotok time before completing with no results
var fetchPauseOnErrors = fetchThenPause.Catch(Observable
.Empty<DataProviderResult>()
.Delay(tnotok, scheduler));
// Now, whenever our observable completes (after its pause), start it again.
var fetchLoop = fetchPauseOnErrors.Repeat();
// Now use Publish(initialValue) so that we remember the most recent value
var fetchLoopWithMemory = fetchLoop.Publish(null);
// YMMV from here on. Lets use RefCount() to start the
// connection the first time someone subscribes
var fetchLoopAuto = fetchLoopWithMemory.RefCount();
// And lets filter out that first null that will arrive before
// we ever get the first result from the data provider
return fetchLoopAuto.Where(t => t != null);
}
public MyClass()
{
Information = CreateObservable();
}
public IObservable<DataProviderResult> Information { get; private set; }
Generate produces cold observable sequences, so that is my first alarm bell.
I tried to pull your code into linqpad* and run it and changed it a bit to focus on the problem. It seems to me that you have the Iterator and ResultSelector functions confused. These are back-to-front. When you iterate, you should take the value from your last iteration and use it to produce your next value. The result selector is used to pick off (Select) the value form the instance you are iterating on.
So in your case, the type you are iterating on is the type you want to produce values of. Therefore keep your ResultSelector function just the identity function x=>x, and your IteratorFunction should be the one that make the WebService call.
Observable.Generate<DataProviderResult, DataProviderResult>(
// we start with some empty data
new DataProviderResult() {
Failures = 0
, Informations = new List<Information>()},
// never stop
(r) => true,
// we get the next value(iterate) by making a call to the webservice
(r) => FetchNextResults(r),
// there is no projection
(r) => r,
// we select time for next msg depending on the current failures
(r) => r.Failures > 0 ? tnotok : tok,
// we pass a TestScheduler
scheduler)
.Suscribe(r => HandleResults(r));
As a side note, try to prefer immutable types instead of mutating values as you iterate.
*Please provide an autonomous working snippet of code so people can better answer your question. :-)