let's say you have an IObservable<T> that may supply a few values immediately, and some being pushed continously:
var immediate_values = new [] { "curerntly", "available", "values" }.ToObservable();
var future_values = Observable.Timer(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(1)).Select(x => "new value!");
IObservable<string> input = immediate_values.Concat(future_values);
Is there any way to transform input into an IObservable<string[]>, where the first array being pushed consists of all immediately available values, and each subsequent array consists of only 1 value (each one being pushed thereafter)?
Above is just example data naturally, this would need to work on any IObservable>T> without knowing the individual input streams.
IObservable<string[]> buffered = input.BufferSomehow();
// should push values:
// First value: string[] = ["currently", "available", "values"]
// Second value: string[] = ["new value!"]
// Third value: string[] = ["new value!"]
// .....
I've thought of the .Buffer() function of course, but I don't really want to buffer by any particular TimeSpan, and can't think of any way to produce an observable with buffer window closing signals.
Can anyone think of a reasonable way to achieve this, or is this not really possible at all?
Thanks!
There is no direct way to distinguish between the on-start-up values of an observable and the subsequent values. My suggestion would be to infer it:
var autoBufferedInput1 = input.Publish(_input => _input
.Buffer(_input.Throttle(TimeSpan.FromSeconds(.1)))
.Select(l => l.ToArray())
);
This sets your buffer boundary to a rolling, extending window of .1 seconds: Each time a value comes in, it extends the window to .1 seconds from the time the value came in, and adds the value to the buffer. If .1 seconds go by with no values, then the buffer is flushed out.
This will have the side-effect that if you have near-simultaneous "hot" values (within .1 seconds of each other), then those will be buffered together. If that's undesired, you can Switch out, though that makes things more complicated:
var autoBufferedInput2 = input.Publish(_input =>
_input.Throttle(TimeSpan.FromSeconds(.1)).Publish(_boundary => _boundary
.Take(1)
.Select(_ => _input.Select(s => new[] { s }))
.StartWith(_input
.Buffer(_boundary)
.Select(l => l.ToArray())
)
.Switch()
)
);
autoBufferedInput2 uses the .1 second inference method until the first buffered list, then switches to simply selecting out and wrapping values in an array.
EDIT: If you want an absolute 1 second gate as well, then the snippets would look like this:
var autoBufferedInput1 = input.Publish(_input => _input
.Buffer(
Observable.Merge(
Observable.Timer(TimeSpan.FromSeconds(1)).Select(_ => Unit.Default),
_input.Throttle(TimeSpan.FromSeconds(.1)).Select(_ => Unit.Default)
)
)
.Select(l => l.ToArray())
);
var autoBufferedInput2 = input.Publish(_input =>
Observable.Merge(
_input.Throttle(TimeSpan.FromSeconds(.1)).Select(_ => Unit.Default),
Observable.Timer(TimeSpan.FromSeconds(1)).Select(_ => Unit.Default)
)
.Publish(_boundary => _boundary
.Take(1)
.Select(_ => _input.Select(s => new[] { s }))
.StartWith(_input
.Buffer(_boundary)
.Select(l => l.ToArray())
)
.Switch()
)
);
For any IObservable<T>, you'd need to do:
var sequence = ongoingSequence.StartWith(initialSequence);
You could take advantage of the fact that the immediately available values are propagated synchronously during the subscription, and toggle some switch after the Subscribe method returns. The implementation below is based on this idea. During the subscription all incoming messages are buffered, after the subscription the buffer is emitted, and after that all future incoming messages are emitted immediately one by one.
public static IObservable<T[]> BufferImmediatelyAvailable<T>(
this IObservable<T> source)
{
return Observable.Create<T[]>(observer =>
{
var buffer = new List<T>();
var subscription = source.Subscribe(x =>
{
if (buffer != null)
buffer.Add(x);
else
observer.OnNext(new[] { x });
}, ex =>
{
buffer = null;
observer.OnError(ex);
}, () =>
{
if (buffer != null)
{
var output = buffer.ToArray();
buffer = null;
observer.OnNext(output);
}
observer.OnCompleted();
});
if (buffer != null)
{
var output = buffer.ToArray();
buffer = null;
observer.OnNext(output);
}
return subscription;
});
}
Related
I am trying to convert the below statement so that I can get the key alongside the selected list:
var feed = new Subject<TradeExecuted>();
feed
.GroupByUntil(x => (x.Execution.Contract.Symbol, x.Execution.AccountId, x.Tenant, x.UserId), x => Observable.Timer(TimeSpan.FromSeconds(5)))
.SelectMany(x => x.ToList())
.Select(trades => Observable.FromAsync(() => Mediator.Publish(trades, cts.Token)))
.Concat() // Ensure that the results are serialized.
.Subscribe(cts.Token); // Check status of calls.
The above works, whereas the below does not - when I try and itterate over the list, it locks up.
feed
.GroupByUntil(x => (x.Execution.Contract.Symbol, x.Execution.AccountId, x.Tenant, x.UserId), x => Observable.Timer(timespan))
.Select(x => Observable.FromAsync(() =>
{
var list = x.ToEnumerable(); // <---- LOCK UP if we use list.First() etc
var aggregate = AggregateTrades(x.Key.Symbol, x.Key.AccountId, x.Key.Tenant, list);
return Mediator.Publish(aggregate, cts.Token);
}))
.Concat()
.Subscribe(cts.Token); // Check status of calls.
I am clearly doing something wrong and probably horrific!
Going back to the original code, how can I get the Key alongside the enumerable list (and avoiding the hack below)?
As a sidenote, the below code works but it a nasty hack where I get the keys from the first list item:
feed
.GroupByUntil(x => (x.Execution.Contract.Symbol, x.Execution.AccountId, x.Tenant, x.UserId), x => Observable.Timer(TimeSpan.FromSeconds(5)))
.SelectMany(x => x.ToList())
.Select(trades => Observable.FromAsync(() =>
{
var firstTrade = trades.First();
var aggregate = AggregateTrades(firstTrade.Execution.Contract.Symbol, firstTrade.Execution.AccountId, firstTrade.Tenant, trades);
return Mediator.Publish(aggregate, cts.Token);
}))
.Concat() // Ensure that the results are serialized.
.Subscribe(cts.Token); // Check status of calls.
All versions of your code suffer from trying to eagerly evaluate the grouped sub-observable. Since in v1 and v3 your group observable will run a maximum of 5 seconds, that isn't horrible/awful, but it's still not great. In v2, I don't know what timespan is, but assuming it's 5 seconds, you have the same problem: Trying to turn the grouped sub-observable into a list or an enumerable means waiting for the sub-observable to complete, blocking the thread (or the task).
You can fix this by using the Buffer operator to lazily evaluate the grouped sub-observable:
var timespan = TimeSpan.FromSeconds(5);
feed
.GroupByUntil(x => (x.Execution.Contract.Symbol, x.Execution.AccountId, x.Tenant, x.UserId), x => Observable.Timer(timespan))
.SelectMany(x => x
.Buffer(timespan)
.Select(list => Observable.FromAsync(() =>
{
var aggregate = AggregateTrades(x.Key.Symbol, x.Key.AccountId, x.Key.Tenant, list));
return Mediator.Publish(aggregate, cts.Token);
}))
)
.Concat() // Ensure that the results are serialized.
.Subscribe(cts.Token); // Check status of calls.
This essentially means that until timespan is up, the items in the group by gather in a list inside Buffer. Once timespan is up, they're released as a list, and the mediator publish happens.
I have the following method which retrieves sales data from the db.
The Db returns the data in 1.5 to 1.9 seconds, which I'd like to be faster but one step at a time
I have to manipulate the data here in the method because I couldn't figure out how to return it in the final form from the query, which is certainly the most efficient way to do so.
Since the results (results type is a List of AuctionDatum )count is rarely going to be more than 1-200 results, I figured this method would suffice for now...NOT
The db runs as noted above BUT the loop takes almost 4-5 seconds, and for this testing it was only 25 records!! (4 of the 25 required conversion to $, I didnt measure that time for that method)
Method
public async Task<CarSalesData> GetCarSalesDataAsync(int modelId)
{
var stopwatch = new Stopwatch();
stopwatch.Start();
var results = await GetVehicleAuctionDataForModelAsync(modelId);
var cnt = results.Count;
var csd = new CarSalesData
{
ModelName =
$"{results[0].Vehicle.Model.Manufacturer.ManufacturerName} {results[0].Vehicle.Model.ModelYear} {results[0].Vehicle.Model.ModelName}"
};
var dates = new List<string>();
var values = new List<double>();
stopwatch.Stop();
var elt = stopwatch.Elapsed;
stopwatch.Start();
foreach (var result in results)
{
if (result.AuctionCurrency == "USD")
{
dates.Add(result.AuctionDate.ToString("MM/dd/yyyy"));
values.Add(Convert.ToDouble(result.SoldPrice));
}
else
{
dates.Add(result.AuctionDate.ToString("MM/dd/yyyy"));
values.Add(Convert.ToDouble(VehicleManagerHelpers.GetPriceInUSD(Convert.ToDecimal(result.SoldPrice), result.AuctionCurrency, result.AuctionDate))); }
}
csd.SaleDates = dates;
csd.SalesValues = values;
stopwatch.Stop();
var elt1 = stopwatch.Elapsed;
//the value of err below is:: The data retrieval method took 01.8641257 seconds and the loop took 05.5960528 seconds
var err = $"The data retrieval method took {elt} seconds and the loop took {elt1} seconds";
return csd;
}
DB method
public async Task<IList<VehicleAuctionData>> GetVehicleAuctionDataForModelAsync(int modelId)
{
return await _context.AuctionDatum
.Include(u => u.Vehicle)
.Include(u => u.Vehicle.Model)
.Include(u => u.Vehicle.Model.Manufacturer)
.Where(u => u.Vehicle.ModelId == modelId && u.SoldPrice !=null)
.OrderBy(u => u.AuctionDate)
.ToListAsync();
}
Any help in getting this to run more efficiently would be appreciated.
If you only need a few properties, create a model that contains those properties you need, and select into it. For example:
public async Task<IList<VehicleAuctionData>> GetVehicleAuctionDataForModelAsync(int modelId)
{
return await _context.AuctionDatum
.Include(u => u.Vehicle)
.Include(u => u.Vehicle.Model)
.Include(u => u.Vehicle.Model.Manufacturer)
.Where(u => u.Vehicle.ModelId == modelId && u.SoldPrice !=null)
.Select(u => new SmallerObject{
Manufacturer = u.Vehicle.Model.Manufacturer.Name,
AuctionDate = u.AuctionDate,
AuctionCurrency = u.AuctionCurrency,
// add properties as needed here
})
.OrderBy(u => u.AuctionDate)
.ToListAsync();
}
With Linq, the query will then be trimmed to only return those values needed to satisfy the output. Depending on how heavy your objects are, this might reduce the amount of data coming back from the db.
I have an observable that emits unique values e.g.
var source=Observable.Range(1,100).Publish();
source.Connect();
I want to observe its values from e.g. two observers but each observer to get notified only for values not seen in other observers.
So if first observer contains the value 10 the second observer should never get notified for the 10 value.
Update
I chose #Asti`s answer cause it was first and although buggy it pointed to the right direction and up-voted #Shlomo's answer. Too bad I cannot accept both answers as #Shlomo answer was more correct and I really appreciate all his help we get on this tag.
Observables aren't supposed to behave differently for different observers; a better approach would be to give each observer its own filtered observable.
That being said, if your constraints require that you need this behavior in a single observable - we can use a Round-Robin method.
public static IEnumerable<T> Repeat<T>(this IEnumerable<T> source)
{
for (; ; )
foreach (var item in source.ToArray())
yield return item;
}
public static IObservable<T> RoundRobin<T>(this IObservable<T> source)
{
var subscribers = new List<IObserver<T>>();
var shared = source
.Zip(subscribers.Repeat(), (value, observer) => (value, observer))
.Publish()
.RefCount();
return Observable.Create<T>(observer =>
{
subscribers.Add(observer);
var subscription =
shared
.Where(pair => pair.observer == observer)
.Select(pair => pair.value)
.Subscribe(observer);
var dispose = Disposable.Create(() => subscribers.Remove(observer));
return new CompositeDisposable(subscription, dispose);
});
}
Usage:
var source = Observable.Range(1, 100).Publish();
var dist = source.RoundRobin();
dist.Subscribe(i => Console.WriteLine($"One sees {i}"));
dist.Subscribe(i => Console.WriteLine($"Two sees {i}"));
source.Connect();
Result:
One sees 1
Two sees 2
One sees 3
Two sees 4
One sees 5
Two sees 6
One sees 7
Two sees 8
One sees 9
Two sees 10
If you already have a list of observers, the code becomes much simpler.
EDIT: #Asti fixed his bug, and I fixed mine based on his answer. Our answers are now largely similar. I have an idea how to do a purely reactive one, if I have time I'll post that later.
Fixed code:
public static IObservable<T> RoundRobin2<T>(this IObservable<T> source)
{
var subscribers = new BehaviorSubject<ImmutableList<IObserver<T>>>(ImmutableList<IObserver<T>>.Empty);
ImmutableList<IObserver<T>> latest = ImmutableList<IObserver<T>>.Empty;
subscribers.Subscribe(l => latest = l);
var shared = source
.Select((v, i) => (v, i))
.WithLatestFrom(subscribers, (t, s) => (t.v, t.i, s))
.Publish()
.RefCount();
return Observable.Create<T>(observer =>
{
subscribers.OnNext(latest.Add(observer));
var dispose = Disposable.Create(() => subscribers.OnNext(latest.Remove(observer)));
var sub = shared
.Where(t => t.i % t.s.Count == t.s.FindIndex(o => o == observer))
.Select(t => t.v)
.Subscribe(observer);
return new CompositeDisposable(dispose, sub);
});
}
Original answer:
I upvoted #Asti's answer, because he's largely correct: Just because you can, doesn't mean you should. And his answer largely works, but it's subject to a bug:
This works fine:
var source = Observable.Range(1, 20).Publish();
var dist = source.RoundRobin();
dist.Subscribe(i => Console.WriteLine($"One sees {i}"));
dist.Take(1).Subscribe(i => Console.WriteLine($"Two sees {i}"));
This doesn't:
var source = Observable.Range(1, 20).Publish();
var dist = source.RoundRobin();
dist.Take(1).Subscribe(i => Console.WriteLine($"One sees {i}"));
dist.Subscribe(i => Console.WriteLine($"Two sees {i}"));
Output is:
One sees 1
Two sees 1
Two sees 2
Two sees 3
Two sees 4
...
I first thought the bug is Halloween related, but now I'm not sure. The .ToArray() in Repeat should take care of that. I also wrote a pure-ish observable implementation which has the same bug. This implementation doesn't guarantee a perfect Round Robin, but that wasn't in the question:
public static IObservable<T> RoundRobin2<T>(this IObservable<T> source)
{
var subscribers = new BehaviorSubject<ImmutableList<IObserver<T>>>(ImmutableList<IObserver<T>>.Empty);
ImmutableList<IObserver<T>> latest = ImmutableList<IObserver<T>>.Empty;
subscribers.Subscribe(l => latest = l);
var shared = source
.Select((v, i) => (v, i))
.WithLatestFrom(subscribers, (t, s) => (t.v, t.i, s))
.Publish()
.RefCount();
return Observable.Create<T>(observer =>
{
subscribers.OnNext(latest.Add(observer));
var dispose = Disposable.Create(() => subscribers.OnNext(latest.Remove(observer)));
var sub = shared
.Where(t => t.i % t.s.Count == t.s.FindIndex(o => o == observer))
.Select(t => t.v)
.Subscribe(observer);
return new CompositeDisposable(dispose, sub);
});
}
This is a simple distributed queue implementation using TPL Dataflow. But with respect to different observers not seeing the same value, there's little chance of it behaving incorrectly. It's not round-robin, but actually has back-pressure semantics.
public static IObservable<T> Distribute<T>(this IObservable<T> source)
{
var buffer = new BufferBlock<T>();
source.Subscribe(buffer.AsObserver());
return Observable.Create<T>(observer =>
buffer.LinkTo(new ActionBlock<T>(observer.OnNext, new ExecutionDataflowBlockOptions { BoundedCapacity = 1 })
);
}
Output
One sees 1
Two sees 2
One sees 3
Two sees 4
One sees 5
One sees 6
One sees 7
One sees 8
One sees 9
One sees 10
I might prefer skipping Rx entirely and just using TPL Dataflow.
I have a stream with letters (A-Z) and numbers (1-9). I do want to join letters that arrive within a timeout (this can change) and always emit numbers immediately. Can you suggest me which functions are best to do this?
Sample working code (not sure this is correct and/or a good solution):
private BehaviorSubject<TimeSpan> sTimeouts = new BehaviorSubject<TimeSpan>(0.ms());
private IObservable<string> lettersJoined(IObservable<char> ob)
{
return Observable.Create<string>(observer =>
{
var letters = new List<char>();
var lettersFlush = new SerialDisposable();
return ob.Subscribe(c =>
{
if (char.IsUpper(c))
{
if ((await sTimeouts.FirstAsync()).Ticks > 0)
{
letters.Add(c);
lettersFlush.Disposable =
VariableTimeout(sTimeouts)
.Subscribe(x => {
observer.OnNext(String.Concat(letters));
letters.Clear();
});
}
else
observer.OnNext(letters.ToString());
}
else if (char.IsDigit(c))
observer.OnNext(c.ToString());
}
}
}
private IObservable<long> VariableTimeout(IObservable<TimeSpan> timeouts)
{
return Observable.Create<long>(obs =>
{
var sd = new SerialDisposable();
var first = DateTime.Now;
return timeouts
.Subscribe(timeout =>
{
if (timeout.Ticks == 0 || first + timeout < DateTime.Now)
{
sd.Disposable = null;
obs.OnNext(timeout.Ticks);
obs.OnCompleted();
}
else
{
timeout -= DateTime.Now - first;
sd.Disposable =
Observable
.Timer(timeout)
.Subscribe(t => {
obs.OnNext(t);
obs.OnCompleted();
});
}
});
});
}
private void ChangeTimeout(int timeout)
{
sTimeouts.OnNext(timeout.ms())
}
// I use the following extension method
public static class TickExtensions
{
public static TimeSpan ms(this int ms)
{
return TimeSpan.FromMilliseconds(ms);
}
}
To modify the timeout, I can simply change the private timeout variable, but probably a Subject for it would be OK if needed/better.
UPDATE
var scheduler = new TestScheduler();
var timeout = scheduler.CreateColdObservable<int>(
ReactiveTest.OnNext(0000.Ms(), 2000),
ReactiveTest.OnNext(4300.Ms(), 1000));
var input = scheduler.CreateColdObservable<char>(
ReactiveTest.OnNext(0100.Ms(), '1'),
ReactiveTest.OnNext(1600.Ms(), '2'),
ReactiveTest.OnNext(1900.Ms(), 'A'),
ReactiveTest.OnNext(2100.Ms(), 'B'),
ReactiveTest.OnNext(4500.Ms(), 'C'),
ReactiveTest.OnNext(5100.Ms(), 'A'),
ReactiveTest.OnNext(5500.Ms(), '5'),
ReactiveTest.OnNext(6000.Ms(), 'B'),
ReactiveTest.OnNext(7200.Ms(), '1'),
ReactiveTest.OnNext(7500.Ms(), 'B'),
ReactiveTest.OnNext(7700.Ms(), 'A'),
ReactiveTest.OnNext(8400.Ms(), 'A'));
var expected = scheduler.CreateColdObservable<string>(
ReactiveTest.OnNext(0100.Ms(), "1"),
ReactiveTest.OnNext(1600.Ms(), "2"),
ReactiveTest.OnNext(4100.Ms(), "AB"),
ReactiveTest.OnNext(5500.Ms(), "5"),
ReactiveTest.OnNext(7000.Ms(), "CAB"),
ReactiveTest.OnNext(7200.Ms(), "1"),
ReactiveTest.OnNext(9400.Ms(), "BAA"));
// if ReactiveTest.OnNext(3800.Ms(), 1000)
// then expected is ReactiveTest.OnNext(3800.Ms(), "AB")
UPDATE #2
Refined solution correctly supporting timeout change during buffering
Several things that may help out here.
First marble diagrams are nice for helping visualize the problem, but when proving if something works or not, lets be prescriptive and unit test with ITestableObservable<T> instances.
Second, I am not sure what your solution should be. If I look at your marble diagrams I see some discrepancies. Here I have added a Timeline to help visualize.
111111111122222222223
Time: 123456789012345678901234567890
Input: 1---2--A-B----C--A-B-1--B-A--A
Output: 1---2----AB-------CAB-1-----BAA
Here I see the "AB" output published at unit 10.
Then I see the "CAB" output published at unit 19.
Further I see the "BAA" output published at unit 29.
But you suggest these should occur at constant timeouts apart.
So then I think it is maybe the gap between values that is important, but this doesn't seem to add up either. This just leads me back to my point above, please provide a unit test that could pass or fail.
Thirdly, regarding your implementation you could make it slightly better by using the SerialDisposable type for the lettersFlush type.
To help me set up the unit test I create the following block of code
var scheduler = new TestScheduler();
var input = scheduler.CreateColdObservable<char>(
ReactiveTest.OnNext(0100.Ms(), '1'),
ReactiveTest.OnNext(0500.Ms(), '2'),
ReactiveTest.OnNext(0800.Ms(), 'A'),
ReactiveTest.OnNext(1000.Ms(), 'B'),
ReactiveTest.OnNext(1500.Ms(), 'C'),
ReactiveTest.OnNext(1800.Ms(), 'A'),
ReactiveTest.OnNext(2000.Ms(), 'B'),
ReactiveTest.OnNext(2200.Ms(), '1'),
ReactiveTest.OnNext(2500.Ms(), 'B'),
ReactiveTest.OnNext(2700.Ms(), 'A'),
ReactiveTest.OnNext(3000.Ms(), 'A'));
var expected = scheduler.CreateColdObservable<string>(
ReactiveTest.OnNext(0100.Ms(), "1"),
ReactiveTest.OnNext(0500.Ms(), "2"),
ReactiveTest.OnNext(1000.Ms(), "AB"),
ReactiveTest.OnNext(2000.Ms(), "CAB"),
ReactiveTest.OnNext(2200.Ms(), "1"),
ReactiveTest.OnNext(3000.Ms(), "BAA"));
I have taken some liberties to change some values to what I think you meant by your marble diagrams.
If I then use the very good answer provided above by #Shlomo, I can see further issues with just using fuzzy marble diagrams. As the buffer boundary would have to happen after the last value to be included occurs, these windows need to close on an off-by-one time.
void Main()
{
var scheduler = new TestScheduler();
var input = scheduler.CreateColdObservable<char>(
ReactiveTest.OnNext(0100.Ms(), '1'),
ReactiveTest.OnNext(0500.Ms(), '2'),
ReactiveTest.OnNext(0800.Ms(), 'A'),
ReactiveTest.OnNext(1000.Ms(), 'B'),
ReactiveTest.OnNext(1500.Ms(), 'C'),
ReactiveTest.OnNext(1800.Ms(), 'A'),
ReactiveTest.OnNext(2000.Ms(), 'B'),
ReactiveTest.OnNext(2200.Ms(), '1'),
ReactiveTest.OnNext(2500.Ms(), 'B'),
ReactiveTest.OnNext(2700.Ms(), 'A'),
ReactiveTest.OnNext(3000.Ms(), 'A'));
var expected = scheduler.CreateColdObservable<string>(
ReactiveTest.OnNext(0100.Ms(), "1"),
ReactiveTest.OnNext(0500.Ms(), "2"),
ReactiveTest.OnNext(1000.Ms()+1, "AB"),
ReactiveTest.OnNext(2000.Ms()+1, "CAB"),
ReactiveTest.OnNext(2200.Ms(), "1"),
ReactiveTest.OnNext(3000.Ms()+1, "BAA"));
/*
111111111122222222223
Time: 123456789012345678901234567890
Input: 1---2--A-B----C--A-B-1--B-A--A
Output: 1---2----AB-------CAB-1-----BAA
*/
var bufferBoundaries = //Observable.Timer(TimeSpan.FromSeconds(1), scheduler);
//Move to a hot test sequence to force the windows to close just after the values are produced
scheduler.CreateHotObservable<Unit>(
ReactiveTest.OnNext(1000.Ms()+1, Unit.Default),
ReactiveTest.OnNext(2000.Ms()+1, Unit.Default),
ReactiveTest.OnNext(3000.Ms()+1, Unit.Default),
ReactiveTest.OnNext(4000.Ms()+1, Unit.Default));
var publishedFinal = input
.Publish(i => i
.Where(c => char.IsLetter(c))
.Buffer(bufferBoundaries)
.Where(l => l.Any())
.Select(lc => new string(lc.ToArray()))
.Merge(i
.Where(c => char.IsNumber(c))
.Select(c => c.ToString())
)
);
var observer = scheduler.CreateObserver<string>();
publishedFinal.Subscribe(observer);
scheduler.Start();
//This test passes with the "+1" values hacked in.
ReactiveAssert.AreElementsEqual(
expected.Messages,
observer.Messages);
}
// Define other methods and classes here
public static class TickExtensions
{
public static long Ms(this int ms)
{
return TimeSpan.FromMilliseconds(ms).Ticks;
}
}
I suppose my point is that Rx is deterministic, therefore we can create tests that are deterministic. So while your question is a very good one, and I do believe that #Shlomo provides a solid final answer, we can do better than just fuzzy marble diagrams and using Random in our examples/tests.
Being precise here should help prevent silly race conditions in production, and help reader better understand these solutions.
Assuming sampleInput as your sample Input:
var charStream = "12ABCAB1BAA".ToObservable();
var random = new Random();
var randomMilliTimings = Enumerable.Range(0, 12)
.Select(i => random.Next(2000))
.ToList();
var sampleInput = charStream
.Zip(randomMilliTimings, (c, ts) => Tuple.Create(c, TimeSpan.FromMilliseconds(ts)))
.Select(t => Observable.Return(t.Item1).Delay(t.Item2))
.Concat();
First, instead of changing a mutable variable, it would be best to instead generate some stream to represent your buffer windows:
Input: 1---2--A-B----C--A-B-1--B-A--A
Window: ---------*--------*---------*--
Output: 1---2----AB-------CAB-1-----BAA
I generated a stream of incrementing TimeSpans and called it bufferBoundaries like so to demonstrate:
var bufferBoundaries = Observable.Range(1, 20)
.Select(t => Observable.Return(t).Delay(TimeSpan.FromSeconds(t)))
.Concat();
This would look like this:
Seconds: 0--1--2--3--4--5--6--7--8--9--10
BB : ---1-----2--------3-----------4-
... next you want to split that sampleInput up into separate streams for letters and numbers, and handle them accordingly:
var letters = sampleInput
.Where(c => char.IsLetter(c))
.Buffer(bufferBoundaries)
.Where(l => l.Any())
.Select(lc => new string(lc.ToArray()));
var numbers = sampleInput
.Where(c => char.IsNumber(c))
.Select(c => c.ToString());
Next, merge the two streams together:
var finalOutput = letters.Merge(numbers);
Lastly, it's generally not a good idea to subscribe twice to the same input (in our case, sampleInput) if you can help it. So in our case, we should replace letters, numbers, and finalOutput with the following:
var publishedFinal = sampleInput
.Publish(_si => _si
.Where(c => char.IsLetter(c))
.Buffer(bufferBoundaries)
.Where(l => l.Any())
.Select(lc => new string(lc.ToArray()))
.Merge( _si
.Where(c => char.IsNumber(c))
.Select(c => c.ToString())
)
);
I'm trying to combine two observables whose values share some key.
I want to produce a new value whenever the first observable produces a new value, combined with the latest value from a second observable which selection depends on the latest value from the first observable.
pseudo code example:
var obs1 = Observable.Interval(TimeSpan.FromSeconds(1)).Select(x => Tuple.create(SomeKeyThatVaries, x)
var obs2 = Observable.Interval(TimeSpan.FromMilliSeconds(1)).Select(x => Tuple.create(SomeKeyThatVaries, x)
from x in obs1
let latestFromObs2WhereKeyMatches = …
select Tuple.create(x, latestFromObs2WhereKeyMatches)
Any suggestions?
Clearly this could be implemented by subcribing to the second observable and creating a dictionary with the latest values indexable by the key. But I'm looking for a different approach..
Usage scenario: one minute price bars computed from a stream of stock quotes. In this case the key is the ticker and the dictionary contains latest ask and bid prices for concrete tickers, which are then used in the computation.
(By the way, thank you Dave and James this has been a very fruitful discussion)
(sorry about the formatting, hard to get right on an iPad..)
...why are you looking for a different approach? Sounds like you are on the right lines to me. It's short, simple code... roughly speaking it will be something like:
var cache = new ConcurrentDictionary<long, long>();
obs2.Subscribe(x => cache[x.Item1] = x.Item2);
var results = obs1.Select(x => new {
obs1 = x.Item2,
cache.ContainsKey(x.Item1) ? cache[x.Item1] : 0
});
At the end of the day, C# is an OO language and the heavy lifting of the thread-safe mutable collections is already all done for you.
There may be fancy Rx approach (feels like joins might be involved)... but how maintainable will it be? And how will it perform?
$0.02
I'd like to know the purpose of a such a query. Would you mind describing the usage scenario a bit?
Nevertheless, it seems like the following query may solve your problem. The initial projections aren't necessary if you already have some way of identifying the origin of each value, but I've included them for the sake of generalization, to be consistent with your extremely abstract mode of questioning. ;-)
Note: I'm assuming that someKeyThatVaries is not shared data as you've shown it, which is why I've also included the term anotherKeyThatVaries; otherwise, the entire query really makes no sense to me.
var obs1 = Observable.Interval(TimeSpan.FromSeconds(1))
.Select(x => Tuple.Create(someKeyThatVaries, x));
var obs2 = Observable.Interval(TimeSpan.FromSeconds(.25))
.Select(x => Tuple.Create(anotherKeyThatVaries, x));
var results = obs1.Select(t => new { Key = t.Item1, Value = t.Item2, Kind = 1 })
.Merge(
obs2.Select(t => new { Key = t.Item1, Value = t.Item2, Kind = 2 }))
.GroupBy(t => t.Key, t => new { t.Value, t.Kind })
.SelectMany(g =>
g.Scan(
new { X = -1L, Y = -1L, Yield = false },
(acc, cur) => cur.Kind == 1
? new { X = cur.Value, Y = acc.Y, Yield = true }
: new { X = acc.X, Y = cur.Value, Yield = false })
.Where(s => s.Yield)
.Select(s => Tuple.Create(s.X, s.Y)));