Develop Reference

Get a several counts using the same state of a list

I have code like this:
count = taskList.Count(x => x.IsCompleted);
successfulCount = taskList.Count(x => x.IsCompleted && x.Result == true);
failedCount = taskList.Count(x => x.IsCompleted && x.Result == false);
I then use it like this:
Console.Write($"\r {count} out of {calls} calls made. {successfulCount} successful, {failedCount} failed.");
The problem is that the taskList is being updated in another thread. So I end up getting messages like this:
48 of 777843 calls made. 96 successful, 0 failed
I know why it is doing this. Between the time that I calculated the count and when I calculated the successfulCount, I had more tasks complete.
But it is still a bit odd to show to my tester (this is for a test harness), so, if it is possible to fix without slowing down what I am observing, then I would like to fix it.
Is there a way to issue 3 count calls in one query so it is more likely to have a consistent result? (I does not seem likely, but I thought I would ask.)

You could do the following
var result = taskList.Where(x => x.IsCompleted)
.GroupBy(x => x.Result)
.ToDictionary(x => x.Key, x => x.Count());
var successfulCount = result.TryGetValue(true, out var success) ? success : 0;
var failedCount = result.TryGetValue(false, out var failed) ? failed : 0;
var count = successfulCount + failedCount;
That will group the items in the list on the Result and create a dictionary of the counts. Then you can get the success and failed counts and create the total by adding them.

Just a variation on juharr worthy answer, Take a snapshot
var current = taskList.Select(x => (x.IsCompleted, x.Result)).ToList();
count = current.Count(x => x.IsCompleted)
successfulCount = current.Count(x => x.IsCompleted && x.Result == true);
failedCount = current .Count(x => x.IsCompleted && x.Result == false);

Observe values not seen in other observers

I have an observable that emits unique values e.g.
var source=Observable.Range(1,100).Publish();
source.Connect();
I want to observe its values from e.g. two observers but each observer to get notified only for values not seen in other observers.
So if first observer contains the value 10 the second observer should never get notified for the 10 value.
Update
I chose #Asti`s answer cause it was first and although buggy it pointed to the right direction and up-voted #Shlomo's answer. Too bad I cannot accept both answers as #Shlomo answer was more correct and I really appreciate all his help we get on this tag.

Observables aren't supposed to behave differently for different observers; a better approach would be to give each observer its own filtered observable.
That being said, if your constraints require that you need this behavior in a single observable - we can use a Round-Robin method.
public static IEnumerable<T> Repeat<T>(this IEnumerable<T> source)
{
for (; ; )
foreach (var item in source.ToArray())
yield return item;
}
public static IObservable<T> RoundRobin<T>(this IObservable<T> source)
{
var subscribers = new List<IObserver<T>>();
var shared = source
.Zip(subscribers.Repeat(), (value, observer) => (value, observer))
.Publish()
.RefCount();
return Observable.Create<T>(observer =>
{
subscribers.Add(observer);
var subscription =
shared
.Where(pair => pair.observer == observer)
.Select(pair => pair.value)
.Subscribe(observer);
var dispose = Disposable.Create(() => subscribers.Remove(observer));
return new CompositeDisposable(subscription, dispose);
});
}
Usage:
var source = Observable.Range(1, 100).Publish();
var dist = source.RoundRobin();
dist.Subscribe(i => Console.WriteLine($"One sees {i}"));
dist.Subscribe(i => Console.WriteLine($"Two sees {i}"));
source.Connect();
Result:
One sees 1
Two sees 2
One sees 3
Two sees 4
One sees 5
Two sees 6
One sees 7
Two sees 8
One sees 9
Two sees 10
If you already have a list of observers, the code becomes much simpler.

EDIT: #Asti fixed his bug, and I fixed mine based on his answer. Our answers are now largely similar. I have an idea how to do a purely reactive one, if I have time I'll post that later.
Fixed code:
public static IObservable<T> RoundRobin2<T>(this IObservable<T> source)
{
var subscribers = new BehaviorSubject<ImmutableList<IObserver<T>>>(ImmutableList<IObserver<T>>.Empty);
ImmutableList<IObserver<T>> latest = ImmutableList<IObserver<T>>.Empty;
subscribers.Subscribe(l => latest = l);
var shared = source
.Select((v, i) => (v, i))
.WithLatestFrom(subscribers, (t, s) => (t.v, t.i, s))
.Publish()
.RefCount();
return Observable.Create<T>(observer =>
{
subscribers.OnNext(latest.Add(observer));
var dispose = Disposable.Create(() => subscribers.OnNext(latest.Remove(observer)));
var sub = shared
.Where(t => t.i % t.s.Count == t.s.FindIndex(o => o == observer))
.Select(t => t.v)
.Subscribe(observer);
return new CompositeDisposable(dispose, sub);
});
}
Original answer:
I upvoted #Asti's answer, because he's largely correct: Just because you can, doesn't mean you should. And his answer largely works, but it's subject to a bug:
This works fine:
var source = Observable.Range(1, 20).Publish();
var dist = source.RoundRobin();
dist.Subscribe(i => Console.WriteLine($"One sees {i}"));
dist.Take(1).Subscribe(i => Console.WriteLine($"Two sees {i}"));
This doesn't:
var source = Observable.Range(1, 20).Publish();
var dist = source.RoundRobin();
dist.Take(1).Subscribe(i => Console.WriteLine($"One sees {i}"));
dist.Subscribe(i => Console.WriteLine($"Two sees {i}"));
Output is:
One sees 1
Two sees 1
Two sees 2
Two sees 3
Two sees 4
...
I first thought the bug is Halloween related, but now I'm not sure. The .ToArray() in Repeat should take care of that. I also wrote a pure-ish observable implementation which has the same bug. This implementation doesn't guarantee a perfect Round Robin, but that wasn't in the question:
public static IObservable<T> RoundRobin2<T>(this IObservable<T> source)
{
var subscribers = new BehaviorSubject<ImmutableList<IObserver<T>>>(ImmutableList<IObserver<T>>.Empty);
ImmutableList<IObserver<T>> latest = ImmutableList<IObserver<T>>.Empty;
subscribers.Subscribe(l => latest = l);
var shared = source
.Select((v, i) => (v, i))
.WithLatestFrom(subscribers, (t, s) => (t.v, t.i, s))
.Publish()
.RefCount();
return Observable.Create<T>(observer =>
{
subscribers.OnNext(latest.Add(observer));
var dispose = Disposable.Create(() => subscribers.OnNext(latest.Remove(observer)));
var sub = shared
.Where(t => t.i % t.s.Count == t.s.FindIndex(o => o == observer))
.Select(t => t.v)
.Subscribe(observer);
return new CompositeDisposable(dispose, sub);
});
}

This is a simple distributed queue implementation using TPL Dataflow. But with respect to different observers not seeing the same value, there's little chance of it behaving incorrectly. It's not round-robin, but actually has back-pressure semantics.
public static IObservable<T> Distribute<T>(this IObservable<T> source)
{
var buffer = new BufferBlock<T>();
source.Subscribe(buffer.AsObserver());
return Observable.Create<T>(observer =>
buffer.LinkTo(new ActionBlock<T>(observer.OnNext, new ExecutionDataflowBlockOptions { BoundedCapacity = 1 })
);
}
Output
One sees 1
Two sees 2
One sees 3
Two sees 4
One sees 5
One sees 6
One sees 7
One sees 8
One sees 9
One sees 10
I might prefer skipping Rx entirely and just using TPL Dataflow.

System.Reactive - Buffer/group immediately available values of an Observable

let's say you have an IObservable<T> that may supply a few values immediately, and some being pushed continously:
var immediate_values = new [] { "curerntly", "available", "values" }.ToObservable();
var future_values = Observable.Timer(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(1)).Select(x => "new value!");
IObservable<string> input = immediate_values.Concat(future_values);
Is there any way to transform input into an IObservable<string[]>, where the first array being pushed consists of all immediately available values, and each subsequent array consists of only 1 value (each one being pushed thereafter)?
Above is just example data naturally, this would need to work on any IObservable>T> without knowing the individual input streams.
IObservable<string[]> buffered = input.BufferSomehow();
// should push values:
// First value: string[] = ["currently", "available", "values"]
// Second value: string[] = ["new value!"]
// Third value: string[] = ["new value!"]
// .....
I've thought of the .Buffer() function of course, but I don't really want to buffer by any particular TimeSpan, and can't think of any way to produce an observable with buffer window closing signals.
Can anyone think of a reasonable way to achieve this, or is this not really possible at all?
Thanks!

There is no direct way to distinguish between the on-start-up values of an observable and the subsequent values. My suggestion would be to infer it:
var autoBufferedInput1 = input.Publish(_input => _input
.Buffer(_input.Throttle(TimeSpan.FromSeconds(.1)))
.Select(l => l.ToArray())
);
This sets your buffer boundary to a rolling, extending window of .1 seconds: Each time a value comes in, it extends the window to .1 seconds from the time the value came in, and adds the value to the buffer. If .1 seconds go by with no values, then the buffer is flushed out.
This will have the side-effect that if you have near-simultaneous "hot" values (within .1 seconds of each other), then those will be buffered together. If that's undesired, you can Switch out, though that makes things more complicated:
var autoBufferedInput2 = input.Publish(_input =>
_input.Throttle(TimeSpan.FromSeconds(.1)).Publish(_boundary => _boundary
.Take(1)
.Select(_ => _input.Select(s => new[] { s }))
.StartWith(_input
.Buffer(_boundary)
.Select(l => l.ToArray())
)
.Switch()
)
);
autoBufferedInput2 uses the .1 second inference method until the first buffered list, then switches to simply selecting out and wrapping values in an array.
EDIT: If you want an absolute 1 second gate as well, then the snippets would look like this:
var autoBufferedInput1 = input.Publish(_input => _input
.Buffer(
Observable.Merge(
Observable.Timer(TimeSpan.FromSeconds(1)).Select(_ => Unit.Default),
_input.Throttle(TimeSpan.FromSeconds(.1)).Select(_ => Unit.Default)
)
)
.Select(l => l.ToArray())
);
var autoBufferedInput2 = input.Publish(_input =>
Observable.Merge(
_input.Throttle(TimeSpan.FromSeconds(.1)).Select(_ => Unit.Default),
Observable.Timer(TimeSpan.FromSeconds(1)).Select(_ => Unit.Default)
)
.Publish(_boundary => _boundary
.Take(1)
.Select(_ => _input.Select(s => new[] { s }))
.StartWith(_input
.Buffer(_boundary)
.Select(l => l.ToArray())
)
.Switch()
)
);

For any IObservable<T>, you'd need to do:
var sequence = ongoingSequence.StartWith(initialSequence);

You could take advantage of the fact that the immediately available values are propagated synchronously during the subscription, and toggle some switch after the Subscribe method returns. The implementation below is based on this idea. During the subscription all incoming messages are buffered, after the subscription the buffer is emitted, and after that all future incoming messages are emitted immediately one by one.
public static IObservable<T[]> BufferImmediatelyAvailable<T>(
this IObservable<T> source)
{
return Observable.Create<T[]>(observer =>
{
var buffer = new List<T>();
var subscription = source.Subscribe(x =>
{
if (buffer != null)
buffer.Add(x);
else
observer.OnNext(new[] { x });
}, ex =>
{
buffer = null;
observer.OnError(ex);
}, () =>
{
if (buffer != null)
{
var output = buffer.ToArray();
buffer = null;
observer.OnNext(output);
}
observer.OnCompleted();
});
if (buffer != null)
{
var output = buffer.ToArray();
buffer = null;
observer.OnNext(output);
}
return subscription;
});
}

How can I merge observables once an item in the first observable satisfies a predicate?

I want to take two observables, and emit from the first observable until one of the items meets a predicate, and then start emitting events merged from both, like this:
letters: -A----B---C----d---------e-----f------g--
numbers: ---1-------2----------3-----4-----5------
predicate: IsLowerCase()
result: -A----B---C----d12----3--e--4--f--5---g--
How would it be possible to do this in C# using System.Reactive?
Thanks

Switch is almost always the key:
var letters = new Subject<string>();
var numbers = new Subject<string>();
var predicate = lettersPublished.Select(s => s.All(c => char.IsLower(c)));
var numbersCached = numbers.Replay().RefCount();
var dummySubscription = numbersCached.Subscribe();
var combined = lettersPublished.Merge(numbersCached);
var switchFlag = predicate.Where(b => b).Take(1);
var result = switchFlag.StartWith(false).Select(b => b ? combined : letters).Switch();
var resultWithDisposal = Observable.Using(() => dummySubscription, _ => result);
Construct the two observables, one before the predicate is true (letters in your case), and one after (combined), then use an observable to switch between them (switchflag). The dummySubscription is necessary to get Replay to cache all values prior to the latter subscription. Using is used to dump the dummySubscription.
You can also do this all in a single expression (given letters and numbers) as follows:
var oneLiner = letters.Publish(_letters => numbers.Replay(_numbers =>
_letters
.Select(s => s.All(c => char.IsLower(c)))
.Where(b => b)
.Take(1)
.StartWith(false)
.Select(b => b ? _letters.Merge(_numbers) : letters)
.Switch()
));

combining one observable with latest from another observable

I'm trying to combine two observables whose values share some key.
I want to produce a new value whenever the first observable produces a new value, combined with the latest value from a second observable which selection depends on the latest value from the first observable.
pseudo code example:
var obs1 = Observable.Interval(TimeSpan.FromSeconds(1)).Select(x => Tuple.create(SomeKeyThatVaries, x)
var obs2 = Observable.Interval(TimeSpan.FromMilliSeconds(1)).Select(x => Tuple.create(SomeKeyThatVaries, x)
from x in obs1
let latestFromObs2WhereKeyMatches = …
select Tuple.create(x, latestFromObs2WhereKeyMatches)
Any suggestions?
Clearly this could be implemented by subcribing to the second observable and creating a dictionary with the latest values indexable by the key. But I'm looking for a different approach..
Usage scenario: one minute price bars computed from a stream of stock quotes. In this case the key is the ticker and the dictionary contains latest ask and bid prices for concrete tickers, which are then used in the computation.
(By the way, thank you Dave and James this has been a very fruitful discussion)
(sorry about the formatting, hard to get right on an iPad..)

...why are you looking for a different approach? Sounds like you are on the right lines to me. It's short, simple code... roughly speaking it will be something like:
var cache = new ConcurrentDictionary<long, long>();
obs2.Subscribe(x => cache[x.Item1] = x.Item2);
var results = obs1.Select(x => new {
obs1 = x.Item2,
cache.ContainsKey(x.Item1) ? cache[x.Item1] : 0
});
At the end of the day, C# is an OO language and the heavy lifting of the thread-safe mutable collections is already all done for you.
There may be fancy Rx approach (feels like joins might be involved)... but how maintainable will it be? And how will it perform?
$0.02

I'd like to know the purpose of a such a query. Would you mind describing the usage scenario a bit?
Nevertheless, it seems like the following query may solve your problem. The initial projections aren't necessary if you already have some way of identifying the origin of each value, but I've included them for the sake of generalization, to be consistent with your extremely abstract mode of questioning. ;-)
Note: I'm assuming that someKeyThatVaries is not shared data as you've shown it, which is why I've also included the term anotherKeyThatVaries; otherwise, the entire query really makes no sense to me.
var obs1 = Observable.Interval(TimeSpan.FromSeconds(1))
.Select(x => Tuple.Create(someKeyThatVaries, x));
var obs2 = Observable.Interval(TimeSpan.FromSeconds(.25))
.Select(x => Tuple.Create(anotherKeyThatVaries, x));
var results = obs1.Select(t => new { Key = t.Item1, Value = t.Item2, Kind = 1 })
.Merge(
obs2.Select(t => new { Key = t.Item1, Value = t.Item2, Kind = 2 }))
.GroupBy(t => t.Key, t => new { t.Value, t.Kind })
.SelectMany(g =>
g.Scan(
new { X = -1L, Y = -1L, Yield = false },
(acc, cur) => cur.Kind == 1
? new { X = cur.Value, Y = acc.Y, Yield = true }
: new { X = acc.X, Y = cur.Value, Yield = false })
.Where(s => s.Yield)
.Select(s => Tuple.Create(s.X, s.Y)));