Develop Reference

IAsyncEnumerable like a Source for akka streams

I want to use IAsyncEnumerable like a Source for akka streams. But I not found, how do it.
No sutiable method in Source class for this code.
using System.Collections.Generic;
using System.Threading.Tasks;
using Akka.Streams.Dsl;
namespace ConsoleApp1
{
class Program
{
static async Task Main(string[] args)
{
Source.From(await AsyncEnumerable())
.Via(/*some action*/)
//.....
}
private static async IAsyncEnumerable<int> AsyncEnumerable()
{
//some async enumerable
}
}
}
How use IAsyncEnumerbale for Source?

This has been done in the past as a part of Akka.NET Streams contrib package, but since I don't see it there anymore, let's go through on how to implement such source. The topic can be quite long, as:
Akka.NET Streams is really about graph processing - we're talking about many-inputs/many-outputs configurations (in Akka.NET they're called inlets and outlets) with support for cycles in graphs.
Akka.NET is not build on top of .NET async/await or even on top of .NET standard thread pool library - they're both pluggable, which means that the lowest barier is basically using callbacks and encoding what C# compiler sometimes does for us.
Akka.NET streams is capable of both pushing and pulling values between stages/operators. IAsyncEnumerable<T> can only pull data while IObservable<T> can only push it, so we get more expressive power here, but this comes at a cost.
The basics of low level API used to implement custom stages can be found in the docs.
The starter boilerplate looks like this:
public static class AsyncEnumerableExtensions {
// Helper method to change IAsyncEnumerable into Akka.NET Source.
public static Source<T, NotUsed> AsSource<T>(this IAsyncEnumerable<T> source) =>
Source.FromGraph(new AsyncEnumerableSource<T>(source));
}
// Source stage is description of a part of the graph that doesn't consume
// any data, only produce it using a single output channel.
public sealed class AsyncEnumerableSource<T> : GraphStage<SourceShape<T>>
{
private readonly IAsyncEnumerable<T> _enumerable;
public AsyncEnumerableSource(IAsyncEnumerable<T> enumerable)
{
_enumerable = enumerable;
Outlet = new Outlet<T>("asyncenumerable.out");
Shape = new SourceShape<T>(Outlet);
}
public Outlet<T> Outlet { get; }
public override SourceShape<T> Shape { get; }
/// Logic if to a graph stage, what enumerator is to enumerable.
protected override GraphStageLogic CreateLogic(Attributes inheritedAttributes) =>
new Logic(this);
sealed class Logic: OutGraphStageLogic
{
public override void OnPull()
{
// method called whenever a consumer asks for new data
}
public override void OnDownstreamFinish()
{
// method called whenever a consumer stage finishes,used for disposals
}
}
}
As mentioned, we don't use async/await straight away here: even more, calling Logic methods in asynchronous context is unsafe. To make it safe we need to register out methods that may be called from other threads using GetAsyncCallback<T> and call them via returned wrappers. This will ensure, that not data races will happen when executing asynchronous code.
sealed class Logic : OutGraphStageLogic
{
private readonly Outlet<T> _outlet;
// enumerator we'll call for MoveNextAsync, and eventually dispose
private readonly IAsyncEnumerator<T> _enumerator;
// callback called whenever _enumerator.MoveNextAsync completes asynchronously
private readonly Action<Task<bool>> _onMoveNext;
// callback called whenever _enumerator.DisposeAsync completes asynchronously
private readonly Action<Task> _onDisposed;
// cache used for errors thrown by _enumerator.MoveNextAsync, that
// should be rethrown after _enumerator.DisposeAsync
private Exception? _failReason = null;
public Logic(AsyncEnumerableSource<T> source) : base(source.Shape)
{
_outlet = source.Outlet;
_enumerator = source._enumerable.GetAsyncEnumerator();
_onMoveNext = GetAsyncCallback<Task<bool>>(OnMoveNext);
_onDisposed = GetAsyncCallback<Task>(OnDisposed);
}
// ... other methods
}
The last part to do are methods overriden on `Logic:
OnPull used whenever the downstream stage calls for new data. Here we need to call for next element of async enumerator sequence.
OnDownstreamFinish called whenever the downstream stage has finished and will not ask for any new data. It's the place for us to dispose our enumerator.
Thing is these methods are not async/await, while their enumerator's equivalent are. What we basically need to do there is to:
Call corresponding async methods of underlying enumerator (OnPull → MoveNextAsync and OnDownstreamFinish → DisposeAsync).
See, if we can take their results immediately - it's important part that usually is done for us as part of C# compiler in async/await calls.
If not, and we need to wait for the results - call ContinueWith to register our callback wrappers to be called once async methods are done.
sealed class Logic : OutGraphStageLogic
{
// ... constructor and fields
public override void OnPull()
{
var hasNext = _enumerator.MoveNextAsync();
if (hasNext.IsCompletedSuccessfully)
{
// first try short-path: values is returned immediately
if (hasNext.Result)
// check if there was next value and push it downstream
Push(_outlet, _enumerator.Current);
else
// if there was none, we reached end of async enumerable
// and we can dispose it
DisposeAndComplete();
}
else
// we need to wait for the result
hasNext.AsTask().ContinueWith(_onMoveNext);
}
// This method is called when another stage downstream has been completed
public override void OnDownstreamFinish() =>
// dispose enumerator on downstream finish
DisposeAndComplete();
private void DisposeAndComplete()
{
var disposed = _enumerator.DisposeAsync();
if (disposed.IsCompletedSuccessfully)
{
// enumerator disposal completed immediately
if (_failReason is not null)
// if we close this stream in result of error in MoveNextAsync,
// fail the stage
FailStage(_failReason);
else
// we can close the stage with no issues
CompleteStage();
}
else
// we need to await for enumerator to be disposed
disposed.AsTask().ContinueWith(_onDisposed);
}
private void OnMoveNext(Task<bool> task)
{
// since this is callback, it will always be completed, we just need
// to check for exceptions
if (task.IsCompletedSuccessfully)
{
if (task.Result)
// if task returns true, it means we read a value
Push(_outlet, _enumerator.Current);
else
// otherwise there are no more values to read and we can close the source
DisposeAndComplete();
}
else
{
// task either failed or has been cancelled
_failReason = task.Exception as Exception ?? new TaskCanceledException(task);
FailStage(_failReason);
}
}
private void OnDisposed(Task task)
{
if (task.IsCompletedSuccessfully) CompleteStage();
else {
var reason = task.Exception as Exception
?? _failReason
?? new TaskCanceledException(task);
FailStage(reason);
}
}
}

As of Akka.NET v1.4.30 this is now natively supported inside Akka.Streams via the RunAsAsyncEnumerable method:
var input = Enumerable.Range(1, 6).ToList();
var cts = new CancellationTokenSource();
var token = cts.Token;
var asyncEnumerable = Source.From(input).RunAsAsyncEnumerable(Materializer);
var output = input.ToArray();
bool caught = false;
try
{
await foreach (var a in asyncEnumerable.WithCancellation(token))
{
cts.Cancel();
}
}
catch (OperationCanceledException e)
{
caught = true;
}
caught.ShouldBeTrue();
I copied that sample from the Akka.NET test suite, in case you're wondering.

You can also use an existing primitive for streaming large collections of data. Here is an example of using Source.unfoldAsync to stream pages of data - in this case github repositories using Octokit - until there is no more.
var source = Source.UnfoldAsync<int, RepositoryPage>(startPage, page =>
{
var pageTask = client.GetRepositoriesAsync(page, pageSize);
var next = pageTask.ContinueWith(task =>
{
var page = task.Result;
if (page.PageNumber * pageSize > page.Total) return Option<(int, RepositoryPage)>.None;
else return new Option<(int, RepositoryPage)>((page.PageNumber + 1, page));
});
return next;
});
To run
using var sys = ActorSystem.Create("system");
using var mat = sys.Materializer();
int startPage = 1;
int pageSize = 50;
var client = new GitHubClient(new ProductHeaderValue("github-search-app"));
var source = ...
var sink = Sink.ForEach<RepositoryPage>(Console.WriteLine);
var result = source.RunWith(sink, mat);
await result.ContinueWith(_ => sys.Terminate());
class Page<T>
{
public Page(IReadOnlyList<T> contents, int page, long total)
{
Contents = contents;
PageNumber = page;
Total = total;
}
public IReadOnlyList<T> Contents { get; set; } = new List<T>();
public int PageNumber { get; set; }
public long Total { get; set; }
}
class RepositoryPage : Page<Repository>
{
public RepositoryPage(IReadOnlyList<Repository> contents, int page, long total)
: base(contents, page, total)
{
}
public override string ToString() =>
$"Page {PageNumber}\n{string.Join("", Contents.Select(x => x.Name + "\n"))}";
}
static class GitHubClientExtensions
{
public static async Task<RepositoryPage> GetRepositoriesAsync(this GitHubClient client, int page, int size)
{
// specify a search term here
var request = new SearchRepositoriesRequest("bootstrap")
{
Page = page,
PerPage = size
};
var result = await client.Search.SearchRepo(request);
return new RepositoryPage(result.Items, page, result.TotalCount);
}
}

Rx.Net GroupBy, How to subscribe multiple observers to a specific Group

I am taking my first leap into the world of Rx and finding it difficult to get the desired results, especially with the GroupBy operator, so any help would be much appreciated.
How can I subscribe multiple observers to a specific group?
My requirements are:
I have a DataProvider class that makes http Api requests at regular intervals.
The http response is a List<Item>. Each Item has a unique Id property.
I need to process each Item as a separate stream based on its Id, which looks like a case for GroupBy.
Each group needs its own pipeline where:
It Starts with a specific value (StartWith operator)
It Buffers the previous Item for comparison with the current Item (Buffer(2,1) operator)
If the current Item is different to the previous (Where) emit the current Item
The result is an IObservable<Item> of changes (ChangeStream). I am no longer dealing with a specific group.
How can I stay within the group pipeline and allow multiple subscribers to Subscribe to a specific group?
Observers can subscribe early (before the Item.Id has appeared on the response stream and before the group is created)
Observers can subscribe late (after the Item.Id has appeared on the response stream and the group has been created)
Late subscribers should receive the last change for the Item.Id (Replay(1)) but I can’t seem to figure this part out either.
What is the Rx way to Multicast a specific group? Any help / advice would be much appreciated. I have provided sample code below.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using System.Threading;
namespace RxTest
{
class Program
{
static void Main(string[] args)
{
var dataService = new MockDataService();
// How do I subscribe to a specific group?
// Eg. I am only interested in changes to Items where Id == 1
// Subscribers can be early (before the stream is hot)
var item1Stream = dataService.SubscribeToItem(1);
// There can be multiple subscribers to a group
var item1Stream2 = dataService.SubscribeToItem(1);
Console.WriteLine("Press Any Key to Start");
Console.ReadLine();
dataService.Start();
// Subscribers can be late (Eg. Subscriber to Item Id == 2 after it has emitted items)
Thread.Sleep(2000);
var item2Stream = dataService.SubscribeToItem(2);
// Subscribers can be early (After connect but before the Item Id appears on the Stream (before group creation))
// Eg. Subscribe to group 4 (Group 4 doesn't get created until 20s after connect in this example)
var item4Stream = dataService.SubscribeToItem(4);
// What is the Rx way to Multicast a Group?
Console.WriteLine("Press Any Key to Exit");
Console.ReadLine();
dataService.Stop();
}
}
public class MockDataService
{
private readonly IConnectableObservable<Item> _itemsStream;
private IDisposable _itemsSubscription;
private readonly IObservable<Item> _changeStream;
private IDisposable _changeSubscription;
public MockDataService()
{
// Simulate Http response pipeline.
//// Time: 1s...............10s..............20s.....etc
//// stream: [[1][2]]repeat...[[2][3]]repeat...[[3][4]]repeat...
IObservable<List<Item>> responseStream = Observable.Interval(TimeSpan.FromSeconds(1))
.Take(50)
.Select(tick =>
{
// Every 10 ticks an item drops off the stream and a new one starts
// Every 2 ticks the Item value increases to generate a change.
int rangeStart = Math.Min(((int)tick / 10) + 1, 5);
return Enumerable.Range(rangeStart, 2).Select(id => new Item(id, (int)tick / 2)).ToList();
});
// Flatten the list into IObservable<Item>
//// Time: 1s.............10s............20s.....etc
//// stream: [1][2]repeat...[2][3]repeat...[3][4]repeat...
_itemsStream = responseStream
.SelectMany(list => list)
.Publish();
// Split into groups by Item.Id and process each group for changes
// ChangeStream is an IObservable<Item> that have changes.
_changeStream = _itemsStream
.GroupBy(item => item.Id)
.SelectMany(grp =>
grp
// Pipeline for each group.
.StartWith(new Item(grp.Key, -1)) // Initial item from Db
//.TakeUntil(Item => Item.IsComplete()) // Logic to complete the group
.LogConsoleWithThread($"Group: {grp.Key}")
.Buffer(2, 1)
.Where(buffer => buffer.Count == 2 && buffer[0].HasChanges(buffer[1]))
.Select(buffer => buffer[1])
.LogConsoleWithThread($"Group.Change : {grp.Key}")
// How do I push changes in this group to Zero..Many subscribed Observers?
// I would also like to Replay(1) to all late subscribers to a group.
);
}
/// <summary>
/// How to get the IObservable for a specific group?
/// </summary>
/// <param name="itemId"></param>
/// <returns></returns>
public IObservable<Item> SubscribeToItem(int itemId)
{
// ????
return null;
}
public void Start()
{
_changeSubscription = _changeStream.SubscribeConsole("ChangeStream");
_itemsSubscription = _itemsStream.Connect();
}
public void Stop()
{
_changeSubscription.Dispose();
_itemsSubscription.Dispose();
}
}
public class Item
{
public int Id { get; private set; }
public int Value { get; private set; }
public Item(int id, int value)
{
Id = id;
Value = value;
}
public bool HasChanges(Item compareItem)
{
return this.Value != compareItem.Value;
}
public override string ToString()
{
return $"Item: Id={Id} Value={Value}";
}
}
public static class RxExtensions
{
public static IDisposable SubscribeConsole<T>(this IObservable<T> observable, string name = "")
{
return observable.Subscribe(new ConsoleObserver<T>(name));
}
/// <summary>
/// Logs to the Console the subscriptions and emissions done on/by the observable
/// each log message also includes the thread it happens on
/// </summary>
/// <typeparam name="T">The Observable Type</typeparam>
/// <param name="observable">The Observable to log.</param>
/// <param name="name">An optional name prefix that will be added before each notification</param>
/// <returns></returns>
public static IObservable<T> LogConsoleWithThread<T>(this IObservable<T> observable, string name = "")
{
return Observable.Defer(() =>
{
Console.WriteLine("{0} Subscription happened on Thread: {1}", name, Thread.CurrentThread.ManagedThreadId);
return observable.Do(
x => Console.WriteLine("{0} - OnNext({1}) Thread: {2}", name, x, Thread.CurrentThread.ManagedThreadId),
ex =>
{
Console.WriteLine("{0} - OnError Thread:{1}", name, Thread.CurrentThread.ManagedThreadId);
Console.WriteLine("\t {0}", ex);
},
() => Console.WriteLine("{0} - OnCompleted() Thread {1}", name, Thread.CurrentThread.ManagedThreadId));
});
}
}
/// <summary>
/// An observer that outputs to the console each time the OnNext, OnError or OnComplete occurs
/// </summary>
/// <typeparam name="T"></typeparam>
public class ConsoleObserver<T> : IObserver<T>
{
private readonly string _name;
public ConsoleObserver(string name = "")
{
_name = name;
}
public void OnNext(T value)
{
Console.WriteLine("{0} - OnNext({1})", _name, value);
}
public void OnError(Exception error)
{
Console.WriteLine("{0} - OnError:", _name);
Console.WriteLine("\t {0}", error);
}
public void OnCompleted()
{
Console.WriteLine("{0} - OnCompleted()", _name);
}
}
}

You probably need a specialized publishing operator, because the existing ones (Publish, PublishLast and Replay) are too narrow or too broad for your needs. So you'll need to use the Multicast operator, supplied with a custom replay subject that buffers only the last element per key. Here is a basic implementation of such a subject:
public class ReplayLastPerKeySubject<T, TKey> : ISubject<T>
{
private readonly Func<T, TKey> _keySelector;
private readonly ReplaySubject<ReplaySubject<T>> _subjects;
private readonly IObservable<T> _mergedSubjects;
private readonly Dictionary<TKey, ReplaySubject<T>> _perKey;
public ReplayLastPerKeySubject(Func<T, TKey> keySelector)
{
_keySelector = keySelector;
_subjects = new ReplaySubject<ReplaySubject<T>>();
_mergedSubjects = _subjects.Merge();
_perKey = new Dictionary<TKey, ReplaySubject<T>>();
}
public void OnNext(T value)
{
var key = _keySelector(value);
ReplaySubject<T> subject;
if (!_perKey.TryGetValue(key, out subject))
{
subject = new ReplaySubject<T>(1);
_perKey.Add(key, subject);
_subjects.OnNext(subject);
}
subject.OnNext(value);
}
public void OnCompleted()
{
// All subjects, inner and outer, must be completed
_subjects.OnCompleted();
_subjects.Subscribe(subject => subject.OnCompleted());
}
public void OnError(Exception error)
{
// Faulting the master (outer) subject is enough
_subjects.OnError(error);
}
public IDisposable Subscribe(IObserver<T> observer)
{
return _mergedSubjects.Subscribe(observer);
}
}
This implementation is based on an answer of a similar question, written by an RX expert. The original answer uses a Concat observable for subscribing the observers, while this one uses a Merge observable, so I am not 100% sure about its correctness and efficiency.
Having such an implementation in place, the rest is easy. You first create a published version of your original observable:
var published = YourObservable
.Multicast(new ReplayLastPerKeySubject<Item, int>(x => x.Id)))
.RefCount();
And finally you can create a change stream for a specific key, by using the Where operator:
var changeStream13 = published.Where(x => x.Id == 13);

Merging multiple observables and updating existing subscribers?

How do I aggregate hot observables which may or may not have subscribers into a new observable and continue to provide all new data to existing subscribers?
As an example, imagine we have some class like this:
class SomeClass
{
IObservable<string> Actions { get; set; } = Observable.Empty<string>();
void AddActionCreator(IObservable<string> creator)
{
Actions = Actions.Merge(creator);
}
}
The problem I am running into is if AddActionCreator adds a new stream of actions then any previous subscribers of SomeClass.Actions which subscribed before that new stream is merged will never get the new actions.

It's easy to do what you want. What you need here is a SelectMany and a Subject<IObservabe<string>>.
Here's the class you need:
public class SomeClass
{
private Subject<IObservable<string>> _sources = new Subject<System.IObservable<string>>();
public IObservable<string> Actions { get; private set; } = null;
public SomeClass()
{
this.Actions = _sources.SelectMany(x => x);
}
public void AddActionCreator(IObservable<string> creator)
{
_sources.OnNext(creator);
}
}
Now you can use it like this:
var sc = new SomeClass();
sc.Actions.Subscribe(x => Console.WriteLine($"1:{x}"));
sc.AddActionCreator(Observable.Return("Hello"));
sc.Actions.Subscribe(x => Console.WriteLine($"2:{x}"));
sc.AddActionCreator(Observable.Range(0, 3).Select(x => $"{x}"));
sc.Actions.Subscribe(x => Console.WriteLine($"3:{x}"));
sc.AddActionCreator(Observable.Return("World"));
You'll get this output:
1:Hello
1:0
1:1
1:2
2:0
2:1
2:2
1:World
2:World
3:World
You can see that the new observables are added to the existing subscribers.

Triggering DynamicData cache update using Reactive Subject

As a caveat I'm a novice with Rx (2 weeks) and have been experimenting with using Rx, RxUI and Roland Pheasant's DynamicData.
I have a service that initially loads data from local persistence and then, upon some user (or system) instruction will contact the server (TriggerServer in the example) to get additional or replacement data. The solution I've come up with uses a Subject and I've come across many a site discussing the pros/cons of using them. Although I understand the basics of hot/cold it's all based on reading rather than real world.
So, using the below as a simplified version, is this 'right' way of going about this problem or is there something I haven't properly understood somewhere?
NB: I'm not sure how important it is, but the actual code is taken from a Xamarin.Forms app, that uses RxUI, the user input being a ReactiveCommand.
Example:
using DynamicData;
using System;
using System.Linq;
using System.Reactive;
using System.Reactive.Disposables;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using System.Threading.Tasks;
public class MyService : IDisposable
{
private CompositeDisposable _cleanup;
private Subject<Unit> _serverSubject = new Subject<Unit>();
public MyService()
{
var data = Initialise().Publish();
AllData = data.AsObservableCache();
_cleanup = new CompositeDisposable(AllData, data.Connect());
}
public IObservableCache<MyData, Guid> AllData { get; }
public void TriggerServer()
{
// This is what I'm not sure about...
_serverSubject.OnNext(Unit.Default);
}
private IObservable<IChangeSet<MyData, Guid>> Initialise()
{
return ObservableChangeSet.Create<MyData, Guid>(async cache =>
{
// inital load - is this okay?
cache.AddOrUpdate(await LoadLocalData());
// is this a valid way of doing this?
var sync = _serverSubject.Select(_ => GetDataFromServer())
.Subscribe(async task =>
{
var data = await task.ConfigureAwait(false);
cache.AddOrUpdate(data);
});
return new CompositeDisposable(sync);
}, d=> d.Id);
}
private IObservable<MyData> LoadLocalData()
{
return Observable.Timer(TimeSpan.FromSeconds(3)).Select(_ => new MyData("localdata"));
}
private async Task<MyData> GetDataFromServer()
{
await Task.Delay(2000).ConfigureAwait(true);
return new MyData("serverdata");
}
public void Dispose()
{
_cleanup?.Dispose();
}
}
public class MyData
{
public MyData(string value)
{
Value = value;
}
public Guid Id { get; } = Guid.NewGuid();
public string Value { get; set; }
}
And a simple Console app to run:
public static class TestProgram
{
public static void Main()
{
var service = new MyService();
service.AllData.Connect()
.Bind(out var myData)
.Subscribe(_=> Console.WriteLine("data in"), ()=> Console.WriteLine("COMPLETE"));
while (Continue())
{
Console.WriteLine("");
Console.WriteLine("");
Console.WriteLine($"Triggering Server Call, current data is: {string.Join(", ", myData.Select(x=> x.Value))}");
service.TriggerServer();
}
}
private static bool Continue()
{
Console.WriteLine("Press any key to call server, x to exit");
var key = Console.ReadKey();
return key.Key != ConsoleKey.X;
}
}

Looks very good for first try with Rx
I would suggest few changes:
1) Remove the Initialize() call from the constructor and make it a public method - helps a lot with unit tests and now you can await it if you need to
public static void Main()
{
var service = new MyService();
service.Initialize();
2) Add Throttle to you trigger - this fixes parallel calls to the server returning the same results
3) Don't do anything that can throw in Subscribe, use Do instead:
var sync = _serverSubject
.Throttle(Timespan.FromSeconds(0.5), RxApp.TaskPoolScheduler) // you can pass a scheduler via arguments, or use TestScheduler in unit tests to make time pass faster
.Do(async _ =>
{
var data = await GetDataFromServer().ConfigureAwait(false); // I just think this is more readable, your way was also correct
cache.AddOrUpdate(data);
})
// .Retry(); // or anything alese to handle failures
.Subscribe();

I'm putting what I've come to as my solution just in case there's others that find this while they're wandering the internets.
I ended up removing the Subjects all together and chaining together several SourceCache, so when one changed it pushed into the other and so on. I've removed some code for brevity:
public class MyService : IDisposable
{
private SourceCache<MyData, Guid> _localCache = new SourceCache<MyData, Guid>(x=> x.Id);
private SourceCache<MyData, Guid> _serverCache = new SourceCache<MyData, Guid>(x=> x.Id);
public MyService()
{
var localdata = _localCache.Connect();
var serverdata = _serverCache.Connect();
var alldata = localdata.Merge(serverdata);
AllData = alldata.AsObservableCache();
}
public IObservableCache<MyData, Guid> AllData { get; }
public IObservable<Unit> TriggerLocal()
{
return LoadLocalAsync().ToObservable();
}
public IObservable<Unit> TriggerServer()
{
return LoadServerAsync().ToObservable();
}
}
EDIT: I've changed this again to remove any chaining of caches - I just manage the one cache internally. Lesson is not to post too early.

Rx - Reactive extensions - conditional switch from first Observable to second

I have 2 data sources: online and offline (cached). Both of them returns IObservable of object which contains 2 flags - IsSuccess and IsCached. I would like to get data from online source but only when IsSuccess=true. If this fail I would like to get data from offline source. Additionally I want to save new data in cache for future. I am not sure how to do it best in RX.
Here is my implementation of that but I think it can be done much better
public IObservable<Result<SampleModel>> GetSampleModel()
{
IObservable<Result<SampleModel>> onlineObservable = _onlineSource.GetData<SampleModel>();
IObservable<Result<SampleModel>> offlineObservable = _offlineSource.GetData<SampleModel>();
var subject = new Subject<Result<SampleModel>>();
onlineObservable.Do(async (result) =>
{
if (result.IsSuccess)
{
await _offlineSource.CacheData(result.Data).ConfigureAwait(false);
}
}).Subscribe((result) =>
{
if (result.IsSuccess)
{
subject.OnNext(result);
}
subject.OnCompleted();
});
return subject.Concat(offlineObservable).Take(1);
}
Result class - wrapper for data:
public class Result<T>
{
public Result(Exception exception)
{
Exception = exception;
}
public Result(T data, bool isCached = false)
{
IsCached = isCached;
IsSuccess = true;
Data = data;
}
public bool IsSuccess { get; private set; }
public bool IsCached { get; private set; }
public T Data { get; private set; }
public Exception Exception { get; private set; }
}

Your implementation will not work reliably, because there is a race condition in there. Consider this:
var temp = GetSampleModel(); // #1
// Do some long operation here
temp.Subscribe(p => Console.WriteLine(p)); // #2
In this case, fetching data will start in #1, and if the data is received and pushed to subject before #2 executes, nothing will be printed no matter how long you wait.
Usually, you should avoid subscribing inside a function returning IObservable to avoid such issues. Using Do is also a bad smell. You could fix the code using ReplaySubject or AsyncSubject, but in such cases I generally prefer Observable.Create. Here is my rewrite:
public IObservable<SampleModel> GetSampleModel(IScheduler scheduler = null)
{
scheduler = scheduler ?? TaskPoolScheduler.Default;
return Observable.Create<SampleModel>(observer =>
{
return scheduler.ScheduleAsync(async (s, ct) =>
{
var onlineResult = await _onlineSource.GetData<SampleModel>().FirstAsync();
if (onlineResult.IsSuccess)
{
observer.OnNext(onlineResult.Data);
await _offlineSource.CacheData(onlineResult.Data);
observer.OnCompleted();
}
else
{
var offlineResult = await _offlineSource.GetData<SampleModel>().FirstAsync();
if (offlineResult.IsSuccess)
{
observer.OnNext(offlineResult.Data);
observer.OnCompleted();
}
else
{
observer.OnError(new Exception("Could not receive model"));
}
}
return Disposable.Empty;
});
});
}
You can see that it still isn't terribly pretty. I think that it's because you chose not to use natural Rx system of handling errors, but instead to wrap your values in Result type. If you alter your repository methods to handle errors in Rx way, resulting code is much more concise. (Note that I changed your Result type to MaybeCached, and I assume that now both sources return IObservable<SampleModel>, which is a cold observable either returning a single result or an error):
public class MaybeCached<T>
{
public MaybeCached(T data, bool isCached)
{
IsCached = isCached;
IsSuccess = true;
}
public bool IsCached { get; private set; }
public T Data { get; private set; }
}
public IObservable<SampleModel> GetSampleModel()
{
_onlineSource
.GetData<SampleModel>()
.Select(d => new MaybeCached(d, false))
.Catch(_offlineSource
.GetData<SampleModel>()
.Select(d => new MaybeCached(d, true))
.SelectMany(data => data.IsCached ? Observable.Return(data.Data) : _offlineSource.CacheData(data.Data).Select(_ => data.Data));
}
Catch is used here in order to obtain a conditional switch you asked for.