How to Separate IObservable and IObserver

How to Separate IObservable and IObserver - c#

Update: check out the example at the bottom
I need to message between classes. The publisher will loop indefinitely, call some method to get data, and then pass the result of that call into OnNext. There can be many subscribers, but there should only ever be one IObservable, and one long-running task. Here is an implementation.
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Diagnostics;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using System.Threading.Tasks;
namespace UnitTestProject1
{
[TestClass]
public class UnitTest1
{
private static string GetSomeData() => "Hi";
[TestMethod]
public async Task RunMessagingAsync()
{
var subject = new Subject<string>();
//Create a class and inject the subject as IObserver
new Publisher(subject);
//Create a class and inject the subject as IObservable
new Subscriber(subject, 1.ToString());
new Subscriber(subject, 2.ToString());
new Subscriber(subject, 3.ToString());
//Run the loop for 3 seconds
await Task.Delay(3000);
}
class Publisher
{
public Publisher(IObserver<string> observer)
{
Task.Run(async () =>
{
//Loop forever
while (true)
{
//Get some data, publish it with OnNext and wait 500 milliseconds
observer.OnNext(GetSomeData());
await Task.Delay(500);
}
});
}
}
class Subscriber
{
public string Name;
//Listen for OnNext and write to the debug window when it happens
public Subscriber(IObservable<string> observable, string name)
{
Name = name;
var disposable = observable.Subscribe((s) => Debug.WriteLine($"Name: {Name} Message: {s}"));
}
}
}
}
Output:
Name: 1 Message: Hi
Name: 2 Message: Hi
Name: 3 Message: Hi
Name: 1 Message: Hi
Name: 2 Message: Hi
Name: 3 Message: Hi
This works fine. Notice that only one IObserver sends messages, but all subscriptions pick up the message. But, how do I separate the IObservable and the IObserver ? They are glued together as a Subject. Here is another approach.
[TestMethod]
public async Task RunMessagingAsync2()
{
var observers = new List<IObserver<string>>();
var observable = Observable.Create(
(IObserver<string> observer) =>
{
observers.Add(observer);
Task.Run(async () =>
{
while (true)
{
try
{
observer.OnNext(GetSomeData());
}
catch (Exception ex)
{
observer.OnError(ex);
}
await Task.Delay(500);
}
});
return Disposable.Create(() => { });
});
//Create a class and inject the subject as IObservable
new Subscriber(observable);
new Subscriber(observable);
//Run the loop for 10 seconds
await Task.Delay(10000);
Assert.IsTrue(ReferenceEquals(observers[0], observers[1]));
}
The problem here is that this creates two separate Tasks and two separate IObservers. Every subscription creates a new IObserver. You can confirm that because the Assert here fails. This doesn't really make any sense to me. From what I understand of Reactive programming, I wouldn't expect the Subscribe method here to create a new IObserver each time. Check out this gist. It is a slight modification of the Observable.Create example. It shows how the Subscribe method causes an IObserver to be created each time it is called. How can I achieve the functionality from the first example without using a Subject?
Here is another approach that does not use Reactive UI at all... You could create a Subject from the publisher if you want to, but it is not necessary.
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Diagnostics;
using System.Threading.Tasks;
namespace UnitTestProject1
{
[TestClass]
public class UnitTest1
{
private static string GetSomeData() => "Hi";
class Publisher
{
public Publisher(Action<string> onNext)
{
Task.Run(async () =>
{
//Loop forever
while (true)
{
//Get some data, publish it with OnNext and wait 500 milliseconds
onNext(GetSomeData());
await Task.Delay(500);
}
});
}
}
class Subscriber
{
//Listen for OnNext and write to the debug window when it happens
public void ReceiveMessage(string message) => Debug.WriteLine(message);
}
[TestMethod]
public async Task RunMessagingAsync()
{
//Create a class and inject the subject as IObservable
var subscriber = new Subscriber();
//Create a class and inject the subject as IObserver
new Publisher(subscriber.ReceiveMessage);
//Run the loop for 10 seconds
await Task.Delay(10000);
}
}
}
Lastly, I should add that ReactiveUI used to have a MessageBus class. I'm not sure if it got removed or not, but it is no longer recommended. What do they suggest we use instead?
Working Example
This version is correct. I guess the only thing I'm asking now is how do I do the equivalent of this with Observable.Create? The problem with Observable.Create is that it runs the action for each subscription. That is not the intended functionality. The long running task here only runs once no matter how many subscriptions there are.
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Reactive.Disposables;
using System.Reactive.Linq;
using System.Threading;
using System.Threading.Tasks;
namespace UnitTestProject1
{
class Subscriber
{
public string Name;
//Listen for OnNext and write to the debug window when it happens
public Subscriber(IObservable<string> observable, string name)
{
Name = name;
var disposable = observable.Subscribe((s) => Debug.WriteLine($"Name: {Name} Message: {s}"));
}
}
internal class BasicObservable<T> : IObservable<T>
{
List<IObserver<T>> _observers = new List<IObserver<T>>();
public BasicObservable(
Func<T> getData,
TimeSpan? interval = null,
CancellationToken cancellationToken = default
) =>
Task.Run(async () =>
{
while (!cancellationToken.IsCancellationRequested)
{
try
{
await Task.Delay(interval ?? new TimeSpan(0, 0, 1));
var data = getData();
_observers.ForEach(o => o.OnNext(data));
}
catch (Exception ex)
{
_observers.ForEach(o => o.OnError(ex));
}
}
_observers.ForEach(o => o.OnCompleted());
}, cancellationToken);
public IDisposable Subscribe(IObserver<T> observer)
{
_observers.Add(observer);
return Disposable.Create(observer, (o) => _observers.Remove(o));
}
}
public static class ObservableExtensions
{
public static IObservable<T> CreateObservable<T>(
this Func<T> getData,
CancellationToken cancellationToken = default)
=> new BasicObservable<T>(getData, default, cancellationToken);
public static IObservable<T> CreateObservable<T>(
this Func<T> getData,
TimeSpan? interval = null,
CancellationToken cancellationToken = default)
=> new BasicObservable<T>(getData, interval, cancellationToken);
}
[TestClass]
public class UnitTest1
{
string GetData() => "Hi";
[TestMethod]
public async Task Messaging()
{
var cancellationSource = new CancellationTokenSource();
var cancellationToken = cancellationSource.Token;
Func<string> getData = GetData;
var publisher = getData.CreateObservable(cancellationToken);
new Subscriber(publisher, "One");
new Subscriber(publisher, "Two");
for (var i = 0; true; i++)
{
if (i >= 5)
{
cancellationSource.Cancel();
}
await Task.Delay(1000);
}
}
}
}

At first you must familiarize yourself with the theory of "cold" and "hot" observables. Here is the definition from the Introduction to RX.
Cold are sequences that are passive and start producing notifications on request (when subscribed to).
Hot are sequences that are active and produce notifications regardless of subscriptions.
What you want is a hot observable, and the problem is that the Observable.Create method creates cold observables. But you can make any observable hot by using the Publish operator. This operator provides a way to have a single underlying subscription shared by multiple independent observers. Example:
int index = 0;
var coldObservable = Observable.Create<int>(observer =>
{
_ = Task.Run(async () =>
{
while (true)
{
observer.OnNext(++index);
await Task.Delay(1000);
}
});
return Disposable.Empty;
});
IConnectableObservable<int> hotObservable = coldObservable.Publish();
hotObservable.Connect(); // Causes the start of the loop
hotObservable.Subscribe(s => Console.WriteLine($"Observer A received #{s}"));
hotObservable.Subscribe(s => Console.WriteLine($"Observer B received #{s}"));
The coldObservable created by the Observable.Create is subscribed when the hotObservable.Connect method is invoked, and then all notifications generated by that single subscription are propagated to all subscribers of the hotObservable.
Output:
Observer A received #1
Observer B received #1
Observer A received #2
Observer B received #2
Observer A received #3
Observer B received #3
Observer A received #4
Observer B received #4
Observer A received #5
Observer B received #5
Observer A received #6
Observer B received #6
...
Important: the purpose of the example above is to demonstrate the Publish operator, and not to serve as an example of good quality RX code. One of its problems is that by subscribing the observers after connecting to the source becomes theoretically possible that the first notification will not be send to some or all of the observers, because it may be created before their subscription. There is a race condition in other words.
There is an alternative method of managing the lifetime of an IConnectableObservable, the operator RefCount:
Returns an observable sequence that stays connected to the source as long as there is at least one subscription to the observable sequence.
var hotObservable = coldObservable.Publish().RefCount();
This way you don't need to Connect manually. The connection occurs automatically with the first subscription, and it is disposed automatically with the last unsubscription.

I've added this as an answer because I feel that the code that Christian posted in his answer is dangerous as it's mixing Tasks and Rx and there are race conditions.
Here's an alternative that fixes most of these issues:
public class UnitTest1
{
private string GetData() => "Hi";
private IDisposable Subscriber(IObservable<string> observable, string name) =>
observable.Subscribe(s => Debug.WriteLine($"Name: {name} Message: {s}"));
public async Task Messaging()
{
var coldObservable =
Observable
.Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1.0))
.Select(_ => GetData());
var publisher = coldObservable.Publish();
var subscriptions =
new CompositeDisposable(
Subscriber(publisher, "One"),
Subscriber(publisher, "Two"),
publisher.Connect());
await Task.Delay(TimeSpan.FromSeconds(5.0));
subscriptions.Dispose();
}
}
Better yet, though, I would look at doing it this way:
public class UnitTest1
{
private string GetData() => "Hi";
private IObservable<string> Subscriber(IObservable<string> observable, string name) =>
observable.Select(s => $"Name: {name} Message: {s}");
public async Task Messaging()
{
var coldObservable =
Observable
.Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1.0))
.Select(_ => GetData())
.Do(_ => Debug.WriteLine("Called GetData()"))
.Publish(published =>
Observable
.Merge(
Subscriber(published, "One"),
Subscriber(published, "Two")))
.TakeUntil(Observable.Timer(TimeSpan.FromSeconds(5.0)))
.Do(x => Debug.WriteLine(x));
await coldObservable;
}
}
It's always best to use the inbuilt operators for Rx rather than hybrid approaches with tasks.

Thanks to the answer above, I eventually got the desired result without having to implement IObservable. Theodor was correct. The answer was to convert the IObservable to hot with the Publish() method.
I wrote an article about this here
While this works, Enigmativity's answer above is far better.
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Diagnostics;
using System.Reactive.Disposables;
using System.Reactive.Linq;
using System.Threading;
using System.Threading.Tasks;
namespace Observables
{
class Subscriber
{
public string Name;
//Listen for OnNext and write to the debug window when it happens
public Subscriber(IObservable<string> observable, string name)
{
Name = name;
observable.Subscribe(s => Debug.WriteLine($"Name: {Name} Message: {s}"));
}
}
[TestClass]
public class UnitTest1
{
static string GetData() => "Hi";
[TestMethod]
public async Task Messaging()
{
var cancellationSource = new CancellationTokenSource();
var cancellationToken = cancellationSource.Token;
var coldObservable = Observable.Create<string>(observer =>
{
_ = Task.Run(async () =>
{
while (!cancellationToken.IsCancellationRequested)
{
var data = GetData();
observer.OnNext(data);
await Task.Delay(1000);
}
}, cancellationToken);
return Disposable.Empty;
});
var publisher = coldObservable.Publish();
var connection = publisher.Connect();
new Subscriber(publisher, "One");
new Subscriber(publisher, "Two");
for (var i = 0; i < 5; i++)
{
if (i == 4)
{
cancellationSource.Cancel();
}
await Task.Delay(1000);
}
connection.Dispose();
}
}
}

Related

Async method deadlocks with TestScheduler in ReactiveUI

I'm trying to use the reactiveui test scheduler with an async method in a test.
The test hangs when the async call is awaited.
The root cause seems to be a command that's awaited in the async method.
[Fact]
public async Task Test()
=> await new TestScheduler().With(async scheduler =>
{
await SomeAsyncMethod();
// *** execution never gets here
Debugger.Break();
});
private async Task SomeAsyncMethod()
{
var command = ReactiveCommand.CreateFromTask(async () =>
{
await Task.Delay(100);
});
// *** this hangs
await command.Execute();
}
How can I do an async call in combination with the test scheduler that does not deadlock?
I'm using reactiveui 9.4.1
EDIT:
I've tried the WithAsync() method as suggested in Funks answer, but the behaviour is the same.

How can I do an async call in combination with the test scheduler?
In short
command.Execute() is a cold observable. You need to subscribe to it, instead of using await.
Given your interest in TestScheduler, I take it you want to test something involving time. However, from the When should I care about scheduling section:
threads created via "new Thread()" or "Task.Run" can't be controlled in a unit test.
So, if you want to check, for example, if your Task completes within 100ms, you're going to have to wait until the async method completes. To be sure, that's not the kind of test TestScheduler is meant for.
The somewhat longer version
The purpose of TestScheduler is to verify workflows by putting things in motion and verifying state at certain points in time. As we can only manipulate time on a TestScheduler, you'd typically prefer not to wait on real async code to complete, given there's no way to fast forward actual computations or I/O. Remember, it's about verifying workflows: vm.A has new value at 20ms, so vm.B should have new val at 120ms,...
So how can you test the SUT?
1\ You could mock the async method using scheduler.CreateColdObservable
public class ViewModelTests
{
[Fact]
public void Test()
{
string observed = "";
new TestScheduler().With(scheduler =>
{
var observable = scheduler.CreateColdObservable(
scheduler.OnNextAt(100, "Done"));
observable.Subscribe(value => observed = value);
Assert.Equal("", observed);
scheduler.AdvanceByMs(99);
Assert.Equal("", observed);
scheduler.AdvanceByMs(1);
Assert.Equal("Done", observed);
});
}
}
Here we basically replaced command.Execute() with var observable created on scheduler.
It's clear the example above is rather simple, but with several observables notifying each other this kind of test can provide valuable insights, as well as a safety net while refactoring.
Ref:
Answer by Paul Betts
Control Time with the TestScheduler
2\ You could reference the IScheduler explicitly
a) Using the schedulers provided by RxApp
public class MyViewModel : ReactiveObject
{
public string Observed { get; set; }
public MyViewModel()
{
Observed = "";
this.MyCommand = ReactiveCommand
.CreateFromTask(SomeAsyncMethod);
}
public ReactiveCommand<Unit, Unit> MyCommand { get; }
private async Task SomeAsyncMethod()
{
await RxApp.TaskpoolScheduler.Sleep(TimeSpan.FromMilliseconds(100));
Observed = "Done";
}
}
public class ViewModelTests
{
[Fact]
public void Test()
{
new TestScheduler().With(scheduler =>
{
var vm = new MyViewModel();
vm.MyCommand.Execute().Subscribe();
Assert.Equal("", vm.Observed);
scheduler.AdvanceByMs(99);
Assert.Equal("", vm.Observed);
scheduler.AdvanceByMs(1);
Assert.Equal("Done", vm.Observed);
});
}
}
Note
CreateFromTask creates a ReactiveCommand with asynchronous execution logic. There's no need to define the Test method as async or await the TestScheduler.
Within the With extension method's scope RxApp.TaskpoolScheduler = RxApp.MainThreadScheduler = the new TestScheduler().
b) Managing your own schedulers through constructor injection
public class MyViewModel : ReactiveObject
{
private readonly IScheduler _taskpoolScheduler;
public string Observed { get; set; }
public MyViewModel(IScheduler scheduler)
{
_taskpoolScheduler = scheduler;
Observed = "";
this.MyCommand = ReactiveCommand
.CreateFromTask(SomeAsyncMethod);
}
public ReactiveCommand<Unit, Unit> MyCommand { get; }
private async Task SomeAsyncMethod()
{
await _taskpoolScheduler.Sleep(TimeSpan.FromMilliseconds(100));
Observed = "Done";
}
}
public class ViewModelTests
{
[Fact]
public void Test()
{
new TestScheduler().With(scheduler =>
{
var vm = new MyViewModel(scheduler); ;
vm.MyCommand.Execute().Subscribe();
Assert.Equal("", vm.Observed);
scheduler.AdvanceByMs(99);
Assert.Equal("", vm.Observed);
scheduler.AdvanceByMs(0);
Assert.Equal("Done", vm.Observed);
});
}
}
Ref:
Kent Boogaert's Answer
Testing Rx code - ISchedulerProvider
Let's close ranks with another quote from Haacked:
Unfortunately, and this next point is important, the TestScheduler doesn’t extend into real life, so your shenanigans are limited to your asynchronous Reactive code. Thus, if you call Thread.Sleep(1000) in your test, that thread will really be blocked for a second. But as far as the test scheduler is concerned, no time has passed.

Have you tried to use ConfigureAwait(false) when calling nested method?
[Fact]
public async Task Test()
=> await new TestScheduler().With(async scheduler =>
{
// this hangs
await SomeAsyncMethod().ConfigureAwait(false);
// ***** execution will never get to here
Debugger.Break();
}

Please try using .ConfigureAwait(false) on all your async methods.
This will provide you non-blocking behavior.
[Fact]
public async Task Test()
=> await new TestScheduler().With(async scheduler =>
{
await SomeAsyncMethod().ConfigureAwait(false);
// *** execution never gets here
Debugger.Break();
}).ConfigureAwait(false);
private async Task SomeAsyncMethod()
{
var command = ReactiveCommand.CreateFromTask(async () =>
{
await Task.Delay(100).ConfigureAwait(false);
}).ConfigureAwait(false);
// *** this hangs
await command.Execute();
}
Another way to test whether the problem is related with ConfigureAwait is to port your project to Asp.Net Core and test it there.
Asp.net core does not need to use ConfigureAwait to prevent this blocking issue.
Check this for Reference

MassTransit Consumer Test Passing But Confusing Error Thrown

I'm trying to Unit test a MassTransit Consumer using the MassTransit.Testing Framework and the InMemoryTestHarness.
I'm able to successfully test that a message is sent for two separate consumers so far.
One of the consumers is also successfully consumed, but I get an error message as follows:
R-FAULT loopback://localhost/vhost/input_queue 49820000-5689-0050-3b5c-08d5ecc4708c Acme.Company.Messages.Commands.ISomeCommand Acme.Company.SomeService.Consumers.SomeCommandConsumer(00:00:00.2328493) Failure: The payload was not found: MassTransit.RabbitMqTransport.ModelContext, StackTrace: at GreenPipes.PipeExtensions.GetPayload[TPayload](PipeContext context) at MassTransit.DeferExtensions.Defer[T](ConsumeContext1 context, TimeSpan delay, Action2 callback)
The code at this point is attempting to defer the message for one minute so I wonder whether that is the reason for the missing payload???
The code is as follows:
[TestFixture]
public class SomeCommandConsumerTests
{
private InMemoryTestHarness _harness;
private Mock<ISomeRepository> _SomeRepositoryMock;
private Mock<IAnotherRepository> _AnotherRepositoryMock;
[OneTimeSetUp]
public async Task OneTimeInit()
{
_harness = new InMemoryTestHarness("vhost");
_harness.Consumer(() => new SomeCommandConsumer(_SomeRepositoryMock.Object, _AnotherRepositoryMock.Object));
await _harness.Start();
}
[SetUp]
public void Init()
{
_SomeRepositoryMock = new Mock<ISomeRepository>();
_AnotherRepositoryMock = new Mock<IAnotherRepository>();
_SomeRepositoryMock.Setup(x => x.UpdateSomeId(It.IsAny<SomeEnum>(), It.IsAny<int>()))
.Returns(Task.Factory.StartNew(() => { }));
_SomeRepositoryMock.Setup(x => x.UpdateProcMessage(It.IsAny<string>(), It.IsAny<int>()))
.Returns(Task.Factory.StartNew(() => { }));
_SomeRepositoryMock.Setup(
x => x.UpdateSomeProcStartTime(It.IsAny<int>()))
.Returns(Task.Factory.StartNew(() => { }));
_SomeRepositoryMock.Setup(
x => x.UpdateSomeProcEndTime(It.IsAny<int>()))
.Returns(Task.Factory.StartNew(() => { }));
}
[Test]
public async Task ProcessMessage_MethodCalledWithSomeCondition_MessageSent()
{
//Arrange
_SomeRepositoryMock.Setup(x => x.GetAsync(It.IsAny<int>())).ReturnsAsync(new Entity
{
Property1 = true,
SomeID = 12345
});
await _harness.InputQueueSendEndpoint.Send(new SomeCommand
{
MessageType = MessageTypeEnum.SomeMessgae,
SomeID = 12345
});
//Assert
_harness.Sent.Select<ISomeCommand>().Any().Should().BeTrue();
}
[Test]
public async Task ProcessMessage_MethodCalledWithSomeCondition_CorrectNextStepReturned()
{
//Arrange
_SomeRepositoryMock.Setup(x => x.GetAsync(It.IsAny<int>())).ReturnsAsync(new Control()
{
Property1 = true,
SomeID = 12345
});
await _harness.InputQueueSendEndpoint.Send(new SomeCommand
{
MessageType = MessageTypeEnum.SomeMessgae,
SomeID = 12345
});
//Assert
_harness.Consumed.Select<ISomeCommand>().Any().Should().BeTrue();
_harness.Consumed
.Select<ISomeCommand>()
.First()
.Context
.Message
.SomeID
.Should()
.Be(12345);
_harness.Consumed
.Select<ISomeCommand>()
.First()
.Context
.Message
.MessageProcessingResult
.Should()
.Be(MessageProcessingResult.DeferProcessing);
}
[OneTimeTearDown]
public async Task Teardown()
{
await _harness.Stop();
}
}
And the code that is being hit in the consumer is:
await context.Defer(TimeSpan.FromMinutes(1));
Basically, what am I missing, is it even an issue?

This is happening because you are using the in-memory test harness with a feature (Defer) that is supported by RabbitMQ. Defer tries to use the RabbitMQ model from the consumer to defer the message, and it isn't there, because in-memory doesn't know anything about it.
If you want to use a more generic solution, use Redeliver instead. You'll need to use the QuartzIntegration library with the in-memory test harness, but it does in-memory message redelivery using that scheduler.
You'll also need to update your RabbitMQ bus configuration to include the cfg.UseDelayedExchangeMessageScheduler(); so that RabbitMQ is used for message scheduling.

Unit Testing Cache Behaviour of Akavache with TestScheduler

So I'm trying to test caching behaviour in an app that's using Akavache.
My test looks like this:
using Akavache;
using Microsoft.Reactive.Testing;
using Moq;
using NUnit.Framework;
using ReactiveUI.Testing;
using System;
using System.Threading.Tasks;
[TestFixture]
public class CacheFixture
{
[Test]
public async Task CachingTest()
{
var scheduler = new TestScheduler();
// replacing the TestScheduler with the scheduler below works
// var scheduler = CurrentThreadScheduler.Instance;
var cache = new InMemoryBlobCache(scheduler);
var someApi = new Mock<ISomeApi>();
someApi.Setup(s => s.GetSomeStrings())
.Returns(Task.FromResult("helloworld")).Verifiable();
var apiWrapper = new SomeApiWrapper(someApi.Object, cache,
TimeSpan.FromSeconds(10));
var string1 = await apiWrapper.GetSomeStrings();
someApi.Verify(s => s.GetSomeStrings(), Times.Once());
StringAssert.AreEqualIgnoringCase("helloworld", string1);
scheduler.AdvanceToMs(5000);
// without the TestScheduler, I'd have to 'wait' here
// await Task.Delay(5000);
var string2 = await apiWrapper.GetSomeStrings();
someApi.Verify(s => s.GetSomeStrings(), Times.Once());
StringAssert.AreEqualIgnoringCase("helloworld", string2);
}
}
The SomeApiWrapper uses an internal api (mocked with new Mock<ISomeApi>()) that - for simplicity's sake - just returns a string. The problem now is that the second string is never returned. The SomeApiWrapper class that handles the caching looks like this:
using Akavache;
using System;
using System.Reactive.Linq;
using System.Threading.Tasks;
public class SomeApiWrapper
{
private IBlobCache Cache;
private ISomeApi Api;
private TimeSpan Timeout;
public SomeApiWrapper(ISomeApi api, IBlobCache cache, TimeSpan cacheTimeout)
{
Cache = cache;
Api = api;
Timeout = cacheTimeout;
}
public async Task<string> GetSomeStrings()
{
var key = "somestrings";
var cachedStrings = Cache.GetOrFetchObject(key, DoGetStrings,
Cache.Scheduler.Now.Add(Timeout));
// this is the last step, after this it just keeps running
// but never returns - but only for the 2nd call
return await cachedStrings.FirstOrDefaultAsync();
}
private async Task<string> DoGetStrings()
{
return await Api.GetSomeStrings();
}
}
Debugging only leads me to the line return await cachedStrings.FirstOrDefaultAsync(); - and it never finishes after that.
When I replace the TestScheduler with the standard (CurrentThreadScheduler.Instance) and the scheduler.AdvanceToMs(5000) with await Task.Delay(5000), everything works as expected but I don't want unit tests running for multiple seconds.
A similar test, where the TestScheduler is advanced past the cache timeout also succeeds. It's just this scenario, where the cache entry should not expire in between the two method calls.
Is there something I'm doing wrong in the way I'm using TestScheduler?

This is a fairly common problem when bouncing between the Task and the IObservable paradigms. It is further exacerbated by trying to wait before moving forward in the tests.
The key problem is that you are blocking* here
return await cachedStrings.FirstOrDefaultAsync();
I say blocking in the sense that the code can not continue to process until this statement yields.
On the first run the cache can not find the key, so it executes your DoGetStrings. The issue surfaces on the second run, where the cache is populated. This time (I guess) the fetching of the cached data is scheduled. You need to invoke the request, observe the sequence, then pump the scheduler.
The corrected code is here (but requires some API changes)
[TestFixture]
public class CacheFixture
{
[Test]
public async Task CachingTest()
{
var testScheduler = new TestScheduler();
var cache = new InMemoryBlobCache(testScheduler);
var cacheTimeout = TimeSpan.FromSeconds(10);
var someApi = new Mock<ISomeApi>();
someApi.Setup(s => s.GetSomeStrings())
.Returns(Task.FromResult("helloworld")).Verifiable();
var apiWrapper = new SomeApiWrapper(someApi.Object, cache, cacheTimeout);
var string1 = await apiWrapper.GetSomeStrings();
someApi.Verify(s => s.GetSomeStrings(), Times.Once());
StringAssert.AreEqualIgnoringCase("helloworld", string1);
testScheduler.AdvanceToMs(5000);
var observer = testScheduler.CreateObserver<string>();
apiWrapper.GetSomeStrings().Subscribe(observer);
testScheduler.AdvanceByMs(cacheTimeout.TotalMilliseconds);
someApi.Verify(s => s.GetSomeStrings(), Times.Once());
StringAssert.AreEqualIgnoringCase("helloworld", observer.Messages[0].Value.Value);
}
}
public interface ISomeApi
{
Task<string> GetSomeStrings();
}
public class SomeApiWrapper
{
private IBlobCache Cache;
private ISomeApi Api;
private TimeSpan Timeout;
public SomeApiWrapper(ISomeApi api, IBlobCache cache, TimeSpan cacheTimeout)
{
Cache = cache;
Api = api;
Timeout = cacheTimeout;
}
public IObservable<string> GetSomeStrings()
{
var key = "somestrings";
var cachedStrings = Cache.GetOrFetchObject(key, DoGetStrings,
Cache.Scheduler.Now.Add(Timeout));
//Return an observerable here instead of "blocking" with a task. -LC
return cachedStrings.Take(1);
}
private async Task<string> DoGetStrings()
{
return await Api.GetSomeStrings();
}
}
This code is green and runs sub-second.

Async TPL deadlock with third party lib aka wild goose chase

After spending a very frustrating and unproductive day on this, I'm posting here in search of help.
I am using a third-party library that initiates a network connection in an unknown manner (I do know however it's a managed wrapper for an unmanaged lib). It lets you know about the status of the connection by invoking an event StatusChanged(status).
Since obviously invoking the network is costly and I may not need it for my Service, I inject an AsyncLazy<Connection> which is then invoked if necessary. The Service is accessed by ParallelForEachAsync which is an extension I made to process Tasks concurrently, based on this post.
If accessed sequentially, all is well. Any concurrency, even 2 parallel tasks will result in a deadlock 90% of the time. I know it's definitely related to how the third-party lib interacts with my code because a) I am not able to reproduce the effect using the same structure but without invoking it and b) the event StatusChanged(Connecting) is received fine, at which point I assume the network operation is started and I never get a callback for StatusChanged(Connected).
Here's a as-faithful-as-possible repro of the code structure which doesn't reproduce the deadlock unfortunately.
Any ideas on how to go about resolving this?
class Program
{
static void Main(string[] args)
{
AsyncContext.Run(() => MainAsync(args));
}
static async Task MainAsync(string[] args)
{
var lazy = new AsyncLazy<Connection>(() => ConnectionFactory.Create());
var service = new Service(lazy);
await Enumerable.Range(0, 100)
.ParallelForEachAsync(10, async i =>
{
await service.DoWork();
Console.WriteLine("did some work");
}, CancellationToken.None);
}
}
class ConnectionFactory
{
public static Task<Connection> Create()
{
var tcs = new TaskCompletionSource<Connection>();
var session = new Session();
session.Connected += (sender, args) =>
{
Console.WriteLine("connected");
tcs.SetResult(new Connection());
};
session.Connect();
return tcs.Task;
}
}
class Connection
{
public async Task DoSomethinElse()
{
await Task.Delay(1000);
}
}
class Session
{
public event EventHandler Connected;
public void Connect()
{
Console.WriteLine("Simulate network operation with unknown scheduling");
Task.Delay(100).Wait();
Connected(this, EventArgs.Empty);
}
}
class Service
{
private static Random r = new Random();
private readonly AsyncLazy<Connection> lazy;
public Service(AsyncLazy<Connection> lazy)
{
this.lazy = lazy;
}
public async Task DoWork()
{
Console.WriteLine("Trying to do some work, will connect");
await Task.Delay(r.Next(0, 100));
var connection = await lazy;
await connection.DoSomethinElse();
}
}
public static class AsyncExtensions
{
public static async Task<AsyncParallelLoopResult> ParallelForEachAsync<T>(
this IEnumerable<T> source,
int degreeOfParallelism,
Func<T, Task> body,
CancellationToken cancellationToken)
{
var partitions = Partitioner.Create(source).GetPartitions(degreeOfParallelism);
bool wasBroken = false;
var tasks =
from partition in partitions
select Task.Run(async () =>
{
using (partition)
{
while (partition.MoveNext())
{
if (cancellationToken.IsCancellationRequested)
{
Volatile.Write(ref wasBroken, true);
break;
}
await body(partition.Current);
}
}
});
await Task.WhenAll(tasks)
.ConfigureAwait(false);
return new AsyncParallelLoopResult(Volatile.Read(ref wasBroken));
}
}
public class AsyncParallelLoopResult
{
public bool IsCompleted { get; private set; }
internal AsyncParallelLoopResult(bool isCompleted)
{
IsCompleted = isCompleted;
}
}
EDIT
I think I understand why it's happening but not sure how to solve it. While the context is waiting for DoWork, DoWork is waiting for the lazy connection.
This ugly hack seems to solve it:
Connection WaitForConnection()
{
connectionLazy.Start();
var awaiter = connectionLazy.GetAwaiter();
while (!awaiter.IsCompleted)
Thread.Sleep(50);
return awaiter.GetResult();
}
Any more elegant solutions?

I suspect that the 3rd-party library is requiring some kind of STA pumping. This is fairly common with old-style asynchronous code.
I have a type AsyncContextThread that you can try, passing true to the constructor to enable manual STA pumping. AsyncContextThread is just like AsyncContext except it runs the context within a new thread (an STA thread in this case).
static void Main(string[] args)
{
using (var thread = new AsyncContextThread(true))
{
thread.Factory.Run(() => MainAsync(args)).Wait();
}
}
or
static void Main(string[] args)
{
AsyncContext.Run(() => async
{
using (var thread = new AsyncContextThread(true))
{
await thread.Factory.Run(() => MainAsync(args));
}
}
}
Note that AsyncContextThread will not work in all STA scenarios. I have run into issues when doing (some rather twisted) COM interop that required a true UI thread (WPF or WinForms thread); for some reason the STA pumping wasn't sufficient for those COM objects.

TPL dataflow blocks inside a WCF duplex

I am new writer to SO, pls bear with me.
I have a WCF service with a duplex service contract. This service contract has an operation contact that suppose to do long data processing. I am constrained to limit the number of concurrent data processing to let's say max 3. My problem is that after the data processing I need to get back to the same service instance context so I call back my initiator endpoint passing the data processing result. I need to mention that due to various reasons I am constrained to TPL dataflows and WCF duplex.
Here is a demo to what I wrote so far
In a console library I simulate WCF calls
class Program
{
static void Main(string[] args)
{
// simulate service calls
Enumerable.Range(0, 5).ToList().ForEach(x =>
{
new System.Threading.Thread(new ThreadStart(async () =>
{
var service = new Service();
await service.Inc(x);
})).Start();
});
}
}
Here is what suppose to be the WCF service
// service contract
public class Service
{
static TransformBlock<Message<int>, Message<int>> transformBlock;
static Service()
{
transformBlock = new TransformBlock<Message<int>, Message<int>>(x => Inc(x), new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 3
});
}
static Message<int> Inc(Message<int> input)
{
System.Threading.Thread.Sleep(100);
return new Message<int> { Token = input.Token, Data = input.Data + 1 };
}
// operation contract
public async Task Inc(int id)
{
var token = Guid.NewGuid().ToString();
transformBlock.Post(new Message<int> { Token = token, Data = id });
while (await transformBlock.OutputAvailableAsync())
{
Message<int> message;
if (transformBlock.TryReceive(m => m.Token == token, out message))
{
// do further processing using initiator service instance members
// something like Callback.IncResult(m.Data);
break;
}
}
}
}
public class Message<T>
{
public string Token { get; set; }
public T Data { get; set; }
}
The operation contract is not really necessary to be async, but I needed the OutputAvailableAsync notification.
Is this a good approach or is there a better solution for my scenario?
Thanks in advance.

First, I think you shouldn't use the token the way you do. Unique identifiers are useful when communicating between processes. But when you're inside a single process, just use reference equality.
To actually answer your question, I think the (kind of) busy loop is not a good idea.
A simpler solution for asynchronous throttling would be to use SemaphoreSlim. Something like:
static readonly SemaphoreSlim Semaphore = new SemaphoreSlim(3);
// operation contract
public async Task Inc(int id)
{
await Semaphore.WaitAsync();
try
{
Thread.Sleep(100);
var result = id + 1;
// do further processing using initiator service instance members
// something like Callback.IncResult(result);
}
finally
{
Semaphore.Release();
}
}
If you really want to (or have to?) use dataflow, you can use TaskCompletionSource for synchronization between the operation and the block. The operation method would wait on the Task of the TaskCompletionSource and the block would set it when it finished computation for that message:
private static readonly ActionBlock<Message<int>> Block =
new ActionBlock<Message<int>>(
x => Inc(x),
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 3
});
static void Inc(Message<int> input)
{
Thread.Sleep(100);
input.TCS.SetResult(input.Data + 1);
}
// operation contract
public async Task Inc(int id)
{
var tcs = new TaskCompletionSource<int>();
Block.Post(new Message<int> { TCS = tcs, Data = id });
int result = await tcs.Task;
// do further processing using initiator service instance members
// something like Callback.IncResult(result);
}

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

How to Separate IObservable and IObserver - c#

Related

Async method deadlocks with TestScheduler in ReactiveUI

MassTransit Consumer Test Passing But Confusing Error Thrown

Unit Testing Cache Behaviour of Akavache with TestScheduler

Async TPL deadlock with third party lib aka wild goose chase

TPL dataflow blocks inside a WCF duplex

Categories

Resources