When using IObservable.LastAsync() to force my console app to wait on the result of an API call using Flurl, that API call is never made and the main thread deadlocks and never returns from LastAsync(). My goals are:
Since this is a console app, I can't really "subscribe" to the API call since that would allow the main thread to continue, likely causing it to exit prior to the API call completing. So I need to block until the value is obtained.
The API call should be deferred until the first subscriber requests a value.
Second and onward subscribers should not cause another API call, instead the last value from the stream should be returned (this is the goal of using Replay(1))
Here is an example that reproduces the issue:
public static class Program
{
public static async Task Main(string[] args)
{
var obs = Observable.Defer(() =>
"https://api.publicapis.org"
.AppendPathSegment("entries")
.GetJsonAsync()
.ToObservable())
.Select(x => x.title)
.Replay(1);
var title = await obs.LastAsync();
Console.WriteLine($"Title 1: {title}");
}
}
How can I modify my example to ensure that all 3 requirements above are met? Why does my example cause a deadlock?
Replay returns "connectable" observable, and you need to call Connect() method on it to start it going. Without that call, it does not subscribe to the underlying observable and does not emit items to its own subscribers, so that's why you have a "deadlock".
In this case instead of manually connecting, you can use RefCount() extension method which will automatically connect it on first subscriber and disconnect on when last subscriber unsubscribes. So:
public static async Task Main(string[] args) {
var obs = Observable.Defer(() =>
"https://api.publicapis.org"
.AppendPathSegment("entries")
.GetJsonAsync()
.ToObservable())
.Select(x => x.count)
.Replay(1)
.RefCount();
// makes request
var title = await obs.LastAsync();
Console.WriteLine($"Title 1: {title}");
// does not make request, obtains from replay cache
title = await obs.LastAsync();
Console.WriteLine($"Title 2: {title}");
}
You can also use AutoConnect method:
.Replay(1)
.AutoConnect(1);
This will automatically connect on first subscriber but will never disconnect (in your case shouldn't matter).
Related
I am reproducing my Rx issue with a simplified test case below. The test below hangs. I am sure it is a small, but fundamental, thing that I am missing, but can't put my finger on it.
public class Service
{
private ISubject<double> _subject = new Subject<double>();
public void Reset()
{
_subject.OnNext(0.0);
}
public IObservable<double> GetProgress()
{
return _subject;
}
}
public class ObTest
{
[Fact]
private async Task SimpleTest()
{
var service = new Service();
var result = service.GetProgress().Take(1);
var task = Task.Run(async () =>
{
service.Reset();
});
await result;
}
}
UPDATE
My attempt above was to simplify the problem a little and understand it. In my case GetProgress() is a merge of various Observables that publish the download progress, one of these Observables is a Subject<double> that publishes 0 everytime somebody calls a method to delete the download.
The race condition identified by Enigmativity and Theodor Zoulias may(??) happen in real life. I display a view which attempts to get the progress, however, quick fingers delete it just in time.
What I need to understand a bit more is if the download is started again (subscription has taken place by now, by virtue of displaying a view, which has already made the subscription) and somebody again deletes it.
public class Service
{
private ISubject<double> _deleteSubject = new Subject<double>();
public void Reset()
{
_deleteSubject.OnNext(0.0);
}
public IObservable<double> GetProgress()
{
return _deleteSubject.Merge(downloadProgress);
}
}
Your code isn't hanging. It's awaiting an observable that sometimes never gets a value.
You have a race condition.
The Task.Run is sometimes executing to completion before the await result creates the subscription to the observable - so it never sees the value.
Try this code instead:
private async Task SimpleTest()
{
var service = new Service();
var result = service.GetProgress().Take(1);
var awaiter = result.GetAwaiter();
var task = Task.Run(() =>
{
service.Reset();
});
await awaiter;
}
The line await result creates a subscription to the observable. The problem is that the notification _subject.OnNext(0.0) may occur before this subscription, in which case the value will pass unobserved, and the await result will continue waiting for a notification for ever. In this particular example the notification is always missed, at least in my PC, because the subscription is delayed for around 30 msec (measured with a Stopwatch), which is longer than the time needed for the task that resets the service to complete, probably because the JITer must load and compile some RX-related assembly. The situation changes when I do a warm-up by calling new Subject<int>().FirstAsync().Subscribe() before running the example. In that case the notification is observed almost always, and the hanging is avoided.
I can think of two robust solutions to this problem.
The solution suggested by Enigmativity, to create an awaitable subscription before starting the task that resets the service. This can be done with either GetAwaiter or ToTask.
To use a ReplaySubject<T> instead of a plain vanilla Subject<T>.
Represents an object that is both an observable sequence as well as an observer. Each notification is broadcasted to all subscribed and future observers, subject to buffer trimming policies.
The ReplaySubject will cache the value so that it can be observed by the future subscription, eliminating the race condition. You could initialize it with a bufferSize of 1 to minimize the memory footprint of the buffer.
Sometimes I need to start an async job which works very slow. I don't care if that job success and I need to continue working on my current thread.
Like sometimes I need to send an Email or SMS which works very slow. I need to respond to the web client as soon as possible so I don't want to await it.
I have googled this question and some articles suggest me to write like this:
// This method has to be async
public async Task<Response> SomeHTTPAction()
{
// Some logic...
// ...
// Send an Email but don't care if it successfully sent.
Task.Run(() => _emailService.SendEmailAsync());
return MyRespond();
}
Or like this:
// This method has to be async
public async Task<Response> SomeHTTPAction()
{
// Some logic...
// ...
// Send an Email but don't care if it successfully sent.
Task.Factory.StartNew(() => _emailService.SendEmailAsync());
return MyRespond();
}
There will be a warning says: before the call is completed. Consider applying the 'await' operator to the result of the call.
So what if I really awaited it? What is the best practice in C# to 'fire and forget', just call an async method without waiting for its completion?
A standalone discard is the best way to avoid this warning.
_ = Task.Run(() => _emailService.SendEmailAsync());
Discards are dummy variables and can be used to ignore the Task object returned by an asynchronous operation.
https://learn.microsoft.com/en-us/dotnet/csharp/discards#a-standalone-discard
If you truly just want to fire and forget. Simply don't call use await.
// It is a good idea to add CancellationTokens
var asyncProcedure = SomeHTTPAction(cancellationToken).ConfigureAwait(false);
// Or If not simply do:
var asyncProcedure = SomeHTTPAction().ConfigureAwait(false);
If you want to use the result output later its gets trickier. But if it is truly fire and forget the above should work
A Cancellation token allows interrupts and canceling procedures. If you are using Cancellation token you will need to use it everywhere from the retrieval straight through to the calling method (Turtles all the way down).
I used ConfigureAwait(false) to prevent deadlocks. Here for more information
EDIT
See the second answer that uses 'Task.Factory.StartNew' I gave this answer some time ago. At the time I didn't realise that the way I did it at the time doesn't ensure completion.
If you need to use async in your function you can also use a discard variable and don't use await. This is also usefull if you have multiple async function calls but you don't need to wait for all of them.
public async function(){
var tmp = await asyncfunction();
...
_ = _httpClient.PutAsync(url, content);
...
}
As Amadan told in the comment that, you need to remove async from your function. then it will stop giving you the warning.
// This method has to be async
public Response SomeHTTPAction()
{
// Some logic...
// ...
// Send an Email but don't care if it successfully sent.
Task.Factory.StartNew(() => _emailService.SendEmailAsync());
return MyRespond();
}
and Task.Factory.StartNew(() => _emailService.SendEmailAsync()); will indeed work on a new thread.
It all depends on what your Async method accepts. Normally it will accept a "special" class that also holds an event. You can subscribe your callback method to that event and pass it along with the method. When it's finished, your callback method will be called.
An example of this (for sockets) would be:
public void CreateSocket()
{
Socket s = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
SocketAsyncEventArgs sockAsync = new SocketAsyncEventArgs();
sockAsync.Completed += SockAsync_Completed;
s.ConnectAsync(sockAsync);
}
private void SockAsync_Completed(object sender, SocketAsyncEventArgs e)
{
//Do stuff with your callback object.
}
It all depends on what the method you are trying to call can accept. I would look at the documentation for more help on that specifically.
I am curious why this hasn't been suggested.
new Thread(() =>
{
Thread.CurrentThread.IsBackground = true;
//what ever code here...e.g.
DoSomething();
UpdateSomething();
}).Start();
It just fires off a separate thread.
I'm using Reactive Extensions in combination with async/await to simplify my socket protocol implementation. There's some actions that have to be performed when specific message arrives (e.g. send 'pong' to each 'ping' -message) and there's also some method's where we have to wait for some specific responses asynchronously. The following example illustrates this:
private Subject<string> MessageReceived = new Subject<string>();
//this method gets called every time a message is received from socket
internal void OnReceiveMessage(string message)
{
MessageReceived.OnNext(message);
ProcessMessage(message);
}
public async Task<string> TestMethod()
{
var expectedMessage = MessageReceived.Where(x => x.EndsWith("D") && x.EndsWith("F")).FirstOrDefaultAsync();
await SendMessage("ABC");
//some code...
//if response we are waiting for comes before next row, we miss it
return await expectedMessage;
}
TestMethod() sends "ABC" to the socket and continues when for example "DEF" is received (there might be some other messages before that).
This works almost, but there's a race condition. It seems that this code won't listen for messages until return await expectedMessage; And this is a problem, since sometimes the message arrives before that.
FirstOrDefaultAsync won't work here nicely: It doesn't subscribe until the await line, which leaves you with a race condition (as you point out). Here's how you can replace it:
var expectedMessage = MessageReceived
.Where(x => x.EndsWith("D") && x.EndsWith("F"))
.Take(1)
.Replay(1)
.RefCount();
using (var dummySubscription = expectedMessage.Subscribe(i => {}))
{
await SendMessage("ABC");
//Some code... goes here.
return await expectedMessage;
}
.Replay(1) makes sure that a new subscription gets the most recent entry, assuming one exists. It only works though if there's a subscriber listening, hence dummySubscription.
I have a static collection, say of tasks to call remote rest api:
static ConcurrentBag<Task<HttpResponseMessage>> _collection = new ConcurrentBag<Task<HttpResponseMessage>>();
static void Main(string[] args)
{
Task.Factory.StartNew(() => Produce());
Task.Factory.StartNew(() => Consume());
Console.ReadKey();
}
One thread adds new items into it:
private static void Produce()
{
while (true)
{
var task = HttpClientFactory.Create().GetAsync("http://example.com");
_collection.Add(task);
Thread.Sleep(500);
}
}
And another thread should process those items:
private static void Consume()
{
_collection.ToObservable()
.Subscribe(
t => Console.WriteLine("++"),
ex => Console.WriteLine(ex.Message),
() => Console.WriteLine("Done"));
}
But it runs only once and completes prematurely. So output is;
++
Done
It would be interesting if it worked like that... but sadly it doesn't. The ToObservable extension method is defined on the IEnumerable<T> interface - so it's getting a point-in-time snap shot of the collection.
You need a collection than can be observed, such as ObservableCollection. With this, you can respond to add events to feed an Rx pipeline (perhaps by wiring the CollectionChanged event up with Observable.FromEventPattern). Bear in mind that this collection doesn't support concurrent adds. Such a technique is one way to "enter the monad" (i.e. obtain an IObservable<T>).
Equivalent is adding your request payloads to a Subject. Either way, you can then project them into asynchronous requests. So say (for arguments sake), your Produce signature looked like this:
private static async Task<HttpResponseMessage> Produce(string requestUrl)
Then you might construct an observable to convert the requestUrls to async web requests using your Produce method like so:
var requests = new Subject<string>();
var responses = requests.SelectMany(
x => Observable.FromAsync(() => Produce(x)));
responses.Subscribe(
t => Console.WriteLine("++"),
ex => Console.WriteLine(ex.Message),
() => Console.WriteLine("Done"));
And submit each request with something like:
requests.OnNext("http://myurl");
If you need concurrent adds, see Observable.Synchronize.
If you need to control the thread(s) that handle the responses, use ObserveOn which I wrote a lengthy explanation of here.
I'm trying to create an async unit test for the project, but cannot understand how to wait for the async subject to complete:
[Test]
public async void MicroTest()
{
var value = 2;
var first = new AsyncSubject<int>();
var second = new AsyncSubject<int>();
first.Subscribe(_ =>
{
value = _;
second.OnCompleted();
});
first.OnNext(1);
// how to wait for the second subject to complete?
Assert.AreEqual(value, 1);
}
Sync version of this test is works well:
[Test]
public void MicroTest()
{
var value = 2;
var first = new Subject<int>();
var second = new Subject<int>();
first.Subscribe(_ =>
{
value = _;
second.OnCompleted();
});
first.OnNext(1);
Assert.AreEqual(value, 1);
}
AsyncSubject versus Subject
First off, it's worth pointing out that AsyncSubject<T> is not an asynchronous version of Subject<T>. Both are in fact free-threaded* (see footnote).
AsyncSubject is a specialization of Subject intended to be used to model an operation that completes asynchronously and returns a single result. It has two noteworthy features:
Only the last result is published
The result is cached and is available to observers subscribing after it has completed.
It is used internally in various places, including by the ToObservable() extension method defined on Task and Task<T>.
The issue with the test
Recall AsyncSubject<T> will only return the final result received. It does this by waiting for OnCompleted() so it knows what the final result is. Because you do not call OnCompleted() on first your test is flawed as the OnNext() handler - the lambda function passed in your Subscribe call - will never be invoked.
Additionally, it is invalid not to call OnNext() at least once on an AsyncSubject<T>, so when you call await second; you will get an InvalidOperationException if you haven't done this.
If you write your test as follows, all is well:
[Test]
public async void MicroTest()
{
var value = 2;
var first = new AsyncSubject<int>();
var second = new AsyncSubject<int>();
first.Subscribe(_ =>
{
// won't be called until an OnCompleted() has
// been invoked on first
value = _;
// you must send *some* value to second
second.OnNext(_);
second.OnCompleted();
});
first.OnNext(1);
// you must do this for OnNext handler to be called
first.OnCompleted();
// how to wait for the second subject to complete
await second;
Assert.AreEqual(value, 1);
}
About asynchronous tests
As a general rule I would avoid writing asynchronous tests that could wait forever. This gets particularly annoying when it causes resource drains on build servers. Use some kind of timeout e.g:
await second.Timeout(TimeSpan.FromSeconds(1));
No need to handle the exception since that is enough for the test to fail.
**I've borrowed this term from the COM lexicon. In this sense I mean that they, as with most of the Rx framework components, will generally run on whatever thread you happen to invoke their methods on. Being free-threaded doesn't necessarily mean being fully thread safe though. In particular, unlike AsyncSubject<T>, Subject<T> doesn't protect you from the Rx grammar violation of making overlapping calls to OnNext. Use Subject.Synchronize or Observable.Synchronize for this protection.*