I am trying to process some tasks asynchronously using Rx, e.g.
var list = Enumerable.Range(0, 100)
.ToObservable()
.SelectMany(x => Observable.Start(() => {
Console.WriteLine("Processing {0} ...", x);
Thread.Sleep(100 * x % 3);
if (x > 90) {
Console.WriteLine("Procesing exception {0} > 90", x);
throw new Exception("Value too large");
}
Console.WriteLine("Processing {0} completed.", x);
return x;
}))
.Subscribe(
x => { Console.WriteLine("Next [{0}]", x); },
e => {
Console.WriteLine("Exception:");
Console.WriteLine(e.Message);
},
() => { Console.WriteLine("Complete"); }
);
The problem I have with this code is that the exception is not passed to the subscriber. So, after a lot of trying I gave up and decided to ask this simple question:
How do you handle the exceptions raised from within asynchronous methods within a SelectMany statement?
Just to make it clear, the final implementation is a synchroneous function call that may or may not throw an exception. The goal is to pass it on to the subscriber so that it can be further processed (in the specific case a message will be shown to the user).
Edit
I moved my findings down to an answer, so that I can mark this question as answered. Personally, I do not agree with self answering ... but sometimes there is no other way, so sorry for it.
Use Materialize to convert your OnError / OnCompleted messages into notifications.
For example,
observable.SelectMany(x => Observable.Start(fn).Materialize())
will get you the error / completion wrapped in a notification to be handled in your actual subscription point way down the line, as opposed to the error being terminated inside the SelectMany.
This is useful for most Async call operations because the method either fails or completes.
The answer
Actually the code is working correctly. However, the debugger breaks at the exceptions as the async operations are still executed in the background - well at least those that were already started when the first exception occurred. Threw me! If you run the code without debugger the exceptions are swallowed.So I guess the problem was really in front of the computer :-)
Still some clarifications on the Observable.Start as I assumed - an this correctly - that the implemtation should have actually some error handling implemented ... see Background.
Background
Observable.Start is a convenience method that uses the Observable.ToAsync method to turn a function/acion into an async operation. If you look at the implementation of the method you'll see that it already does the exception handling/forwarding.
public static Func<IObservable<TResult>> ToAsync<TResult>(this Func<TResult> function, IScheduler scheduler) {
if (function != null) {
if (scheduler != null) {
return () => {
AsyncSubject<TResult> asyncSubject = new AsyncSubject<TResult>();
scheduler.Schedule(() => {
TResult result = default(TResult);
try {
result = function();
} catch (Exception exception1) {
Exception exception = exception1;
asyncSubject.OnError(exception);
return;
}
asyncSubject.OnNext(result);
asyncSubject.OnCompleted();
});
return asyncSubject.AsObservable<TResult>();
};
} else {
throw new ArgumentNullException("scheduler");
}
} else {
throw new ArgumentNullException("function");
}
}
Related
I would like to introduce some specific error catching routine and then just retry subscribing to the same observable. Does this have any drawbacks:
public static IObservable<int> GetObservable()
{
return Observable.Interval(TimeSpan.Zero)
.Select(x =>
{
Console.WriteLine($"Throwing");
throw new Exception("ups");
return Observable.Return(1);
}).SelectMany(x => x)
.Catch<int, Exception>(ex =>
{
return GetObservable().SubscribeOn(Scheduler.Default);
});
}
Does the Catch actually hold some stuff and should not be used in this way? What are the implications? Should the Catch be treated as a "use once" routine and just get a Exception check on the Do and then at the bottom use Retry?
I have a basic ReactiveCommand. No async wizardry, just plain old ReactiveCommand.Create(). I Subscribe() with the overload that takes an exception handler, but never hit the breakpoint in said exception handler (I did not expect this). I subscribe to ThrownErrors, never hit the breakpoint in that exception handler either (I sort of expected this).
Here's the example code:
var myCommand = ReactiveCommand.Create();
// this does not seem to work
myCommand.Subscribe(
_ => { throw new Exception("oops"); },
ex => {
Console.WriteLine(ex.Mesage);
Debugger.Break();
});
//this does not seem to work either
myCommand.ThrownExceptions.Subscribe(
ex => {
Console.WriteLine(ex.Mesage);
Debugger.Break();
});
I did my homework and checked the questions and answers in the topic.
ReactiveUI exception handling
How to catch exception from ReactiveCommand?
I have checked the mail list as well, and found this:
https://groups.google.com/forum/#!topic/reactivexaml/Dkc-cSesKPY
So I decided to change this to some async solution:
var myCommand = ReactiveCommand.CreateAsyncObservable(_ => this.Throw());
myCommand.Subscribe(
_ => { Console.WriteLine("How did we get here?"); },
// this is not expected to work
ex => {
Console.WriteLine(ex.Message);
Debugger.Break();
});
// however, I sort of expect this to work
myCommand.ThrownExceptions.Subscribe(
ex => {
Console.WriteLine(ex.Message);
Debugger.Break();
});
[...]
private IObservable<object> Throw()
{
Debugger.Break();
throw new Exception("oops");
}
And yet, I never hit any of my breakpoints, except the one in the Throw() method. :(
What am I doing wrong? How am I supposed to catch exceptions here?
Edit:
I do, however, hit the exception handler breakpoint when I throw the exception from within the observable, like this
private IObservable<object> Throw()
{
Debugger.Break();
return Task.Factory.StartNew(() =>
{
throw new Exception("oops");
return new object();
}).ToObservable();
}
Question modified to: "am I capable of handling an exception from within the method and not the observable?"
This is a bug in the current release of ReactiveUI - the exception thrown while creating the 'execute' observable is swallowed and the internal state of the ReactiveCommand is left in a broken state. This has been fixed in the next release.
See this github issue for details.
I have a Parallel.Foreach loop which downloads files like so:
try
{
var parallelOptions = new ParallelOptions();
parallelOptions.MaxDegreeOfParallelism = 8;
int failedFiles = 0;
Parallel.ForEach(FilesToDownload, parallelOptions, tile =>
{
bool downloaded = DownloadTile(File);
if (downloaded)
{
//Downloaded :)
}
else
{
failedFiles++;
if (failedFiles > 10)
{
throw new Exception("10 Files Failed to download. Failing download");
}
}
parallelOptions.CancellationToken.ThrowIfCancellationRequested();
});
}
catch (Exception ex)
{
throw; //This throws it up to a main method that cancels my program
}
and I was wondering what is the correct way for throwing an Exception from inside the Parallel.Foreach method? In my instance I think I am going to see the exception thrown 8 times as soon as the first exception is thrown.
What is the correct way to throw exceptions in Parallel.ForEach loops?
First, better to use Interlocked.Increment(ref failedFiles) instead of failedFiles++. Otherwise it can happen that you have 10-15 failures, but you end up with a counter with a value something like 7-8, because of the lack of cache synchronization and the effect of compiler/jitter optimizations.
The loop of your program might throw more exceptions, but at the end it will be aggregated into a single AggregateException, and the outter catch() will receive that single exception instance. If you do not want more exceptions in the AggregateException, you can use == instead of >
if (Interlocked.Increment(ref failedFiles) == 10)
When an exception is thrown inside the loop, Parallel.ForEach prevents other iterations to start, then it waits for the currently running iterations to finish, then it aggregates all the exceptions it caught, packs it into an AggregateException and throws that single instance. It means that in your case a single exception will prevent further downloads.
You can use CancellationToken.Register. Also have a look at this example.
You can also stop your Parallel.Foreach and throw an exception.
This example demonstrates how to stop a For loop; however, you can stop a ForEach loop in the same way. See the snippet below.
private static void StopLoop()
{
double[] source = MakeDemoSource(1000, 1);
ConcurrentStack<double> results = new ConcurrentStack<double>();
// i is the iteration variable. loopState is a
// compiler-generated ParallelLoopState
Parallel.For(0, source.Length, (i, loopState) =>
{
// Take the first 100 values that are retrieved
// from anywhere in the source.
if (i < 100)
{
// Accessing shared object on each iteration
// is not efficient. See remarks.
double d = Compute(source[i]);
results.Push(d);
}
else
{
loopState.Stop();
return;
}
});
Console.WriteLine("Results contains {0} elements", results.Count());
}
If you want, you can use ConcurrentStack the same way to gather exceptions to return as soon as the loop is stopped.
I have this class for explaining my problem:
public class DataObserver: IDisposable
{
private readonly List<IDisposable> _subscriptions = new List<IDisposable>();
private readonly SomeBusinessLogicServer _server;
public DataObserver(SomeBusinessLogicServer server, IObservable<SomeData> data)
{
_server = server;
_subscriptions.Add(data.Subscribe(TryHandle));
}
private void TryHandle(SomeData data)
{
try
{
_server.MakeApiCallAsync(data).Wait();
}
catch (Exception)
{
// Handle exceptions somehow!
}
}
public void Dispose()
{
_subscriptions.ForEach(s => s.Dispose());
_subscriptions.Clear();
}
}
A) How can I avoid blocking inside the TryHandle() function?
B) How would you publish exceptions caught inside that function for handling them properly?
The Rx Design Guidelines provide a lot of useful advice when writing your own Rx operators:
http://go.microsoft.com/fwlink/?LinkID=205219
I'm sure I'll get lambasted for linking to an external article, but this link has been good for a couple of years and it's too big to republish on SO.
First, take a look at CompositeDisposable instead of re-implementing it yourself.
Other than that, there are many answers to your question. I have found that the best insight I've had when working with Rx is realizing that most cases where you want to subscribe are really just more chains in the observable you are building and you don't really want to subscribe but instead want to apply yet another transform to the incoming observable. And let some code that is further "on the edge of the system" and has more knowledge of how to handle errors do the actual subscribing
In the example you have presented:
A) Don't block by just transforming the IObservable<SomeData> into an IObservable<Task> (which is really better expressed as an IObservable<IObservable<Unit>>).
B) Publish exceptions by just ending the observable with an error or, if you don't want the exception to end the observable, exposing an IObservable<Exception>.
Here's how I'd re-write your example, assuming you did not want the stream to end on error, but instead just keep running after reporting the errors:
public static class DataObserver
{
public static IObservable<Exception> ApplyLogic(this IObservable<SomeData> source, SomeBusinessLogicServer server)
{
return source
.Select(data =>
{
// execute the async method as an observable<Unit>
// ignore its results, but capture its error (if any) and yield it.
return Observable
.FromAsync(() => server.MakeApiCallAsync(data))
.IgnoreElements()
.Select(_ => (Exception)null) // to cast from IObservable<Unit> to IObservable<Exception>
.Catch((Exception e) => Observable.Return(e));
})
// runs the Api calls sequentially (so they will not run concurrently)
// If you prefer to let the calls run in parallel, then use
// .Merge() instead of .Concat()
.Concat() ;
}
}
// Usage (in Main() perhaps)
IObservable<SomeData> dataStream = ...;
var subscription = dataStream.ApplyLogic(server).Subscribe(error =>
{
Console.WriteLine("An error occurred processing a dataItem: {0}", error);
}, fatalError =>
{
Console.WriteLine("A fatal error occurred retrieving data from the dataStream: {0}", fatalError);
});
When handling exceptions in TPL tasks I have come across two ways to handle exceptions. The first catches the exception within the task and returns it within the result like so:
var task = Task<Exception>.Factory.StartNew(
() =>
{
try
{
// Do Something
return null;
}
catch (System.Exception e)
{
return e;
}
});
task.ContinueWith(
r =>
{
if (r.Result != null)
{
// Handle Exception
}
});
The second is the one shown within the documentation and I guess the proper way to do things:
var task = Task.Factory.StartNew(
() =>
{
// Do Something
});
task.ContinueWith(
r =>
{
if (r.Exception != null)
{
// Handle Aggregate Exception
r.Exception.Handle(y => true);
}
});
I am wondering if there is anything wrong with the first approach? I have received 'unhandled aggregate exception' exceptions every now and again using this technique and was wondering how this can happen?
To clarify, I think the second pattern is the better one but I have a chunk of code which makes use of the first pattern and I am trying to find out if it needs re-factoring i.e. if it turns out that not all exceptions will be trapped.
The first approach assumes exceptions will be raised for every invocation. While this might be true, the exceptions don't seem "exceptional" and smells of a design issue. If the exceptions are not exceptional, then the result doesn't make much sense. The other problem is that if you do want a "result" (i.e. something other than Exception) you can't because the one and only Result slot is used for an Exception. Another problem is that you don't get the re-throwing of the exception back on the main thread (you could do that manually) so you don't get the catch semantics (i.e. you're using the Handle method).
The second method will be better understood by more people.