I have a couple of asynchronous APIs that use callbacks or events instead of async. I successfully used TaskCompletionSource to wrap them as described here.
Now, I would like to use an API that returns IObservable<T> and yields multiple objects. I read about Rx for .NET, which seems the way to go. However, I'm hesitant to include another dependency and another new paradigm, since I'm already using a lot of things that are new for me in this app (like XAML, MVVM, C#'s async/await).
Is there any way to wrap IObservable<T> analogously to how you wrap a single callback API? I would like to call the API as such:
foreach (var t in GetMultipleInstancesAsync()) {
var res = await t;
Console.WriteLine("Received item:", res);
}
If the observable emits multiple values, you can convert them to Task<T> and then add them to any IEnumerable structure.
Check IObservable ToTask. As discussed here, the observable must complete before awaiting otherwise more values might come over.
This guide here might do the trick for you too
public static Task<IList<T>> BufferAllAsync<T>(this IObservable<T> observable)
{
List<T> result = new List<T>();
object gate = new object();
TaskCompletionSource<IList<T>> finalTask = new TaskCompletionSource<IList<T>>();
observable.Subscribe(
value =>
{
lock (gate)
{
result.Add(value);
}
},
exception => finalTask.TrySetException(exception),
() => finalTask.SetResult(result.AsReadOnly())
);
return finalTask.Task;
}
If you would like to use Rx then you can use your returning list:
GetMultipleInstancesAsync().ToObservable().Subscribe(...);
You can subscribe the OnCompleted/OnError handler.
Also you can wrap it a task list:
var result = await Task.WhenAll(GetMultipleInstancesAsync().ToArray());
So you got an array of your results and you are done.
Related
I am trying to you use yield and return a result from converting X into Y in an async task. But, I am getting an error on select.
The error is:
Error CS1942 The type of the expression in the select clause is
incorrect. Type inference failed in the call to 'Select'.
public async Task<Result<dynamic>> GetYAsync(IEnumerable<X> infos)
{
return Task.WhenAll(from info in infos.ToArray() select async ()=>
{
yield return await new Y(info.Id, "Start");
});
}
Short answer: you can't use an asynchronous yield statement.
But in most cases, you don't need to. Using LINQ you can aggregate all tasks before passing them into Task.WaitAll. I simplified your example to return an IEnumerable<int>, but this will work with every type.
public class Program
{
public static Task<int> X(int x)
{
return Task.FromResult(x);
}
public static async Task<IEnumerable<int>> GetYAsync(IEnumerable<int> infos)
{
var res = await Task.WhenAll(infos.Select(info => X(info)));
return res;
}
public static async void Main()
{
var test = await GetYAsync(new [] {1, 2, 3});
Console.WriteLine(test);
}
}
Your example has another error await new Y(...), a constructor cannot be asynchronous, therefore I replaced it with an asynchronous function. (As hinted in the comments, it is technically possible to create a custom awaitable type and create this type with new, although this is rarely used),
The example above uses infos.Select to create a list of pending tasks, returned by invoking the function X. This list of tasks will then be awaited and returned.
This workaround should fit most cases. Real asynchronous iterators, as for example in JavaScript, are not supported in .Net.
Update: This feature is currently suggested as a language proposal: Async Streams. So maybe we will see this in the future.
Update: If you need asynchronous iterators, there are a few options currently available:
Reactive Stream, RX.Net, which provides you with asynchronous observable streams based on events.
There are implementations of asynchronous iterators or asynchronous enumerables AsyncEnumerable or
.Net Async Enumerable
You do not. Async Enum support (and yield is there to implement enumerable) comes with C# 8 some point in 2019 as it looks. So, for now the answer is simply that you do not.
The reason you get the error is that you can also not returna Result. Yield (return) is specific to implementing enumerations. Your method signature does not match.
I have a class library that (amongst other things) acts as a wrapper for an external client library for a web API.
My (simplified) code here takes a query string, generates a ReportUtilities object, uses this to download a report, then returns the report string back to the caller:
public string GetXmlReport(string queryString)
{
ReportUtilities reportUtil = new ReportUtilities(_user,queryString,"XML");
byte[] data = reportUtil.GetResponse().Download();
return Encoding.UTF8.GetString(data);
}
The issue is that this method is downloading data from a webservice using a syncronous method. I would like to make this available asyncronously in my library by adding a GetXmlReportAsync() method to my class.
Now, this would be straighforward if the ReportUtilities class provided a GenerateAndDownloadAsync() method returning Task<byte[]> but unfortunately it does not and it is in the external library so I am stuck with what it provides.
The ReportUtilities class does have a GetResponseAsync() method that returns void and provides a delegate for a a OnReadyCallback method together with an OnReadyCallback OnReady{get;set;} property.
I should add that .GetResponse() returns a ReportResponse object which does have a DownloadAsync() method but, again, this returns void rather than Task<byte[]>. Again ReportResponse comes with OnDownloadSuccessCallback delegate and OnDownloadSuccessCallback OnDownloadSuccess { get; set; } property.
It's almost as if the external library authors are 'rolling thier own' async API rather than using the one built into C#?
My question is: How can I implement a GetXmlReportAsync() method in my class to make the most efficient use of the asyncronous functions in the client library?
Obviously I could just do:
public async Task<string> GetXmlReportAsync(string queryString)
{
ReportUtilities reportUtil = new ReportUtilities(_user,queryString,"XML");
byte[] data = await Task.Run(() => { return reportUtil.GetResponse().Download(); });
return Encoding.UTF8.GetString(data);
}
but then the thread gets tied up with 2 syncronous input/output method calls to the external library: .GetResponse() and .Download() which surely isnt optimal?
Alternatively, I could imagine a situation where I just exposed a similar API to the external library in my own with clients having to provide callbacks for when thier reports are ready but I would much prefer to wrap this up into the more familiar async/await style API.
Am I trying to fit a square peg in a round hole or am I missing a neat way to wrap this into an async/await style API?
It's almost as if the external library authors are 'rolling thier own' async API rather than using the one built into C#?
A not uncommon situation for older libraries, particularly ones that were directly ported from other platforms/languages.
Am I trying to fit a square peg in a round hole or am I missing a neat way to wrap this into an async/await style API?
The pattern they're using is very similar to EAP, and there's a common pattern for converting EAP to TAP. You can do something similar with a few adjustments.
I recommend creating extension methods for the third-party library types that give you nice TAP endpoints, and then building your logic on top of that. That way, the TAP method doesn't mix concerns (translating asynchronous patterns, and doing business logic - i.e., converting to a string).
The ReportUtilities class does have a GetResponseAsync() method that returns void and provides a delegate for a a OnReadyCallback method together with an OnReadyCallback OnReady{get;set;} property.
Something like this, then:
public static Task<ReportResponse> GetResponseTaskAsync(this ReportUtilities #this)
{
var tcs = new TaskCompletionSource<ReportResponse>();
#this.OnReady = response =>
{
// TODO: check for errors, and call tcs.TrySetException if one is found.
tcs.TrySetResult(response);
};
#this.GetResponseAsync();
return tcs.Task;
}
Similarly for the next level:
public static Task<byte[]> DownloadTaskAsync(this ReportResponse #this)
{
var tcs = new TaskCompletionSource<byte[]>();
// TODO: how to get errors? Is there an OnDownloadFailed?
#this.OnDownloadSuccess = result =>
{
tcs.TrySetResult(result);
};
#this.DownloadAsync();
return tcs.Task;
}
Then your business logic can use the clean TAP endpoints:
public async Task<string> GetXmlReportAsync(string queryString)
{
ReportUtilities reportUtil = new ReportUtilities(_user, queryString, "XML");
var response = await reportUtil.GetResponseTaskAsync().ConfigureAwait(false);
var data = await response.DownloadTaskAsync().ConfigureAwait(false);
return Encoding.UTF8.GetString(data);
}
How can I implement a GetXmlReportAsync() method in my class to make
the most efficient use of the asyncronous functions in the client
library?
You could wrap the asynchoronous GetResponseAsync call with a TaskCompletionSource<string>. It would register the delegate once complete and set the completion of the task via SetResult. It would look something of this sort:
public Task<string> GetXmlReportAsync()
{
var tcs = new TaskCompletionSource<string>();
ReportUtilities reportUtil = new ReportUtilities(_user,queryString,"XML");
reportUtil.GetResponseAsync(callBack =>
{
// I persume this would be the callback invoked once the download is done
// Note, I am assuming sort sort of "Result" property provided by the callback,
// Change this to the actual property
byte[] data = callBack.Result;
tcs.SetResult(Encoding.UTF8.GetString(data));
});
return tcs.Task;
}
I'm trying to use the Reactive Extensions (Rx) to buffer an enumeration of Tasks as they complete. Does anyone know if there is a clean built-in way of doing this? The ToObservable extension method will just make an IObservable<Task<T>>, which is not what I want, I want an IObservable<T>, that I can then use Buffer on.
Contrived example:
//Method designed to be awaitable
public static Task<int> makeInt()
{
return Task.Run(() => 5);
}
//In practice, however, I don't want to await each individual task
//I want to await chunks of them at a time, which *should* be easy with Observable.Buffer
public static void Main()
{
//Make a bunch of tasks
IEnumerable<Task<int>> futureInts = Enumerable.Range(1, 100).Select(t => makeInt());
//Is there a built in way to turn this into an Observable that I can then buffer?
IObservable<int> buffered = futureInts.TasksToObservable().Buffer(15); //????
buffered.Subscribe(ints => {
Console.WriteLine(ints.Count()); //Should be 15
});
}
You can use the fact that Task can be converted to observable using another overload of ToObservable().
When you have a collection of (single-item) observables, you can create a single observable that contains the items as they complete using Merge().
So, your code could look like this:
futureInts.Select(t => t.ToObservable())
.Merge()
.Buffer(15)
.Subscribe(ints => Console.WriteLine(ints.Count));
I want to write a class to simplify the asynchronous programing, like string s = mylib.BeginInvoek(test,"1"); here is my code:
public T BeginInvokeExWithReturnValue<T>(Func<T> actionFunction)
{
ExecWithReturnType<T> execWtihReturnValue = new ExecWithReturnType<T>(actionFunction);
IAsyncResult iar = execWtihReturnValue.BeginInvoke(new AsyncCallback(EndInvokeExWithReturnValue<T>), execWtihReturnValue);
// how to code here to return value
}
private void EndInvokeExWithReturnValue<T>(IAsyncResult iar)
{
ExecWithReturnType<T> execWtihReturnValue = (ExecWithReturnType<T>)iar.AsyncState;
execWtihReturnValue.EndInvoke(iar);
}
this BeginInvokeExWithReturnValue function has no input parameter, but returns a value,
But I don't know how to return a value from BeginInvokeExWithReturnValue function. Anyone who know this, Could you pls help me ? thx very much.
What you are trying to do right now is not async; if you want to return the T, just use:
return actionFunction();
It will be less overhead.
If you want async, and you are on 4.0, then the TPL may be a good option:
public Task<T> BeginInvokeExWithReturnValue<T>(Func<T> actionFunction)
{
var task = new Task<T>(actionFunction);
task.Start();
return task;
}
Now the caller can use:
var task = BeginInvokeExWithReturnValue(() => Whatever());
and then when desired, check for completion, block (Wait) for completion, register continuations, etc. Or just:
var result = task.Result; // implicit wait
Console.WriteLine(result);
This allows you to seamlessly write async code. Or in C# 5.0, seamlessly write continuations:
var result = await task; // continuation - this is **not** a wait
Console.WriteLine(result);
As David pointed out, the Invoke method is probably what you want, but if you're looking to write you're own variant, you just need to type cast the value to the generic (T, in your example) to satisfy your comment.
return (T) iar;
Following from the comments,
There are 3 models of Asyncronous development in .NET
APM - (BeginXXX EndXXX) Which you are using here, when the long running task completes it calls back into your code in the EndXXX method
EAP - Event based. In this model, when the long running task completes, an event is raised to inform your code.
TPL - New in .NET 4, this is the 'Task' based version. it looks most like Syncronous programming to client code, using a fluent interface. Its calls back to your code using continueWith.
Hope this helps
With business logic encapsulated behind synchronous service calls e.g.:
interface IFooService
{
Foo GetFooById(int id);
int SaveFoo(Foo foo);
}
What is the best way to extend/use these service calls in an asynchronous fashion?
At present I've created a simple AsyncUtils class:
public static class AsyncUtils
{
public static void Execute<T>(Func<T> asyncFunc)
{
Execute(asyncFunc, null, null);
}
public static void Execute<T>(Func<T> asyncFunc, Action<T> successCallback)
{
Execute(asyncFunc, successCallback, null);
}
public static void Execute<T>(Func<T> asyncFunc, Action<T> successCallback, Action<Exception> failureCallback)
{
ThreadPool.UnsafeQueueUserWorkItem(state => ExecuteAndHandleError(asyncFunc, successCallback, failureCallback), null);
}
private static void ExecuteAndHandleError<T>(Func<T> asyncFunc, Action<T> successCallback, Action<Exception> failureCallback)
{
try
{
T result = asyncFunc();
if (successCallback != null)
{
successCallback(result);
}
}
catch (Exception e)
{
if (failureCallback != null)
{
failureCallback(e);
}
}
}
}
Which lets me call anything asynchronously:
AsyncUtils(
() => _fooService.SaveFoo(foo),
id => HandleFooSavedSuccessfully(id),
ex => HandleFooSaveError(ex));
Whilst this works in simple use cases it quickly gets tricky if other processes need to coordinate about the results, for example if I need to save three objects asynchronously before the current thread can continue then I'd like a way to wait-on/join the worker threads.
Options I've thought of so far include:
having AsyncUtils return a WaitHandle
having AsyncUtils use an AsyncMethodCaller and return an IAsyncResult
rewriting the API to include Begin, End async calls
e.g. something resembling:
interface IFooService
{
Foo GetFooById(int id);
IAsyncResult BeginGetFooById(int id);
Foo EndGetFooById(IAsyncResult result);
int SaveFoo(Foo foo);
IAsyncResult BeginSaveFoo(Foo foo);
int EndSaveFoo(IAsyncResult result);
}
Are there other approaches I should consider? What are the benefits and potential pitfalls of each?
Ideally I'd like to keep the service layer simple/synchronous and provide some easy to use utility methods for calling them asynchronously. I'd be interested in hearing about solutions and ideas applicable to C# 3.5 and C# 4 (we haven't upgraded yet but will do in the relatively near future).
Looking forward to your ideas.
Given your requirement to stay .NET 2.0 only, and not work on 3.5 or 4.0, this is probably the best option.
I do have three remarks on your current implementation.
Is there a specific reason you're using ThreadPool.UnsafeQueueUserWorkItem? Unless there is a specific reason this is required, I would recommend using ThreadPool.QueueUserWorkItem instead, especially if you're in a large development team. The Unsafe version can potentially allow security flaws to appear as you lose the calling stack, and as a result, the ability to control permissions as closely.
The current design of your exception handling, using the failureCallback, will swallow all exceptions, and provide no feedback, unless a callback is defined. It might be better to propogate the exception and let it bubble up if you're not going to handle it properly. Alternatively, you could push this back onto the calling thread in some fashion, though this would require using something more like IAsyncResult.
You currently have no way to tell if an asynchronous call is completed. This would be the other advantage of using IAsyncResult in your design (though it does add some complexity to the implementation).
Once you upgrade to .NET 4, however, I would recommend just putting this in a Task or Task<T>, as it was designed to handle this very cleanly. Instead of:
AsyncUtils(
() => _fooService.SaveFoo(foo),
id => HandleFooSavedSuccessfully(id),
ex => HandleFooSaveError(ex));
You can use the built-in tools and just write:
var task = Task.Factory.StartNew(
() => return _fooService.SaveFoo(foo) );
task.ContinueWith(
t => HandleFooSavedSuccessfully(t.Result),
TaskContinuationOptions.NotOnFaulted);
task.ContinueWith(
t => try { t.Wait(); } catch( Exception e) { HandleFooSaveError(e); },
TaskContinuationOptions.OnlyOnFaulted );
Granted, the last line there is a bit odd, but that's mainly because I tried to keep your existing API. If you reworked it a bit, you could simplify it...
Asynchronous interface (based on IAsyncResult) is useful only when you have some non-blocking call under the cover. The main point of the interface is to make it possible to do the call without blocking the caller thread.
This is useful in scenarios when you can make some system call and the system will notify you back when something happens (e.g. when a HTTP response is received or when an event happens).
The price for using IAsyncResult based interface is that you have to write code in a somewhat awkward way (by making every call using callback). Even worse, asynchronous API makes it impossible to use standard language constructs like while, for, or try..catch.
I don't really see the point of wrapping synchronous API into asynchronous interface, because you won't get the benefit (there will always be some thread blocked) and you'll just get more awkward way of calling it.
Of course, it makes a perfect sense to run the synchronous code on a background thread somehow (to avoid blocking the main application thread). Either using Task<T> on .NET 4.0 or using QueueUserWorkItem on .NET 2.0. However, I'm not sure if this should be done automatically in the service - it feels like doing this on the caller side would be easier, because you may need to perform multiple calls to the service. Using asynchronous API, you'd have to write something like:
svc.BeginGetFooId(ar1 => {
var foo = ar1.Result;
foo.Prop = 123;
svc.BeginSaveFoo(foo, ar2 => {
// etc...
}
});
When using synchronous API, you'd write something like:
ThreadPool.QueueUserWorkItem(() => {
var foo = svc.GetFooId();
foo.Prop = 123;
svc.SaveFoo(foo);
});
The following is a response to Reed's follow-up question. I'm not suggesting that it's the right way to go.
public static int PerformSlowly(int id)
{
// Addition isn't so hard, but let's pretend.
Thread.Sleep(10000);
return 42 + id;
}
public static Task<int> PerformTask(int id)
{
// Here's the straightforward approach.
return Task.Factory.StartNew(() => PerformSlowly(id));
}
public static Lazy<int> PerformLazily(int id)
{
// Start performing it now, but don't block.
var task = PerformTask(id);
// JIT for the value being checked, block and retrieve.
return new Lazy<int>(() => task.Result);
}
static void Main(string[] args)
{
int i;
// Start calculating the result, using a Lazy<int> as the future value.
var result = PerformLazily(7);
// Do assorted work, then get result.
i = result.Value;
// The alternative is to use the Task as the future value.
var task = PerformTask(7);
// Do assorted work, then get result.
i = task.Result;
}