I've a method called by my controller and i'm trying to add new value during the iteration
public async Task<MyResult> GetItems(int itemId) {
try
{
//returns a lists of items
var resList = await _unit.repository.GetAll(x => x.ItemId.Equals(itemId));
if(resList.Count() == 0)
throw new Exception(err.Message);
//Here i need to list the sub-items, but the method returns async
resList.ToList().ForEach(async res => {
switch (res.Type)
{
case Itemtype.None :
res.SubItems = await _unit.repository.GetAll(y => y.ItemId(res.ItemId));
break;
case Itemtype.Low :
//get from another place
break;
}
});
return new MyResult {
res = resList
};
}
catch (Exception ex)
{
throw ex;
}
}
It was shown my Items, but without the sub-items
Note that using await inside a foreach loop will pause the iteration until the Task completes.
This can lead to pretty poor performance if you have a large number of items.
Allow the tasks to run simultaneously:
var tasks = resList.Select(async res =>
{
switch (res.Type)
{
case Itemtype.None :
res.SubItems = await _unit.repository.GetAll(y => y.ItemId(res.ItemId));
break;
case Itemtype.Low :
//get from another place
break;
}
});
Then use Task.WhenAll to allow them all to complete:
await Task.WhenAll(tasks);
Enumerable.Select works differently to List<T>.ForEach in that it returns from the provided delegate, which in this case is Task.
It's important to realize that the await keyword will return when it acts on an incomplete Task. Usually it returns its own incomplete Task that the caller can use to wait until it's done, but if the method signature is void, it will return nothing.
Whatever that ForEach() is, it's likely that it does not accept a delegate that returns a Task, which means that your anonymous method (async res => { ... }) has a void return type.
That means your anonymous method returns as soon as the network request in GetAll() is sent. Because ForEach() isn't waiting for it, it moves on to the next iteration. By the time ForEach() is done everything, all it has done is sent the requests, but not waited for them. So by the time you get to your return statement, you can't be sure anything has been done.
Replace that .ForEach() call with just a regular foreach so that you aren't doing that work inside a void delegate. Then you should see it behave more like you expect. Even better, use Johnathan's answer to start all the tasks inside the loop, then wait for them after.
Microsoft has a very well written series of articles on Asynchronous programming with async and await that I think you will benefit from reading. You can find the rest of the series in the table of contents on the left side of that page.
Switch from resList.ToList().ForEach(...) to ordinary foreach:
foreach (var res in resList)
{
switch (res.Type)
{
case Itemtype.None:
res.SubItems = await _unit.repository.GetAll(y => y.ItemId(res.ItemId));
break;
case Itemtype.Low:
//get from another place
break;
}
}
It seems that your are using List<T>.ForEach. It accepts Action<T> and since the action you pass to it is asynchronous your current code will just create resList.Count() of tasks and proceed to return statement without actually awaiting them. Simple reproducer can look like this:
class MyClass { public int i; }
var col = new[] { new MyClass { i = 1 }, new MyClass { i = 2 } };
col.ToList().ForEach(async i => { await Task.Delay(1); i.i *= 10; });
//col.ToList().ForEach(i => { i.i *= 10; });
Console.WriteLine(string.Join(" ", col.Select(mc => mc.i))); // will print "1 2"
Related
I have a chain of task-returning methods, all returning some Task<SomeResponse<T>>. SomeResponse<T> is a generic response class exposing properties like whether the response was successful (IsSuccess), if it's successful a T Data property containing the returning object, and if not an accompanying error message.
Let's assume I have 3 such methods (all of them returning SomeResponse<T>). I only want to keep executing the tasks one-by-one until one of them fails or all of them succeed. The flow would look like this:
var first = await firtTask(someParam);
if (!first.IsSuccess) return first;
var second = await secondTask(first.Data);
if (!second.IsSuccess) return second;
var third = await thirdTask(second.Data);
return third; // doesn't matter if it succeeded or not as it's the last one, no need to check.
My issue here is that the SomeResponse<T> of each call needs to be validated for success before proceeding to the next await, which adds a lot of repetitive validation code. Checking if each task completed successfully is not enough, as I then have to inspect it's SomeResponse<T>.IsSuccess property before proceeding to the next task.
I tried creating an extension method on top of Task<SomeResponse<T>> for this:
public static Task<SomeResponse<T>> OnSuccessChainAsync<T>(this Task<SomeResponse<T>> startingTask, Func<T, Task<SomeResponse<T>>> continuationTask)
{
// omitting null checks etc
var continuation = startingTask.ContinueWith(
async previousTask =>
{
var response = await previousTask.ConfigureAwait(false);
if (!response.IsSuccess)
{
return response;
}
return await continuationTask(response.Data).ConfigureAwait(false);
}, TaskScheduler.Current);
return continuation.Unwrap();
}
This now allows me to write:
public override Task<SomeResponse<TValue>> AddAsync(TValue someValue)
{
return firstTask(someValue)
.OnSuccessChainAsync(secondTask)
.OnSuccessChainAsync(thirdTask);
}
I'm not sure if I'm heading in the wrong direction here. I am mixing async-await with TPL's ContinueWith, and on top of that I get a VSTHRD003 Avoid awaiting foreign Tasks from my analyzers.
Don't mix the old-style ContinueWith with async/await.
In fact, try and avoid ContinueWith completely: it's horrendously complex with lots of bits of subtle behaviour, half of which you never want to think about and half of which are much more clearly expressed with async/await.
Why not simplify things a bit:
public static async Task<SomeResponse<T>> ExecuteInSequence<T>(
T firstData,
params Func<T, Task<Response<T>>>[] funcs)
{
T data = firstData;
foreach (var func in funcs)
{
var response = await func(data);
if (!response.IsSuccess)
{
return response;
}
data = response.Data;
}
return data;
}
Then you can write:
ExecuteInSequence(someValue, task1, task2, task3);
There's no mixing of anything, no chaining, just a straightforward loop.
If you were going to write this as an extension method on Task<SomeResponse<T>>, I'd still keep everything as awaits:
public static async Task<SomeResponse<T>> OnSuccessChainAsync<T>(
this Task<SomeResponse<T>> startingTask,
Func<T, Task<SomeResponse<T>>> continuationTask)
{
// startingTask will probably have already completed (especially if
// it's one which we created on a previous invocation), in which case
// this await will be synchronous.
var result = await startingTask;
if (!result.IsSuccess)
{
return result;
}
return await continuationTask(result.Data);
}
I have this iterator- return type is IEnumerable<Task<Event>> so that I can await for each item later:
private IEnumerable<Task<Event>> GetEventsAsync(long length)
{
var pos = _reader.BaseStream.Position;
while (_reader.BaseStream.Position - pos < length)
{
yield return ReadEvent(); // this method is async
}
}
Now I want to pass this in to constructor.
private async Task<EventManager> ReadEventManagerAsync()
{
// some other stuff
var length = Reverse(await _reader.ReadInt32()); // bytes to read
var events = GetEventsAsync(length); // cant await iterator. so use linq
return new EventManager(events.Select(async e => await e).Select(x => x.Result));
}
The constructor takes this parameter.
internal EventManager([NotNull, NoEnumeration]IEnumerable<Event> events) {...}
Will this code run asynchronously?
Because I cant await inside lambda even if method is marked as async. I tried to do some hack.
I Understand why Select(async e => await e) returns IEnumerable<Task<Event>> because async e => await e is async method, which return type must be wrapped inside Task
My question is will .Select(x => x.Result) still run async? because I'm awaiting items before this so this shouldn't be a problem right? its like writing this
await e;
return e.Result; // is it safe?
I don't want to use Task.WhenAll because that will enumerate all stuff and that's the thing I'm trying to avoid. I want to keep this iterator untouched until its passed in constructor. I want to keep the execution deferred. (I will use factory method if this approach is not possible)
Will this code run asynchronously?
No. When the enumerable is enumerated, it will run synchronously, complete with possibility of deadlocks.
Select(async e => await e) doesn't do anything. It's the same as Select(e => e). It is not the same as await e;.
The whole converting-stream-to-IEnumerable<T> is extremely odd. Perhaps you meant to convert it to IObservable<T>?
I will use factory method if this approach is not possible
Yes, you cannot use await in a constructor, and blocking can cause deadlocks. An asynchronous factory method is probably your best bet.
No, it will run synchronously (constructors can't be async after all).
That doesn't mean your performance is impacted if (since) you never actually execute the tasks. events.Select(async e => await e).Select(x => x.Result) returns an enumerable after all. It won't call the first Select, which means that the awaited code is never called.
You can use WhenAll for this, and await for it asynchronously.
private async Task<EventManager> ReadEventManagerAsync()
{
// some other stuff
var length = Reverse(await _reader.ReadInt32()); // bytes to read
var events = GetEventsAsync(length); // cant await iterator. so use linq
await Task.WhenAll(events);
return new EventManager(events.Select(x => x.Result));
}
yield does not work with await as expected, may be next c# version will have better provision. As of now, you just have to await and store event in a list. Also if your ReadEvent depends on Stream.Position you cannot use Task.WhenAll or any other way but simply enumerate and store result in list.
private async Task<IEnumerable<Event>> GetEventsAsync(long length)
{
List<Event> events = new List<Event>();
var pos = _reader.BaseStream.Position;
while (_reader.BaseStream.Position - pos < length)
{
// if ReadEvent depends on _reader.BaseStream.Postion
// you cannot use Task.WhenAll, because it executes all
// tasks in parallel, where else, you must await to finish
// your previous ReadEvent
Event e = await ReadEvent(); // this method is async
events.Add(e);
}
return events;
}
I seem to be experiencing a deadlock with the following code, but I do not understand why.
From a certain point in code I call this method.
public async Task<SearchResult> Search(SearchData searchData)
{
var tasks = new List<Task<FolderResult>>();
using (var serviceClient = new Service.ServiceClient())
{
foreach (var result in MethodThatCallsWebservice(serviceClient, config, searchData))
tasks.Add(result);
return await GetResult(tasks);
}
Where GetResult is as following:
private static async Task<SearchResult> GetResult(IEnumerable<Task<FolderResult>> tasks)
{
var result = new SearchResult();
await Task.WhenAll(tasks).ConfigureAwait(false);
foreach (var taskResult in tasks.Select(p => p.MyResult))
{
foreach (var folder in taskResult.Result)
{
// Do stuff to fill result
}
}
return result;
}
The line var result = new SearchResult(); never completes, though the GUI is responsive because of the following code:
public async void DisplaySearchResult(Task<SearchResult> searchResult)
{
var result = await searchResult;
FillResultView(result);
}
This method is called via an event handler that called the Search method.
_view.Search += (sender, args) => _view.DisplaySearchResult(_model.Search(args.Value));
The first line of DisplaySearchResult gets called, which follows the path down to the GetResult method with the Task.WhenAll(...) part.
Why isn't the Task.WhenAll(...) ever completed? Did I not understand the use of await correctly?
If I run the tasks synchronously, I do get the result but then the GUI freezes:
foreach (var task in tasks)
task.RunSynchronously();
I read various solutions, but most were in combination with Task.WaitAll() and therefore did not help much. I also tried to use the help from this blogpost as you can see in DisplaySearchResult but I failed to get it to work.
Update 1:
The method MethodThatCallsWebservice:
private IEnumerable<Task<FolderResult>> MethodThatCallsWebservice(ServiceClient serviceClient, SearchData searchData)
{
// Doing stuff here to determine keys
foreach(var key in keys)
yield return new Task<FolderResult>(() => new FolderResult(key, serviceClient.GetStuff(input))); // NOTE: This is not the async variant
}
Since you have an asynchronous version of GetStuff (GetStuffAsync) it's much better to use it instead of offloading the synchronous GetStuff to a ThreadPool thread with Task.Run. This wastes threads and limits scalability.
async methods return a "hot" task so you don't need to call Start:
IEnumerable<Task<FolderResult>> MethodThatCallsWebservice(ServiceClient serviceClient, SearchData searchData)
{
return keys.Select(async key =>
new FolderResult(key, await serviceClient.GetStuffAsync(input)));
}
You need to start your tasks before you return them. Or even better use Task.Run.
This:
yield return new Task<FolderResult>(() =>
new FolderResult(key, serviceClient.GetStuff(input)))
// NOTE: This is not the async variant
Is better written as:
yield return Task.Run<FolderResult>(() =>
new FolderResult(key, serviceClient.GetStuff(input)));
We're requesting thousands of objects back from Dapper and hit the parameter limit (2100) so have decided to load them in chunks.
I thought it would be a good opportunity to try Async Await - this the first time I've had a go so maybe making a school boy error!
Breakpoints are being hit, but the whole thing is just not returning. It's not throwing an error - it just seems like everything is going in a black hole!
Help please!
This was my original method - it now calls the Async method
public List<MyObject> Get(IEnumerable<int> ids)
{
return this.GetMyObjectsAsync(ids).Result.ToList();
} //Breakpoint on this final bracket never gets hit
I added this method to split the ids into chunks of 1000 and then await for the tasks to complete
private async Task<List<MyObject>> GetMyObjectsAsync(IEnumerable<int> ids)
{
var subSets = this.Partition(ids, 1000);
var tasks = subSets.Select(set => GetMyObjectsTask(set.ToArray()));
//breakpoint on the line below gets hit ...
var multiLists = await Task.WhenAll(tasks);
//breakpoint on line below never gets hit ...
var list = new List<MyObject>();
foreach (var myobj in multiLists)
{
list.AddRange(myobj);
}
return list;
}
and below is the task ...
private async Task<IEnumerable<MyObject>> GetMyObjectsTask(params int[] ids)
{
using (var db = new SqlConnection(this.connectionString))
{
//breakpoint on the line below gets hit
await db.OpenAsync();
return await db.QueryAsync<MyObject>(#"SELECT Something FROM Somewhere WHERE ID IN #Ids",
new { ids});
}
}
The following method just splits the list of ids in chunks - this appears to work fine ...
private IEnumerable<IEnumerable<T>> Partition<T>(IEnumerable<T> source, int size)
{
var partition = new List<T>(size);
var counter = 0;
using (var enumerator = source.GetEnumerator())
{
while (enumerator.MoveNext())
{
partition.Add(enumerator.Current);
counter++;
if (counter % size == 0)
{
yield return partition.ToList();
partition.Clear();
counter = 0;
}
}
if (counter != 0)
yield return partition;
}
}
You are creating a deadlock situation the way you use async/await in combination with Task<T>.Result.
The offending line is:
return this.GetMyObjectsAsync(ids).Result.ToList();
GetMyObjectsAsync(ids).Result blocks the current sync context. But GetMyObjectsAsync uses await which schedules a continuation point for the current sync context. I'm sure you can see the problem with that approach: the continuation can never be executed because the current sync contextis blocked by Task.Result.
One solution that can work in some cases would be to use ConfigureAwait(false) which means that the continuation can be run on any sync context.
But in general I think it's best to avoid Task.Result with async/await.
Please note that whether this deadlock situation actually occurs depends on the synchronization context that is used when calling the asynchronous method. For ASP.net, Windows Forms and WPF this will result in a deadlock, but as far as I know it won't for a console application. (Thanks to Marc Gravell for his comment)
Microsoft has a good article about Best Practices in Asynchronous Programming. (Thanks to ken2k)
I think parameters are case sensitive it should be:
return await db.QueryAsync<MyObject>(#"SELECT Something FROM Somewhere WHERE ID IN #ids",
new { ids});
instead of "#Ids" below in query:
return await db.QueryAsync<MyObject>(#"SELECT Something FROM Somewhere WHERE ID IN **#Ids**",
new { ids});
not sure though but just try.
I'm trying to implement this trivial task of listing all objects in an AmazonS3 bucket with paged requests asynchronously in C#4. I have it working in C#5 using the following snippet:
var listRequest = new ListObjectsRequest().WithBucketName(bucketName);
ListObjectsResponse listResponse = null;
var list = new List<List<S3Object>>();
while (listResponse == null || listResponse.IsTruncated)
{
listResponse = await Task<ListObjectsResponse>.Factory.FromAsync(
client.BeginListObjects, client.EndListObjects, listRequest, null);
list.Add(listResponse.S3Objects);
if (listResponse.IsTruncated)
{
listRequest.Marker = listResponse.NextMarker;
}
}
return list.SelectMany(l => l);
I'm calling the BeginListObjects/EndListObjects pair asynchronously, but I have to repeat that call every time the response says it's truncated. This piece of code works for me.
However, I now want to do this in C#4's TPL, where I don't have the luxury of using async/await and want to understand if this can be done using continuations.
How do I do this same thing in C#4?
Okay, so rather than putting the items into a list with each task/continuation it's easier in a non-await model to just have each task/continuation return the entire sequence. Given that, I used the following helper method to add each one's iterative results onto the aggregate total.
public static Task<IEnumerable<T>> Concat<T>(Task<IEnumerable<T>> first
, Task<IEnumerable<T>> second)
{
return Task.Factory.ContinueWhenAll(new[] { first, second }, _ =>
{
return first.Result.Concat(second.Result);
});
}
Next, I used the follow method to take a task of a single result and turn it into a task of a sequence (containing just that one item).
public static Task<IEnumerable<T>> ToSequence<T>(this Task<T> task)
{
var tcs = new TaskCompletionSource<IEnumerable<T>>();
task.ContinueWith(_ =>
{
if (task.IsCanceled)
tcs.SetCanceled();
else if (task.IsFaulted)
tcs.SetException(task.Exception);
else
tcs.SetResult(Enumerable.Repeat(task.Result, 1));
});
return tcs.Task;
}
Note here that you have some fields/locals not defined; I'm assuming you can add them to the appropriate method without difficulty.
private Task<IEnumerable<S3Object>> method(object sender, EventArgs e)
{
ListObjectsResponse listResponse = null;
return Task<ListObjectsResponse>.Factory.FromAsync(
client.BeginListObjects, client.EndListObjects, listRequest, null)
.ToSequence()
.ContinueWith(continuation);
}
Here is where the real magic happens. Basically,
public Task<IEnumerable<S3Object>> continuation(Task<IEnumerable<S3Object>> task)
{
if (task.Result == null) //not quite sure what null means here//may need to edit this recursive case
{
return Task<ListObjectsResponse>.Factory.FromAsync(
client.BeginListObjects, client.EndListObjects, listRequest, null)
.ToSequence()
.ContinueWith(continuation);
}
else if (task.Result.First().IsTruncated)
{
//if the results were trunctated then concat those results with
//TODO modify the request marker here; either create a new one or store the request as a field and mutate.
Task<IEnumerable<S3Object>> nextBatch = Task<ListObjectsResponse>.Factory.FromAsync(
client.BeginListObjects, client.EndListObjects, listRequest, null)
.ToSequence()
.ContinueWith(continuation);
return Concat(nextBatch, task);//recursive continuation call
}
else //if we're done it means the existing results are sufficient
{
return task;
}
}