I'm trying to combine several resources in a single collection (s variable below). GetLocations2 returns a Task and I'm expecting to be able to add that to task results collection (again, the s variable). However, it's complaining that I can't add the task results to the collection because of the following error:
Error CS1503 Argument 1: cannot convert from 'string' to
'System.Threading.Tasks.Task'
Called
public static async Task<string> WebRequest2(Uri uri)
{
using (var client = new WebClient())
{
var result = await client.OpenReadTaskAsync(uri).ConfigureAwait(false);
using (var sr = new StreamReader(result))
{
return sr.ReadToEnd();
}
}
}
Caller
private static async Task GetLocations2()
{
var s = new List<Task<string>>();
foreach (var lob in _lobs)
{
var r = await Helper.WebRequest2(new Uri(lob));
var x = Helper.DeserializeResponse<SLICS>(r);
s.Add(r); //Getting red squiggly here
}
//var w = await Task.WhenAll();
}
You do not need to await before adding task to list, it basically unwraps Task created by Helper.WebRequest2 (also it waits for Task to finish, so tasks in your original code will be executed sequentially):
private static async Task GetLocations2()
{
var s = new List<Task<string>>();
foreach (var lob in _lobs)
{
var r = Helper.WebRequest2(new Uri(lob));
// var x = Helper.DeserializeResponse<SLICS>(r); this should be done after Task.WhenAll
// or using `ContinueWith`
s.Add(r);
}
var w = await Task.WhenAll(s);
}
You can also wrap both operations in an async lambda to avoid having to iterate twice:
private static async Task GetLocations2()
{
IEnumerable<SLICS> w = await Task.WhenAll(_lobs.Select(async lob =>
{
var r = await Helper.WebRequest2(new Uri(lob));
return Helper.DeserializeResponse<SLICS>(r);
}));
}
Using Select will enumerate all the Tasks returned by the lambda expressions, then Task.WhenAll will wait for them all to complete, and unwrap the results.
Related
How to get the individual API call status success response in C#.
I am creating a mobile application using Xamarin Forms,
In my application, I need to prefetch certain information when app launches to use the mobile application.
Right now, I am calling the details like this,
public async Task<Response> GetAllVasInformationAsync()
{
var userDetails = GetUserDetailsAsync();
var getWageInfo = GetUserWageInfoAsync();
var getSalaryInfo = GetSalaryInfoAsync();
await Task.WhenAll(userDetails,
getWageInfo,
getSalaryInfo,
);
var resultToReturn = new Response
{
IsuserDetailsSucceeded = userDetails.Result,
IsgetWageInfoSucceeded = getWageInfo.Result,
IsgetSalaryInfoSucceeded = getSalaryInfo.Result,
};
return resultToReturn;
}
In my app I need to update details based on the success response. Something like this (2/5) completed. And the text should be updated whenever we get a new response.
What is the best way to implement this feature? Is it possible to use along with Task.WhenAll. Because I am trying to wrap everything in one method call.
In my app I need to update details based on the success response.
The proper way to do this is IProgress<string>. The calling code should supply a Progress<string> that updates the UI accordingly.
public async Task<Response> GetAllVasInformationAsync(IProgress<string> progress)
{
var userDetails = UpdateWhenComplete(GetUserDetailsAsync(), "user details");
var getWageInfo = UpdateWhenComplete(GetUserWageInfoAsync(), "wage information");
var getSalaryInfo = UpdateWhenComplete(GetSalaryInfoAsync(), "salary information");
await Task.WhenAll(userDetails, getWageInfo, getSalaryInfo);
return new Response
{
IsuserDetailsSucceeded = await userDetails,
IsgetWageInfoSucceeded = await getWageInfo,
IsgetSalaryInfoSucceeded = await getSalaryInfo,
};
async Task<T> UpdateWhenComplete<T>(Task<T> task, string taskName)
{
try { return await task; }
finally { progress?.Report($"Completed {taskName}"); }
}
}
If you also need a count, you can either use IProgress<(int, string)> or change how the report progress string is built to include the count.
So here's what I would do in C# 8 and .NET Standard 2.1:
First, I create the method which will produce the async enumerable:
static async IAsyncEnumerable<bool> TasksToPerform() {
Task[] tasks = new Task[3] { userDetails, getWageInfo, getSalaryInfo };
for (i = 0; i < tasks.Length; i++) {
await tasks[i];
yield return true;
}
}
So now you need to await foreach on this task enumerable. Every time you get a return, you know that a task has been finished.
int numberOfFinishedTasks = 0;
await foreach (var b in TasksToPerform()) {
numberOfFinishedTasks++;
//Update UI here to reflect the finished task number
}
No need to over-complicate this. This code will show how many of your tasks had exceptions. Your await task.whenall just triggers them and waits for them to finish. So after that you can do whatever you want with the tasks :)
var task = Task.Delay(300);
var tasks = new List<Task> { task };
var faultedTasks = 0;
tasks.ForEach(t =>
{
t.ContinueWith(t2 =>
{
//do something with a field / property holding ViewModel state
//that your view is listening to
});
});
await Task.WhenAll(tasks);
//use this to respond with a finished count
tasks.ForEach(_ => { if (_.IsFaulted) faultedTasks++; });
Console.WriteLine($"{tasks.Count() - faultedTasks} / {tasks.Count()} completed.");
.WhenAll() will allow you to determine if /any/ of the tasks failed, they you just count the tasks that have failed.
public async Task<Response> GetAllVasInformationAsync()
{
var userDetails = GetUserDetailsAsync();
var getWageInfo = GetUserWageInfoAsync();
var getSalaryInfo = GetSalaryInfoAsync();
await Task.WhenAll(userDetails, getWaitInfo, getSalaryInfo)
.ContinueWith((task) =>
{
if(task.IsFaulted)
{
int failedCount = 0;
if(userDetails.IsFaulted) failedCount++;
if(getWaitInfo.IsFaulted) failedCount++;
if(getSalaryInfo.IsFaulted) failedCount++;
return $"{failedCount} tasks failed";
}
});
var resultToReturn = new Response
{
IsuserDetailsSucceeded = userDetails.Result,
IsgetWageInfoSucceeded = getWageInfo.Result,
IsgetSalaryInfoSucceeded = getSalaryInfo.Result,
};
return resultToReturn;
}
I have two loops that use SemaphoreSlim and a array of strings "Contents"
a foreachloop:
var allTasks = new List<Task>();
var throttle = new SemaphoreSlim(10,10);
foreach (string s in Contents)
{
await throttle.WaitAsync();
allTasks.Add(
Task.Run(async () =>
{
try
{
rootResponse.Add(await POSTAsync(s, siteurl, src, target));
}
finally
{
throttle.Release();
}
}));
}
await Task.WhenAll(allTasks);
a for loop:
var allTasks = new List<Task>();
var throttle = new SemaphoreSlim(10,10);
for(int s=0;s<Contents.Count;s++)
{
await throttle.WaitAsync();
allTasks.Add(
Task.Run(async () =>
{
try
{
rootResponse[s] = await POSTAsync(Contents[s], siteurl, src, target);
}
finally
{
throttle.Release();
}
}));
}
await Task.WhenAll(allTasks);
the first foreach loop runs well, but the for loops Task.WhenAll(allTasks) returns a OutOfRangeException and I want the Contents[] index and List index to match.
Can I fix the for loop? or is there a better approach?
This would fix your current problems
for (int s = 0; s < Contents.Count; s++)
{
var content = Contents[s];
allTasks.Add(
Task.Run(async () =>
{
await throttle.WaitAsync();
try
{
rootResponse[s] = await POSTAsync(content, siteurl, src, target);
}
finally
{
throttle.Release();
}
}));
}
await Task.WhenAll(allTasks);
However this is a fairly messy and nasty piece of code. This looks a bit neater
public static async Task DoStuffAsync(Content[] contents, string siteurl, string src, string target)
{
var throttle = new SemaphoreSlim(10, 10);
// local method
async Task<(Content, SomeResponse)> PostAsyncWrapper(Content content)
{
await throttle.WaitAsync();
try
{
// return a content and result pair
return (content, await PostAsync(content, siteurl, src, target));
}
finally
{
throttle.Release();
}
}
var results = await Task.WhenAll(contents.Select(PostAsyncWrapper));
// do stuff with your results pairs here
}
There are many other ways you could do this, PLinq, Parallel.For,Parallel.ForEach, Or just tidying up your captures in your loops like above.
However since you have an IO bound work load, and you have async methods that run it. The most appropriate solution is the async await pattern which neither Parallel.For,Parallel.ForEach cater for optimally.
Another way is TPL DataFlow library which can be found in the System.Threading.Tasks.Dataflow nuget package.
Code
public static async Task DoStuffAsync(Content[] contents, string siteurl, string src, string target)
{
async Task<(Content, SomeResponse)> PostAsyncWrapper(Content content)
{
return (content, await PostAsync(content, siteurl, src, target));
}
var bufferblock = new BufferBlock<(Content, SomeResponse)>();
var actionBlock = new TransformBlock<Content, (Content, SomeResponse)>(
content => PostAsyncWrapper(content),
new ExecutionDataflowBlockOptions
{
EnsureOrdered = false,
MaxDegreeOfParallelism = 100,
SingleProducerConstrained = true
});
actionBlock.LinkTo(bufferblock);
foreach (var content in contents)
actionBlock.Post(content);
actionBlock.Complete();
await actionBlock.Completion;
if (bufferblock.TryReceiveAll(out var result))
{
// do stuff with your results pairs here
}
}
Basically this creates a BufferBlock And TransformBlock, You pump your work load into the TransformBlock, it has degrees of parallel in its options, and it pushes them into the BufferBlock, you await completion and get your results.
Why Dataflow? because it deals with the async await, it has MaxDegreeOfParallelism, it designed for IO bound or CPU Bound workloads, and its extremely simple to use. Additionally, as most data is generally processed in many ways (in a pipeline), you can then use it to pipe and manipulate streams of data in sequence and parallel or in any way way you choose down the line.
Anyway good luck
I'm downloading 100K+ files and want to do it in patches, such as 100 files at a time.
static void Main(string[] args) {
Task.WaitAll(
new Task[]{
RunAsync()
});
}
// each group has 100 attachments.
static async Task RunAsync() {
foreach (var group in groups) {
var tasks = new List<Task>();
foreach (var attachment in group.attachments) {
tasks.Add(DownloadFileAsync(attachment, downloadPath));
}
await Task.WhenAll(tasks);
}
}
static async Task DownloadFileAsync(Attachment attachment, string path) {
using (var client = new HttpClient()) {
using (var fileStream = File.Create(path + attachment.FileName)) {
var downloadedFileStream = await client.GetStreamAsync(attachment.url);
await downloadedFileStream.CopyToAsync(fileStream);
}
}
}
Expected
Hoping it to download 100 files at a time, then download next 100;
Actual
It downloads a lot more at the same time. Quickly got an error Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host
Running tasks in "batch" is not a good idea in terms of performance. A long running task would make whole batch block. A better approach would be starting a new task as soon as one is finished.
This can be implemented with a queue as #MertAkcakaya suggested. But I will post another alternative based on my other answer Have a set of Tasks with only X running at a time
int maxTread = 3;
System.Net.ServicePointManager.DefaultConnectionLimit = 50; //Set this once to a max value in your app
var urls = new Tuple<string, string>[] {
Tuple.Create("http://cnn.com","temp/cnn1.htm"),
Tuple.Create("http://cnn.com","temp/cnn2.htm"),
Tuple.Create("http://bbc.com","temp/bbc1.htm"),
Tuple.Create("http://bbc.com","temp/bbc2.htm"),
Tuple.Create("http://stackoverflow.com","temp/stackoverflow.htm"),
Tuple.Create("http://google.com","temp/google1.htm"),
Tuple.Create("http://google.com","temp/google2.htm"),
};
DownloadParallel(urls, maxTread);
async Task DownloadParallel(IEnumerable<Tuple<string,string>> urls, int maxThreads)
{
SemaphoreSlim maxThread = new SemaphoreSlim(maxThreads);
var client = new HttpClient();
foreach(var url in urls)
{
await maxThread.WaitAsync();
DownloadFile(client, url.Item1, url.Item2)
.ContinueWith((task) => maxThread.Release() );
}
}
async Task DownloadFile(HttpClient client, string url, string fileName)
{
var stream = await client.GetStreamAsync(url);
using (var fileStream = File.Create(fileName))
{
await stream.CopyToAsync(fileStream);
}
}
PS: DownloadParallel will return as soon as it starts the last download. So don't await it. If you really want to await it you should add for (int i = 0; i < maxThreads; i++) await maxThread.WaitAsync(); at the end of the method.
PS2: Don't forget to add exception handling to DownloadFile
I'm a bit new to async programming in C#, and struggling with a small but frustrating challenge.
I have a ASP.Net MVC WEB API that runs nicely. on the other side I have created this WEB API client method:
public async Task<DTOVArt> GetVArtFormArt(DTOArt art)
{
using (var client = GetHttpClient())
{
var response = await client.GetAsync("api/APIVArt/Get/" + art.ART_ID);
if (!response.IsSuccessStatusCode) return null;
var arts = await response.Content.ReadAsAsync<DTOVArt>();
return arts;
}
}
which seems to be work very fine...
This method is called from a WPF view model, an that's where my problem comes..
private DTOVObs TransFormAaretsGang(DTOAaretsGang aaretsGang)
{
var dtovObs = new DTOVObs();
using (var artservice = new ArtService())
{
artservice.GetVArtFormArt(new DTOArt() {ART_ID = aaretsGang.ART_ID}).ContinueWith(t =>
{
dtovObs.Familie_id = t.Result.Familie_id;
dtovObs.Gruppe_id = t.Result.Gruppe_id;
dtovObs.ART_ID = t.Result.ART_ID;
if (aaretsGang.Dato != null) dtovObs.Aarstal = aaretsGang.Dato.Value.Year;
return dtovObs;
});
}
return dtovObs;
}
The problem is that this last methods performs the return statement before it hits the ContinueWith statement block, that actually sets the values inside the class that should be returned.
Any attempt to do any kind of Wait() or using .Result instead of ContinueWith just blocks everything.
And if I do the return inside the ContinueWith block, the C# compiler says the method is missing a return statement, which is true.
That's the nature of async. Because your call is async, it will be executed later and the code below just continue executing.
Try adding an await and just remove the ContinueWith if this is the root of the call, usually it's an event handler:
private async Task<DTOVObs> TransFormAaretsGang(DTOAaretsGang aaretsGang)
{
var dtovObs = new DTOVObs();
DTOVArt Result = await artservice.GetVArtFormArt(new DTOArt() {ART_ID = aaretsGang.ART_ID});
dtovObs.Familie_id = Result.Familie_id;
dtovObs.Gruppe_id = Result.Gruppe_id;
dtovObs.ART_ID = Result.ART_ID;
if (aaretsGang.Dato != null)
dtovObs.Aarstal = aaretsGang.Dato.Value.Year;
return dtovObs;
}
If you still want to return an asynch Task so that any caller that calls this method can await the result, try:
private async Task<DTOVObs> TransFormAaretsGang(DTOAaretsGang aaretsGang)
{
using (var artservice = new ArtService())
{
return artservice.GetVArtFormArt(new DTOArt() {ART_ID = aaretsGang.ART_ID}).ContinueWith(t =>
{
dtovObs.Familie_id = t.Result.Familie_id;
dtovObs.Gruppe_id = t.Result.Gruppe_id;
dtovObs.ART_ID = t.Result.ART_ID;
if (aaretsGang.Dato != null) dtovObs.Aarstal = aaretsGang.Dato.Value.Year;
return dtovObs;
});
}
}
When you use async / await, you don't have to use ContinueWith anymore. ContinueWith means: wait until the previous is finished and use the result to do the next.
async await does this for you.
Suppose you have an async function. All async functions return either Task (for void return) or Task<TResult> if the return is of type TResult
private async Task<int> SlowAdditionAsync(int a, int b)
{
await Task.Delay(TimeSpan.FromSeconds(5); // causing the slow part
return a + b;
}
usage:
private async Task PerformSlowAddition()
{
int a = ...;
int b = ...;
int x =await SlowAditionAsync(a, b);
// the statement after the await, the task is finished, the result is in x.
// You can used the result as if you used Continuewith:
DisplayAddition(x);
}
Or if you want to do something else during the calculation:
private async Task PerformSlowAddition()
{
int a = ...;
int b = ...;
var taskSlowAddition = SlowAditionAsync(a, b);
DoSomethingElse(); // while the calculator does its thing
// now we need the result:
int x = await taskSlowAddition;
// no need to use ContinueWith, the next statement will be executed:
DisplayAddition(x);
}
Remember:
All functions that use a function that returns a Task or Task should be declared async
all async functions return Task is they return void or Task if they return TResult.
There is one exception to Task return: event handlers return void
After calling an async function, you can do other things.
When you need the result use await
you can only await a Task or a Task
The value of await Task is the TResult
There is only one async function that doesn't have to return a task and that is the eventhandler:
private async void OnButton1_Clicked(object sender, ...)
{
var taskX = DosomethingAsync(...)
DoSomethingElse();'
// now we need the result of taskX:
var x = await TaskX;
ProcessReault(x)
}
Note that although the event handler doesn't return a task it is still async
If you have some statements that needs to run in the background while your user interface keeps responsive, use Task.Factory.StartNew() or the more modern one Task.Run():
private int SlowCalculation(int a, int b)
{
// do something really difficult and slow
System.Threading.Thread.Sleep(TimeSpan.FromSeconds(5));
return a + b;
}
// make it async:
private async Task<int> SlowCalculationAsync(int a, int b)
{
return await Task.Run( () => SlowCalculation(a, b));
}
usage:
private async Task CalculateAsync()
{
int a = ...;
int b = ...;
int x = await SlowCalculationAsync(a, b);
Display(x);
}
private async void OnButton1_clicked(object sender, ...)
{
await CalculateAsync();
}
I'm asynchronously retrieving some rss articles with my Portable Class Library that uses the Microsoft.Bcl library (which doesn't have Task.WhenAll). Each article has a url to rss comments that I need to asynchronously retrieve as well.
The code below is my library. I call GetArticles() but it does not return any of the which creates a list of tasks that call GetComments() to asynchronously get the comments.
I've tried using Task.WaitAll in GetArticles to wait for the comments but it does not block the thread. Any help would be appreciated.
private const string ArticlesUri = "";
public async Task<List<ArticleBrief>> GetArticles()
{
var results = new List<ArticleBrief>();
try
{
var wfw = XNamespace.Get("http://wellformedweb.org/CommentAPI/");
var media = XNamespace.Get("http://search.yahoo.com/mrss/");
var dc = XNamespace.Get("http://purl.org/dc/elements/1.1/");
var t = await WebHttpRequestAsync(ArticlesUri);
StringReader stringReader = new StringReader(t);
using (var xmlReader = System.Xml.XmlReader.Create(stringReader))
{
var doc = System.Xml.Linq.XDocument.Load(xmlReader);
results = (from e in doc.Element("rss").Element("channel").Elements("item")
select
new ArticleBrief()
{
Title = e.Element("title").Value,
Description = e.Element("description").Value,
Published = Convert.ToDateTime(e.Element("pubDate").Value),
Url = e.Element("link").Value,
CommentUri = e.Element(wfw + "commentRss").Value,
ThumbnailUri = e.Element(media + "thumbnail").FirstAttribute.Value,
Categories = GetCategoryElements(e.Elements("category")),
Creator = e.Element(dc + "creator").Value
}).ToList();
}
var tasks = new Queue<Task>();
foreach (var result in results)
{
tasks.Enqueue(
Task.Factory.StartNew(async ()=>
{
result.Comments = await GetComments(result.CommentUri);
}
));
}
Task.WaitAll(tasks.ToArray());
}
catch (Exception ex)
{
// should do some other
// logging here. for now pass off
// exception to callback on UI
throw ex;
}
return results;
}
public async Task<List<Comment>> GetComments(string uri)
{
var results = new List<Comment>();
try
{
var wfw = XNamespace.Get("http://wellformedweb.org/CommentAPI/");
var media = XNamespace.Get("http://search.yahoo.com/mrss/");
var dc = XNamespace.Get("http://purl.org/dc/elements/1.1/");
var t = await WebHttpRequestAsync(uri);
StringReader stringReader = new StringReader(t);
using (var xmlReader = System.Xml.XmlReader.Create(stringReader))
{
var doc = System.Xml.Linq.XDocument.Load(xmlReader);
results = (from e in doc.Element("rss").Element("channel").Elements("item")
select
new Comment()
{
Description = e.Element("description").Value,
Published = Convert.ToDateTime(e.Element("pubDate").Value),
Url = e.Element("link").Value,
Creator = e.Element(dc + "creator").Value
}).ToList();
}
}
catch (Exception ex)
{
// should do some other
// logging here. for now pass off
// exception to callback on UI
throw ex;
}
return results;
}
private static async Task<string> WebHttpRequestAsync(string url)
{
//TODO: look into getting
var request = WebRequest.Create(url);
request.Method = "GET";
var response = await request.GetResponseAsync();
return ReadStreamFromResponse(response);
}
private static string ReadStreamFromResponse(WebResponse response)
{
using (Stream responseStream = response.GetResponseStream())
using (StreamReader sr = new StreamReader(responseStream))
{
string strContent = sr.ReadToEnd();
return strContent;
}
}
private List<string> GetCategoryElements(IEnumerable<XElement> categories)
{
var listOfCategories = new List<string>();
foreach (var category in categories)
{
listOfCategories.Add(category.Value);
}
return listOfCategories;
}
Updated Code from Solution, just added .UnWrap() on the Enqueue method:
var tasks = new Queue<Task>();
foreach (var result in results)
{
tasks.Enqueue(
Task.Factory.StartNew(async ()=>
{
result.Comments = await GetComments(result.CommentUri);
}
).Unwrap());
}
Task.WaitAll(tasks.ToArray());
It is waiting appropriately. The problem is that you are creating a Task which creates another task (i.e. StartNew is returning a Task<Task> and you are only waiting on the outer Task which completes rather quickly (it completes before the inner Task is complete)).
The questions will be:
Do you really want that inner task?
If yes, then you can use Task.Unwrap to get a proxy task that represents the completion of both the inner and outer Task and use that to Wait on.
If no, then you could remove the use of async/await in StartNew so that there is not an inner task (I think this would be prefered, it's not clear why you need the inner task).
Do you really need to do a synchronous Wait on an asynchronous Task? Read some of Stephen Cleary's blog: http://blog.stephencleary.com/2012/02/async-unit-tests-part-1-wrong-way.html
As an aside, if you are not using C# 5, then watch out for closing over the foreach variable result See
Has foreach's use of variables been changed in C# 5?, and
http://blogs.msdn.com/b/ericlippert/archive/2009/11/12/closing-over-the-loop-variable-considered-harmful.aspx)
In Microsoft.Bcl.Async we couldn't add any static methods to Task. However, you can find most of the methods on TaskEx, for example, TaskEx.WhenAll() does exist.