I have 2 methods: the first sends HTTP GET request on one address and the second calls it multiple times (so, it sends request to many IPs). Both methods are async, so they don't block code execution while the requests are proccessing remotely. The problem is, due to my poor C# knowledge, I don't know how to send all the requests simultaneously, not one after another (which my code does). That's my code:
public static async Task<string> SendRequest(Uri uri)
{
using (var client = new HttpClient())
{
var resp = await client.GetStringAsync(uri).ConfigureAwait(false);
return resp;
}
}
public static async Task<string[]> SendToAllIps(string req)
{
string[] resp = new string[_allIps.Length];
for (int i = 0; i < _allIps.Length; i++)
{
resp[i] = await SendRequest(new Uri(_allIps[i] + req));
}
return resp;
}
How to make SendToAllIps send requests without awaiting for previous task result? Also SendToAllIps must return an array of responses when all the requests are finished. As far as I understand, this can be done with Task.WaitAll, but how to use it in this particular situation?
You can use Task.WhenAll to await a collection of tasks:
public static async Task<string[]> SendToAllIps(string req)
{
var tasks = _allIps.Select(ip => SendRequest(new Uri(ip + req)));
return await Task.WhenAll(tasks);
}
Answers provided above provide the correct way of doing it but doesn't provide rationale, let me explain what's wrong with your code:
Following line creates an issue:
resp[i] = await SendRequest(new Uri(_allIps[i] + req));
Why ?
As you are awaiting each individual request, it will stop the processing of the remaining requests, that's the behavior of the async-await and it will be almost the synchronous processing of each SendRequest when you wanted then to be concurrent.
Resolution:
SendRequest being an async method returns as Task<string>, you need add that to an IEnumerable<Task<string>> and then you have option:
Task.WaitAll or Task.WhenAll
Yours is Web API (Rest Application), which needs a Synchronization Context, so you need Task.WhenAll, which provides a Task as result to wait upon and integrate all the task results in an array, if you try using Task.WaitAll it will lead to deadlock, as it is not able to search the Synchronization Context, check the following:
WaitAll vs WhenAll
You can use the Task.WaitAll only for the Console application not the Web Application, Console App doesn't need any Synchronization Context or UI Thread
public static async Task<string[]> SendToAllIps(string req)
{
var tasks = new List<Task<string>>();
for (int i = 0; i < _allIps.Length; i++)
{
// Start task and assign the task itself to a collection.
var task = SendRequest(new Uri(_allIps[i] + req));
tasks.Add(task);
}
// await all the tasks.
string[] resp = await Task.WhenAll(tasks);
return resp;
}
The key here is to collect all the tasks in a collection and then await them all using await Task.WhenAll. Even though I think the solution from Lee is more elegant...
Related
Scenario 1 - For each website in string list (_websites), the caller method wraps GetWebContent into a task, waits for all the tasks to finish and return results.
private async Task<string[]> AsyncGetUrlStringFromWebsites()
{
List<Task<string>> tasks = new List<Task<string>>();
foreach (var website in _websites)
{
tasks.Add(Task.Run(() => GetWebsiteContent(website)));
}
var results = await Task.WhenAll(tasks);
return results;
}
private string GetWebContent(string url)
{
var client = new HttpClient();
var content = client.GetStringAsync(url);
return content.Result;
}
Scenario 2 - For each website in string list (_websites), the caller method calls GetWebContent (returns Task< string >), waits for all the tasks to finish and return the results.
private async Task<string[]> AsyncGetUrlStringFromWebsites()
{
List<Task<string>> tasks = new List<Task<string>>();
foreach (var website in _websites)
{
tasks.Add(GetWebContent(website));
}
var results = await Task.WhenAll(tasks);
return results;
}
private async Task<string> GetWebContent(string url)
{
var client = new HttpClient();
var content = await client.GetStringAsync(url);
return content;
}
Questions - Which way is the correct approach and why? How does each approach impact achieving asynchronous processing?
With Task.Run() you occupy a thread from the thread pool and tell it to wait until the web content has been received.
Why would you want to do that? Do you pay someone to stand next to your mailbox to tell you when a letter arrives?
GetStringAsync already is asynchronous. The cpu has nothing to do (with this process) while the content comes in over the network.
So the second approach is correct, no need to use extra threads from the thread pool here.
Always interesting to read: Stephen Cleary's "There is no thread"
#René Vogt gave a great explanation.
There a minor 5 cents from my side.
In the second example there is not need to use async / await in GetWebContent method. You can simply return Task<string> (this would also reduce async depth).
I have been reasearching once again the async tasks. No mater how i set the Tasks, my application suffers from UI freeze all the time. I have the following code for downloading the string from a webpage:
internal string DownloadString(string URL)
{
var result = LoadCompanyContracts(URL);
return result.Result;
}
internal async Task<string> LoadCompanyContracts(string URL)
{
Task<string> task2 = Task<string>.Factory.StartNew(() =>
{
for (int i = 0; i <= 10000000; i++) Console.WriteLine(i);
WebClient wc = new WebClient();
string tmp = wc.DownloadString(new Uri(URL));
return tmp;
});
return task2.Result;
}
When i execute this task and during the for loop the UI of my application is freezing. Even though i believe that this code should not freeze the UI i am not able to find a solution. I have tried many different options and really want to use tasks instead of threads or events with webclient async.
Info: I am using .net 4.5 for my project. The difference in my code is that these functions are inside a class library(don't know if it matters).
Is it possible to run this code without blocking the user interface with async await by calling the DownloadString function from my code? If not what are the alternatives(any good nuget packages)?
The async keyword doesn't make something run asynchronously, it enables you to use await to await an already asynchronous operation. You need to use
DownloadStringTaskAsync to truly download in an asynchronous manner:
internal async Task<string> LoadCompanyContracts(string URL)
{
....
using(var wc = new WebClient())
{
string tmp = await wc.DownloadStringTaskAsync(new Uri(URL));
return tmp;
}
}
await by itself returns execution in the original execution context (ie the UI thread). This may or may not be desirable, which is why library code typically uses ConfigureAwait(false); and lets the final user of the library to decide how to await:
string tmp = await wc.DownloadStringTaskAsync(new Uri(URL))
.ConfigureAwait(false);
Finally, there's no point in awaiting if you are going to call .Result from the top-level function. There is no point in using await at all if you don't want to do use the method's result in your code. LoadCompanyContracts could be just:
internal Task<string> LoadCompanyContracts(string URL)
{
....
using(var wc = new WebClient())
{
return wc.DownloadStringTaskAsync(new Uri(URL))
.ConfigureAwait(false);
}
}
Oops
Typically, you don't need to use await at all if you just return the result of an asynchronous operation. The method could just return wc.DownloadStringTaskAsync(..); BUT that would cause the method to return and dispose the WebClient before download finishes. Avoiding the using block isn't a solution either, as it will let an expensive object like WebClient alive longer than necessary.
That's why HttpClient is preferable to WebClient: a single instance supports multiple concurrent calls, which means you can have just one instance eg as a field and reuse it, eg:
HttpClient _myClient =new HttpClient();
internal Task<string> LoadCompanyContractsAsync(string URL)
{
....
return _myClient.GetStringAsync(new Uri(URL))
.ConfigureAwait(false);
}
}
You could get rid of your DownloadString since it doesn't do anything on top of LoadCompanyContracts. If it does use the result of LoadCompanyContracts, it should be rewritten as:
internal async Task<string> DownloadString(string URL)
{
var result = await LoadCompanyContracts(URL);
//Do something with the result
return result;
}
EDIT
The original answer used DownloadStringAsync which is a legacy method that raises an event when download completes. The correct method is DownloadStringTaskAsync
EDIT 2
Since we are talking about a UI, the code can be made asynchronous all the way to the top event handler by using the async void syntax for the handler, eg async void Button1_Click, eg:
async void LoadCustomers_Click(...)
{
var contracts=await LoaCompanyContracts(_companyUrls);
txtContracts>Text=contracts;
}
In this case we want to return to the original thread, so we don't use ConfigureAwait(false);
I have to call 100,000 urls and I don't need to wait for the response. I have searched a lot . some people say there is no way to call http request without waiting for the response, also there are few answers for questions like mine that say to set Method="POST".
The following is the source code based on all of them. I have tried to call the urls asynchronously using WhenAll.
The problem is this that when I see CPU usage in task manager, it is fully busy for 140 seconds and within that time the system is almost unusable.
protected async void btnStartCallWhenAll_Click(object sender, EventArgs e)
{
// Make a list of web addresses.
List<string> urlList = SetUpURLList(Convert.ToInt32(txtNoRecordsToAdd.Text));
// One-step async call.
await ProcessAllURLSAsync(urlList);
}
private async Task ProcessAllURLSAsync(List<string> urlList)
{
// Create a query.
IEnumerable<Task<int>> CallingTasksQuery =
from url in urlList select ProcessURLAsync(url);
// Use ToArray to execute the query and start the Calling tasks.
Task<int>[] CallingTasks = CallingTasksQuery.ToArray();
// Await the completion of all the running tasks.
int[] lengths = await Task.WhenAll(CallingTasks);
int total = lengths.Sum();
}
private async Task<int> ProcessURLAsync(string url)
{
await CallURLAsync(url);
return 1;
}
private async Task CallURLAsync(string url)
{
// Initialize an HttpWebRequest for the current URL.
var webReq = (HttpWebRequest)WebRequest.Create(url);
webReq.Method="POST";
// Send the request to the Internet resource and wait for the response.
Task<WebResponse> responseTask = webReq.GetResponseAsync() ;
}
private List<string> SetUpURLList(int No)
{
List<string> urls = new List<string>
{
};
for (int i = 1; i <= No; i++)
urls.Add("http://msdn.microsoft.com/library/windows/apps/br211380.aspx");
return urls;
}
By the way the compiler hints that "this async method lacks 'await' operators and will run synchronously consider using the await operator to await non-blocking api calls or await task.run(..) to do cpu bound work on background thread" for this line:
private async Task CallURLAsync(string url).
I don't know if this affect my problem or not but after searching same problem in other questions, they said that I need to disable compiler message before this line.
Is the following the correct way to make a async method, where the code in the method have to do multiple async calls that needs to be waited on.
The plan is to start multiple of this method, and when wait for all of them to finish before the code continues.
public static Task<string> Get(string url)
{
return Task.Run(async () =>
{
var client = getBaseHttpClient();
var result = await client.GetAsync(url).ConfigureAwait(false);
if (result.IsSuccessStatusCode)
{
return await result.Content.ReadAsStringAsync();
}
return null;
});
}
Your code:
starts a threadpool thread (Task.Run),
which will start an async I/O operation (GetAsync), and then go back to the threadpool.
When the I/O is done (await), another threadpool thread will be started (ConfigureAwait(false)),
which will start another async I/O operation to read the content of the HTTP response (GetAsStringAsync), and go back to the threadpool.
When the I/O is done (await), another threadpool thread will be started to return the content to the calling method.
You could skip step 1. altogether. All it does is defer the call to getBaseHttpClient to a threadpool thread, which I'll assume is not intensive CPU-bound work - in which case, it could/should be done synchronously.
public static async Task<string> Get(string url)
{
var client = getBaseHttpClient();
var result = await client.GetAsync(url).ConfigureAwait(false);
if (result.IsSuccessStatusCode)
{
return await result.Content.ReadAsStringAsync();
}
return null;
}
Calling code would be:
var tasks = urls.Select(Get);
var responses = await Task.WhenAll(tasks);
I'm needing to create a list of tasks to execute a routine that takes one parameter and then wait for those tasks to complete before continuing with the rest of the program code. Here is an example:
List<Task> tasks = new List<Task>();
foreach (string URL in LIST_URL_COLLECTION)
{
tasks[i] = Task.Factory.StartNew(
GoToURL(URL)
);
}
//wait for them to finish
Console.WriteLine("Done");
I've have googled and searched this site but I just keep hitting a dead end, I did this once but can't remember how.
The Task Parallel Library exposes a convinent way to asynchronously wait for the completion of all tasks via the Task.WhenAll method. The method returns a Task by itself which is awaitable and should be awaited:
public async Task QueryUrlsAsync()
{
var urlFetchingTasks = ListUrlCollection.Select(url => Task.Run(url));
await Task.WhenAll(urlFetchingTasks);
Console.WriteLine("Done");
}
Note that in order to await, your method must be marked with the async modifier in the method signature and return either a Task (if it has no return value) or a Task<T> (if it does have a return value, which type is T).
As a side note, your method looks like it's fetching urls, which i am assuming is generating a web request to some endpoint. In order to do that, there's no need to use extra threads via Task.Factory.StartNew or Task.Run, as these operations are naturally asynchronous. You should look into HttpClient as a starting point. For example, your method could look like this:
public async Task QueryUrlsAsync()
{
var urlFetchingTasks = ListUrlCollection.Select(url =>
{
var httpClient = new HttpClient();
return httpClient.GetAsync(url);
});
await Task.WhenAll(urlFetchingTasks);
Console.WriteLine("Done");
}