Develop Reference

c# a task was canceled at the await call

I'm trying to hit my simple server's endpoint using the following code, but I keep getting "A task was canceled." during the await call. The server's logs don't show any errors and cts.IsCancellationRequested == false, however e.CancellationToken.IsCancellationRequested == true. Any advice on how to track down the cause of this cancellation? At the very least, how can I tell if it's coming from the front end or the server endpoint?
private async Task<string> SendSingleRequestToDlis(
HttpClient client,
StringContent requestData)
{
int timeout = 600000; // in ms
string dlisEndpoint = "myendpointhere";
string response;
using (var cts = new CancellationTokenSource(timeout))
{
//send
HttpResponseMessage request;
try
{
request = await client.PostAsync(dlisEndpoint, requestData);
}
catch (Exception e)
{
throw new Exception("Could not establish conection to model hosted on DLIS.", e);
}
....

You are not using your CancellationToken so don't implement it. Just use this:
string dlisEndpoint = "myendpointhere";
string response;
HttpResponseMessage request;
try
{
request = await client.PostAsync(dlisEndpoint, requestData);
}
catch (Exception e)
{
throw new Exception("Could not establish conection to model hosted on DLIS.", e);
}
If you wanted to use the cancellation token you actually have to pass it down into the client.
request = await client.PostAsync(dlisEndpoint, requestData, cts.Source);

At the very least, how can I tell if it's coming from the front end or the server endpoint?
You can know it's at the client because you're seeing an OperationCanceledException. A server can just return a response code, which you would see as an HttpRequestException.
i timed it just now and the cancellation happens at exactly 100s
This is the default value of the HttpClient.Timeout property. Set this property higher (or to Timeout.InfiniteTimeSpan) at the point where the HttpClient is created (or configured, if the HttpClient is injected).

cURL request with HttpClient - how to set the timeout on the server connection (WinInet)

I am sending cURL request using HttpClient through the method described here under.
The parameter used for this method are:
SelectedProxy = a custom class that stores my proxy's parameters
Parameters.WcTimeout = the timeout
url, header, content = the cURL request (based on this tool to convert to C# https://curl.olsh.me/).
const SslProtocols _Tls12 = (SslProtocols)0x00000C00;
const SecurityProtocolType Tls12 = (SecurityProtocolType)_Tls12;
ServicePointManager.SecurityProtocol = Tls12;
string source = "";
using (var handler = new HttpClientHandler())
{
handler.UseCookies = usecookies;
WebProxy wp = new WebProxy(SelectedProxy.Address);
handler.Proxy = wp;
using (var httpClient = new HttpClient(handler))
{
httpClient.Timeout = Parameters.WcTimeout;
using (var request = new HttpRequestMessage(new HttpMethod(HttpMethod), url))
{
if (headers != null)
{
foreach (var h in headers)
{
request.Headers.TryAddWithoutValidation(h.Item1, h.Item2);
}
}
if (content != "")
{
request.Content = new StringContent(content, Encoding.UTF8, "application/x-www-form-urlencoded");
}
HttpResponseMessage response = new HttpResponseMessage();
try
{
response = await httpClient.SendAsync(request);
}
catch (Exception e)
{
//Here the exception happens
}
source = await response.Content.ReadAsStringAsync();
}
}
}
return source;
If I am running this without proxy, it works like a charm.
When I send a request using a proxy which I tested first from Chrome, I have the following error on my try {} catch {}. Here is the error tree
{"An error occurred while sending the request."}
InnerException {"Unable to connect to the remote server"}
InnerException {"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond [ProxyAdress]"}
SocketErrorCode: TimedOut
By using a Stopwatch I see that the TimedOut occurred after around 30 sec.
I tried a few different handler based on the following links What's the difference between HttpClient.Timeout and using the WebRequestHandler timeout properties?, HttpClient Timeout confusion or with the WinHttpHandler.
It's worth noting that WinHttpHandler allow for a different error code, i.e. Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'. The underlying reason is the same though it helped to target where it bugs (i.e. WinInet) which confirms also what #DavidWright was saying regarding that timeouts from HttpClient manages a different part of the request sending.
Hence my issue is coming from the time it takes to establish a connection to the server, which triggers the 30sec timeout from WinInet.
My question is then How to change those timeout?
On a side note, it's worth noting that Chrome, which uses WinInet, does not seem to suffer from this timeout, nor Cefsharp on which a big part of my app is based, and through which the same proxies can properly send requests.

So thanks to #DavidWright I understand a few things:
Before that the HttpRequestMessage is sent and the timeout from HttpClient starts, a TCP connection to the server is initiated
The TCP connection has its own timeout, defined at OS level, and we do not identified a way to change it at run time from C# (question pending if anyone want to contribute)
Insisting on trying to connect works as each try benefits from previous tries, though proper exception management & manual timeout counter needs to be implemented (I actually considered a number of tries in my code, assuming each try is around 30sec)
All this together ended up in the following code:
const SslProtocols _Tls12 = (SslProtocols)0x00000C00;
const SecurityProtocolType Tls12 = (SecurityProtocolType)_Tls12;
ServicePointManager.SecurityProtocol = Tls12;
var sp = ServicePointManager.FindServicePoint(endpoint);
sp.ConnectionLeaseTimeout = (int)Parameters.ConnectionLeaseTimeout.TotalMilliseconds;
string source = "";
using (var handler = new HttpClientHandler())
{
handler.UseCookies = usecookies;
WebProxy wp = new WebProxy(SelectedProxy.Address);
handler.Proxy = wp;
using (var client = new HttpClient(handler))
{
client.Timeout = Parameters.WcTimeout;
int n = 0;
back:
using (var request = new HttpRequestMessage(new HttpMethod(HttpMethod), endpoint))
{
if (headers != null)
{
foreach (var h in headers)
{
request.Headers.TryAddWithoutValidation(h.Item1, h.Item2);
}
}
if (content != "")
{
request.Content = new StringContent(content, Encoding.UTF8, "application/x-www-form-urlencoded");
}
HttpResponseMessage response = new HttpResponseMessage();
try
{
response = await client.SendAsync(request);
}
catch (Exception e)
{
if(e.InnerException != null)
{
if(e.InnerException.InnerException != null)
{
if (e.InnerException.InnerException.Message.Contains("A connection attempt failed because the connected party did not properly respond after"))
{
if (n <= Parameters.TCPMaxTries)
{
n++;
goto back;
}
}
}
}
// Manage here other exceptions
}
source = await response.Content.ReadAsStringAsync();
}
}
}
return source;
On a side note, my current implementation of HttpClient may be problematic in the future. Though being disposable, HttpClient should be defined at App level through a static, and not within a using statement. To read more about this go here or there.
My issue is that I want to renew the proxy at each request and that it is not set on a per request basis. While it explains the reasdon of the new ConnectionLeaseTimeout parameter (to minimize the time the lease remains open) it is a different topic

I have had the same problem with HttpClient. Two things need to happen for SendAsync to return: first, setting up the TCP channel over which the communication occurs (the SYN, SYN/ACK, ACK handshake, if you're familiar with that) and second getting back the data that constitutes the HTTP response over that TCP channel. HttpClient's timeout only applies to the second part. The timeout for the first part is governed by the OS's network subsystem, and it's quite difficult to change that timeout in .NET code.
(Here's how you can reproduce this effect. Set up a working client/server connection between two machines, so you know that name resolution, port access, listening, and client and server logic all works. Then unplug the network cable on the server and re-run the client request. It will time out with the OS's default network timeout, regardless of what timeout you set on your HttpClient.)
The only way I know around this is to start your own delay timer on a different thread and cancel the SendAsync task if the timer finishes first. You can do this using Task.Delay and Task.WaitAny or by creating a CancellationTokenSource with your desired timeone (which essentially just does the first way under the hood). In either case you will need to be careful about cancelling and reading exceptions from the task that loses the race.

HttpClient query occasionally hangs

I initialise HttpClient like so:
public static CookieContainer cookieContainer = new CookieContainer();
public static HttpClient httpClient = new HttpClient(new HttpClientHandler() { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate, CookieContainer = cookieContainer }) { Timeout = TimeSpan.FromSeconds(120) };
so all queries should throw TaskCanceledException if no response is received within 120 seconds.
But some queries (like 1 of 100 000-1 000 000) hang infinitely.
I wrote following code:
public static async Task<HttpResponse> DownloadAsync2(HttpRequestMessage httpRequestMessage)
{
HttpResponse response = new HttpResponse { Success = false, StatusCode = (int)HttpStatusCode.RequestTimeout, Response = "Timeout????????" };
Task task;
if (await Task.WhenAny(
task = Task.Run(async () =>
{
try
{
HttpResponseMessage r = await Global.httpClient.SendAsync(httpRequestMessage).ConfigureAwait(false);
response = new HttpResponse { Success = true, StatusCode = (int)r.StatusCode, Response = await r.Content.ReadAsStringAsync().ConfigureAwait(false) };
}
catch (TaskCanceledException)
{
response = new HttpResponse { Success = false, StatusCode = (int)HttpStatusCode.RequestTimeout, Response = "Timeout" };
}
catch (Exception ex)
{
response = new HttpResponse { Success = false, StatusCode = -1, Response = ex.Message + ": " + ex.InnerException };
}
}),
Task.Run(async () =>
{
await Task.Delay(TimeSpan.FromSeconds(150)).ConfigureAwait(false);
})
).ConfigureAwait(false) != task)
{
Log("150 seconds passed");
}
return response;
}
which actually occasionally executes Log("150 seconds passed");.
I call it like so:
HttpResponse r = await DownloadAsync2(new HttpRequestMessage
{
RequestUri = new Uri("https://address.com"),
Method = HttpMethod.Get
}).ConfigureAwait(false);
Why TaskCanceledException sometimes isn't thrown after 120 seconds?

I don't know with how frequency you call DownloadAsync2, but your code smells a lot for bursting and starving ThreadPool.
Initial number of threads in ThreadPool by default is limited to number of CPU logical cores (usually 12 for today normal systems) and in case of unavailability of threads in ThreadPool, 500ms takes for each new thread to be generated.
So for example:
for (int i = 0; i < 1000; i++)
{
HttpResponse r = await DownloadAsync2(new HttpRequestMessage
{
RequestUri = new Uri("https://address.com"),
Method = HttpMethod.Get
}).ConfigureAwait(false);
}
This code with a high chance will be freezed, specially if you have some lock or any cpu intensive tasks somewhere in your code. Because you invoke new thread per calling DownloadAsync2 so all threads of ThreadPool consumed and many more of them still needed.
I know maybe you say "all of my tasks have been awaited and they release for other works". but they also consumed for starting new DownloadAsync2 threads and you will reach the point that after finishing await Global.httpClient.SendAsync no thread remains for re-assigning and completing the task.
So method have to wait until one thread being available or generated to complete (even after timeout). Rare but feasible.

On Windows & .NET, the number of concurrent outgoing HTTP request to the same endpoint is limited to 2 (as per HTTP 1.1 specification). If you create a ton of concurrent requests to the same endpoint they will queue up. That is one possible explanation to what you experience.
Another possible explanation is this: you don't set the Timeout property of HttpClient explicitly, so it defaults to 100 seconds. If you keep making new requests, while the previous ones didn't finish, system resources will become used up.
I suggest setting the Timeout property to a low value - something proportional to the frequency of the calls you make (1 sec?) and optionally increasing the number of conncurrent outgoing connections with ServicePointManager.DefaultConnectionLimit

I discovered it was httpClient.SendAsync method which occasionally hangs. Therefore I added a cancellation token set to X seconds. But even with a cancelletion token it may sometimes remain stuck and never throw TaskCanceledException.
Therefore I proceeded to workaround that keeps the SendAsync task forever stuck on background and continue with other work.
Here is my workaround:
public static async Task<Response> DownloadAsync3(HttpRequestMessage httpRequestMessage, string caller)
{
Response response;
try
{
using CancellationTokenSource timeoutCTS = new CancellationTokenSource(httpTimeoutSec * 1000);
using HttpResponseMessage r = await Global.httpClient.SendAsync(httpRequestMessage, HttpCompletionOption.ResponseContentRead, timeoutCTS.Token).WithCancellation(timeoutCTS.Token).ConfigureAwait(false);
response = new Response { Success = true, StatusCode = (int)r.StatusCode, Message = await r.Content.ReadAsStringAsync().ConfigureAwait(false) };
}
catch (TaskCanceledException)
{
response = new Response { Success = false, StatusCode = (int)HttpStatusCode.RequestTimeout, Message = "Timeout" };
}
catch (Exception ex)
{
response = new Response { Success = false, StatusCode = -1, Message = ex.Message + ": " + ex.InnerException };
}
httpRequestMessage.Dispose();
return response;
}
public static Task<T> WithCancellation<T>(this Task<T> task, CancellationToken cancellationToken)
{
return task.IsCompleted
? task
: task.ContinueWith(
completedTask => completedTask.GetAwaiter().GetResult(),
cancellationToken,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Default);
}

With Flurl, you can configure Timeout per client, per request or globally.
// call once at application startup
FlurlHttp.Configure(settings => settings.Timeout = TimeSpan.FromSeconds(120));
string url = "https://address.com";
// high level scenario
var response = await url.GetAsync();
// low level scenario
await url.SendAsync(
HttpMethod.Get, // Example
httpContent, // optional
cancellationToken, // optional
HttpCompletionOption.ResponseHeaderRead); // optional
// Timeout at request level
await url
.WithTimeout(TimeSpan.FromSeconds(120))
.GetAsync();
Fluent HTTP documentation
Flurl configuration documentation

And the answer is:
ThreadPool.SetMinThreads(MAX_THREAD_COUNT, MAX_THREAD_COUNT);
where MAX_THREAD_COUNT is some number (I use 200). You MUST set at least the second parameter (completionPortThreads), and most probably the first (workerThreads). I had already set the first, but not the second, and now that it is working I am keeping both set.
Alas, this isn't the answer. See comments below

OK, I officially give up! I'm replacing the code with:
try
{
return Task.Run(() => httpClient.SendAsync(requestMessage)).Result;
}
catch (AggregateException e)
{
if (e.InnerException != null)
throw e.InnerException;
throw;
}

HttpClient - Send a batch of requests

I want to iterate a batch of requests, sending each one of them to an external API using HttpClient class.
foreach (var MyRequest in RequestsBatch)
{
try
{
HttpClient httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMilliseconds(5);
HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endpoint), myRequest);
JObject resultResponse = await response.Content.ReadAsAsync<JObject>();
}
catch (Exception ex)
{
continue;
}
}
The context here is I need to set a very small timeout value, so in case the response takes more than that time, we simply get the "Task was cancelled" exception and continue iterating.
Now, in the code above, comment these two lines:
HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endpoint), myRequest);
resultResponse = await response.Content.ReadAsAsync<JObject>();
The iteration ends very fast. Uncomment them and try again. It takes a lot of time.
I wonder if calling PostAsJsonAsync/ReadAsAsync methods with await takes more time than the timeout value?
Based on the answer below, supposing it will create different threads, we have this method:
public Task<JObject> GetResponse(string endPoint, JObject request, TimeSpan timeout)
{
return Task.Run(async () =>
{
try
{
HttpClient httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMilliseconds(5);
HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endPoint), request).WithTimeout<HttpResponseMessage>(timeout);
JObject resultResponse = await response.Content.ReadAsAsync<JObject>().WithTimeout<JObject>(timeout);
return resultResponse;
}
catch (Exception ex)
{
return new JObject() { new JProperty("ControlledException", "Invalid response. ")};
}
});
}
An exception is raised there and the JObject exception should be returned, very fast, however, if using httpClient methods, even if it raises the exception it takes a lot of time. Is there a behind the scenes processing affecting the Task even if the return value was a simple exception JObject?
If yes, which another approach could be used to send a batch of requests to an API in a very fast way?

I agree with the accepted answer in that the key to speeding things up is to run the requests in parallel. But any solution that forces additional threads into the mix by use of Task.Run or Parallel.ForEach is not gaining you any efficiency with I/O bound asynchronous operations. If anything it's hurting.
You can easily get all calls running concurrently while letting the underlying async subsystems decide how many threads are required to complete the tasks as efficiently as possible. Chances are that number is much smaller than the number of concurrent calls, because they don't require any thread at all while they're awaiting a response.
Further, the accepted answer creates a new instance of HttpClient for each call. Don't do that either - bad things can happen.
Here's a modified version of the accepted answer:
var httpClient = new HttpClient {
Timeout = TimeSpan.FromMilliseconds(5)
};
var taskList = new List<Task<JObject>>();
foreach (var myRequest in RequestsBatch)
{
// by virtue of not awaiting each call, you've already acheived parallelism
taskList.Add(GetResponseAsync(endPoint, myRequest));
}
try
{
// asynchronously wait until all tasks are complete
await Task.WhenAll(taskList.ToArray());
}
catch (Exception ex)
{
}
async Task<JObject> GetResponseAsync(string endPoint, string myRequest)
{
// no Task.Run here!
var response = await httpClient.PostAsJsonAsync<string>(
string.Format("{0}api/GetResponse", endpoint),
myRequest);
return await response.Content.ReadAsAsync<JObject>();
}

It doesn't look like you're actually running a seperate thread for each request. Try something like this:
var taskList = new List<Task<JObject>>();
foreach (var myRequest in RequestsBatch)
{
taskList.Add(GetResponse(endPoint, myRequest));
}
try
{
Task.WaitAll(taskList.ToArray());
}
catch (Exception ex)
{
}
public Task<JObject> GetResponse(string endPoint, string myRequest)
{
return Task.Run(() =>
{
HttpClient httpClient = new HttpClient();
HttpResponseMessage response = httpClient.PostAsJsonAsync<string>(
string.Format("{0}api/GetResponse", endpoint),
myRequest,
new CancellationTokenSource(TimeSpan.FromMilliseconds(5)).Token);
JObject resultResponse = response.Content.ReadAsAsync<JObject>();
});
}

How to call web API from an http client using a small timeout

I'm working with two web API projects that get communicated. The first web API calls the second one using an HttpClient class.
What I would like to do is to set a short timeout (500 ms) when I call the second web API, and if I don't get response in that time, just to skip the next lines that process the result in the client, but continue processing the request at server side(second API).
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Clear();
client.Timeout = this.Timeout; // (500ms)
HttpResponseMessage response = client.PostAsJsonAsync(EndPoint, PostData).Result;
if (response.IsSuccessStatusCode)
{
return response.Content.ReadAsAsync<T>().Result;
}
else
{
throw new CustomException()
}
}
It works in the first API side, however in the second API(server), I get the following exceptions:
"A task was canceled."
"The operation was cancelled."
at System.Threading.CancellationToken.ThrowOperationCanceledException()
at System.Threading.CancellationToken.ThrowIfCancellationRequested()
I think it is caused by the small timeout of the call, that ends when the second API is still processing the result.
How could I avoid this behaviour in the second API and continue processing the request ?
Thanks in advance.

That is the expected behavior. When you set a timeout and the call does not respond in that amount of time, the task is canceled and that exception is thrown.
And by the way, do not use .Result. That will cause blocking. Mark your method async and use await.
The whole thing should look something like this:
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Clear();
client.Timeout = this.Timeout; // (500ms)
try
{
HttpResponseMessage response = await client.PostAsJsonAsync(EndPoint, PostData);
if (response.IsSuccessStatusCode)
{
return await response.Content.ReadAsAsync<T>();
}
else
{
throw new CustomException()
}
}
catch (TaskCanceledException)
{
// request did not complete in 500ms.
return null; // or something else to indicate no data, move on
}
}