Develop Reference

HttpClient.GetAsync fails after running for some time with TaskCanceledException "A task was cancelled" [duplicate]

It works fine when have one or two tasks however throws an error "A task was cancelled" when we have more than one task listed.
List<Task> allTasks = new List<Task>();
allTasks.Add(....);
allTasks.Add(....);
Task.WaitAll(allTasks.ToArray(), configuration.CancellationToken);
private static Task<T> HttpClientSendAsync<T>(string url, object data, HttpMethod method, string contentType, CancellationToken token)
{
HttpRequestMessage httpRequestMessage = new HttpRequestMessage(method, url);
HttpClient httpClient = new HttpClient();
httpClient.Timeout = new TimeSpan(Constants.TimeOut);
if (data != null)
{
byte[] byteArray = Encoding.ASCII.GetBytes(Helper.ToJSON(data));
MemoryStream memoryStream = new MemoryStream(byteArray);
httpRequestMessage.Content = new StringContent(new StreamReader(memoryStream).ReadToEnd(), Encoding.UTF8, contentType);
}
return httpClient.SendAsync(httpRequestMessage).ContinueWith(task =>
{
var response = task.Result;
return response.Content.ReadAsStringAsync().ContinueWith(stringTask =>
{
var json = stringTask.Result;
return Helper.FromJSON<T>(json);
});
}).Unwrap();
}

There's 2 likely reasons that a TaskCanceledException would be thrown:
Something called Cancel() on the CancellationTokenSource associated with the cancellation token before the task completed.
The request timed out, i.e. didn't complete within the timespan you specified on HttpClient.Timeout.
My guess is it was a timeout. (If it was an explicit cancellation, you probably would have figured that out.) You can be more certain by inspecting the exception:
try
{
var response = task.Result;
}
catch (TaskCanceledException ex)
{
// Check ex.CancellationToken.IsCancellationRequested here.
// If false, it's pretty safe to assume it was a timeout.
}

I ran into this issue because my Main() method wasn't waiting for the task to complete before returning, so the Task<HttpResponseMessage> was being cancelled when my console program exited.
C# ≥ 7.1
You can make the main method asynchronous and await the task.
public static async Task Main(){
Task<HttpResponseMessage> myTask = sendRequest(); // however you create the Task
HttpResponseMessage response = await myTask;
// process the response
}
C# < 7.1
The solution was to call myTask.GetAwaiter().GetResult() in Main() (from this answer).

var clientHttp = new HttpClient();
clientHttp.Timeout = TimeSpan.FromMinutes(30);
The above is the best approach for waiting on a large request.
You are confused about 30 minutes; it's random time and you can give any time that you want.
In other words, request will not wait for 30 minutes if they get results before 30 minutes.
30 min means request processing time is 30 min.
When we occurred error "Task was cancelled", or large data request requirements.

Another possibility is that the result is not awaited on the client side. This can happen if any one method on the call stack does not use the await keyword to wait for the call to be completed.

Promoting #JobaDiniz's comment to an answer:
Do not do the obvious thing and dispose the HttpClient instance, even though the code "looks right":
async Task<HttpResponseMessage> Method() {
using (var client = new HttpClient())
return client.GetAsync(request);
}
Disposing the HttpClient instance can cause following HTTP requests started by other instances of HttpClient to be cancelled!
The same happens with C#'s new RIAA syntax; slightly less obvious:
async Task<HttpResponseMessage> Method() {
using var client = new HttpClient();
return client.GetAsync(request);
}
Instead, the correct approach is to cache a static instance of HttpClient for your app or library, and reuse it:
static HttpClient client = new HttpClient();
async Task<HttpResponseMessage> Method() {
return client.GetAsync(request);
}
(The Async() request methods are all thread safe.)

in my .net core 3.1 applications I am getting two problem where inner cause was timeout exception.
1, one is i am getting aggregate exception and in it's inner exception was timeout exception
2, other case was Task canceled exception
My solution is
catch (Exception ex)
{
if (ex.InnerException is TimeoutException)
{
ex = ex.InnerException;
}
else if (ex is TaskCanceledException)
{
if ((ex as TaskCanceledException).CancellationToken == null || (ex as TaskCanceledException).CancellationToken.IsCancellationRequested == false)
{
ex = new TimeoutException("Timeout occurred");
}
}
Logger.Fatal(string.Format("Exception at calling {0} :{1}", url, ex.Message), ex);
}

In my situation, the controller method was not made as async and the method called inside the controller method was async.
So I guess its important to use async/await all the way to top level to avoid issues like these.

I was using a simple call instead of async. As soon I added await and made method async it started working fine.
public async Task<T> ExecuteScalarAsync<T>(string query, object parameter = null, CommandType commandType = CommandType.Text) where T : IConvertible
{
using (IDbConnection db = new SqlConnection(_con))
{
return await db.ExecuteScalarAsync<T>(query, parameter, null, null, commandType);
}
}

Another reason can be that if you are running the service (API) and put a breakpoint in the service (and your code is stuck at some breakpoint (e.g Visual Studio solution is showing Debugging instead of Running)). and then hitting the API from the client code. So if the service code a paused on some breakpoint, you just hit F5 in VS.

Cancellable Async Web Request

I am trying to make a generic method that will cancel an Async web request and any additional operations associated with it. I found another question regarding this such as this.
And I have written a helper class that will do just this. I present it below:
public static class Helpers
{
public static async Task<string> GetJson(string url,
CancellationTokenSource cancellationTokenSource,
bool useSynchronizationContext = true)
{
try
{
var request = (HttpWebRequest)WebRequest.Create(url);
string jsonStringResult;
using (WebResponse response = await request.GetResponseAsync()
.WithCancellation(cancellationTokenSource.Token,
request.Abort, useSynchronizationContext))
{
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
jsonStringResult = await reader.ReadToEndAsync();
reader.Close();
dataStream.Close();
}
cancellationTokenSource.Token.ThrowIfCancellationRequested();
return jsonStringResult;
}
catch (Exception ex) when (ex is OperationCanceledException
|| ex is TaskCanceledException)
{
}
catch (Exception ex) when (ex is WebException
&& ((WebException)ex).Status == WebExceptionStatus.RequestCanceled)
{
}
catch (Exception ex)
{
//Any other exception
}
finally
{
cancellationTokenSource.Dispose();
}
return default;
}
public static async Task<T> WithCancellation<T>(this Task<T> task,
CancellationToken cancellationToken, Action action,
bool useSynchronizationContext)
{
using (cancellationToken.Register(action, useSynchronizationContext))
{
return await task;
}
}
}
Notice the line
cancellationTokenSource.Token.ThrowIfCancellationRequested();
right before returning the JSON string.
When the operation is canceled and the flow of execution is on line
StreamReader reader = new StreamReader(dataStream);
for example, all lines below (reader.Close() etc.) will be executed and the exception will be thrown when ThrowIfCancelationRequested() is executed - is that correct? Am I missing something?
If so, is there a way to cancel everything at once?
Thanks everyone for their response,
After the answer provided and all really useful comments I updated the implementation.I used HttpClient and the extension method in the link for having a task that cannot actually being canceled to behave like one that can - readAsStringAsync is actually executed since it cannot accept a cancellation token.
public static class Helpers
{
public static async Task<string> GetJson(string url,CancellationToken cancellationToken)
{
try
{
string jsonStringResult;
using (var client = new HttpClient())
{
cancellationToken.ThrowIfCancellationRequested();
using (var response = await client.GetAsync(url, cancellationToken))
{
jsonStringResult = await response.Content.ReadAsStringAsync().WithCancellation(cancellationToken);
}
}
return jsonStringResult;
}
catch (Exception ex) when (ex is OperationCanceledException)
{
}
catch (Exception ex) when (ex is WebException exception && exception.Status == WebExceptionStatus.RequestCanceled)
{
}
catch (Exception ex)
{
//LogException(ex);
}
return default;
}
public static Task<T> WithCancellation<T>(this Task<T> task, CancellationToken cancellationToken)
{
return task.IsCompleted
? task: task.ContinueWith(completedTask => completedTask.GetAwaiter().GetResult(),cancellationToken,TaskContinuationOptions.ExecuteSynchronously,TaskScheduler.Default);
}
}

all lines below (reader.Close() etc.) will be executed and the
exception will be thrown when ThrowIfCancelationRequested() is
executed - is that correct? Am I missing something?
If so, is there a way to cancel everything at once?
First of all, operation that you want to cancel has to explicitly support being canceled. So, you have to push your best to use a code for executing some operation in a way which somehow takes CancellationToken as an argument. If it is not possible, then you have no other chance.
That's why this concept is called Cooperative cancellation. Because almost always both sides should be aware of that cancellation happened. Client-side should be aware of that the code was actually canceled, it’s not enough for a client to know that cancellation was just requested. For the callee, it’s important to know about the fact that cancellation was requested in order to properly finish itself.
Regarding to checking whether operation is cancelled while executing Close method of stream and reader. You have to call cleanup methods always whether operation is cancelled or not if you want to avoid memory leaks. Of course I suggest you to use using statement which will automatically do that cleanup.
And by the way, for making some function cancelable you don't have to check whether cancellation is requested before executing each line. You just have to check whether cancellation is request before and after executing some long-running operation. And pass cancellation token if that long-running operations supports cancelling through cancellation token property.
Also, you have to take a look for side-effects. Don’t cancel if you’ve already incurred side-effects that your method isn’t prepared to revert on the way out that would leave you in an inconsistent state.
Some general code-block might be so:
if(ct.IsCancellationRequested)
{
break; // or throw
}
await DoSomething(ct);
if (ct.IsCancellationRequested)
{
// if there is no side-effect
return; // or throw
// or, we already did something in `DoSomething` method
// do some rollback
}
As a solution, you can use some different objects like HttpClient or WebRequest for executing async, awaitable and cancelable web request. You can take a look to that link for implementation details.

DbContext getting disposed too early within Async method

When converting an existing synchronous method to async, I accidentally used "async void" on one of the methods, which resulted in some unexpected behavior.
Below is a simplified example of the kind of change I had actually performed,
public IActionResult Index()
{
var vm = new ViewModel();
try
{
var max = 0;
if (_dbContext.OnlyTable.Any())
{
max = _dbContext.OnlyTable.Max(x => x.SomeColumn);
}
_dbContext.Add(new TestTable() { SomeColumn = max + 1 });
_dbContext.SaveChanges();
MakePostCallAsync("http:\\google.com", vm);
if (!string.IsNullOrEmpty(vm.TextToDisplay))
{
vm.TextToDisplay = "I have inserted the value " + newmax + " into table (-1 means error)";
}
else
{
vm.TextToDisplay = "Errored!";
}
}
catch (Exception ex)
{
vm.TextToDisplay = "I encountered error message - \"" + ex.Message + "\"";
}
return View("Index", vm);
}
private async void MakePostCallAsync(string url, ViewModel vm)
{
var httpClient = new HttpClient();
var httpResponse = await httpClient.PostAsync("http://google.com", null).ConfigureAwait(true);
newmax = _dbContext.OnlyTable.Max(x => x.SomeColumn);
}
The issue is that, the MakePostCallAsync() method, when trying to query the database using DbContext, throws an exception saying DbContext is already disposed.
_dbContext in the example is injected using ASP .Net Core's DI (through AddDbContext() extension) with its default scope (Scoped).
I fail to and need help understand the following,
why the DB context is disposed even before the request is served, while Scoped object's lifetime should be the entire duration of the current request
Even though I have used ConfigureAwait(true) (which explicitly means that once the awaited method returns MakePostCallAsync() should continue in Request context?) - this doesn't seem to be happening
In the actual code, this was fixed once I made all the methods async (all the way up to controller) - in the repro I have shared, even that doesn't help prevent the exception - why is it so, how do I solve this?
Repro is available in https://github.com/jjkcharles/SampleAsync

You should never use async void unless you are writing a event handler. If you want to use async/await you need to go all the way up the call stack till you get to return a Task<IActionResult>
public async Task<IActionResult> Index()
{
var vm = new ViewModel();
try
{
var max = 0;
if (_dbContext.OnlyTable.Any())
{
max = _dbContext.OnlyTable.Max(x => x.SomeColumn);
}
_dbContext.Add(new TestTable() { SomeColumn = max + 1 });
_dbContext.SaveChanges();
await MakePostCallAsync("http:\\google.com", vm);
if (!string.IsNullOrEmpty(vm.TextToDisplay))
{
vm.TextToDisplay = "I have inserted the value " + newmax + " into table (-1 means error)";
}
else
{
vm.TextToDisplay = "Errored!";
}
}
catch (Exception ex)
{
vm.TextToDisplay = "I encountered error message - \"" + ex.Message + "\"";
}
return View("Index", vm);
}
private async Task MakePostCallAsync(string url, ViewModel vm)
{
var httpClient = new HttpClient();
var httpResponse = await httpClient.PostAsync("http://google.com", null).ConfigureAwait(true);
newmax = _dbContext.OnlyTable.Max(x => x.SomeColumn);
}

It seems to me you have some trouble understanding how to use async-await, because I see several major errors in your program.
This article, written by the ever so helpful Stephen Cleary helped me to understand how to use it properly.
Every function that wants to use async-await has to return Task instead of void and Task<TResult> instead of TResult. The only exception to this rule are event handlers that are not interested in the result of the actions.
So first change Index such that it returns a Task<IActionResult>. Once you've done this, your compiler will probably warn you that you forgot to await somewhere inside your Index function.
If you call an async function, you don't have to wait for it to finish, you can do other useful stuff, that will be performed whenever the async function has to await for something. But before you can use any result of the async function you have to await until it is ready:
var taskMakePostCall = MakePostCallAsync(...)
// if you have something useful to do, don't await
DoSomethingUseful(...);
// now you need the result of MakePostCallAsync, await until it is finished
await taskMakePostCall;
// now you can use the results.
If your MakePostCallAsync would have returned something, for instance an int, the return value would have been Task<int> and your code would have been:
Task<int> taskMakePostCall = MakePostCallAsync(...)
DoSomethingUseful(...);
int result = await taskMakePostCall;
If you don't have something useful to do, just await immediately:
int result = await MakePostCallAsync(...);
The reason for your exception is that your MakePostCallAsync is not finished completely before you somewhere Dispose your dbContext, probably via a using statement. After adding this await before returning you are certain that MakePostCallAsync is completely finished before returning Index()

In your example you are not awaiting your call to MakePostCallAsync("http:\\google.com", vm). This means the request continues execution immediately and it eventually executes the code that disposes of your _dbContext, presumably while MakePostCallAsync is still waiting for the HTTP client to return a response. Once the HTTP client does return a response your MakePostCallAsync tries to call newmax = _dbContext.OnlyTable.Max(x => x.SomeColumn) but your request has already been handled and your DB context disposed of by then.

C# async HttpWebRequest deadlocking

For few hours I am struggling with async code in C# and I can't really get why is my code deadlocked.
So far I've red many articles and anything ringed the bell for me.
Hope you can help me.
Here is the code I am trying to run.
Main
Task.Run(async () =>
{
Task<EventDetailed[]> events = getDetailedEvents();
await events;
}).Wait();
getDetailedEvents:
static async Task<EventDetailed[]> getDetailedEvents()
{
...
EventDetailed[] result = await LoadDetailedEventsDetailsAsync(evnts).ConfigureAwait(false);
return result;
}
And the core of my problem.
LoadDetailedEventsDetailsAsync
async static Task<EventDetailed[]> LoadDetailedEventsDetailsAsync(Event[] events)
{
List<EventDetailed> detailed = new List<EventDetailed>();
List<Task<WebResponse>> responses = new List<Task<WebResponse>>();
List<Event> tasksWithStream = new List<Event>();
foreach (Event e in events)
{
var httpWebRequest = (HttpWebRequest)WebRequest.Create("http://...");
... some headers etc ...
e.Stream = httpWebRequest.GetRequestStreamAsync();
e.WebRequest = httpWebRequest;
tasksWithStream.Add(e);
}
foreach (var tsk in tasksWithStream)
{
try {
await tsk.Stream.ConfigureAwait(false);
using (var streamWriter = new StreamWriter(tsk.Stream.Result))
{
streamWriter.Write("...");
streamWriter.Flush();
streamWriter.Close();
}
responses.Add(tsk.WebRequest.GetResponseAsync());
}
catch (Exception ex)
{
Logger.mes("Failed to get event data.");
}
}
foreach (var response in responses)
{
try
{
await response.ConfigureAwait(false);
}
catch (Exception ex)
{
Logger.mes("Failed to get event data.");
continue;
}
parseData.Add(ParseData(response));
}

A couple points:
First, it's important to note that you should almost never call .Wait (or .Result) on an async task - you should use await instead. One of the very few exceptions is in the Main method of a console app. The reason is that if you don't block the main thread, your program will simply exit prematurely.
Second, if you need to make multiple HTTP requests that do not depend on each other (i.e. request B does not need the results of request A), then there are huge performance gains to be had by executing them in parallel. Better still, you are not consuming a thread per request because the calls are asynchronous, i.e. they don't block a thread while waiting for a response, so the same thread can effectively fire off many concurrent requests.
I won't re-write your code, but I will suggest how I'd restructure it:
static void Main(string[] args)
{
// start all async tasks in parallel.
var tasks = GetEvents().Select(GetEventDetailsAsync);
// wait for them all to complete. normally you should use await instead of Wait,
// but you can't because you're in the main method of a console app.
Task.WhenAll(task).Wait();
}
static IEnumerable<Event> GetEvents()
{
// build a list of whatever metadata is needed to do your async work.
// do NOT do any actual async work here.
}
async static Task<EventDetailed> GetEventDetailsAsync(Event e)
{
// do all async work here, use await as needed,
// but only for one event (no loops).
}

Is it safe to call .ConfigureAwait(false) in finally block?

I have a "rest client" that wraps HttpClient and whose methods are async.
Besides other reasons, I need to control signin/signout process with my rest client so that number of sessions is not exceeded.
The rest client implements IDisposable and upon disposing the client I need to check if the client is "still signed in" and sign out if it is.
Since doing any kind of external calls in Dispose method is considered bad practice, I have something as following
public class MappingsController : RestController
{
[HttpGet]
public async Task<HttpResponseMessage> GetYears()
{
return await ProcessRestCall(async rc => await rc.GetYearsAsync());
}
}
public class RestController : ApiController
{
protected async Task<HttpResponseMessage> ProcessRestCall<T>(Func<RestClient, Task<T>> restClientCallback)
{
RestClient restClient = null;
try
{
var credentials = GetCredentialsFromRequestHeader();
if (credentials == null)
{
return Request.CreateErrorResponse(HttpStatusCode.Unauthorized, "Missing credentials from header!");
}
var username = credentials["Username"];
var password = credentials["Password"];
restClient = new RestClient(username, password);
var authenticated = await restClient.SignInAsync();
if (!authenticated)
{
return CreateErrorResponseWithRestStatus(HttpStatusCode.Unauthorized, restClient);
}
var result = await restClientCallback(restClient);
// Following works, but since I need to do it in finally block in case exception happens, perhaps It should be done in finally anyways...
//await restClient.SignOutAsync();
var response = Request.CreateResponse(HttpStatusCode.OK, result);
return response;
}
catch (Exception e)
{
return CreateErrorResponseWithRestStatus(HttpStatusCode.BadRequest, restClient, e);
}
finally
{
if (restClient != null)
{
if (restClient.IsSignedIn)
{
//var signedOutOk = restClient.SignOutAsync();//.Result; //<-- problem - this blocks!!!
restClient.SignOutAsync().ConfigureAwait(false); // seems to work, but I am not sure if this is kosher + I can't get return var
//Logger.Warn(CultureInfo.InvariantCulture, m => m("Client was still signed in! Attempt to to sign out was {0}", signedOutOk ? "successful" : "unsuccessful"));
}
restClient.Dispose();
}
}
}
}

The use of .ConfigureAwait(false) is a non-issue. You aren't awaiting on the task at all. Since you don't await it, it doesn't matter what await is configured to do.
What you're doing is just basic fire and forget (which may or may not be acceptable for you).
You should remove the ConfigureAwait(false) no matter what, just because it does nothing and is confusing to the reader. If it's okay for you to send the request to sign out but not actually sign out, then this is okay.
If you need to ensure that restClient.Dispose(); isn't called until the sign out request returns, then you have a bit of a...problem. The problem stems from the fact that the sign out request might be unsuccessful, or much worse, it might not respond at all. You'd need some way of dealing with that.
You can't use await in a finally block, but you can more or less mimic its behavior through continuations. You may need to do something like this:
public static async Task DoStuff()
{
IDisposable disposable = null;
try { }
finally
{
var task = GenerateTask();
var continuation = Task.WhenAny(task, Task.Delay(5000))
.ContinueWith(t =>
{
if (task.IsCompleted) //if false we timed out or it threw an exception
{
var result = task.Result;
//TODO use result
}
disposable.Dispose();
});
}
}
Note that since you aren't using await the task returned from DoStuff will indicate that it is "done" as soon as it hits the finally block for the first time; not when the continuation fires and the object is disposed. That may or may not be acceptable.