I created a FileResult : IHttpActionResult webapi return type for my api calls. The FileResult downloads a file from another url and then returns the stream to the client.
Initially my code had a using statement like below:
public async Task<HttpResponseMessage> ExecuteAsync(CancellationToken cancellationToken)
{
try
{
HttpResponseMessage response;
using (var httpClient = new HttpClient())
{
response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new System.Net.Http.StreamContent(
await httpClient.GetStreamAsync(this.filePath))
};
}
return response;
}
catch (WebException exception)
{...}
}
However this would intermittently cause a TaskCanceledException. I know that if the HttpClient is disposed before the asychronous call is finished the Task's state will change to canceled. However since I use an await in: Content = new System.Net.Http.StreamContent(await httpClient.GetStreamAsync(this.filePath)) that should prevent the HttpClient from being disposed off in the middle of the task completion.
Why does that task get canceled? It is not because of a timeout since this has happened on the smallest requests and doesn't always occur on large requests.
When I removed the using statement the code worked properly:
public async Task<HttpResponseMessage> ExecuteAsync(CancellationToken cancellationToken)
{
try
{
HttpResponseMessage response;
var httpClient = new HttpClient();
response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new System.Net.Http.StreamContent(
await httpClient.GetStreamAsync(this.filePath))
};
return response;
}
catch (WebException exception)
{...}
}
Any idea why the using caused the issue?
I know that if the HttpClient is disposed before the asychronous call is finished the Task's state will change to canceled. However since I use an await in: Content = new System.Net.Http.StreamContent(await httpClient.GetStreamAsync(this.filePath)) that should prevent the HttpClient from being disposed off in the middle of the task completion.
But what does that task do? It gets the stream. So, your code ends up with a Stream that may or may not be completely read when it closes the HttpClient.
HttpClient is specifically designed for reuse (and simultaneous use), so I recommend removing the using completely and moving the HttpClient declaration to a static class member. But if you want to close and reopen the clients, you should be able to get it working by reading the stream entirely into memory before closing the HttpClient.
I had a similar issue with Task Canceled exceptions. If you try catching AggregateException or having a catch all Exception block underneath your WebException, you may well find that you catch it, with one exception with the entry stating "A task was canceled"
I did some investigation and found that the AggregateException is quite misleading as described in various threads;
Setting HttpClient to a too short timeout crashes process
How can I tell when HttpClient has timed out?
Bug in httpclientgetasync should throw webexception not taskcanceledexception
I ended up changing my code to set an explicit timeout (where asyncTimeoutInMins is read from the app.config file);
string jsonResponse = string.Empty;
try
{
using (HttpClient httpClient = new HttpClient())
{
httpClient.BaseAddress = new Uri(Properties.Settings.Default.MyWebService);
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.Timeout = new TimeSpan(0, asyncTimeoutInMins, 0);
HttpResponseMessage response;
response = await httpClient.GetAsync("/myservice/resource");
// Check the response StatusCode
if (response.IsSuccessStatusCode)
{
// Read the content of the response into a string
jsonResponse = await response.Content.ReadAsStringAsync();
}
else if (response.StatusCode == HttpStatusCode.Forbidden)
{
jsonResponse = await response.Content.ReadAsStringAsync();
Logger.Instance.Warning(new HttpRequestException(string.Format("The response StatusCode was {0} - {1}", response.StatusCode.ToString(), jsonResponse)));
Environment.Exit((int)ExitCodes.Unauthorised);
}
else
{
jsonResponse = await response.Content.ReadAsStringAsync();
Logger.Instance.Warning(new HttpRequestException(string.Format("The response StatusCode was {0} - {1}", response.StatusCode.ToString(), jsonResponse)));
Environment.Exit((int)ExitCodes.ApplicationError);
}
}
}
catch (HttpRequestException reqEx)
{
Logger.Instance.Error(reqEx);
Console.WriteLine("HttpRequestException : {0}", reqEx.InnerException.Message);
Environment.Exit((int)ExitCodes.ApplicationError);
}
catch (Exception ex)
{
Logger.Instance.Error(ex);
throw;
}
return jsonResponse;
Related
I'm trying to hit my simple server's endpoint using the following code, but I keep getting "A task was canceled." during the await call. The server's logs don't show any errors and cts.IsCancellationRequested == false, however e.CancellationToken.IsCancellationRequested == true. Any advice on how to track down the cause of this cancellation? At the very least, how can I tell if it's coming from the front end or the server endpoint?
private async Task<string> SendSingleRequestToDlis(
HttpClient client,
StringContent requestData)
{
int timeout = 600000; // in ms
string dlisEndpoint = "myendpointhere";
string response;
using (var cts = new CancellationTokenSource(timeout))
{
//send
HttpResponseMessage request;
try
{
request = await client.PostAsync(dlisEndpoint, requestData);
}
catch (Exception e)
{
throw new Exception("Could not establish conection to model hosted on DLIS.", e);
}
....
You are not using your CancellationToken so don't implement it. Just use this:
string dlisEndpoint = "myendpointhere";
string response;
HttpResponseMessage request;
try
{
request = await client.PostAsync(dlisEndpoint, requestData);
}
catch (Exception e)
{
throw new Exception("Could not establish conection to model hosted on DLIS.", e);
}
If you wanted to use the cancellation token you actually have to pass it down into the client.
request = await client.PostAsync(dlisEndpoint, requestData, cts.Source);
At the very least, how can I tell if it's coming from the front end or the server endpoint?
You can know it's at the client because you're seeing an OperationCanceledException. A server can just return a response code, which you would see as an HttpRequestException.
i timed it just now and the cancellation happens at exactly 100s
This is the default value of the HttpClient.Timeout property. Set this property higher (or to Timeout.InfiniteTimeSpan) at the point where the HttpClient is created (or configured, if the HttpClient is injected).
I am facing an issue regarding not getting response from GetAsync API of HttpClient in MVC Applications(Target Framework - .Net Framework 4.7) whereas getting response in web services and console applications with same snippet. Here I have attached code snippet which I am trying to execute.
public void Get()
{
var response = Gettasks().Result;
}
public static async Task<HttpResponseMessage> GetTasks()
{
var response = new HttpResponseMessage();
try
{
using (var client = new HttpClient())
{
response = await client.GetAsync("https://www.google.com");
}
}
catch (Exception exception)
{
Console.WriteLine(exception.Message);
}
return response;
}
I am getting stuck on response = await client.GetAsync("https://www.google.com"); this line and not getting any response after executing this statement.
If anyone can please suggest solution for this or provide fix/solution which works for you.
You're seeing a deadlock because the code is blocking on an asynchronous method.
The best fix is to remove the blocking:
public async Task Get()
{
var response = await Gettasks();
}
This deadlock happens because await captures a context, and ASP.NET (pre-Core) has a context that only allows one thread at a time, and the code blocks a thread (.Result) in that context, which prevents GetTasks from completing.
Both the context and the blocking are necessary to see this kind of deadlock. In the other scenarios, there is no context, so that is why the deadlock does not occur. Since ASP.NET (pre-Core) has a context, the proper solution here is to remove the blocking.
Not sure whether you have tried following which is working for me.
using (HttpClient client = new HttpClient())
{
client.BaseAddress = new Uri(Environment.GetEnvironmentVariable("BaseAddress"));
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var requestUri = Environment.GetEnvironmentVariable("Uri");
HttpResponseMessage response = await client.GetAsync(requestUri);
if (response.IsSuccessStatusCode)
{
var data = await response.Content.ReadAsStringAsync();
}
}
Does anyone know why HttpClient - PostAsync doesn’t return. It just does nothing. I have had it working occasionally especially for one off posts but it seems sometimes to not do anything especially under load and it doesn't throw an exception which of course makes my code unreliable and hard to debug.
I have tried adding ConfigureAwait(false) It makes not difference.
I suspect the Task is failing to 'pack'
This is in a core 3.0 console app run on macOS Catalina using visual studio code
This code is pretty much copied from Microsoft's documentation and I’m calling Microsoft Graph when posting.
public static async Task PostAsync(HttpClient httpClient, string url, string token, HttpContent content, Action<JObject> processResult, ILogger log)
{
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
// content.Headers.Clear();
content.Headers.Add("Content-Type", "application/json");
try
{
HttpResponseMessage response = await httpClient.PostAsync(url, content);
if (response.IsSuccessStatusCode)
{
var json = await response.Content.ReadAsStringAsync();
var result = JsonConvert.DeserializeObject(json) as JObject;
processResult(result);
}
else
{
var errorContent = await response.Content.ReadAsStringAsync();
log.LogError(errorContent);
}
}
catch (System.Exception ex)
{
log.LogError(ex, ex.Message);
throw;
}
}
Here is an example of the calling code
public async Task SendInvitation(string token, Invitation invitation, ILogger logger)
{
var stringContent = new StringContent(JsonConvert.SerializeObject(invitation), Encoding.UTF8, "application/json");
await HttpHelpers.PostAsync(
Client,
"https://graph.microsoft.com/v1.0/invitations",
token,
stringContent,
result => logger.LogInformation(DebugHelpers.Print(result)),
logger);
}
Answered (Sort of)
If I change
HttpResponseMessage response = await httpClient.PostAsync(url, content);
to
HttpResponseMessage response = httpClient.PostAsync(url, content).GetAwaiter().GetResult();
It seems to work but it's slow because what I'm doing is using blocking code. I think this is a quirk of core 3 on macOS. I don't like that this is happening.
More Info
I'm doing a lot of looping.
It seems that if I put all the things I'm awaiting in a taskList it behaves properly.
\\ Pseudo Code
var taskList = new List<Task>();
foreach(var thing in things){
taskList.Add(HttpHelpers.PostAsync(...things));
}
await Task.WhenAll(taskList);
Please check whether all calls you make in the code execution path support asynchronousity. For example once I spent quite some time figuring out a nuget package called CommandLineParser did not support async calls. I was using it like so :
public static void Main(string[] args)
{
Parser.Default.ParseArguments<Options>(args)
.WithParsed(async options =>
{ await httphelper.PostAsync(...);
}
}
I fixed the issue by changing it to something like
public static void Main(string[] args)
{
Parser.Default.ParseArguments<Options>(args)
.WithParsed(options =>
{ httphelper.PostAsync(...).Result;
}
}
So please check you are not using some calls that do not support async in the way.
You can wait for HttpResponseMessage certain condition. Inspect HttpResponseMessage in debug, with 2 given steps. Screenshot of debug process: postAsync debug
return HttpResponseMessage from method with postAsync:
private static async Task<HttpResponseMessage> methodWithPostAsync(){
...
response = await client.PostAsync(url, data);
return response
}
call method and wait for response message status:
Task<HttpResponseMessage> content= methodWithPostAsync();
while (!content.IsCompleted)
{
Console.WriteLine("busy");
System.Threading.Thread.Sleep(1000);
}
try ConfigureAwait like this
HttpResponseMessage response = await httpClient.PostAsync(url, content).ConfigureAwait(false);
I get somehow a Deadlock/hanging after I purposely catch a WebServiceException to let my application continue. However, even though the application contiues. Doing a webservice hangs and it probably seems like it is still trying to do something from the previous call.
I tried using CancellationTokenSource but that did not seem to solve my problem.
RetryHandler:
public class RetryHandler : DelegatingHandler
{
private const int MaxRetries = 2;
public RetryHandler(HttpMessageHandler innerHandler)
: base(innerHandler)
{
}
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
HttpResponseMessage response = null;
Exception lastException = null;
for (var i = 0; i < MaxRetries; i++)
{
try
{
response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
{
lastException = ex;
}
if (IsSuccessful(response))
{
return response;
}
}
throw GetException(response, lastException);
}
Calling Post twice makes my program hang:
public async Task<T> Post<T>(
string path,
HttpContent content,
string username,
string token,
HttpMessageHandler handler)
{
var client = new HttpClient(new RetryHandler(handler));
var authString = GetAuthenticationString(username, token);
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", authString);
AddUsernameAndTokenToClientRequestHeader(client, username, token);
client.DefaultRequestHeaders.TryAddWithoutValidation("Content-Type", content.Headers.ContentType.MediaType);
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("*/*"));
var result = await client.PostAsync(new Uri(path), content, _cancelHttpRequests.Token).ConfigureAwait(false);
var resultContent = result.Content.ReadAsStringAsync().Result;
return JsonConvert.DeserializeObject<T>(resultContent);
}
What is happening here is, although you are catching the exception, and supposedly let your application continue, the service itself continues it's work, async, so that even when you are trying to force the service to continue, it will still go on attempting to complete all of the desired action.
In your case: causing deadlock. Cancellation token won't help here, as your service running async and you already stopped it by catching the exception, so, you are basically doing nothing with this token.
Two ways to solve this:
Either, disconnect the service when you are getting the exception, this way forcing the service to shut.
Or try to work with your service in a sync way so that you can stop the service when ever needed, this way insuring it won't do any additional work when you stop it.
I want to iterate a batch of requests, sending each one of them to an external API using HttpClient class.
foreach (var MyRequest in RequestsBatch)
{
try
{
HttpClient httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMilliseconds(5);
HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endpoint), myRequest);
JObject resultResponse = await response.Content.ReadAsAsync<JObject>();
}
catch (Exception ex)
{
continue;
}
}
The context here is I need to set a very small timeout value, so in case the response takes more than that time, we simply get the "Task was cancelled" exception and continue iterating.
Now, in the code above, comment these two lines:
HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endpoint), myRequest);
resultResponse = await response.Content.ReadAsAsync<JObject>();
The iteration ends very fast. Uncomment them and try again. It takes a lot of time.
I wonder if calling PostAsJsonAsync/ReadAsAsync methods with await takes more time than the timeout value?
Based on the answer below, supposing it will create different threads, we have this method:
public Task<JObject> GetResponse(string endPoint, JObject request, TimeSpan timeout)
{
return Task.Run(async () =>
{
try
{
HttpClient httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMilliseconds(5);
HttpResponseMessage response = await httpClient.PostAsJsonAsync<string>(string.Format("{0}api/GetResponse", endPoint), request).WithTimeout<HttpResponseMessage>(timeout);
JObject resultResponse = await response.Content.ReadAsAsync<JObject>().WithTimeout<JObject>(timeout);
return resultResponse;
}
catch (Exception ex)
{
return new JObject() { new JProperty("ControlledException", "Invalid response. ")};
}
});
}
An exception is raised there and the JObject exception should be returned, very fast, however, if using httpClient methods, even if it raises the exception it takes a lot of time. Is there a behind the scenes processing affecting the Task even if the return value was a simple exception JObject?
If yes, which another approach could be used to send a batch of requests to an API in a very fast way?
I agree with the accepted answer in that the key to speeding things up is to run the requests in parallel. But any solution that forces additional threads into the mix by use of Task.Run or Parallel.ForEach is not gaining you any efficiency with I/O bound asynchronous operations. If anything it's hurting.
You can easily get all calls running concurrently while letting the underlying async subsystems decide how many threads are required to complete the tasks as efficiently as possible. Chances are that number is much smaller than the number of concurrent calls, because they don't require any thread at all while they're awaiting a response.
Further, the accepted answer creates a new instance of HttpClient for each call. Don't do that either - bad things can happen.
Here's a modified version of the accepted answer:
var httpClient = new HttpClient {
Timeout = TimeSpan.FromMilliseconds(5)
};
var taskList = new List<Task<JObject>>();
foreach (var myRequest in RequestsBatch)
{
// by virtue of not awaiting each call, you've already acheived parallelism
taskList.Add(GetResponseAsync(endPoint, myRequest));
}
try
{
// asynchronously wait until all tasks are complete
await Task.WhenAll(taskList.ToArray());
}
catch (Exception ex)
{
}
async Task<JObject> GetResponseAsync(string endPoint, string myRequest)
{
// no Task.Run here!
var response = await httpClient.PostAsJsonAsync<string>(
string.Format("{0}api/GetResponse", endpoint),
myRequest);
return await response.Content.ReadAsAsync<JObject>();
}
It doesn't look like you're actually running a seperate thread for each request. Try something like this:
var taskList = new List<Task<JObject>>();
foreach (var myRequest in RequestsBatch)
{
taskList.Add(GetResponse(endPoint, myRequest));
}
try
{
Task.WaitAll(taskList.ToArray());
}
catch (Exception ex)
{
}
public Task<JObject> GetResponse(string endPoint, string myRequest)
{
return Task.Run(() =>
{
HttpClient httpClient = new HttpClient();
HttpResponseMessage response = httpClient.PostAsJsonAsync<string>(
string.Format("{0}api/GetResponse", endpoint),
myRequest,
new CancellationTokenSource(TimeSpan.FromMilliseconds(5)).Token);
JObject resultResponse = response.Content.ReadAsAsync<JObject>();
});
}