Develop Reference

c# a task was canceled at the await call

I'm trying to hit my simple server's endpoint using the following code, but I keep getting "A task was canceled." during the await call. The server's logs don't show any errors and cts.IsCancellationRequested == false, however e.CancellationToken.IsCancellationRequested == true. Any advice on how to track down the cause of this cancellation? At the very least, how can I tell if it's coming from the front end or the server endpoint?
private async Task<string> SendSingleRequestToDlis(
HttpClient client,
StringContent requestData)
{
int timeout = 600000; // in ms
string dlisEndpoint = "myendpointhere";
string response;
using (var cts = new CancellationTokenSource(timeout))
{
//send
HttpResponseMessage request;
try
{
request = await client.PostAsync(dlisEndpoint, requestData);
}
catch (Exception e)
{
throw new Exception("Could not establish conection to model hosted on DLIS.", e);
}
....

You are not using your CancellationToken so don't implement it. Just use this:
string dlisEndpoint = "myendpointhere";
string response;
HttpResponseMessage request;
try
{
request = await client.PostAsync(dlisEndpoint, requestData);
}
catch (Exception e)
{
throw new Exception("Could not establish conection to model hosted on DLIS.", e);
}
If you wanted to use the cancellation token you actually have to pass it down into the client.
request = await client.PostAsync(dlisEndpoint, requestData, cts.Source);

At the very least, how can I tell if it's coming from the front end or the server endpoint?
You can know it's at the client because you're seeing an OperationCanceledException. A server can just return a response code, which you would see as an HttpRequestException.
i timed it just now and the cancellation happens at exactly 100s
This is the default value of the HttpClient.Timeout property. Set this property higher (or to Timeout.InfiniteTimeSpan) at the point where the HttpClient is created (or configured, if the HttpClient is injected).

Polly HandleTransientHttpError not catching HttpRequestException

I've created a retry policy on my HttpClient in the Startup.ConfigureServices method. Note also that by default, asp.net core 2.1 logs 4 [Information] lines for each call made by the HttpClient which are shows in the logs at the end of my question.
services.AddHttpClient("ResilientClient")
.AddPolicyHandler(
Policy.WrapAsync(
PollyRetryPolicies.TransientErrorRetryPolicy(),
Policy.TimeoutAsync<HttpResponseMessage>(TimeSpan.FromSeconds(60))));
The policy is defined as follows. Note that I write the retry attempt to logs, so I will know if the retry policy is invoked.
public static IAsyncPolicy < HttpResponseMessage > TransientErrorRetryPolicy() {
return HttpPolicyExtensions
.HandleTransientHttpError()
.Or < TimeoutRejectedException > ()
.WaitAndRetryAsync(sleepDurations: ExponentialBackoffPolicy.DecorrelatedJitter(3, SEED_DELAY, MAX_DELAY),
onRetry: (message, timespan, attempt, context) => {
context.GetLogger() ? .LogInformation($ "Retrying request to {message?.Result?.RequestMessage?.RequestUri} in {timespan.TotalSeconds} seconds. Retry attempt {attempt}.");
});
}
HandleTransientHttpError() is a Polly extension that states in it's comments:
The conditions configured to be handled are:
• Network failures (as System.Net.Http.HttpRequestException)
My httpclient usage is like this:
using (HttpResponseMessage response = await _httpClient.SendAsync(request))
{
response.EnsureSuccessStatusCode();
try
{
string result = await response.Content.ReadAsStringAsync();
if (result == null || result.Trim().Length == 0) {
result = "[]";
}
return JArray.Parse(result);
} catch (Exception ex) {
_logger.LogInformation($ "Failed to read response from {url}. {ex.GetType()}:{ex.Message}");
throw new ActivityException($ "Failed to read response from {url}.", ex);
}
}
The following logs are captured:
[Information] System.Net.Http.HttpClient.ResilientClient.LogicalHandler: Start processing HTTP request GET https://api.au.... obfuscated
[Information] System.Net.Http.HttpClient.ResilientClient.CustomClientHandler: Sending HTTP request GET https://api.au..... obfuscated
[Information] System.Net.Http.HttpClient.ResilientClient.CustomClientHandler: Received HTTP response after 2421.8895ms - 200
[Information] System.Net.Http.HttpClient.ResilientClient.LogicalHandler: End processing HTTP request after 2422.1636ms - OK
Unknown error responding to request: HttpRequestException:
System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: The server returned an invalid or unrecognized response.
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.CopyToAsyncCore(Stream destination, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.HttpConnectionResponseContent.SerializeToStreamAsync(Stream stream, TransportContext context, CancellationToken cancellationToken)
at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
--- End of inner exception stack trace ---
at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at nd_activity_service.Controllers.ActivityController.GetND(String url) in /codebuild/output/src251819872/src/src/nd-activity-service/Controllers/ActivityController.cs:line 561
The Http call succeeds, and I can see it returns 200 - OK. But then the HttpRequestException is thrown. I assume the policy is not being invoked because the HttpClient message pipeline has already resolved, as we can see it returned 200 - OK. So how is it throwing an exception outside of this?
And how do I handle it? Wrap another policy around the method that handles HttpRequestExceptions specifically?
This error does appear to be transient. It is a scheduled job and works the next time it is called.

Your policy is defined against the HttpClient not against the HttpResponseMessage.
So, the response.EnsureSuccessStatusCode() will not trigger retry even if you receive for example 428.
The HandleTransientHttpError will trigger retry if you receive 408 or 5XX status codes from the downstream system. And when the SendAsync throws the HttpRequestException
Because your exception StackTrace looks like this:
System.Net.Http.HttpRequestException: Error while copying content to a stream.
System.IO.IOException: The server returned an invalid or
unrecognized response.
that's why my educated guess is that this exception is thrown by the HttpContent class while you try to read the response body (ReadAsStringAsync).
This will not trigger retry since you have defined your policy on the HttpClient.
If you want to retry in those cases as well when either the response.EnsureSuccessStatusCode() throws HRE or when the response.Content.ReadAsStringAsync() does then you have to wrap your whole http communication and response processing logic into a retry policy.
Let me show you how to do that.
First use a PolicyRegistry instead of AddPolicyHandler:
//services.AddHttpClient("ResilientClient")
// .AddPolicyHandler(
// Policy.WrapAsync(
// TransientErrorRetryPolicy(),
// Policy.TimeoutAsync<HttpResponseMessage>(TimeSpan.FromSeconds(60))));
services.AddHttpClient("ResilientClient");
var registry = services.AddPolicyRegistry();
registry.Add("retry", Policy.WrapAsync(
TransientErrorRetryPolicy(),
Policy.TimeoutAsync<HttpResponseMessage>(TimeSpan.FromSeconds(60))));
Then ask the DI for the register, for example:
private readonly IHttpClientFactory factory;
private readonly IReadOnlyPolicyRegistry<string> registry;
public TestController(IHttpClientFactory factory, IReadOnlyPolicyRegistry<string> registry)
{
this.factory = factory;
this.registry = registry;
}
Finally retrieve the combined policy and execute the http call:
var retryPolicy = registry.Get<IAsyncPolicy<HttpResponseMessage>>("retry");
await retryPolicy.ExecuteAsync(async () => await IssueRequest());
private async Task<HttpResponseMessage> IssueRequest()
{
var _httpClient = factory.CreateClient("ResilientClient");
HttpResponseMessage response = await _httpClient.GetAsync("http://httpstat.us/428");
response.EnsureSuccessStatusCode();
return response;
}
I've used the httpstat.us to simulate 428 response.

Losing error message with Http Status code 503 in System.net.Http library

We have 2 backend services communicating with each other, both of them are using .net framework with System.net.Http library
One of them returns HTTP status code 503 with some response message("data is not available") when certain condition is met, as shown below in the screenshot
the controller code implementation looks like this:
HttpResponseMessage response = new HttpResponseMessage { StatusCode = HttpStatusCode.ServiceUnavailable, ReasonPhrase = "XXXXXX data is currently not available XXXXXXXXX" };
response.Content = new StringContent("XXXXXX is currently not available XXXXXXXXXX");
return response;
the other service which is calling this API is consuming the response and gets this WebException, which is desired case but the problem is, we lose the message("data is not available") being sent from the first service, when I tried to print the response in the debug window, it showed the following result :
HttpWebResponse aresponse = request.GetResponse() as HttpWebResponse;
'HttpWebResponse aresponse = request.GetResponse() as HttpWebResponse' threw an exception of type 'System.Net.WebException'
Data: {System.Collections.ListDictionaryInternal}
HResult: -2146233079
HelpLink: null
InnerException: null
Message: "The remote server returned an error: (503) Server Unavailable."
Response: {System.Net.HttpWebResponse}
Source: "System"
StackTrace: null
Status: ProtocolError
TargetSite: null
As shown in the output message is not what I sent from the first service. It's the default message for HTTP status code 503.
Is there a way for me to read the response content from the WebException?

Found the solution, you can read the response from the System.net.WebException
catch(WebException ex)
{
HttpWebResponse errorResponse = webException.Response as HttpWebResponse;
var statusDescription = errorResponse.StatusDescription;
if(statusDescription.Contains("XXXXX"))
{
// Added the condition here
}
}

WebRequest Strange NotFound Error

I have 2 different ASP.NET Core websites: admin and public.
Both running on staging server and on local machine.
I send GET request to different pages to determine execution time of different pages and encountered problem with admin site: all urls on local instance and staging always returns 404 error:
An unhandled exception of type 'System.Net.WebException' occurred in
System.dll
Additional information: The remote server returned an error: (404) Not
Found.
Meanwhile, same requests in browser return html pages normally. Requests through HttpWebRequest to public site always also return 200 Status Code (OK).
Code for request I took here.
I tried to add all headers and cookies from browser request, but it didn't help. Also tried to debug local instance and found that no exceptions thrown while request executed.
Any ideas?

404 is way to generic. The code provided in answer in your link (https://stackoverflow.com/a/16642279/571203) does no error handling - this is brilliant example of how you can get to troubles when you blindly copy code from stackoverflow :)
Modified code with error handling should look like:
string urlAddress = "http://google.com/rrr";
var request = (HttpWebRequest)WebRequest.Create(urlAddress);
string data = null;
string errorData = null;
try
{
using (var response = (HttpWebResponse)request.GetResponse())
{
data = ReadResponse(response);
}
}
catch (WebException exception)
{
using (var response = (HttpWebResponse)exception.Response)
{
errorData = ReadResponse(response);
}
}
static string ReadResponse(HttpWebResponse response)
{
if (response.CharacterSet == null)
{
using (var reader = new StreamReader(response.GetResponseStream()))
{
return reader.ReadToEnd();
}
}
using (var reader = new StreamReader(response.GetResponseStream(), Encoding.GetEncoding(response.CharacterSet)))
{
return reader.ReadToEnd();
}
}
So when there is an exception, you'll get not just the status code, but entire response from server in the errorData variable.
One thing to check is proxy - browser can use http proxy while your server client uses none.

RestSharp + Server Down - How do you know if server is down?

I am wondering how can I check if the RestSharp request I made failed because the server is down vs something else.
When I shutdown my server I get a status code of "NotFound" but that could be a particular record was not found(which I do on my site if say they try to find a record that might be recently deleted).
How can I figure out the server is actually down?
Edit
here is my code
private readonly RestClient client = new RestClient(GlobalVariables.ApiUrl);
var request = new RestRequest("MyController", Method.POST);
request.AddParameter("UserId", "1");
request.AddParameter("Name", name.Trim());
var asyncHandle = client.ExecuteAsync(request, response =>
{
var status = response.StatusCode;
});

When the server is down, it should not return a "404 NotFound" error.
The most appropriate in this case is HTTP Error 503 - Service unavailable
The Web server (running the Web site) is currently unable to handle
the HTTP request due to a temporary overloading or maintenance of the
server. The implication is that this is a temporary condition which
will be alleviated after some delay. Some servers in this state may
also simply refuse the socket connection, in which case a different
error may be generated because the socket creation timed out.
That way checking that RestResponse.StatusCode is 503 it will tell you that the server is down.

I am having the same issue.
I decided to check for the ContentLength too. At least my webservice always returns ContentLength>0 (even for NotFound occurrences). This seems to work out.
if ( response.StatusCode == System.Net.HttpStatusCode.NotFound &&
response.ContentLength == -1 ){
==> client couldn't connect to webservice
}

In my scenario, I needed to check if anything happened to the connection, e.g. connection time out, can't resolve the host name, etc. So I also added following code:
try
{
var client = new RestClient(_configuration.ServerUrl);
var request = new RestRequest
{
Resource = _configuration.SomeUrl,
Method = Method.POST
};
var response = client.Execute(request);
if (response.ErrorException != null)
{
throw response.ErrorException;
}
}
catch (WebException ex)
{
// TODO: check ex.Status, if it matches one of needed conditions.
}
For more information about WebExceptionStatus Enumeration, please, see following link https://msdn.microsoft.com/en-us/library/system.net.webexceptionstatus(v=vs.110).aspx