Develop Reference

.NET Core 3.1 HttpClient throws exception intermittently: SocketException: The I/O operation has been aborted because of either a thread exit or an

Intermittently I get the exceptions below:
IOException: Unable to read data from the transport connection: The
I/O operation has been aborted because of either a thread exit or an
application request..
SocketException: The I/O operation has been aborted because of either
a thread exit or an application request.
The system is querying an external resource and from time to time the exceptions happen without anything seeming to be out of the ordinary. I have tried to set a longer timeout for HttpClient but it did not help. It could be anywhere from 5000-50000 searches before the exception happens but I would still like to mitigate it. If I retry the same search directly after exception it works so the receiving party does not seem to have a problem even though I can't access that applications logs. Runs on .NET Core 3.1.
MyService.cs
public class MyService
{
private readonly HttpClient _httpClient;
public MyService(HttpClient client)
{
client.BaseAddress = new Uri("https://example.com/");
client.Timeout = TimeSpan.FromMinutes(5);
_httpClient = client;
}
private async Task<List<string>> GetValuesFromSearch(string search)
{
//Exception is thrown here
var response = await _httpClient.GetAsync("search/" + search);
using var responseStream = await response.Content.ReadAsStreamAsync();
response.EnsureSuccessStatusCode();
var searchResultList = await JsonSerializer.DeserializeAsync
<List<string>>(responseStream);
return searchResultList;
}
}
Called like this:
var myService = new MyService(new HttpClient());
foreach (var search in listToIterate)
{
//Can be called up to 200 000 times
var result = await myService.GetValuesFromSearch(search);
}

The issue could be due to socket exhaustion. This is a known issue with HttpClient and the solution is to use HttpClientFactory. I haven't tested this but here's a quick re-write of your MyService class:
public class MyService
{
private readonly IHttpClientFactory _httpClientFactory;
public MyService(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory ??
throw new ArgumentNullException(nameof(httpClientFactory));
}
private async Task<List<string>> GetValuesFromSearch(string search)
{
var _httpClient = _httpClientFactory.CreateClient("MyClient");
_httpClient.BaseAddress = new Uri("https://example.com/");
_httpClient.Timeout = TimeSpan.FromMinutes(5);
// You could also set the above in Startup.cs or wherever you add your services:
//services.AddHttpClient("MyClient", c => {
// c.BaseAddress = new Uri("https://example.com/");
// c.Timeout = TimeSpan.FromMinutes(5);
//});
//Exception is thrown here
var response = await _httpClient.GetAsync("search/" + search);
using var responseStream = await response.Content.ReadAsStreamAsync();
response.EnsureSuccessStatusCode();
var searchResultList = await JsonSerializer.DeserializeAsync
<List<string>>(responseStream);
return searchResultList;
}
}

If your request fails to return a HttpResponseMessage, HttpClient will throw this as the inner exception stack trace of type TaskCancelledException. To confirm, try using Polly to add a TimeoutAsync policy; the exception should change to a TimeOutRejectedException.
In a similar use case, the best solution I have found is to:
use HttpClientFactory, and skip the using statement when you use your client.
use Polly to build in some additional resiliency.
wrap the HttpClient code in a try/catch block so your process doesn't bomb out in the event there is no HttpResponseMessage to work with (this is the actual 'fix,' the other two pieces are to keep the exception from throwing as frequently):
private async Task<List<string>> GetValuesFromSearch(string search)
{
try
{
//Exception is thrown here
var response = await _httpClient.GetAsync("search/" + search);
var responseStream = await response.Content.ReadAsStreamAsync();
response.EnsureSuccessStatusCode();
var searchResultList = await JsonSerializer.DeserializeAsync
<List<string>>(responseStream);
return searchResultList;
}
catch (Exception ex)
{
// Log the exception.
// Do what you want, or return null and handle a dropped request in your calling method.
}
}

HttpClient GetAsync not working as expected

When testing my web API with Postman my API get executes fine!
When it comes to running the code with HttpClient in my client application the code executes without error but without the expected result on the server.
What could be happening?
From my client application:
private string GetResponseFromURI(Uri u)
{
var response = "";
HttpResponseMessage result;
using (var client = new HttpClient())
{
Task task = Task.Run(async () =>
{
result = await client.GetAsync(u);
if (result.IsSuccessStatusCode)
{
response = await result.Content.ReadAsStringAsync();
}
});
task.Wait();
}
return response;
}
Here is the API controller:
[Route("api/[controller]")]
public class CartsController : Controller
{
private readonly ICartRepository _cartRepo;
public CartsController(ICartRepository cartRepo)
{
_cartRepo = cartRepo;
}
[HttpGet]
public string GetTodays()
{
return _cartRepo.GetTodaysCarts();
}
[HttpGet]
[Route("Add")]
public string GetIncrement()
{
var cart = new CountedCarts();
_cartRepo.Add(cart);
return _cartRepo.GetTodaysCarts();
}
[HttpGet]
[Route("Remove")]
public string GetDecrement()
{
_cartRepo.RemoveLast();
return _cartRepo.GetTodaysCarts();
}
}
Note these API calls work as expected when called from Postman.

You shouldn't use await with client.GetAsync, It's managed by .Net platform, because you can only send one request at the time.
just use it like this
var response = client.GetAsync("URL").Result; // Blocking call!
if (response.IsSuccessStatusCode)
{
// Parse the response body. Blocking!
var dataObjects = response.Content.ReadAsAsync<object>().Result;
}
else
{
var result = $"{(int)response.StatusCode} ({response.ReasonPhrase})";
// logger.WriteEntry(result, EventLogEntryType.Error, 40);
}

You are doing fire-and-forget approach. In your case, you need to wait for the result.
For example,
static async Task<string> GetResponseFromURI(Uri u)
{
var response = "";
using (var client = new HttpClient())
{
HttpResponseMessage result = await client.GetAsync(u);
if (result.IsSuccessStatusCode)
{
response = await result.Content.ReadAsStringAsync();
}
}
return response;
}
static void Main(string[] args)
{
var t = Task.Run(() => GetResponseFromURI(new Uri("http://www.google.com")));
t.Wait();
Console.WriteLine(t.Result);
Console.ReadLine();
}

Simple sample used to get page data.
public string GetPage(string url)
{
HttpResponseMessage response = client.GetAsync(url).Result;
if (response.IsSuccessStatusCode)
{
string page = response.Content.ReadAsStringAsync().Result;
return "Successfully load page";
}
else
{
return "Invalid Page url requested";
}
}

I've had a problem with chace control when using httpclient.
HttpBaseProtocalFilter^ filter = ref new HttpBaseProtocolFilter();
filter->CacheControl->ReadBehavior = Windows::Web::Http::Filters::HttpCacheReadBehavior::MostRecent;
HttpClient^ httpClient = ref new HttpClient(filter);
I'm not really sure what the expected results are or what results your getting at all so this is really just a guessing game right now.
When I POST something using HttpClient I found adding headers by hand seemed to work more often than using default headers.
auto httpClient = ref new HttpClient();
Windows::Web::Http::Headers::HttpMediaTypeHeaderValue^ type = ref new Windows::Web::http::Headers::HttpMediaTypeHeaderValue("application/json");
content->Headers->ContentType = type;
If I don't do these 2 things I found, for me anyways, that half the time my web requests were either not actually being sent or the headers were all messed up and the other half of the time it worked perfectly.
I just read a comment where you said it would only fire once, that makes me think it is the cachecontrol. I think what happens is something (Windows?) sees 2 requests being sent that are the exact same, so to speed things up it just assumes the same answer and never actually sends the request a 2nd time

Correct way to make async web request

Hi I need to write a proper async web request function in C# to get some data in JSON format and deserialize it in strongly typed object. I came up with this solution:
public async Task<TReturnType> MakeAsyncRequest<TReturnType>(string url) where TReturnType : class
{
try
{
var request = (HttpWebRequest)WebRequest.Create(url);
var response = await request.GetResponseAsync();
var json = await Task.Run(() => ReadStreamFromResponse(response));
var deserializedObject = await Task.Run(() => JsonConvert.DeserializeObject<TReturnType>(json));
return deserializedObject;
}
catch (Exception)
{
// TODO: Handle exception here
throw;
}
}
private string ReadStreamFromResponse(WebResponse response)
{
using (var responseStream = response.GetResponseStream())
using (var sr = new StreamReader(responseStream))
{
return sr.ReadToEnd();
}
}
And usage would be like this:
var myData = await MakeAsyncRequest<IEnumerable<SomeObject>>(http://example.com/api/?getdata=all);
The issue I'm facing is that sometimes my web request fires and never returns, is there a proper way to handle that? Maybe playing with timers is an option but I don't like to go there.
Also I would be thankful if you could improve on my function, maybe there is a better way to do things and I just don't see it.

HttpClient in using statement causes Task cancelled

I created a FileResult : IHttpActionResult webapi return type for my api calls. The FileResult downloads a file from another url and then returns the stream to the client.
Initially my code had a using statement like below:
public async Task<HttpResponseMessage> ExecuteAsync(CancellationToken cancellationToken)
{
try
{
HttpResponseMessage response;
using (var httpClient = new HttpClient())
{
response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new System.Net.Http.StreamContent(
await httpClient.GetStreamAsync(this.filePath))
};
}
return response;
}
catch (WebException exception)
{...}
}
However this would intermittently cause a TaskCanceledException. I know that if the HttpClient is disposed before the asychronous call is finished the Task's state will change to canceled. However since I use an await in: Content = new System.Net.Http.StreamContent(await httpClient.GetStreamAsync(this.filePath)) that should prevent the HttpClient from being disposed off in the middle of the task completion.
Why does that task get canceled? It is not because of a timeout since this has happened on the smallest requests and doesn't always occur on large requests.
When I removed the using statement the code worked properly:
public async Task<HttpResponseMessage> ExecuteAsync(CancellationToken cancellationToken)
{
try
{
HttpResponseMessage response;
var httpClient = new HttpClient();
response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new System.Net.Http.StreamContent(
await httpClient.GetStreamAsync(this.filePath))
};
return response;
}
catch (WebException exception)
{...}
}
Any idea why the using caused the issue?

I know that if the HttpClient is disposed before the asychronous call is finished the Task's state will change to canceled. However since I use an await in: Content = new System.Net.Http.StreamContent(await httpClient.GetStreamAsync(this.filePath)) that should prevent the HttpClient from being disposed off in the middle of the task completion.
But what does that task do? It gets the stream. So, your code ends up with a Stream that may or may not be completely read when it closes the HttpClient.
HttpClient is specifically designed for reuse (and simultaneous use), so I recommend removing the using completely and moving the HttpClient declaration to a static class member. But if you want to close and reopen the clients, you should be able to get it working by reading the stream entirely into memory before closing the HttpClient.

I had a similar issue with Task Canceled exceptions. If you try catching AggregateException or having a catch all Exception block underneath your WebException, you may well find that you catch it, with one exception with the entry stating "A task was canceled"
I did some investigation and found that the AggregateException is quite misleading as described in various threads;
Setting HttpClient to a too short timeout crashes process
How can I tell when HttpClient has timed out?
Bug in httpclientgetasync should throw webexception not taskcanceledexception
I ended up changing my code to set an explicit timeout (where asyncTimeoutInMins is read from the app.config file);
string jsonResponse = string.Empty;
try
{
using (HttpClient httpClient = new HttpClient())
{
httpClient.BaseAddress = new Uri(Properties.Settings.Default.MyWebService);
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.Timeout = new TimeSpan(0, asyncTimeoutInMins, 0);
HttpResponseMessage response;
response = await httpClient.GetAsync("/myservice/resource");
// Check the response StatusCode
if (response.IsSuccessStatusCode)
{
// Read the content of the response into a string
jsonResponse = await response.Content.ReadAsStringAsync();
}
else if (response.StatusCode == HttpStatusCode.Forbidden)
{
jsonResponse = await response.Content.ReadAsStringAsync();
Logger.Instance.Warning(new HttpRequestException(string.Format("The response StatusCode was {0} - {1}", response.StatusCode.ToString(), jsonResponse)));
Environment.Exit((int)ExitCodes.Unauthorised);
}
else
{
jsonResponse = await response.Content.ReadAsStringAsync();
Logger.Instance.Warning(new HttpRequestException(string.Format("The response StatusCode was {0} - {1}", response.StatusCode.ToString(), jsonResponse)));
Environment.Exit((int)ExitCodes.ApplicationError);
}
}
}
catch (HttpRequestException reqEx)
{
Logger.Instance.Error(reqEx);
Console.WriteLine("HttpRequestException : {0}", reqEx.InnerException.Message);
Environment.Exit((int)ExitCodes.ApplicationError);
}
catch (Exception ex)
{
Logger.Instance.Error(ex);
throw;
}
return jsonResponse;

How to deal with HTML error codes and timeout from HttpClient()

I'm using HttpClient to connect to a server (see simplified code below). I cant figure out how I would respond to HTML error codes (e.g. 403) and timeouts so I can report what the result is.
When I encounter a 403 error code an error pop-up occurs in Visual Studio. But I can figure out how I convert this into try in the code. i.e. is the name of the exception present in the error pop-up?
using System.Net.Http;
HttpClient client = new HttpClient();
var response = client.PostAsync(dutMacUrl, null).Result;
var result = response.Content.ReadAsStringAsync().Result;

you can use async/await feature to simplify your code and avoid using Result.
for example
public async Task<string> Foo(string uri)
{
var client = new HttpClient();
try
{
var response = await client.PostAsync(uri, null);
}
catch (Exception ex)
{
//here you handle exceptions
}
// use this if (response.StatusCode != HttpStatusCode.OK) { do what you want }
// or this if (response.IsSuccessStatusCode) { do what you want }
var result = await response.Content.ReadAsStringAsync();
return result;
}

If you are using webAPI another option is to use IHttpActionResult
public object IHttpActionResult mymethod()
{
// instantiate your class or object
IEnumerable<yourClass> myobject = new IEnumerable<yourClass>();
// assuming you want to return a collection
try
{
// do stuff
// handle dto or map result back to object
return Ok(myobject)
}
catch(Exception e)
{
// return a bad request if the action fails
return BadRequest(e.Message)
}
}
This would allow you to make a call to your api endpoint and either return a successful response with the updated object or return a bad request if the endpoint fails.