No response when request takes 5 minutes or longer - c#

I have encountered a very strange issue - HttpClient's SendAsync never returns when request to the specific web server takes 5 minutes or longer.
This is a sample WebApi controller method that I try to get the response from
[HttpGet]
[Route("api/Entity/Ping")]
public async Task<HttpResponseMessage> Ping([FromUri] int time)
{
await Task.Delay(TimeSpan.FromMinutes(time));
var bytes = Enumerable.Repeat((byte)42, 100_000_000).ToArray();
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new ByteArrayContent(bytes);
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = "result.bin";
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
return response;
}
And this is a code for sending a request
using (var client = HttpClientFactory.Create(handler))
{
client.Timeout = TimeSpan.FromMinutes(10);
var url = "http://problem-server/WebApp/api/Entity/Ping?time=5";
var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri(url)
};
var response = await client.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead,
default);
var stream = await response.Content.ReadAsStreamAsync();
if (response.IsSuccessStatusCode)
return stream;
return default;
}
As you can see, everything is pretty simple and should work without issues. But it doesn't and SendAsync call just hangs forever (for 10 minutes).
In the same time it works when [time] parameter is less then 5.
Additionally, when you open the URL in a browser it successfully downloads a result.bin file after 5 minutes of processing, so method works.
Firstly I thought this is due to a deadlock.
But synchronous request using old WebRequest class to the same URL also hangs
var url = "http://problem-server/WebApp/api/Entity/Ping?time=5";
var request = WebRequest.Create(url);
request.Timeout = (int)TimeSpan.FromMinutes(10).TotalMilliseconds;
var response = request.GetResponse();
var stream = response.GetResponseStream();
if (stream != null)
return stream;
return default;
Next, I copied the WebApp folder to another server, lets call it ok-server.
Modified the URLs in http client and web request methods.
And, magically, everything works - the response is received after [time] minutes.
So the issue is with the problem-server.
But how to debug \ investigate it - IIS request tracing or logs "say" that the request has completed successfully after [time] minutes and the response was sent.
Both machines, problem-server and ok-server, have IIS 8.5 and Windows Server 2012 R2.
Web Api uses .NET Framework 4.5.
(I have also tried to use .NET Core 3.1 with ASP.NET Core hosted on IIS for the Web Api - the result is the same)
Can you help me find the reason for this issue?
Perhaps, I need to look into global machine configs or maybe network setting.
I am truly lost right now.
UPDATE
problem_server and ok_server are in different network segments.
problem_server IP is 192.168.114.100 and ok_server IP is 192.150.0.15.
To diagnose possible network misconfigurations I decided to send a request to the problem_server from the machine in its IP segment.
Here is the result when executing the test client from 192.168.114.125 machine
My workstation is yet in another IP segment - 192.135.9/24. Perhaps there are some router settings between 192.150.0/24 and 192.135.9/24 segments that allow the request to the ok_server to succeed.

I would really recommend that you not execute a five minute delay in your API controller. It will give you more grief than it's worth. For example, when IIS restarts your AppPool, it will wait up to 90 seconds for requests to process. During these autonomous restarts, this request will be aborted.
The problem server may have TCP KeepAlive set to Microsoft's recommended (but not default) value of 5 minutes. Because the HttpClient doesn't implement TCP keepalives by default, the problem server OS is likely disconnecting the TCP socket before the response is sent to the client because the client fails to respond to the keepalive being sent by the problem server OS.
You could adjust the TCP KeepAlive setting at the OS level on the problem server by editing the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters\ subkey as described here.
Alternatively, you can configure the client to support TCP keepalive before sending the request by configuring the ServicePoint. If there is a network device, such as a stateful firewall, between the client and server, a high frequency keep-alive setting may help keep the connection open.
var sp = ServicePointManager.FindServicePoint(new Uri(url));
sp.SetTcpKeepAlive(true, 6000, 3000);

Related

.Net Maui HttpClient GetStringAsync() slow only on physical device

I am using HttpClient service for simple GetStringAsync() request. While running the app on android emulator, avarage response time is less than ~0.5sec. The problem appears on a physical devices where response is 3 to even 9 seconds.
Because at the same moment, the app have another running service in background (simple while (true) loop downloading other data every 5 sec) also using HttpClient, I've tried disabling it but after that, response time is only ~0.2 secs faster (I also have no idea why it have any impact on other requests since I am reusing the same HttpClient, not creating new one).
I also tried disabling firewall on my host computer but didn't
worked. Using HttpClientFactory is not solving problem either.
I wonder if there is any different way to download string from url for .Net maui or am I using it in wrong way or should I add any options to my HttpClient?
HttpClient:
public static HttpClient _httpclient;
and
_hhtpclient = new HttpClient();
Request:
var jsoN = await _httpclient.GetStringAsync(#"http://192.168.0.209:8032/getStatus?Id=" +id);
if (jsoN != null)
{
var status = JsonConvert.DeserializeObject<Status>(jsoN);
return status ;
}
else
{
return null;
}
Devices i tested on:
Samsung a73 (Android 12.0)
Huwawei Nova Y90 (EMUI 12)
My internet speed (tested on host computer):
DOWNLOAD 295.03 Mb/s | UPLOAD 30.70 Mb/s

HttpClient timeouts without reason

In my .net Application I am using the Microsoft's HttpClient to send Requests to a server. In my team we have a problem that several people get a timeout on each single request without any knowledgable reason.
Following code is used to send the request:
var _httpClient = new HttpClient();
// Here are some values that are identical on each system.
var values = new Dictionary<string, string>();
var content = new FormUrlEncodedContent(values);
var response = await _httpClient.PostAsync("http://myservice.com/endpoint", content);
Is there any known issue or reason that the httpClient runs into a timeout on specific systems?
Important fact: Calling the same endpoint with the same data via Postman works on every collegues system.
EDIT:
The error message is:
"The request was canceled due to the configured HttpClient.Timeout of
100 seconds elapsing."
When navigating through the inner Exceptions the native error code is 995 and SocketErrorCode is OperationAborted
What I expect:
After max. 5 Secs there should a 200 Response with Json Data.
Ok the problem was that c# couldn't resolve the DNS name on particular systems and Postman could...

HttpClient requests not working for a particular domain in Azure

I have an Azure function that sends a request to a URL and sends back the response. This function kept failing with timeout error for URLs from a particular domain (confidential).
To debug this, I created a very minimal Azure function:
var content = string.Empty;
using (var response = await _httpClient.GetAsync(url))
{
response.EnsureSuccessStatusCode();
content = await response.Content.ReadAsByteArrayAsync();
}
return new OkObjectResult(content);
This code works fine in local. When I try using the deployed Azure function, it works for all the other domains I tried (ex: https://google.com) but it hits request timeout error for a particular domain after trying for about 90 seconds. The error happens at this particular line: _httpClient.GetAsync(url). Again, it works fine for this (confidential) domain in local.
I have tried deploying the Azure function to two completely different Azure service plans and regions. Same result. It doesn't work for URLs from the required domain. Works for URLs of other domains.
Error:
System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
Update (solution):
I tried sending a request from Postman, copied the code from there for C# and deployed it to the Azure function and it is now working for the problematic domain. Something like below:
var client = new RestClient(url);
client.Timeout = -1;
var request = new RestRequest(Method.GET);
IRestResponse response = client.Execute(request);
The key here is client.Timeout = -1, which seems to have fixed the problem.
Now, in my original code, I tried setting HttpClient's timeout to Timeout.InfiniteTimeSpan both in Startup configuration as well as at individual request level but it did not work.
services.AddHttpClient("AzureTestClient", options =>
{
options.Timeout = Timeout.InfiniteTimeSpan;
});
Am I setting the timeout wrong in the HttpClient solution?
If you are using a Consumption plan then maybe the confidential URL need to whitelist the whole Azure Data center. You can follow the guide here or consider upgrading the Consumption plan to a premium one and have a dedicated linked VNET.
Maybe your local machine is already linked to the domain/whitelisted so azure function operates from different range.
Another reason maybe the URL returns a different HttpStatusCode that is't Successful range (200-299) so it fails with "EnsureSuccessStatusCode" in the old code?
Normally for the http code initialization, I did something like that:
public void Configure(IWebJobsBuilder builder)
{
builder.Services.AddHttpClient("AzureTestClient",
options => { options.Timeout = Timeout.InfiniteTimeSpan; });
}
Then when I want to use it, I do like that in any other function and it worked:
var client = clientFactory.CreateClient("AzureTestClient");

HttpClient PostAsync does not return

I've seen a lot of question about this, and all points to me using ConfigureAwait(false), but even after doing so, it still doesn't returned any response. When I run the debugger, the code stops at the PostAsync and does not continue with my code. Am I doing something wrong here? Does it have to do with me calling an API via HTTPS?
Here's the code:
public async static Task<PaymentModel> AddAsync(Card card)
{
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes(string.Format("{0}:", "hidden"))));
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
var cardJson = JsonConvert.SerializeObject(card);
var postRequest = new StringContent(cardJson, Encoding.UTF8, "application/json");
var request = await client.PostAsync(new Uri("https://sample-3rd-party-api/api/endpoint/here"), postRequest).ConfigureAwait(false);
var content = await request.Content.ReadAsStringAsync().ConfigureAwait(false);
}
EDIT:
In response to the comments below, the code is contained from a method AddAsync(Card card) called from a button click with a handler:
public async void OnExecute(object sender, EventArgs args)
{
//some code here
payment = await PaymentModel.AddAsync(card).ConfigureAwait(false);
}
EDIT 2:
I tried pinging the API, but it returns a request timed out, but when I tried it using Postman, it's doing fine (the API is just a Sandbox which is open for all, so it's okay to share this):
EDIT 3:
I think the problem lies with where I don't have an SSL certificate to access the API. I have a PHP server that connects to the same API and I have to set SSL_VERIFYPEER to false just so I can access it (don't worry, I added a cacert now so its on true again). Can the same issue be happening here? If so, what can I do to create a valid certificate for my xamarin forms app
You can use this
var json = JsonConvert.SerializeObject(card);
using (var client = new HttpClient())
{
var t = await client.PostAsJsonAsync("https://sample-3rd-party-api/api/endpoint/here", json);
Response R =JsonConvert.DeserializeObject<Response>((JsonConvert.DeserializeObject(t.Content.ReadAsStringAsync().Result.ToString())).ToString());
}
What's most likely happening here is your OnExecute method has a return type of void instead of Task which prevents the UI thread from being able to await it. Try either changing that return type to Task or creating a new UI thread to perform this work. I wouldn't worry about the ping timing out as long as Postman works. Many public web servers disable their ping response.
Does it have to do with me calling an API via HTTPS?
As you are remaining in the same network and calling the same API from POSTMAN and .NET HTTP Client and only getting success with POSTMAN.So this issue gets cancelled.
Next
tried pinging the API, but it returns a request timed out
This is answered on top of mine.
Can you Please try setting the timeout option for HTTPClient while initializing.
client.Timeout = TimeSpan.FromSeconds(10);
and if still Problem persists please setup Fiddler and compare both the req sent from POstman and .NET client
So I think the problem is resolved now since I'm able to receive content from the request, what I did was simply follow the docs here: https://learn.microsoft.com/en-us/xamarin/cross-platform/app-fundamentals/transport-layer-security?tabs=windows
It looks like my settings are outdated in platform level.
Update the HttpClient implementation and SSL/TLS implementation
options to enable TLS 1.2 security.
Update the HttpClient Implementation option to enable TSL 1.2
security. (NSUrlSession (iOS 7.0+)
I was having the same issue and below trick fixed the issue.
Change your var request = await client.PostAsync(...); as below
var task = client.PostAsync(new Uri("https://sample-3rd-party-api/api/endpoint/here"), postRequest);
var request = task.GetAwaiter().GetResult();

Why is this HttpWebRequest slow in C#

I am using this code to perform a simple REST request. (The code mostly comes from this q: How to post JSON to the server?).
Why is it so slow? I'm using VS 2013 and it takes about 15 secs on first try and then about 4 secs. on subsequent tries, yet in another language (Delphi) I can make a http request and it takes about 1 sec consistently.
var request = (HttpWebRequest)WebRequest.Create("http://jsonplaceholder.typicode.com/posts");
request.ContentType = "application/json";
request.Method = "POST";
request.ServicePoint.Expect100Continue = false;
using (var streamWriter = new StreamWriter(request.GetRequestStream()))
{
string json = new JavaScriptSerializer().Serialize(new
{
title = "foo",
body = "bar",
userId = "1"
});
streamWriter.Write(json);
}
var response = (HttpWebResponse)request.GetResponse();
using (var streamReader = new StreamReader(response.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
textBox1.Text = result;
}
P.S. You can test this code for yourself, it is simply using a test REST server from the internet at above url.
What do you mean by first try? It means the first try after I leave the computer for a while
Before reaching the server, there is a process of finding the IP address of the server. This process is called Dns Resolution.
First time, your application has to go through the process of Dns Resolution in order to find the IP address. Once you resolved the IP address, the IP address will be cached in the local machine.
So, further calls doesn't go through the process of Dns Resolution; it can use the cached IP. After a while, the cache will be dropped and again you'll hit the DNS server for resolving the IP address.
This is the only explanation I can come up for the delay you're noticing. Whenever you're noticing a delay, that probably means that you're hitting the Dns Server just because it is either the first time or cache is expired.
Why it is faster in other environment(Delphi)?
I'm sorry I can't come up with a good reason for this.

Categories

Resources