I am using this code to perform a simple REST request. (The code mostly comes from this q: How to post JSON to the server?).
Why is it so slow? I'm using VS 2013 and it takes about 15 secs on first try and then about 4 secs. on subsequent tries, yet in another language (Delphi) I can make a http request and it takes about 1 sec consistently.
var request = (HttpWebRequest)WebRequest.Create("http://jsonplaceholder.typicode.com/posts");
request.ContentType = "application/json";
request.Method = "POST";
request.ServicePoint.Expect100Continue = false;
using (var streamWriter = new StreamWriter(request.GetRequestStream()))
{
string json = new JavaScriptSerializer().Serialize(new
{
title = "foo",
body = "bar",
userId = "1"
});
streamWriter.Write(json);
}
var response = (HttpWebResponse)request.GetResponse();
using (var streamReader = new StreamReader(response.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
textBox1.Text = result;
}
P.S. You can test this code for yourself, it is simply using a test REST server from the internet at above url.
What do you mean by first try? It means the first try after I leave the computer for a while
Before reaching the server, there is a process of finding the IP address of the server. This process is called Dns Resolution.
First time, your application has to go through the process of Dns Resolution in order to find the IP address. Once you resolved the IP address, the IP address will be cached in the local machine.
So, further calls doesn't go through the process of Dns Resolution; it can use the cached IP. After a while, the cache will be dropped and again you'll hit the DNS server for resolving the IP address.
This is the only explanation I can come up for the delay you're noticing. Whenever you're noticing a delay, that probably means that you're hitting the Dns Server just because it is either the first time or cache is expired.
Why it is faster in other environment(Delphi)?
I'm sorry I can't come up with a good reason for this.
Related
I have encountered a very strange issue - HttpClient's SendAsync never returns when request to the specific web server takes 5 minutes or longer.
This is a sample WebApi controller method that I try to get the response from
[HttpGet]
[Route("api/Entity/Ping")]
public async Task<HttpResponseMessage> Ping([FromUri] int time)
{
await Task.Delay(TimeSpan.FromMinutes(time));
var bytes = Enumerable.Repeat((byte)42, 100_000_000).ToArray();
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new ByteArrayContent(bytes);
response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = "result.bin";
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
return response;
}
And this is a code for sending a request
using (var client = HttpClientFactory.Create(handler))
{
client.Timeout = TimeSpan.FromMinutes(10);
var url = "http://problem-server/WebApp/api/Entity/Ping?time=5";
var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri(url)
};
var response = await client.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead,
default);
var stream = await response.Content.ReadAsStreamAsync();
if (response.IsSuccessStatusCode)
return stream;
return default;
}
As you can see, everything is pretty simple and should work without issues. But it doesn't and SendAsync call just hangs forever (for 10 minutes).
In the same time it works when [time] parameter is less then 5.
Additionally, when you open the URL in a browser it successfully downloads a result.bin file after 5 minutes of processing, so method works.
Firstly I thought this is due to a deadlock.
But synchronous request using old WebRequest class to the same URL also hangs
var url = "http://problem-server/WebApp/api/Entity/Ping?time=5";
var request = WebRequest.Create(url);
request.Timeout = (int)TimeSpan.FromMinutes(10).TotalMilliseconds;
var response = request.GetResponse();
var stream = response.GetResponseStream();
if (stream != null)
return stream;
return default;
Next, I copied the WebApp folder to another server, lets call it ok-server.
Modified the URLs in http client and web request methods.
And, magically, everything works - the response is received after [time] minutes.
So the issue is with the problem-server.
But how to debug \ investigate it - IIS request tracing or logs "say" that the request has completed successfully after [time] minutes and the response was sent.
Both machines, problem-server and ok-server, have IIS 8.5 and Windows Server 2012 R2.
Web Api uses .NET Framework 4.5.
(I have also tried to use .NET Core 3.1 with ASP.NET Core hosted on IIS for the Web Api - the result is the same)
Can you help me find the reason for this issue?
Perhaps, I need to look into global machine configs or maybe network setting.
I am truly lost right now.
UPDATE
problem_server and ok_server are in different network segments.
problem_server IP is 192.168.114.100 and ok_server IP is 192.150.0.15.
To diagnose possible network misconfigurations I decided to send a request to the problem_server from the machine in its IP segment.
Here is the result when executing the test client from 192.168.114.125 machine
My workstation is yet in another IP segment - 192.135.9/24. Perhaps there are some router settings between 192.150.0/24 and 192.135.9/24 segments that allow the request to the ok_server to succeed.
I would really recommend that you not execute a five minute delay in your API controller. It will give you more grief than it's worth. For example, when IIS restarts your AppPool, it will wait up to 90 seconds for requests to process. During these autonomous restarts, this request will be aborted.
The problem server may have TCP KeepAlive set to Microsoft's recommended (but not default) value of 5 minutes. Because the HttpClient doesn't implement TCP keepalives by default, the problem server OS is likely disconnecting the TCP socket before the response is sent to the client because the client fails to respond to the keepalive being sent by the problem server OS.
You could adjust the TCP KeepAlive setting at the OS level on the problem server by editing the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters\ subkey as described here.
Alternatively, you can configure the client to support TCP keepalive before sending the request by configuring the ServicePoint. If there is a network device, such as a stateful firewall, between the client and server, a high frequency keep-alive setting may help keep the connection open.
var sp = ServicePointManager.FindServicePoint(new Uri(url));
sp.SetTcpKeepAlive(true, 6000, 3000);
So the goal of this service is to determine if the machine is on a known network such as different office locations or not based on the public IP. I've created a few lists of known IP addresses to compare against. If the public IP is not known, I trigger a process to start an OpenVPN connection to my server.
I have been utilizing http://ipv4.icanhazip.com for retrieving the public IP.
The current behavior is, during the first Timer interval the web request returns the correct public IP. The VPN connection is then called and shortly establishes. In CMD I am able to run curl http://ipv4.icanhazip.com and confirm the VPN server IP as well the IPv4 route table has in fact been updated. All my web browser traffic is being routed correctly.
However the service requests never return the new IP during the 15 second intervals.
Here is my WebRequest function
public static string ReturnPublicIP()
{
string url = "http://ipv4.icanhazip.com";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.AutomaticDecompression = DecompressionMethods.GZip;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (Stream stream = response.GetResponseStream())
using (StreamReader reader = new StreamReader(stream))
{
return reader.ReadToEnd().Trim((char)0x0a, (char)0x0d);
}
}
Any idea's on what I can do to remedy this? I would imagine the networking\process thread would be aware of the updated route table and flow the traffic accordingly..
Thanks in advance for any help!
I would like to know how Reusing HttpWebRequests works to avoid SSL handshake process everytime.
I use the keep alive headr in the request and the first handshake is successfull but i would like to reuse the request in order to avoid future handshakes against the same certificate.
Think is i dont know if i had to reuse the HttpWebRequest object instance or even if i create a new request object it will use the same connection since the keep alive is already on place and working.
Should i store the existing request object lets say at class level and reuse it? or i can safely dispose the object and next time i create a request it will be under the effect of the keep alive connection?
I am asking this cause i need to lower the timings in an application, and worst part is always ssl handshake, that can take over 3seconds in a phone with medium signal from carrier.
I am using C# to develop.
I have tried to look for this kind of information but all i read over internet is how to set up the SSL Server and enabling certain settings but not how to make the client work with these features.
EDIT: FINDINGS RESULTS
I created a sample program in .NET C# whith the following code:
Stopwatch sw = new Stopwatch();
sw.Start();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(#"https:\\www.gmail.com"));
request.KeepAlive = true;
request.Method = "GET";
request.ContentType = "application/json";
request.ContentLength = 0;
request.ConnectionGroupName = "test";
//request.UnsafeAuthenticatedConnectionSharing = true;
//request.PreAuthenticate = true;
var response = request.GetResponse();
//response.Close();
request.Abort();
sw.Stop();
listBox1.Items.Add("Connection in : " + sw.Elapsed.ToString());
sw.Reset();
sw.Start();
HttpWebRequest request2 = (HttpWebRequest)WebRequest.Create(new Uri(#"https:\\www.gmail.com"));
request2.KeepAlive = true;
request2.Method = "GET";
//request2.UnsafeAuthenticatedConnectionSharing = true;
//request2.PreAuthenticate = true;
request2.ContentType = "application/json";
request2.ContentLength = 0;
request2.ConnectionGroupName = "test";
var response2 = request2.GetResponse();
//response2.Close();
request2.Abort();
sw.Stop();
listBox1.Items.Add("Connection 2 in : " + sw.Elapsed.ToString());
Results was that the first connection triggered the CertificatevalidationCallback 3 times (one for each certificate) and then the second connection only once, but when i CLOSED THE RESPONSE before performing the next request, no callback was triggered.
I suppose that keeping a response open keeps the socket open and thats why the partial handshake takes place (not the full certificate chain).
Sorry if I sound kind of noob in this matter, SSL and timings was coded by a work mate and the code was not clear to follow. But i think i have the answer. Thanks Poupou for your tremendous help
This is already built-in the the SSL/TLS stack shipped with Xamarin.iOS (i.e. at a lower level than HttpWebRequest). There's nothing to set up to enabled this. In fact you would need extra code if you wanted to disable it.
If the server supports it then subsequent handshake will already faster because a Session ID cache will be used (see TLS 1.0 RFC page 30).
However the server does not have to honor the session id (given to it). In such case a full handshake will need be done again. IOW you cannot force this from the client (only offer).
You can verify this using a network analyzer, e.g. wireshark, by looking at the exchanges (and comparing them to the RFC).
I'm sending an HTTPWebRequest to a 3rd party with the code below. The response takes between 2 and 22 seconds to come back. The 3rd party claims that once they receive it, they are sending back a response immediately, and that none of their other partners are reporting any delays (but I'm not sure I believe them -- they've lied before).
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.example.com");
request.Timeout = 38000;
request.Method = "POST";
request.ContentType = "text/xml";
StreamWriter streamOut = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(XMLToSend); // XMLToSend is just a string that is maybe 1kb in size
streamOut.Close();
HttpWebResponse resp = null;
resp = (HttpWebResponse)request.GetResponse(); // This line takes between 2 and 22 seconds to return.
StreamReader responseReader = new StreamReader(resp.GetResponseStream(), Encoding.UTF8);
Response = responseReader.ReadToEnd(); // Response is merely a string to hold the response.
Is there any reason that the code above would just...pause? The code is running in a very solid hosting provider (Rackspace Intensive Segment), and the machine it is on isn't being used for anything else. I'm merely testing some code that we are about to put into production. So, it's not that the machine is taxed, and given that it is Rackspace and we are paying a boatload, I doubt it is their network either.
I'm just trying to make sure that my code is as fast as possible, and that I'm not doing anything stupid, because in a few weeks, this code will be ramped up to run 20,000 requests to this 3rd part every hour.
Try doing a flush before you close.
streamOut.Flush();
streamOut.Close();
Also download microsoft network monitor to see for certain if the hold up is you or them, you can download it here...
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=983b941d-06cb-4658-b7f6-3088333d062f&displaylang=en
There is a few things that I would do:
I would profile the code above and get some definitive timings.
Implement the using statements in order to dispose of resources correctly.
Write the code in an async style there's going to be an awful lot of IO wait once its ramped.
Can you hit the URL in a regular ole browser? How fast is that?
Can you hit other URL's (not your partner's) in this code? How fast is that?
It is entirely possible you're getting bitten by the 'latency bug' where even an instant response from your partner results in unpredictable delays from your perspective.
Another thought: I noticed the https in your URL. Is it any faster with http?
I use C#.
The first time I use WebRequest GetRequestStream() in my code, it takes up to 20 seconds. After that it takes it takes under 1 second.
Below is my code. The row "this.requestStream = httpRequest.GetRequestStream()" is causing the delay.
StringBuilder postData = new StringBuilder(100);
postData.Append("param=");
postData.Append("test");
byte[] dataArray = Encoding.UTF8.GetBytes(postData.ToString());
this.httpRequest = (HttpWebRequest)WebRequest.Create("http://myurl.com");
httpRequest.Method = "POST";
httpRequest.ContentType = "application/x-www-form-urlencoded";
httpRequest.ContentLength = dataArray.Length;
this.requestStream = httpRequest.GetRequestStream();
using (requestStream)
requestStream.Write(dataArray, 0, dataArray.Length);
this.webResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream responseStream = webResponse.GetResponseStream();
StreamReader responseReader = new System.IO.StreamReader(responseStream, Encoding.UTF8);
String responseString = responseReader.ReadToEnd();
How can I see what causes this? (for instance: DNS lookup? Server not responding?)
Thanks and regards, Koen
You could also try to set the .Proxy = null. Sometimes it tries to autodetect a proxy which takes up time.
That sounds like your application is pre-compiling when you first hit it. This is how .net works.
Here is a way to speed up your web app. link text
It's actually the framework for HTML operations doing startup network proxy checking to setup the property HttpWebRequest.DefaultWebProxy.
In my application as part of the startup actions I create a fully formed request as a back ground task to get this overhead out of the way.
HttpWebRequest web = (HttpWebRequest)WebRequest.Create(m_ServletURL);
web.UserAgent = "Mozilla/4.0 (Windows 7 6.1) Java/1.6.0_26";
Setting the UserAgent field in my case is triggers the startup overhead.
I had the same issue but .proxy = null didn't solve it for me. Depending on the network structure the problem might be connected to IPv6. The first request nearly took exactly 21sec each time the application run. Therefore I argue it must be a timeout value. If this value is reached the fallback solution IPv4 is used (for subsequent calls as well). Forcing the use of IPv4 in the first place solved the issue for me!
HttpWebRequest request = WebRequest.Create("http://myurl.com") as HttpWebRequest;
request.ServicePoint.BindIPEndPointDelegate = (servicePount, remoteEndPoint, retryCount) =>
{
if (remoteEndPoint.AddressFamily == System.Net.Sockets.AddressFamily.InterNetwork)
{
return new IPEndPoint(IPAddress.Any, 0);
}
throw new System.InvalidOperationException("No IPv4 address found.");
};
One problem may be the fact that .NET, by default, only allows 2 connections at a time.
You can increase the number of simultaneous connections with:
ServicePointManager.DefaultConnectionLimit = newConnectionLimit;
We leave the determination of the optimal value to the user.