My problem is when using WebClient or WebRequest/HttpRequest that the first request often takes a bit more than 40 seconds and all subsequent ones only around 250 milliseconds regardless of size (if it is serveral k characters or only 40). Im only trying to get JSON data from some endpoint.
with WebClient I use wc.DownloadString("some address");
with WeRequest I use (HttpWebRequest) WebRequest.Create("some address") and webRequest.GetResponse(); with according stream handling afterwards
I already looked around for a Solution, and found:
WebRequest.DefaultWebProxy = GlobalProxySelection.GetEmptyWebProxy();
WebRequest.DefaultWebProxy = null;
webclient.Proxy = null;
webclient.Proxy = GlobalProxySelection.GetEmptyWebProxy();
The above mentionen options are NOT reliable working for me.
Are there any other ways to reliable fix this huge delay for first Request? I would prefere a solution without external libs if possible.
As a note I use .Net Framework 4.5.
Related
I would like to know how Reusing HttpWebRequests works to avoid SSL handshake process everytime.
I use the keep alive headr in the request and the first handshake is successfull but i would like to reuse the request in order to avoid future handshakes against the same certificate.
Think is i dont know if i had to reuse the HttpWebRequest object instance or even if i create a new request object it will use the same connection since the keep alive is already on place and working.
Should i store the existing request object lets say at class level and reuse it? or i can safely dispose the object and next time i create a request it will be under the effect of the keep alive connection?
I am asking this cause i need to lower the timings in an application, and worst part is always ssl handshake, that can take over 3seconds in a phone with medium signal from carrier.
I am using C# to develop.
I have tried to look for this kind of information but all i read over internet is how to set up the SSL Server and enabling certain settings but not how to make the client work with these features.
EDIT: FINDINGS RESULTS
I created a sample program in .NET C# whith the following code:
Stopwatch sw = new Stopwatch();
sw.Start();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(#"https:\\www.gmail.com"));
request.KeepAlive = true;
request.Method = "GET";
request.ContentType = "application/json";
request.ContentLength = 0;
request.ConnectionGroupName = "test";
//request.UnsafeAuthenticatedConnectionSharing = true;
//request.PreAuthenticate = true;
var response = request.GetResponse();
//response.Close();
request.Abort();
sw.Stop();
listBox1.Items.Add("Connection in : " + sw.Elapsed.ToString());
sw.Reset();
sw.Start();
HttpWebRequest request2 = (HttpWebRequest)WebRequest.Create(new Uri(#"https:\\www.gmail.com"));
request2.KeepAlive = true;
request2.Method = "GET";
//request2.UnsafeAuthenticatedConnectionSharing = true;
//request2.PreAuthenticate = true;
request2.ContentType = "application/json";
request2.ContentLength = 0;
request2.ConnectionGroupName = "test";
var response2 = request2.GetResponse();
//response2.Close();
request2.Abort();
sw.Stop();
listBox1.Items.Add("Connection 2 in : " + sw.Elapsed.ToString());
Results was that the first connection triggered the CertificatevalidationCallback 3 times (one for each certificate) and then the second connection only once, but when i CLOSED THE RESPONSE before performing the next request, no callback was triggered.
I suppose that keeping a response open keeps the socket open and thats why the partial handshake takes place (not the full certificate chain).
Sorry if I sound kind of noob in this matter, SSL and timings was coded by a work mate and the code was not clear to follow. But i think i have the answer. Thanks Poupou for your tremendous help
This is already built-in the the SSL/TLS stack shipped with Xamarin.iOS (i.e. at a lower level than HttpWebRequest). There's nothing to set up to enabled this. In fact you would need extra code if you wanted to disable it.
If the server supports it then subsequent handshake will already faster because a Session ID cache will be used (see TLS 1.0 RFC page 30).
However the server does not have to honor the session id (given to it). In such case a full handshake will need be done again. IOW you cannot force this from the client (only offer).
You can verify this using a network analyzer, e.g. wireshark, by looking at the exchanges (and comparing them to the RFC).
In C# on .net 4.0 I'm trying to get the header date/time of a Internet web site.
My goal is to validate a local systems time (within seconds) of Internet time by using HTTP rather then SNTP. I'm a SNTP fan but it won't do in this scenario. I found this concept of using HTTP headers for time called "HTP" and want to replicate it in C#.
Tried to use HttpWebRequest.Headers collection using MSDN example on the page, which doesn't return me the Date (or much else).
If HttpWebRequest.Headers is a good way to go about getting this value, why can't I see Date in this result? Is there a better way?
var myHttpWebRequest=(HttpWebRequest)WebRequest.Create("http://www.microsoft.com");
myHttpWebRequest.GetResponse();
Console.WriteLine("\nThe HttpHeaders are \n\n\tName\t\tValue\n{0}",
myHttpWebRequest.Headers);
You seem to be reading the request headers, instead of reading the response headers:
var myHttpWebRequest = (HttpWebRequest)WebRequest.Create("http://www.microsoft.com");
var response = myHttpWebRequest.GetResponse();
Console.WriteLine("\nThe HttpHeaders are \n\n\tName\t\tValue\n{0}", response.Headers);
It seems to be occurring only one machine and none of the other machines.
HttpWebRequest myRequest =(HttpWebRequest)WebRequest.Create("https://connect.zystemsgo.com/auto/");
myRequest.Method = "GET";
SetCertificatePolicy();
Application.DoEvents();
WebResponse myResponse = myRequest.GetResponse();
StreamReader sr = new StreamReader(myResponse.GetResponseStream(),System.Text.Encoding.UTF8);
string result = sr.ReadToEnd();
I tried searching other 400 request errors, but it is not clear. How do I go about debugging this?
HTTP Error 400 means Bad Request. This is being returned by the server.
Usually, when I'm debugging HTTP requests, I use Fiddler to monitor the requests and responses and find out what's going on. It never fails.
(Not really an answer, but too big for comment)
For what it's worth, I ran the following Python code (too lazy to spin up C# :), and it worked fine:
import httplib
conn = httplib.HTTPSConnection('connect.zystemsgo.com')
conn.request('GET', '/auto/')
resp = conn.getresponse()
data = resp.read()
print data # expected ouput, just like visiting in a browser
print resp.status # 200
Are you sure you are showing us the URL that is actually failing, or is your code a more general example?
Perhaps the server certificate is not installed on that machine? I wouldn't expect a HTTP 400 in that case, but it's the only thing I can think of so far...
it is a bad request error .Are there no parameters in the request?
Can you post the response message,it will give some idea of what is going wrong.
The code that i supplied in the comment above works.
WebClient webClient = new WebClient();
webClient.DownloadFile("Your complete url for the file", #"c:\myfile.txt");
you need to have permission to write in the directory of your choice.
You could also try and use the async download if you want.I am not getting why it would not work on a certain machine.
I'm sending an HTTPWebRequest to a 3rd party with the code below. The response takes between 2 and 22 seconds to come back. The 3rd party claims that once they receive it, they are sending back a response immediately, and that none of their other partners are reporting any delays (but I'm not sure I believe them -- they've lied before).
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.example.com");
request.Timeout = 38000;
request.Method = "POST";
request.ContentType = "text/xml";
StreamWriter streamOut = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.ASCII);
streamOut.Write(XMLToSend); // XMLToSend is just a string that is maybe 1kb in size
streamOut.Close();
HttpWebResponse resp = null;
resp = (HttpWebResponse)request.GetResponse(); // This line takes between 2 and 22 seconds to return.
StreamReader responseReader = new StreamReader(resp.GetResponseStream(), Encoding.UTF8);
Response = responseReader.ReadToEnd(); // Response is merely a string to hold the response.
Is there any reason that the code above would just...pause? The code is running in a very solid hosting provider (Rackspace Intensive Segment), and the machine it is on isn't being used for anything else. I'm merely testing some code that we are about to put into production. So, it's not that the machine is taxed, and given that it is Rackspace and we are paying a boatload, I doubt it is their network either.
I'm just trying to make sure that my code is as fast as possible, and that I'm not doing anything stupid, because in a few weeks, this code will be ramped up to run 20,000 requests to this 3rd part every hour.
Try doing a flush before you close.
streamOut.Flush();
streamOut.Close();
Also download microsoft network monitor to see for certain if the hold up is you or them, you can download it here...
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=983b941d-06cb-4658-b7f6-3088333d062f&displaylang=en
There is a few things that I would do:
I would profile the code above and get some definitive timings.
Implement the using statements in order to dispose of resources correctly.
Write the code in an async style there's going to be an awful lot of IO wait once its ramped.
Can you hit the URL in a regular ole browser? How fast is that?
Can you hit other URL's (not your partner's) in this code? How fast is that?
It is entirely possible you're getting bitten by the 'latency bug' where even an instant response from your partner results in unpredictable delays from your perspective.
Another thought: I noticed the https in your URL. Is it any faster with http?
I use C#.
The first time I use WebRequest GetRequestStream() in my code, it takes up to 20 seconds. After that it takes it takes under 1 second.
Below is my code. The row "this.requestStream = httpRequest.GetRequestStream()" is causing the delay.
StringBuilder postData = new StringBuilder(100);
postData.Append("param=");
postData.Append("test");
byte[] dataArray = Encoding.UTF8.GetBytes(postData.ToString());
this.httpRequest = (HttpWebRequest)WebRequest.Create("http://myurl.com");
httpRequest.Method = "POST";
httpRequest.ContentType = "application/x-www-form-urlencoded";
httpRequest.ContentLength = dataArray.Length;
this.requestStream = httpRequest.GetRequestStream();
using (requestStream)
requestStream.Write(dataArray, 0, dataArray.Length);
this.webResponse = (HttpWebResponse)httpRequest.GetResponse();
Stream responseStream = webResponse.GetResponseStream();
StreamReader responseReader = new System.IO.StreamReader(responseStream, Encoding.UTF8);
String responseString = responseReader.ReadToEnd();
How can I see what causes this? (for instance: DNS lookup? Server not responding?)
Thanks and regards, Koen
You could also try to set the .Proxy = null. Sometimes it tries to autodetect a proxy which takes up time.
That sounds like your application is pre-compiling when you first hit it. This is how .net works.
Here is a way to speed up your web app. link text
It's actually the framework for HTML operations doing startup network proxy checking to setup the property HttpWebRequest.DefaultWebProxy.
In my application as part of the startup actions I create a fully formed request as a back ground task to get this overhead out of the way.
HttpWebRequest web = (HttpWebRequest)WebRequest.Create(m_ServletURL);
web.UserAgent = "Mozilla/4.0 (Windows 7 6.1) Java/1.6.0_26";
Setting the UserAgent field in my case is triggers the startup overhead.
I had the same issue but .proxy = null didn't solve it for me. Depending on the network structure the problem might be connected to IPv6. The first request nearly took exactly 21sec each time the application run. Therefore I argue it must be a timeout value. If this value is reached the fallback solution IPv4 is used (for subsequent calls as well). Forcing the use of IPv4 in the first place solved the issue for me!
HttpWebRequest request = WebRequest.Create("http://myurl.com") as HttpWebRequest;
request.ServicePoint.BindIPEndPointDelegate = (servicePount, remoteEndPoint, retryCount) =>
{
if (remoteEndPoint.AddressFamily == System.Net.Sockets.AddressFamily.InterNetwork)
{
return new IPEndPoint(IPAddress.Any, 0);
}
throw new System.InvalidOperationException("No IPv4 address found.");
};
One problem may be the fact that .NET, by default, only allows 2 connections at a time.
You can increase the number of simultaneous connections with:
ServicePointManager.DefaultConnectionLimit = newConnectionLimit;
We leave the determination of the optimal value to the user.