I have some logic issues with HttpWebRequest class.
I using HttpWebRequest class from System.Net namespace, and when I doing this:
while(true)
{
HttpWebRequest request = WebRequest.Create("http://somesite.com/") as HttpWebRequest;
HttpWebResponse responce = request.GetResponse() as HttpWebResponse;
}
I get response one by one with overage one sec interval, but I think my internet connection can work faster because the received data is very small. Then I try this:
while(true)
{
HttpWebRequest request = WebRequest.Create("http://google.com/") as HttpWebRequest;
HttpWebResponse response = request.BeginGetResponse(EndReceive, obj);
}
internal void EndReceive(IAsyncResult ar)
{
obj.Response = obj.Request.EndGetResponse(ar) as HttpWebResponse;
}
And I get very small speed increasing, something about 10-30%, but I use async request, I send to server 5 request instead of one, why speed wasn't increase by more than 100%?
It's ok if server can't handle more than one request from one ip at the same time... But when I run 10 console app with code:
void SendRequest()
{
HttpWebRequest request = WebRequest.Create("http://google.com/") as HttpWebRequest;
HttpWebResponse responce = request.BeginGetResponse(EndReceive, obj);
}
void EndReceive(IAsyncResult ar)
{
obj.Response = obj.Request.EndGetResponse(ar) as HttpWebResponse;
}
I get speed increasing for like 4-8 times, is problem with HttpWebRequest class? And why i cant get such speed with one application but many async requests?
I strongly suspect you're basically being bounded by the built-in connection pool, which will limit the number of requests a single process (or possibly AppDomain) makes to a given host. If you want to change the number of concurrent requests you can make to a single host, use the <connectionManagement> element in your app.config.
As an aside, you should have a using statement when you use the response, otherwise you're not disposing of it properly... which can cause horrible problems with the connection pool in itself.
Related
I have a custom WebUploadTraceListener : TraceListener that I use to send HTTP (and eventually HTTPS) POST data to a web service that writes it to a database.
I have tested doing this with both WebClient and HttpWebRequest and empirically I'm seeing better performance with the latter.
Because of the one-way nature of the data, I don't care about the server response. But I found that if I don't handle the HttpWebResponse my code locks up on the third write. I think this is because of the DefaultConnectionLimit setting and the system not reusing the resource...
Per Jon Skeet
Note that you do need to dispose of the WebResponse returned by request.GetResponse - otherwise the underlying infrastructure won't know that you're actually done with it, and won't be able to reuse the connection.
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(ServiceURI);
httpRequest.Method = "POST";
httpRequest.ContentType = "application/x-www-form-urlencoded";
try
{
using (Stream stream = httpRequest.GetRequestStream())
{
stream.Write(postBytes, 0, postBytes.Length);
}
using (HttpWebResponse response = (HttpWebResponse)httpRequest.GetResponse())
{
/// discard response
}
}
catch (Exception)
{
/// ...
}
I want to maximize the speed of sending the POST data and get back to the main program flow as quickly as possible. Because I'm tracing program flow, a synchronous write is preferable, but not mandatory as I can always add a POST field including a Tick count.
Is an HttpWebRequest and stream.Write the quickest method for doing this in .Net4.0 ?
Is there a cheaper way of discarding the unwanted response?
Attually, httpRequest.GetResponse only post the data to server and don't download anything to the client, except the information to tell client if the request is successfully processed by server.
You only get the respone data when you call GetResponseStream. If even the information about the success/error of the request you don't want to received, you don't have anyway to tell if server is success process the request or not.
So the answer is:
Yes, that is lowest level for managed code, unless you want mess up
with socket (which you shouldn't)
With HttpWebRequest, no. You almost don't get any data
you don't want.
I currently have a python server running that handles many different kinds of requests and I'm trying to do this using C#. My code is as follows:
try
{
ServicePointManager.DefaultConnectionLimit = 10;
System.Net.ServicePointManager.Expect100Continue = false;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Proxy = null;
request.Method = "GET";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
response.Close();
}
}
catch (WebException e)
{
Console.WriteLine(e);
}
My first get request is almost instant but after that, the time it takes for one request to go through is almost 30 seconds to 1 min. I have researched everywhere online and tried to change the settings to make it run faster but it can't seem to work. Are there anymore settings that I could change to make it faster?
By using my psychic debugging skills I guess your server only accepts one connection at the time. You do not close your request, so you connection is kept alive. Keep alive attribute. The server will accept new connection when you current one is closed, which is default 100000ms or the server timeout. In your case I guess 30 to 60 seconds. You can start by setting the keepalive attribe to false.
I'm calling a third party web API to update some of our data on their side. I've been submitting about five jobs in quick succession and, without fail, the first two requests are working properly. The last three however never update. The application seems to be indicating that the request is timing out, but I want to make sure that I'm not messing anything up on my side.
I'm calling the function below with an Action<string, Dictionary<string,object>> Delegate and I'm using BeginInvoke to call the API asynchronously. I don't really care about the response. Am I misunderstanding something about WebRequest.GetResponse() or is this a problem with the endpoint?
private void UpdateJobInfo(string jobId, Dictionary<string, object> updates)
{
var postData = GetJsonEncodedValues(updates);
var request = WebRequest.Create(string.Format(JobResultEndpoint, _username, jobId));
request.ContentType = "application/json; charset=utf-8";
request.Method = WebRequestMethods.Http.Put;
request.Headers[HttpRequestHeader.Authorization] = GetAuthenticationCredentials();
request.GetRequestStream().Write(Encoding.ASCII.GetBytes(postData), 0, Encoding.ASCII.GetBytes(postData).Length);
request.GetResponse();
}
You're not disposing of the response (or indeed the request stream, although that's a slightly different matter). That means you're leaving the connection to the server open until the finalizer happens to notice that the response can be finalized. The connections are pooled with (by default) two connections per URL. So your later requests are waiting for the earlier responses to be finalized before they can obtain a connection.
Better code:
// Is this definitely what you want? What about non-ASCII data?
byte[] binaryPostData = Encoding.ASCII.GetBytes(postData);
using (var requestStream = request.GetRequestStream())
{
requestStream.Write(binaryPostData, 0, binaryPostData.Length);
}
using (var response = request.GetResponse())
{
// We don't care about the response, but we have to fetch it
// and dispose it.
}
Hy,
I am using HttpWebRequest in 10 concurent threads to download a list of images. I sorted the images after the hostName so each of this threads are getting an image from the same Hostname.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.ServicePoint.ConnectionLimit = 10;
myReq.Timeout = 2000;
myReq.KeepAlive = true;
HttpWebResponse myResp = (HttpWebResponse )myReq.GetResponse();
After the program is running for a while I keep getting timeout exception.
My thoughts are that I get exception because maybe the Host server has some limitation regarding the concurent connections from the same user.
So how is a connection reused in .net ?
In my program each thread is creating a new connection to a hostname, or is reusing the existing one because of the KeepAlive property ??
It seems that the problem was some servers that were using http/1.0.
HttpWebRequest is using 100 Continue behaviour but the server doesn't support it.
So i changed the property System.Net.ServicePointManager.Expect100Continue to false and then everything worked fine.
If the website isn't responding after one second or so, it's probably safe to assume that it's a bad link and that I should move on to my next link in a set of "possible links."
How do I tell the WebClient to stop attempting to download after some predetermined amount of time?
I suppose I could use a thread, but if it's taking longer than one second to download, that's ok. I just want to make sure that it's connecting with the site.
Perhaps I should modify the WebClient headers, but I don't see what I should modify.
Perhaps I should use some other class in the System.Net namespace?
If you use the System.Net.WebRequest class you can set the Timeout property to be short and handle timeout exceptions.
try{
var request = WebRequest.Create("http://www.contoso.com");
request.Timeout = 5000; //set the timeout to 5 seconds
request.Method = "GET";
var response = request.GetResponse();
}
catch(WebException webEx){
//there was an error, likely a timeout, try another link here
}