I am writing code to surf a website using spring webflow.
I have to update my cookie with his set-cookie header every request else my session will be terminated.
Sometime a request could bump into webException so I need to catch the webException response headers for the cookie update so my next request is still valid
The problem happens with timeout exception, other exceptions like 503 works fine. The webRespones is always null but I can see the server had respond the header part and only stuck in sending the body part in fiddler. My session will be terminated without catching the header response properly. Is there any work around for this?
try
{
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("http://example.com");
webRequest.CookieContainer = new CookieContainer();
webRequest.CookieContainer.Add(cookieCollection);
webRequest.Timeout = 20000;
webRequest.ReadWriteTimeout = 20000;
using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
{
//A function handling the cookie adding
AddCookies(webResponse.Headers, cookieCollection, webResponse.ResponseUri.Host);
}
}
catch (WebException ex)
{
HttpWebResponse webResponse = ex.Response as HttpWebResponse;
if (webResponse == null) throw ex;
//Null webResponse when timeout, the received header information is dropped.
AddCookies(webResponse.Headers, cookieCollection, webResponse.ResponseUri.Host);
}
The timeout exception is implemented on the client side so the response will be null. You could use raw sockets to implement the get, but the easiest solution would probably be to increase the timeout value so you get the response.
Related
I want to download one image from url using console application.
I have used following code:
string sourceUrl = "http://i.ytimg.com/vi/pvBnYBsUi9A/default.jpg"; // Not Found
//string sourceUrl = "http://i.ytimg.com/vi/OrxZAN1FZUY/default.jpg"; // Found
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(sourceUrl);
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (Exception)
{
}
Above code throws exception in line "response = (HttpWebResponse)request.GetResponse();"
but when I am accessing "http://i.ytimg.com/vi/pvBnYBsUi9A/default.jpg" url in my browser then image will be display.
What I am missing here?
I tried that url "http://i.ytimg.com/vi/pvBnYBsUi9A/default.jpg" in Chrome
developer tools. It also receives a 404, but the response includes the image, which displays.
Your code is not the cause of the exception. The site is returning a 404 and your code gets an exception.
You could write logic to look at the response even if you get a 404 and decide whether to take it anyway, as the browser does.
It looks like you can get the response returned by the site if you catch WebException, which allows you to see the http request status and the response, per the documentation.
Example from the .Net 4.5 doc...
try
{
// Creates an HttpWebRequest for the specified URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
// Sends the HttpWebRequest and waits for a response.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
if (myHttpWebResponse.StatusCode == HttpStatusCode.OK)
Console.WriteLine("\r\nResponse Status Code is OK and StatusDescription is: {0}",
myHttpWebResponse.StatusDescription);
// Releases the resources of the response.
myHttpWebResponse.Close();
}
catch(WebException e)
{
Console.WriteLine("\r\nWebException Raised. The following error occured : {0}",e.Status);
}
catch(Exception e)
{
Console.WriteLine("\nThe following Exception was raised : {0}",e.Message);
WebException has Response and Status properties. So it looks like the .Net way to deal with this is to catch WebException and determine how to handle based on the status and response content (if necessary).
is seems to me that WebRequest.Create(url) fails a litle to soon..
Explanations:
on a url that it failed (or throwed a System.Net.WebException) ... copy-paste -ing that url into the browser seems to work..browser gives response!.. sometimes with a remarcable delay (~10 seconds).. but WebRequest in less than 3 seconds throws exception
Example of valid url on which it fails:
http://tracker.podtropolis.com:2710/announce?info_hash=%92%FD%2F%0B%40%F64%C5%86%19%D6%3E%B1%28%B2%81%A1J%D4%F6&peer_id=-AZ2060-%AD%18%05o%11%A6%26%B3%C3%D16%AC&port=6881&downloaded=0&uploaded=0&left=647749313&numwant=30&compact=1&event=started
if it has any meaning firefox reacts 10 times more slowly on this url that explorer, also sometimes firefox crashes on loading such a url
So question Why is WebRequest failing so soon?? I would like it to try a litle harder to get response from URL...
And this is the method that catches exeption (here i check if url is valid OR ~ "is tracker alive??")
public static bool isURLValid(string url)
{
try
{
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.Method = "HEAD";
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
//Returns TURE if the Status code == 200
return (response.StatusCode == HttpStatusCode.OK);
}
catch (Exception ex) //(WebException ex)
{
Logger.e(TAG, "isURLValid", ex);
return false; //Any exception will returns false.
}
}
if valid i get stream (i know .. I know ..double contact.. but still)
inputStream = WebRequest.Create(fullURL).GetResponse().GetResponseStream();
So.. thoughts?
I am writing an application to check the status of some internal web applications. Some of these applications use Windows authentication. When I use this code to check the status, it throws The remote server returned an error: (401) Unauthorized.. Which is understandable because I haven't provided any credentials to the webiste so I am not authorized.
WebResponse objResponse = null;
WebRequest objRequest = HttpWebRequest.Create(website);
objResponse = objRequest.GetResponse();
Is there a way to ignore the 401 error without doing something like this?
WebRequest objRequest = HttpWebRequest.Create(website);
try
{
objResponse = objRequest.GetResponse();
}
catch (WebException ex)
{
//Catch and ignore 401 Unauthorized errors because this means the site is up, the app just doesn't have authorization to use it.
if (!ex.Message.Contains("The remote server returned an error: (401) Unauthorized."))
{
throw;
}
}
I would suggest to try this:
try
{
objResponse = objRequest.GetResponse() as HttpWebResponse;
}
catch (WebException ex)
{
objResponse = ex.Response as HttpWebResponse;
}
finally
The WebException has the response all information you want.
When the server is down or unreachable you will get a timeout exception. I know that the only way to handle that is with a try/catch.
I'm quite sure this is the case for most errors (401/404/501), so: No, you can't ignore (prevent) the exceptions but you will have to handle them. They are the only way to get most of the StatusCodes your App is looking for.
The short of it is you'll want to check the myHttpWebResponse.StatusCode for the status code and act accordingly.
Sample code from reference:
public static void GetPage(String url)
{
try
{
// Creates an HttpWebRequest for the specified URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
// Sends the HttpWebRequest and waits for a response.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
if (myHttpWebResponse.StatusCode == HttpStatusCode.OK)
Console.WriteLine("\r\nResponse Status Code is OK and StatusDescription is: {0}",
myHttpWebResponse.StatusDescription);
// Releases the resources of the response.
myHttpWebResponse.Close();
}
catch(WebException e)
{
Console.WriteLine("\r\nWebException Raised. The following error occured : {0}",e.Status);
}
catch(Exception e)
{
Console.WriteLine("\nThe following Exception was raised : {0}",e.Message);
}
}
I'm writing a port scanner to detect web services running on the local network. Some of these web services require basic authentication - I don't know the username/ password for these services, I just want to list them, so I can't provide the credentials at this stage. I'm using the code:
var request = (HttpWebRequest)WebRequest.Create("http://" + req);
request.Referer = "";
request.Timeout = 3000;
request.UserAgent = "Mozilla/5.0";
request.AllowAutoRedirect = false;
request.Method = WebRequestMethods.Http.Head;
HttpWebResponse response = null;
try
{
response = (HttpWebResponse) request.GetResponse();
// I want to parse the headers here for the server name but as the exception is thrown the response object is null.
}
catch (Exception ex)
{
//401 error is caught here - response is null
}
I'm then parsing out the server name from the headers that are returned - I know they are being returned because I can see them with fiddler but the HttpWebResponse object is set to null as the GetResponse() method is throwing an exception. Basically - how do I get it to not throw and exception but return the headers along with a status code of 401.
If you catch a WebException you'll have access to ex.Response and you can retrieve your headers from there.
I am making a Http Webrequest to an available site that I can visit fine, but the HTTP Web request keeps timing out. Is there any reason why this code might allow it to timeout when it shouldn't?
I've tried upping the timeout setting, but it still continues to timeout.
Uri CameraUrl = new Uri("http://" + cfg_cameraIps[i]);
HttpWebRequest myRequest = (HttpWebRequest)WebRequest.Create(CameraUrl);
myRequest.Timeout = 5000;
myRequest.Method = "HEAD";
try
{
HttpWebResponse webresponse;
webresponse = (HttpWebResponse)myRequest.GetResponse();
if (webresponse.StatusCode.ToString() == "OK")
{
continue;
}
You're not closing your web response - if you find that the first couple of requests work, but ones after that don't, then that's the problem. It's trying to reuse the existing connection to the server, but it can't because you haven't closed the response.
Change your code to:
using (HttpWebResponse webresponse = (HttpWebResponse) myRequest.GetResponse())
{
if (webresponse.StatusCode == HttpStatusCode.OK)
{
continue;
}
...
}
and see if that helps.
If it's failing on the very first request to the server, then that's something different. In that case, use Wireshark to see what's going on at the network level.
Note that in the code above I've also removed the string conversion in favour of comparing the status codes directly.