We have a url and we need to check whether web page is active or not. We tried following code:
WebResponse objResponse = null;
WebRequest objRequest = HttpWebRequest.Create(URL);
objRequest.Method = "HEAD";
try
{
objResponse = objRequest.GetResponse();
objResponse.Close();
}
catch (Exception ex)
{
}
Above code gave exception if unable to get a response but also works fine even if we have a "server error" on that page? Any help how to get server error?
The HttpResponse class has a StatusCode property which you can check. If it's 200 everything is ok.
You can change your code to this:
HttpWebResponse objResponse = null;
var objRequest = HttpWebRequest.Create("http://google.com");
objResponse = (HttpWebResponse) objRequest.GetResponse();
if(objResponse.StatusCode != HttpStatusCode.OK)
{
Console.WriteLine("It failed");
}else{
Console.WriteLine("It worked");
}
For one thing, use a using statement on the response - that way you'll dispose of it whatever happens.
Now, if a WebException is thrown, you can catch that and look at WebException.Response to find out the status code and any data sent back:
WebRequest request = WebRequest.Create(URL);
request.Method = "HEAD";
try
{
using (WebResponse response = request.GetResponse())
{
// Use data for success case
}
}
catch (WebException ex)
{
HttpWebResponse errorResponse = (HttpWebResponse) ex.Response;
HttpStatusCode status = errorResponse.StatusCode;
// etc
}
Related
I'm trying to verify the existence of a Url using HttpWebRequest. I found a few examples that do basically this:
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Url);
request.Method = "HEAD";
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
return response.StatusCode;
}
However, if the url is indeed broken, it's not returning a response, it's instead throwing an exception.
I modified my code to this:
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Url);
request.Method = "HEAD";
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
return response.StatusCode;
}
}
catch (System.Net.WebException ex)
{
var response = ex.Response as HttpWebResponse;
return response == null ? HttpStatusCode.InternalServerError : response.StatusCode;
}
which seems to finally do what I want.
But I would like to know, why is the request throwing an exception instead of returning the response with a NotFound status code?
Ya this can be quite annoying when web pages use status codes heavily and not all of them are errors. Which can make processing the body quite a pain. Personally I use this extension method for getting the response.
public static class HttpWebResponseExt
{
public static HttpWebResponse GetResponseNoException(this HttpWebRequest req)
{
try
{
return (HttpWebResponse)req.GetResponse();
}
catch (WebException we)
{
var resp = we.Response as HttpWebResponse;
if (resp == null)
throw;
return resp;
}
}
}
Why not? They're both valid design options, and HttpWebRequest was just designed to work this way.
Just like #Will, I wrote similar extension method to get the response content in string from WebException.
/// <summary>
/// Reads Response content in string from WebException
/// </summary>
/// <param name="webException"></param>
/// <returns></returns>
public static (HttpStatusCode statusCode, string? responseString) GetResponseStringNoException(this WebException webException)
{
if (webException.Response is HttpWebResponse response)
{
Stream responseStream = response.GetResponseStream();
StreamReader streamReader = new(responseStream, Encoding.Default);
string responseContent = streamReader.ReadToEnd();
HttpStatusCode statusCode = response.StatusCode;
streamReader.Close();
responseStream.Close();
response.Close();
return (statusCode, responseContent);
}
else
{
return (HttpStatusCode.InternalServerError, null);
}
}
The above code is non-optimised solution.
I have the following code :
public static QHttpResponse Execute(QHttpRequest request)
{
//Setup the request
HttpWebRequest webrequest = (HttpWebRequest) WebRequest.Create(request.GetFinalUrl());
webrequest.AllowAutoRedirect = request.IsAllowRedirects;
webrequest.Method = request.Method;
webrequest.Accept = "application/json, text/javascript;q=0.9, */*;q=0.5";
webrequest.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
//request.Host is set automatically
webrequest.UserAgent = request.UserAgent;
if (!String.IsNullOrEmpty(request.Referrer))
webrequest.Referer = request.Referrer;
webrequest.Timeout = 50000;
webrequest.KeepAlive = false;
webrequest.CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Revalidate);
webrequest.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
if (request.IsAjax)
{
webrequest.Headers.Add("X-Request", "JSON");
webrequest.Headers.Add("X-Requested-With", "XMLHttpRequest");
webrequest.Headers.Add("X-Prototype-Version", "1.7");
}
// Cookies
webrequest.CookieContainer = request.Cookies;
// Write the data to the body for POST and other methods
if (request.IsMethodPost())
{
byte[] dataBytes = Encoding.UTF8.GetBytes(request.GetDataParamString());
webrequest.ContentLength = dataBytes.Length;
using (Stream requestStream = webrequest.GetRequestStream())
requestStream.Write(dataBytes, 0, dataBytes.Length);
}
// Get the response
HttpWebResponse webresponse;
try
{
webresponse = webrequest.GetResponse() as HttpWebResponse;
}
catch (WebException wex)
{
if(request.IsBypassError)
webresponse = wex.Response as HttpWebResponse;
else
throw;
}
// Read to QHttpResponse object
QHttpResponse response = new QHttpResponse();
response.StatusCode = webresponse.StatusCode;
response.NewCookies = webresponse.Cookies;
using (Stream responseStream = webresponse.GetResponseStream())
using (StreamReader reader = new StreamReader(responseStream))
response.Reply = reader.ReadToEnd();
webresponse.Close();
return response;
}
I have this code run multiple times from various locations and randomly (Every couple of hours), it hangs at this line :
webresponse = webrequest.GetResponse() as HttpWebResponse;
I tried setting webrequest.KeepAlive = false;, but I continued to receive the error.
I'd like any available help on solving this, thanks in advance.
EDIT : I'd like to add that this code is executed from two threads. Occasionally they may connect to the same host, but only from these 2 threads. Also, as I see, the response is closed appropriately.
EDIT 2 : Visual studio's debugger says the execution is really stuck at System.dll!System.Net.Sockets.Socket.Receive.
EDIT 3 : In an attempt to see exactly what was causing the bug, I modified the "Get the response" code from above to
// Get the response
HttpWebResponse webresponse = null;
try
{
webresponse = webrequest.GetResponse() as HttpWebResponse;
}
catch (WebException wex)
{
Console.WriteLine("Time : " + DateTime.Now);
Console.WriteLine("Thread name : " + Thread.CurrentThread.Name);
Console.WriteLine("Exception : " + wex);
Console.WriteLine("Exc msg : " + wex.Message);
Console.WriteLine("Url : " + request.GetFinalUrl());
if (request.IsBypassError)
webresponse = wex.Response as HttpWebResponse;
else
{
if (webresponse != null)
{
webresponse.Close();
webresponse.Dispose();
}
throw;
}
}
I received this output :
Time : 5/11/2015 3:13:35 AM
Thread name : BOT A
Exception : System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
at System.Net.HttpWebRequest.GetResponse()
at Gameloop.Util.Web.QWebClient.Execute(QHttpRequest request) in e:\Visual Studio - Workspace\Gameloop.Util\Gameloop.Util\Web\QWebClient.cs:line 52
Exc msg : The remote server returned an error: (500) Internal Server Error.
Url : https://website1.com/url/path/to/something (I changed this)
This was the only displayed error and was encountered by thread "BOT A". However, this was not the url the threads appear to have actually frozen at. "BOT A" was actually frozen at 12:00pm at website2.com and "BOT B" was actually frozen at 7:00am at website3.com. I doubt the hanging has much to do with the exception since the requests would have been made a large number of times after that before the actual hang.
My first inclination is that you may need to dispose of your HttpWebResponse. Normally you might wrap that in a using block, but since you have two places where webresponse might be assigned, you might just want to dispose it explicitly, like this.
webresponse.Close();
webresponse.Dispose();
I would start there.
Just add below in the try block:
httpWReq.Timeout = 3000;
I have a really simple question. I am uploading files to a server using HTTP POST. The thing is I need to specially handle connection timeouts and add a bit of a waiting algorithm after a timeout has occurred to relive the server.
My code is pretty simple:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("SomeURI");
request.Method = "POST";
request.ContentType = "application/octet-stream";
request.KeepAlive = true;
request.Accept = "*/*";
request.Timeout = 300000;
request.AllowWriteStreamBuffering = false;
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
WebHeaderCollection headers = response.Headers;
using (Stream Answer = response.GetResponseStream())
{
// Handle.
}
}
}
catch (WebException e)
{
if (Timeout_exception)
{
//Handle timeout exception
}
}
I omitted the file reading code as it is not our concern. Now I need to make sure that once a WebException is thrown, I filter the exception to see if it is indeed a timeout exception. I thought of comparing against the exception message yet I am not sure if this is the right way since the application in question is a commercial app and I am afraid that the message varies between different languages. And what message should I be looking for.
Any suggestions?
You can look at WebException.Status. The WebExceptionStatus enum has a Timeout flag:
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
WebHeaderCollection headers = response.Headers;
using (Stream answer = response.GetResponseStream())
{
// Do stuff
}
}
}
catch (WebException e)
{
if (e.Status == WebExceptionStatus.Timeout)
{
// Handle timeout exception
}
else throw;
}
Using C# 6 exception filters can come in handy here:
try
{
var request = WebRequest.Create("http://www.google.com");
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
WebHeaderCollection headers = response.Headers;
using (Stream answer = response.GetResponseStream())
{
// Do stuff
}
}
}
catch (WebException e) when (e.Status == WebExceptionStatus.Timeout)
{
// If we got here, it was a timeout exception.
}
Yuval's answer is quite a direct hit but here's a version of mine which I've tried since I've undergone in the same circumstance if you want to target via Status codes:
catch (WebException ex)
{
var hwr = (HttpWebResponse)ex.Response;
if (hwr != null)
{
var responseex = hwr.StatusCode;
int statcode = (int)responseex;
if (statcode == 404)
{
Utility.Instance.log(logPath, "The file might not be availble yet at the moment. Please try again later or contact your system administrator.", true);
}
if (statcode == 401)
{
Utility.Instance.log(logPath, "Username and Password do not match.", true);
}
if (statcode == 408)
{
Utility.Instance.log(logPath, "The operation has timed out", true);
}
}
else
{
Utility.Instance.log(logPath, ex + ". Please contact your administrator.", true);//Or you can do a different thing here
}
}
This question already exists:
How to properly catch a 404 error in .NET [duplicate]
Closed 8 years ago.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.google.com/afakepage");
request.Method = WebRequestMethods.Http.Head;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
bool pageExists = response.StatusCode == HttpStatusCode.OK;
When the address is an invalid one the software crashes on the 3rd line of the code when its supposed to get response, any way to work around this?
You can get the response like this
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.google.com/afakepage");
request.Method = WebRequestMethods.Http.Head;
try
{
using (WebResponse response = request.GetResponse())
{
}
}
catch (WebException e)
{
using (WebResponse response = e.Response)
{
HttpWebResponse httpResponse = (HttpWebResponse) response;
MessageBox.Show(httpRespnse.StatusCode.ToString());
}
}
How to let Httpwebresponse ignore the 404 error and continue with it? It's easier than looking for exceptions in input as it is very rare when this happens.
I'm assuming you have a line somewhere in your code like:
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
Simply replace it with this:
HttpWebResponse response;
try
{
response = request.GetResponse() as HttpWebResponse;
}
catch (WebException ex)
{
response = ex.Response as HttpWebResponse;
}
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://mysite.com");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
}
catch(WebException ex)
{
HttpWebResponse webResponse = (HttpWebResponse)ex.Response;
if (webResponse.StatusCode == HttpStatusCode.NotFound)
{
//Handle 404 Error...
}
}
If you look at the properties of the WebException that gets thrown, you'll see the property Response. Is this what you are looking for?