Develop Reference

Error Handling for WebRequest & Response

I am currently running a method that takes a string (a domain name) and checks to see if the site is available or not and passes the information into a Domain object I have created. Currently, I am running into an issue where one of the sites is down and is in turn crashing the application. Below is the method:
private Domain GetStatus(string x)
{
string status = "";
WebRequest req = HttpWebRequest.Create("http://www." + x);
WebResponse res = req.GetResponse();
HttpWebResponse response = (HttpWebResponse)res;
if ((int)response.StatusCode > 226 || response.StatusCode == HttpStatusCode.NotFound)
{
status = "ERROR: " + response.StatusCode.ToString();
}
else
{
status = "LIVE";
}
Domain temp = new Domain(x, status);
return temp;
}
Initial thoughts were that the response.StatusCode == HttpStatusCode.NotFound would handle such an error but it is currently crashing on the line WebResponse res = req.GetResponse(); with the following response:
System.Net.WebException: 'The remote name could not be resolved: 'www.DOMAIN.com''

The issue is due to the fact that your own code is raising an exception.
This can be due to the lack of an internet connection, or a dns resolve issue (which could be caused by the remote party).
So, if the remote server throws an error, you'll get HTTP 500 Internal Server Error, if you can't reach it; your code throws an exception and you'll need to handle that.
To fix this, you can use a try/catch block, something like this:
private Domain GetStatus(string x)
{
string status = "";
try
{
WebRequest req = HttpWebRequest.Create("http://www." + x);
WebResponse res = req.GetResponse();
HttpWebResponse response = (HttpWebResponse)res;
if ((int)response.StatusCode > 226 ||
response.StatusCode == HttpStatusCode.NotFound)
{
status = "ERROR: " + response.StatusCode.ToString();
}
else
{
status = "LIVE";
}
}
catch (Exception e)
{
status = "ERROR: Something bad happend: " + e.ToString();
}
Domain temp = new Domain(x, status);
return temp;
}
By the way, the message,
The remote name could not be resolved
indicates that the host cannot be resolved.
Most likely cause is that your internet is down or, the domain is misspelled or the route to the domain is faulty (e.g. on intranet environments).

HttpWebRequest is all about HTTP protocol, which is kind of a agreed upon language.
But if the person on the other end doesn't exists, so how should you expect him to return you an "Hello" for example ?
So StatusCode is really just about if the actual remote site did response, what did the response state was according to the request resource, is it Successful(200) ? Not Found(404) ? Unauthorized(401) and so on.
Exception means, i couldn't reach the site because of many reasons.
StatusCode means the resource you requested has return this response type.
But a more actual check if the site is alive or not, is querying a static page and not getting exception, a more healthy check, will querying a static page, you will always count as being Healthy; meaning will return a 200 OK response.
So it all depends on what LIVE means for you (or the client using it).
Is it the remote host is actually receiving requests, meaning no Exceptions.
Or it actually means, he's able to get requests and returning me a valid StatusCode response that i expect him to return (Healthy).

HttpWebRequest GetRequestStream SystemException after several calls randomly

I have a program in C# .NET 3.5 CompactFramework that requests data from a C# WebService:
public SynchronisationResult<J> Get<J>(IEnumerable<DomainProxy> existingObjects, string controller, string parameters) where J : IdJsonObject)
{
string existingObjectsJson = JsonConvert.SerializeObject(existingObjects);
string url = GenerateUrl(controller, parameters);
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.ContentType = "text/json";
request.Method = "POST";
request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version11;
request.Timeout = 60000;
request.ContentLength = existingObjectsJson.Length;
using (Stream requestStream = request.GetRequestStream())
{
using (var streamWriter = new StreamWriter(requestStream))
{
streamWriter.Write(existingObjectsJson);
streamWriter.Flush();
streamWriter.Close();
}
requestStream.Close();
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
string responseData = "";
using (var streamReader = new StreamReader(response.GetResponseStream()))
{
responseData = streamReader.ReadToEnd();
}
if (response.StatusCode == HttpStatusCode.OK)
{
result = JsonConvert.DeserializeObject<SynchronisationResult<J>>(responseData);
}
else
{
throw new Exception(responseData);
}
response.Close();
}
}
I can call this method several times with different parameters (different controllers on the WebServer) and suddenly everything gets stuck. The application does not react anymore, when i press PAUSE in Visual Studio i see the program pointer at the location
using (Stream requestStream = request.GetRequestStream())
Sometimes a SystemException is thrown in Timer.ring of System.Net.Connection..., although in case, the application does not continue to run, even not by bubbling the exception to the next catch-Block. This means, that i have to reset the device it does never continue to run.
I have tried the following changes to solve the problem:
request.KeepAlive = true / false
request.Pipelines = true / false
ServicePointManager.DefaultConnectionLimit = 1000;
request.AutomaticDecompression = DecompressionMethods.GZip or nothing
Nothing, the request works fine in Postman for example.
Weird thing is, if i implement this in a for loop, asking for about 200 objects to be updated it crashes faster. In case i implement the request method on a button click and click it with a frequency of about 10 seconds it works way longer. I tried with a development IIS backend on port 888, with a production machine on port 80, firewall locally is turned off. There is no certain request that fails it could be a request for type A or B or C,... each run is different.
Would someone explain:
a) why code gets stuck and does not continue?
b) why code gets stuck even when an exception is thrown
c) how to configure ServicePointManager or Request to get things working properly
EDIT: This is the Exception that sometimes occurs when request.GetRequestStream() is executed:
at System.Threading.Timer.startTimer(UInt32 dueTime)
at System.Threading.Timer.Change(UInt32 dueTime, UInt32 period)
at System.Threading.Timer.Change(Int32 dueTime, Int32 period)
at System.Threading.ThreadPool.QueueUserWorkItem(WaitCallback callBack, Object state, Boolean IsHttpRequest)
at System.Net.Connection.actionSending()
at System.Net.Connection.changeState(ConnectionState state)
at System.Net.Connection.transitionRequestSent(Event e)
at System.Net.Connection.processEvent(Event e)
at System.Net.Connection.actionRequestSent()
at System.Net.Connection.changeState(ConnectionState state)
at System.Net.Connection.transitionIdle(Event e)
at System.Net.Connection.processEvent(Event e)
at System.Net.Connection.submitRequest(HttpWebRequest request)
at System.Net.ServicePoint.SubmitRequest(HttpWebRequest request, String connGroupName)
at System.Net.HttpWebRequest.SubmitRequest()
at System.Net.HttpWebRequest.finishGetRequestStream()
at System.Net.HttpWebRequest.GetRequestStream()

I was also stuck on this issue for couple of days. Finally what I found was if I run Fiddler in the background there was no exception thrown in the request.GetRequestStream(). Which means this is something related to the connection pool where fiddler is handling this. So I did some research and found the below link which solved my issue:
https://www.telerik.com/blogs/help!-running-fiddler-fixes-my-app-
Also after the request is completed make sure you abort that as well. What I did is:
if (webrequest != null) webrequest.Abort();
For me everything is working fine now.

C# Rest Exception Handling [duplicate]

I am initiating an HttpWebRequest and then retrieving it's response. Occasionally, I get a 500 (or at least 5##) error, but no description. I have control over both endpoints and would like the receiving end to get a little bit more information. For example, I would like to pass the exception message from server to client. Is this possible using HttpWebRequest and HttpWebResponse?
Code:
try
{
HttpWebRequest webRequest = HttpWebRequest.Create(URL) as HttpWebRequest;
webRequest.Method = WebRequestMethods.Http.Get;
webRequest.Credentials = new NetworkCredential(Username, Password);
webRequest.ContentType = "application/x-www-form-urlencoded";
using(HttpWebResponse response = webRequest.GetResponse() as HttpWebResponse)
{
if(response.StatusCode == HttpStatusCode.OK)
{
// Do stuff with response.GetResponseStream();
}
}
}
catch(Exception ex)
{
ShowError(ex);
// if the server returns a 500 error than the webRequest.GetResponse() method
// throws an exception and all I get is "The remote server returned an error: (500)."
}
Any help with this would be much appreciated.

Is this possible using HttpWebRequest and HttpWebResponse?
You could have your web server simply catch and write the exception text into the body of the response, then set status code to 500. Now the client would throw an exception when it encounters a 500 error but you could read the response stream and fetch the message of the exception.
So you could catch a WebException which is what will be thrown if a non 200 status code is returned from the server and read its body:
catch (WebException ex)
{
using (var stream = ex.Response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
Console.WriteLine(reader.ReadToEnd());
}
}
catch (Exception ex)
{
// Something more serious happened
// like for example you don't have network access
// we cannot talk about a server exception here as
// the server probably was never reached
}

I came across this question when trying to check if a file existed on an FTP site or not. If the file doesn't exist there will be an error when trying to check its timestamp. But I want to make sure the error is not something else, by checking its type.
The Response property on WebException will be of type FtpWebResponse on which you can check its StatusCode property to see which FTP error you have.
Here's the code I ended up with:
public static bool FileExists(string host, string username, string password, string filename)
{
// create FTP request
FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://" + host + "/" + filename);
request.Credentials = new NetworkCredential(username, password);
// we want to get date stamp - to see if the file exists
request.Method = WebRequestMethods.Ftp.GetDateTimestamp;
try
{
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
var lastModified = response.LastModified;
// if we get the last modified date then the file exists
return true;
}
catch (WebException ex)
{
var ftpResponse = (FtpWebResponse)ex.Response;
// if the status code is 'file unavailable' then the file doesn't exist
// may be different depending upon FTP server software
if (ftpResponse.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
{
return false;
}
// some other error - like maybe internet is down
throw;
}
}

I faced a similar situation:
I was trying to read raw response in case of an HTTP error consuming a SOAP service, using BasicHTTPBinding.
However, when reading the response using GetResponseStream(), got the error:
Stream not readable
So, this code worked for me:
try
{
response = basicHTTPBindingClient.CallOperation(request);
}
catch (ProtocolException exception)
{
var webException = exception.InnerException as WebException;
var rawResponse = string.Empty;
var alreadyClosedStream = webException.Response.GetResponseStream() as MemoryStream;
using (var brandNewStream = new MemoryStream(alreadyClosedStream.ToArray()))
using (var reader = new StreamReader(brandNewStream))
rawResponse = reader.ReadToEnd();
}

You can also use this library which wraps HttpWebRequest and Response into simple methods that return objects based on the results. It uses some of the techniques described in these answers and has plenty of code inspired by answers from this and similar threads. It automatically catches any exceptions, seeks to abstract as much boiler plate code needed to make these web requests as possible, and automatically deserializes the response object.
An example of what your code would look like using this wrapper is as simple as
var response = httpClient.Get<SomeResponseObject>(request);
if(response.StatusCode == HttpStatusCode.OK)
{
//do something with the response
console.Writeline(response.Body.Id); //where the body param matches the object you pass in as an anonymous type.
}else {
//do something with the error
console.Writelint(string.Format("{0}: {1}", response.StatusCode.ToString(), response.ErrorMessage);
}
Full disclosure
This library is a free open source wrapper library, and I am the author of said library. I make no money off of this but have found it immensely useful over the years and am sure anyone who is still using the HttpWebRequest / HttpWebResponse classes will too.
It is not a silver bullet but supports get, post, delete with both async and non-async for get and post as well as JSON or XML requests and responses. It is being actively maintained as of 6/21/2020

Sometimes ex.Response also throws NullReferenceException so below is the best way to handle
catch (WebException ex)
{
using (var stream = ex?.Response?.GetResponseStream())
if(stream != null)
using (var reader = new StreamReader(stream))
{
Console.WriteLine(reader.ReadToEnd());
}
// todo...
}
catch (Exception ex)
{
// todo...
}

**Answer Updated on 14-03-2022**
HttpWebRequest myHttprequest = null;
HttpWebResponse myHttpresponse = null;
try
{
myHttpRequest = (HttpWebRequest)WebRequest.Create(URL);
myHttpRequest.Method = "POST";
myHttpRequest.ContentType = "application/x-www-form-urlencoded";
myHttpRequest.ContentLength = urinfo.Length;
StreamWriter writer = new StreamWriter(myHttprequest.GetRequestStream());
writer.Write(urinfo);
writer.Close();
myHttpresponse = (HttpWebResponse)myHttpRequest.GetResponse();
if (myHttpresponse.StatusCode == HttpStatusCode.OK)
{
//Success code flow
}
myHttpresponse.Close();
}
catch(WebException e) {
Console.WriteLine("This program is expected to throw WebException on successful run."+
"\n\nException Message :" + e.Message);
if(e.Status == WebExceptionStatus.ProtocolError) {
Console.WriteLine("Status Code : {0}",
((HttpWebResponse)e.Response).StatusCode);
Console.WriteLine("Status Description : {0}",
((HttpWebResponse)e.Response).StatusDescription);
}
**Updated Answer with try catch block**
[docs.microsoft][1]

HttpWebRequest doesn't throw exception

I have a problem with httpwebrequest exception. I use the following code to make a request and catch the exception.
try
{
Uri url= new Uri("https://www.example.com");
HttpWebRequest request2 =(HttpWebRequest)WebRequest.Create(url);
request2.Timeout = 10000;
HttpWebResponse response2 = (HttpWebResponse)request2.GetResponse();
response2.Close();
}
catch (TimeoutException)
{
listBox.Items.Insert(0, "Timeout");
}
catch (WebException ex)
{
using (WebResponse response = ex.Response)
{
HttpWebResponse httpResponse = (HttpWebResponse)response;
listBox.Items.Insert(0, "Status code(Benchmark):" + httpResponse.StatusCode);
}
}
catch
{
listBox.Items.Insert(0, "Failure");
}
At company network when I enter a non-existing url such as www.oiuahsdupiasduiuhid.com; it throws webexception . I got status code: not found or service unavailable. However If I try it at home, it doesn't throw any exception. It waits around 1 second and then without any error stops working. I delete all exceptions to see what is happening but the problem is it doesn’t show any error.
Do you have any idea about what is the problem?
Or any recommendation, how can I handle this problem with a different way?

Without knowing more about your application design, specifically exception handling further up the call stack, it is hard to say why it is behaving like it is when you are at home.
But when I just tried your exact code, it did throw a WebException, however httpResponse.StatusCode throws a NullReferenceException because httpResponse is null. If you are potentially swallowing this exception further up the call stack, it could explain the situation you are seeing.
httpResponse is going to be null in many WebException cases because your request did not receive any response, specifically in the timeout scenario.
Before casting WebException.Response, you need to check the WebException.Status property. If that status suggests a response was received, then you can go check WebException.Response, otherwise it is just going to be null. Try something like:
if(e.Status == WebExceptionStatus.ProtocolError) {
listBox.Items.Insert("Status Code : {0}",
((HttpWebResponse)e.Response).StatusCode);
}
else
{
listBox.Items.Insert("Status : {0}", ex.Status);
}
As a side note, your response2.Close(); is never going to be called when HttpWebResponse response2 = (HttpWebResponse)request2.GetResponse(); throws an exception, so you should be wrapping it in a using block:
using(HttpWebResponse response2 = (HttpWebResponse)request2.GetResponse())
{
// do something with response
}

Thanks to psubsee2003. I got my answer. here is my code which is working properly. I added following codes to webexception.
if (ex.Status == WebExceptionStatus.ProtocolError)
{
using (WebResponse response = ex.Response)
{
HttpWebResponse httpResponse = (HttpWebResponse)response;
listBox2.Items.Insert(0, "Status:" + httpResponse.StatusCode);
}
}
else
{
listBox2.Items.Insert(0, "Status: " + ex.Status);
}

How to get error information when HttpWebRequest.GetResponse() fails

I am initiating an HttpWebRequest and then retrieving it's response. Occasionally, I get a 500 (or at least 5##) error, but no description. I have control over both endpoints and would like the receiving end to get a little bit more information. For example, I would like to pass the exception message from server to client. Is this possible using HttpWebRequest and HttpWebResponse?
Code:
try
{
HttpWebRequest webRequest = HttpWebRequest.Create(URL) as HttpWebRequest;
webRequest.Method = WebRequestMethods.Http.Get;
webRequest.Credentials = new NetworkCredential(Username, Password);
webRequest.ContentType = "application/x-www-form-urlencoded";
using(HttpWebResponse response = webRequest.GetResponse() as HttpWebResponse)
{
if(response.StatusCode == HttpStatusCode.OK)
{
// Do stuff with response.GetResponseStream();
}
}
}
catch(Exception ex)
{
ShowError(ex);
// if the server returns a 500 error than the webRequest.GetResponse() method
// throws an exception and all I get is "The remote server returned an error: (500)."
}
Any help with this would be much appreciated.

Is this possible using HttpWebRequest and HttpWebResponse?
You could have your web server simply catch and write the exception text into the body of the response, then set status code to 500. Now the client would throw an exception when it encounters a 500 error but you could read the response stream and fetch the message of the exception.
So you could catch a WebException which is what will be thrown if a non 200 status code is returned from the server and read its body:
catch (WebException ex)
{
using (var stream = ex.Response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
Console.WriteLine(reader.ReadToEnd());
}
}
catch (Exception ex)
{
// Something more serious happened
// like for example you don't have network access
// we cannot talk about a server exception here as
// the server probably was never reached
}

I came across this question when trying to check if a file existed on an FTP site or not. If the file doesn't exist there will be an error when trying to check its timestamp. But I want to make sure the error is not something else, by checking its type.
The Response property on WebException will be of type FtpWebResponse on which you can check its StatusCode property to see which FTP error you have.
Here's the code I ended up with:
public static bool FileExists(string host, string username, string password, string filename)
{
// create FTP request
FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://" + host + "/" + filename);
request.Credentials = new NetworkCredential(username, password);
// we want to get date stamp - to see if the file exists
request.Method = WebRequestMethods.Ftp.GetDateTimestamp;
try
{
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
var lastModified = response.LastModified;
// if we get the last modified date then the file exists
return true;
}
catch (WebException ex)
{
var ftpResponse = (FtpWebResponse)ex.Response;
// if the status code is 'file unavailable' then the file doesn't exist
// may be different depending upon FTP server software
if (ftpResponse.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
{
return false;
}
// some other error - like maybe internet is down
throw;
}
}

I faced a similar situation:
I was trying to read raw response in case of an HTTP error consuming a SOAP service, using BasicHTTPBinding.
However, when reading the response using GetResponseStream(), got the error:
Stream not readable
So, this code worked for me:
try
{
response = basicHTTPBindingClient.CallOperation(request);
}
catch (ProtocolException exception)
{
var webException = exception.InnerException as WebException;
var rawResponse = string.Empty;
var alreadyClosedStream = webException.Response.GetResponseStream() as MemoryStream;
using (var brandNewStream = new MemoryStream(alreadyClosedStream.ToArray()))
using (var reader = new StreamReader(brandNewStream))
rawResponse = reader.ReadToEnd();
}

You can also use this library which wraps HttpWebRequest and Response into simple methods that return objects based on the results. It uses some of the techniques described in these answers and has plenty of code inspired by answers from this and similar threads. It automatically catches any exceptions, seeks to abstract as much boiler plate code needed to make these web requests as possible, and automatically deserializes the response object.
An example of what your code would look like using this wrapper is as simple as
var response = httpClient.Get<SomeResponseObject>(request);
if(response.StatusCode == HttpStatusCode.OK)
{
//do something with the response
console.Writeline(response.Body.Id); //where the body param matches the object you pass in as an anonymous type.
}else {
//do something with the error
console.Writelint(string.Format("{0}: {1}", response.StatusCode.ToString(), response.ErrorMessage);
}
Full disclosure
This library is a free open source wrapper library, and I am the author of said library. I make no money off of this but have found it immensely useful over the years and am sure anyone who is still using the HttpWebRequest / HttpWebResponse classes will too.
It is not a silver bullet but supports get, post, delete with both async and non-async for get and post as well as JSON or XML requests and responses. It is being actively maintained as of 6/21/2020

Sometimes ex.Response also throws NullReferenceException so below is the best way to handle
catch (WebException ex)
{
using (var stream = ex?.Response?.GetResponseStream())
if(stream != null)
using (var reader = new StreamReader(stream))
{
Console.WriteLine(reader.ReadToEnd());
}
// todo...
}
catch (Exception ex)
{
// todo...
}

**Answer Updated on 14-03-2022**
HttpWebRequest myHttprequest = null;
HttpWebResponse myHttpresponse = null;
try
{
myHttpRequest = (HttpWebRequest)WebRequest.Create(URL);
myHttpRequest.Method = "POST";
myHttpRequest.ContentType = "application/x-www-form-urlencoded";
myHttpRequest.ContentLength = urinfo.Length;
StreamWriter writer = new StreamWriter(myHttprequest.GetRequestStream());
writer.Write(urinfo);
writer.Close();
myHttpresponse = (HttpWebResponse)myHttpRequest.GetResponse();
if (myHttpresponse.StatusCode == HttpStatusCode.OK)
{
//Success code flow
}
myHttpresponse.Close();
}
catch(WebException e) {
Console.WriteLine("This program is expected to throw WebException on successful run."+
"\n\nException Message :" + e.Message);
if(e.Status == WebExceptionStatus.ProtocolError) {
Console.WriteLine("Status Code : {0}",
((HttpWebResponse)e.Response).StatusCode);
Console.WriteLine("Status Description : {0}",
((HttpWebResponse)e.Response).StatusDescription);
}
**Updated Answer with try catch block**
[docs.microsoft][1]