Develop Reference

How to validate HttpStatusCode before exception [duplicate]

I'm trying to verify the existence of a Url using HttpWebRequest. I found a few examples that do basically this:
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Url);
request.Method = "HEAD";
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
return response.StatusCode;
}
However, if the url is indeed broken, it's not returning a response, it's instead throwing an exception.
I modified my code to this:
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Url);
request.Method = "HEAD";
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
return response.StatusCode;
}
}
catch (System.Net.WebException ex)
{
var response = ex.Response as HttpWebResponse;
return response == null ? HttpStatusCode.InternalServerError : response.StatusCode;
}
which seems to finally do what I want.
But I would like to know, why is the request throwing an exception instead of returning the response with a NotFound status code?

Ya this can be quite annoying when web pages use status codes heavily and not all of them are errors. Which can make processing the body quite a pain. Personally I use this extension method for getting the response.
public static class HttpWebResponseExt
{
public static HttpWebResponse GetResponseNoException(this HttpWebRequest req)
{
try
{
return (HttpWebResponse)req.GetResponse();
}
catch (WebException we)
{
var resp = we.Response as HttpWebResponse;
if (resp == null)
throw;
return resp;
}
}
}

Why not? They're both valid design options, and HttpWebRequest was just designed to work this way.

Just like #Will, I wrote similar extension method to get the response content in string from WebException.
/// <summary>
/// Reads Response content in string from WebException
/// </summary>
/// <param name="webException"></param>
/// <returns></returns>
public static (HttpStatusCode statusCode, string? responseString) GetResponseStringNoException(this WebException webException)
{
if (webException.Response is HttpWebResponse response)
{
Stream responseStream = response.GetResponseStream();
StreamReader streamReader = new(responseStream, Encoding.Default);
string responseContent = streamReader.ReadToEnd();
HttpStatusCode statusCode = response.StatusCode;
streamReader.Close();
responseStream.Close();
response.Close();
return (statusCode, responseContent);
}
else
{
return (HttpStatusCode.InternalServerError, null);
}
}
The above code is non-optimised solution.

How to get html content from amazon using HttpWebRequest

I am trying to get HTML content from the amazon website. Here is my code to create request, response, and get string:
public static HttpWebResponse GetHttpWebResponse(string url)
{
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.ContentType = "text/xml";
try
{
return (HttpWebResponse)webRequest.GetResponse();
}
catch (WebException e)
{
if (e.Response == null)
throw new Exception("Cannot get response");
return (HttpWebResponse)e.Response;
}
}
public static string GetString(HttpWebResponse response)
{
Encoding encoding = Encoding.UTF8;
using (var reader = new StreamReader(response.GetResponseStream(), encoding))
{
string responseText = reader.ReadToEnd();
return responseText;
}
}
It is working fine with other web sites. However, when I try to get content from amazon, for example:
https://www.amazon.com/gp/product/B00AEISSHA/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1
I am seeing encoded content:
I tried to change Encoding and used HttpUtility.HtmlDecode(html); but it couldn't help. Is there any simple way to get content from Amazon?

You're not catering for compression. If you update your webrequest like this, it should do the trick.
public static HttpWebResponse GetHttpWebResponse(string url)
{
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.ContentType = "text/xml";
webRequest.AutomaticDecompression = DecompressionMethods.GZip;
try
{
return (HttpWebResponse)webRequest.GetResponse();
}
catch (WebException e)
{
if (e.Response == null)
throw new Exception("Cannot get response");
return (HttpWebResponse)e.Response;
}
}

How to get http content using an given api key like this in C#?

http://ssw.com/profile/?apikey = skdwkdkfkkdj
I tried to use
public async Task<string> GetFromUriAsync(string requestUri, string token)
{
var client = new HttpClient();
client.BaseAddress = new Uri(BaseUri);
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("apikey", "=" + token);
HttpResponseMessage response = await client.GetAsync(requestUri);
response.EnsureSuccessStatusCode();
string responseBody = await response.Content.ReadAsStringAsync();
return responseBody;
}
Then it returns null
Am I missing something or is it just totally wrong?
Thanks

You're trying to pass the API key in the header information of your HTTP request. What you need to do is just pass that whole URL without any additional header information.
IE: use "http://ssw.com/profile?apikey=abcdef" as the requestUri and send token as null. Also, remove the setting of the client.DefaultRequestHeaders.Authorization property. Authorization was meant to be a user/pass system and not a token-based system.
To test this, download Fiddler 4 (https://www.telerik.com/download/fiddler). Once you have fiddler installed, on the "Composer" tab, you can test different queries you need by putting the URL directly into the URL box and clicking "Execute". You'll then be able to use the inspectors to see the responses and figure out where you need to go from there.
Here are the classes I use for HTTP GET and POST operations:
public static string HTTPGET(string url)
{
try
{
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url);
request.Timeout = 100000;
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Stream responseStream = response.GetResponseStream();
if (responseStream != null)
using (StreamReader resStream = new StreamReader(responseStream))
return resStream.ReadToEnd();
return null;
}
catch (Exception e)
{
Console.WriteLine(url);
Console.WriteLine(e);
return null;
}
}
public static string HTTPPOST(string url, string postData)
{
try
{
HttpWebRequest webRequest = (HttpWebRequest) WebRequest.Create(url);
webRequest.Method = "POST";
webRequest.ContentType = "x-www-form-urlencoded";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
using (Stream requestStream = webRequest.GetRequestStream())
requestStream.Write(byteArray, 0, byteArray.Length);
using (Stream responseStream = webRequest.GetResponse().GetResponseStream())
if (responseStream != null)
using (StreamReader responseReader = new StreamReader(responseStream))
return responseReader.ReadToEnd();
return null;
}
catch (Exception e)
{
Console.WriteLine(url);
Console.WriteLine(postData);
Console.WriteLine(e);
return null;
}
}

Getting data back from an HttpWebRequest.BeginGetResponse callback

I am writing a Windows Phone 8 app that is supposed to send an GET+POST request to a server and parse the answer.
The code I am using to send the request and to get a response back is the following (it is written in a separate static class):
// server to POST to
string url = "http://myserver.com/?page=hello&param2=val2";
// HTTP web request
var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.Method = "POST";
// Write the request Asynchronously
using (var stream = await Task.Factory.FromAsync<Stream>(httpWebRequest.BeginGetRequestStream,
httpWebRequest.EndGetRequestStream, null))
{
// Create the post data
string postData = "pseudo=pseudo&titre=test&texte=\"Contenu du message\"";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
// Write the bytes to the stream
await stream.WriteAsync(byteArray, 0, byteArray.Length);
stream.Close();
IAsyncResult ar = httpWebRequest.BeginGetResponse(new AsyncCallback(GetResponsetStreamCallback), httpWebRequest);
}
}
private static void GetResponsetStreamCallback(IAsyncResult callbackResult)
{
HttpWebRequest request = (HttpWebRequest)callbackResult.AsyncState;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(callbackResult);
using (StreamReader httpWebStreamReader = new StreamReader(response.GetResponseStream()))
{
string result = httpWebStreamReader.ReadToEnd();
//For debug: show results
System.Diagnostics.Debug.WriteLine(result);
}
My problem is : I have no idea how to get this answer (the string result) back in my behind-code in my app (or any other method in my app to be honest).
How could I do that?

You can try the following code,
string url = "http://myserver.com/?page=hello&param2=val2";
// HTTP web request
var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.Method = "POST";
httpWebRequest.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), webRequest);
}
private void GetRequestStreamCallback(IAsyncResult asynchronousResult)
{
HttpWebRequest webRequest = (HttpWebRequest)asynchronousResult.AsyncState;
using (var postStream = webRequest.EndGetRequestStream(asynchronousResult))
{
//send yoour data here
}
webRequest.BeginGetResponse(new AsyncCallback(GetResponseCallback), webRequest);
}
void GetResponseCallback(IAsyncResult asynchronousResult)
{
try
{
HttpWebRequest myrequest = (HttpWebRequest)asynchronousResult.AsyncState;
using (HttpWebResponse response = (HttpWebResponse)myrequest.EndGetResponse(asynchronousResult))
{
System.IO.Stream responseStream = response.GetResponseStream();
using (var reader = new System.IO.StreamReader(responseStream))
{
data = reader.ReadToEnd();
}
responseStream.Close();
}
}
catch (Exception e)
{
//Handle Exception
}
else
throw;
}
}

public static string GetPageAsString(Uri address)
{
string result = "";
// Create the web request
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
// Get response
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream(), Constants.EncodingType);
// Read the whole contents and return as a string
result = reader.ReadToEnd();
}
return result;
}

Does it have to be a static class? Because if you have a new webrequest object for each request, then each response will come back into it's own object.v
You need to put the result somewhere that you can access it from the place you want to use it.
e.g. if you put it into another public static variable member then you can read it off where you need to. But you probably need to signal the UI to action it, or bind it to the UI.
If you use a static place to store it, then you will only have one active at a time. Unless you add it to a static list of items or results that you are working with
See also: http://blogs.msdn.com/b/devfish/archive/2011/04/07/httpwebrequest-fundamentals-windows-phone-services-consumption-part-2.aspx

You can: make a global variable in App.xaml.cs:
public string result;
In code use it as
(App.Current as App).result = httpWebStreamReader.ReadToEnd();
If you will need to get notified in your current active page when the result is updated - use delegates after you get the response which will signal to your page.

Sending xml using Apache HttpComponents

I am attempting to replicate the following C# code in Java. This code is a helper class that sends a request containing xml, and reads a response.
internal static String Send(String url, String body)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
try
{
// create the new httpwebrequest with the uri
request.ContentLength = 0;
// set the method to POST
request.Method = "POST";
if (!String.IsNullOrEmpty(body))
{
request.ContentType = "application/xml; charset=utf-8";
byte[] postData = Encoding.Default.GetBytes(body);
request.ContentLength = postData.Length;
using (Stream s = request.GetRequestStream())
{
s.Write(postData, 0, postData.Length);
}
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
String responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
if (response.StatusCode != HttpStatusCode.OK)
{
throw new ResponseException(((int)response.StatusCode),
response.StatusCode.ToString(), request.RequestUri.ToString(),
responseString);
}
return responseString;
}
}
catch (WebException e)
{
using (WebResponse response = e.Response)
{
HttpWebResponse httpResponse = response as HttpWebResponse;
if (httpResponse != null)
{
using (Stream data = response.GetResponseStream())
{
data.Position = 0;
throw new ResponseException(((int)httpResponse.StatusCode),
httpResponse.StatusCode.ToString(), request.RequestUri.ToString(),
new StreamReader(data).ReadToEnd()
);
}
}
else
{
throw;
}
After reading other threads I determined that the Apache HttpComponents library would be my best bet to get the same functionality. After reading the documentation and following the example here:
http://hc.apache.org/httpcomponents-client-ga/quickstart.html
I am unable to figure out how to send the body string as xml. When I attempt to set the entity for the request it requires that I declare a BasicNameValuePair, and I do not understand what this is, or how I would format the body string to meet this specification.
Below is what I have currently done.
protected static String Send(String url, String body)
{
HttpPost request = new HttpPost(url);
try
{
request.setHeader("ContentType", "application/xml; charset=utf=8");
// Encode the body if needed
request.setEntity(new UrlEncodedFormEntity());
//get the response
// if the response code is not valid throw a ResponseException
// else return the response string.
} finally {
request.releaseConnection();
}
return null;
}
EDIT : or should I use a StringEntity and do the following
protected static String SendToJetstream(String url, String body)
{
HttpPost request = new HttpPost(url);
try
{
StringEntity myEntity = new StringEntity(body,
ContentType.create("application/xml", "UTF-8"));
// Encode the body if needed
request.setEntity(myEntity);
//get the response
// if the response code is not valid throw a ResponseException
// else return the response string.
} finally {
request.releaseConnection();
}
return null;
}

Use a FileEntity
File file = new File("somefile.xml");
FileEntity entity = new FileEntity(file, ContentType.create("application/xml", "UTF-8"));
Lots of good examples here: http://hc.apache.org/httpcomponents-client-ga/tutorial/html/fundamentals.html#d5e165