I'm writing a port scanner to detect web services running on the local network. Some of these web services require basic authentication - I don't know the username/ password for these services, I just want to list them, so I can't provide the credentials at this stage. I'm using the code:
var request = (HttpWebRequest)WebRequest.Create("http://" + req);
request.Referer = "";
request.Timeout = 3000;
request.UserAgent = "Mozilla/5.0";
request.AllowAutoRedirect = false;
request.Method = WebRequestMethods.Http.Head;
HttpWebResponse response = null;
try
{
response = (HttpWebResponse) request.GetResponse();
// I want to parse the headers here for the server name but as the exception is thrown the response object is null.
}
catch (Exception ex)
{
//401 error is caught here - response is null
}
I'm then parsing out the server name from the headers that are returned - I know they are being returned because I can see them with fiddler but the HttpWebResponse object is set to null as the GetResponse() method is throwing an exception. Basically - how do I get it to not throw and exception but return the headers along with a status code of 401.
If you catch a WebException you'll have access to ex.Response and you can retrieve your headers from there.
Related
I'm trying to get the order book from GDAX (link to documentation of the call) but when doing it from the c# executable I always get Error 400 - Bad request.
When taking the actual URL and pasting it into my browser, it works fine.
String URL = "https://api.gdax.com/products/BTC-USD/book?level=2";
WebRequest request = WebRequest.Create(URL);
WebResponse response = request.GetResponse();
The actual issue with your API call is , the API is expecting a user-agent string while making the request: Below is the code in working condition:
try
{
String URL = "http://api.gdax.com/products/BTC-USD/book?level=2";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL);
request.UserAgent = ".NET Framework Test Client";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
var encoding = ASCIIEncoding.ASCII;
using (var reader = new System.IO.StreamReader(response.GetResponseStream(), encoding))
{
string responseText = reader.ReadToEnd();
}
}
catch(WebException ex)
{
HttpWebResponse xyz = ex.Response as HttpWebResponse;
var encoding = ASCIIEncoding.ASCII;
using (var reader = new System.IO.StreamReader(xyz.GetResponseStream(), encoding))
{
string responseText = reader.ReadToEnd();
}
}
Basically ProtocolError indicates that you have received the response but there is an error related to protocol, which you can observe, when you read the response content from exception. I have added catch to handle the exception and read ex.Response (which is HttpWebResponse) and could see that the API is asking for user-agent to be suppllied while making the call. I got to see the error as "{"message":"User-Agent header is required."}"
You can ignore the code inside the exception block, I used it only to see what is the actual response message, which contains actual error details
Note: I have boxed WebRequest to HttpWebRequest to have additional http protocol related properties and most importantly "UserAgent" property which is not available with the WebRequest class.
You need to Accept the certificarte, Google for access to a https webrequest.
Like this
I want to download one image from url using console application.
I have used following code:
string sourceUrl = "http://i.ytimg.com/vi/pvBnYBsUi9A/default.jpg"; // Not Found
//string sourceUrl = "http://i.ytimg.com/vi/OrxZAN1FZUY/default.jpg"; // Found
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(sourceUrl);
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (Exception)
{
}
Above code throws exception in line "response = (HttpWebResponse)request.GetResponse();"
but when I am accessing "http://i.ytimg.com/vi/pvBnYBsUi9A/default.jpg" url in my browser then image will be display.
What I am missing here?
I tried that url "http://i.ytimg.com/vi/pvBnYBsUi9A/default.jpg" in Chrome
developer tools. It also receives a 404, but the response includes the image, which displays.
Your code is not the cause of the exception. The site is returning a 404 and your code gets an exception.
You could write logic to look at the response even if you get a 404 and decide whether to take it anyway, as the browser does.
It looks like you can get the response returned by the site if you catch WebException, which allows you to see the http request status and the response, per the documentation.
Example from the .Net 4.5 doc...
try
{
// Creates an HttpWebRequest for the specified URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
// Sends the HttpWebRequest and waits for a response.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
if (myHttpWebResponse.StatusCode == HttpStatusCode.OK)
Console.WriteLine("\r\nResponse Status Code is OK and StatusDescription is: {0}",
myHttpWebResponse.StatusDescription);
// Releases the resources of the response.
myHttpWebResponse.Close();
}
catch(WebException e)
{
Console.WriteLine("\r\nWebException Raised. The following error occured : {0}",e.Status);
}
catch(Exception e)
{
Console.WriteLine("\nThe following Exception was raised : {0}",e.Message);
WebException has Response and Status properties. So it looks like the .Net way to deal with this is to catch WebException and determine how to handle based on the status and response content (if necessary).
I am writing code to surf a website using spring webflow.
I have to update my cookie with his set-cookie header every request else my session will be terminated.
Sometime a request could bump into webException so I need to catch the webException response headers for the cookie update so my next request is still valid
The problem happens with timeout exception, other exceptions like 503 works fine. The webRespones is always null but I can see the server had respond the header part and only stuck in sending the body part in fiddler. My session will be terminated without catching the header response properly. Is there any work around for this?
try
{
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("http://example.com");
webRequest.CookieContainer = new CookieContainer();
webRequest.CookieContainer.Add(cookieCollection);
webRequest.Timeout = 20000;
webRequest.ReadWriteTimeout = 20000;
using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
{
//A function handling the cookie adding
AddCookies(webResponse.Headers, cookieCollection, webResponse.ResponseUri.Host);
}
}
catch (WebException ex)
{
HttpWebResponse webResponse = ex.Response as HttpWebResponse;
if (webResponse == null) throw ex;
//Null webResponse when timeout, the received header information is dropped.
AddCookies(webResponse.Headers, cookieCollection, webResponse.ResponseUri.Host);
}
The timeout exception is implemented on the client side so the response will be null. You could use raw sockets to implement the get, but the easiest solution would probably be to increase the timeout value so you get the response.
I am getting The remote server returned an error: (400) Bad Request error while running the following code.
I am trying to upload xml file on the http server.
My xml file contains tag for the username,password and domain and when i am trying to connect is manually i am able to connect it,but using same credentials when i am trying to connect it through this code, i am getting 400 Bad Request error.
Please suggest me how to overcome this issue.
Thanks
`
public static void UploadHttp(string xml)
{
string txtResults = string.Empty;
try
{
string url = "http://my.server.com/upload.aspx ";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.KeepAlive = false;
request.SendChunked = true;
request.AllowAutoRedirect = true;
request.Method = "Post";
request.ContentType = "text/xml";
var encoder = new UTF8Encoding();
var data = encoder.GetBytes(xml);
request.ContentLength = data.Length;
var reqStream = request.GetRequestStream();
reqStream.Write(data, 0, data.Length);
reqStream.Close();
WebResponse response = null;
response = request.GetResponse();
var reader = new StreamReader(response.GetResponseStream());
var str = reader.ReadToEnd();
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError)
{
HttpWebResponse err = ex.Response as HttpWebResponse;
if (err != null)
{
string htmlResponse = new StreamReader(err.GetResponseStream()).ReadToEnd();
txtResults = string.Format("{0} {1}", err.StatusDescription, htmlResponse);
}
}
else
{
}
}
catch (Exception ex)
{
txtResults = ex.ToString();
}
}`
Are you sure you should be using POST not PUT?
POST is usually used with application/x-www-urlencoded formats. If you are using a REST API, you should maybe be using PUT? If you are uploading a file you probably need to use multipart/form-data. Not always, but usually, that is the right thing to do..
Also you don't seem to be using the credentials to log in - you need to use the Credentials property of the HttpWebRequest object to send the username and password.
400 Bad request Error will be thrown due to incorrect authentication entries.
Check if your API URL is correct or wrong. Don't append or prepend spaces.
Verify that your username and password are valid. Please check any spelling mistake(s) while entering.
Note: Mostly due to Incorrect authentication entries due to spell changes will occur 400 Bad request.
What type of authentication do you use?
Send the credentials using the properties Ben said before and setup a cookie handler.
You already allow redirection, check your webserver if any redirection occurs (NTLM auth does for sure). If there is a redirection you need to store the session which is mostly stored in a session cookie.
//use "ASCII" or try with another encoding scheme instead of "UTF8".
using (StreamWriter postStream = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.UTF8))
{
postStream.Write(postData);
postStream.Close();
}
I am writing an application to check the status of some internal web applications. Some of these applications use Windows authentication. When I use this code to check the status, it throws The remote server returned an error: (401) Unauthorized.. Which is understandable because I haven't provided any credentials to the webiste so I am not authorized.
WebResponse objResponse = null;
WebRequest objRequest = HttpWebRequest.Create(website);
objResponse = objRequest.GetResponse();
Is there a way to ignore the 401 error without doing something like this?
WebRequest objRequest = HttpWebRequest.Create(website);
try
{
objResponse = objRequest.GetResponse();
}
catch (WebException ex)
{
//Catch and ignore 401 Unauthorized errors because this means the site is up, the app just doesn't have authorization to use it.
if (!ex.Message.Contains("The remote server returned an error: (401) Unauthorized."))
{
throw;
}
}
I would suggest to try this:
try
{
objResponse = objRequest.GetResponse() as HttpWebResponse;
}
catch (WebException ex)
{
objResponse = ex.Response as HttpWebResponse;
}
finally
The WebException has the response all information you want.
When the server is down or unreachable you will get a timeout exception. I know that the only way to handle that is with a try/catch.
I'm quite sure this is the case for most errors (401/404/501), so: No, you can't ignore (prevent) the exceptions but you will have to handle them. They are the only way to get most of the StatusCodes your App is looking for.
The short of it is you'll want to check the myHttpWebResponse.StatusCode for the status code and act accordingly.
Sample code from reference:
public static void GetPage(String url)
{
try
{
// Creates an HttpWebRequest for the specified URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
// Sends the HttpWebRequest and waits for a response.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
if (myHttpWebResponse.StatusCode == HttpStatusCode.OK)
Console.WriteLine("\r\nResponse Status Code is OK and StatusDescription is: {0}",
myHttpWebResponse.StatusDescription);
// Releases the resources of the response.
myHttpWebResponse.Close();
}
catch(WebException e)
{
Console.WriteLine("\r\nWebException Raised. The following error occured : {0}",e.Status);
}
catch(Exception e)
{
Console.WriteLine("\nThe following Exception was raised : {0}",e.Message);
}
}