Develop Reference

HttpWebRequest.GetResponse() returning 404 Error

I have some code that calls HttpWebRequest's GetResponse() method to retrieve HTML from a URL and return it to the calling method.
This has been working perfectly fine within my Development and QA environments but now that I have uploaded it to my UAT server, I keep getting the following error:
The remote server returned an error: (404) Not Found.
The main difference between Dev/QA and UAT is that UAT uses SSL/HTTPS based URLs whereas Dev/QA uses HTTP. I introduced the following line of code to help progress me a little futher:
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AcceptAllCertifications);
where AcceptAllCertifications always returns true but I still get my 404 error.
I that people who previously had this error have been able to resolve the issue by merely ensuring the URI used for the HttpWebRequest doesn't have a slash at the end (see: Simple HttpWebRequest over SSL (https) gives 404 Not Found under C#) but this does not make a difference to me.
I have now tried what was suggested at this post (see: HttpWebResponse returns 404 error) where I render the exception on the page. This bypassed the yellow-warning screen and gives me a bit more informtion, including the URL it is trying to get a response from. However, when I copy and paste the URL into my browser, it works perfectly fine and renders the HTML on the page. I'm quite happy therefore that the correct URL is being used in the GetResponse call.
Has anyone got any ideas as to what may be causing me this grief? As said, it only seems to be a problem on my UAT server where I am using SSL.
Here is my code to assist:
public static string GetHtmlValues()
{
var webConfigParentUrlValue = new Uri(ConfigurationManager.AppSettings["ParentUrl"]);
var destinationUrl = HttpContext.Current.Request.Url.AbsoluteUri;
var path = "DestinationController" + "/" + "DestinationAction" + "?destinationUrl=" + destinationUrl;
var redirect = new Uri(webConfigParentUrlValue, path).AbsoluteUri;
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AcceptAllCertifications);
var request = (HttpWebRequest)WebRequest.Create(redirect);
//Ensures that if the user has already signed in to the application,
// their authorisation is carried on through to this new request
AttachAuthorisedCookieIfExists(request);
HttpWebResponse result;
try
{
result = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
result = ex.Response as HttpWebResponse;
}
String responseString;
using (Stream stream = result.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
responseString = reader.ReadToEnd();
}
return responseString;
}
More details of the error as it is rendered on the page:

I ran into a similar situation, but with a different error message. My problem turned out to be that my UAT environment was Windows 2008 with .NET 4.5. In this environment the SSL handshake/detecting is performed differently than most web browsers. So I was seeing the URL render without error in a web browser but my application would generate an error. My error message included "The underlying connection was closed: An unexpected error occurred on a send". This might be your issue.
My solution was to force the protocol change. I detect the specific error, then I force a change in the security protocol of my application and try again.
This is the code I use:
catch (Exception ex)
{
if(ex.Message.Contains("The underlying connection was closed: An unexpected error occurred on a send."))
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
// retry the retrieval
}
}

I finally found the solution to my problem...
The first clue to get me on the right track was the wrong physical path being displayed in the 404 error from IIS. It turns out that this incorrect physical path was mapped to another site in my IIS setup. This particular naturally had a binding also; port 443. As you may know, port 443 is the default port for https.
Now looking at my URL that I was trying to pass into the HTTPWebRequest.GetResponse() method, it looked something like this:
https://www.my-web-site.com
Taking this into account, when this application was hosted on IIS within the bounds of SSL, the error was occuring as follows:
Code enters the aforementioned method GetHtmlValues()
The code gets https://www.my-web-site.com from the web.config file
A response is requested from https://www.my-web-site.com
At this point, as no port has been specified and application is now out there on the open internet, it tries to get a response from https://www.my-web-site.com:443
The problem is, my application isn't hosted via IIS on port 443. A different application lives here. Subsequently, as the page can't be found on port 443, a 404 error is produced.
Now for the solution...
Looking in IIS, I found the port that my application sits on. Let's say port 16523.
Whereas previously in my web.config I had my key of ParentUrl decalred with a value of https://www.my-web-site.com, this is to be changed to http://www.my-web-site.com:16523
Note how the https has become http and the port number is specified at the end. Now when the application tries to get the response, it no longer uses the default ssl port as the correct one was specified.

HttpWebRequest timeout not obeyed

I have set up the following web request:
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(gotoWebinerUrl);
request.Accept = "text/json";
request.Timeout = 5000;
// Allows us to track with Fiddler, for dev use only
request.Proxy = new WebProxy("127.0.0.1", 8888);
try
{
WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
...
}
catch (Exception exception)
{
...
}
In Fiddler, I have installed this plugin which delays requests http://fiddlerdelayext.codeplex.com/. Using the plugin, I have added a rule which delays the webinar request URL by 60,000ms (1 min).
I would expect my application to delay for 5 seconds, fail and be caught by my exception. However, it delays for the full 60 seconds.
I'm not sure if this is the plugins problem or my application, but I suspect the latter. In the 60 seconds delay, I can navigate to other web pages and see the requests in Fiddler, so I don't believe the problem exists with the plugin.
I found a couple of similar questions (How to terminate HttpWebRequest Connection in C#?It doesn't work even set timeout or readwritetimeout), but I'm not very familiar with threads and if they apply here or not.
Additional Info
A bit more research into this stepping through the code and I've noticed some interesting behaviour. The GetResponse is fired successfully and I see the request made in Fiddler. I then have my 60 seconds wait time. However, when I can continue stepping through, I notice that it has been caught as a TimeoutException, despite the fact it's actually waited the full 60s and has received a successful response. This does suggest that maybe something in the plugin is fooling the application into holding on somehow.

How to build TCP IP listner to read incoming messages using chilkat socket class c#

I am using chilkat socket class. The problem is I want to keep my socket open, lets say I executed my form and the very first time which opened the port on a specific IP to listen the messages.I am able to receive the messages first time only successfully, now after this message I want to keep my application to keep listening and receive when ever a new message comes.
We have several clients who will connect and send some text messages on the same port and ip.
But I am unable to achieve this. I need to build a Listener, which will keep on listening and as soon as I will get any message I need to process it. Any body who has used chilkat class or having experience in this kind of application kindly suggest me how can I achieve this functionality as I could't find good example for this kind of application on CHILKAT website or may be I am inexperienced don't know how to exactly code this type of functionality.
Edit 1: Jermy,
yes we have developed REST WCF services and they are working perfect, but the problem is in the response of REST WCF Service big response headers are appearing, which we don't want because in our enterprise application Windows Phone 7 mobiles will also communicate and send text messages and only for the sake of mobiles we are trying to reduce the data we need to pass back and by using sockets we can avoid extra response headers and SMS is not an option for us because of cost. If you have any suggestions towards Webservices to minimize the data kindly share it.

Have you considered a Web Service? They can be consumed by pretty much any language that can send Http requests. If you have control of the client applications then a Web Service is definitely the correct route.
http://sarangasl.blogspot.com/2010/09/create-simple-web-service-in-visual.html
Edit:
Have you considered simple http upload of bytes, with a http response code. Ie Http Ok, Http Failure. You can customize the status codes to anything that suits your project.
Edit 2:
Perhaps an RPC styled method with ONLY http status codes as the response could be suitable. Checks this question for hints. json call with C#
Basically you are just sending some string to a url, then receiving the status code back. That is quite minimal.
Edit 3:
Here is something I pulled out of some old code with Reflector. This is just for the general gist of the procedure. Obviously there should be a using statement on the first request.
public void SMS(Uri address, string data)
{
// Perhaps string data is JSON, or perhaps its something delimited who knows.
// Json seems to be the pretty lean.
try
{
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(address);
request.Method = "POST";
// If we don't setup proxy information then IE has to resolve its current settings
// and adds 500+ms to the request time.
request.Proxy = new WebProxy();
request.Proxy.IsBypassed(address);
request.ContentType = "application/json;charset=utf-8";
// If your only sending two bits of data why not add custom headers?
// If you only send headers, no need for the StreamWriter.
// request.Headers.Add("SMS-Sender","234234223");
// request.Headers.Add("SMS-Body","Hey mom I'm keen for dinner tonight :D");
request.Headers.Add("X-Requested-With", "XMLHttpRequest");
StreamWriter writer = new StreamWriter(request.GetRequestStream());
writer.WriteLine(data);
writer.Close();
using (HttpWebResponse response = (HttpWebResponse) request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
// Either read the stream or get the status code and description.
// Perhaps you won't even bother reading the response stream or the code
// and assume success if no HTTP error status causes an exception.
}
}
}
catch (WebException exception)
{
if (exception.Status == WebExceptionStatus.ProtocolError)
{
// Something,perhaps a HTTP error is used for a failed SMS?
}
}
}
Remember to respond only with Http status codes and descriptions. And ensure that the request's proxy is setup to bypass the requesting Url to save time resolving the IE proxy.

WinForm Security Context

I have written a WinForms app that uploads addresses from a spreadsheet, and geocodes them using an external geocoding service. This all works fine on my local machine, but the time has come for it to be installed on other peoples computers for testing. The app no longer works now though, generating the below error:
System.Net.WebException: The remote server returned an error: (407) Proxy Authentication Required.
Having read a lot and chatted breifly to our network guys, it seems i need to establish the Security Context for the users account and work with this to correct the error.
Has anyone got any pointers about how I should be going about this?
Thanks in advance!
C

It depends on how your uploading the data. If your using a http request (as it looks like you are) it will look something like;
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("https://test.example.com/");
req.Method = "POST";
req.ContentType = "text/xml";
req.Credentials = new NetworkCredential("TESTACCOUNT", "P#ssword");
StreamWriter writer = new StreamWriter(req.GetRequestStream());
writer.Write(input);
writer.Close();
var rsp = req.GetResponse().GetResponseStream();

Using .NET's HttpWebRequest to download a multitude of files in a row

I have an application that needs to download several files in a row in succession (sometimes a few thousand). However, what ends up happening when several files need to be downloaded is I get an exception with an inner exception of type SocketException and the error code 10048 (WSAEADDRINUSE). I did some digging and basically it's because the server has run out of sockets (and they are all waiting for 240s or so before they become available again) - not coincidentally it starts happening around the 1024 file range. I would expect that the HttpWebRequest/ServicePointManager would be reusing my connection, but apparently it is not (and the files are https, so that may be part of it). I never saw this problem in the C++ code that this was ported from (but that doesn't mean it didn't ever happen - I'd be surprised if it was, though).
I am properly closing the WebRequest object and the HttpWebRequest object has KeepAlive set to true by default. Next my intent is to fiddle around with ServicePointManager.SetTcpKeepAlive(). However, I can't see how more people haven't run into this problem.
Has anyone else run into the problem, and if so, what did you do to get around it? Currently I have a retry scheme that detects this error and waits it out, but that doesn't seem like the right thing to do.
Here's some basic code to verify what I'm doing (just in case I'm missing closing something):
WebRequest webRequest = WebRequest.Create(uri);
webRequest.Method = "GET";
webRequest.Credentials = new NetworkCredential(username, password);
WebResponse webResponse = webRequest.GetResponse();
try
{
using(Stream stream = webResponse.GetResponseStream())
{
// read the stream
}
}
finally
{
webResponse.Close()
}

What kind of application is this? You mentioned that the server is running out of ports, but then you mentioned HttpWebRequest. Are you running this code in a webservice or ASP.NET page, which is trying to then download multiple files for the same incoming request from the client?
What kind of authentication is the page using? If it is using NTLM authentication, then the connections cannot be shared if the credentials being used are different for each request.
What I would suggest is to group your request per credential. So, for eg, all requests using username "John" would be grouped. You can specify the "ConnectionGroupName" property on the service point, so the system will try to reuse connections for the same credential and server.
If that also doesnt work, you will need to do one or more of the following:
1) Throttle your requests.
2) Increase the wildcard port range.
3) Use the BindIPConnectionCallback on ServicePoint to make it bind to a non-wildcard port (i.e a port in the range 1024-16384)

More digging seems to point to it possibly being due to authentication and the UnsafeAuthenticatedConnectionSharing property might alleviate this. However, I'm not sure that's the best thing, either.