My current idea was to get the number representations of the status codes and decide what to do based on those. However I'm hitting a roadblock of sorts.
When I'm checking browser requests in chrome there is this part to every response:
Request URL: SiteURL
Request Method: GET
Status Code: 200 OK //note this part
Remote Address: 127.0.0.1:8888
Referrer Policy: no-referrer-when-downgrade
When I do something like this:
var response = Task.Run(() => this.HttpClient.GetAsync($"some_link"));
string pageHtml = response.Result.Content.ReadAsStringAsync().Result;
string statusCode = response.Result.StatusCode.ToString();
I get just "OK" for a status code instead of "200 OK" when everything is fine.
Is there a way to get the number representation of a status code? I'm not sure how to work with the text versions at all since I've no idea what most of those would say, and also I'm not at all sure if there are OK status codes that would work. Is it possible to get a status code of something different than "OK" and still get the requested page? If the answer is "No" I guess that would solve my problems right than and there.
The ultimate idea is when I get a bad status code like 504, 502 (or possibly something different than a good status code) the program just retries the response every 5 minutes until the situation is fixed and continues working after that.
I'm trying to connect to a companies web service, but I'm getting a BadRequest value returned. I suppose that isn't a huge issue, and I'm not asking for help on how to fix the bad request. The issue I'm running into, is how to get more details on what is causing it. So I have this HttpResponseMessage Class variable:
HttpResponseMessage response = c.Post(address, content);
//...fill in content
response = cAssessments.Post(addressInfo, content); //return "Bad Response"
I send it off to their servers with the proper content, and I get back 400 response stating "Bad Request". My question is how do I dig deeper into that repsonse variable to see exactly what is causing it. I'm figuring that there is some dependance on the service I'm hitting to supply information so I'm going to assume it's in there somewhere. But what are something things inside that variable I should be looking into? For example:
"response.StatusCode" gives me the the 400 code. What other properties can I look into?
Any help is appreciated.
Filling content of BadRequest response with some useful tip or information what could possibly go wrong is totally up to web service which serves response. You cant do anything from your side.
Try the Content itself, also the URL of the originating request.
Depending on the API you're requesting against, they may include what specifically caused the 400 in the response's Content.
Also the format you are requesting it in (json vs xml vs whatever)
It all depends on what the service is returning. To check that, you can read the response stream:
using (var stream = response.GetResponseStream ()){
using (var reader = new StreamReader(stream, Encoding.UTF8)
{
var text = reader.ReadToEnd();
// text contains your info
}
}
Another useful option for debugging is using something like postman, where you can paste in your request and see the full response immediately
If the response doesn't include any useful info, you're stuck though, there's nothing you can do about that.
TL;DR version
When a transfer error occurs while writing to the request stream, I can't access the response, even though the server sends it.
Full version
I have a .NET application that uploads files to a Tomcat server, using HttpWebRequest. In some cases, the server closes the request stream prematurely (because it refuses the file for one reason or another, e.g. an invalid filename), and sends a 400 response with a custom header to indicate the cause of the error.
The problem is that if the uploaded file is large, the request stream is closed before I finish writing the request body, and I get an IOException:
Message: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.
InnerException: SocketException: An existing connection was forcibly closed by the remote host
I can catch this exception, but then, when I call GetResponse, I get a WebException with the previous IOException as its inner exception, and a null Response property. So I can never get the response, even though the server sends it (checked with WireShark).
Since I can't get the response, I don't know what the actual problem is. From my application point of view, it looks like the connection was interrupted, so I treat it as a network-related error and retry the upload... which, of course, fails again.
How can I work around this issue and retrieve the actual response from the server? Is it even possible? To me, the current behavior looks like a bug in HttpWebRequest, or at least a severe design issue...
Here's the code I used to reproduce the problem:
var request = HttpWebRequest.CreateHttp(uri);
request.Method = "POST";
string filename = "foo\u00A0bar.dat"; // Invalid characters in filename, the server will refuse it
request.Headers["Content-Disposition"] = string.Format("attachment; filename*=utf-8''{0}", Uri.EscapeDataString(filename));
request.AllowWriteStreamBuffering = false;
request.ContentType = "application/octet-stream";
request.ContentLength = 100 * 1024 * 1024;
// Upload the "file" (just random data in this case)
try
{
using (var stream = request.GetRequestStream())
{
byte[] buffer = new byte[1024 * 1024];
new Random().NextBytes(buffer);
for (int i = 0; i < 100; i++)
{
stream.Write(buffer, 0, buffer.Length);
}
}
}
catch(Exception ex)
{
// here I get an IOException; InnerException is a SocketException
Console.WriteLine("Error writing to stream: {0}", ex);
}
// Now try to read the response
try
{
using (var response = (HttpWebResponse)request.GetResponse())
{
Console.WriteLine("{0} - {1}", (int)response.StatusCode, response.StatusDescription);
}
}
catch(Exception ex)
{
// here I get a WebException; InnerException is the IOException from the previous catch
Console.WriteLine("Error getting the response: {0}", ex);
var webEx = ex as WebException;
if (webEx != null)
{
Console.WriteLine(webEx.Status); // SendFailure
var response = (HttpWebResponse)webEx.Response;
if (response != null)
{
Console.WriteLine("{0} - {1}", (int)response.StatusCode, response.StatusDescription);
}
else
{
Console.WriteLine("No response");
}
}
}
Additional notes:
If I correctly understand the role of the 100 Continue status, the server shouldn't send it to me if it's going to refuse the file. However, it seems that this status is controlled directly by Tomcat, and can't be controlled by the application. Ideally, I'd like the server not to send me 100 Continue in this case, but according to my colleagues in charge of the back-end, there is no easy way to do it. So I'm looking for a client-side solution for now; but if you happen to know how to solve the problem on the server side, it would also be appreciated.
The app in which I encounter the issue targets .NET 4.0, but I also reproduced it with 4.5.
I'm not timing out. The exception is thrown long before the timeout.
I tried an async request. It doesn't change anything.
I tried setting the request protocol version to HTTP 1.0, with the same result.
Someone else has already filed a bug on Connect for this issue: https://connect.microsoft.com/VisualStudio/feedback/details/779622/unable-to-get-servers-error-response-when-uploading-file-with-httpwebrequest
I am out of ideas as to what can be a client side solution to your problem. But I still think the server side solution of using a custom tomcat valve can help here. I currently doesn`t have a tomcat setup where I can test this but I think a server side solution here would be along the following lines :
RFC section 8.2.3 clearly states :
Requirements for HTTP/1.1 origin servers:
- Upon receiving a request which includes an Expect request-header
field with the "100-continue" expectation, an origin server MUST
either respond with 100 (Continue) status and continue to read
from the input stream, or respond with a final status code. The
origin server MUST NOT wait for the request body before sending
the 100 (Continue) response. If it responds with a final status
code, it MAY close the transport connection or it MAY continue
to read and discard the rest of the request. It MUST NOT
perform the requested method if it returns a final status code.
So assuming tomcat confirms to the RFC, while in the custom valve you would have recieved the HTTP request header, but the request body would not be sent since the control is not yet in the servlet that reads the body.
So you can probably implement a custom valve, something similar to :
import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.valves.ErrorReportValve;
public class CustomUploadHandlerValve extends ValveBase {
#Override
public void invoke(Request request, Response response) throws IOException, ServletException {
HttpServletRequest httpRequest = (HttpServletRequest) request;
String fileName = httpRequest.getHeader("Filename"); // get the filename or whatever other parameters required as per your code
bool validationSuccess = Validate(); // perform filename check or anyother validation here
if(!validationSuccess)
{
response = CreateResponse(); //create your custom 400 response here
request.SetResponse(response);
// return the response here
}
else
{
getNext().invoke(request, response); // to pass to the next valve/ servlet in the chain
}
}
...
}
DISCLAIMER : Again I haven`t tried this to success, need sometime and a tomcat setup to try it out ;).
Thought it might be a starting point for you.
I had the same problem. The server sends a response before the client end of the transmission of the request body, when I try to do async request. After a series of experiments, I found a workaround.
After the request stream has been received, I use reflection to check the private field _CoreResponse of the HttpWebRequest. If it is an object of class CoreResponseData, I take his private fields (using reflection): m_StatusCode, m_StatusDescription, m_ResponseHeaders, m_ContentLength. They contain information about the server's response!
In most cases, this hack works!
What are you getting in the status code and response of the second exception not the internal exception?
If a WebException is thrown, use the Response and Status properties of the exception to determine the response from the server.
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.getresponse(v=vs.110).aspx
You are not saying what exactly version of Tomcat 7 you are using...
checked with WireShark
What do you actually see with WireShark?
Do you see the status line of response?
Do you see the complete status line, up to CR-LF characters at its end?
Is Tomcat asking for authentication credentials (401), or it is refusing file upload for some other reason (first acknowledging it with 100 but then aborting it mid-flight)?
The problem is that if the uploaded file is large, the request stream
is closed before I finish writing the request body, and I get an IOException:
If you do not want the connection to be closed but all the data transferred over the wire and swallowed at the server side, on Tomcat 7.0.55 and later it is possible to configure maxSwallowSize attribute on HTTP connector, e.g. maxSwallowSize="-1".
http://tomcat.apache.org/tomcat-7.0-doc/config/http.html
If you want to discuss Tomcat side of connection handling, you would better ask on the Tomcat users' mailing list,
http://tomcat.apache.org/lists.html#tomcat-users
At .Net side:
Is it possible to perform stream.Write() and request.GetResponse() simultaneously, from different threads?
Is it possible to performs some checks at the client side before actually uploading the file?
hmmm... i don't get it - that is EXACTLY why in many real-life scenarios large files are uploaded in chunks (and not as a single large file)
by the way: many internet servers have size limitations. for instance in tomcat that is representad by maxPostSize (as seen in this link: http://tomcat.apache.org/tomcat-5.5-doc/config/http.html)
so tweaking the server configurations seems like the easy way, but i do think that the right way is to split the file to several requests
EDIT: replace Uri.EscapeDataString with HttpServerUtility.UrlEncode
Uri.EscapeDataString(filename) // a problematic .net implementation
HttpServerUtility.UrlEncode(filename) // the proper way to do it
I am experience a pretty similar problem currently also with Tomcat and a Java client. The Tomcat REST service sends a HTTP returncode with response body before reading the whole request body. The client however fails with IOException. I inserted a HTTP Proxy on the client to sniff the protocol and actually the HTTP response is sent to the client eventually. Most likly the Tomcat closed the request input stream before sending the response.
One solution is to use a different HTTP server like Jetty which does not have this problem. The other solution is a add a Apache HTTP server with AJP in front of Tomcat. Apache HTTP server has a different handling of streams and with that the problem goes away.
Does someone know what happen when i receive this error, I am lock for some hours on this problem. My code is :
HttpPost httpPost = new HttpPost(SERVICE_DATA_ADRESSE);
HttpResponse response = httpClient.execute(httpPost);
HttpEntity entity = response.getEntity();
Scanner sc = new Scanner(entity.getContent());
On the server side the method call is working well when i debug it (It is a Csharp server)
When the code run i receive this error
ECONNRESET(Connection reset by peer)
but i have no idea what does it really mean. Does someone can translate to me this error ?
Thanks,
Eliott
It was due to a problem in serialisation when the server want to send the answer.
It seems to be occurring only one machine and none of the other machines.
HttpWebRequest myRequest =(HttpWebRequest)WebRequest.Create("https://connect.zystemsgo.com/auto/");
myRequest.Method = "GET";
SetCertificatePolicy();
Application.DoEvents();
WebResponse myResponse = myRequest.GetResponse();
StreamReader sr = new StreamReader(myResponse.GetResponseStream(),System.Text.Encoding.UTF8);
string result = sr.ReadToEnd();
I tried searching other 400 request errors, but it is not clear. How do I go about debugging this?
HTTP Error 400 means Bad Request. This is being returned by the server.
Usually, when I'm debugging HTTP requests, I use Fiddler to monitor the requests and responses and find out what's going on. It never fails.
(Not really an answer, but too big for comment)
For what it's worth, I ran the following Python code (too lazy to spin up C# :), and it worked fine:
import httplib
conn = httplib.HTTPSConnection('connect.zystemsgo.com')
conn.request('GET', '/auto/')
resp = conn.getresponse()
data = resp.read()
print data # expected ouput, just like visiting in a browser
print resp.status # 200
Are you sure you are showing us the URL that is actually failing, or is your code a more general example?
Perhaps the server certificate is not installed on that machine? I wouldn't expect a HTTP 400 in that case, but it's the only thing I can think of so far...
it is a bad request error .Are there no parameters in the request?
Can you post the response message,it will give some idea of what is going wrong.
The code that i supplied in the comment above works.
WebClient webClient = new WebClient();
webClient.DownloadFile("Your complete url for the file", #"c:\myfile.txt");
you need to have permission to write in the directory of your choice.
You could also try and use the async download if you want.I am not getting why it would not work on a certain machine.