UPDATE: I am trying to POST data to https URI. The POST works for HTTP but it fails for HTTPS uri
Hi I am creating a c# winforms exe to post data to a website. The code is below. The issue is, the stream duplicates my post data..
eg: suppose I want to post this -> username=bob
Then when I check the traffic, what is actually sent is, username=bobusername=bob
See? It duplicates, it adds the same line once more to the end of the buffer and sends it.
I am going crazy trying to find the issue from two days.. Can any body solve this or give me some hints please? thank you..
(content length is correctly set to 12, but it sends 24 bytes, after appending same data once again to the tail of buffer)
There are the headers
POST /login/ HTTP/1.0
Content-Type: application/x-www-form-urlencoded
Host: abc.test.com
Content-Length: 12
username=bobusername=bob
-
This is the code I am currently using
string post_data = "username=bob";
string uri = "https://abc.test.com/login/";
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create(uri);
request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
request.Method = "POST";
byte[] postBytes = Encoding.ASCII.GetBytes(post_data);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postBytes.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(postBytes, 0, postBytes.Length);
MessageBox.Show(postBytes.Length.ToString());
requestStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader sr = new StreamReader(response.GetResponseStream());
string tmp = sr.ReadToEnd().Trim();
I put a breakpoint on line byte[] postBytes = Encoding.ASCII.GetBytes(post_data); and postBytes contains the correct data... but it gets output twice.
Why is this happening? I hope I am clear..
I tried out your code and it seemed to work as expected (sent a HTTP Post with a 12byte payload) after I changed the host in the URI to something that was addressable (used http://adsf.com/login). Here's the trace from wireshark:
You might try out the URI I used to see what you get, this will at least rule out your computer or code as possible sources of the problem. If the problem disappears when using a different URI then the problem might be between your network equipment and the web server (reverse-proxy configuration, webserver configuration, network switch configuration, etc).
You can try to get more information by setting trace configuration as described in this page. When I tried your code, I get the following output:
System.Net Verbose: 0 : [2324] Data from ConnectStream#26756241::Write
System.Net Verbose: 0 : [2324] 00000000 : 75 73 65 72 6E 61 6D 65-3D 62 6F 62 : username=bob
System.Net Verbose: 0 : [2324] Exiting ConnectStream#26756241::Write()
Looks like data is correctly written to the ConnectStream. Something wrong somewhere else?
And don't forget to close the WebResponse object.
Related
I appreciate this is a little bit niche, but I thought I would ask anyway. I'm writing a small c# application to utilise the HMRC web portal and electronically submit VAT returns in XML format. According to the HMRC specification it is just a simple Http 1.1 POST action required, and retrieving the response in XML. The application is built, however, I am having trouble with this code. I get an "OK" HttpWebResponse, but the HMRC server returns this spurious error message which they can't seem to tell me what it means. Here is my code:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(destinationUrl);
byte[] bytes;
bytes = System.Text.Encoding.ASCII.GetBytes(requestXml);
request.ContentType = "text/xml; charset='utf-8'";
request.ContentLength = bytes.Length;
request.Method = "POST";
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();
HttpWebResponse response;
response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
string responseStr = new StreamReader(responseStream).ReadToEnd();
return responseStr;
}
And the error is:
1001 - The submitted XML document either failed to validate against the GovTalk schema for this class of document or its body was badly formed.
I know the XML is ok, because when I test it using a third-party tool like "Postman" it submits 100% and the Transaction Engine returns no errors, so it must be my code. Does anything look glaringly wrong to post an XML? I have tried different Content/MIME Types and also I have confirmed that 'utf-8' is the correct encoding.
I was just wondering if any developers out there had worked on the Transaction Engine and could share their submit/post code?
The answer is that the code converting the file contents into the Byte Array was actually converting the file name.
I changed this line:
bytes = System.Text.Encoding.ASCII.GetBytes(requestXml);
To this:
bytes = File.ReadAllBytes(requestXml);
I have built wcf. it is working well
The issue is when I call it many times it displays the following error:
The server encountered an error processing the request. See server
logs for more details
I configured a WCF Tracing File but it remains always empty. what can be the reason of this sudden stop of the service and how to fix it?
Here is the code that I use at the client's side every 20 seconds:
string url = "http://host/Service.svc/method";
HttpWebRequest webrequest = (HttpWebRequest)WebRequest.Create(url);
webrequest.Method = "GET";
ASCIIEncoding encoding = new ASCIIEncoding();
HttpWebResponse webresponse = (HttpWebResponse)webrequest.GetResponse();
Encoding enc = System.Text.Encoding.GetEncoding("utf-8");
StreamReader loResponseStream =
new StreamReader(webresponse.GetResponseStream(), enc);
string strResult = loResponseStream.ReadToEnd();
loResponseStream.Close();
webresponse.Close();
I fixed the issue. it was due to open database connections. I missed to close, at the server side, the database connections. Thank you for answer
It could be a working memory issue on the server/host. If there's less than 5% available you get no response.
I am new to .net and APIs and am currently using .Net 4.5 to connect to an API using rest. The problem I am having is I get an exception thrown back in the return xml that says "Cannot forward request to server with name", "Cannot read data from connection", Connection reset", full error below.
What is odd is this script works fine on smaller datasets but when the response is large enough, I get that exception from the server thrown back. What has helped setting the keep alive to true, using httpversion10, and specifying gzip and sendchunked. I am using advanced rest client to test the server in chrome addins and it returns data fine on there with these larger dataset. It will not with the script below. I am suspecting there is a difference in the way I am telling the server to handle my response verses the chrome add in. Any suggestions on how I improve the performance of this?
This is what the advanced rest client settings look like that work for the Chrome add in.
This is the code I have which appears to need changes to make it handle the request/response better.
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(#"magicalwebsite");
req.KeepAlive = true;
req.ProtocolVersion = HttpVersion.Version10;
req.ServicePoint.ConnectionLimit = 24;
req.Timeout = 2000000000;
req.Method = "Post";
req.Accept = "*/*";
req.SendChunked = true;
req.AutomaticDecompression = DecompressionMethods.GZip;
//Xml request file for data
string postData = System.IO.File.ReadAllText(#"C:\Users\yo\Desktop\testtest.txt");
//sending header and content
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
req.ContentType = "text/xml";
req.ContentLength = byteArray.Length;
req.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes("xxxx:xxxxx"));
Stream dataStream = req.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
//Requesting response of data
HttpWebResponse resp = req.GetResponse() as HttpWebResponse;
//Grabbing response
using (Stream stream = resp.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
String responseString = reader.ReadToEnd();
}
This is the exception I am getting in the xml being thrown back.
<?xml version="1.0" encoding="UTF-8"?>
<response success="false">
<messages>
<message key="exception-caught">Caught Exception: Caught Exception:
Cannot forward request to server with name=prod-euapp01
com.magicalpony.exception.APException: Cannot forward request to server with name=prod-euapp01
at com.magicalpony.webservices.APIForwarder.forward(APIForwarder.java:105)
at com.magicalpony.webservices.APIServlet.forwardRequest(APIServlet.java:270)
at com.magicalpony.webservices.APIServlet.wrongServer(APIServlet.java:253)
at com.magicalpony.webservices.APIServlet.service(APIServlet.java:124)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
at org.apache.catalina.core.ApplicationFilterChain.doFilter
(ApplicationFilterChain.java:210)
at com.magicalpony.system.WebServiceMonitor.doFilter(WebServiceMonitor.java:61)
at org.apache.catalina.core.
ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at com.magicalpony.system.HitTracer.doFilter(HitTracer.java:133)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:399)
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:303)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:183)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:169)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:311)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.magicalpony.exception.APException:
Cannot read data from connection
at com.magicalpony.webservices.NetUtil.readData(NetUtil.java:61)
at com.magicalpony.webservices.APIForwarder.forward(APIForwarder.java:102)
... 26 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at
java.net.SocketInputStream.read(SocketInputStream.java:122)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at com.magicalpony.webservices.NetUtil.readData(NetUtil.java:58)
... 27 more</message>
</messages>
</response>
The problem is with DNS resolution.
Step 1: Enter your domain name in a browser and see if Server is available.
Step 2: If server is available with domain name then you got to fix the IP Address or DNS resolution.
You can fix this by updating the IP Address in your PC (Follow steps below)
Go to a folder: C:\Windows\System32\drivers\etc
Copy and paste "hosts" file to desktop.
Update your host file with your IP Address and domain name.
Step 3: Copy and Paste hosts file in original folder (C:\Windows\System32\drivers\etc).
Step 4: Test your API.
I'm developing some API for testing, and I have a problem when I make a webrequest and especially when i retrieve the webresponse.
I use this code:
string request = HttpPost("http://iunlocker.net/check_imei.php", "ime_i=013270000134001");
public static string HttpPost(string URI, string Parameters)
{
try
{
System.Net.WebRequest req = System.Net.WebRequest.Create(URI);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(Parameters);
req.ContentLength = bytes.Length;
System.IO.Stream os = req.GetRequestStream();
os.Write(bytes, 0, bytes.Length);
os.Close();
System.Net.WebResponse resp= req.GetResponse();
if (resp == null) return null;
System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());
return sr.ReadToEnd().Trim();
}
catch (Exception ex) { }
return null;
}
The website in the call is an example, because with this and with other websites I can't retrieve the result correctly. I receive an exception "Error 403"
Can anyone maybe help me by telling what I may be doing wrong?
I thought the problem was on encoding/decoding -- in fact using Fiddler it asks me if I want to decode before see the text -- but with another website, used for examples, I receive the same message from Fiddler but I can retrieve the response without a problem.
Thanks in advance.
HTTP 403 error means "access forbidden". The destination website is refusing to fulfill your request, for reasons of its own.
Given this particular website http://iunlocker.net/, I'm going to hazard a guess that it may be checking the HTTP_REFERER. In other words it's refusing to fulfill your request because it knows it didn't come from a browser that was viewing the form.
[EDIT] After viewing the response from
curl --form ime_i=013270000134001 -i http://iunlocker.net/check_imei.php
I can see that the immediate response is setting a cookie and a redirect.
HTTP/1.1 307 Temporary Redirect
Server: nginx
Date: Wed, 03 Jul 2013 04:00:27 GMT
Content-Type: text/html
Content-Length: 180
Connection: keep-alive
Set-Cookie: PMBC=35e9e4cd3a7f9d50e7f3bb39d43750d1; path=/
Location: http://iunlocker.net/check_imei.php?pmtry=1
<html>
<head><title>307 Temporary Redirect</title></head>
<body bgcolor="white">
<center><h1>307 Temporary Redirect</h1></center>
<hr><center>nginx</center>
</body>
</html>
This site does not want you scraping it; if you wish to defeat this you will have to make use of its cookies.
http://en.wikipedia.org/wiki/HTTP_403 - The web server is denying you access to that URL.
Perhaps the IP address you are using, is not allowed to access that resource. Check web server.
I have developed web api which accept files using POST method, makes manipulation and return them back using HTTP Response. The web api return additional data in http header like output file name. The problem is that then I am posting and receiving response with HttpWebResponse I get scrambled file name in response header value and unicode characters are lost.
For example if I submit наталья.docx file I get наÑалÑÑ.pdf.
The full response header
Pragma: no-cache
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Result: True
StoreFile: false
Timeout: 300
OutputFileName: наÑалÑÑ.pdf
Content-Disposition: attachment; filename=наÑалÑÑ.pdf
Cache-Control: no-cache, no-store
Content-Type: application/pdf
Date: Wed, 12 Sep 2012 07:21:37 GMT
Expires: -1
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4
I am reading header values like this
HttpWebResponse webResponse = FormUpload.MultipartFormDataPost(postdatatoserver);
using (Stream clientResponse = webResponse.GetResponseStream())
if (webResponse.StatusCode == HttpStatusCode.OK)
{
Helpers.CopyStream(clientResponse, outStream);
webHeaderCollection = webResponse.Headers;
}
I am not sure should I just decode scrambled characters to unicode when I read them from response header or maybe I need to include encoding into response header when I send data from web api server?
See http://msdn.microsoft.com/en-us/library/system.net.webresponse.getresponsestream.aspx:
Stream ReceiveStream = myWebResponse.GetResponseStream();
Encoding enc = System.Text.Encoding.UTF8;
// Pipe the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(ReceiveStream, enc);
You might also try
System.Text.Encoding.Default
or
System.Text.Encoding.UTF7
or
System.Text.Encoding.Unicode
or
System.Text.Encoding.GetEncoding(1251)
or
System.Text.Encoding.GetEncoding(1252)
or
System.Text.Encoding.GetEncoding(20866)
See here for a longer list:
http://www.pcreview.co.uk/forums/system-text-encoding-getencoding-whatvalidstrings-t1406242.html
Edit:
Current [RFC 2045] grammar restricts parameter values (and hence
Content-Disposition filenames) to US-ASCII.
So the HTTP-Headers are always transmitted in ASCII format, irrespective of the StreamReader encoding.
IE doesn't conform to the standard, so there is a workaround: UrlEncode the filename
So you need to do this when you write the file back:
// IE needs url encoding, FF doesn't support it, Google Chrome doesn't care
if (Request.Browser.IsBrowser ("IE"))
{
fileName = Server.UrlEncode(fileName);
}
Response.Clear ();
Response.AddHeader ("content-disposition", String.Format ("attachment;filename=\"{0}\"", fileName));
Response.AddHeader ("Content-Length", data.Length.ToString (CultureInfo.InvariantCulture));
Response.ContentType = mimeType;
Response.BinaryWrite(data);
As per
Unicode in Content-Disposition header
you can add an asterisk, and append the proper encoding.