I need to make from my app an authentificated httpwebrequest. the response to my request should be in json format. for this i'm using the code below:
// Create the web request
Uri address = new Uri("http://www.mysite.com/remote/user/login/format/json");
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
request.Method = "POST";
request.UseDefaultCredentials = false;
request.Credentials = new NetworkCredential(UserName, Password);
request.PreAuthenticate = true;
request.ContentType = "application/x-www-form-urlencoded";
request.Accept = "application/json";
string data = string.Format("username={0}&password={1}", otherusername, otherpassword);
// Create a byte array of the data we want to send
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data);
// Set the content length in the request headers
request.ContentLength = byteData.Length;
//Write data
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
// Get response
try
{
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Console application output
jsonResponse = reader.ReadToEnd();
reader.Close();
}
user = new User();
JObject o = JObject.Parse(jsonResponse);
user.Unguessable_id = (string)o["unguessable_id"];
user.Print_id = (string)o["print_id"];
user.Rrid = (string)o["rrid"];
user.Raid = (string)o["raid"];
}
catch (WebException ex) {
errorMessage = ex.Message;
}
the problem is that the very first call it always gives a 500 error on the server. and the request fails. if i redo the call(by making an refresh in my browser) the request is successful.
the request should look like this in normal conditions:
POST /remote/user/login/format/json HTTP/1.1
Host: <yourhost>
username=user&password=pass
but when the server sends out the 500 error he received something like this:
username=user&password=passwordPOST /remote/user/login/format/json HTTP/1.1
any idea why this is happening? in my test app if i refresh the page that makes the httpwebrequest the call is successful.
EDIT:
after installing Fiddler the requests made look like this:
=> the one that generates 500
POST http://www.mysite.com/remote/user/login/format/json HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/json
Host: www.mysite.com
Content-Length: 30
Expect: 100-continue
Connection: Keep-Alive
username=user&password=pass
=> the one made on refresh
POST http://www.mysite.com/remote/user/login/format/json HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/json
Authorization: Basic ZGNpOkFpR2g3YWVj
Host: www.mysite.com
Content-Length: 30
Expect: 100-continue
Connection: Keep-Alive
username=user&password=pass
it seems that Authorization: Basic ZGNpOkFpR2g3YWVj is not included in the first request...why is that happening?(i'm using the same code for both requests)
I would advice you to install Fiddler to see what's really happening
I needed to add:
request.Headers.Add("Authorization: Basic ZGNpOkFpR2g3YWVj");
weird though that for the second request it was added automatically..
Related
I have intercepted an HTTP POST as follows
Header
Key Value
Request POST /east-berkshire/local/quick_search HTTP/1.1
Accept text/html, application/xhtml+xml, */*
Referer https://www.netmums.com/east-berkshire/local/index/childcare/nannies-au-pairs
Accept-Language en-GB
User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Content-Type application/x-www-form-urlencoded
Accept-Encoding gzip, deflate
Host www.netmums.com
Content-Length 107
DNT 1
Connection Keep-Alive
Cache-Control no-cache
Cookie AMCV_44326DF2572396FB7F000101%40AdobeOrg=817868104%7CMCMID%7C34574735755395522184062187835447062918%7CMCAAMLH-1486721296%7C6%7CMCAAMB-1486721296%7CNRX38WO0n5BH8Th-nqAG_A%7CMCOPTOUT-1486123696s%7CNONE; _ga=GA1.2.258060262.1486116497; _gat=1; _lp4_u=dZXxbBpqGf; __qca=P0-238174588-1486116496764; _tynt_crtg=; aam_uuid=34158303305859258534090346121149142657; __gads=ID=b3ba42a045f2be6a:T=1486116505:S=ALNI_MZHsVecqphdMO7SI-l4IEGrCyFpsg; AMCVS_44326DF2572396FB7F000101%40AdobeOrg=1; ABTastySession=LiwioHashMRASN%3Anull%5E%7C%5ELiwioUTMC%3A1; ABTasty=ABTastyUTMB%3A1%5E%7C%5ELiwioTracking%3A17020310101198682%5E%7C%5EsegmentationTracking%3A17020310101198682%5E%7C%5ELiwioUTMA%3A0.1.1486116611618.0.1486116611618.2; firstvisit=1; Cake=3qdc1afjmdvq0fg9kdunu2okn4; NetmumsLocation=east-berkshire; OX_plg=swf|sl|shk|pm
Body
_method=POST&data%5BListing%5D%5Blisting_category_id%5D=2&data%5BListing%5D%5Blisting_subcategory_id%5D=211
I have written the following C# code to try simulate this
var request = WebRequest.Create("https://www.netmums.com/east-berkshire/local/quick_search") as HttpWebRequest;
if (request == null) throw new HttpRequestException("Could not create web request");
request.Method = "post";
request.ContentType = "application/x-www-form-urlencoded";
var bs = Encoding.ASCII.GetBytes("[Listing][listing_category_id]=2&[Listing][listing_subcategory_id]=211");
using (var reqStream = request.GetRequestStream())
reqStream.Write(bs, 0, bs.Length);
string result;
using (var response = request.GetResponse())
{
var stream = response.GetResponseStream();
if (stream == null) throw new HttpRequestException("No data returned");
var sr = new StreamReader(stream);
result = sr.ReadToEnd();
sr.Close();
}
However when I execute it, on the GetResponse() call I get the error
The remote server returned an error: (404) Not Found.
What am I doing wrong?
I don't know but content length is always negative even if repsonse has corrent Content-Length header. For example:
class Program
{
static void Main()
{
try
{
string result = Get("http://stackoverflow.com/");
Console.WriteLine("Response length = {0}", result.Length);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
static string Get(string adr)
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(adr);
req.UserAgent = "Mozilla/5.0 (Windows NT 6.3; rv:36.0) Gecko/20100101 Firefox/36.0";
req.Proxy = null;
req.KeepAlive = false;
req.Headers.Add("Accept-Language", "ru-RU,ru;q=0.9,en;q=0.8");
req.AllowAutoRedirect = true;
req.Timeout = 10000;
req.ReadWriteTimeout = 10000;
req.MaximumAutomaticRedirections = 10;
req.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
req.Method = WebRequestMethods.Http.Get;
using (var response = (HttpWebResponse) req.GetResponse())
{
using (var stream = response.GetResponseStream())
{
if (stream == null)
throw new NullReferenceException("Response stream is nulL!");
using (var reader = new StreamReader(stream, Encoding.Default))
{
Console.WriteLine("Content length = {0}", response.ContentLength);
return WebUtility.HtmlDecode(reader.ReadToEnd());
}
}
}
}
}
fiddler shows following output:
HTTP/1.1 200 OK
Date: Wed, 02 Sep 2015 20:36:42 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Cache-Control: private, max-age=0
Cf-Railgun: fe8c0e42fd 44.42 0.042796 0030 3350
X-Frame-Options: SAMEORIGIN
X-Request-Guid: bc3ccb1f-1de5-4375-b30d-f3c89134cf86
Server: cloudflare-nginx
CF-RAY: 21fc0233f6652bca-AMS
Content-Length: 251540
But in program I get this:
how can it be fixed?
A couple of things:
When I try, stackoverflow.com does not set a Content-Length header, so it will come through as -1.
But, changing the URL to use a server which definitely does set a Content-Length header, e.g., www.theguardian.com, still produces the same result: -1.
I think it's your use of AutomaticDecompression on the HttpWebRequest object.
If you don't set that property the ContentLength property comes through correctly.
This means that it is Fiddler that is returning the content-length. The reason that Fiddler would do that is that you have the 'Stream' button pressed (or some otherway to indicate to fiddler you want to return 'Stream' or 'Chunked' data.
When a server returns 'Chunked' data, it sets the content-length to -1 -- you don't know the length of the content since you can have unlimited chunks.
If you unclick (or turn off) streaming, then fiddler returns its default 'buffered' response, which is an exact copy of the response from the server. Which of course will include that Content-Length header.
Literally just went through this -- HTH!
I need a little help setting up a HTTP Post in C#. I appreciate any assistance I receive in advance.
Using Fiddler here is my RAW POST:
POST http://www.domain.com/tester.aspx HTTP/1.1
User-Agent: Tegan
Content-Type: multipart/form-data; boundary=myboundary
Host: www.domain.com
Content-Length: 1538
Expect: 100-continue
<some-xml>
<customer>
<user-id>george</user-id>
<first-name>George</first-name>
<last-name>Jones</last-name>
</customer>
</some-xml>
My requirements are a little tricky. They require a multi-part post with a boundary. I'm not familiar with setting up a boundary. If any one can assist I would appreciate it.
Here are my requirements:
POST http://www.domain.com/tester.aspx HTTP/1.0(CRLF)
User-Agent: myprogramname(CRLF)
Content-Type: multipart/form-data; boundary=myboundary(CRLF)
Content-Length: nnn(CRLF)
(CRLF)
(CRLF)
--myboundary(CRLF)
Content-Disposition: form-data; name=”xmlrequest”(CRLF)
Content-Type: text/xml(CRLF)
(CRLF)
(XML request message)(CRLF)
(CRLF)
--myboundary--(CRLF)
So I think this is what the POST should look like but I need some help with my C#.
POST http://www.domain.com/tester.aspx HTTP/1.1
User-Agent: Tegan
Content-Type: multipart/form-data; boundary=myboundary
Content-Length: 1538
--myboundary
Content-Disposition: form-data; name="xmlrequest"
Content-Type: text/xml
<some-xml>
<customer>
<user-id>george</user-id>
<first-name>George</first-name>
<last-name>Jones</last-name>
</customer>
</some-xml>
(CRLF)
--myboundary--
Here is the C# code I'm using to create the WebRequest.
HttpWebRequest request = null;
Uri uri = new Uri("http://domain.com/tester.aspx");
request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.UserAgent = "NPPD";
request.ContentType = "multipart/form-data; boundary=myboundary";
request.ContentLength = postData.Length;
using (Stream writeStream = request.GetRequestStream())
{
writeStream.Write(postData, 0, postData.Length);
}
string result = string.Empty;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (StreamReader readStream = new StreamReader(responseStream, Encoding.UTF8))
{
result = readStream.ReadToEnd();
}
}
}
return result;
I blogged about this and presented a sample method that could be used to send multipart/form-data requests. Checkout here: http://www.bratched.com/en/home/dotnet/69-uploading-multiple-files-with-c.html
This website uses POST to send data whenever the user clicks on a calendar to change the date. I used Firebug to inspect it. The target URL is this. The post data (space-separated) for a particular example is LeagueID=9 GameDate=4-29-2011 Season=2010-2011 Refresh=false LastUpdateTime=01-01-1900 type=Matchups RefreshStartTime=15-5-2011-1308094688683 Week= conferenceID=.
And here are the headers:
Host scores.covers.com
User-Agent Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language en-gb,en;q=0.5
Accept-Encoding gzip, deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive 115
Connection keep-alive
Referer http://scores.covers.com/basketball-scores-matchups.aspx
Content-Length 169
Content-Type text/plain; charset=UTF-8
Cookie __loooooooooongCookieString
I'd like to make that POST request using WebRequest (or whetever else does the trick). Here's my attempt:
string parameters = "LeagueID=\"9\"&GameDate=\"4-29-2011\"&Season=\"2010-2011\"&Refresh=\"false\"&LastUpdateTime=\"01-01-1900\"&type=\"Matchups\"&RefreshStartTime=\"15-5-2011-1308094688683\"&Week=&conferenceID=";
byte[] bytes = Encoding.ASCII.GetBytes(parameters);
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://scores.covers.com/ajax/SportsDirect.Controls.LiveScoresControls.Scoreboard,SportsDirect.Controls.LiveScoresControls.ashx?_method=UpdateScoreboard&_session=no");
req.Method = "POST";
req.ContentLength = bytes.Length;
req.ContentType = "text/plain; charset=UTF-8";
Console.WriteLine(req.ContentLength); // 175
Stream reqStream = req.GetRequestStream();
reqStream.Write(bytes, 0, bytes.Length);
reqStream.Close();
WebResponse resp = req.GetResponse();
Console.WriteLine(((HttpWebResponse)resp).StatusDescription); // OK
Stream respStream = resp.GetResponseStream();
StreamReader reader = new StreamReader(respStream);
Console.WriteLine(reader.ReadToEnd());
resp.Close();
But it doesn't work. The response code is OK, but the response itself is this:
new Object();r.error = new ajax_error('System.FormatException','Input string was
not in a correct format.\r\nCould not retreive parameters from HTTP request.',0
)new Object();r.error = new ajax_error('System.ArgumentException','Object of typ
e \'System.DBNull\' cannot be converted to type \'System.Int32\'.',0)
What's the deal? I can see that something's wrong with the params since the content length of the request is 175 (as opposed to the 169 from the request made by Firefox).
Why not use NameValueCollection to POST your parameters using a WebClient? It does the tricky stuff for you. The code at the bottom of the linked page is about as simple as it comes. Unlike the sample, you should probably deal thoughtfully with disposal of the WebClient.
Don't ASCII encode when you specify UTF-8 later. Make sure to url encode parameters. Try changing the content-type to 'x-www-form-urlencoded'.
Recently I've been experiencing a nightmare with .NET (C#) and SOAP Transmissions.
I've got to consume a webservice (which was supposed to be an easy task) but it become terrible and nothing seem to works.
HttpWebRequest webRequest = (HttpWebRequest)System.Net.WebRequest.Create("http://api.myapi.com/apis/services/theapi");
webRequest.AllowAutoRedirect = true;
webRequest.Timeout = 1000 * 30;
webRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
webRequest.PreAuthenticate = true;
webRequest.Method = "POST";
webRequest.Headers.Add("SOAPAction: \"\"");
webRequest.Accept = "text/xml";
WebResponse webResponse = null;
try
{
webResponse = webRequest.GetResponse();
Stream Stream = webRequest.GetRequestStream();
string SoapEnvelope = "<soap:Envelope>...SOAP CODE ...</soap:Envelope>";
StreamReader streamReader = new StreamReader(webResponse.GetResponseStream(), System.Text.Encoding.UTF8);
XmlDocument SoapEnvelopeXML = new XmlDocument();
SoapEnvelopeXML.LoadXml(SoapEnvelope);
SoapEnvelopeXML.Save(Stream);
string result = streamReader.ReadToEnd();
return result;
}
When I try to sniff the packages by using Wireshark, that's what I get:
---- CLIENT INPUT ------
POST /apis/services/theapi HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
SOAPAction: ""
Accept: text/xml
Host: api.myapi.com
Connection: Keep-Alive
---- SERVER ANSWER ------
HTTP/1.1 500 Internal Server Error
Date: Sat, 14 May 2011 15:35:32 GMT
X-Powered-By: Servlet 2.4; JBoss-4.0.5.GA (build: CVSTag=Branch_4_0 date=200610162339)/Tomcat-5.5
Content-Type: text/xml;charset=ISO-8859-1
Content-Length: 225
Connection: close
X-Pad: avoid browser bug
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><soap:Fault><faultcode>soap:Client</faultcode><faultstring>Error reading XMLStreamReader.</faultstring></soap:Fault></soap:Body></soap:Envelope>
As expected, since I haven't posted a Soap Request (no XML in the request), I receive a SOAP Fault and ERROR 500.
Any ideas?
I've got to somehow do this manually. I've tried to use even TCPClient (to deal with it in a lower level), but all my attempts were frustrated.
You should use the VS Add Service Reference wizard to load the service into your project. The Add Service Reference generates classes to work with the api automatically at a much higher level, from the url of the service API endpoint. It will look something like this:
MyApiClient client = new MyApiClient();
MyApiResult result;
try {
client.Open();
result = client.CallMethod(param1, param2, ...);
client.Close();
} catch (Exception ex) {
// do something with FaultException or API error
}
// do something with the result returned, if needed
If you've done it correct, you shouldn't have to deal with HttpWebRequest, explicit URL's, or hand-typing out any SOAP XML at all!!