Develop Reference

Posting to a form, can't get the address right

I have intercepted an HTTP POST as follows
Header
Key Value
Request POST /east-berkshire/local/quick_search HTTP/1.1
Accept text/html, application/xhtml+xml, */*
Referer https://www.netmums.com/east-berkshire/local/index/childcare/nannies-au-pairs
Accept-Language en-GB
User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Content-Type application/x-www-form-urlencoded
Accept-Encoding gzip, deflate
Host www.netmums.com
Content-Length 107
DNT 1
Connection Keep-Alive
Cache-Control no-cache
Cookie AMCV_44326DF2572396FB7F000101%40AdobeOrg=817868104%7CMCMID%7C34574735755395522184062187835447062918%7CMCAAMLH-1486721296%7C6%7CMCAAMB-1486721296%7CNRX38WO0n5BH8Th-nqAG_A%7CMCOPTOUT-1486123696s%7CNONE; _ga=GA1.2.258060262.1486116497; _gat=1; _lp4_u=dZXxbBpqGf; __qca=P0-238174588-1486116496764; _tynt_crtg=; aam_uuid=34158303305859258534090346121149142657; __gads=ID=b3ba42a045f2be6a:T=1486116505:S=ALNI_MZHsVecqphdMO7SI-l4IEGrCyFpsg; AMCVS_44326DF2572396FB7F000101%40AdobeOrg=1; ABTastySession=LiwioHashMRASN%3Anull%5E%7C%5ELiwioUTMC%3A1; ABTasty=ABTastyUTMB%3A1%5E%7C%5ELiwioTracking%3A17020310101198682%5E%7C%5EsegmentationTracking%3A17020310101198682%5E%7C%5ELiwioUTMA%3A0.1.1486116611618.0.1486116611618.2; firstvisit=1; Cake=3qdc1afjmdvq0fg9kdunu2okn4; NetmumsLocation=east-berkshire; OX_plg=swf|sl|shk|pm
Body
_method=POST&data%5BListing%5D%5Blisting_category_id%5D=2&data%5BListing%5D%5Blisting_subcategory_id%5D=211
I have written the following C# code to try simulate this
var request = WebRequest.Create("https://www.netmums.com/east-berkshire/local/quick_search") as HttpWebRequest;
if (request == null) throw new HttpRequestException("Could not create web request");
request.Method = "post";
request.ContentType = "application/x-www-form-urlencoded";
var bs = Encoding.ASCII.GetBytes("[Listing][listing_category_id]=2&[Listing][listing_subcategory_id]=211");
using (var reqStream = request.GetRequestStream())
reqStream.Write(bs, 0, bs.Length);
string result;
using (var response = request.GetResponse())
{
var stream = response.GetResponseStream();
if (stream == null) throw new HttpRequestException("No data returned");
var sr = new StreamReader(stream);
result = sr.ReadToEnd();
sr.Close();
}
However when I execute it, on the GetResponse() call I get the error
The remote server returned an error: (404) Not Found.
What am I doing wrong?

C# HTTP POST with Boundary

I need a little help setting up a HTTP Post in C#. I appreciate any assistance I receive in advance.
Using Fiddler here is my RAW POST:
POST http://www.domain.com/tester.aspx HTTP/1.1
User-Agent: Tegan
Content-Type: multipart/form-data; boundary=myboundary
Host: www.domain.com
Content-Length: 1538
Expect: 100-continue
<some-xml>
<customer>
<user-id>george</user-id>
<first-name>George</first-name>
<last-name>Jones</last-name>
</customer>
</some-xml>
My requirements are a little tricky. They require a multi-part post with a boundary. I'm not familiar with setting up a boundary. If any one can assist I would appreciate it.
Here are my requirements:
POST http://www.domain.com/tester.aspx HTTP/1.0(CRLF)
User-Agent: myprogramname(CRLF)
Content-Type: multipart/form-data; boundary=myboundary(CRLF)
Content-Length: nnn(CRLF)
(CRLF)
(CRLF)
--myboundary(CRLF)
Content-Disposition: form-data; name=”xmlrequest”(CRLF)
Content-Type: text/xml(CRLF)
(CRLF)
(XML request message)(CRLF)
(CRLF)
--myboundary--(CRLF)
So I think this is what the POST should look like but I need some help with my C#.
POST http://www.domain.com/tester.aspx HTTP/1.1
User-Agent: Tegan
Content-Type: multipart/form-data; boundary=myboundary
Content-Length: 1538
--myboundary
Content-Disposition: form-data; name="xmlrequest"
Content-Type: text/xml
<some-xml>
<customer>
<user-id>george</user-id>
<first-name>George</first-name>
<last-name>Jones</last-name>
</customer>
</some-xml>
(CRLF)
--myboundary--
Here is the C# code I'm using to create the WebRequest.
HttpWebRequest request = null;
Uri uri = new Uri("http://domain.com/tester.aspx");
request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.UserAgent = "NPPD";
request.ContentType = "multipart/form-data; boundary=myboundary";
request.ContentLength = postData.Length;
using (Stream writeStream = request.GetRequestStream())
{
writeStream.Write(postData, 0, postData.Length);
}
string result = string.Empty;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (StreamReader readStream = new StreamReader(responseStream, Encoding.UTF8))
{
result = readStream.ReadToEnd();
}
}
}
return result;

I blogged about this and presented a sample method that could be used to send multipart/form-data requests. Checkout here: http://www.bratched.com/en/home/dotnet/69-uploading-multiple-files-with-c.html

How to read HTTP POST Request Message Parameters in c#

I am able to read the url and entire page but not able to read the HTTP POST Request Message Parameters in c#.
In my situation i am posting a post url to a site after they verify they send me a HTTP Post message with parameters like id.
here is my code in c#
HttpWebRequest request1 = (HttpWebRequest)WebRequest.Create(uri);
postsourcedata = "processing=true&Sal=5000000";
request1.Method = "POST";
request1.ContentType = "application/x-www-form-urlencoded";
request1.ContentLength = postsourcedata.Length;
request1.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)";
Stream writeStream1 = request1.GetRequestStream();
UTF8Encoding encoding1 = new UTF8Encoding();
byte[] bytes1 = encoding1.GetBytes(postsourcedata);
writeStream1.Write(bytes1, 0, bytes1.Length);
writeStream1.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader readStream = new StreamReader(responseStream, Encoding.UTF8);
string page = readStream.ReadToEnd();
//page.Close();
return page.ToString();
They are sending me request parameters like id and text , how to read these parameters on my side.I am posting to the website through a web service.
Can anyone help me with this?

If they are sending you an HTTP Post message that means that you either need to have a web server or something that understands HTTP protocol to handle the requests, correct?
What I mean is that by your description, it looks like they are sending you an HTTP Request to port 80 or port 443 (https) and you should have asp.net page to handle the request. Once they hit that page, you can simply do:
Request.Parameters("Id")
Request.Parameters("Text")
And so on.

HttpWebRequest POST gives 500 server error

I need to make from my app an authentificated httpwebrequest. the response to my request should be in json format. for this i'm using the code below:
// Create the web request
Uri address = new Uri("http://www.mysite.com/remote/user/login/format/json");
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
request.Method = "POST";
request.UseDefaultCredentials = false;
request.Credentials = new NetworkCredential(UserName, Password);
request.PreAuthenticate = true;
request.ContentType = "application/x-www-form-urlencoded";
request.Accept = "application/json";
string data = string.Format("username={0}&password={1}", otherusername, otherpassword);
// Create a byte array of the data we want to send
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data);
// Set the content length in the request headers
request.ContentLength = byteData.Length;
//Write data
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
// Get response
try
{
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Console application output
jsonResponse = reader.ReadToEnd();
reader.Close();
}
user = new User();
JObject o = JObject.Parse(jsonResponse);
user.Unguessable_id = (string)o["unguessable_id"];
user.Print_id = (string)o["print_id"];
user.Rrid = (string)o["rrid"];
user.Raid = (string)o["raid"];
}
catch (WebException ex) {
errorMessage = ex.Message;
}
the problem is that the very first call it always gives a 500 error on the server. and the request fails. if i redo the call(by making an refresh in my browser) the request is successful.
the request should look like this in normal conditions:
POST /remote/user/login/format/json HTTP/1.1
Host: <yourhost>
username=user&password=pass
but when the server sends out the 500 error he received something like this:
username=user&password=passwordPOST /remote/user/login/format/json HTTP/1.1
any idea why this is happening? in my test app if i refresh the page that makes the httpwebrequest the call is successful.
EDIT:
after installing Fiddler the requests made look like this:
=> the one that generates 500
POST http://www.mysite.com/remote/user/login/format/json HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/json
Host: www.mysite.com
Content-Length: 30
Expect: 100-continue
Connection: Keep-Alive
username=user&password=pass
=> the one made on refresh
POST http://www.mysite.com/remote/user/login/format/json HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/json
Authorization: Basic ZGNpOkFpR2g3YWVj
Host: www.mysite.com
Content-Length: 30
Expect: 100-continue
Connection: Keep-Alive
username=user&password=pass
it seems that Authorization: Basic ZGNpOkFpR2g3YWVj is not included in the first request...why is that happening?(i'm using the same code for both requests)

I would advice you to install Fiddler to see what's really happening

I needed to add:
request.Headers.Add("Authorization: Basic ZGNpOkFpR2g3YWVj");
weird though that for the second request it was added automatically..

.Net C# : Read attachment from HttpWebResponse

Is it possible to read an image attachment from System.Net.HttpWebResponse?
I have a url to a java page, which generates images.
When I open the url in firefox, the download dialog appears. Content-type is application/png.
Seems to work.
When I try this in c#, and make a GET request I retrieve the content-type: text/html and no content-disposition header.
Simple Code:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
response.GetResponseStream() is empty.
A try with java was successful.
Do I have to prepare webrequest or something else?

You probably need to set a User-Agent header.
Run Fiddler and compare the requests.

Writing something in the UserAgent property of the HttpWebRequest does indeed make a difference in a lot of cases. A common practice for web services seem to be to ignore requests with an empty UserAgent.
See: Webmasters: Interpretation of empty User-agent
Simply set the UserAgent property to a non-empty string. You can for example use the name of your application, assembly information, impersonate a common UserAgent, or something else identifying.
Examples:
request.UserAgent = "my example program v1";
request.UserAgent = $"{System.Reflection.Assembly.GetExecutingAssembly().GetName().Name.ToString()} v{System.Reflection.Assembly.GetExecutingAssembly().GetName().Version.ToString()}";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36";
And just to give a full working example:
using System.IO;
using System.Net;
void DownloadFile(Uri uri, string filename)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.Timeout = 10000;
request.Method = "GET";
request.UserAgent = "my example program v1";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream receiveStream = response.GetResponseStream())
{
using (FileStream fileStream = File.Create(filename))
{
receiveStream.CopyTo(fileStream);
}
}
}
}