how to set up request body with httpWebRequest - c#

I want using httpWebRequest to "POST" data to a website. So i used firebug analyzing what really send to server. First step, i use browser to browsing www.mytargetURL.net, second, turn on firebug, after that, i fill all form data and click the submit button (it's mean POST data to the server). So, I watching firebug and there was a lot of Parameters in request body part. Something like:
param1=
param2=
param3=default_value1
param4=default_value2
param5=value_I_set_byhand1
param6=value_I_set_byhand2
The question is: I should set up the request body of httpWebRequest obj with all the parameters i saw in firebug parameter table (it's mean all 6 parameters) or just parameters which has value (parameter 3-6) or just parameters i have filled in the submit form(just param5 and param6)?
Thanks you for all supports

You create the HttpWebRequest object, get the request stream, and write your parameters to it. The example at HttpRequest.GetRequestStream should point you in the right direction.

Related

Pass model via "redirect to action" visible to URL String in Browser

The thing is I'm passing some data in model from first action to second action of same controller
but the data in model is visible in URL query string.
This is how it is visible to Query string
Now
1 - is there any solution to hide that data in query string Other than using Temp Data
2 - and why redirection to action is happening from client ??
3 - It seems to be passing data via Get method in redirection, is there any way I can pass data in Body (i.e. Post).
The MVC RedirectToAction method simply sends the client (browser, in this case) a HTTP header called "Location" in the response, and sets the status code to 302 "Found", to indicate a suggested redirect. That header contains the URL of the view you specified, and puts any model data onto the querystring.
The client can choose whether to visit that URL or not. Browsers generally follow the link automatically, because it's user-friendly to do so. Other HTTP clients may not do so. That's how it works - this is general to the whole web, not just MVC. The redirect is always done as a GET, it's not possible to redirect via any other HTTP method.
If you want to just display the view without a redirect, then use
return View("addLeaveEntitlement", result.Model)
Then there's just one request, there's no redirect, and the URL does not change at all (not even to the regular URL of the "add" view - this may or may not be desirable).

Send POST request to asp.net mvc action via Fiddler

I have an ASP.NET MVC web site. One of my routes is a URL that takes 5 parameters. For the sake of illustration, these parameters are named parameter1, parameter2, parameter3, parameter4, and parameter5. Currently, I'm constructing a URL in some C# code that will POST to the mvc action via a WebClient. that code looks like this:
WebClient myWebClient = new WebClient();
myWebClient.UploadStringCompleted += myWebClient_UploadStringCompleted;
string url = "http://www.example.com/customer/" + parameter1 + "/orders/" + parameter2 + "/" + parameter3 + "/" + parameter4 + "/" + parameter5;
myWebClient.UploadStringAsync(new Uri(url, UriKind.Absolute));
I'm confident that the UploadString method does a POST. I need to do a POST, because my parameter values can be very long. In fact, I estimate that occasionally, the total url length may be 20000 characters long. Regardless, I get a 400 error when I attempt to post my data. In an effort to debug this, I'm trying to figure out how to simulate a POST in Fiddler.
Assuming that I am passing values via a query string as shown above, what values do I enter into Fiddler? From the Composer tab, I'm not sure what to enter for the Request Headers area. I'm also not entirely sure what to enter for the url. I'm not sure if I put the entire URL in there, including the parameter values, or if those belong in the Request Headers.
What I need to enter into Fiddler, so that I can debug my issue?
Basically all your parameters are a part of the URL, and this is the root of your problem. Here is what is going on: you are hitting the URL length limitation, and receiving a "400 Bad request" error. In real world most web browsers do not work with URLs more than 2000 characters long.
To resolve this problem I would suggest doing a bit of refactoring, so that request is posted to the URL http://www.example.com/customer/parameter1/orders or even http://www.example.com/customer/orders with parameters send in request body. Here is how test such request in Fiddler:
On Composer tab choose POST request verb
Specify the URL as
http://www.example.com/customer/parameter1/orders
or
http://www.example.com/customer/orders
In Request Headers section you can set content type header like
Content-Type: application/x-www-form-urlencoded
or any other header you might require. Or just leave it blank which will work in your case.
Finally in Request Body field list your parameters in query string form
parameter1name=parameter1value&parameter2name=parameter2value
In this new case here is how you can send such a request using WebClient:
WebClient myWebClient = new WebClient();
myWebClient.UploadStringCompleted += myWebClient_UploadStringCompleted;
string url = "http://www.example.com/customer/orders";
string data = "parameter1name=parameter1value&parameter2name=parameter2value";
myWebClient.UploadStringAsync(new Uri(url, UriKind.Absolute), data);
I simply mimic the exact request that was sent.
This is how I do it:
Open Fiddler
Go to the page that I want to re-issue the command i.e. repeat the bug step but watch for the request in the list
Select it from the list and right-click, go to replay > reissue and edit
This build a replicated request but hits a break point before it is sent (You will see the red bar on the right)
Above this you can edit the values that were sent by double-clicking on any of them in Headers, QueryString etc
Then hit Run to Complete

crawling / scraping a search form based webpages

I want to crawl/scrape a webpage which has a form
to be precise following is the URL
http://lafayetteassessor.com/propertysearch.cfm
The problem is, i want to make a search and save the result in a webpage.
my search string will always give a unique page, so result count won't be a problem.
the search over there doesn't search on URL (e.g. google searching url contains parameters to search). How can i search from starting page (as above) and get the result page ?
please give me some idea.
I am using C#/.NET.
If you look at the forms on that page, you will notice that they use the POST method, rather than the GET method. As I'm sure you know, GET forms pass their parameters as part of the URL, eg mypage?arg1=value&arg2=value
However, for POST requests, you need to pass the parameters as the request body. It takes the same format, it's just passed in differently. To do this, use code similar to this:
HttpRequest myRequest = (HttpRequest)WebRequest.Create(theURL);
myRequest.Method = "post";
using(TextWriter body = new StreamWriter(myRequest.GetRequestStream())) {
body.Write("arg1=value1&arg2=value2");
}
WebResponse theResponse = myRequest.GetResponse();
//do stuff with the response
Don't forget that you still need to escape the arguments, etc.

C# httpwebrequest on javascript

As some of you may know, you are able to POST with C#. This means you can "push" buttons on a website with webrequest/response. Now there are also buttons on sites which work with javascript, they start like:
(function($j){
$j.data(document, 'maxPictureSize', 764327);
share_init();
})(jQuery.noConflict());
Is there any solution you can make those function calls in C# with like httprequests or any other kind of library?
I'm assuming you have a program that wants to manipulate the server "back end" for a web page by making the server think that someone pushed a button that POSTs, and sending the data that the web page would include with its POST.
The first tool you need is Microsoft Network Monitor 3.3, or another network packet tracing tool. Use this to look at the POST from the real web page. NetMon (at least) decomposes the packet into the HTTP pieces and headers, so you can easily see what's going on.
Now you will know what data the real POST is sending, and the URL to which it is sending the data (with any possible "query string" - which is unusual for a POST).
Next you need to write C# to create the same sort of POST to the same URL. It seems that you already know about HttpWebRequest/HttpWebResponse so I won't explain them in detail. You may have noticed in your NetMon trace that the Content-Type header was application/x-www-form-urlencoded. This is most often data from an HTML form which is URL-Encoded (like the name), so you need to URL-Encode your data before POSTing it, and you need to know the size of the encoded data for the Content-Length. HttpUtility.UrlEncode() is one method to use for this encoding.
Once you think you have it, try it and use NetMon to inspect your POST request and the response from the server. Keep going until you have duplicated what the mystery web page is doing.
Ok use webBrowser form to load the page:
webBrowser.Navigate( url );
then save the contents of the web broweser form to a file or a string:
File.WriteAllText(#"c:\test\ajax_test.txt", webBrowser1.Document.Body.Parent.OuterHtml, Encoding.GetEncoding(webBrowser1.Document.Encoding));
now if you look to the txt file it should have the html tags you look for.
Even when using JavaScript to do a POST there is a POST somewhere in the JS which works the same way as button submit. You just have to dig to the place where the JS code posts and see how it does it. Then craft the same post in C#.
Take for example ASP.NET's own __doPostBack function
var theForm = document.forms['aspnetForm'];
if (!theForm) {
theForm = document.aspnetForm;
}
function __doPostBack(eventTarget, eventArgument) {
if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
theForm.__EVENTTARGET.value = eventTarget;
theForm.__EVENTARGUMENT.value = eventArgument;
theForm.submit();
}
}
You can see that it digs for the form sets several values for input fields and does a submit. Basically you need to fill the same values for the inputs and submit the same form and you've got the JS submit done yourself.
You need to capture the requests and headers those buttons are sending and simulate them with HttpWebRequest. You could also take a look at WatiN if you want to automate user actions on web sites.

HTTP Post in C# console app doesn't return the same thing as a browser request

I have a C# console app (.NET 2.0 framework) that does an HTTP post using the following code:
StringBuilder postData = new StringBuilder(100);
postData.Append("post.php?");
postData.Append("Key1=");
postData.Append(val1);
postData.Append("&Key2=");
postData.Append(val2);
byte[] dataArray = Encoding.UTF8.GetBytes(postData.ToString());
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create("http://example.com/");
httpRequest.Method = "POST";
httpRequest.ContentType = "application/x-www-form-urlencoded";
httpRequest.ContentLength = dataArray.Length;
Stream requestStream = httpRequest.GetRequestStream();
requestStream.Write(dataArray, 0, dataArray.Length);
requestStream.Flush();
requestStream.Close();
HttpWebResponse webResponse = (HttpWebResponse)httpRequest.GetResponse();
if (httpRequest.HaveResponse == true) {
Stream responseStream = webResponse.GetResponseStream();
StreamReader responseReader = new System.IO.StreamReader(responseStream, Encoding.UTF8);
String responseString = responseReader.ReadToEnd();
}
The outputs from this are:
webResponse.ContentLength = -1
webResponse.ContentType = text/html
webResponse.ContentEncoding is blank
The responseString is HTML with a title and body.
However, if I post the same URL into a browser (http://example.com/post.php?Key1=some_value&Key2=some_other_value), I get a small XML snippet like:
<?xml version="1.0" ?>
<RESPONSE RESULT="SUCCESS"/>
with none of the same HTML as in the application. Why are the responses so different? I need to parse the returned result which I am not getting in the HTML. Do I have to change how I do the post in the application? I don't have control over the server side code that accepts the post.
If you are indeed supposed to use the POST HTTP method, you have a couple things wrong. First, this line:
postData.Append("post.php?");
is incorrect. You want to post to post.php, you don't want post the value "post.php?" to the page. Just remove this line entirely.
This piece:
... WebRequest.Create("http://example.com/");
needs post.php added to it, so...
... WebRequest.Create("http://example.com/post.php");
Again this is assuming you are actually supposed to be POSTing to the specified page instead of GETing. If you are supposed to be using GET, then the other answers already supplied apply.
You'll want to get an HTTP sniffer tool like Fiddler and compare the headers that are being sent from your app to the ones being sent by the browser. There will be something different that is causing the server to return a different response. When you tweak your app to send the same thing browser is sending you should get the same response. (It could be user-agent, cookies, anything, but something is surely different.)
I've seen this in the past.
When you run from a browser, the "User-Agent" in the header is "Mozilla ...".
When you run from a program, it's different and generally specific to the language used.
I think you need to use a GET request, instead of POST. If the url you're using has querystring values (like ?Key1=some_value&Key2=some_other_value) then it's expecting a GET. Instead of adding post values to your webrequest, just put this data in the querystring.
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create("http://example.com/?val1=" + val1 + "&val2=" + val2);
httpRequest.Method = "GET";
httpRequest.ContentType = "application/x-www-form-urlencoded";
....
So, the result you're getting is different when you POST the data from your app because the server-side code has a different output when it can't read the data it's expecting in the querystring.
In your code you a specify the POST method which sends the data to the PHP file without putting the data in the web address. When you put the information in the address bar, that is not the POST method, that is the GET method. The name may be confusing, but GET just means that the data is being sent to the PHP file through the web address, instead of behind the scenes, not that it is supposed to get any information. When you put the address in the browser it is using a GET.
Create a simple html form and specify POST as the method and your url as the action. You will see that the information is sent without appearing in the address bar.
Then do the same thing but specify GET. You will see the information you sent in the address bar.
I believe the problem has something to do with the way your headers are set up for the WebRequest.
I have seen strange cases where attempting to simulate a browser by changing headers in the request makes a difference to the server.
The short answer is that your console application is not a web browser and the web server of example.com is expecting to interact with a browser.
You might also consider changing the ContentType to be "multipart/form-data".
What I find odd is that you are essentially posting nothing. The work is being done by the query string. Therefore, you probably should be using a GET instead of a POST.
Is the form expecting a cookie? That is another possible reason why it works in the browser and not from the console app.

Categories

Resources