I am new to C# and just messing around with it myself, now, i have been trying to create a WinForm that can post some parameters in a webpage and do something something on the resultant webpage obtained. Now I have accomplished this on a page that uses POST method, But i am not able to do so with A webpage that has a html code like this :
<form method="post" action="test.asp" name=FrontPage_Form1 onsubmit="return FrontPage_Form1_Validator(this)">
<div align="center"><center><p>
<input name="name" size="8" maxlength=8><font color="#faebd7">---
Now i don't how How To implement this "ONSUBMIT" with HttpWebRequest..
This is my current Code :
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://cbseresults.nic.in/aieee/cbseaieee.asp");
request.Method = "POST";
string r = "regno=" + rno.ToString();
Bytes = Encoding.UTF8.GetBytes(r);
request.ContentLength = Bytes.Length;
request.ContentType = "application/x-www-form-urlencoded";
RequestStream = request.GetRequestStream();
RequestStream.Write(Bytes, 0, Bytes.Length);
RequestStream.Close();
Response = (HttpWebResponse)request.GetResponse();
StreamReader ResponseStream = new StreamReader(Response.GetResponseStream(), Encoding.ASCII);
string Result = ResponseStream.ReadToEnd();
ResponseStream.Close();
But its not working, Any Help is greatly appreciated...
Try using Fiddler in order to understand what the page is sending and receiving from the server.
Then make the request as it is shown in fiddler...
You can also use WebClient to open some pages or sending and receiving data from server.
There are some ways to click on buttons or links:
Use a WebBrowser object in your app and Iterate through objects on the page by using SelectNextControl method of WebBrowser object and then sending Enter key like so: SendKeys.Send("{Enter}");
Using JavaScript and invoking functions and reading elements using getElementById and some other methods
Related
I had a problem scraping data from a web page which I got a solution
Scrape data from web page that using iframe c#
My problem is that they changed the webpage which is now https://webportal.thpa.gr/ctreport/container/track and I don't think that is using iFrames and I cannot get any data back.
Can someone tell me if I can use the same method to get data from this webpage or should I use a different aproach?
I don't know how #coder_b found that I should use https://portal.thpa.gr/fnet5/track/index.php as web page and that I should use
var reqUrlContent =
hc.PostAsync(url,
new StringContent($"d=1&containerCode={reference}&go=1", Encoding.UTF8,
"application/x-www-form-urlencoded"))
.Result;
to pass the variables
EDIT: When I check the webpage there is an input which contains the number
input type="text" id="report_container_containerno"
name="report_container[containerno]" required="required"
class="form-control" minlength="11" maxlength="11" placeholder="E/K
για αναζήτηση" value="ARKU2215462"
Can I use something to pass with HtmlAgilityPack and then it should be easy to read the result
Also when I check the DocumentNode it seems to show me the cookies page that I should agree.
Can I bypass or auto allow cookies?
Try this:
public static string Download(string search)
{
var request = (HttpWebRequest)WebRequest.Create("https://webportal.thpa.gr/ctreport/container/track");
var postData = string.Format("report_container%5Bcontainerno%5D={0}&report_container%5Bsearch%5D=", search);
var data = Encoding.ASCII.GetBytes(postData);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
using (var response = (HttpWebResponse)request.GetResponse())
using (var stream = new StreamReader(response.GetResponseStream()))
{
return stream.ReadToEnd();
}
}
Usage:
var html = Download("ARKU2215462");
UPDATE
To find the post parameters to use, press F12 in the browser to show dev tools, then select Network tab. Now, fill the search input with your ARKU2215462 and press the button.
That do a request to the server to get the response. In that request, you can inspect both request and response. There are lots of request (styles, scripts, iamges...) but you want the html pages. In this case, look this:
This is the Form data requested. If you click in "view source", you get the data encoded like "report_container%5Bcontainerno%5D=ARKU2215462&report_container%5Bsearch%5D=", as you need in your code.
I want to scrape specific websites.For example in that website(https://www.accessdata.fda.gov/scripts/cder/cliil/index.cfm) in index page when you select a data field(you can choose country) and in Country keyword you can choose USA it navigate search page( https://www.accessdata.fda.gov/scripts/cder/cliil/dsp_Search.cfm ) I want to download search page.I want to scrape it.But there is no query string.How can I do this?
Are there any solution that I can post form in index with parameters?
Edited:
I use webrequest but it does not show page with data.Are my parameters is false?
here is my code
System.Net.WebRequest request1 = System.Net.WebRequest.Create("https://www.accessdata.fda.gov/scripts/cder/cliil/dsp_Search.cfm");
var Deger1 = "{'DataField':'COUNTRY','COUNTRY':'USA','Keywords':'','Submit':'Submit'}";
request1.Method = "POST";
byte[] byteArray = System.Text.Encoding.UTF8.GetBytes(Deger1);
request1.ContentType = "text/xml";
request1.ContentLength = byteArray.Length;
Stream dataStream = request1.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
System.Net.WebResponse response = request1.GetResponse();
Console.WriteLine(((System.Net.HttpWebResponse)response).StatusDescription);
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream, System.Text.Encoding.UTF8, true);
string responseFromServer = reader.ReadToEnd();
reader.Close();
dataStream.Close();
response.Close();
I don't know how to make http requests with #c but to get your requirement fulfilled I think the below information is enough.
1. FormData={
'DataField':'COUNTRY','COUNTRY':'USA','Keywords':'','Submit':'Submit'
}
2. You should make a post request with the below url along with the above form data.
"https://www.accessdata.fda.gov/scripts/cder/cliil/dsp_Search.cfm"
I've tested it in other language and found it working.
Btw, I've got above information by satisfying the below parameter in the search field using the following url:
url = "https://www.accessdata.fda.gov/scripts/cder/cliil/index.cfm"
Search Fields:
1. Country
2. USA
I need to load website using webview in metro apps. I could post login parameters using following method. how to load the same in webview??????
string post_data = "userName=test123&password=test#321";
// this is where we will send it
string uri = "http://tesproject.com";
// create a request
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
// turn our request string into a byte stream
byte[] postBytes = Encoding.UTF8.GetBytes(post_data);
// this is important - make sure you specify type this way
request.ContentType = "application/x-www-form-urlencoded";
Stream requestStream = await request.GetRequestStreamAsync();
// now send it
requestStream.Write(postBytes, 0, postBytes.Length);
// grab te response and print it out to the console along with the status code
WebResponse response = await request.GetResponseAsync();
WebView class doesn't support navigating to an URL with POST parameters directly.
You could however use your code and the WebResponse to get the HTML. And then use the NavigateToString method of the WebView class to render the HTML:
HttpWebResponse httpResponse= (HttpWebResponse)response;
StreamReader reader=new StreamReader(httpResponse.GetResponseStream());
string htmlString= reader.ReadToEnd();
if (!string.IsNullOrEmpty(htmlString))
webView.NavigateToString(htmlString);
Or you could create your own HTML with a form with your post values and some javascript to submit the form, but that might be a bit more cumbersome.
I'm trying to log onto the following website using HttpWebRequest: http://mostanmeldung.moessinger.at/login.php
Texts are in German, but they don't really matter. If you look at the source code (which by the way was not written by me, so don't blame me for its bad style :P), you will see a form tag that contains two input tags. The name of the first one is "BN" (username), and the name of the second one is "PW" (password). I am trying to send data containing values for these two inputs to the webserver using the HttpWebRequest class. However, posting the values redirects the request to another page called "einloggen.php". On that site I am told whether my login was successful.
My problem is that I am able to send the data without any problems, however, all I receive is the content of "login.php", the site you have to enter your username and password on.
This is what my code looks like:
string post = String.Format(PostPattern, Username, Password);
byte[] postBytes = Encoding.ASCII.GetBytes(post);
CookieContainer cookies = new CookieContainer();
// "Address": http://mostanmeldung.moessinger.at/login.php
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Address);
req.CookieContainer = cookies;
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = postBytes.Length;
req.AllowAutoRedirect = true;
MessageBox.Show(post); // shows me "BN=boop;PW=hi"
Stream reqStream = req.GetRequestStream();
reqStream.Write(postBytes, 0, postBytes.Length);
reqStream.Close();
WebResponse res;
if (/*req.HaveResponse &&*/ (res = req.GetResponse()) != null)
{
StreamReader reader = new StreamReader(res.GetResponseStream());
MessageBox.Show(reader.ReadToEnd());
return AuthResult.Success;
}
return AuthResult.NoResponse;
The message box at line 22 (5 lines before the end) shows me the content of "login.php" instead of "einloggen.php" which I am redirected to. Why is that?
The ACTION on that form points to einloggen.php, not login.php, so you need to send your POST data to einloggen.php instead.
I need to call a web and retrieve the resulting data from the model in my asp.net mvc application. When accessed on the web, the form looks like this:
<form id="textEntryForm" name="textEntryForm" method="post" action="/project/evaluate_to_pdf">
<textarea id="p" rows="20" name="p" cols="132"/><br/>
<input type="button" value="parse" name="do_parse" onclick="new Ajax.Updater('parsedProject','/project/parse',{asynchronous:true,evalScripts:true,on404:function(e){alert('not found!')},parameters:Form.serialize(this.form)});return false"/>
<input type="button" value="evaluate_to_html" name="do_evaluate_to_html" onclick="new Ajax.Updater('parsedProject','/project/evaluate_to_html',{asynchronous:true,evalScripts:true,on404:function(e){alert('not found!')},parameters:Form.serialize(this.form)});return false"/>
<input type="button" value="evaluate" name="do_evaluate" onclick="new Ajax.Updater('parsedProject','/project/evaluate',{asynchronous:true,evalScripts:true,on404:function(e){alert('not found!')},parameters:Form.serialize(this.form)});return false"/>
<input type="button" value="evaluate to pdf source" name="do_evaluate_to_pdf_source" onclick="new Ajax.Updater('parsedProject','/project/evaluate_to_pdf_source',{asynchronous:true,evalScripts:true,on404:function(e){alert('not found!')},parameters:Form.serialize(this.form)});return false"/>
<input type="submit" id="do_evaluate_to_pdf" value="evaluate_to_pdf" name="do_evaluate_to_pdf"/>
</form>
I need to pass the data that would be entered into textarea id="p". How do add that in, using a WebClient to connect?
Thanks!
Edit This isn't for testing purposes, I need to retrieve the data for use in my application.
I just used this: http://www.eggheadcafe.com/community/aspnet/2/69261/you-can-use-the-webclient.aspx
Another option is the Lightweight Test Automation Framework by Microsoft <- Here Steve Sanderson applies it to MVC.
(source: codeville.net)
(source: codeville.net)
You create a Stream and pass it into your HttpWebRequest.
// Create a request using a URL that can receive a post.
WebRequest request =
WebRequest.Create("http://www.contoso.com/PostAccepter.aspx ");
// Set the Method property of the request to POST.
request.Method = "POST";
// Create POST data and convert it to a byte array.
string postData = "p=Some text here from the textarea";
byte[] byteArray = Encoding.UTF8.GetBytes (postData);
// Set the ContentType property of the WebRequest.
request.ContentType = "application/x-www-form-urlencoded";
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
Stream dataStream = request.GetRequestStream ();
// Write the data to the request stream.
dataStream.Write (byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close ();
// Get the response.
WebResponse response = request.GetResponse ();
// Display the status.
Console.WriteLine (((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream ();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader (dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd ();
// Display the content.
Console.WriteLine (responseFromServer);
// Clean up the streams.
reader.Close ();
dataStream.Close ();
response.Close ();
http://msdn.microsoft.com/en-us/library/debx8sh9.aspx
These things have a habit of becoming more and more complex, for example should you need to handle cookies, authentication or multipart form uploads for uploading files etc. I suggest using curl (http://sourceforge.net/projects/libcurl-net/)
Something like this:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
string data = "&p=" + dataThatNeedsToBeInTextArea;
byte[] byteArray = Encoding.UTF8.GetBytes (data);
req.ContentLength = byteArray.Length;
Stream stream= req.GetRequestStream ();
stream.Write (byteArray, 0, byteArray.Length);
stream.Close ();
StreamReader streamIn = new StreamReader(req.GetResponse().GetResponseStream());
string response = streamIn.ReadToEnd();
streamIn .Close();
Agreeing with #wentbackward, WatiN is another alternative.