I want to scrape specific websites.For example in that website(https://www.accessdata.fda.gov/scripts/cder/cliil/index.cfm) in index page when you select a data field(you can choose country) and in Country keyword you can choose USA it navigate search page( https://www.accessdata.fda.gov/scripts/cder/cliil/dsp_Search.cfm ) I want to download search page.I want to scrape it.But there is no query string.How can I do this?
Are there any solution that I can post form in index with parameters?
Edited:
I use webrequest but it does not show page with data.Are my parameters is false?
here is my code
System.Net.WebRequest request1 = System.Net.WebRequest.Create("https://www.accessdata.fda.gov/scripts/cder/cliil/dsp_Search.cfm");
var Deger1 = "{'DataField':'COUNTRY','COUNTRY':'USA','Keywords':'','Submit':'Submit'}";
request1.Method = "POST";
byte[] byteArray = System.Text.Encoding.UTF8.GetBytes(Deger1);
request1.ContentType = "text/xml";
request1.ContentLength = byteArray.Length;
Stream dataStream = request1.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
System.Net.WebResponse response = request1.GetResponse();
Console.WriteLine(((System.Net.HttpWebResponse)response).StatusDescription);
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream, System.Text.Encoding.UTF8, true);
string responseFromServer = reader.ReadToEnd();
reader.Close();
dataStream.Close();
response.Close();
I don't know how to make http requests with #c but to get your requirement fulfilled I think the below information is enough.
1. FormData={
'DataField':'COUNTRY','COUNTRY':'USA','Keywords':'','Submit':'Submit'
}
2. You should make a post request with the below url along with the above form data.
"https://www.accessdata.fda.gov/scripts/cder/cliil/dsp_Search.cfm"
I've tested it in other language and found it working.
Btw, I've got above information by satisfying the below parameter in the search field using the following url:
url = "https://www.accessdata.fda.gov/scripts/cder/cliil/index.cfm"
Search Fields:
1. Country
2. USA
Related
I have a problem in "translating" this HTML page into c # code.
I have to pass an xml file to a production machine and I would like to do it from a c # application instead of manually, as shown in the attached screenshot.
I wrote this code c #:
WebRequest request = WebRequest.Create(#"http://Machine_IP/JTI/");
byte[] bytes;
bytes = System.Text.Encoding.ASCII.GetBytes(d.InnerXml);
request.ContentType = "text/xml; encoding='utf-8'";
request.ContentLength = bytes.Length;
request.Method = "POST";
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();
HttpWebResponse response;
response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
string responseStr = new StreamReader(responseStream).ReadToEnd();
}
I can't figure out how to pass name = "ImportJobs".
If I make a "generic" POST, the machine does not receive the xml file.
I should do a POST as an ImportJobs method.
The supplier, as specifications, gives me the following:
Request
HTTP method: POST
Encryption type (Enctype): Multipart/form-data
URL: http://MachineName/JTI
Command: ImportJobs
Parameters: None
Multipart data section:The XML job description
Response
Data none
Can anyone help me?
thanks a lot
HTML Example
I've been fiddling quite a bit with my uploading to vimeo.
I've made a ticket request.
I've uploaded the file.
I've checked the file if its uploaded.
I need to run the method DELETE with the complete_uri response i should get from my ticket.
However, im not receiving any complete_URI from the ticket response.
Here is my code:
public static dynamic GenerateTicket()
{
const string apiUrl = "https://api.vimeo.com/me/videos?type=streaming";
var req = (HttpWebRequest)WebRequest.Create(apiUrl);
req.Accept = "application/vnd.vimeo.*+json;version=3.0";
req.Headers.Add(HttpRequestHeader.Authorization, "bearer " + AccessToken);
req.Method = "POST";
var res = (HttpWebResponse)req.GetResponse();
var dataStream = res.GetResponseStream();
var reader = new StreamReader(dataStream);
var result = Json.Decode(reader.ReadToEnd());
return result;
}
This response gives me:
form
ticket_id
upload_link
upload_link_secure
uri
user
In order to finish my upload i need to run step 4 in this guide: https://developer.vimeo.com/api/upload
Sending parameter type=streaming as body:
ASCIIEncoding encoding = new ASCIIEncoding();
string stringData = "type=streaming"; //place body here
byte[] data = encoding.GetBytes(stringData);
req.Method = "PUT";
req.ContentLength = data.Length;
Stream newStream = req.GetRequestStream();
newStream.Write(data, 0, data.Length);
newStream.Close();
At the moment, type=streaming must be sent in the body of the request, not as a url parameter.
This will probably change to allow either option.
the important point is :
"The first thing you need to do is request upload access for your application. You can do so from your My Apps page."
If you get all values without complete_uri, it means: you dont have an upload access token. So go to your apps and make an upload request
I need to load website using webview in metro apps. I could post login parameters using following method. how to load the same in webview??????
string post_data = "userName=test123&password=test#321";
// this is where we will send it
string uri = "http://tesproject.com";
// create a request
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
// turn our request string into a byte stream
byte[] postBytes = Encoding.UTF8.GetBytes(post_data);
// this is important - make sure you specify type this way
request.ContentType = "application/x-www-form-urlencoded";
Stream requestStream = await request.GetRequestStreamAsync();
// now send it
requestStream.Write(postBytes, 0, postBytes.Length);
// grab te response and print it out to the console along with the status code
WebResponse response = await request.GetResponseAsync();
WebView class doesn't support navigating to an URL with POST parameters directly.
You could however use your code and the WebResponse to get the HTML. And then use the NavigateToString method of the WebView class to render the HTML:
HttpWebResponse httpResponse= (HttpWebResponse)response;
StreamReader reader=new StreamReader(httpResponse.GetResponseStream());
string htmlString= reader.ReadToEnd();
if (!string.IsNullOrEmpty(htmlString))
webView.NavigateToString(htmlString);
Or you could create your own HTML with a form with your post values and some javascript to submit the form, but that might be a bit more cumbersome.
I am working with a REST API and I am trying to do a PUT method to it. I found this code I was going to give a try:
static void Main()
{
string xml = "<xml>...</xml>";
byte[] arr = System.Text.Encoding.UTF8.GetBytes(xml);
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://localhost/");
request.Method = "PUT";
request.ContentType = "text/xml";
request.ContentLength = arr.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(arr, 0, arr.Length);
dataStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string returnString = response.StatusCode.ToString();
Console.WriteLine(returnString);
}
One thing I want to do if possible, and can't seem to find anything about it. I would like to pass the data of text fields so, txtEmail.Text, txtFirstName.Text, etc. Is this possible? If, so how would I go about doing this? Does this code look like it would work? Unfortunately the API I'm working with has very very little documentation. Thanks!
The code lines
Stream dataStream = request.GetRequestStream();
dataStream.Write(arr, 0, arr.Length);
dataStream.Close();
do write something to the remote website. The request stream is the way to provide data to the server, so you would create a string / object that you write to the stream to transfer to the server. In your example <xml>...</xml> is sent to the server.
I'm trying to log onto the following website using HttpWebRequest: http://mostanmeldung.moessinger.at/login.php
Texts are in German, but they don't really matter. If you look at the source code (which by the way was not written by me, so don't blame me for its bad style :P), you will see a form tag that contains two input tags. The name of the first one is "BN" (username), and the name of the second one is "PW" (password). I am trying to send data containing values for these two inputs to the webserver using the HttpWebRequest class. However, posting the values redirects the request to another page called "einloggen.php". On that site I am told whether my login was successful.
My problem is that I am able to send the data without any problems, however, all I receive is the content of "login.php", the site you have to enter your username and password on.
This is what my code looks like:
string post = String.Format(PostPattern, Username, Password);
byte[] postBytes = Encoding.ASCII.GetBytes(post);
CookieContainer cookies = new CookieContainer();
// "Address": http://mostanmeldung.moessinger.at/login.php
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Address);
req.CookieContainer = cookies;
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = postBytes.Length;
req.AllowAutoRedirect = true;
MessageBox.Show(post); // shows me "BN=boop;PW=hi"
Stream reqStream = req.GetRequestStream();
reqStream.Write(postBytes, 0, postBytes.Length);
reqStream.Close();
WebResponse res;
if (/*req.HaveResponse &&*/ (res = req.GetResponse()) != null)
{
StreamReader reader = new StreamReader(res.GetResponseStream());
MessageBox.Show(reader.ReadToEnd());
return AuthResult.Success;
}
return AuthResult.NoResponse;
The message box at line 22 (5 lines before the end) shows me the content of "login.php" instead of "einloggen.php" which I am redirected to. Why is that?
The ACTION on that form points to einloggen.php, not login.php, so you need to send your POST data to einloggen.php instead.