I'm use Asp.Net Mvc 4
www.hostname.com to my site from my report.hostname2.com
send and receive data to move directly to that address by the string bi codebehind.
querysstring not, because sending a very long string
I mean rapor.coskunoglu.net/Pdf address to send string data to move directly to
that address
PDF of the screen to make it appear so.
How can I do this?
Thank you, take it easy.
I'm sorry, my english is not good.
EDIT0:
I want to use POST.
sb -> my StringBuilder.
byte[] bytt = Encoding.UTF8.GetBytes(sb.ToString());
WebRequest wr = WebRequest.Create("http://report.hostname2.com/Pdf");
wr.ContentType = "application/x-www-form-urlencoded";
wr.ContentLength = bytt.Length;
wr.Method = "POST";
Stream st = wr.GetRequestStream();
st.Write(bytt, 0, bytt.Length);
st.Close();
After you send the POST I want to go to report.hostname2.com.
Did you see this my job?
One way to achieve that is to store the data you want to transmit into some commonly shared database between the two sites and then simply send an id to the other site as a query string so that it could retrieve the data. If you cannot use a shared database then all that's left is standard HTTP protocol means:
GET - query string - impractical in your case if the data is large
POST - generate a form and then submit this form to the remote site - could be a good solution for your case because you are not limited in size
Alternatively you could use a WebClient to POST some data:
StringBuilder sb = ... the data to send
using (var client = new WebClient())
{
var values = new NameValueCollection
{
{ "data", sb.ToString() }
};
byte[] result = client.UploadValues("http://report.hostname2.com/Pdf", values);
}
And then on the remote site you could read the data POST parameter from the request.
Related
Web-page data into the application
You can replicate the request the website makes to get a list of relevant numbers. The following code might be a good start.
var httpRequest = (HttpWebRequest) WebRequest.Create("<url>");
httpRequest.Method = "POST";
httpRequest.Accept = "application/json";
string postData = "{<json payload>}";
using (var streamWriter = new StreamWriter(httpRequest.GetRequestStream())) {
streamWriter.Write(postData);
}
var httpResponse = (HttpWebResponse) httpRequest.GetResponse();
string result;
using (var streamReader = new StreamReader(httpResponse.GetResponseStream())) {
result = streamReader.ReadToEnd();
}
Console.WriteLine(result);
Now, for the <url> and <json payload> values:
Open the web inspector in your browser.
Go to the Network tab.
Set it so Fetch/XHR/AJAX requests are shown.
Refresh the page.
Look for a request that you want to replicate.
Copy the request URL.
Copy the Payload (JSON data, to use it in your code you'll have to add a \ before every ")
Side note: The owner of the website you are making automated requests to might not be very happy about your tool, and you/it might be blocked if it makes too many requests in a short time.
I had a problem scraping data from a web page which I got a solution
Scrape data from web page that using iframe c#
My problem is that they changed the webpage which is now https://webportal.thpa.gr/ctreport/container/track and I don't think that is using iFrames and I cannot get any data back.
Can someone tell me if I can use the same method to get data from this webpage or should I use a different aproach?
I don't know how #coder_b found that I should use https://portal.thpa.gr/fnet5/track/index.php as web page and that I should use
var reqUrlContent =
hc.PostAsync(url,
new StringContent($"d=1&containerCode={reference}&go=1", Encoding.UTF8,
"application/x-www-form-urlencoded"))
.Result;
to pass the variables
EDIT: When I check the webpage there is an input which contains the number
input type="text" id="report_container_containerno"
name="report_container[containerno]" required="required"
class="form-control" minlength="11" maxlength="11" placeholder="E/K
για αναζήτηση" value="ARKU2215462"
Can I use something to pass with HtmlAgilityPack and then it should be easy to read the result
Also when I check the DocumentNode it seems to show me the cookies page that I should agree.
Can I bypass or auto allow cookies?
Try this:
public static string Download(string search)
{
var request = (HttpWebRequest)WebRequest.Create("https://webportal.thpa.gr/ctreport/container/track");
var postData = string.Format("report_container%5Bcontainerno%5D={0}&report_container%5Bsearch%5D=", search);
var data = Encoding.ASCII.GetBytes(postData);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
using (var response = (HttpWebResponse)request.GetResponse())
using (var stream = new StreamReader(response.GetResponseStream()))
{
return stream.ReadToEnd();
}
}
Usage:
var html = Download("ARKU2215462");
UPDATE
To find the post parameters to use, press F12 in the browser to show dev tools, then select Network tab. Now, fill the search input with your ARKU2215462 and press the button.
That do a request to the server to get the response. In that request, you can inspect both request and response. There are lots of request (styles, scripts, iamges...) but you want the html pages. In this case, look this:
This is the Form data requested. If you click in "view source", you get the data encoded like "report_container%5Bcontainerno%5D=ARKU2215462&report_container%5Bsearch%5D=", as you need in your code.
I am writing my own specific product crawler. Now there is a product selling website which uses post data for pages. I really really need to able to post data and read the response. But they are using asp.net validation and it is so messed up. I really could not figure how to properly post data and read. I am using htmlagilitypack. If it is possible to post data with htmlagilitypack and read the response it would be really really awesome.
Now this is the example page : http://www.hizlial.com/HizliListele.aspx?CatID=482643
When you opened the page look at the class "urun_listele"
You will see the options there
20 Ürün Listele
40 Ürün Listele
60 Ürün Listele
Tümünü Listele
Those numbers are product counts to be displayed. Tümünü listele means list all products. Now I really need to post data and get all of the products under that product category. I used firebug to debug and tried to code below but i still got default number of products
private void button11_Click(object sender, RoutedEventArgs e)
{
StringBuilder srBuilder = new StringBuilder();
AppendPostParameter(srBuilder, "ctl00$ContentPlaceHolder1$cmbUrunSayi", "full");
srBuilder = srBuilder.Replace("&", "", srBuilder.Length - 1, 1);
byte[] byteArray = Encoding.UTF8.GetBytes(srBuilder.ToString());
HttpWebRequest hWebReq = (HttpWebRequest)WebRequest.Create("http://www.hizlial.com/HizliListele.aspx?CatID=482643");
hWebReq.Method = "POST";
hWebReq.ContentType = "application/x-www-form-urlencoded";
using (Stream requestStream = hWebReq.GetRequestStream())
{
requestStream.Write(byteArray, 0, byteArray.Length);
}
HtmlDocument hd = new HtmlDocument();
using (HttpWebResponse response = (HttpWebResponse)hWebReq.GetResponse())
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
var htmlstring = sr.ReadToEnd();
}
}
}
static private void AppendPostParameter(StringBuilder sb, string name, string value)
{
sb.AppendFormat("{0}={1}&", name, HttpUtility.UrlEncode(value));
}
After i get the data I will load it to the htmlagilitypack HtmlDocument
Any help is appreciated.
C# 4.0 , wpf application, htmlagiltiypack
ASP .Net uses __EVENTTARGET and __EVENTARGUMENT fields to simulate Windows Forms behavior. To simulate Change event of combobox on server you need to append to form field to request they are __EVENTTARGET as 'ctl00$ContentPlaceHolder1$cmbUrunSayi' and __EVENTARGUMENT as ''.
If you look onchange code of combo and __doPostBack method you will understand what I mean. You can insert the code below after your declaration of srBuilder. That way code will work.
AppendPostParameter(srBuilder, "__EVENTTARGET", "ctl00$ContentPlaceHolder1$cmbUrunSayi");
AppendPostParameter(srBuilder, "__EVENTARGUMENT", string.Empty);
You will also need to extract __VIEWSTATE & __EVENTVALIDATION values. To get them just send a dummy request and extaract that values and cookies from that request and then append them into new one...
I am working on creating an Internet Explorer add on using BandOjects and C# Windows Forms Application, and am testing out parsing HTML source code. I have been currently parsing information based on the URL of the site.
I would like to get HTML source of the current page of an example site I have that uses a login. if I use the URL of the page I am on, it will always grab the source of the login page rather than the actual page, as my app doesn't recognize that I logged in. would i need to store my login credentials for the site using some kind of api? or is there a way to grab the current page of the HTML regardless? I would prefer the latter as it seemingly would be less trouble. Thanks!
I use this method in one of my apps:
private static string RetrieveData(string url)
{
// used to build entire input
var sb = new StringBuilder();
// used on each read operation
var buf = new byte[8192];
try
{
// prepare the web page we will be asking for
var request = (HttpWebRequest)
WebRequest.Create(url);
/* Using the proxy class to access the site
* Uri proxyURI = new Uri("http://proxy.com:80");
request.Proxy = new WebProxy(proxyURI);
request.Proxy.Credentials = new NetworkCredential("proxyuser", "proxypassword");*/
// execute the request
var response = (HttpWebResponse)
request.GetResponse();
// we will read data via the response stream
Stream resStream = response.GetResponseStream();
string tempString = null;
int count = 0;
do
{
// fill the buffer with data
count = resStream.Read(buf, 0, buf.Length);
// make sure we read some data
if (count != 0)
{
// translate from bytes to ASCII text
tempString = Encoding.ASCII.GetString(buf, 0, count);
// continue building the string
sb.Append(tempString);
}
} while (count > 0); // any more data to read?
}
catch(Exception exception)
{
MessageBox.Show(#"Failed to retrieve data from the network. Please check you internet connection: " +
exception);
}
return sb.ToString();
}
You have to just pass the url of the web page for which you need to retrieve the code.
For example:
string htmlSourceGoggle = RetrieveData("www.google.com")
Note: You can get un-comment the proxy configuration if you use proxy to access the internet. Replace the proxy address, username and password with the one you use.
For logging in via code. check this: Login to website, via C#
I'm using the following code to POST data to a URL on button click. I need to be able to send a form name along with this data. Any suggestions?
string url = "http://www.someurl.com";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
string proxy = null;
string data = String.Format("{0}={1}&{2}={3}&{4}={5}&{6}={7}&{8}={9}&{10}={11}",
txtName.ClientID, txtName.Text,
txtEmail.ClientID, txtEmail.Text,
txtLanguages.ClientID, txtLanguages.Text,
txtPhone.ClientID, txtPhone.Text,
txtAdditional.ClientID, txtAdditional.Text);
byte[] buffer = Encoding.UTF8.GetBytes(data);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = buffer.Length;
req.Proxy = new WebProxy(proxy, true); // ignore for local addresses
req.CookieContainer = new CookieContainer(); // enable cookies
Stream reqst = req.GetRequestStream(); // add form data to request stream
reqst.Write(buffer, 0, buffer.Length);
reqst.Flush();
reqst.Close();
If you mean the action, just append it to the URL.
You can append any key/value pair you want to the POST data - doesn't matter whether it's a control name, form name, form action or nowhere on the page at all.
By the way, if you're going to construct POST data manually you should URL-encode the values, eg.
string data = String.Format("{0}={1}&{2}={3}&{4}={5}&{6}={7}&{8}={9}&{10}={11}",
txtName.ClientID, HttpUtility.UrlEncode(txtName.Text),
...
If you are managing different activities with different forms, you could also send a hidden variable, between your form with the kind of activity you are doing, so you can evaluate the hidden hidden value, and act acording to it.
Or put a value and name to the submit button... it will come to your script as apost variable as well.