I'm currently trying to read from an HTML file hosted online. My code should read to the end and then change xylosNotice1.Text to the source read from the HTML file.
Here is what I've tried:
WebRequest req = HttpWebRequest.Create("http://www.example.com/UpdateMe/updates.html");
req.Method = "GET";
string source;
using (StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream()))
{
source = reader.ReadToEnd();
}
Console.WriteLine(source);
xylosNotice1.Text = (source);
It doesn't update the textbox with the source.
Sorry guys stupid mistake, The code needed to be placed in the Main_Load part this is resolved.
Related
First of all what I want to do is legal (since they let you download the pdf).
I just wanted to make a faster and automatic method of downloading the pdf.
For example: http://www.lasirena.es/article/&path=10_17&ID=782
It has an embedded flash pdf and when I download that page source code, the link to the pdf:
http://issuu.com/lasirena/docs/af_fulleto_setembre_andorra_sense_c?e=3360093/9079351
Doesn't show up, the only thing that I have on the source code is this: 3360093/9079351
I tried to find a way to build the pdf link from it, but I can't find the name "af_fulleto_setembre_andorra_sense_c" anywhere...
I've made plenty of automatic downloads like this, but it's the first time that I can't build or get the pdf link and I can't seem to find a way, is it even possible?
I tried to try and find jpg's links but without success either. Either way (jpg or pdf) is fine...
PS: the Document ID doesn't show on the downloaded source code either.
Thank you.
I thought a workaround for this, some might not consider this a solution but in my case works fine because it depends on the ISSUU publisher account.
The Solution itself is making a Request to ISSUU API connected with the publisher account I'm looking for.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://api.issuu.com/query?action=issuu.documents.list" +
"&apiKey=Inser Your API Key" +
"&format=json" +
"&documentUsername=User of the account you want to make a request" +
"&pageSize=100&resultOrder=asc" +
"&responseParams=name,documentId,pageCount" +
"&username=Insert your ISSUU username" +
"&token=Insert Your Token here");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.Accept = "application/json";
try
{
using (WebResponse response = request.GetResponse())
{
var responseValue = string.Empty;
// grab the response
using (var responseStream = response.GetResponseStream())
{
using (var reader = new StreamReader(responseStream))
{
responseValue = reader.ReadToEnd();
}
}
if (responseValue != "")
{
List<string> lista_linkss = new List<string>();
JObject ApiRequest = JObject.Parse(responseValue);
//// get JSON result objects into a list
IList<JToken> results = ApiRequest["rsp"]["_content"]["result"]["_content"].Children()["document"].ToList();
for (int i = 0; i < results.Count(); i++)
{
Folheto folheto = new Folheto();
folheto.name = results[i]["name"].ToString();
folheto.documentId = results[i]["documentId"].ToString();
folheto.pageCount = Int32.Parse(results[i]["pageCount"].ToString());
string _date = Newtonsoft.Json.JsonConvert.SerializeObject(results[i]["uploadTimestamp"], Formatting.None, new IsoDateTimeConverter() { DateTimeFormat = "yyyy-MM-dd hh:mm:ss" }).Replace(#"""", string.Empty);
folheto.uploadTimestamp = Convert.ToDateTime(_date);
if (!lista_nomes_Sirena.Contains(folheto.name))
{
list.Add(folheto);
}
}
}
}
}
catch (WebException ex)
{
// Handle error
}
You have to pay attention to the Parameter "pageSize" the maximum permitted by the API is 100, this means the maximum number of results you get is 100, since the account I'm following has around 240 pdf's, I used this request once with the Parameter "resultOrder = asc" and another time with the value "resultOrder=desc".
This allowed me to get the first 100 pdfs and the latest 100 pdfs inserted.
Since I didn't need a history but just the pdf's they will be sending out from now, it didn't make a difference.
Finalizing my code I'm sending all the document's ID's to a sql database I made, and when I start the program, I make a check to see if the ID was already downloaded, if not it downloads the pdf, if yes it doesn't.
Hope someone can find this work around useful
I have tried both WebClient's DownloadSring and WebRequest+Stream to try and scrape a page (This one) and get the Raw Paste data from it. I have scoured the net but have found no answers.
I have this code:
WebRequest request = WebRequest.Create("http://pastebin.com/raw.php?i=" + textBox1.Text);
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
string pasteContent = "";
using (StreamReader sr = new StreamReader(data))
{
pasteContent = sr.ReadToEnd();
}
new Note().txtMain.Text += pasteContent;
new Note().txtMain.Refresh();
and I have multiple forms so I am editing Note's txtMain textbox to add the paste content but it seems to return nothing, no matter which function I use. I know cross-form editing works because I have multiple things that can return to it.
How can I scrape the raw data?
Thank you VERY much,
P
There is no problem in downloading the content of your site. You simply doesn't use the instance of the Node class you created.
var note = new Note();
note.txtMain.Text += pasteContent;
note.Show();
I ran into a problem. I am .Net Developer and don't know about php, I am working on a CRM which has an API. My Client says it should be simple page should work with simple post. now i don't understand how i can do a simple Post in .Net. I have created an asp.net WebForm. All is working well. The only thing that i have problem with is that i have to return a list of parameters to response. I am using
Response.Write("100 - Click Recorded Successfully.");
but this return a full html Document with the parameter string at the top of the document. I saw one php Api which return only the prameter string like this with out HTML Document:
response=1
&responsetext=SUCCESS
&authcode=123456
&transactionid=2154229522
&avsresponse=N
&cvvresponse=N
&orderid=3592
&type=sale
&response_code=100
can some one suggest me any better way how i can do this. I found many article that explains how to do a simple Get Post in .Net but none of these solved my problem.
Update:
this is the code that i am using from another application to call the page and get response stream
string result = "";
WebRequest objRequest = WebRequest.Create(url + query);
objRequest.Method = "POST";
objRequest.ContentLength = 0;
objRequest.Headers.Add("x-ms-version", "2012-08-01");
objRequest.ContentType = "application/xml";
WebResponse objResponse = objRequest.GetResponse();
using (StreamReader sr =
new StreamReader(objResponse.GetResponseStream()))
{
result = sr.ReadToEnd();
// Close and clean up the StreamReader
sr.Close();
}
string temp = result;
where url + query is the address to my page. The result shows this code http://screencast.com/t/eKn4cckXc. I want to get the header line only, that is "100 - Click Recorded Successfully."
You have two options. First is to clear whatever response was already generated on the page, write the text, and then end the response so that nothing else added:
Response.Clear();
Response.ClearHeaders();
Response.AddHeader("Content-Type", "text/plain");
Response.Write(Request.Url.Query);
Response.End();
That is if you want to process it on the Page. However a better approach would be to implement Http Handler, in which case all you need to do is:
public void ProcessRequest
{
Response.AddHeader("Content-Type", "text/plain");
Response.Write(Request.Url.Query);
}
Does anyone (please) know how to do this? I thought that there would be an easy way to achieve this but can't find anything about saving the contents of WebBrowser HTML.
You might try something like this:
(Assuming C# 4 and WPF 4)
dynamic doc = webBrowser.Document;
var htmlText = doc.documentElement.InnerHtml;
Works for me...
Yous should use HttpWebRequest and HttpWebResponse objects. Simple sample (found in web, tested, working):
HttpWebRequest myWebRequest = (HttpWebRequest)HttpWebRequest.Create(#"http://www.[pagename].com");
myWebRequest.Method = "GET";
HttpWebResponse myWebResponse = (HttpWebResponse)myWebRequest.GetResponse();
StreamReader myWebSource = new StreamReader(myWebResponse.GetResponseStream());
string myPageSource = string.Empty;
myPageSource = myWebSource.ReadToEnd();
myWebResponse.Close();
I am creating a HttpWebRequest object from another aspx page to save the response stream to my data store. The Url I am using to create the HttpWebRequest object has querystring to render the correct output. When I browse to the page using any old browser it renders correctly. When I try to retrieve the output stream using the HttpWebResponse.GetResponseStream() it renders my built in error check.
Why would it render correctly in the browser, but not using the HttpWebRequest and HttpWebResponse objects?
Here is the source code:
Code behind of target page:
protected void PageLoad(object sender, EventsArgs e)
{
string output = string.Empty;
if(Request.Querystring["a"] != null)
{
//generate output
output = "The query string value is " + Request.QueryString["a"].ToString();
}
else
{
//generate message indicating the query string variable is missing
output = "The query string value was not found";
}
Response.Write(output);
}
Code behind of page creating HttpWebRequest object
string url = "http://www.mysite.com/mypage.aspx?a=1";
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url)
//this if statement was missing from original example
if(User.Length > 0)
{
request.Credentials = new NetworkCredentials("myaccount", "mypassword", "mydomain");
request.PreAuthenticate = true;
}
request.UserAgent = Request.UserAgent;
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Stream resStream = response.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(resStream, encode, true, 2000);
int count = readStream.Read(read, 0, read.Length);
string str = Server.HtmlEncode(" ");
while (count > 0)
{
// Dumps the 256 characters on a string and displays the string to the console.
string strRead = new string(read, 0, count);
str = str.Replace(str, str + Server.HtmlEncode(strRead.ToString()));
count = readStream.Read(read, 0, 256);
}
// return what was found
result = str.ToString();
resStream.Close();
readStream.Close();
Update
#David McEwing - I am creating the HttpWebRequest with the full page name. The page is still generating the error output. I updated the code sample of the target page to demonstrate exactly what I am doing.
#Chris Lively - I am not redirecting to an error page, I generate a message indicating the query string value was not found. I updated the source code example.
Update 1:
I tried using Fiddler to trace the HttpWebRequest and it did not show up in the Web Sessions history window. Am I missing something in my source code to get a complete web request and response.
Update 2:
I did not include the following section of code in my example and it was culprit causing the issue. I was setting the Credentials property of the HttpWebRequest with a sevice account instead of my AD account which was causing the issue.
I updated my source code example
What webserver are you using? I can remember at one point in my past when doing something with IIS there was an issue where the redirect between http://example.com/ and http://example.com/default.asp dropped the query string.
Perhaps run Fiddler (or a protocol sniffer) and see if there is something happening that you aren't expecting.
Also check if passing in the full page name works. If it does the above is almost certainly the problem.
Optionally, you can try to use the AllowAutoRedirect property of the HttpRequestObject.
I need to replace the following line of code:
request.Credentials = new NetworkCredentials("myaccount", "mypassword", "mydomain");
with:
request.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;