I have tried both WebClient's DownloadSring and WebRequest+Stream to try and scrape a page (This one) and get the Raw Paste data from it. I have scoured the net but have found no answers.
I have this code:
WebRequest request = WebRequest.Create("http://pastebin.com/raw.php?i=" + textBox1.Text);
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
string pasteContent = "";
using (StreamReader sr = new StreamReader(data))
{
pasteContent = sr.ReadToEnd();
}
new Note().txtMain.Text += pasteContent;
new Note().txtMain.Refresh();
and I have multiple forms so I am editing Note's txtMain textbox to add the paste content but it seems to return nothing, no matter which function I use. I know cross-form editing works because I have multiple things that can return to it.
How can I scrape the raw data?
Thank you VERY much,
P
There is no problem in downloading the content of your site. You simply doesn't use the instance of the Node class you created.
var note = new Note();
note.txtMain.Text += pasteContent;
note.Show();
Related
I encountered som issue when using Sharepoint LIst Source on an SSIS package (too many redirection...)
My goal is to retrieve (whatever the type of result (xml, rows...) the content of a sharepoint list based on a specific view.
I only have a http://xxxxxx/_vti_bin/Lists.asmx url access.
I currently try this kind of code:
string serviceURL = "https://xxxxx/_vti_bin/Lists.asmx";
HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(serviceURL);
wr.UseDefaultCredentials = true;
wr.PreAuthenticate = true;
wr.Credentials = CredentialCache.DefaultCredentials;
wr.CookieContainer = new CookieContainer();
wr.AllowAutoRedirect = false;
// wr.MaximumAutomaticRedirections = 500;
HttpWebResponse ws = (HttpWebResponse)wr.GetResponse();
Stream str = ws.GetResponseStream();
// StreamReader readStream = new StreamReader(str, Encoding.UTF8);
string xmlData;
using (StreamReader sr = new StreamReader(str, Encoding.UTF8))
{
xmlData = sr.ReadToEnd();
sr.Close();
}
At this point it seems I have no longer error (too many redirection...) but now I'm block to get columns name and value, or xml value containing all data.
Using Web Service connected to the lists.asmx failed too, so I can't use this kind of solution.
Any ideas ?
thanks in advance.
regards,
I am trying to make an app, that can read a text from a website and store it in a string.
For example my app could open this random generator website, which would generate a random number string and then my program would read it and store it in a string.
Is that even possible?
I didn't get your goal but you may get the whole HTML page and parse it as you wish:
var httpClient = new HttpClient();
var htmlString = await httpClient.GetStringAsync(new Uri("http://google.com"));
You can also use that, and precise the encoding :
string text = null;
using (WebResponse response = WebRequest.Create(url).GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.GetEncoding("iso-8859-1")))
{
text = reader.ReadToEnd();
reader.Close();
}
response.Close();
}
I'm currently trying to read from an HTML file hosted online. My code should read to the end and then change xylosNotice1.Text to the source read from the HTML file.
Here is what I've tried:
WebRequest req = HttpWebRequest.Create("http://www.example.com/UpdateMe/updates.html");
req.Method = "GET";
string source;
using (StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream()))
{
source = reader.ReadToEnd();
}
Console.WriteLine(source);
xylosNotice1.Text = (source);
It doesn't update the textbox with the source.
Sorry guys stupid mistake, The code needed to be placed in the Main_Load part this is resolved.
I ran into a problem. I am .Net Developer and don't know about php, I am working on a CRM which has an API. My Client says it should be simple page should work with simple post. now i don't understand how i can do a simple Post in .Net. I have created an asp.net WebForm. All is working well. The only thing that i have problem with is that i have to return a list of parameters to response. I am using
Response.Write("100 - Click Recorded Successfully.");
but this return a full html Document with the parameter string at the top of the document. I saw one php Api which return only the prameter string like this with out HTML Document:
response=1
&responsetext=SUCCESS
&authcode=123456
&transactionid=2154229522
&avsresponse=N
&cvvresponse=N
&orderid=3592
&type=sale
&response_code=100
can some one suggest me any better way how i can do this. I found many article that explains how to do a simple Get Post in .Net but none of these solved my problem.
Update:
this is the code that i am using from another application to call the page and get response stream
string result = "";
WebRequest objRequest = WebRequest.Create(url + query);
objRequest.Method = "POST";
objRequest.ContentLength = 0;
objRequest.Headers.Add("x-ms-version", "2012-08-01");
objRequest.ContentType = "application/xml";
WebResponse objResponse = objRequest.GetResponse();
using (StreamReader sr =
new StreamReader(objResponse.GetResponseStream()))
{
result = sr.ReadToEnd();
// Close and clean up the StreamReader
sr.Close();
}
string temp = result;
where url + query is the address to my page. The result shows this code http://screencast.com/t/eKn4cckXc. I want to get the header line only, that is "100 - Click Recorded Successfully."
You have two options. First is to clear whatever response was already generated on the page, write the text, and then end the response so that nothing else added:
Response.Clear();
Response.ClearHeaders();
Response.AddHeader("Content-Type", "text/plain");
Response.Write(Request.Url.Query);
Response.End();
That is if you want to process it on the Page. However a better approach would be to implement Http Handler, in which case all you need to do is:
public void ProcessRequest
{
Response.AddHeader("Content-Type", "text/plain");
Response.Write(Request.Url.Query);
}
I want to generate html content based on a result returned by http url.
http://www.zillow.com/webservice/GetDeepSearchResults.htm?zws-id=X1-ZWz1c239bjatxn_5taq0&address=2114+Bigelow+Ave&citystatezip=Seattle%2C+WA
This page will give you some XML results. I want to convert to use that XML to generate HTML. I am not getting any idea where to start? Would someone offer any guidelines or sample code for asp.net?
For details: http://www.zillow.com/howto/api/GetDeepSearchResults.htm
To fetch the data you can use the HttpWebRequest class, this is an example I have to hand but it may be slightly overdone for your needs (and you need to make sure you're doing the right thing - I suspect the above to be a GET rather than a POST).
Uri baseUri = new Uri(this.RemoteServer);
HttpWebRequest rq = (HttpWebRequest)HttpWebRequest.Create(new Uri(baseUri, action));
rq.Method = "POST";
rq.ContentType = "application/x-www-form-urlencoded";
rq.Accept = "text/xml";
rq.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
Encoding encoding = Encoding.GetEncoding("UTF-8");
byte[] chars = encoding.GetBytes(body);
rq.ContentLength = chars.Length;
using (Stream stream = rq.GetRequestStream())
{
stream.Write(chars, 0, chars.Length);
stream.Close();
}
XDocument doc;
WebResponse rs = rq.GetResponse();
using (Stream stream = rs.GetResponseStream())
{
using (XmlTextReader tr = new XmlTextReader(stream))
{
doc = XDocument.Load(tr);
responseXml = doc.Root;
}
if (responseXml == null)
{
throw new Exception("No response");
}
}
return responseXml;
Once you've got the data back you need to render HTML, lots and lots of choices - if you just want to convert what you've got into HTML with minimal further processing then you can use XSLT - which is a question all on its own. If you need to do stuff with it then the question is too vague and you'll need to be more specific.
Create a xsl stylesheet, and inject the stylesheet element into the resulting xml from teh page