I'm trying on form load, to make it count the number of lines in a pastebin raw & return the value to a textbox. Been racking my brains and still cant figure it out.
textBox1.Text = new WebClient().DownloadString("yourlink").
I'm expanding my comment to an answer.
As already mentioned, you need a HttpRequest or WebRequest to get the content of your string.
Maybe new WebClient().DownloadString(url);, but I prefer to use the WebRequest since it's also supported in .NET Core.
What you need to do is, extract the content of the RAW TextArea object from html. I know, people will probably hate me for that, but I used regex for that task. Alternatively you can use a html parser.
The Raw data is contained within a textarea with following attributes:
<textarea id="paste_code" class="paste_code" name="paste_code" onkeydown="return catchTab(this,event)">
So the regex pattern looks like this:
private static string rgxPatternPasteBinRawContent = #"<textarea id=""paste_code"" class=""paste_code"" name=""paste_code"" onkeydown=""return catchTab\(this,event\)"">(.*)<\/textarea>";
Since the html code is spread over multiple lines, our Regex has to be use with a single line option.
Regex rgx = new Regex(rgxPatternPasteBinRawContent, RegexOptions.Singleline);
Now find the match, that contains the RAW data:
string htmlContent = await GetHtmlContentFromPage("SomePasteBinURL");
//Possibly your new WebClient().DownloadString("SomePasteBinURL");
//await not necesseraly needed here!
Match match = rgx.Match(htmlContent);
string rawContent = "ERROR: No Raw content found!";
if (match.Groups.Count > 0)
{
rawContent = match.Groups[1].Value;
}
int numberOfLines = rawContent.Split('\n').Length + 1;
And you're done.
The WebRequest looks like this for me:
private static async Task<string> GetHtmlContentFromPage(string url)
{
WebRequest request = WebRequest.CreateHttp(url);
WebResponse response = await request.GetResponseAsync();
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
readStream = new StreamReader(receiveStream);
string data = readStream.ReadToEnd();
response.Dispose();
readStream.Dispose();
return data;
}
Related
I am trying to make an app, that can read a text from a website and store it in a string.
For example my app could open this random generator website, which would generate a random number string and then my program would read it and store it in a string.
Is that even possible?
I didn't get your goal but you may get the whole HTML page and parse it as you wish:
var httpClient = new HttpClient();
var htmlString = await httpClient.GetStringAsync(new Uri("http://google.com"));
You can also use that, and precise the encoding :
string text = null;
using (WebResponse response = WebRequest.Create(url).GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.GetEncoding("iso-8859-1")))
{
text = reader.ReadToEnd();
reader.Close();
}
response.Close();
}
I have tried both WebClient's DownloadSring and WebRequest+Stream to try and scrape a page (This one) and get the Raw Paste data from it. I have scoured the net but have found no answers.
I have this code:
WebRequest request = WebRequest.Create("http://pastebin.com/raw.php?i=" + textBox1.Text);
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
string pasteContent = "";
using (StreamReader sr = new StreamReader(data))
{
pasteContent = sr.ReadToEnd();
}
new Note().txtMain.Text += pasteContent;
new Note().txtMain.Refresh();
and I have multiple forms so I am editing Note's txtMain textbox to add the paste content but it seems to return nothing, no matter which function I use. I know cross-form editing works because I have multiple things that can return to it.
How can I scrape the raw data?
Thank you VERY much,
P
There is no problem in downloading the content of your site. You simply doesn't use the instance of the Node class you created.
var note = new Note();
note.txtMain.Text += pasteContent;
note.Show();
Can anyone suggest a way to search a string on the web page, using ASP .net C#
Scenario: A textbox have some value(string), and on the click of button it searches that value(string) on some xyz page.
Example: I have "youtube"(string) in the textbox, and when I press submit button. That submit button gets the entire information of say Google.com(rendered page), and searches for "youtube"(string) on that Google.com
Possible solution:
I think that can be achieved by some-how rendering the page temporarily some-where, or some-how storing the response in any string or array, and then look that string.
Can anyone suggest a way to solve the above scenario. If possible with an example.
use a web request to get the page:
WebRequest request = WebRequest.Create(http://www.google.com);
request.Method = "GET";
WebResponse response = request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string content = reader.ReadToEnd();
reader.Close();
response.Close();
then just search on your content string:
int i = content.IndexOf("youtube");
or however you want to search for it.
You could do this with async/await if you are using .NET 4.5
static void Main(string[] args)
{
var content = GetUrlContents("http://www.google.com");
if (content.Result.Contains("Google"))
Console.WriteLine("Google found!");
Console.Read();
}
static async Task<string> GetUrlContents(string url)
{
HttpClient client = new HttpClient();
var content = await client.GetStringAsync(url);
return content;
}
Or if you want synchronous, you could do this kind of method
public static string GetUrlContents(string url)
{
return new WebClient().DownloadString(address);
}
I make a simple web scraper that scrapes lyrics for me then writes it to a database. everything works but for some reason it's replacing some characters with question marks and when I view this information on a simple php web page I'm seeing a lot of mistakes in the lyrics.
I?m = I'm
Let?s = Let's
haven?t = haven't
stuff like that.
I know the error is in c# and my code because I put a breakpoints before it writes to the database and I display it in a rich text box. How would I get it to display these characters correctly?
public static string getSourceCode(string url)
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
StreamReader sr = new StreamReader(resp.GetResponseStream());
string sourceCode = sr.ReadToEnd();
sr.Close();
resp.Close();
return sourceCode;
}
........
string url = txbURL2.Text;
string sourceCode = sourceCode = WorkerClass.getSourceCode(url);
int startIndex = sourceCode.IndexOf("<td valign=\"top\" width=\"100%\">");
sourceCode = sourceCode.Substring(startIndex, sourceCode.Length - startIndex);
........
//Gets Lyric
startIndex = sourceCode.IndexOf("<br><b>Lyrics:</b><br><br>") + 30;
endIndex = sourceCode.IndexOf(" <br><br>", startIndex);
string lyric = sourceCode.Substring(startIndex, endIndex - startIndex) + "";
rtbLyric.Text = lyric;
//End Lyric
The problem is probably character encoding. My guess is that the web page you're scraping is encoded in UTF8, but somewhere along the line you're converting to ASCII.
Check out the excellent article called "What every developer should know about character encoding" for more details.
Update
You could try this, although the StreamReader should default to UTF-8 anyway:
var encoding = System.Text.Encoding.GetEncoding("utf-8");
StreamReader sr = new StreamReader(resp.GetResponseStream(), encoding);
Check the encoding by searching for charset in the html code.
Your code snipplet misses the actual load process, so it is impossible to tell where it goes wrong.
You can also try using the WebClient:
WebClient client = new WebClient { Encoding = Encoding.UTF8 };
string html = client.DownloadString(url);
I want to generate html content based on a result returned by http url.
http://www.zillow.com/webservice/GetDeepSearchResults.htm?zws-id=X1-ZWz1c239bjatxn_5taq0&address=2114+Bigelow+Ave&citystatezip=Seattle%2C+WA
This page will give you some XML results. I want to convert to use that XML to generate HTML. I am not getting any idea where to start? Would someone offer any guidelines or sample code for asp.net?
For details: http://www.zillow.com/howto/api/GetDeepSearchResults.htm
To fetch the data you can use the HttpWebRequest class, this is an example I have to hand but it may be slightly overdone for your needs (and you need to make sure you're doing the right thing - I suspect the above to be a GET rather than a POST).
Uri baseUri = new Uri(this.RemoteServer);
HttpWebRequest rq = (HttpWebRequest)HttpWebRequest.Create(new Uri(baseUri, action));
rq.Method = "POST";
rq.ContentType = "application/x-www-form-urlencoded";
rq.Accept = "text/xml";
rq.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
Encoding encoding = Encoding.GetEncoding("UTF-8");
byte[] chars = encoding.GetBytes(body);
rq.ContentLength = chars.Length;
using (Stream stream = rq.GetRequestStream())
{
stream.Write(chars, 0, chars.Length);
stream.Close();
}
XDocument doc;
WebResponse rs = rq.GetResponse();
using (Stream stream = rs.GetResponseStream())
{
using (XmlTextReader tr = new XmlTextReader(stream))
{
doc = XDocument.Load(tr);
responseXml = doc.Root;
}
if (responseXml == null)
{
throw new Exception("No response");
}
}
return responseXml;
Once you've got the data back you need to render HTML, lots and lots of choices - if you just want to convert what you've got into HTML with minimal further processing then you can use XSLT - which is a question all on its own. If you need to do stuff with it then the question is too vague and you'll need to be more specific.
Create a xsl stylesheet, and inject the stylesheet element into the resulting xml from teh page