So I'm working on a little fun project and keep in mind I'm a beginner, I want to grab the info of songs that have played from this radio channel:
ilikeradio (sorry the site is in Swedish).
I want to just simply put that in a textBox.
I have tried:
WebClient web = new WebClient();
string htmlContent = new System.Net.WebClient().DownloadString(URL);
But this only gave me the source code and not the code with the list items for artist song etc.
Any help is appreciated Keep in mind I am a beginner.
It seems that the URL you provided returns HTML, but if you compare the HTML you get with that which is rendered in the browser (by right-clicking the webpage and inspecting the HTML), you will see that what you get is actually different than what is finally rendered. The reason for this is that the website is using Ajax to load the song list. In other words, when you call DownloadString(), you get the results from the web serve before it has had the javascript run and update it.
It is not easy to get the final HTML render result. But you are in luck!
If you go to that website and open the debug tools in Chrome and click the Network tab. Next, sort all the requests by Method and GET requests should be at the top. Amongst those GET requests is the one you are looking for:
https://unison.mtgradio.se/api/v2/timeline?channel_id=6&client_id=6690709&to=2018-10-02T08%3A00%3A50&from=2018-10-02T07%3A00%3A50&limit=40
This URL returns JSON which the web server eventually loads and renders for you to see as a "song list".
The JSON returned is a list of songs with some metadata. You will need to parse this JSON to extract and display the list of songs in your own webpage. I suspect that you can view the source code of that website and find the Javascript to do this ;)
Newtonsoft JSONConvert is the best library for parsing JSON.
If you want to view the JSON with the song list, copy the URL above and paste it into your browser address bar (and hit enter). Next, copy the JSON result and then open this. Paste JSON into the Text tab and then click the Viewer tab. You will note that the first element is the Current Song, while other elements are in the song list. Also note that each element has a child element called song, which contains the title.
To get you going, try this:
using System;
using System.Net;
using Newtonsoft.Json.Linq;
public class Program
{
public static void Main()
{
WebClient web = new WebClient();
using (WebClient wc = new WebClient())
{
var json = wc.DownloadString("https://unison.mtgradio.se/api/v2/timeline?channel_id=6&client_id=6690709&to=2018-10-02T08%3A00%3A50&from=2018-10-02T07%3A00%3A50&limit=40");
dynamic stuff = JArray.Parse(json);
string name = stuff[1].song.title;
Console.WriteLine(name);
}
}
}
NOTE
By the time you try this out, you will notice that the song name printed to console does not exist in the list on the webpage. This is because if you look at the JSON URL that I posted above, there are query parameters... one of which is date and time. You will need to modify the URL accordingly to get the most recent (displayed right now on the website) playlist.
Related
I know there is another question with practically identical title here: Webclient.DownloadString does not retrieve the whole page
But the solution doesn't help me, maybe somebody else have the same problem.
I'm trying to get the html code of this URL:
https://cubebrush.co/?freebies=true
To achieve that, I'm using the following code in C#:
WebClient webClient = new WebClient();
string webString = webClient.DownloadString("https://cubebrush.co/?freebies=true");
But the retrieved html lacks some information, for example, all the button tags inside the website. This can be quickly checked using the library HtmlAgilityPack and checking for all the tags inside the website with the following code:
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(webString);
HashSet<string> hs = new HashSet<string>();
foreach (var dec in doc.DocumentNode.Descendants())
{
hs.Add(dec.Name);
}
If we run this, it will show 26 tags, but none of them will be a button tag. This makes sense, since the initial webString also lacks that "button information".
I've tried to copy webString into a file, to check if, as the initial commented post says, was a problem with the visualizer, but it doesn't, visualizer and file looks exactly equal.
Can somebody tells me what I'm doing wrong? Thanks!
i have a problem i want to find a specific string in a web page then save the web page that i found the string.
I am using firefox for web browser
Problem :
1. I open a page (Containing a random word)
2. Then my C# program doing searching in the page, if the word find in the page then program will automaticaly save the page to Drive . If not the program will do click on Next Button on the page then do search again in the page.
Is that possible ?
Ok, so it sounds like you might want to do something like the following.
You can use WebClient to load the response from a url into a string:
using(WebClient client = new WebClient()) {
string s = client.DownloadString(your_url);
}
You can then search for a occurrence of the string you a looking for in "s" using indexOf:
if (s.IndexOf("string you are searching for") > -1)
{
// s contains "string you are searching for"
}
Then you can save "s" to disk using a StreamWriter:
using(StreamWriter sw = new StreamWriter("file name"))
{
sw.WriteLine(s);
}
In terms of clicking the "next" button can you define the urls as a list of strings and then just iterate over them using the previous code for each.
i recently made an app using windows app studio online, and suprisingly the app turn out alright it handles the rss feeds quite nicely. Ive made some changes to the app, as the feed contains a enclosure url which is a download link. i cant get it to detect the enclosure url. i think im just expressing it wrong.
i tried every online rss source viewer to see if it appeared any different in yahoo pipes it appears as item.enclosure.url, on code beauty it appears as http://bla.com"/> ive tried every combination i can think of
i also tried putting the feed through yahoo pipes to give me a new feed with the enclosure url in a new tag
< downloadlink >< /downloadlink> and used the code
rssItem.FeedUrl = item.GetSafeElementString("downloadlink")
and the whole app works as it should so i know the only thing im doing wrong is retieving the enclosure url the only problem with yahoo pipes is you cant change the input url which i need to do so it has to come from the enclosure url
so my question is how should i retrieve the tag
rssItem.FeedUrl = item.GetSafeElementString("whatgoeshere")
thanks
This Worked Nicely
XElement element = item.Element("enclosure"); //new element
int length = (int)element.Attribute("length"); //seprate attributes
string type = (string)element.Attribute("type");
string url = (string)element.Attribute("url");
rssItem.FeedUrl = url; // use the result
I'm trying to get the FINAL source of a webpage. I am using webclient openRead method, but this method is only returning the initial page source. After the source downloads, there is a javascript that runs and collect the data that I need in a different format and my method will be looking for something that got completely changed.
What I am talking about is exactly like the difference between:
right-click on a webpage -> select view source
access the developer tools
Look at this site to know what I am talking about: http://www.augsburg.edu/history/fac_listing.html and watch how any of the email is displayed using each option. I think what happening is that the first will show you the initial load of the page. The second will show you the final page html. The webclient only lets me do option #1.
here is the code that will only return option #1. Oh I need to do this from a console application. Thank you!
private static string GetReader(string site)
{
WebClient client = new WebClient();
try
{
data = client.OpenRead(site);
reader = new StreamReader(data);
}
catch
{
return "";
}
return reader.ReadToEnd();
}
I've found a solution to my problem.
I ended up using Selenium-WebDriver PageSource property. It worked beautifully!
Learn about Selenium and Webdriver. It is an easy thing to learn. It helps for testing and on this!
I have an RSS feed URL, that I can view in any Feed Reader.
This RSS feed is not controlled by me, it is only consumed by me.
This RSS Feed (Office of Inspector General's Excluded Provider List) links to a page with download-able files.
These files are updated approximately once a month, and the RSS feed displays new "unread" items.
What I want to do is write something (in C#) that checks this RSS Feed once a week, and when a new item (i.e. a new download-able file) is available, triggers off an executable.
This is essentially like a very scaled-down RSS Reader, with the sole purpose of triggering an executable when a new item appears.
Any guidance, advice would be greatly appreciated.
Edit:
I need help in determining when a new
item becomes available for
download.
The running of an
executable I can do.
The
executable that will run, will process
the downloaded file.
As a commenter already noted, this question is quite broad, but here's an attempt to answer:
You can either write a Windows Service (use a template that comes with VS/MonoDevelop) or you can write a simple console app that would be called by Windows Scheduler or Cron.
The main code will use one of the many RSS feed parsers available:
There are plenty of examples here on SO. IMO, the simplest LINQ-based is here
I personally like this approach, also using LINQ.
Once you parse the feed, you need to look for the value of the Link element, found by doing this from the SO example above:
....
var feeds = from feed in feedXML.Descendants("item")
select new
{
Title = feed.Element("title").Value,
**Link** = feed.Element("link").Value,
Description = feed.Element("description").Value
};
....
So, now that you have the executable, you'll need to download it to your machine. I suggest you look into this example from MSDN:
Now, that you have the file downloaded, simple use Process.Start("Path to EXE"); to execute it.
Watch out for viruses in the exes!!!
If you are using .Net 3.5 or above you can you the various classes within the System.ServiceModel.Syndication namespace, specifically the SyndicationFeed class which exposes a LastUpdatedTime property that you can use to compare dates to know when to call your executable using the Process.Start method in the System.Diagnostics namespace.
using (XmlReader reader = XmlReader.Create(path))
{
SyndicationFeed feed = SyndicationFeed.Load(reader);
if ((feed != null) && (feed.LastUpdateTime > feedLastUpdated))
{
// Launch Process
}
}
So you have to read the RSS feed from the URL, and then parse the data to determine whether a new item is available.
To read the feed, you'll want to use a WebClient. The simplest way:
var MyClient = new WebClient();
string rssData = MyClient.DownloadString("http://whatever");
You can then create an XML document from the returned string.
var feedXML = new XMlDocument();
feedXML.Load(rssData);
#dawebber shows how to parse the XML with LINQ. You'll want to check the date on each item to see if it's newer than the last date checked. Or perhaps you have a database of items that you've already seen and you want to check to see if the items you received are in the database.
Whenever you find a new item, you can fire off your executable using Process.Start.
You could write a System Tray application. I've done several that screen scrape/monitor sites on a scheduled basis. Here is a VERY simple start. I think you could do what you're looking for in a few hours.
http://alperguc.blogspot.com/2008/11/c-system-tray-minimize-to-tray-with.html