I want to get captions from youtube videos and I have seen that it's easy to do with video that have been subbed by humans with an HTTP request as following:
https://www.youtube.com/api/timedtext?v=[LINK]&lang=en
The problem emerges when I try to do it with auto generated subtitle because it gives me a blank page.
I tried going to the network tab of chrome to find out what kind of HTTP request the browser does, and I found out that it uses the same link but with more specific parameters that change in every youtube video:
Signature: Two Hexadecimal numbers separated by a dot with length 40 each
Expire : A decimal number with length 10
The other parameters are equals, so I need to figure out how to get them. I can't find any API that helps me with this specific problem.
Well why do you want to inspect network calls and figure it out from there. I think Google transits data with protobuf, so good luck trying to decrypt the params.
Why not use their official api and do it the correct and legal way ?
What you want is captions.
Here's the documentation on its api:
https://developers.google.com/youtube/v3/docs/captions
You, probably, will want to first call the 'list' method to get a list of all the available caption tracks for the video. Then the 'download' method to fetch the captions.
The aforementioned link to the docs also includes code examples you will need.
Another Way to do it would be:
1) Call http://www.youtube.com/get_video_info?&video_id=videoID
2) Parse the response as a query string (URL Encoded String)
3) Convert the string to an array/object
4) The Url to fetch caption tracks (Including the auto-generated ones) will be present in "player_response" property.
Related
Going to Google code page I couldn't find the API I should use to perform a basic web search. All other resources I found point to Google Base API but it is no longer available.
What I need is to be able to sumbit a query string and get back a list with site names. For example, I need to find the first results when searching for "champions league" as if typing the query on the Google page.
What is the correct API to use for text searches these days? Are there any librariries for PHP or C Sharp?
EDIT: I found PHP code on the Net that sends requests to ajax.googleapis.com/ajax/services/search/web. I checked it out and it actually returns search results :) Do you know where I can find info for this endpoint and from what API is it part? Also, Custom Search API as suggested by #Rickard doesn't seem to provide this basic functionality. I tried to use it but it asks me to enter the sites I want to search in. I don't want to search particular sites but all.
Thank you
Check out the Google Custom Search API
I found this to work just fine, as the Google API is great unless you need to search +100 times per day, then they charge you. This is a simple solution to this problem, but only works for string searches and not image searches.
Search for "turtle":
string searchString = "Turtle";
System.Diagnostics.Process.Start("www.google.com/search?q=" + searchString);
This website that keeps updating some live information about the bus timings in Helsinki.
I want to parse the live information from the website and display it on my WP7 phone. The user needs to enter the bus stop number and the WP7 app should show the buses/trams currently in the bus stop.
Is there any way I could obtain the real time information from the website?
If you look at the source of the website (http://www.omatlahdot.fi/omatlahdot/web?command=fullscreen&stop=1020455) -- in IE right-click on the page and select View Source -- you'll see that there's really very little in the actual source file, in particular none of the data is there. All of the hard work is coming from the referenced javascript file scripts/fullscreen_header.js (full path is http://www.omatlahdot.fi/omatlahdot/scripts/fullscreen_header.js). You want to download that .js file and study how it retrieves data with AJAX calls. Start with the reloadPage function.
You can make these same calls (e.g., using WebClient) to retrieve the data into your application. If you want to extract the data from the returned HTML, I'd consider parsing it simply as a string since I am assuming that it would have a very regular structure and dragging in a general-purpose HTML parser would probably be overkill.
Alternatively, you might find out if the omatlahodot.fi provides the data as JSON or XML feeds, so you don't have to "screen-scrape" the HTML. I don't read Finnish, so I can't help you with that. Look around on their websites (maybe a section called "dev" or "api") or send them an email inquiry.
Please let us know how it works out!
Bing Maps API has been great for figuring out all my GPS information sent from a WP7 phone. But I'm not sure how to deal with Zip Codes. I know it varies per country. I'm curious what is the best way to get a Zip code (or similar) from a latitude/longitude. Bing Maps doesn't always return a City with Area code. Sometimes it will bring back "Populated Area" such as the bay area or San Francisco. The zip node in the XML will be blank.
Are there hacks to get around this? Or am I looking at some other complicated calculation and a second web service call?
I ended up using the google api to get this type of information. I wanted to get the suburb I was in and Bing would not return this. May be worth a look to see if Google will return the zip code.
You can use the Locations API to get the location information for a specified point. If you only want the postal code for the specified location, then you just set the includeEntityTypes parameter to PostCode1, though you can request several as a comma-separated list.
As you are no doubt aware, you can also choose between an XML or JSON response.
I'm not sure if there's an easier way to do this with the API, but you can definitely accomplish this without using another service although it's a bit of extra work. You can add a custom tile layer using shape files such as the ones provided here: http://www.census.gov/geo/www/cob/zt_metadata.html
I want to submit Google queries like these:
http://www.google.ch/search?q=100+eur+to+chf
http://www.google.ch/search?q=1.5*17.5
...from a C# console application and capture the result reported back by Google (and ignore any links to other sites). Is there a specific Google API that helps me with this task?
I got this idea from the tool Launchy (launchy.net). The plugin GCalc does this, I also found the source file for this module:
http://launchy.svn.sourceforge.net/viewvc/launchy/tags/2.5/plugins/gcalc/gcalc.cpp?revision=614&view=markup
It looks like GCalc does not use any Google API at all. But I've got no clue how to do the same in C#, and I would prefer to use a proper API. But if there isn't one, I could use some help/pointers on how to copy the GCalc functionality to C# (.net libraries/classes...?)
Google calculator results don't show up when using the API. So if you want them, you'll have to scrape the page. Be careful doing so as it's against Google' terms of service so your IP will be banned if you send too many frequent requests.
Once you've got the results page, use an html parser. The result is in a <b> tag (e.g. <b>1 + 1 = 2</b>; if it's not present, then you have no calculator result). Be careful of <sup> tags within the result (e.g. <b>(1 (m^2) kg) / 2 = 0.5 m<sup>2</sup> kg</b>). You might also want to decode the html entities.
You can use WebClient.DownloadString(String url). This way you get page (html) as string.
You have to parse result, but that shouldn't be hard. HttpAgilityPack is good c# html parser that uses XPath for data retrieval.
why not use HTTPWebRequest and then parse the result as macrog stated in his answer.
I need to write a C# code for grabbing contents of a web page. Steps looks like following
Browse to login page
I have user name and a password, provide it programatically and login
Then you are in detail page
You have to get some information there, like (prodcut Id, Des, etc.)
Then need to click(by code) on Detail View
Then you can get the price for that product from there.
Now it is done, so we can write detail line into text file like this...
ABC Printer::225519::285.00
Please help me on this, (Even VB.Net Code is ok, I can convert it to C#)
The WatiN library is probably what you want, then. Basically, it controls a web browser (native support for IE and Firefox, I believe, though they may have added more since I last used it) and provides an easy syntax for programmatically interacting with page elements within that browser. All you'll need are the names and/or IDs of those elements, or some unique way to identify them on the page.
You should be able to achieve this using the WebRequest class to retrieve pages, and the HTML Agility Pack to extract elements from HTML source.
yea I downloaded that library. Nice one.
Thanks for sharing it with me. But I have a issue with that library. The site I want to get data is having a "captcha" on the login page.
I can enter that value if this can show image and wait for my input.
Can we achive that from this library, if you can like to have a sample.
You should be able to achieve this by using two classes in C#, HttpWebRequest (to request the web pages) and perhaps XmlTextReader (to parse the HTML/XML response).
If you do not wish to use XmlTextReader, then I'd advise looking into Regular Expressions, as they are fantastically useful for extracting information from large bodies of text where-in patterns exist.
How to: Send Data Using the WebRequest Class