Develop Reference

Connecting To A Website To Look Up A Word(Compiling Mass Data/Webcrawler)

I am currently developing a Word-Completion application in C# and after getting the UI up and running, keyboard hooks set, and other things of that nature, I came to the realization that I need a WordList. The only issue is, I cant seem to find one with the appropriate information. I also don't want to spend an entire week formatting and gathering a WordList by hand.
The information I want is something like "TheWord, The definition, verb/etc."
So, it hit me. Why not download a basic word list with nothing but words(Already did this; there are about 109,523 words), write a program that iterates through every word, connects to the internet, retrieves the data(definition etc) from some arbitrary site, and creates XML data from said information. It could be 100% automated, and I would only have to wait for maybe an hour depending on my internet connection speed.
This however, brought me to a few questions.
How should I connect to a site to look up these words? << This my actual question.
How would I read this information from the website?
Would I piss off my ISP or the website for that matter?
Is this a really bad idea? Lol.
How do you guys think I should go about this?
EDIT
Someone noticed that Dictionary.com uses the word as a suffix in the url. This will make it easy to iterate through the word file. I also see that the webpage is stored in XHTML(Or maybe just HTML). Here is the source for the Word "Cat". http://pastebin.com/hjZj6AC1

For what you marked as your actual question - you just need to download the data from the website and find what you need.
A great tool for this is CsQuery which allows you to use jquery selectors.
You could do something like this:
var dom = CQ.CreateFromUrl("http://www.jquery.com");
string definition = dom.Select(".definitionDiv").Text();

Stream live audio to website without Flash

I'm trying to find out what libraries there are out there for streaming live audio to a website.
The current technology stack that I'd like to stay within is...
MVC3/C#
HTML/HTML5
Basically, I'm wanting to build something that will allow the end user to have nothing but their (modern) browser installed and stream the audio. If possible I'd prefer that the same end user interface support previously recorded content as well.
As far as the input side of things I don't care what it uses as long as it's a .net/c# based technology or if not .net/c# I'd need it to be pre-built and capable of sending to a .net endpoint.
I have searched on SO for this topic and have found several posts on the subject, but most were extremely vague and didn't fit within the confines of the tech stack that I have in use (java/flash/etc) so please if you manage to find out be sure that it is actually inline with the question before you link it!

I would suggest you start here at the W3Schools - HTML5 Audio page and implement a C# handler that returns a byte-stream of the audio content for the source tag. Failing that, there are a bunch of JavaScript libraries that should be able to help you implement this properly.
The last one that I linked above ThunderJS, looks promising only because of the introductory line:
Thunder works its magic by creating HTML5 Audio elements with a src value containing a base64 data URI.
You could also check out similar questions here on StackOverflow, because you want to do this client-side so the language of choice isn't as much of an issue...
Maybe start here, or here, or here.
Failing THAT, there's always "the Google" which led me to these, few, lists?
Hope this helps to point you in the right direction. Good luck.

Where to create an RSS feed for dynamic website

I'm currently creating a website a little bit like Digg.com. There are different category like "Technology", "Sports", etc.. I want to create an RSS feeds for my website and while doing research on this, I have question that I can't find the answer.
First, this is what I have:
-I have the .NET code in C# that create a file with the last 15 news from a query from my database.
What I need to know:
-Is the RSS feeds (the xml file) needs to be generated at each load of the page (I saw that on some tutorial page but maybe it was only for a educational purpose). Personaly, I'm thinking about regenerating the .xml file each time someone submit something new. Is this a good idea?
-Do I need to create a different file for each categorie. Example: feedSports.xml, feedTechnology.xml, etc??? Or is there another way (I saw something about channel.???)
-What does feedburner do with all of this?
Thanks a lot for you help. I know this must be very newbie question so that's why I can't find anything answering this clearly on google.
DarkJaf

Your feeds would be generated just as your HTML pages are generated, after each request. But instead of outputting HTML it would be outputting RSS.
I probably would not make a file for each feed but it sure is possible. A better approach may be to pass a variable via GET or POST to your page generating the RSS and grab the data that pertains to the variable passed. You most likely can use the same logic you use for generate your HTML news lists if you isolate your code well.
I would also take a look at the article posted by Raj. It looks like C# has a nice namespace (System.ServiceModel.Syndication) that contains some objects that make the job pretty easy.
Have fun!
Nick
nickgs.com

How to get rss old items like google reader

I'm creating RSS reader application. I need to get any linked rss old items. For example some web rss result count is too less. My application check time range is too long. Sometimes loss some news.
How can I get rss old items?
When scrolling down on the google reader,reader shown previous items.

try this http://www.google.com/reader/atom/feed/{complete url to rssfeed without {} }?n=5000

I guess, Google saves these items and can display them, even if they are no longer in the feed. Google Reader might even show you items from before you added the feed, because the feeds might be stored globally and not per user.

Yes, the strategy recommended by #someone can help on this. Expanding on that:
Google Reader unofficial API lets you ask for old items from feeds, but it'll be very slow (if you're asking for 10000 items for instance), so you should ask for that once and cache it on your side.
If you need more than 10000~20000 you'll probably get timeouts on the Google server side. To help with this you can probably ask for 1000 or something items each time (http://www.google.com/reader/atom/feed/), and then use the continuation parameter for paging. I've never used this one, but it contains a parameter (c, for continuation) that can be promising for what you need. As described here (in the 'Atom set of items' section):
a string used for continuation process. Each feed return not all items, but only a certain number of items. You'll find in the atom feed (under the name gr:continuation) a string called continuation. Just add that string as argument for this parameter, and you'll retrieve next items.
One more thing, you'll need to login to Google Reader before using that API. If you want code for that, check my answer to this other question.
Hope it helps!

Since Google Reader shut down about a year ago, I'd suggest you give a shot at Superfeedr if you're looking for a replacement.

How to allow simple HTML tags in comments or anywhere?

In my web application I am developing a comment functionality, where user's can comment. But I am facing a problem which is I want to allow simple HTML tags in the comment box. HTML tags like <b>, <strong>, <i>, <em>, <u>, etc., that are normally allowed to enter in a commenting box. But then I also want when user presses enter then it will be automatically converted into breaks (<br /> tags) and get stored into database, so that when I'll display them in the web page then they'll look like as user entered.
Can you please tell me how to parse that user entered only allowed set of HTML tags and how to convert enters into <br /> tags and then store them in database.
Or if anyone have some better idea or suggestion to implement this kind of functionality. I am using ASP.NET 2.0 (C#)
I noticed that StackOverflow.com is doing the same thing on Profile Editing. When we edit our profile then below the "About Me" field "basic HTML allowed" line is written, I want to do almost the same functionality.

I don't have a C# specific answer for you, but you can go about it a few different ways. One is to let the user input whatever they want, then you run a filter over it to strip out the "bad" html. There are numerous open source filters that do this for PHP, Python, etc. In general, it's a pretty difficult problem, and it's best to let some well developed 3rd party code do this rather than write it yourself.
Another way to handle it is to allow the user to enter comments in some kind of simpler markup language like BBCode, Textile, or Markdown (stackoverflow is using Markdown), perhaps in conjunction with a nice Javascript editor. You then run the user's text through a processor for one of these markup languages to get the HTML. You can usually obtain implementations of these processors for whatever language you are using. These processors usually strip out the "bad" HTML.

Its rather "simple" to do that in php and python due to the large number of functions.I am still learning c# .lol. but havent yet come across the function.The chances are that it exists and all you need to do is search for it.I mean a function that can take the user input,search for the allowed tags (which are in an array of course) and replace the <> with something else like [] then use a function to escape the other html tags.In php we use htmlentities().
Something like
<code>
$txt=$_POST['comment'];
$txt=strreplace("<b>*</b>","[b]*[/b],"$txt");
$securetxt=htmlentities($txt);
$finaltxt=strreplace("[b]*[/b]","<b>*</b>","$securetxt");
//Now save to Db

I'm not sure, but I think you have to escape html characters when inserting in database and when retrieving echo them unescaped, so the browser can see it just like html.

I don´t know asp.net, but in php there´s an easy function, strip_tags, that let you add exceptions (in your case, b, em, etc.). If there´s nothing like that in C# you can write a regular expression that strips out all tags except the allowed ones but chances are that such an expression already exists so it should be easy to find.
replacing \n (or something similar) with br shouldn´t be a problem either with a simple search and replace.

This is a dangerous road to go down. You might think you can do some awesome regexes, or find someone who can help you with it, but sanitizing SOME markup and leaving other is just crazy talk.
I highly recommend you look into BBCode or another token system. Even something untokenized such as what SO uses, is probably a much better solution.