Develop Reference

scraping data from website with a C# console application

I'm trying to learn Spanish and making some flash cards (for my personal use) to help me learn the verbs.
Here is an example, page example. So near the top of the page you will see the past participle: bloqueado & gerund: bloqueando. It is these two values that I wish to obtain in my code and use for my flash cards.
If this is possible I will use a C# console application. I am aware that scraping data from a website is not ideal however this is a once off.
Any guidance on how to start something like this and pitfalls to avoid would be very helpful!

I know this isn't an exact answer, but here is the process I would suggest.
https://www.gnu.org/software/wget/ and mirror the website to a
folder. Wget is a web spider and will follow the links on the site until it has downloaded everything. You'll have to run it with a few different parameters until you figure out the correct settings you want.
Use C# to run through each file in the folder and extract the
words from <section class="verb-mood-section"> in each file. It's your choosing of whether you want to output them to the console or store them in a database or flat file.
Should be that easy, in theory.

Use SGMLReader. SGMLReader is a versatile and robust component that will stream HTML to an XMLReader:
XmlDocument FromHtml(TextReader reader) {
// setup SgmlReader
Sgml.SgmlReader sgmlReader = new Sgml.SgmlReader();
sgmlReader.DocType = "HTML";
sgmlReader.WhitespaceHandling = WhitespaceHandling.All;
sgmlReader.CaseFolding = Sgml.CaseFolding.ToLower;
sgmlReader.InputStream = reader;
// create document
XmlDocument doc = new XmlDocument();
doc.PreserveWhitespace = true;
doc.XmlResolver = null;
doc.Load(sgmlReader);
return doc;
}
You can see that you need to create a TextReader first. TThis would in reality be a StreamReader as a TextReader is an abstract class.
Then you create the XMLDocument over that. Once you've got it into the XMLDocument you can use the various methods supported by XMLDocument to isolate and extract the nodes you need. I'll leave you to explore that aspect of it.
You might try using the XDocument class as it's a lot easier to handle than the XMLDocument, especially if you're a newbie. It also supports LINQ.

Xamarin crossplatform Read an xml from online

I have an xml file on server www.testsite.com/sample_file.xml
the structure of xml is like this
<mobileclients>
<clientitem>
<code>SXPFBD</code>
<api>http://SPFD.azurewebsites.net/APIs/</api>
</clientitem>
<clientitem>
<code>STYPFBD</code>
<api>http://SPFD.azurewebsites.net/APIs/</api>
</clientitem>
</mobileclients>
I like to check the code input is present in xml or not using a xamarin cross platform application
I cant keep the file as a resource Since it will keep updating So is it possible to read from an online xml file on the go

Sure, although I would recommend switching to JSON if you can, it is still possible to work with XML.
Here is a code snippet from an app I have developed.
public List<NewsItem> GetNewsItems()
{
var newsXml = XDocument.Load("http://xxxxxxxx.nl/index.php?page=news");
var newsItems = (from newsitem in newsXml.Descendants("newsitem")
select new NewsItem
{
Title = WebUtility.HtmlDecode(newsitem.Element("title").Value),
Message = newsitem.Element("message").Value,
Date = DateTime.ParseExact(newsitem.Element("date").Value, "dd-MM-yyyy HH:mm", CultureInfo.InvariantCulture), // TODO better error handling
User = newsitem.Element("user").Value,
Replies = Convert.ToInt32(newsitem.Element("replies").Value),
Url = newsitem.Element("budgetgamingurl").Value,
Views = Convert.ToInt32(newsitem.Element("views").Value)
}).ToList();
return newsItems;
}
As you can see it loads the document from a URL and handles the XPath queries in-memory.
It is a bit rough, it could use some error handling, but you'll get the basic idea.

Saving, loading and manipulating (listview) data C#

I'm working on an UWP application where a user can input data which is placed in a listview. All fine and dandy, but how can I save the user data to a separate file and load it the next time a user boots up the app?
I've tried to find a solution, but I had great difficulty to understand these code snippets and on how to apply these (since I'm fairly new to C# and App development). Would somebody like to explain how I can achieve the saving/loading of the data and explain what the code does?
Thanks in advance! :)

You can create a file like this:
StorageFile ageFile = await local.CreateFileAsync("Age.txt", CreationCollisionOption.FailIfExists);
I can read and write to a file like this:
StorageFolder local = Windows.Storage.ApplicationData.Current.LocalFolder;
var ageFile = await local.OpenStreamForReadAsync(#"Age.txt");
// Read the data.
using (StreamReader streamReader = new StreamReader(ageFile))
{
//Use like a normal streamReader
}
if you are trying to write, use OpenStreamForWriteAsync;

If I understood well, you have some kind of object structure that serves as a model for your ListView. When the application is started, you want to read a file where the data is present. When closing the application (or some other event) write the file with the changes done. Right?
1) When your application is loaded / closed (or upon modifications or some event of your choice), use the Windows.Storage API to read / write the text into the file.
2) If the data you want to write is just a liste of strings, you can save this as is in the file. If it is more complicated, I would recommend serializing it in JSON format. Use JSON.NET to serialize (object -> string) and deserialize (object <- string) the content of your file and object structure.
Product product = new Product();
product.Name = "Apple";
...
string json = JsonConvert.SerializeObject(product);

C# Reading Video Metadata in Windows (Title,description,year)

Hey everyone just trying to make a program that browses video files and reads the title and description from the files metadata. I found some docs from microsoft here giving whats needed but how do I access these functions? what using namespaces are needed in c#? I would love any help that can be provided.

In that link you posted, scroll to the bottom and click "Shell Metadata Providers". There's more more information and some sample C++ code.
Here are some other relevant links:
Reading/Writing metadata of audio/video files
http://www.codeproject.com/Articles/14535/Accessing-WMF-metadata-with-C
https://social.msdn.microsoft.com/Forums/pt-BR/0f36a3b2-4d3d-4842-88a4-bea493bbbace/read-video-filemov-avi-mpg-etc-meta-data?forum=csharpgeneral
https://web.archive.org/web/20170225230114/https://stackoverflow.com/questions/7396265/c-sharp-to-read-properties-of-video-files
Sorry I can't give you anything more concrete, however it looks like some tag libraries (i.e. for reading MP3 metadata) may work as well, as the metadata for videos seems to be stored in a similar, if not identical, format. That being said, you can give TagLib# a shot.
https://www.nuget.org/packages/taglib/

I've made a simple C# code (portable to Unity, too) csatomreader. It's optimized for speed and can read the atoms over HTTP, too.
E.g. Get title:
using (FileStream stream = new FileStream(fileName, FileMode.Open))
{
var mp4Reader = new AtomReader(stream);
string value = mp4Reader.GetMetaAtomValue(AtomReader.TitleTypeName);
Console.WriteLine($"{atomTypeName}: {value}");
}
If you need to get more metadata values at once, then iterate over ParseAtoms(), e.g. see the GetMetaAtomValue() source.

Silverlight Loading Reference Data On Demand from a 'dumb' server

I have a text file with a list of 300,000 words and the frequency with wich they occur. Each line is in the format Word:FequencyOfOccurence.
I want this information to be accessible from within the C# code. I can't hard code the list since it is too long, and I'm not sure how to go about accessing it from a file on the server. Ideally I'd ideally like the information to be downloaded only if it's used (To save on bandwidth) but this is not a high priority as the file is not too big and internet speeds are always increasing.
It doesn't need to be useable for binding.
The information does not need to be editable once the project has been built.

Here is another alternative. Zip the file up and stick it in the clientBin folder next to the apllication XAP. Then at the point in the app where the content is needed do something like this:-
public void GetWordFrequencyResource(Action<string> callback)
{
WebClient client = new WebClient();
client.OpenReadAsync += (s, args) =>
{
try
{
var zipRes = new StreamResourceInfo(args.Result, null)
var txtRes = Application.GetResourceStream(zipRes, new Uri("WordFrequency.txt", UriKind.Relative));
string result = new StreamReader(txtRes.Stream).ReadToEnd();
callback(result);
}
catch
{
callback(null); //Fetch failed.
}
}
client.OpenReadAsync(new Uri("WordFrequency.zip", UriKind.Relative"));
}
Usage:-
var wordFrequency = new Dictionary<string, int>();
GetWordFrequencyResource(s =>
{
// Code here to burst string into dictionary.
});
// Note code here is asynchronous with the building of the dictionary don't attempt to
// use the dictionary here.
The above code allows you to store the file in an efficient zip format but not in the XAP itself. Hence you can download it on demand. It makes use of the fact that a XAP is a zip file so Application.GetResourceStream which is designed to pull resources from XAP files can be used on a zip file.
BTW, I'm not actually suggesting you use a dictionary, I'm just using a dictionary as simple example. In reality I would imagine the file is in sorted order. If that is the case you could use a KeyValuePair<string, int> for each entry but create a custom collection type that holds them in an array or List and then use some Binary search methods to index into it.

Based on your comments, you could download the word list file if you are required to have a very thin server layer. The XAP file containing your Silverlight application is nothing more than a ZIP file with all the referenced files for your Silverlight client layer. Try adding the word list as content that gets compiled into the XAP and see how big the file gets. Text usually compresses really well. In general, though, you'll want to be friendly with your users in how much memory your application consumes. Loading a huge text file into memory, in addition to everything else you need in your app, may untimately make your app a resource hog.
A better practice, in general, would be to call a web service. The service could would perform whatever look up logic you need. Here's a blog post from a quick search that should get you started: (This was written for SL2, but should apply the same for SL3.)
Calling web services with Silverlight 2
Even better would be to store your list in a SQL Server. It will be much easier and quicker to query.

You could create a WCF service on the server side that will send the data to the Silverlight application. Once you retrieve the information you could cache it in-memory inside the client. Here's an example of calling a WCF service method from Silverlight.
Another possibility is to embed the text file into the Silverlight assembly that is deployed to the client:
using (var stream = Assembly.GetExecutingAssembly()
.GetManifestResourceStream("namespace.data.txt"))
using (var reader = new StreamReader(stream))
{
string data = reader.ReadToEnd();
// Do something with the data
}