Develop Reference

crawling price gives null , HtmlAgilityPack (C#)

Im trying to get stock data from a website with webcrawler as a hobby project. I got the link to work, i got the Name of the stock but i cant get the price... I dont know how to handle the html code. Here is my code,
var htmlDocument = new HtmlDocument();
htmlDocument.LoadHtml(html);
var divs = htmlDocument.DocumentNode.Descendants("div").Where(n => n.GetAttributeValue("class", "").Equals("Flexbox__StyledFlexbox-sc-1ob4g1e-0 eYavUv Row__StyledRow-sc-1iamenj-0 foFHXj Rows__AlignedRow-sc-1udgki9-0 dnLFDN")).ToList();
var stocks = new List<Stock>();
foreach (var div in divs)
{
var stock = new Stock()
{
Name = div.Descendants("a").Where(a=>a.GetAttributeValue("class","").Equals("Link__StyledLink-sc-apj04t-0 foCaAq NameCell__StyledLink-sc-qgec4s-0 hZYbiE")).FirstOrDefault().InnerText,
changeInPercent = div.Descendants("span").Where((a)=>a.GetAttributeValue("class", "").Equals("Development__StyledDevelopment-sc-hnn1ri-0 kJLDzW")).FirstOrDefault()?.InnerText
};
stocks.Add(stock);
}
foreach (var stock in stocks)
{
Console.WriteLine(stock.Name + " ");
}
I got the Name correct, but i dont really know how the get the ChangeInPercent.... I will past in the html code below,
The top highlight show where i got the name from, and the second one is the "span" i want. I want the -4.70
Im a litle bit confused when it comes to get the data with my code. I tried everything. My changeInPercent property is a string.
it has to be the code somehow...

There's probably an easier to select a single attribute/node than the way you're doing it right now:
If you know the exact XPath expression to select the node you're looking for, then you can do the following:
var htmlDocument = new HtmlDocument();
htmlDocument.LoadHtml(html);
var changeInPercent = htmlDocument.DocumentNode
.SelectSingleNode("//foo/bar")
.InnerText;
Getting the right XPath expression (the //foo/bar example above) is the tricky part. But this can be found quite easy using your browser's dev tools. You can navigate to the desired element and just copy it's XPath expression - simple as that! See here for a sample on how to copy the expression.

check if collection of objects where each contain another collection of objects contain all values of List<string> via LINQ

I have a collection of objects where each object contains another collection of objects. I need to find out the fastest way to check if it contains all values of List<string>.
Here is an example:
class Video {
List<Tag> Tags;
}
class Tag{
public Tag (string name){
Name = name;
}
string Name;
}
List<string> selectedTags = new List<string>();
selectedTags.Add("Foo");
selectedTags.Add("Moo");
selectedTags.Add("Boo");
List<Video> videos = new List<Video>();
// Case A
Video videoA = new Video();
videoA.Tags = new List<Tag>();
videoA.Tags.Add(new Tag("Foo"));
videoA.Tags.Add(new Tag("Moo"));
videos.Add(videoA);
videoA should not be selected by LINQ because it doesn't contain all tags.
// Case B
Video videoB = new Video();
videoB.Tags = new List<Tag>();
videoB.Tags.Add(new Tag("Foo"));
videoB.Tags.Add(new Tag("Moo"));
videoB.Tags.Add(new Tag("Boo"));
videos.Add(videoB);
videoB should be selected by LINQ because it contains all tags.
I tried this with foreach loops, but it's too slow so I'm looking for a LINQ solution.
foreach (Video video in videos) {
if (video.Tags.Count() > 0) {
bool containAllTags = true;
foreach (string tagToFind in selectedTags) {
bool tagFound = false;
foreach (Tag tagItem in video.Tags) {
if (tagToFind == tagItem.Name)
tagFound = true;
}
if (!tagFound)
containAllTags = false;
}
if (containAllTags)
result.Add(videoItem);
}
}
The resulting LINQ should look like this:
IEnumerable<Video> = from vid in videos
where vid.Tags.( ..I dont know.. )
select vid;
I tried several ways with .Any, .All, etc.. but I can't find the solution and I can't use .Intersect because one is a List of strings and the other is a List of objects. Note that in the production version, Video and Tag elements have many more properties.

With your current code, you logically want:
IEnumerable<Video> result = from vid in videos
where selectedTags.All(tag =>
vid.Tags.Any(t => t.Name == tag))
select vid;
Or equivalently:
var result = videos.Where(vid => selectedTags.All(tag =>
vid.Tags.Any(t => t.Name == tag)));
This is assuming you've made Tag.Name and Video.Tags public, of course - ideally as properties rather than as fields.
Note how we're calling All against selectedTags, as (assuming I've read your requirements correctly) it's important that all the selected tags are present in the video - it's not important that all the video's tags are selected.
Now that could be relatively slow, if you have a lot of tags to check and a lot of tags per video.
However, knowing how to optimize it will really depend on some other choices:
If the order of the tags isn't important, could you change Video.Tags to be a set instead of a list?
Are you always looking through the same set of videos, so you could perform some pre-processing?
Is the total number of tags available large? What about the number of tags per video? What about the number of selected tags?
Alternatively, you can project each video to its "list of tags" and check whether there are any in the selected set which aren't in the video's set:
var result = videos.Where(vid => !selectedTags.Except(vid.Tags.Select(t => t.Name))
.Any());

Looping through to return all the required node IDs

I am kinda struggling with returning all the node ID's inside a given Treeview.
Basically I have a function which runs a query to the database using a given NodeID
and returns its Parent Node ID as shown below, which works fine and returns me the
Parent Node ID of the Node ID which is passed through.
public static string Parent_ID(string NodeID)
{
GeneralConnection gc = ConnectionHelper.GetConnection();
int intNodeID = Convert.ToInt32(NodeID);
QueryDataParameters para = new QueryDataParameters();
para.Add("#NodeID", intNodeID);
int parent = (int)gc.ExecuteScalar("custom.product.searchForParentNodeID", para);
return Convert.ToString(parent);
}
Now the ParentID returned is actually a NodeID of the Node above. So basically
I want to use the newly obtained ID to obtains its ParentID and keeping looping
through the table (using the above query) till I can obtain all NodeIDs within the current level. Obviously I will be storing these NodeIDs in a StringBuilder list which I plan on using it for further checks.
Anybody have suggestions?

Your question is a little confusing, but maybe this will help. You should try using the API instead of custom queries, if possible, to make hotfixes/upgrades easier in the future. For example, if you have a Node ID and you want all of it's siblings, you can do something like this:
// Version 7 code, untested
var treeProvider = new TreeProvider();
var childNodeId = 1;
// Get the child Node
// Node ID, Culture, Site name
var childNodeDataSet = treeProvider.SelectNodes(childNodeId, null, CMSContext.CurrentSiteName);
if (childNodeDataSet.Items.Any())
{
// Assuming the child node was found, get the parent ID
var parentNodeId = childNodeDataSet.Items[0].NodeParentID;
// Now get the children of the parent, aka the siblings
// Site name, Culture, Combine with default culture, Where clause, Order by clause
var siblingNodes = treeProvider.SelectNodes(CMSContext.CurrentSiteName, "/%", null, false, null, "NodeParentID = '" + parentNodeId + "'", null);
}
Hope that helps.

c# return all specific elements from an xml

Greetings. Im having a small trouble i would like to have some help with. Im having a very large xml file with about 1000 customers with diffrent customer information. And I would like to do methods to retrive this information. Ive been searching everywhere but cant seem to find what im looking for. Currently im trying:
public custInformation getcustInfo(string file) {
//Load the xml file
var xe = XDocument.Load(_server.MapPath(file)).Root;
//Get information
return (from p in xe.Descendants("cust-account").Descendants("cust-info")
select new custInformation
{
firstName = (string)p.Element("cust-fname"),
lastName = (string)p.Element("cust-lname"),
address = (string)p.Element("cust-address1"),
}).(All elements)??
}
(All elements) is where id like to retrive all the information. Using FirstOrDefault will only retrive the first element and LastOrDefault will only retrive the first element. If some one could help me i would be very greatefull.

you want a list of customers. Change the return value to IEnumerable
and transform the query to IEnumerable with ToList()/ToArray():
public IEnumerable<custInformation> getcustInfo(string file) {
//Load the xml file
var xe = XDocument.Load(_server.MapPath(file)).Root;
//Get information
return (from p in xe.Descendants("cust-account").Descendants("cust-info")
select new custInformation
{
firstName = (string)p.Element("cust-fname"),
lastName = (string)p.Element("cust-lname"),
address = (string)p.Element("cust-address1"),
}).ToList();
}

How does one get all child terms of a SharePoint term in C#?

I am writing a webpart for SharePoint 2010 that recuperates the latest page of a certain (custom) type, according to publishing date. It only takes into account pages tagged with a specified term. I would like it to be able to also do so with pages that are tagged with terms which are children of the selected terms.
If I have a term tree like so:
England
Kent
Dover
Canterbury
Surrey
Croydon
Crawley
then by selecting Kent, I want my webpart to show the latest page tagged with Kent, Dover, or Canterbury.
Is this possible in C# ?
Thanks for your time.

The function you are looking for is Term.GetTerms
You will need to get a TaxonomyValue from your field
Then you have to get the current TaxonomySession, then use the TaxonomySession to get the Term used in the field. From that term you can use the Parent field to get the parent Term.
Here is some rough code to show you the objects used.
TaxonomyFieldValue v = null; // Notsurehowtodothisbit();
TaxonomySession session = new TaxonomySession(site);
if (session.TermStores != null && session.TermStores.Count > 0)
{
TermStore termStore = session.TermStores[0];
Term t = termStore.GetTerm(v.TermGuid);
Term parentTerm = t.Parent;
TermCollection childTerms = t.GetTerms();
}
Once you have the tree, you may be able to use a caml query to generate a SPList.GetList query that brings back anything tagged that way.
I have not done an experiment in this regard...
But Bart-Jan Hoeijmakers has
private SPListItemCollection GetItemsByTerm(Term term, SPList list)
{
// init some vars SPListItemCollection items = null;
SPSite site = SPContext.Current.Site; // set up the TaxonomySession
TaxonomySession session = new TaxonomySession(site);
// get the default termstore TermStore termStore = session.TermStores[0];
// If no wssid is found, the term is not used yet in the sitecollection, so no items exist using the term
int[] wssIds = TaxonomyField.GetWssIdsOfTerm(SPContext.Current.Site, termStore.Id, term.TermSet.Id, term.Id, false, 1);
if (wssIds.Length > 0)
{
// a TaxonomyField is a lookupfield. Constructing the SPQuery
SPQuery query = new SPQuery();
query.Query = String.Format("<Where><Eq><FieldRef Name='MyTaxonomyField' LookupId='TRUE' /><Value Type='Lookup'>{0}</Value></Eq></Where>", wssIds[0]);
items = list.GetItems(query);
}
return items;
}

Nat's partial answer using the GetTerms method for the parent is great. The code for querying one list looks good too.
To get the id for the parent term, you can use TermStore.GetTerms against the title.
To search across all lists and libraries in the the site collection, you can use the Search API's FullTextSQLQuery method specifying the guids in the where clause with the owstaxIdMyTaxonomyField as the column.
There is a great example of getting id's by title and searching for a term store by id at Using taxonomy fields in SharePoint 2010: Part III