Develop Reference

HtmlAgilityPack throws stackoverflow exception when calling OuterHtml

var doc = new HtmlDocument();
var table = HtmlNode.CreateNode("table");
var tbody = HtmlNode.CreateNode("tbody");
table.AppendChild(tbody);
doc.DocumentNode.AppendChild(table);
var s = doc.DocumentNode.OuterHtml; //exception is thrown
I am unsure why this throws an exception. I have created 2 nodes and appended the child (tbody) to the parent (table).
In my mind, this should return
<table><tbody></tbody></table>
I'm not sure what I've done wrong

That HtmlNode.CreateNode expects an HTML structure instead of an element name.
From the method definition:
// Creates an HTML node from a string representing literal HTML.
//
// Parameters:
// html:
// The HTML text.
//
// Returns:
// The newly created node instance.
public static HtmlNode CreateNode(string html)
You are looking for the CreateElement method on the HtmlDocument instance.
var doc = new HtmlDocument();
var table = doc.CreateElement("table");
var tbody = doc.CreateElement("tbody");
table.AppendChild(tbody);
doc.DocumentNode.AppendChild(table);
In case you do want to go for HtmlNode.CreateNode then pass the corresponding html tags for the table and tbody.
var table = HtmlNode.CreateNode("<table></table");
var tbody = HtmlNode.CreateNode("<tbody></tbody>");

C# HtmlDocument Extract Classes

I am writing some code to loop through every element in a HTML page and extract all ID and Classes.
My current code is able to extract the ID's but I can't see a way to get the classes, does anybody know where I can access these?
private void ParseElements()
{
// GET: Document from Browser
HtmlDocument ThisDocument = Browser.Document;
// DECLARE: List of IDs
List<string> ListIdentifiers = new List<string>();
// LOOP: Through Each Element
for (int LoopA = 0; LoopA < ThisDocument.All.Count; LoopA += 1)
{
// DETERMINE: Whether ID Exists in Element
if (ThisDocument.All[LoopA].Id != null)
{
// ADD: Identifier to List
ListIdentifiers.Add(ThisDocument.All[LoopA].Id);
}
}
}

You could get the inner HTML of each node and use a regular expression to get the class. Or you could try HTML Agility pack.
Something like...
HtmlAgilityPack.HtmlDocument AgilePack = new HtmlAgilityPack.HtmlDocument();
AgilePack.LoadHtml(ThisDocument.Body.OuterHtml);
HtmlNodeCollection Nodes = AgilePack.DocumentNode.SelectNodes(#"//*");
foreach (HtmlAgilityPack.HtmlNode Node in Nodes)
{
if (Node.Attributes["class"] != null)
MessageBox.Show(Node.Attributes["class"].Value);
}

A better way to handle XML updation

I have a DataGridView control where some values are popluted.
And also I have an xml file. The user can change the value in the Warning Column of DataGridView.And that needs to be saved in the xml file.
The below program just does the job
XDocument xdoc = XDocument.Load(filePath);
//match the record
foreach (var rule in xdoc.Descendants("Rule"))
{
foreach (var row in dgRulesMaster.Rows.Cast<DataGridViewRow>())
{
if (rule.Attribute("id").Value == row.Cells[0].Value.ToString())
{
rule.Attribute("action").Value = row.Cells[3].Value.ToString();
}
}
}
//save the record
xdoc.Save(filePath);
Matching the grid values with the XML document and for the matched values, updating the needed XML attribute.
Is there a better way to code this?
Thanks

You could do something like this:
var rules = dgRulesMaster.Rows.Cast<DataGridViewRow>()
.Select(x => new {
RuleId = x.Cells[0].Value.ToString(),
IsWarning = x.Cells[3].Value.ToString() });
var tuples = from n in xdoc.Descendants("Rule")
from r in rules
where n.Attribute("id").Value == r.RuleId
select new { Node = n, Rule = r };
foreach(var tuple in tuples)
tuple.Node.Attribute("action").Value = tuple.Rule.IsWarning;
This is basically the same, just a bit more LINQ-y. Whether or not this is "better" is debatable. One thing I removed is the conversion of IsWarning first to string, then to int and finally back to string. It now is converted to string once and left that way.

XPath allows you to target nodes in the xml with alot of power. Microsoft's example of using the XPathNavigator to modify an XML file is as follows:
XmlDocument document = new XmlDocument();
document.Load("contosoBooks.xml");
XPathNavigator navigator = document.CreateNavigator();
XmlNamespaceManager manager = new XmlNamespaceManager(navigator.NameTable);
manager.AddNamespace("bk", "http://www.contoso.com/books");
foreach (XPathNavigator nav in navigator.Select("//bk:price", manager))
{
if (nav.Value == "11.99")
{
nav.SetValue("12.99");
}
}
Console.WriteLine(navigator.OuterXml);
Source: http://msdn.microsoft.com/en-us/library/zx28tfx1(v=vs.80).aspx

extract content from html page

I'm trying to extract the content inside div tag with id job_title1 in a html page. I'm using htmlagilitypack to fetch the data. Here is my code
var obj = new HtmlWeb();
var document = obj.Load("url of website ");
var bold = document.DocumentNode.SelectNodes("//div[#class='job_title1']");
foreach (var i in document.DocumentNode.SelectNodes("//div[#class='job_title1']"))
{
Response.Write(i.InnerHtml);
}
When i tried to run this code i'm getting error at foreach saying the Object reference not set to an instance of an object. Please help me solving this.

You said "div tag with id job_title1", shouldn't the xpath be:
document.DocumentNode.SelectNodes("//div[#id='job_title1']")

check if null like this:
var nodes = document.DocumentNode.SelectNodes("//div[#class='job_title1']");
if(nodes != null)
foreach (var i in document.DocumentNode.SelectNodes("//div[#class='job_title1']"
...
Edit: Use \" instead ':
var obj = new HtmlWeb();
var document = obj.Load("url of website ");
var bold = document.DocumentNode.SelectNodes("//div[#class=\"job_title1\"]");
if(bold!= null)
foreach (var i in bold)
{
Response.Write(i.InnerHtml);
}

Removing commented lines from InnerText

i'm currently using the below code which extracts the InnerText, however, what happens is i'm stuck with a bunch of comment out lines of html <-- how do I remove these using the code below?
HtmlWeb hwObject = new HtmlWeb();
HtmlAgilityPack.HtmlDocument htmldocObject = hwObject.Load(htmlURL);
foreach (var script in htmldocObject.DocumentNode.Descendants("script").ToArray())
script.Remove();
HtmlNode body = htmldocObject.DocumentNode.SelectSingleNode("//body");
resultingHTML = body.InnerText.ToString();

Just filter the nodes by comment nodes and call remove on them.
var rootNode = doc.DocumentNode;
var query = rootNode.Descendants().OfType<HtmlCommentNode>().ToList();
foreach (var comment in query)
{
comment.Remove();
}

This is probably a better answer:
public static void RemoveComments(HtmlNode node)
{
foreach (var n in node.ChildNodes.ToArray())
RemoveComments(n);
if (node.NodeType == HtmlNodeType.Comment)
node.Remove();
}