Develop Reference

C# Using XPath with XmlDocument - Can't select nodes in a namespace (returning null)

I'm trying to do something which ought to be quite simple but I'm having terrible trouble. I have tried code from multiple similar questions in StackOverflow but to no avail.
I'm trying to get various pieces of information from an ABN lookup with the Australian government. Here is anonymised return XML value:
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<ABRSearchByABNResponse xmlns="http://abr.business.gov.au/ABRXMLSearch/">
<ABRPayloadSearchResults>
<request>
<identifierSearchRequest>
<authenticationGUID>00000000-0000-0000-0000-000000000000</authenticationGUID>
<identifierType>ABN</identifierType>
<identifierValue>00 000 000 000</identifierValue>
<history>N</history>
</identifierSearchRequest>
</request>
<response>
<usageStatement>The Registrar of the ABR monitors the quality of the information available on this website and updates the information regularly. However, neither the Registrar of the ABR nor the Commonwealth guarantee that the information available through this service (including search results) is accurate, up to date, complete or accept any liability arising from the use of or reliance upon this site.</usageStatement>
<dateRegisterLastUpdated>2017-01-01</dateRegisterLastUpdated>
<dateTimeRetrieved>2017-01-01T00:00:00.2016832+10:00</dateTimeRetrieved>
<businessEntity>
<recordLastUpdatedDate>2017-01-01</recordLastUpdatedDate>
<ABN>
<identifierValue>00000000000</identifierValue>
<isCurrentIndicator>Y</isCurrentIndicator>
<replacedFrom>0001-01-01</replacedFrom>
</ABN>
<entityStatus>
<entityStatusCode>Active</entityStatusCode>
<effectiveFrom>2017-01-01</effectiveFrom>
<effectiveTo>0001-01-01</effectiveTo>
</entityStatus>
<ASICNumber>000000000</ASICNumber>
<entityType>
<entityTypeCode>PRV</entityTypeCode>
<entityDescription>Australian Private Company</entityDescription>
</entityType>
<goodsAndServicesTax>
<effectiveFrom>2017-01-01</effectiveFrom>
<effectiveTo>0001-01-01</effectiveTo>
</goodsAndServicesTax>
<mainName>
<organisationName>COMPANY LTD</organisationName>
<effectiveFrom>2017-01-01</effectiveFrom>
</mainName>
<mainBusinessPhysicalAddress>
<stateCode>NSW</stateCode>
<postcode>0000</postcode>
<effectiveFrom>2017-01-01</effectiveFrom>
<effectiveTo>0001-01-01</effectiveTo>
</mainBusinessPhysicalAddress>
</businessEntity>
</response>
</ABRPayloadSearchResults>
</ABRSearchByABNResponse>
</soap:Body>
</soap:Envelope>
so I want to get for example the whole response using xpath="//response" then use various xpath statement within that node to get the <organisationName> ("//mainName/organisationName") and other values.
It should be simple right? Those xpath statements appear to work when testing in Notepad++but I use this code in Visual Studio:
XmlDocument xdoc = new XmlDocument();
xdoc.LoadXml(ipxml);
XmlNode xnode = xdoc.SelectSingleNode("//response");
XmlNodeList xlist = xdoc.SelectNodes("//mainName/organisationName");
xlist = xdoc.GetElementsByTagName("mainName");
But it always returns null, whatever I put in the xpath I get a null return for the node and 0 count for the list whether I'm selecting something with child nodes, a value or not.
I can get the nodes using GetElementsByTagName() as in the example which returns the correct node, but I wanted to do it 'properly' selecting the proper field using xpath.
I also tried using XElement and Linq but still no luck. Is there something weird about the XML?
I'm sure it must something simple but I've been struggling for ages.

You aren't dealing with the namespaces present in the document. Specifically, the high level element:
<ABRSearchByABNResponse xmlns="http://abr.business.gov.au/ABRXMLSearch/">
places ABRSearchByABNResponse, and all its child elements (unless overridden by another xmlns) into the namespace http://abr.business.gov.au/ABRXMLSearch/. In order to navigate to these nodes (without hacks like GetElementsByTagName or using local-name()), you'll need to register the namespaces with an XmlNamespaceManager, like so. The xmlns aliases don't necessarily need to match those used in the original document, but it's a good convention to do so:
XmlDocument
var xdoc = new XmlDocument();
var ns = new XmlNamespaceManager(xdoc.NameTable);
ns.AddNamespace("soap", "http://schemas.xmlsoap.org/soap/envelope/");
ns.AddNamespace("abr", "http://abr.business.gov.au/ABRXMLSearch/");
xdoc.LoadXml(ipxml);
// NB need to use the overload accepting a namespace
var xresponse = xdoc.SelectSingleNode("//abr:response", ns);
var xlist = xdoc.SelectNodes("//abr:mainName/abr:organisationName", ns);
XDocument
More recently, the powers of LINQ can be harnessed with XDocument, which makes working with namespaces much easier (Descendants finds child nodes at any depth)
var xdoc = XDocument.Parse(ipxml);
XNamespace soap = "http://schemas.xmlsoap.org/soap/envelope/";
XNamespace abr = "http://abr.business.gov.au/ABRXMLSearch/";
var xresponse = xdoc.Descendants(abr + "response");
var xlist = xdoc.Descendants(abr + "organisationName");
XDocument + XPath
You can also resort to using XPath in Linq to Xml, especially for more complicated expressions:
var xdoc = XDocument.Parse(ipxml);
var ns = new XmlNamespaceManager(new NameTable());
ns.AddNamespace("soap", "http://schemas.xmlsoap.org/soap/envelope/");
ns.AddNamespace("abr", "http://abr.business.gov.au/ABRXMLSearch/");
var xresponse = xdoc.XPathSelectElement("//abr:response", ns);
var xlist = xdoc.XPathSelectElement("//abr:mainName/abr:organisationName", ns);

You need to call SelectSingleNode and SelectNodes on the DocumentElement. You are calling them on the document itself.
For example:
XmlNode xnode = xdoc.DocumentElement.SelectSingleNode("//response");

How to use XElement.XPathSelectElement with namespace [duplicate]

Why is this Xpath not working using XDocument.XPathSelectElement?
Xpath:
//Plugin/UI[1]/PluginPageCategory[1]/Page[1]/Group[1]/CommandRef[2]
XML
<Plugin xmlns="http://www.MyNamespace.ca/MyPath">
<UI>
<PluginPageCategory>
<Page>
<Group>
<CommandRef>
<Images>
</Images>
</CommandRef>
<CommandRef>
<Images>
</Images>
</CommandRef>
</Group>
</Page>
</PluginPageCategory>
</UI>
</Plugin>
C# Code:
myXDocument.XPathSelectElement("//Plugin/UI[1]/PluginPageCategory[1]/Page[1]/Group[1]/CommandRef[2]", myXDocument.Root.CreateNavigator());

When namespaces are used, these must be used in the XPath query also. Your XPath query would only work against elements with no namespace (as can be verified by removing the namespace from your XML).
Here's an example showing how you create and pass a namespace manager:
var xml = ... XML from your post ...;
var xmlReader = XmlReader.Create( new StringReader(xml) ); // Or whatever your source is, of course.
var myXDocument = XDocument.Load( xmlReader );
var namespaceManager = new XmlNamespaceManager( xmlReader.NameTable ); // We now have a namespace manager that knows of the namespaces used in your document.
namespaceManager.AddNamespace( "prefix", "http://www.MyNamespace.ca/MyPath" ); // We add an explicit prefix mapping for our query.
var result = myXDocument.XPathSelectElement(
"//prefix:Plugin/prefix:UI[1]/prefix:PluginPageCategory[1]/prefix:Page[1]/prefix:Group[1]/prefix:CommandRef[2]",
namespaceManager
); // We use that prefix against the elements in the query.
Console.WriteLine(result); // <CommandRef ...> element is printed.
Hope this helps.

This should probably be a comment on #Cumbayah's post, but I can't seem to leave comments on anything.
You are probably better off using something like this instead of using XmlReader to get the nametable.
var xml = ... XML from your post ...;
var myXDocument = XDocument.Parse(xml);
var namespaceManager = new XmlNamespaceManager(new NameTable());
namespaceManager.AddNamespace("prefix", "http://www.MyNamespace.ca/MyPath");
var result = ...;

The easiest way in your case is to use XPath axes and node test for node name and position to select the element.
Your XPath selection:
myXDocument.XPathSelectElement("//Plugin/UI[1]/PluginPageCategory[1]/Page[1]/Group[1]/CommandRef[2]", myXDocument.Root.CreateNavigator());
Can be easily translate to:
myXDocument.XPathSelectElement("/child::node()[local-name()='Plugin']/child::node()[local-name()='UI'][position()=1]/child::node()[local-name()='PluginPageCategory'][position()=1]/child::node()[local-name()='Page'][position()=1]/child::node()[local-name()='Group'][position()=1]/child::node()[local-name()='CommandRef'][position()=2]");
There is no need to create and pass XmlNamespaceManager as parameter.

There is a way to do it without any change to the xpath. The solution I've found is to remove the namespace when parsing the XML into the XDocument.
Here is an exemple:
var regex = #"(xmlns:?[^=]*=[""][^""]*[""])";
var myXDocument = XDocument.Parse(Regex.Replace("MyXmlContent", regex, "", RegexOptions.IgnoreCase | RegexOptions.Multiline))
Now that the namespace is gone, it is easyer to manipulate.

Unable to use XmlDocument.SelectSingleNode on XML with Two Namespaces

I'm trying to parse the following XML:
<?xml version="1.0" encoding="utf-8"?>
<A2AAnf:MPPPPPP xsi:schemaLocation="urn:A2AAnf:xsd:$MPPPPPP.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:A2AAnf="urn:A2AAnf:xsd:$MPPPPPP">
<A2AAnf:Num>0</A2AAnf:Num>
<A2AAnf:FIT xmlns="urn:iso:std:iso:20022:xsd:003.001">
<Hdr>
<Inf>
<Mtd>TEST</Mtd>
</Inf>
</Hdr>
</A2AAnf:FIT>
I want to access the <Mtd> tag.
XMLQuire shows the path to be /A2AAnf:MPPPPPP/A2AAnf:FIT/dft:Hdr/dft:Inf/dft:Mtd, but when I'm trying to parse it using the following code:
XmlDocument xmldocument = new XmlDocument();
var xmlNameSpaceManager = new XmlNamespaceManager(xmldocument.NameTable);
xmlNameSpaceManager.AddNamespace("A2AAnf", "urn:A2AAnf:xsd:$MPPPPPP");
try
{
xmldocument.LoadXml(m_XML);
var node = xmldocument.SelectSingleNode("/A2AAnf:MPPPPPP/A2AAnf:FIT/dft:Hdr/dft:Inf/dft:Mtd", xmlNameSpaceManager);
}
I receive the following error:
namespace prefix 'dft' is not defined
And since I can't find "dft" in my XML, I also tried to remove the "dft" prefix and search for the same string without "dft". This time, nothing was returned.
What am I missing?

You have to add dft to your XmlNamespaceManager:
var xmlNameSpaceManager = new XmlNamespaceManager(xmldocument.NameTable);
xmlNameSpaceManager.AddNamespace("A2AAnf", "urn:A2AAnf:xsd:$MPPPPPP");
xmlNameSpaceManager.AddNamespace("dft", "urn:iso:std:iso:20022:xsd:003.001");
The prefixes you use in your XPath query have nothing to do with the prefixes used in the XML document. They are instead the prefixes you define in your XmlNamespaceManager.

SelectSingleNode return null - even with namespace

I know this question has been asked in a similar fashion before, but I can't seem to get this working.
I have some xml:
<?xml version="1.0" encoding="ISO-8859-1" ?>
<Research xmlns="http://www.rixml.org/2005/3/RIXML" xmlns:xalan="http://xml.apache.org/xalan" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" createDateTime="2011-03-29T15:41:48Z" language="eng" researchID="MusiJvs3008">
<Product productID="MusiJvs3008">
<StatusInfo currentStatusIndicator="Yes" statusDateTime="2011-03-29T15:41:48Z" statusType="Published" />
<Source>
<Organization type="SellSideFirm" primaryIndicator="Yes">
<OrganizationID idType="Reuters">9999</OrganizationID>
And I'm trying to read values using xpath:
XPathDocument xmldoc = new XPathDocument(xmlFile);
XPathNavigator nav = xmldoc.CreateNavigator();
XmlNamespaceManager nsMgr = new XmlNamespaceManager(nav.NameTable);
nsMgr.AddNamespace(string.Empty, "http://www.rixml.org/2005/3/RIXML");
XPathNavigator result = nav.SelectSingleNode("/Research", nsMgr); // <-- Returns null!
But even a simple select of the root node returns null! I am sure I have something wrong with my namespace. Can someone please help?
Ideally I want simple lines that will let me select values from the xml file, i.e.
String a = xmlDoc.SelectSingleNode(#"/Research/Product/Content/Title").Value;
BTW, I have no (direct) control over the XML file content.

I don't believe you can use an empty namespace alias and have it used automatically by the XPath expression. As soon as you use an actual alias, it should work though. This test is fine, for example:
using System;
using System.Xml;
using System.Xml.XPath;
class Test
{
static void Main()
{
string xmlFile = "test.xml";
XPathDocument xmldoc = new XPathDocument(xmlFile);
XPathNavigator nav = xmldoc.CreateNavigator();
XmlNamespaceManager nsMgr = new XmlNamespaceManager(nav.NameTable);
nsMgr.AddNamespace("x", "http://www.rixml.org/2005/3/RIXML");
XPathNavigator result = nav.SelectSingleNode("/x:Research", nsMgr);
Console.WriteLine(result);
}
}
Do you have to use XPath and XPathDocument, by the way? I tend to find that LINQ to XML is a much more pleasant API, particularly when it comes to namespaces. If you're using .NET 3.5 and you have no particular requirement to use XPath, I'd suggest you check it out.

Make the following changes
nsMgr.AddNamespace("x", "http://www.rixml.org/2005/3/RIXML");
XPathNavigator result = nav.SelectSingleNode("/x:Research", nsMgr);

i could post, and answer my own question, for the native equivalent of this code. Instead i'll just add it as an answer to the end.
When using the native IXMLDOMDocument (version 6) object:
//Give the default namespace as alias of "x"
document.setProperty("SelectionNamespaces","xmlns:x='http://www.rixml.org/2005/3/RIXML'");
//Query for the nodes we want
document.selectSingleNode("/x:Research");
Bonus Question: Why, oh why, does no Xml Document object model query the default namespace when no namespace is specified... sigh

SelectSingleNode returning null for known good xml node path using XPath

Consider this simple XML document. The serialized XML shown here is the result of an XmlSerializer from a complex POCO object whose schema I have no control over.
<My_RootNode xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="">
<id root="2.16.840.1.113883.3.51.1.1.1" extension="someIdentifier" xmlns="urn:hl7-org:v3" />
<creationTime xsi:nil="true" xmlns="urn:hl7-org:v3" />
</My_RootNode>
The goal is to extract the value of the extension attribute on the id node. In this case, we are using the SelectSingleNode method, and given an XPath expression as such:
XmlNode idNode = myXmlDoc.SelectSingleNode("/My_RootNode/id");
//idNode is evaluated to null at this point in the debugger!
string msgID = idNode.Attributes.GetNamedItem("extension").Value;
The problem is that the SelectSingleNode method returns null for the given XPath expression.
Question: any ideas on this XPath query's correctness, or why this method call + XPath expression would return a null value? Perhaps the namespaces are part of the problem?

I strongly suspect the problem is to do with namespaces. Try getting rid of the namespace and you'll be fine - but obviously that won't help in your real case, where I'd assume the document is fixed.
I can't remember offhand how to specify a namespace in an XPath expression, but I'm sure that's the problem.
EDIT: Okay, I've remembered how to do it now. It's not terribly pleasant though - you need to create an XmlNamespaceManager for it. Here's some sample code that works with your sample document:
using System;
using System.Xml;
public class Test
{
static void Main()
{
XmlDocument doc = new XmlDocument();
XmlNamespaceManager namespaces = new XmlNamespaceManager(doc.NameTable);
namespaces.AddNamespace("ns", "urn:hl7-org:v3");
doc.Load("test.xml");
XmlNode idNode = doc.SelectSingleNode("/My_RootNode/ns:id", namespaces);
string msgID = idNode.Attributes["extension"].Value;
Console.WriteLine(msgID);
}
}

If you want to ignore namespaces completely, you can use this:
static void Main(string[] args)
{
string xml =
"<My_RootNode xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns=\"\">\n" +
" <id root=\"2.16.840.1.113883.3.51.1.1.1\" extension=\"someIdentifier\" xmlns=\"urn:hl7-org:v3\" />\n" +
" <creationTime xsi:nil=\"true\" xmlns=\"urn:hl7-org:v3\" />\n" +
"</My_RootNode>";
XmlDocument doc = new XmlDocument();
doc.LoadXml(xml);
XmlNode idNode = doc.SelectSingleNode("/*[local-name()='My_RootNode']/*[local-name()='id']");
}

This should work in your case without removing namespaces:
XmlNode idNode = myXmlDoc.GetElementsByTagName("id")[0];

Sorry, you forgot the namespace. You need:
XmlNamespaceManager ns = new XmlNamespaceManager(myXmlDoc.NameTable);
ns.AddNamespace("hl7","urn:hl7-org:v3");
XmlNode idNode = myXmlDoc.SelectSingleNode("/My_RootNode/hl7:id", ns);
In fact, whether here or in web services, getting null back from an XPath operation or anything that depends on XPath usually indicates a problem with XML namespaces.

Just to build upon solving the namespace issues, in my case I've been running into documents with multiple namespaces and needed to handle namespaces properly. I wrote the function below to get a namespace manager to deal with any namespace in the document:
private XmlNamespaceManager GetNameSpaceManager(XmlDocument xDoc)
{
XmlNamespaceManager nsm = new XmlNamespaceManager(xDoc.NameTable);
XPathNavigator RootNode = xDoc.CreateNavigator();
RootNode.MoveToFollowing(XPathNodeType.Element);
IDictionary<string, string> NameSpaces = RootNode.GetNamespacesInScope(XmlNamespaceScope.All);
foreach (KeyValuePair<string, string> kvp in NameSpaces)
{
nsm.AddNamespace(kvp.Key, kvp.Value);
}
return nsm;
}

Well... I had the same issue and it was a headache. Since I didn't care much about the namespace or the xml schema, I just deleted this data from my xml and it solved all my issues. May not be the best answer? Probably, but if you don't want to deal with all of this and you ONLY care about the data (and won't be using the xml for some other task) deleting the namespace may solve your problems.
XmlDocument vinDoc = new XmlDocument();
string vinInfo = "your xml string";
vinDoc.LoadXml(vinInfo);
vinDoc.InnerXml = vinDoc.InnerXml.Replace("xmlns=\"http://tempuri.org\/\", "");

The rule to keep in mind is: if your document specifies a namespace, you MUST use an XmlNamespaceManager in your call to SelectNodes() or SelectSingleNode(). That's a good thing.
See the article Advantages of namespaces . Jon Skeet does a great job in his answer showing how to use XmlNamespaceManager. (This answer should really just be a comment on that answer, but I don't quite have enough Rep Points to comment.)

just use //id instead of /id. It works fine in my code

Roisgoen's answer worked for me, but to make it more general, you can use a RegEx:
//Substitute "My_RootNode" for whatever your root node is
string strRegex = #"<My_RootNode(?<xmlns>\s+xmlns([\s]|[^>])*)>";
var myMatch = new Regex(strRegex, RegexOptions.None).Match(myXmlDoc.InnerXml);
if (myMatch.Success)
{
var grp = myMatch.Groups["xmlns"];
if (grp.Success)
{
myXmlDoc.InnerXml = myXmlDoc.InnerXml.Replace(grp.Value, "");
}
}
I fully admit that this is not a best-practice answer, but but it's an easy fix and sometimes that's all we need.