Develop Reference

OutOfMemoryException when save XmlDocument from XmlReader.ReadInnerXml

I am using XmlReader.ReadInnerXml() to load part of an XML file and save it as an XmlDocument. I ran into OutOfMemoryException when the innerXml part was over 2 GB (an estimate). What is the best way to handle this error? Is there a better way to create a large xml from XmlReader? Can I save the content without loading into memory?
using (XmlReader xmlRdr = XmlReader.Create(file))
{
xmlRdr.MoveToContent();
while (xmlRdr.Read())
{
//when read to XmlNodeType.Element and xmlRdr.Name meets certain criteria
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.PreserveWhitespace = true;
try
{
xmlDoc.LoadXml(xmlRdr.ReadInnerXml());
//get a few data from within the innerXml and eventually use XmlWritter to save the file
}
catch(Exception e)
{
string content = $"{e.GetType()} {e.Message} {NewLine} {objId}";
//send content to log file and email
}
}
}

As said in one of the comments maybe try using StreamReader and StreamWriter
This tutorial might help

Unable to save changes to XML document stored in Sharepoint 2010 Document Library

I am working on a project that requires all SQL connection and query information to be stored in XML files. To make my project configurable, I am trying to create a means to let the user configure his sql connection string information (datasource, catalog, username and password) via a series of text boxes. This input will then be saved to the appropriate node within the SQL document.
I can get the current information from the XML file, and display that information within text boxes for the user's review and correction, but I'm encountering an error when it comes time to save the changes.
Here is the code I'm using to update and save the xml document.
protected void submitBtn_Click(object sender, EventArgs e)
{
SPFile file = methods.web.GetFile("MyXMLFile.xml");
myDoc = new XmlDocument();
byte[] bites = file.OpenBinary();
Stream strm1 = new MemoryStream(bites);
myDoc.Load(strm1);
XmlNode node;
node = myDoc.DocumentElement;
foreach (XmlNode node1 in node.ChildNodes)
{
foreach (XmlNode node2 in node1.ChildNodes)
{
if (node2.Name == "name1")
{
if (node2.InnerText != box1.Text)
{
}
}
if (node2.Name == "name2")
{
if (node2.InnerText != box2.Text)
{
}
}
if (node2.Name == "name3")
{
if (node2.InnerText != box3.Text)
{
node2.InnerText = box3.Text;
}
}
if (node2.Name == "name4")
{
if (node2.InnerText != box4.Text)
{
}
}
}
}
myDoc.Save(strm1);
}
Most of the conditionals are empty at this point because I'm still testing.
The code works great until the last line, as I said. At that point, I get the error "Memory Stream is not expandable." I understand that using a memory stream to update a stored file is incorrect, but I can't figure out the right way to do this.
I've tried to implement the solution given in the similar question at Memory stream is not expandable but that situation is different from mine and so the implementation makes no sense to me. Any clarification would be greatly appreciated.

Using the MemoryStream constructor that takes a byte array as an argument creates a non-resizable instance of a MemoryStream. Since you are making changes to the file (and therefore the underlying bytes), you need a resizable MemoryStream. This can be accomplished by using the parameterless constructor of the MemoryStream class and writing the byte array into the MemoryStream.
Try this:
SPFile file = methods.web.GetFile("MyXMLFile.xml");
myDoc = new XmlDocument();
byte[] bites = file.OpenBinary();
using(MemoryStream strm1 = new MemoryStream()){
strm1.Write(bites, 0, (int)bites.Length);
strm1.Position = 0;
myDoc.Load(strm1);
// all of your edits to the file here
strm1.Position = 0;
// save the file back to disk
using(var fs = new FileStream("FILEPATH",FileMode.Create,FileAccess.ReadWrite)){
myDoc.Save(fs);
}
}
To get the FILEPATH for a Sharepoint file, it'd be something along these lines (I don't have a Sharepoint development environment set up right now):
SPFile file = methods.web.GetFile("MyXMLFile.xml")
var filepath = file.ParentFolder.ServerRelativeUrl + "\\" + file.Name;
Or it might be easier to just use the SaveBinary method of the SPFile class like this:
// same code from above
// all of your edits to the file here
strm1.Position = 0;
// don't use a FileStream, just SaveBinary
file.SaveBinary(strm1);
I didn't test this code, but I've used it in Sharepoint solutions to modify XML (mainly OpenXML) documents in Sharepoint lists. Read this blogpost for more information

You could look into using the XDocument class instead of XmlDocument class.
http://msdn.microsoft.com/en-us/library/system.xml.linq.xdocument.aspx
I prefer it because of the simplicity and it eliminates having to use Memory Stream.
Edit: You can append to the file like this:
XDocument doc = XDocument.Load('filePath');
doc.Root.Add(
new XElement("An Element Name",
new XAttribute("An Attribute", "Some Value"),
new XElement("Nested Element", "Inner Text"))
);
doc.Save(filePath);
Or you can search for an element and update like this:
doc.Root.Elements("The element").First(m =>
m.Attribute("An Attribute").Value == "Some value to match").SetElementValue(
"The element to change", "Value to set element to");
doc.Save('filePath');

Creating an XML with multiple root elements

I'm trying to create an XML with multiple root elements. I can't change that because that is the way I'm supposed to send the XML to the server. This is the error I get when I try to run the code:
System.InvalidOperationException: This operation would create an incorrectly structured document.
Is there a way to overwrite this error and have it so that it ignores this?
Alright so let me explain this better:
Here is what I have
XmlDocument doc = new XmlDocument();
doc.LoadXml(_application_data);
Now that creates the XML document and I can add a fake root element to it so that it works. However, I need to get rid of that and convert it into a DocumentElement object.
How would I go about doing that?

Specify Fragment when creating XmlWriter as shown here
XmlWriterSettings settings = new XmlWriterSettings();
settings.OmitXmlDeclaration = true;
settings.ConformanceLevel = ConformanceLevel.Fragment;
settings.CloseOutput = false;
// Create the XmlWriter object and write some content.
MemoryStream strm = new MemoryStream();
using (XmlWriter writer = XmlWriter.Create(strm, settings))
{
writer.WriteElementString("orderID", "1-456-ab");
writer.WriteElementString("orderID", "2-36-00a");
writer.Flush();
}

If it has multiple root elements, it's not XML. If it resembles XML in other ways, you could place everything under a root element, then when you send the string to the server, you just combine the serialized child elements of this root element, or as #Austin points out, use an inner XML method if available.

just create an XML with single root then get it's content as XML text.
you are talking about XML fragment anyways, since good xml has only one root.
this is sample to help you started:
var xml = new XmlDocument();
var root = xml.CreateElement("root");
root.AppendChild(xml.CreateElement("a"));
root.AppendChild(xml.CreateElement("b"));
Console.WriteLine(root.InnerXml); // outputs "<a /><b />"

Process XML in C# using external entity file

I am processing an XML file (which does not contain any dtd or ent declarations) in C# that contains entities such as é and à. I receive the following exception when attempting to load an XML file...
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(record);
Reference to undeclared entity
'eacute'.
I was able to track down the proper ent file here. How do I tell XmlDocument to use this ent file when loading my XML file?

In versions of the framework prior to .Net 4 you use ProhibitDtd of an XmlReaderSettings instance.
var settings = new XmlReaderSettings();
settings.ProhibitDtd = false;
string DTD = #"<!DOCTYPE doc [
<!ENTITY % iso-lat1 PUBLIC ""ISO 8879:1986//ENTITIES Added Latin 1//EN//XML""
""http://www.oasis-open.org/docbook/xmlcharent/0.3/iso-lat1.ent"">
%iso-lat1;
]> ";
string xml = string.Concat(DTD,"<xml><txt>rené</txt></xml>");
XmlDocument xd = new XmlDocument();
xd.Load(XmlReader.Create(new MemoryStream(
UTF8Encoding.UTF8.GetBytes(xml)), settings));
From .Net 4.0 onward use the DtdProcessing property with a value of DtdProcessing.Parse which you set on the XmlTextReader.
XmlDocument xd = new XmlDocument();
using (var rdr = new XmlTextReader(new StringReader(xml)))
{
rdr.DtdProcessing = DtdProcessing.Parse;
xd.Load(rdr);
}

I ran into the same problem, and not wanting to modify my XML (or DTD), I decided to create my own XmlResolver to add entities on the fly.
My implementation actually reads entities from the config file, but this should be enough to do what you're asking for. In this example, I'm converting a right single curly quote into an apostrophe.
class XmlEntityResolver : XmlResolver {
public override object GetEntity(Uri absoluteUri,
string role,
Type ofObjectToReturn)
{
if (absoluteUri.toString() == "-//MY PUB ID") {
MemoryStream ms = new MemoryStream();
StreamWriter sw = new StreamWriter(ms);
sw.Write("<!ENTITY rsquo \"'\">");
sw.Flush();
ms.Position = 0;
return ms;
}
else {
return base.GetEntity(absoluteUri, role, ofObjectToReturn);
}
}
}
Then, when you declare your XmlDocument, just set the resolver prior to load.
XmlDocument doc = new XmlDocument();
doc.XmlResolver = new XmlEntityResolver();
doc.Load(XML_FILE);

é is not a valid XML entity by default whereas it is a valid HTML entity by default.
You would need to define é as a valid XML entity for XML parsing purposes.
EDIT:
To add a reference to your external ent file you need to do that within the XML file itself. Save the ent file to disk and place it within the same directory as the document being parsed.
<!ENTITY % stuff SYSTEM "iso-lat1.ent">
%stuff;
If you want to go a different route check out the information on ENTITY declaration.

According to this, you have to reference them within the file; you cannot tell LoadXml to do this for you.

Your question has been answered in 2004 itself at MSDN Article........ You can find it here.......
http://msdn.microsoft.com/en-us/library/aa302289.aspx

determine if xml file contains data - c#

How do i know if my XML file has data besides the name space info:
Some of the files contain this:
<?xml version="1.0" encoding="UTF-8"?>
And if i encounter such a file, i want to place the file in an error directory

You could use the XmlReader to avoid the overhead of XmlDocument. In your case, you will receive an exception because the root element is missing.
string xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>";
using (StringReader strReader = new StringReader(xml))
{
//You can replace the StringReader object with the path of your xml file.
//In that case, do not forget to remove the "using" lines above.
using (XmlReader reader = XmlReader.Create(strReader))
{
try
{
while (reader.Read())
{
}
}
catch (XmlException ex)
{
//Catch xml exception
//in your case: root element is missing
}
}
}
You can add a condition in the while(reader.Read()) loop after you checked the first nodes to avoid to read the entire xml file since you just want to check if the root element is missing.

I think the only way is to catch an exception when you try and load it, like this:
try
{
System.Xml.XmlDocument doc = new System.Xml.XmlDocument();
doc.Load(Server.MapPath("XMLFile.xml"));
}
catch (System.Xml.XmlException xmlEx)
{
if (xmlEx.Message.Contains("Root element is missing"))
{
// Xml file is empty
}
}
Yes, there is some overhead, but you should be performing sanity checks like this anyway. You should never trust input and the only way to reliably verify it is XML is to treat it like XML and see what .NET says about it!

XmlDocument xDoc = new XmlDocument();
if (xDoc.ChildNodes.Count == 0)
{ // xml document is empty }
if (xDoc.ChildNodes.Count == 1)
{ // in xml document is only declaration node. (if you are shure that declaration is allways at the begining }
if (xDoc.ChildNodes.Count > 1)
{ // there is declaration + n nodes (usually this count is 2; declaration + root node) }

Haven't tried this...but should work.
try
{
XmlDocument doc = new XmlDocument();
doc.Load("test.xml");
}
catch (XmlException exc)
{
//invalid file
}
EDIT: Based on feedback comments
For large XML documents see Thomas's answer. This approach can have performance issues.
But, if it is a valid xml and the program wants to process it then this approach seems better.

If you aren't worried about validity, just check to see if there is anything after the first ?>. I'm not entirely sure of the C# syntax (it's been too long since I used it), but read the file, look for the first instance of ?>, and see if there is anything after that index.
However, if you want to use the XML later or you want to process the XML later, you should consider PK's answer and load the XML into an XmlDocument object. But if you have large XML documents that you don't need to process, then a solution more like mine, reading the file as text, might have less overhead.

You could check if the xml document has a node (the root node) and check it that node has inner text or other children.

As long as you aren't concerned with the validity of the XML document, and only want to ensure that it has a tag other than the declaration, you could use simple text processing:
var regEx = new RegEx("<[A-Za-z]");
bool foundTags = false;
string curLine = "";
using (var reader = new StreamReader(fileName)) {
while (!reader.EndOfStream) {
curLine = reader.ReadLine();
if (regEx.Match(curLine)) {
foundTags = true;
break;
}
}
}
if (!foundTags) {
// file is bad, copy.
}
Keep in mind that there's a million other reasons that the file may be invalid, and the code above would validate a file consisting only of "<a". If your intent is to validate that the XML document is capable of being read, you should use the XmlDocument approach.