OutOfMemoryException when save XmlDocument from XmlReader.ReadInnerXml - c#

I am using XmlReader.ReadInnerXml() to load part of an XML file and save it as an XmlDocument. I ran into OutOfMemoryException when the innerXml part was over 2 GB (an estimate). What is the best way to handle this error? Is there a better way to create a large xml from XmlReader? Can I save the content without loading into memory?
using (XmlReader xmlRdr = XmlReader.Create(file))
{
xmlRdr.MoveToContent();
while (xmlRdr.Read())
{
//when read to XmlNodeType.Element and xmlRdr.Name meets certain criteria
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.PreserveWhitespace = true;
try
{
xmlDoc.LoadXml(xmlRdr.ReadInnerXml());
//get a few data from within the innerXml and eventually use XmlWritter to save the file
}
catch(Exception e)
{
string content = $"{e.GetType()} {e.Message} {NewLine} {objId}";
//send content to log file and email
}
}
}

As said in one of the comments maybe try using StreamReader and StreamWriter
This tutorial might help

Related

How to resolve IOException "file used by another process" while saving XmlDocument?

When I'm trying to save the XML Document I edited the IOException "file used by another process" occured when I try to save that document.
Any ideas how to solve this?
Note: This method is called everytime a new element in the XmlDocument should be written.
public void saveRectangleAsXMLFragment()
{
XmlDocument doc = new XmlDocument();
doc.Load("test.xml");
XmlDocumentFragment xmlDocFrag = doc.CreateDocumentFragment();
String input = generateXMLInput();
xmlDocFrag.InnerXml = input;
XmlElement mapElement = doc.DocumentElement;
mapElement.AppendChild(xmlDocFrag);
input = null;
mapElement = null;
xmlDocFrag = null;
doc.Save("test.xml");
}
Its probably one of your other methods, or other part of the code which opened the file and didnt calose it well. Try to search for this kind of problem.
try this if your's application is only access that .xml file
1. Create a Object globally
object lockData = new object();
2.Use than object to lock statement where you save and load xml
lock(lockData )
{
doc.Load("test.xml");
}
lock(lockData )
{
doc.Save("test.xml");
}
From Jon Skeet's related answer (see https://stackoverflow.com/a/8354736/4151626)
There seems to be a bug in XmlDocument.Save()'s treatment of the file stream, where it becomes pinned and is neither Closed() nor Disposed(). By taking direct control of the creation and disposition of the stream outside of the XmlDocument.Save() I was able to get around this halting error.
//e.g.
XmlWriter xw = new XmlWriter.Create("test.xml");
doc.Save(xw);
xw.Close();
xw.Dispose();

Read only root-element from XML

I have a very large xml-file (let's say it has about 300000 elements). In my part of the application I only have to know if the name of the root-element is ApplicationLog and if there is an attribute called LogId in the root-element.
To read the XML I use:
XDocument document;
using (StreamReader streamReader = new StreamReader(filePath, true))
{
document = XDocument.Load(streamReader);
}
and to get the information I need:
try
{
if (document.Root != null)
{
if (string.Equals(document.Root.Name.LocalName, "ApplicationLog", StringComparison.InvariantCultureIgnoreCase) &&
document.Root.HasAttributes && (from o in document.Root.Attributes() where string.Equals(o.Name.LocalName, "LogId", StringComparison.InvariantCultureIgnoreCase) select o).Any())
{
isRelevantFile = true;
}
}
}
catch (Exception e)
{
}
This just works fine.
The problem is that the XDocument.Load takes about 15 seconds to load a XML-File which is about 20MB.
I also tried it with the XmlDocument, but there I have the same problem.
My first idea for a solution was to read the file as text and parse the first lines for the searched element/attribute. But this seems to be not so professional to me.
Does anybody know a better way to achieve this?
Use the XmlReader API with
using (XmlReader xr = XmlReader.Create(filePath))
{
xr.MoveToContent();
if (xr.LocalName == "ApplicationLog" ...)
}
You can try the solution provided here or use/develop a SAX reader such as this one. You can find more information on SAX here.

C# avoid overwritten xml file

I created a C# Application to write data to xml file. It overrides all the new Data,
How to Avoid..?
Please help me.
This is my code
namespace BarcodeScaner
{
class WriteFile
{
DateTime dt=DateTime.Now;
public WriteFile()
{
}
public void createFile(string ptr,string value)
{
using (XmlWriter writer = XmlWriter.Create("Data.xml"))
{
writer.WriteStartDocument();
writer.WriteStartElement("Products");
writer.WriteStartElement("Details");
writer.WriteAttributeString("PTR", ptr);
writer.WriteAttributeString("Value", value);
writer.WriteAttributeString("DateTime", dt.ToString());
writer.WriteEndElement();
writer.WriteEndDocument();
}
}
}
}
You can use XmlDocument to add custom nodes to an existing Xml Document:
XmlDocument doc = new XmlDocument();
doc.Load("Data.xml");
XmlElement el = doc.CreateElement("child");
el.InnerText = "Example of data being appendeed";
doc.DocumentElement.AppendChild(el);
doc.Save("test.xml");
You need to read the existing file, add your data to it. And then write the combined results to the file.
Because of the format of XML files, you can't simply append new data to the file as you might with some file formats.
I'm assuming that what you're wanting to do is add new data to an existing XML file, correct?
You can't just append data to an XML file, and your code actually creates a new file each time it's run, overwriting the old one.
What you need to do is read the XML data into memory, add the new nodes, and write the whole file out again. If there is a lot of data then it may be more efficient to stream from one file to another and insert the new nodes as you go.

Unable to save changes to XML document stored in Sharepoint 2010 Document Library

I am working on a project that requires all SQL connection and query information to be stored in XML files. To make my project configurable, I am trying to create a means to let the user configure his sql connection string information (datasource, catalog, username and password) via a series of text boxes. This input will then be saved to the appropriate node within the SQL document.
I can get the current information from the XML file, and display that information within text boxes for the user's review and correction, but I'm encountering an error when it comes time to save the changes.
Here is the code I'm using to update and save the xml document.
protected void submitBtn_Click(object sender, EventArgs e)
{
SPFile file = methods.web.GetFile("MyXMLFile.xml");
myDoc = new XmlDocument();
byte[] bites = file.OpenBinary();
Stream strm1 = new MemoryStream(bites);
myDoc.Load(strm1);
XmlNode node;
node = myDoc.DocumentElement;
foreach (XmlNode node1 in node.ChildNodes)
{
foreach (XmlNode node2 in node1.ChildNodes)
{
if (node2.Name == "name1")
{
if (node2.InnerText != box1.Text)
{
}
}
if (node2.Name == "name2")
{
if (node2.InnerText != box2.Text)
{
}
}
if (node2.Name == "name3")
{
if (node2.InnerText != box3.Text)
{
node2.InnerText = box3.Text;
}
}
if (node2.Name == "name4")
{
if (node2.InnerText != box4.Text)
{
}
}
}
}
myDoc.Save(strm1);
}
Most of the conditionals are empty at this point because I'm still testing.
The code works great until the last line, as I said. At that point, I get the error "Memory Stream is not expandable." I understand that using a memory stream to update a stored file is incorrect, but I can't figure out the right way to do this.
I've tried to implement the solution given in the similar question at Memory stream is not expandable but that situation is different from mine and so the implementation makes no sense to me. Any clarification would be greatly appreciated.
Using the MemoryStream constructor that takes a byte array as an argument creates a non-resizable instance of a MemoryStream. Since you are making changes to the file (and therefore the underlying bytes), you need a resizable MemoryStream. This can be accomplished by using the parameterless constructor of the MemoryStream class and writing the byte array into the MemoryStream.
Try this:
SPFile file = methods.web.GetFile("MyXMLFile.xml");
myDoc = new XmlDocument();
byte[] bites = file.OpenBinary();
using(MemoryStream strm1 = new MemoryStream()){
strm1.Write(bites, 0, (int)bites.Length);
strm1.Position = 0;
myDoc.Load(strm1);
// all of your edits to the file here
strm1.Position = 0;
// save the file back to disk
using(var fs = new FileStream("FILEPATH",FileMode.Create,FileAccess.ReadWrite)){
myDoc.Save(fs);
}
}
To get the FILEPATH for a Sharepoint file, it'd be something along these lines (I don't have a Sharepoint development environment set up right now):
SPFile file = methods.web.GetFile("MyXMLFile.xml")
var filepath = file.ParentFolder.ServerRelativeUrl + "\\" + file.Name;
Or it might be easier to just use the SaveBinary method of the SPFile class like this:
// same code from above
// all of your edits to the file here
strm1.Position = 0;
// don't use a FileStream, just SaveBinary
file.SaveBinary(strm1);
I didn't test this code, but I've used it in Sharepoint solutions to modify XML (mainly OpenXML) documents in Sharepoint lists. Read this blogpost for more information
You could look into using the XDocument class instead of XmlDocument class.
http://msdn.microsoft.com/en-us/library/system.xml.linq.xdocument.aspx
I prefer it because of the simplicity and it eliminates having to use Memory Stream.
Edit: You can append to the file like this:
XDocument doc = XDocument.Load('filePath');
doc.Root.Add(
new XElement("An Element Name",
new XAttribute("An Attribute", "Some Value"),
new XElement("Nested Element", "Inner Text"))
);
doc.Save(filePath);
Or you can search for an element and update like this:
doc.Root.Elements("The element").First(m =>
m.Attribute("An Attribute").Value == "Some value to match").SetElementValue(
"The element to change", "Value to set element to");
doc.Save('filePath');

determine if xml file contains data - c#

How do i know if my XML file has data besides the name space info:
Some of the files contain this:
<?xml version="1.0" encoding="UTF-8"?>
And if i encounter such a file, i want to place the file in an error directory
You could use the XmlReader to avoid the overhead of XmlDocument. In your case, you will receive an exception because the root element is missing.
string xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>";
using (StringReader strReader = new StringReader(xml))
{
//You can replace the StringReader object with the path of your xml file.
//In that case, do not forget to remove the "using" lines above.
using (XmlReader reader = XmlReader.Create(strReader))
{
try
{
while (reader.Read())
{
}
}
catch (XmlException ex)
{
//Catch xml exception
//in your case: root element is missing
}
}
}
You can add a condition in the while(reader.Read()) loop after you checked the first nodes to avoid to read the entire xml file since you just want to check if the root element is missing.
I think the only way is to catch an exception when you try and load it, like this:
try
{
System.Xml.XmlDocument doc = new System.Xml.XmlDocument();
doc.Load(Server.MapPath("XMLFile.xml"));
}
catch (System.Xml.XmlException xmlEx)
{
if (xmlEx.Message.Contains("Root element is missing"))
{
// Xml file is empty
}
}
Yes, there is some overhead, but you should be performing sanity checks like this anyway. You should never trust input and the only way to reliably verify it is XML is to treat it like XML and see what .NET says about it!
XmlDocument xDoc = new XmlDocument();
if (xDoc.ChildNodes.Count == 0)
{ // xml document is empty }
if (xDoc.ChildNodes.Count == 1)
{ // in xml document is only declaration node. (if you are shure that declaration is allways at the begining }
if (xDoc.ChildNodes.Count > 1)
{ // there is declaration + n nodes (usually this count is 2; declaration + root node) }
Haven't tried this...but should work.
try
{
XmlDocument doc = new XmlDocument();
doc.Load("test.xml");
}
catch (XmlException exc)
{
//invalid file
}
EDIT: Based on feedback comments
For large XML documents see Thomas's answer. This approach can have performance issues.
But, if it is a valid xml and the program wants to process it then this approach seems better.
If you aren't worried about validity, just check to see if there is anything after the first ?>. I'm not entirely sure of the C# syntax (it's been too long since I used it), but read the file, look for the first instance of ?>, and see if there is anything after that index.
However, if you want to use the XML later or you want to process the XML later, you should consider PK's answer and load the XML into an XmlDocument object. But if you have large XML documents that you don't need to process, then a solution more like mine, reading the file as text, might have less overhead.
You could check if the xml document has a node (the root node) and check it that node has inner text or other children.
As long as you aren't concerned with the validity of the XML document, and only want to ensure that it has a tag other than the declaration, you could use simple text processing:
var regEx = new RegEx("<[A-Za-z]");
bool foundTags = false;
string curLine = "";
using (var reader = new StreamReader(fileName)) {
while (!reader.EndOfStream) {
curLine = reader.ReadLine();
if (regEx.Match(curLine)) {
foundTags = true;
break;
}
}
}
if (!foundTags) {
// file is bad, copy.
}
Keep in mind that there's a million other reasons that the file may be invalid, and the code above would validate a file consisting only of "<a". If your intent is to validate that the XML document is capable of being read, you should use the XmlDocument approach.

Categories

Resources