Develop Reference

Convert a really-large collection to Xml in .NET

I have a database result (as a list of items) that is huge. 450K items are returned.
I then use Linq-To-Xml to convert this collection to an XDocument. It works - but it's a huge memory hit.
Is it possible to convert the collection items to XML but not all in memory, at once .. but streaming to a file as the conversation happens?
The XML data finally saves to disk at over 1Gig. So it's fair to assume that the memory consumption will at least be this.
So - is there a way to stream the XML segments to disk, as we iterate over each item in the list instead of converting the entire result-set to an InMemory XDocument and then saving this to disk?
NOTE: Please do not make suggestions about breaking up the list into smaller parts, etc. I understand that, but I've ruled that out.
Here's some sample code I'm doing (to help give you an idea).
// Create the xml doc.
var elements = from user in userResults
select new XElement("user",
new XElement("id", user.Id),
.....<snip>...... );
return new XDocument(new XDeclaration("1.0", "utf-8", "yes"),
new XElement("users", elements));
// Save the doc to the filesystem.
using (var writer = _fileSystemWrapper.CreateText(destinationXmlFileName))
{
xmldDocument.Save(writer);
}
Update
Maybe there's some other tricks like using Linq-To-Xml to create element segments in batches of 10 .. and for each 10, append that to the end of a file?

You can use XStreamingElement. It would require some small refactoring though, mainly not creating XDocument and using different Save method.
Here is the corresponding sample to yours just to give you an idea:
var elements = from user in userResults
select new XElement("user",
new XElement("id", user.Id),
.....<snip>...... );
var content = new XStreamingElement("users", elements);
using (var output = _fileSystemWrapper.CreateText(destinationXmlFileName))
using (var writer = XmlWriter.Create(output, new XmlWriterSettings { Encoding = Encoding.UTF8, Indent = true })
{
// Use the next line if you don't require standalone="yes" attribute
// content.Save(writer);
writer.WriteStartDocument(true);
content.WriteTo(writer);
writer.WriteEndDocument();
}

Linq to XML is Reducing File Size

I have an XML file that begins at 1,030KB. After I run my code, which is adding an element to a section of the xml file, the file reduces to a size of 580kb. I think this might be a reason why a secondary application reading this file, cannot read the newly generated file. I am not sure why this is happening. Is there a way to stop Linq to XML from compressing the file?
XDocument xDoc = XDocument.Load(cust_file);
XElement parentXElement = xDoc.XPathSelectElement("LastLayout2/CommandBars");
XElement refXElement = xDoc.XPathSelectElement("LastLayout2/CommandBars/CommandBar[#Title = 'Standard']/Controls");
XElement temp = refXElement.XPathSelectElement("Control[#Parameter = 'GLOBAL!QMS_Launcher.Main']");
if (temp == null)
{
XElement newElement = new XElement("Control");
XAttribute classAt = new XAttribute("Class", "CXTPControlButton");
XAttribute idAt = new XAttribute("Id", "0");
XAttribute paramAt = new XAttribute("Parameter", "GLOBAL!QMS_Launcher.Main");
XAttribute custIdAt = new XAttribute("CustomIconId", "68267");
XElement customIcon = new XElement("CustomIcon");
XElement icon = new XElement("Icon");
XAttribute width = new XAttribute("Width", "16");
XAttribute data = new XAttribute("Data", "ABAAAAAAFCAAAAAAEDAAICAAAAAAABAAAAAAABAAAAAABAAAACAADAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPPAAAAPPAAAAPPAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPGAAAAAAJNAAAAAAFCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAAAAAAAHFAAAAAACHAAAAAAALAAAAAAPPAAAAAAJPAAAAAAHCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPFAAAAAAJOAAAAAAPPAAAAAAPPAAAAAAPPAAAAAALOABJAAACGDNJIAADNGJCGAAGJGAEAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGDAAAAAANPAAAAAAMPAAAAAAJLAAAAAAMHAAAAAAFFAAAAAAODAAAAAAAKMGFEAAEIPPGKAAPPPHDFAAPHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEMAAAAAAPPAAAAAAPGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANGAAAAAAPPCBMAAAIFIHOEAAIHJFJDAAJFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOOAAAAAAEPAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPHAAAAAAPPGBOAAAKGILHHAAILILHHAAILJJEGAAJJFEMCAAFEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEPAAAAAAEOAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHJAAAAAAPPGCJBAAAHPPGKAAPPPPGKAAPPPPGKAAPPMPEKAAMPPDJCAAPDAAAAAAAAAAAAAAAAAAAAAAOMAAAAAAPPAAAAAAJBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAALLAAAAAAPPBCFBAAKFILHHAAILILHHAAILILHHAAILILHHAAILKGFEAAKGAAAAAAAAAAAAAAAAAAAAAANJAAAAAAPPAAAAAANMAAAAAABCAAAAAANBAAAAAABIAAAAAAPPAAAAAAHOAEKCAAMEIHOEAAIHIHOEAAIHIHOEAAIHIHOEAAIHJGDEAAJGGCJBAAGCAAAAAAAAAAAAAAFBAAAAAADOAAAAAAPPAAAAAAPPAAAAAAPPAAAAAAPPAAAAAALPMAIAAAHGNOKJAANOPPGKAAPPPPGKAAPPPPGKAAPPPPGKAAPPPPGKAAPPOPFKAAOPKFKDAAKFAAAAAAAAAAAAAAKBAAAAAANJAAAAAABPAAAAAABPAAAAAAAMJBPAAALELMEIAALMPPGKAAPPPPGKAAPPPPGKAAPPPPGKAAPPPPGKAAPPPPGKAAPPPPGKAAPPINMIAAINAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
icon.Add(width, data);
customIcon.Add(icon);
newElement.Add(customIcon);
newElement.Add(classAt, idAt, paramAt, custIdAt);
xDoc.Element("LastLayout2").Element("CommandBars").Elements("CommandBar")
.FirstOrDefault(item => item.Attribute("Title") != null && item.Attribute("Title").Value == "Standard")
.Element("Controls").Add(newElement);
xDoc.Save(cust_file);

It sounds like an encoding issue. You're probably saving down as a different encoding to the original document, and different from what the target application is expecting. Hence the change in file size.
It's possible to change the encoding used for the save as described by dbc above:
Create an XmlWriterSettings, set XmlWriterSettings.Encoding as
appropriate, then create an XmlWriter and pass it to XDocument.Save().

Adding child nodes using c# Xdocument class

I have an xml file as given below.
<?xml version="1.0" encoding="utf-8"?>
<file:Situattion xmlns:file="test">
<file:Properties>
</file:Situattion>
I would like to add the child element file:Character using xDocument.So that my final xml would be like given below
<?xml version="1.0" encoding="utf-8"?>
<file:Situattion xmlns:file="test">
<file:Characters>
<file:Character file:ID="File0">
<file:Value>value0</file:Value>
<file:Description>
Description0
</file:Description>
</file:Character>
<file:Character file:ID="File1">
<file:Value>value1</file:Value>
<file:Description>
Description1
</file:Description>
</file:Character>
</file:Characters>
Code in c# i tried using Xdocument class is given below.
XNamespace ns = "test";
Document = XDocument.Load(Folderpath + "\\File.test");
if (Document.Descendants(ns + "Characters") != null)
{
Document.Add(new XElement(ns + "Character"));
}
Document.Save(Folderpath + "\\File.test");
At line "Document.Add(new XElement(ns + "Character"));", I am getting an error:
"This operation would create an incorrectly structured document.".
How can I add the node under "file:Characters".

You're trying to add an extra file:Character element directly into the root. You don't want to do that - you want to add it under the file:Characters element, presumably.
Also note that Descendants() will never return null - it will return an empty sequence if there are no matching elements. So you want:
var ns = "test";
var file = Path.Combine(folderPath, "File.test");
var doc = XDocument.Load(file);
// Or var characters = document.Root.Element(ns + "Characters")
var characters = document.Descendants(ns + "Characters").FirstOrDefault();
if (characters != null)
{
characters.Add(new XElement(ns + "Character");
doc.Save(file);
}
Note that I've used more conventional naming, Path.Combine, and also moved the Save call so that you'll only end up saving if you've actually made a change to the document.

Document.Root.Element("Characters").Add(new XElement("Character", new XAttribute("ID", "File0"), new XElement("Value", "value0"), new XElement("Description")),
new XElement("Character", new XAttribute("ID", "File1"), new XElement("Value", "value1"), new XElement("Description")));
Note: I have not included the namespace for brevity. You have to add those.

How to generate an XML file dynamically using XDocument?

As I wrote in the subject itself , how can I do that?
Note that solution like this are not appropriate as I want to create child nodes dynamically through running..
new XDocument(
new XElement("root",
new XElement("someNode", "someValue")
)
)
.Save("foo.xml");
I guess this was clear enough the first time but I will write it again:
I need to be able to add child nodes to given parent node while running, in the current syntax I've written this is static generated xml which doesn't contribute me at all because all is known in advance, which is not as my case.
How would you do it with Xdocument, is there away?

If a document has a defined structure and should be filled with dynamic data, you can go like this:
// Setup base structure:
var doc = new XDocument(root);
var root = new XElement("items");
doc.Add(root);
// Retrieve some runtime data:
var data = new[] { 1, 2, 3, 4, 5 };
// Generate the rest of the document based on runtime data:
root.Add(data.Select(x => new XElement("item", x)));

Very simple
Please update your code accordingly
XmlDocument xml = new XmlDocument();
XmlElement root = xml.CreateElement("children");
xml.AppendChild(root);
XmlComment comment = xml.CreateComment("Children below...");
root.AppendChild(comment);
for(int i = 1; i < 10; i++)
{
XmlElement child = xml.CreateElement("child");
child.SetAttribute("age", i.ToString());
root.AppendChild(child);
}
string s = xml.OuterXml;

C# - Unable to save a customized linq-to-xml file, with new elements in it?

I'm creating a new XDocument and inserting a root element "profiles" in it, then saving.
if (!System.IO.File.Exists("profiles.xml"))
{
XDocument doc = new XDocument(
new XElement("profiles")
);
doc.Save("profiles.xml", SaveOptions.None);
}
And then later I wanna take users input and add profiles into the already created xml file:
XElement profile =
new XElement(Player.Name,
new XElement("level", Player.Level),
new XElement("cash", Player.Cash)
);
XDocument doc = XDocument.Load("profiles.xml");
List<XElement> profiles = doc.Root.Elements().ToList();
for (int i = 0; i < profiles.Count; i++)
{
if (profiles[i].Name.ToString() == Player.name)
{
profiles[i] = profile;
return;
}
}
profile.Add(profile);
doc.Save("profiles.xml", SaveOptions.None);
But for some reason, it will never add any new profiles?
EDIT: Also, if I manually create a new profile into the xml file, it won't customize either, so the problem is within Saving the file?

You're never actually doing anything to change any of the elements within the XDocument that doc refers to:
If you find an element with the existing name, you're modifying the list, but that won't modify the document. You probably want to use XElement.ReplaceWith:
profiles[i].ReplaceWith(profile);
Note that in this case you're not even trying to save the XML file again (due to the return statement), so it's not really clear what you're trying to achieve in this case.
If you don't find the element, you're adding the profile element to itself, which certainly isn't going to modify the document. I suspect you want:
doc.Root.Add(profile);
In other words, add the new profile element as a new final child of the root element.
EDIT: Here's a different approach to try instead - I'm assuming any one name should only occur once:
XDocument doc = XDocument.Load("profiles.xml");
var existingElement = doc.Root
.Elements()
.Where(x => x.Name.ToString() == Player.name)
.FirstOrDefault();
if (existingElement != null)
{
existingElement.ReplaceWith(profile);
}
else
{
doc.Root.Add(profile);
}
doc.Save("profiles.xml", SaveOptions.None);
Also, I would strongly advise you not to use the player name as the element name. Use it as an attribute value or text value instead, e.g.
XElement profile =
new XElement("player",
new XAttribute("name", Player.Name),
new Attribute("level", Player.Level),
new XAttribute("cash", Player.Cash)
);
That way you won't have problems if the player name has spaces etc. You'd then need to change your query to:
var existingElement = doc.Root
.Elements()
.Where(x => (string) x.Attribute("name)" == Player.name)
.FirstOrDefault();