Develop Reference

Convert a really-large collection to Xml in .NET

I have a database result (as a list of items) that is huge. 450K items are returned.
I then use Linq-To-Xml to convert this collection to an XDocument. It works - but it's a huge memory hit.
Is it possible to convert the collection items to XML but not all in memory, at once .. but streaming to a file as the conversation happens?
The XML data finally saves to disk at over 1Gig. So it's fair to assume that the memory consumption will at least be this.
So - is there a way to stream the XML segments to disk, as we iterate over each item in the list instead of converting the entire result-set to an InMemory XDocument and then saving this to disk?
NOTE: Please do not make suggestions about breaking up the list into smaller parts, etc. I understand that, but I've ruled that out.
Here's some sample code I'm doing (to help give you an idea).
// Create the xml doc.
var elements = from user in userResults
select new XElement("user",
new XElement("id", user.Id),
.....<snip>...... );
return new XDocument(new XDeclaration("1.0", "utf-8", "yes"),
new XElement("users", elements));
// Save the doc to the filesystem.
using (var writer = _fileSystemWrapper.CreateText(destinationXmlFileName))
{
xmldDocument.Save(writer);
}
Update
Maybe there's some other tricks like using Linq-To-Xml to create element segments in batches of 10 .. and for each 10, append that to the end of a file?

You can use XStreamingElement. It would require some small refactoring though, mainly not creating XDocument and using different Save method.
Here is the corresponding sample to yours just to give you an idea:
var elements = from user in userResults
select new XElement("user",
new XElement("id", user.Id),
.....<snip>...... );
var content = new XStreamingElement("users", elements);
using (var output = _fileSystemWrapper.CreateText(destinationXmlFileName))
using (var writer = XmlWriter.Create(output, new XmlWriterSettings { Encoding = Encoding.UTF8, Indent = true })
{
// Use the next line if you don't require standalone="yes" attribute
// content.Save(writer);
writer.WriteStartDocument(true);
content.WriteTo(writer);
writer.WriteEndDocument();
}

Adding child nodes to an XElement

I am trying to append objects into an XML file. The problem I currently have is it appends everything at the first level itself. I am trying to have the list as the parent element and list items as the child elements.
What I've tried: I came across a few posts where they use loops but I am unable to relate that to my context and code.
Code:
XDocument xDocument = XDocument.Load(#"C:\Users\hci\Desktop\Nazish\TangramsTool\TangramsTool\patterndata.xml");
XElement root = xDocument.Element("Patterns");
foreach (Pattern currentPattern in PatternDictionary.Values)
{
String filePath = currentPattern.Name.ToString();
IEnumerable<XElement> rows = root.Descendants("Pattern"); // Returns a collection of the descendant elements for this document or element, in document order.
XElement firstRow = rows.First(); // Returns the first element of a sequence.
if (currentPattern.PatternDistancesList.Count() == 9)
{
firstRow.AddBeforeSelf( //Adds the specified content immediately before this node.
new XElement("Pattern"),
new XElement("Name", filePath.Substring(64)),
new XElement("PatternDistancesList"),
new XElement("PatternDistance", currentPattern.PatternDistancesList[0].ToString()),
new XElement("PatternDistance", currentPattern.PatternDistancesList[1].ToString()),
}
}
Current XML File:
<Pattern/>
<Name>match.jpg</Name>
<PatternDistancesList/>
<PatternDistance>278</PatternDistance>
<PatternDistance>380</PatternDistance>
What I would like as the end result:
<Pattern>
<Name>match.jpg</Name>
<PatternDistancesList>
<PatternDistance>278</PatternDistance>
<PatternDistance>380</PatternDistance>
</PatternDistancesList>
<Pattern/>
Any tips will be much appreciated. I'm new to WPF and C# so still trying to learn things.

This should do the trick:
firstRow.AddBeforeSelf(
new XElement("Pattern",
new XElement("Name", filePath.Substring(64)),
new XElement("PatternDistancesList",
new XElement("PatternDistance", currentPattern.PatternDistancesList[0].ToString()),
new XElement("PatternDistance", currentPattern.PatternDistancesList[1].ToString()))));

How to generate an XML file dynamically using XDocument?

As I wrote in the subject itself , how can I do that?
Note that solution like this are not appropriate as I want to create child nodes dynamically through running..
new XDocument(
new XElement("root",
new XElement("someNode", "someValue")
)
)
.Save("foo.xml");
I guess this was clear enough the first time but I will write it again:
I need to be able to add child nodes to given parent node while running, in the current syntax I've written this is static generated xml which doesn't contribute me at all because all is known in advance, which is not as my case.
How would you do it with Xdocument, is there away?

If a document has a defined structure and should be filled with dynamic data, you can go like this:
// Setup base structure:
var doc = new XDocument(root);
var root = new XElement("items");
doc.Add(root);
// Retrieve some runtime data:
var data = new[] { 1, 2, 3, 4, 5 };
// Generate the rest of the document based on runtime data:
root.Add(data.Select(x => new XElement("item", x)));

Very simple
Please update your code accordingly
XmlDocument xml = new XmlDocument();
XmlElement root = xml.CreateElement("children");
xml.AppendChild(root);
XmlComment comment = xml.CreateComment("Children below...");
root.AppendChild(comment);
for(int i = 1; i < 10; i++)
{
XmlElement child = xml.CreateElement("child");
child.SetAttribute("age", i.ToString());
root.AppendChild(child);
}
string s = xml.OuterXml;

How to iterate through XElement Nodes of XDocument and add to another XElelement?

I have a XML document that looks like this:
<Person>
<LastName>LastName1</LastName>
<FirstName>FirstName1</FirstName>
<MiddleName>MiddleName1</MiddleName>
</Person>
Originally I had a method to create this structure like below:
public XElement ToXML()
{
return new XElement("Person",
new XElement(this.LastName.ToXML()),
new XElement(this.FirstName.ToXML()),
new XElement(this.MiddleName.ToXML()));
}
The problem is there are a lot of other nodes other than just Person that use the Name values within the root. So what I tried doing was refactoring out the FirstName, LastName, and MiddleName elements to be in their own reusable class with a ToXMLDoc() method that returns those elements as a XDocument instead of a XElement (since the root will be dictated by the class needing the name children; might be Person, Employee, etc.)
This is what my new ToXMLDoc returns:
return new XDocument(new XElement(this.LastName.ToXML()),
new XElement(this.FirstName.ToXML()),
new XElement(this.MiddleName.ToXML()));
My problem is I want to now add this content to within the root XElement added by my Person class. I tried doing something like below, but I'm not using the constructor properly and getting a "Ambiguous constructor reference" error.
return new XElement("Person",
foreach (XElement xe in NameType.ToXMLDoc().Nodes())
{
new XElement(xe.Value);
}
);
How can I take the contents from the ToXMLDoc() method and add them to the XElement node being created for Person? Any help is appreciated, thanks!

You're currently trying to embed a foreach loop within a constructor call. That's not going to work - but it's actually pretty simple:
return new XElement("Person", NameType.ToXMLDoc()
.Nodes()
.Select(xe => new XElement(xe.Value));
Are you sure you don't just want to copy the elements wholesale though? In which case it would just be:
return new XElement("Person", NameType.ToXMLDoc().Nodes());

I would return an XElement, rather than an XDocument
This way you could just do:
new XElement("Person", NameType.ToXMLDoc())

C# - Unable to save a customized linq-to-xml file, with new elements in it?

I'm creating a new XDocument and inserting a root element "profiles" in it, then saving.
if (!System.IO.File.Exists("profiles.xml"))
{
XDocument doc = new XDocument(
new XElement("profiles")
);
doc.Save("profiles.xml", SaveOptions.None);
}
And then later I wanna take users input and add profiles into the already created xml file:
XElement profile =
new XElement(Player.Name,
new XElement("level", Player.Level),
new XElement("cash", Player.Cash)
);
XDocument doc = XDocument.Load("profiles.xml");
List<XElement> profiles = doc.Root.Elements().ToList();
for (int i = 0; i < profiles.Count; i++)
{
if (profiles[i].Name.ToString() == Player.name)
{
profiles[i] = profile;
return;
}
}
profile.Add(profile);
doc.Save("profiles.xml", SaveOptions.None);
But for some reason, it will never add any new profiles?
EDIT: Also, if I manually create a new profile into the xml file, it won't customize either, so the problem is within Saving the file?

You're never actually doing anything to change any of the elements within the XDocument that doc refers to:
If you find an element with the existing name, you're modifying the list, but that won't modify the document. You probably want to use XElement.ReplaceWith:
profiles[i].ReplaceWith(profile);
Note that in this case you're not even trying to save the XML file again (due to the return statement), so it's not really clear what you're trying to achieve in this case.
If you don't find the element, you're adding the profile element to itself, which certainly isn't going to modify the document. I suspect you want:
doc.Root.Add(profile);
In other words, add the new profile element as a new final child of the root element.
EDIT: Here's a different approach to try instead - I'm assuming any one name should only occur once:
XDocument doc = XDocument.Load("profiles.xml");
var existingElement = doc.Root
.Elements()
.Where(x => x.Name.ToString() == Player.name)
.FirstOrDefault();
if (existingElement != null)
{
existingElement.ReplaceWith(profile);
}
else
{
doc.Root.Add(profile);
}
doc.Save("profiles.xml", SaveOptions.None);
Also, I would strongly advise you not to use the player name as the element name. Use it as an attribute value or text value instead, e.g.
XElement profile =
new XElement("player",
new XAttribute("name", Player.Name),
new Attribute("level", Player.Level),
new XAttribute("cash", Player.Cash)
);
That way you won't have problems if the player name has spaces etc. You'd then need to change your query to:
var existingElement = doc.Root
.Elements()
.Where(x => (string) x.Attribute("name)" == Player.name)
.FirstOrDefault();