Develop Reference

XmlReader behaves different with line breaks

If the data is on a single line the
index=int.Parse(logDataReader.ReadElementContentAsString());
and
value=double.Parse(logDataReader.ReadElementContentAsString(),
cause the cursor to move forward. If I take those calls out I see it loop 6 times in debug.
In the following only 3 <data> are read (and they are wrong as the value is for the next index) on the first (<logData id="Bravo">). On the second (<logData id="Bravo">) all <data> are read.
It is not an option to edit the xml and put in line breaks as that file is created dynamically (by XMLwriter). The NewLineChars setting is a line feed. From XMLwriter it is actually just one line - I broke it down to figure out where it was breaking. In the browser it is displayed properly.
How to fix this?
Here is my XML:
<?xml version="1.0" encoding="utf-8"?>
<log>
<logData id="Alpha">
<data><index>100</index><value>150</value></data>
<data><index>110</index><value>750</value></data>
<data><index>120</index><value>750</value></data>
<data><index>130</index><value>150</value></data>
<data><index>140</index><value>0</value></data>
<data><index>150</index><value>222</value></data>
</logData>
<logData id="Bravo">
<data>
<index>100</index>
<value>25</value>
</data>
<data>
<index>110</index>
<value>11</value>
</data>
<data>
<index>120</index>
<value>1</value>
</data>
<data>
<index>130</index>
<value>25</value></data>
<data>
<index>140</index>
<value>0</value>
</data>
<data>
<index>150</index>
<value>1</value>
</data>
</logData>
</log>
And my code:
static void Main(string[] args)
{
List<LogData> logDatas = GetLogDatasFromFile("singleVersusMultLine.xml");
Debug.WriteLine("Main");
Debug.WriteLine("logData");
foreach (LogData logData in logDatas)
{
Debug.WriteLine($" logData.ID {logData.ID}");
foreach(LogPoint logPoint in logData.LogPoints)
{
Debug.WriteLine($" logData.Index {logPoint.Index} logData.Value {logPoint.Value}");
}
}
Debug.WriteLine("end");
}
public static List<LogData> GetLogDatasFromFile(string xmlFile)
{
List<LogData> logDatas = new List<LogData>();
using (XmlReader reader = XmlReader.Create(xmlFile))
{
// move to next "logData"
while (reader.ReadToFollowing("logData"))
{
var logData = new LogData(reader.GetAttribute("id"));
using (var logDataReader = reader.ReadSubtree())
{
// inside "logData" subtree, move to next "data"
while (logDataReader.ReadToFollowing("data"))
{
// move to index
logDataReader.ReadToFollowing("index");
// read index
var index = int.Parse(logDataReader.ReadElementContentAsString());
// move to value
logDataReader.ReadToFollowing("value");
// read value
var value = double.Parse(logDataReader.ReadElementContentAsString(), CultureInfo.InvariantCulture);
logData.LogPoints.Add(new LogPoint(index, value));
}
}
logDatas.Add(logData);
}
}
return logDatas;
}
public class LogData
{
public string ID { get; }
public List<LogPoint> LogPoints { get; } = new List<LogPoint>();
public LogData (string id)
{
ID = id;
}
}
public class LogPoint
{
public int Index { get; }
public double Value { get; }
public LogPoint ( int index, double value)
{
Index = index;
Value = value;
}
}

Your problem is as follows. According to the documentation for XmlReader.ReadElementContentAsString():
This method reads the start tag, the contents of the element, and moves the reader past the end element tag.
And from the documentation for XmlReader.ReadToFollowing(String):
It advances the reader to the next following element that matches the specified name and returns true if a matching element is found.
Thus, after the call to ReadElementContentAsString(), since the reader has been advanced to the next node, it might already be positioned on the next <value> or <data> node. Then when you call ReadToFollowing() this element node is skipped because the method unconditionally moves on to the next node with the correct name. But if the XML is indented then the next node immediately after the call to ReadElementContentAsString() will be an XmlNodeType.Whitespace node, protecting against this bug.
The solution is to check whether the reader is already positioned correctly after the call to ReadElementContentAsString(). First, introduce the following extension method:
public static class XmlReaderExtensions
{
public static bool ReadToFollowingOrCurrent(this XmlReader reader, string localName, string namespaceURI)
{
if (reader == null)
throw new ArgumentNullException(nameof(reader));
if (reader.NodeType == XmlNodeType.Element && reader.LocalName == localName && reader.NamespaceURI == namespaceURI)
return true;
return reader.ReadToFollowing(localName, namespaceURI);
}
}
Then modify your code as follows:
public static List<LogData> GetLogDatasFromFile(string xmlFile)
{
List<LogData> logDatas = new List<LogData>();
using (XmlReader reader = XmlReader.Create(xmlFile))
{
// move to next "logData"
while (reader.ReadToFollowing("logData", ""))
{
var logData = new LogData(reader.GetAttribute("id"));
using (var logDataReader = reader.ReadSubtree())
{
// inside "logData" subtree, move to next "data"
while (logDataReader.ReadToFollowing("data", ""))
{
// move to index
logDataReader.ReadToFollowing("index", "");
// read index
var index = XmlConvert.ToInt32(logDataReader.ReadElementContentAsString());
// move to value
logDataReader.ReadToFollowingOrCurrent("value", "");
// read value
var value = XmlConvert.ToDouble(logDataReader.ReadElementContentAsString());
logData.LogPoints.Add(new LogPoint(index, value));
}
}
logDatas.Add(logData);
}
}
return logDatas;
}
Notes:
Always prefer to use XmlReader methods in which the local name and namespace are specified separately, such as XmlReader.ReadToFollowing (String, String). When you use a method such as XmlReader.ReadToFollowing(String) which accepts a single qualified name, you are implicitly hardcoding the choice of XML prefix, which is generally not a good idea. XML parsing should be independent of prefix choice.
While you correctly parsed your double using the CultureInfo.InvariantCulture locale, it's even easier to use the methods from the XmlConvert class to handle parsing and formatting correctly.
XmlReader.ReadSubtree() leaves the XmlReader positioned on the EndElement node of the element being read, so you shouldn't need to call ReadToFollowingOrCurrent() afterwards. (Nice use of ReadSubtree() to avoid reading too little or too much by the way; by using this method one can avoid several frequent mistakes with XmlReader.)
As you have found, code that manually reads XML using XmlReader should always be unit-tested with both formatted and unformatted XML, because certain bugs will only arise with one or the other. (See e.g. this answer, this one and this one also for other examples of such.)
Working sample .Net fiddle here.

Indeed that code (which I provided to you in your another question) is wrong. ReadToFollowing will read to the next element with this name even if it's cursor is already positioned on element with this name. When there is a whitespace - after you read index, cursor moves to that whitespace and ReadToFollowing("value") works as you expect. However, if there is no whitespace, cursor is already on value node and so ReadToFollowing("value") reads to the next "value" in subsequent "data" node.
I think the following would be a safer approach:
public static List<LogData> GetLogDatasFromFile(string xmlFile) {
List<LogData> logDatas = new List<LogData>();
using (XmlReader reader = XmlReader.Create(xmlFile)) {
LogData currentData = null;
while (reader.Read()) {
if (reader.IsStartElement("logData")) {
// we are positioned on start of logData
if (currentData != null)
logDatas.Add(currentData);
currentData = new LogData(reader.GetAttribute("id"));
}
else if (reader.IsStartElement("data")) {
// we are on start of "data"
// we always have "currentData" at this point
Debug.Assert(currentData != null);
reader.ReadToFollowing("index");
var index = int.Parse(reader.ReadElementContentAsString());
// check if we are not already on "value"
if (!reader.IsStartElement("value"))
reader.ReadToFollowing("value");
var value = double.Parse(reader.ReadElementContentAsString(), CultureInfo.InvariantCulture);
currentData.LogPoints.Add(new LogPoint(index, value));
}
}
if (currentData != null)
logDatas.Add(currentData);
}
return logDatas;
}

I found a fix but to me not an acceptable answer. XMLreader should not behave differently with line breaks.
In XmlWriter this will put line breaks in the text:
XmlWriterSettings xmlWriterSettings = new XmlWriterSettings();
xmlWriterSettings.NewLineOnAttributes = true;
xmlWriterSettings.Indent = true;
using (XmlWriter xmlWriter = XmlWriter.Create(fileNameXML, xmlWriterSettings))
{
I found this here.

Using XhtmlTextWriter with XmlTextReader

After reading this article, I have decided to update the following code (using XmlDocument) with XmlReader:
Rendering controls
public string Rendering(Control baseControl)
{
StringBuilder stringBuilder = new StringBuilder();
using (StringWriter stringWriter = new StringWriter(stringBuilder))
using (XhtmlTextWriter htmlWriter = new XhtmlTextWriter(stringWriter))
{
baseControl.RenderControl(htmlWriter);
return PretifyWithNewlines(stringBuilder.ToString());
}
}
Adding newline after each node
private string PretifyWithNewlines(string minifiedMarkup)
{
XmlDocument xmlDocument = new XmlDocument();
xmlDocument.XmlResolver = null;
try
{
xmlDocument.LoadXml("<base>" + minifiedMarkup + "</base>");
}
catch // when minifiedMarkup contains the whole HTML with DTD tag defined,
{ // it throws an exception with <base>
xmlDocument.LoadXml(minifiedMarkup);
}
return recursiveOperation(xmlDocument.ChildNodes)
.Replace(Environment.NewLine + Environment.NewLine, Environment.NewLine)
.Replace(Environment.NewLine + "<base>" + Environment.NewLine, "")
.Replace(Environment.NewLine + "</base>" + Environment.NewLine, "");
}
Recursively traverse each node and plant new element
private static string recursiveOperation(XmlNodeList xmlNodeList)
{
string result = "";
foreach (XmlNode currentNode in xmlNodeList)
{
XmlNode clonedNode = currentNode;
string interimMarkup = recursiveOperation(currentNode.ChildNodes);
try
{
clonedNode.InnerXml = interimMarkup;
}
finally
{
result += Environment.NewLine + clonedNode.OuterXml + Environment.NewLine;
}
}
return result;
}
Questions:
Is there a room for optimizing the existing code?
How would I go about directly instantiating XmlTextReader from Control, StringWriter or XhtmlTextWriter object? Or do I really need to render it as a string first then instantiate XmlTextReader?
Edit
As per Jon Skeet's answer, here is the update. The idea is to implode a newline after each element:
Before prettify:
<div class="tag"><span>text<span class="another-span"></span></span></div><div>Text<img src="some/relative/URL/" />namely</div>
After prettify:
<div class="tag">
<span>
text
<span class="another-span"></span>
</span>
</div>
<div>
Text
<img src="some/relative/URL/" />
namely
</div>
Notice how span.another-span keep collapsed while everything else (with child nodes) expanded. The indentation will be asserted by Visual Studio.

Is there a room for optimizing the existing code?
Absolutely. The first place I'd change has nothing to do with how you load the XML - it's string concatenation. I'd change your recursiveOperation method to:
private static string RecursiveOperation(XmlNodeList xmlNodeList)
{
StringBuilder result = new StringBuilder();
foreach (XmlNode currentNode in xmlNodeList)
{
XmlNode clonedNode = currentNode;
// Remove try/finally block - if an exception is thrown your
// result will be lost anyway
string interimMarkup = RecursiveOperation(currentNode.ChildNodes);
clonedNode.InnerXml = interimMarkup;
result.Append(Environment.NewLine)
.Append(clonedNode.OuterXml)
.Append(Environment.NewLine);
}
return result.ToString();
}
It's possible you could optimize this further using a single StringBuilder passed into RecursiveOperation, but I haven't quite got to grips with your code sufficiently to say yet. (It's before the first coffee of the morning.)
In terms of the XML handling itself, you're currently doing a lot of reparsing by setting the OuterXml node in each child (recursively). I suspect if I had a better grasp of what you were doing, it would be feasible to change the whole approach. Given that this is functionality that XmlReader really doesn't have (it wouldn't make sense), it's not clear that you should be taking much notice of the other article at the moment.
How would I go about directly instantiating XmlTextReader from Control, StringWriter or XhtmlTextWriter object? Or do I really need to render it as a string first then instantiate XmlTextReader?
It's not clear what it would even mean to create an XmlTextReader from any of those objects - they're not inherently a source of XML data. I think what you've got already looks reasonable to me.
If you're still concerned about the performance, you should avoid guesswork and use a profiler to measure where the time is being taken. You should set yourself a target first though, otherwise you won't know when you've finished optimizing.

Remove items from a colletion with the Rally REST for .NET API

I'm using the REST toolkit, and there's lots of cool stuff, but I don't see a way in it to remove an item from a collection as is mentioned in the API documentation:
Items can be removed from Modifiable collections by sending a POST request to a collection reference url with a body containing an array of items to remove. Example payload to remove a tag from a collection by a POST request to https://rally1.rallydev.com/slm/webservice/v2.0/defect/49103/tags/remove:
Anybody done this? Got code?

v3.1.1 of the toolkit added support for adding to and removing from collections.
Check it out!
http://rallytools.github.io/RallyRestToolkitFor.NET/html/efed9f73-559a-3ef8-5cd7-e3039040c87d.htm

Since there is no convenience method in the toolkit yet to remove a collection element form the collection, in order to remove a tag from a collection of tags you have to replace an existing collection with a copy of that collection minus the change you intended to make to the original collection, and then update the story with that new tag collection.
This code removes a tag from a story. In this example I strip the last tag of the collection. For example, if I have a collection of 2 tags, the code leaves one tag, and when this code is run again, the last tag is removed and the story is no longer tagged.
As an alternative, a commented out condition skips an unwanted tag based on tag name.
using System;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System.Text;
using Rally.RestApi;
using Rally.RestApi.Response;
namespace aRESTremoveTagFromStory
{
class Program
{
static void Main(string[] args)
{
//Initialize the REST API
RallyRestApi restApi;
restApi = new RallyRestApi("user#co.com", "secret", "https://rally1.rallydev.com", "v2.0");
//Set our Workspace and Project scopings
String workspaceRef = "/workspace/1111"; //replace this OID with an OID of your workspace
String projectRef = "/project/2222"; //replace this OID with an OID of your project
bool projectScopingUp = false;
bool projectScopingDown = true;
Request storyRequest = new Request("HierarchicalRequirement");
storyRequest.Workspace = workspaceRef;
storyRequest.Project = projectRef;
storyRequest.ProjectScopeUp = projectScopingUp;
storyRequest.ProjectScopeDown = projectScopingDown;
storyRequest.Fetch = new List<string>()
{
"Name",
"FormattedID",
"Tags"
};
storyRequest.Query = new Query("FormattedID", Query.Operator.Equals, "US123"); //replace this FormattedID with FormattedID valid in your workspace
QueryResult queryStoryResults = restApi.Query(storyRequest);
//create an array to contain modified list of tags for this story.
ArrayList tagList = new ArrayList();
foreach (var s in queryStoryResults.Results)
{
Console.WriteLine("FormattedID: " + s["FormattedID"] + " Name: " + s["Name"]);
Console.WriteLine("----------");
///query for Tags collection on this story
Request tagRequest = new Request(s["Tags"]);
QueryResult queryTagResult = restApi.Query(tagRequest);
int count = 0;
if (queryTagResult.TotalResultCount > 0)
{
foreach (var t in queryTagResult.Results)
{
var tagName = t["Name"];
var tagRef = t["_ref"];
Console.WriteLine(tagName + " " + tagRef + " " + tagOID);
if (count < queryTagResult.TotalResultCount - 1) //skips the last tag
//if (tagName != "tag2") //skips the tag with name "tag2"
{
DynamicJsonObject newTag = new DynamicJsonObject();
newTag["_ref"] = tagRef;
tagList.Add(newTag);
}
count++;
}
}
else
{
Console.WriteLine("no tags found");
}
DynamicJsonObject newTagCollection = new DynamicJsonObject();
newTagCollection["Tags"] = tagList;
OperationResult updateResult = restApi.Update(s["_ref"], newTagCollection);
}
}
}
}

using List.Add() method is breaking my code out of a foreach loops

So, essentially, I'm running into an interesting issue where, when the call to the "CreateXML()" function in the following code is made, an xelement is created as intended, but then, when I attempt to add it to a collection of xeleents, instead of continuing the foreach loop from which the call to "CreateXML()" originated, the foreach loop is broken out of, and a call is made to "WriteXML()". Additionally, though an XElement is created and populated, it is not added to the List. [for clarification, the foreach loops I am referring to live in the "ParseDoc()" method]
private List<XElement> _xelemlist;
private void WriteXml()
{
XElement head = new XElement("header", new XAttribute("headerattributename", "attribute"));
foreach (XElement xelem in _xelemlist)
{
head.Add(xelem);
}
XDocument doc = new XDocument();
doc.Add(head);
}
private void CreateXML(string attname, string att)
{
XElement xelem = new XElement("name", new XElement("child", new XAttribute(attname, att), segment));
_xelemlist.Add(xelem);
}
private void ExtractSegment(HtmlNode node)
{
HtmlAttribute[] segatts = node.Attributes.ToArray();
string attname = segatts[0].Value.ToString();
string att = node.InnerText.ToString();
CreateXML(attname, att);
}
private HtmlDocument ParseDoc(HtmlDocument document)
{
try
{
HtmlNode root = document.DocumentNode.FirstChild;
foreach (HtmlNode childnode1 in root.SelectNodes(".//child1"))
{
foreach (HtmlNode childnode2 in node.SelectNodes(".//child2"))
{
ExtractSegment(childnode2);
}
}
}
catch (Exception e) { }
WriteXml();
return document;
}
When I comment out the "List.Add()" in "CreateXML()" and step through the code, the foreach loop is not broken out of after the first iteration, and the code works properly.
I have no idea what I'm doing wrong (And yes, the code is instantiated by a public member, don't worry: I am only posting the relevant internal methods to my problem)... if anyone has come across this sort of behavior before, I would really appreciate a push in the right direction to attempt to correct it... Sepcifically: is the problem just poor coding, or is this behavior a result of a property of one of the methods/libraries I am using?
One Caveat: I know that I am using HTMLAgilityPack to parse a file and extract information, but a requirement on this code forces me to use XDocument to write said information... don't ask me why.

I have no idea what I'm doing wrong
This, for starters:
catch (Exception e) { }
That's stopping you from seeing what on earth's going on. I strongly suspect you've got a NullReferenceException due to _xelemlist being null, but that's a secondary problem. The main problem is that by pretending everything's fine whatever happens, with no logging whatsoever, the only way of getting anywhere is by debugging, and that's an awful experience when you don't need to go through it.
It's extremely rarely a good idea catch exceptions and swallow them without any logging at all. It's almost never a good idea to do that with Exception.
Whenever you have a problem which is difficult to diagnose, improve your diagnostic capabilities first. That way, when you next run into a problem, it'll be easier to diagnose.

Declare the List this way,
private List<XElement> _xelemlist = new List<XElement>();

In your foreach loop, you are attempting to use XElement head as a list of XElements when you add() to it. This should probably be a list of XElements?

Might I suggest switching to using XmlDocument?
Here is some sample code which I have written for work (changed to protect my work :D), and we are using it rather well.
Code:
XmlDocument doc = new XmlDocument();
XmlNode root;
if(File.Exists(path + "\\MyXmlFile.xml"))
{
doc.Load(path + "\\MyXmlFile.xml");
root = doc.SelectSingleNode("//Library");
}
else
{
XmlDeclaration dec = doc.CreateXmlDeclaration("1.0", "UTF-8", null);
doc.AppendChild(dec);
root = doc.CreateElement("Library");
doc.AppendChild(root);
}
XmlElement book = doc.CreateElement("Book");
XmlElement title = doc.CreateElement("Title");
XmlElement author = doc.CreateElement("Author");
XmlElement isbn = doc.CreateElement("ISBN");
title.InnerText = "Title of a Book";
author.InnerText = "Some Author";
isbn.InnerText = "RandomNumbers";
book.AppendChild(title);
book.AppendChild(author);
book.AppendChild(isbn);
root.AppendChild(book);
doc.Save(path + "\\MyXmlFile.xml");

Parsing XML String in C#

I have looked over other posts here on the same subject and searched Google but I am extremely new to C# NET and at a loss. I am trying to parse this XML...
<whmcsapi version="4.1.2">
<action>getstaffonline</action>
<result>success</result>
<totalresults>1</totalresults>
<staffonline>
<staff>
<adminusername>Admin</adminusername>
<logintime>2010-03-03 18:29:12</logintime>
<ipaddress>127.0.0.1</ipaddress>
<lastvisit>2010-03-03 18:30:43</lastvisit>
</staff>
</staffonline>
</whmcsapi>
using this code..
XDocument doc = XDocument.Parse(strResponse);
var StaffMembers = doc.Descendants("staff").Select(staff => new
{
Name = staff.Element("adminusername").Value,
LoginTime = staff.Element("logintime").Value,
IPAddress = staff.Element("ipaddress").Value,
LastVisit = staff.Element("lastvisit").Value,
}).ToList();
label1.Text = doc.Element("totalresults").Value;
foreach (var staff in StaffMembers)
{
listBox1.Items.Add(staff.Name);
}
I have printed out the contents of strResponse and the XML is definitely there. However, when I click this button, nothing is added to the listBox1 or the label1 so I something is wrong.

Add Root here to start navigating from the root element (whmcsapi):
string label1_Text = doc.Root.Element("totalresults").Value;