Develop Reference

C# Read and Store Data from XML

I am trying to read and store data from an xml file. I have been reading about various methods to read the data such as XmlReader, XmlTextReader, LinQ, etc.
My XML file is
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<circuit name="local">
<Device>device1</Device>
<Point>point1></Point>
</circuit>
<circuit name ="remote">
<Device>device2</Device>
<Point>point2</Point>
</circuit>
</configuration>
I am trying to extract Device and Point set so I can pass those along to be used in a database query. I used this code and the foreach loop to verify the contents, but it only gets the first set.
XDocument msrDoc = XDocument.Load("BNOC MSR.config");
var data = from item in msrDoc.Descendants("circuit")
select new
{
device = item.Element("Device").Value,
point = item.Element("Point").Value
};
foreach (var p in data)
Console.WriteLine(p.ToString());
I have also tried this, but my arrays were all null
String[] deviceList = new String[1];
String[] pointList = new String[1];
int n = 0;
XmlDocument msrDoc = new XmlDocument();
msrDoc.Load("BNOC MSR.config");
var itemNodes = msrDoc.SelectNodes("circuit");
foreach (XmlNode node in itemNodes)
{
var circuit = node.SelectNodes("circuit");
foreach (XmlNode cir in circuit)
{
deviceList[n] = cir.SelectSingleNode("Device").InnerText;
pointList[n] = cir.SelectSingleNode("Point").InnerText;
}
}
Any help would be greatly appreciated.

Are you sure you don't want to use the built-in Properties.Settings for this?
Circuit local = Properties.Settings.Default.localCircuit;
Circuit remote = Properties.Settings.Default.remoteCircuit;
https://learn.microsoft.com/en-us/dotnet/framework/winforms/advanced/using-application-settings-and-user-settings

I believe there is something wrong with the way you are testing the result. The code:
void Main()
{
var fileLocation = #"C:\BrianTemp\input.txt";
var xml = File.ReadAllText(fileLocation);
XDocument msrDoc = XDocument.Load(fileLocation);
var data = from item in msrDoc.Descendants("circuit")
select new
{
device = item.Element("Device").Value,
point = item.Element("Point").Value
};
foreach (var p in data)
{
//It is best practice to use statement blocks {} to prevent silly errors.
//Sometimes you want to execute multiple statements, especially as code changes later
Console.WriteLine($"{p}");
}
}
Produces the expected output:
{ device = device1, point = point1 }
{ device = device2, point = point2 }
You said:
I used this code and the foreach loop to verify the contents, but it
only gets the first set.
As you can see the code produces 2 results as it should.
Note: I corrected the XML file to remove the extra >
<Point>point1></Point><==

I see two problems in your code (and I only tried the second method you posted):
Your string arrays are too small, change to:
String[] deviceList = new String[1];
String[] pointList = new String[1];
The line var itemNodes = msrDoc.SelectNodes("circuit"); should be
var itemNodes = msrDoc.SelectNodes("configuration");

C# Query On MongoDB Not Returning Correct Results

Im currently running into an issue when querying MongoDb using c#. The problem is that I am not returned the correct results or the correct number of results. I do not know the exact number of results but it should be less than 100; instead, I am receiving around 350k-500k results (many of which are null). The other problem is that the program takes upwards of 10 minutes to finish processing.
You can see the problematic portion of code in the following:
public List<BsonDocument> find_All_Documents_With_pIDs()
{
List<string> databases = new List<string>();
List<BsonDocument> pDocs = new List<BsonDocument>();
databases.AddRange(mongo_Server.GetDatabaseNames());
//iterate through each db in mongo
foreach (string dbName in databases)
{
List<string> collections = new List<string>();
var database = mongo_Server.GetDatabase(dbName);
collections.AddRange(database.GetCollectionNames());
//iterate through each collection
foreach (string colName in collections)
{
var collection = database.GetCollection(colName);
//Iterate through each document
foreach (var document in collection.FindAllAs<BsonDocument>())
{
//Get all documents that have a pID in either the main document or its sub document
IMongoQuery query = Query.Exists(document.GetElement("_id").ToString().Remove(0,4) + ".pID");
IMongoQuery subQuery = Query.Exists(document.GetElement("_id").ToString() + ".SubDocument.pID");
pDocs.AddRange(collection.Find(query));
pDocs.AddRange(collection.Find(subQuery));
}
}
}
//Theres a collection used earlier in the program to backup the documents before processing. Not removing the documents from the list found in this location will result in duplicates.
return remove_Backup_Documents_From_List(pIDs);
}
Any help is appreciated!
EDIT:
The following is a screen capture of the data received. Not all the data is null like the following but a very large amount is:

Your script is first bringing all your documents from the database
collection.FindAllAs<BsonDocument>()
and then assembling a query for each one. That's probably the reason the query is so slow.
As an alternative you could do the following:
foreach (string colName in collections)
{
var collection = database.GetCollection(colName);
//Query for all documents that have pID
IMongoQuery query = Query.And([Query.Exists("pID"), // The field exists
Query.NE("pID", BsonNull.Value), //It is not "null"
Query.NE("pID", BsonString.Null)]); //It is not empty i.e. = ""
//Query for all documents that have Subdocument.pID
IMongoQuery subQuery = Query.And([Query.Exists("SubDocument.pID"), // The field exists
Query.NE("SubDocument.pID", BsonNull.Value), //It is not "null"
Query.NE("SubDocument.pID", BsonString.Null)]); //It is not empty i.e. = ""
IMongoQuery totalQuery = Query.Or([query, subQuery]);
List<BsonDocument> results = collection.Find(totalQuery);
if (results.Count > 0) {
pDocs.AddRange(results); //Only add to pDocs if query returned at least one result
}
}
That way you assemble a query that returns only the documents that have either pID or Subdocument.pID fields set.

converting xml of sharepoint list to dataset

its been more than one week and i still cant figure out what is the problem here. hope you could help me. i am successfully retrieving xml from share point server using SOAP web service, then i am converting the xml to dataset object, i am getting the dataset successfully but its "damaged" - there are few columns that has a missing values from the xml. here is the code for importing the xml using SOAP:
private void button2_Click(object sender, EventArgs e)
{
oportal.Lists list = new oportal.Lists();
list.Credentials = System.Net.CredentialCache.DefaultCredentials;
list.Url = "http://xxx/xxx/xxx/xxx/_vti_bin/Lists.asmx";
XmlDocument xmlDoc = new System.Xml.XmlDocument();
XmlNode ndQUery = xmlDoc.CreateNode(XmlNodeType.Element, "Query", "");
XmlNode ndViewFields = xmlDoc.CreateNode(XmlNodeType.Element, "ViewFields", "");
XmlNode ndQueryOptions = xmlDoc.CreateNode(XmlNodeType.Element, "QueryOptions", "");
ndQueryOptions.InnerXml =
"<IncludeMandatoryColumns>TRUE</IncludeMandatoryColumns>" +
"<DateInUtc>FALSE</DateInUtc>";
ndViewFields.InnerXml = #"<FieldRef Name='שם לקוח' />
<FieldRef Name='שם מתל'/>";
try
{
XmlNode ndListItems = list.GetListItems("{DD1CF626-62E1-4E36-BF2B-C7D08EA73674}",null, ndQUery, ndViewFields, "14000", ndQueryOptions, null);
System.Diagnostics.Debug.WriteLine(ndListItems.OuterXml);
dataGridView1.DataSource = ConverttYourXmlNodeToDataSet(ndListItems).Tables[1];
}
catch(System.Web.Services.Protocols.SoapException ex) {
MessageBox.Show(ex.Message + Environment.NewLine + ex.Detail.InnerText + Environment.NewLine + ex.StackTrace);
}
}
the xml i am getting looks ok, the column (field) names are in hebrew language but the xml shows them in HTML Entity (Hexadecimal) - maybe thats the root of the problem?
after i am getting the xml i am converting it into dataset with ConverttYourXmlNodeToDataSet() function here is the code:
public static DataSet ConverttYourXmlNodeToDataSet(XmlNode xmlnodeinput)
{
DataSet dataset = null;
if (xmlnodeinput != null)
{
XmlTextReader xtr = new XmlTextReader(xmlnodeinput.OuterXml, XmlNodeType.Element,null);
dataset = new DataSet();
dataset.ReadXml(xtr);
}
return dataset;
}
i am getting the dataset succefuly but like i mentioned its damaged because of the missing values, they exist in the xml but not in the dataset (the columns exist but not the values).
please have a look at this screen shoot:
iv`e surrounded with red color one of the columns that dont get their value from the XML. here is a screen shoot of the xml and the missing value that should be in the dataset surronded with red color:
also tryed to convert the xml to dataset like that but the results are the same:
public static DataSet read(XmlNode x) {
DataSet ds = new DataSet();
XmlReader r = new XmlNodeReader(x);
ds.ReadXml(r);
return ds;
}
hope someone can help me here. tnx.
UPDATE:
ok i have not solved it yet but i discovered few things that may lead to the solution:
i have noticed that all the columns that appears without values in the dataset are the columns that filled by the user in the website controls, and guess what? all the captions for those columns are in hebrew language , hence the columns that appears with values on the dataset are sharepint default columns, and their captions are in English, and they dont have HTML Entity (Hexadecimal)! name (look at the xml). So it makes me suspect that the problem is related to the HTML Entity (Hexadecimal) column names that related to the Hebrew captions... my assumption is that the dataset cant interpret this HTML Entity (Hexadecimal) encode. another clue is that the column name as its spelled in the dataset (for example look at the screen shoot of the datagridview above - column 4 from the left side (index 3)) is not interpreted right, the column name should be 'שם מתל' and thats all - as you can see (you don`t have to understand Hebrew for that) only half of this Hebrew string is there and concatenated to it part of the encoded HTML Entity (Hexadecimal).
i have noticed that when im sorting columns in share point website the requested url using the hexadecimal html entity of the column and not the hebrew name of the column:
http://xxx/xxx/xxx/xxx/Lists/1/view9.aspx?View={c2538b95-efae-453b-b536-aad6f98265ed}&SortField=_x05e9__x05dd__x0020__x05de__x05&SortDir=Desc
and i expected to see something like:
http://xxx/xxx/xxx/xxx/Lists/1/view9.aspx?View={c2538b95-efae-453b-b536-aad6f98265ed}&SortField=_'שם מתל'=Desc
so i made a change in my code in order to explicitly declare the column names in the encoded HTML Entity (Hexadecimal) and i did this (the original code is above):
ndViewFields.InnerXml = #"<FieldRef Name='_x05d0__x05d9__x05e9__x05d5__x05' />
<FieldRef Name='_x05e9__x05dd__x0020__x05de__x05'/>";
now the result i was getting in the dataset has changed! the change was that the columns i explicitly declared them moved to the first column indexes of the dataset but still there aren`t any values in those columns.
so, to summery all of that digging, here are my assumptions:
*. the problem is the interpreter between the xml and the dataset
*. the interpreter is defective because he cant interpret ecoded HTML Entity (Hexadecimal) properly
*. column captions written in HTML Entity (Hexadecimal) because their captions are in hebrew
*. solution can be or making columns captions to plain hebrew (in xml) or doing something that will make the interpreter between xml and dataset work properly (maybe using XmlParserContext class?? - tryed a little with no success or other class that can manipulate encoded xml text).

finally, after i have accomplish to solve this. the solution i found was super simple.
i have been searching and struggling some time for a solution, never found one and then this simple solution crossed my mind.
only one line of code was needed:
String s = xmlnodeinput.OuterXml.Replace("ows__x05e9__x05dd__x0020__x05de__x05",
"AccountManager");
just replacing the hex value and the dataset loads properly.
also i checked to see that there are no time proccessing issues(replacing string takes less then a second):
start reading 12000 rows: 26/03/2016 17:18:00
start replacing strings: 26/03/2016 17:18:04
load xml string to dataset: 26/03/2016 17:18:04
finish loading dataset: 26/03/2016 17:18:04
the complte converstion from xml to dataset function:
public static DataSet ConverttYourXmlNodeToDataSet(XmlNode xmlnodeinput)
{
//declaring data set object
DataSet dataset = null;
if (xmlnodeinput != null)
{
NameTable nt = new NameTable();
nt.Add("row");
XmlNamespaceManager nsmgr = new XmlNamespaceManager(nt);
XmlParserContext context = new XmlParserContext(nt, null, "heb",null, null, null, null, null, XmlSpace.None,Encoding.Unicode);
String s = xmlnodeinput.OuterXml.Replace("ows__x05e9__x05dd__x0020__x05de__x05", "AccountManager");
XmlTextReader xtr = new XmlTextReader(s, XmlNodeType.Element,context);
dataset = new DataSet();
dataset.ReadXml(xtr);
}
return dataset;
}

Encountered the same problem (Missing values after Xml load via/to DataSet).
Seems there is a problem with some chars (in my case "-").
Solution from jonathana works (replacing chars from attribute names before loading data to the dataset).
I will additionally provide a solution for .NET2 that changes all attribute names in the resulting SharePoint SOAP query XML to make sure the conversation into dataset won't result in an error (can be done more nicly with .NET3+ but im forced to .NET2 in my situation):
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;
XmlDocument doc = new XmlDocument();
doc.LoadXml(spResXml.OuterXml);
System.Xml.XmlNamespaceManager nm = new System.Xml.XmlNamespaceManager(doc.NameTable);
nm.AddNamespace("rs", "urn:schemas-microsoft-com:rowset");
nm.AddNamespace("z", "#RowsetSchema");
nm.AddNamespace("rootNS", "http://schemas.microsoft.com/sharepoint/soap");
var zRows = doc.SelectNodes("//z:row", nm);
for (int i = 0; i < zRows.Count; i++)
{
XmlNode zRow = zRows[i];
List<XmlAttribute> attsList = new List<XmlAttribute>();
for (int j = 0; j < zRow.Attributes.Count; j++)
{ attsList.Add(zRow.Attributes[j]); }
foreach (XmlAttribute att in attsList)
{
string patchedAttName = att.Name;
patchedAttName = patchedAttName.Replace("_x", "%u");
patchedAttName = HttpUtility.UrlDecode(patchedAttName);
patchedAttName = Regex.Replace(patchedAttName,"[^A-Za-z0-9_]", "_", RegexOptions.None);
if (att.Name.Equals(patchedAttName))
{ continue; }
var newAtt = doc.CreateAttribute(att.Prefix, patchedAttName, att.NamespaceURI);
newAtt.Value = att.Value;
zRow.Attributes.Remove(att);
zRow.Attributes.Append(newAtt);
}
}
DataSet ds = new DataSet();
ds.ReadXml(new XmlNodeReader(doc));
DataTable t = ds.Tables[1];

Set Fields dynamically in linq to xml Expression using C# in Razor application

I am trying to export the XML file from the database table using XElement. I am using EF 6.0 Code first.
I am preparing XML using XElement by below code
TooltypeXml =
new XElement("ToolTypes",
(from tbl in db.ToolType
where tbl.CreationDate >= objLastSyncByDevice.LocaltoServerLastAccessDate
|| tbl.LastModifieDate >= objLastSyncByDevice.LocaltoServerLastAccessDate
|| tbl.IsDeleted == true
select new
{
tbl.ToolTypeId,
tbl.ToolTypeName,
tbl.Action,
tbl.UpdatedBy,
tbl.CreationDate,
tbl.CreatedBy,
tbl.LastModifieDate,
tbl.IsDeleted
}).ToList()
.Select(x =>
new XElement("ToolType",
new XElement("ToolTypeName", x.ToolTypeName),
new XElement("Action", x.Action),
new XElement("UpdatedBy", x.UpdatedBy),
new XElement("CreationDate", x.CreationDate),
new XElement("CreatedBy", x.CreatedBy),
new XElement("LastModifieDate", x.LastModifieDate),
new XElement("IsDeleted", x.IsDeleted))));
So It does create the XML format successfully , what I want is how I can write my Linq expression so I don't need to specify each and every fields in expression to select. Because I always want all the fields from table and if I do change something in table I don't need to change anything in code. Please help. Thanks in advance.

I got the solution of this problem,
I have added following code to get all properties of a class and create each fields as XElement.
private void CreateXElemetsOfClass(System.Type typeOfClass, string TableName, dynamic objData, string p_mainElementName, string MachineID)
{
try
{
if (objData != null && objData.Count > 0)
{
System.Reflection.PropertyInfo[] properties = typeOfClass.GetProperties();
List<XElement> lstXElements = new List<XElement>();
List<XElement> lstmainElements = new List<XElement>();
XElement rootElement = new XElement(TableName);
foreach (var item in objData)
{
lstXElements = new List<XElement>();
XElement mainElement = new XElement(p_mainElementName);
foreach (System.Reflection.PropertyInfo property in properties)
{
var notMapped = property.GetCustomAttributes(typeof(NotMappedAttribute), false);
if (notMapped.Length == 0)
{
lstXElements.Add(new XElement(property.Name, property.GetValue(item, null)));
}
}
mainElement.Add(lstXElements);
lstmainElements.Add(mainElement);
}
rootElement.Add(lstmainElements);
string XMLFilePath = serializetoxmlNew(rootElement, MachineID, TableName);
}
}
catch (Exception ex)
{
throw;
}
}
Here, type is System.Type classType = typeof(FileTypes);,
objData is a list i want to retrieve in XML,
p_mainElementName = "FileType".
It will generate an XML as below,
<?xml version="1.0" encoding="utf-8"?>
<FileTypes>
<FileType>
<FileTypeId>1b254bc3-7516-4f9c-89ea-d9b20ecbf005</FileTypeId>
<Description>Excel</Description>
<CreatedBy>22a8be24-9272-4d7e-9248-e9064f917884</CreatedBy>
<UpdatedBy>22a8be24-9272-4d7e-9248-e9064f917884</UpdatedBy>
<CreateDate>2014-08-22T16:05:53.177</CreateDate>
<UpdateDate>2014-08-22T16:05:53.177</UpdateDate>
<FileExtension>.xls</FileExtension>
<FileTypeImage>excel.png</FileTypeImage>
<Action>Create</Action>
<IsDeleted>false</IsDeleted>
</FileType>
<FileType>
<FileTypeId>f3362487-d96e-4cc8-bc4b-5120866f95ce</FileTypeId>
<Description>Adobe Acrobat Reader</Description>
<CreatedBy>22a8be24-9272-4d7e-9248-e9064f917884</CreatedBy>
<UpdatedBy>22a8be24-9272-4d7e-9248-e9064f917884</UpdatedBy>
<CreateDate>2014-08-22T16:05:12.407</CreateDate>
<UpdateDate>2014-08-22T16:05:12.407</UpdateDate>
<FileExtension>.pdf</FileExtension>
<FileTypeImage>pdf.png</FileTypeImage>
<Action>Create</Action>
<IsDeleted>false</IsDeleted>
</FileType>
</FileTypes>
Hope this helps you! It is perfectly working.

It looks like you would better use serialization of your objects to XML and do not create the custom serialization function.

C# LINQ - reading an XML

i need to store all the informationen from the xml in an array. My code doesn't work, because I always get just the first item from the xml.
Does anyone know how to fix this?
XDocument xdoc = XDocument.Load("http://www.thefaxx.de/xml/nano.xml");
var items = from item in xdoc.Descendants("items")
select new
{
Title = item.Element("item").Element("title").Value,
Description = item.Element("item").Element("description").Value
};
foreach (var item in items)
{
listView1.Items.Add(item.Title);
}

How about:
var items = from item in xdoc.Descendants("item")
select new
{
Title = item.Element("title").Value,
// *** NOTE: xml has "desc", not "description"
Description = item.Element("desc").Value
};
It is a little hard to be sure without sample xml - but it looks like you intend to loop over all the <item>...</item> elements - which is what the above does. Your original code loops over the (single?) <items>...</items> element(s), then fetches the first <item>...</item> from within it.
edit after looking at the xml; this would be more efficient:
var items = from item in xdoc.Root.Elements("item")
select new {
Title = item.Element("title").Value,
Description = item.Element("desc").Value
};