MVC best way to parse MySQL XML Table Export

MVC best way to parse MySQL XML Table Export - c#

I am looking for a solution that help me to read fast an XML DataBase Export file,
what looks like this :
<?xml version="1.0" encoding="utf-8"?>
<database name="backup_db">
<table name="users">
<column name="username">user1</column>
<column name="password">pass1</column>
<column name="email">email1</column>
<column name="regdate">2015-01-22 23:31:59</column>
<column name="lastlogin">2016-12-11 01:56:24</column>
<column name="banned">0</column>
</table>
<table name="users">
...
</table>
...
200000 table rows
</database>
I make an solution that enumerate all the element and store the column Value, but have a long execution.
var xd = new XmlDocument();
string path = Server.MapPath("~/App_Data/users.xml");
var fs = new FileStream(path, FileMode.Open);
xd.Load(fs);
var list = xd.GetElementsByTagName("table");
for (var i = 0; i < list.Count; i++)
{
...
}
fs.Close()
Whot to make that work faster ?

Try following. It only adds data as string and will only work if the columns are always in the same order (no skipped columns)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication1
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XmlReader reader = XmlReader.Create(FILENAME);
if (reader.Name != "database")
{
reader.ReadToFollowing("database");
}
reader.MoveToFirstAttribute();
string databaseName = reader.Value;
DataSet ds = new DataSet(databaseName);
DataTable dt = null;
while (!reader.EOF)
{
if (reader.Name != "table")
{
reader.ReadToFollowing("table");
}
if (!reader.EOF)
{
XElement table = (XElement)XElement.ReadFrom(reader);
string tableName = (string)table.Attribute("name");
if (!ds.Tables.Contains(tableName))
{
dt = new DataTable(tableName);
ds.Tables.Add(dt);
dt.Columns.AddRange(table.Elements("column").Select(x => new DataColumn((string)x.Attribute("name"))).ToArray());
}
DataRow newRow = ds.Tables[tableName].Rows.Add();
newRow.ItemArray = table.Elements("column").Select(x => (string)x).ToArray();
}
}
}
}
}

Related

Convert XML to csv in c# with multiple collection

I am looking for a way to convert an xml stream to csv, but I only find solution for 1 collection, i.e. my xml looks like :
<?xml version="1.0" encoding="UTF-8"?>
<CompactData>
<Header>
<ID>id_ofç=_file</ID>
<Test>false</Test>
</Header>
<data:DataSet>
<data:Series FREQ="M" item="item1" unit="unit1">
<data:Obs TIME_PERIOD="2015-01" OBS_VALUE="5.47" />
<data:Obs TIME_PERIOD="2015-02" OBS_VALUE="5.01" />
<data:Obs TIME_PERIOD="2015-03" OBS_VALUE="5.39" />
</data:Series>
<data:Series FREQ="M" item="item2" unit="unit2">
<data:Obs TIME_PERIOD="2015-01" OBS_VALUE="5.47" />
<data:Obs TIME_PERIOD="2015-02" OBS_VALUE="5.01" />
<data:Obs TIME_PERIOD="2015-03" OBS_VALUE="5.39" />
</data:Series>
</data:DataSet>
</CompactData>
Here I want a csv with the format :
FREQ,item,unit,TIME_PERIOD,OBS_VALUE
what is the best way to do that?
Thanks!

Try following :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.Xml;
using System.Xml.Linq;
using System.IO;
namespace ConsoleApplication166
{
class Program
{
const string XML_FILENAME = #"c:\temp\test.xml";
const string CSV_FILENAME = #"c:\temp\test.csv";
static void Main(string[] args)
{
StreamWriter writer = new StreamWriter(CSV_FILENAME);
writer.WriteLine(string.Join(",", new string[] {"FREQ","item","unit","TIME_PERIOD","OBS_VALUE"}));
XDocument doc = XDocument.Load(XML_FILENAME);
XElement dataSet = doc.Descendants().Where(x => x.Name.LocalName == "DataSet").FirstOrDefault();
XNamespace nsData = dataSet.GetNamespaceOfPrefix("data");
foreach (XElement series in dataSet.Elements(nsData + "Series"))
{
string freq = (string)series.Attribute("FREQ");
string item = (string)series.Attribute("item");
string unit = (string)series.Attribute("unit");
foreach (XElement obs in series.Elements(nsData + "Obs"))
{
DateTime time = DateTime.ParseExact((string)obs.Attribute("TIME_PERIOD"), "yyyy-MM", System.Globalization.CultureInfo.InvariantCulture);
double value = (double)obs.Attribute("OBS_VALUE");
writer.WriteLine(string.Join(",", new string[] {freq, item, unit,time.ToString(), value.ToString()}));
}
}
writer.Flush();
writer.Close();
}
}
}

XML to DataTable (list item)

Here is my schema of xml.
<?xml version="1.0" encoding="utf-8"?>
<ArrayOfResultstring xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Resultstring>
<Key>Blablabla : </Key>
<Values>
<string>79,0441326460292</string>
<string>76,0959542079328</string>
<string>74,3061819154758</string>
<string>78,687039788779</string>
</Values>
<Type>list</Type>
</Resultstring>
<Resultstring>
<Key>Blablabla : </Key>
<Values>
<string>87,7110395931923</string>
</Values>
<Type>double</Type>
</Resultstring>
</ArrayOfResultstring>
I need read this XML file and fill the datatable.
I'm trying with DataSet.
DataSet ds = new DataSet();
ds.ReadXml(path);
DataTable dt = ds.Tables[0];
And my output datatable is like.
I need to show my items on table. Is there any way to read properly ?

Try following xml linq which creates a separate row for each string :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
using System.Data;
namespace ConsoleApplication51
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
DataTable dt = new DataTable();
dt.Columns.Add("KEY", typeof(string));
dt.Columns.Add("RESULTING_ID", typeof(string));
dt.Columns.Add("TYPE", typeof(string));
XDocument doc = XDocument.Load(FILENAME);
XElement root = doc.Root;
XNamespace ns = root.GetDefaultNamespace();
foreach (XElement resultingString in doc.Descendants(ns + "Resultstring"))
{
string key = (string)resultingString.Element("Key");
string type = (string)resultingString.Element("Type");
string ids = string.Join(";", resultingString.Descendants("string").Select(x => (string)x));
dt.Rows.Add(new object[] { key, ids, type });
}
}
}
}
Code to put each string in a separate column
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
using System.Data;
namespace ConsoleApplication51
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XDocument doc = XDocument.Load(FILENAME);
XElement root = doc.Root;
XNamespace ns = root.GetDefaultNamespace();
int maxString = doc.Descendants(ns + "Resultstring").Select(x => x.Descendants("string").Count()).Max();
DataTable dt = new DataTable();
dt.Columns.Add("KEY", typeof(string));
dt.Columns.Add("TYPE", typeof(string));
for (int i = 0; i < maxString; i++)
{
dt.Columns.Add("RESULTING_ID_" + (i + 1).ToString(), typeof(string));
}
foreach (XElement resultingString in doc.Descendants(ns + "Resultstring"))
{
string key = (string)resultingString.Element("Key");
string type = (string)resultingString.Element("Type");
List<string> row = resultingString.Descendants("string").Select(x => (string)x).ToList();
row.Insert(0, key);
row.Insert(1, type);
dt.Rows.Add(row.ToArray());
}
}
}
}

With Cinchoo ETL - an open source library, you can easily convert Xml file to DataTable with few lines of code.
For your sample xml file, you can extract the data into datatable as below
using (var p = new ChoXmlReader(** YOUR XML FILE **)
.WithField("Key")
.WithField("Value", xPath: "/Values/string")
)
{
var dt = p.SelectMany(r => ((Array)r.Value).OfType<string>().Select(r1 => new { Key = r.Key, Value = r1})).AsDataTable();
}
Output:
Hope it helps.

your Resultstring xml node is repeating. Below code is working for me
<Resultstring>
<Key>Blablabla : </Key>
<Values>
<string>79,0441326460292</string>
<string>76,0959542079328</string>
<string>74,3061819154758</string>
<string>78,687039788779</string>
</Values>
<Type>list</Type>
<Key>Blablabla : </Key>
<Values>
<string>87,7110395931923</string>
</Values>
<Type>double</Type>
</Resultstring>
Dataset code
DataSet ds = new DataSet();
ds.ReadXml(path);
DataTable dtKey = ds.Tables[0];
DataTable dtValues = ds.Tables[1];
DataTable dtstring = ds.Tables[2];
DataTable dtType = ds.Tables[3];

Read XML with C# and fill List<T>

I have an XML document that contains data about two table.
The structure is like this
<doc>
<table name="ordini">
<row>
<field name="id">0431524493258932</field>
<field name="anno">2018</field>
<field name="att">0000</field>
<field name="cen">01</field>
...
</row>
<row>
<field name="id">1041524493596749</field>
<field name="anno">2018</field>
<field name="att">0000</field>
<field name="cen">01</field>
...
</row>
...
</table>
<table name="righe">
<row>
<field name="id">0431524493258932</field>
<field name="anno">2018</field>
<field name="att">0000</field>
<field name="cen">4U</field>
....
</table>
<doc>
I am trying this way with first table in XML with a lot of confusion
var doc = XDocument.Load(filename);
var ordcli = doc.Descendants("table")
.Where(i => (string)i.Attribute("name") == "ordini")
.Descendants("row")
.Select(e => e.Elements());
foreach (var item in ordcli)
{
foreach (var i in item)
{
Console.WriteLine(i);
}
}
How can I read every row element and get field name to instantiate my Order class and OrderDetails class?
When I have my List<Order> and List<OrderDetails> filled I can populate relative tables.
I hope I've explained my requirement well.
I've never used XlmReader or Linq2Xml. It's first time.

try following code :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication1
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
new XML_DataSet(FILENAME);
}
}
public class XML_DataSet
{
static DataSet ds = new DataSet();
public XML_DataSet(string filename)
{
XDocument doc = XDocument.Load(filename);
foreach(XElement table in doc.Descendants("table"))
{
ds.Tables.Add(ReadTable(table));
}
}
private DataTable ReadTable(XElement table)
{
DataTable dt = new DataTable();
Boolean first = true;
foreach (XElement xRow in table.Elements("row"))
{
if (first)
{
foreach (XElement field in xRow.Elements("field"))
{
dt.Columns.Add((string)field.Attribute("name"), typeof(string));
}
first = false;
}
DataRow newRow = dt.Rows.Add();
foreach (XElement field in xRow.Elements("field"))
{
newRow[(string)field.Attribute("name")] = (string)field;
}
}
return dt;
}
}
}

Retrieving Xml Attribute Value in C#

I'm new to C# & I'm trying to read XML attribute from XML file in my Asp.net web project, when i retrieve attribute content i get this result :
"System.XML.Attribute;"
here is XML file:
<?xml version="1.0" encoding="UTF-8" standalone="true"?>
<!--creating XML file-->
<Document PagesInDocument="1" Filename="YahyaXML">
<Page Filename="YahyaXML" ZonesInPage="1" PageID="0">
<Zone TextLinesInZone="2" zoneLRY="558" zoneLRX="1202"
zoneULY="3" zoneULX="886" ZoneID="0">
<TextLine Content="aaaaaaaaaaa" TextLineID="0"/>
<TextLine Content="aaaaaaaaaaaaaaaaaa" TextLineID="1"/>
</Zone>
</Page>
</Document>
Here is my c# code:
XmlDocument doc = new XmlDocument();
doc.Load(#"C:\Program Files (x86)\NovoDynamics\NovoVerus\api\examples\C#Samples\WebAppTest\Uploads\test-ht.xml");
StringBuilder Content_Value = new StringBuilder();
XmlNodeList aNodes = doc.SelectNodes("/Document/Page/Zone");
XmlNodeList bNodes = doc.SelectNodes("/Document/Page/Zone/TextLine");
foreach (XmlNode aNode in aNodes)
{
XmlAttribute ULX_xml = aNode.Attributes["zoneULX"];
XmlAttribute ULY_xml = aNode.Attributes["zoneULY"];
XmlAttribute LRX_xml = aNode.Attributes["zoneLRX"];
XmlAttribute LRY_xml = aNode.Attributes["zoneLRY"];
if (ULX_xml.Value == TextBox5.Text && ULY_xml.Value == TextBox6.Text && LRX_xml.Value == TextBox7.Text && LRY_xml.Value == TextBox8.Text )
{
foreach (XmlNode bNode in bNodes)
{
//XmlAttribute ContentAttribute = bNode.Attributes["Content"];
if (bNode.Attributes["Content"].Value != null)
{
Content_Value.Append(bNode.Attributes["Content"].Value.ToString());
}
}
Content_Value.Append("\n");
}
}
TextBox9.Text = Content_Value.ToString();

Your code works. Try change first string in your xml file to:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
More: https://msdn.microsoft.com/en-us/library/ms256048(v=vs.120).aspx

Using xml Linq
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
StreamReader reader = new StreamReader(FILENAME);
reader.ReadLine(); //skip identification
XDocument doc = XDocument.Load(reader);
var results = doc.Descendants("Page").Select(x => new
{
filename = (string)x.Attribute("Filename"),
page = (int)x.Attribute("ZonesInPage"),
id = (int)x.Attribute("PageID"),
zones = x.Elements("Zone").Select(y => new {
textLines = (int)y.Attribute("TextLinesInZone"),
ulx = (int)y.Attribute("zoneULX"),
uly = (int)y.Attribute("zoneULY"),
lrx = (int)y.Attribute("zoneLRX"),
lry = (int)y.Attribute("zoneLRY"),
textLine = y.Elements("TextLine").Select(z => new {
content = (string)z.Attribute("Content"),
id = (int)z.Attribute("TextLineID")
}).ToList()
}).ToList()
}).ToList();
}
}
}

How to read XML with name/value pair in c#? [duplicate]

This question already has answers here:
How do I read and parse an XML file in C#?
(12 answers)
Closed 6 years ago.
How to read this type of XML in c#? Do you have any sample code?
<?xml version="1.0" encoding="ISO-8859-1" ?>
<row rownum="1">
<column colnum="1" name="PROJECT NUMBER">1</column>
<column colnum="2" name="ROLE">Project Manager</column>
<column colnum="3" name="FIRSTNAME">Angie</column>
<column colnum="4" name="LASTNAME">Emanual</column>
</row><row rownum="2">
<column colnum="1" name="PROJECT NUMBER">2</column>
<column colnum="2" name="ROLE">Developer</column>
<column colnum="3" name="FIRSTNAME">Ruha</column>
<column colnum="4" name="LASTNAME">Rao</column>
</row>

The following code will read xml fragments
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
namespace ConsoleApplication29
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XmlReaderSettings settings = new XmlReaderSettings();
settings.ConformanceLevel = ConformanceLevel.Fragment;
XmlReader reader = XmlTextReader.Create(FILENAME,settings);
List<object> results = new List<object>();
while (!reader.EOF)
{
if (reader.Name != "row")
{
reader.ReadToFollowing("row");
}
if (!reader.EOF)
{
XElement row = (XElement)XElement.ReadFrom(reader);
results.Add(new object[] {
row.Elements("column").Select(y => new {
rowNum = (int)row.Attribute("rownum"),
colNum = (int)y.Attribute("colnum"),
colName = (string)y.Attribute("name"),
value = (string)y
}).FirstOrDefault()
});
}
}
}
}
}

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

MVC best way to parse MySQL XML Table Export - c#

Related

Convert XML to csv in c# with multiple collection

XML to DataTable (list item)

Read XML with C# and fill List<T>

Retrieving Xml Attribute Value in C#

How to read XML with name/value pair in c#? [duplicate]

Categories

Resources