I am using the following code snippet to parse and convert some XML data to CSV. I can convert the entire XML data and dump it into a file, however my requirements have changed and now I'm confused.
public void xmlToCSVfiltered(string p, int e)
{
string all_lines1 = File.ReadAllText(p);
all_lines1 = "<Root>" + all_lines1 + "</Root>";
XmlDocument doc_all = new XmlDocument();
doc_all.LoadXml(all_lines1);
StreamWriter write_all = new StreamWriter(FILENAME2);
XmlNodeList rows_all = doc_all.GetElementsByTagName("XML");
List<string[]> filtered = new List<string[]>();
foreach (XmlNode rowtemp in rows_all)
{
List<string> children_all = new List<string>();
foreach (XmlNode childtemp in rowtemp.ChildNodes)
{
children_all.Add(Regex.Replace(childtemp.InnerText, "\\s+", " ")); // <------- Fixed the Bug , Advisories dont span
}
string.Join(",", children_all.ToArray());
//write_all.WriteLine(string.Join(",", children_all.ToArray()));
if (children_all.Contains(e.toString()))
{
filtered.Add(children_all.ToArray());
write_all.WriteLine(children_all);
}
}
write_all.Flush();
write_all.Close();
foreach (var res in filtered)
{
Console.WriteLine(string.Join(",", res));
}
}
My input looks something like the following... My objective now is to only convert those "events" and compile into a CSV which have a certain number. Lets say, for example, I only want to convert to CSV those events who's 2nd data value under element <EVENT> is 4627. It would only convert those events and in the case of the input below, both mentioned below.
<XML><HEADER>1.0,770162,20121009133435,3,</HEADER>20121009133435,721,5,1,0,0,0,00:00,00:00,<EVENT>00032134826064957,4627,</EVENT><DRUG>1,1872161156,7,0,10000</DRUG><DOSE>1,0,5000000,0,10000000,0</DOSE><CAREAREA>1 </CAREAREA><ENCOUNTER></ENCOUNTER><ADVISORY>Keep it simple or spell
tham ALL out. For some reason
that is not the case
please press the on button
when trying to activate
device codes also available on
list</ADVISORY><CAREGIVER></CAREGIVER><PATIENT></PATIENT><LOCATION>20121009133435,00-1d-71-0a-71-80,-66</LOCATION><ROUTE></ROUTE><SITE></SITE><POWER>0,50</POWER></XML>
<XML><HEADER>2.0,773162,20121009133435,3,</HEADER>20121004133435,761,5,1,0,0,0,00:00,00:00,<EVENT>00032134826064957,4627,</EVENT><DRUG>1,18735166156,7,0,10000</DRUG><DOSE>1,0,5000000,0,10000000,0</DOSE><CAREAREA>1 </CAREAREA><ENCOUNTER></ENCOUNTER><ADVISORY>Keep it simple or spell
tham ALL out. For some reason
that is not the case
please press the on button
when trying to activate
device codes also available on
list</ADVISORY><CAREGIVER></CAREGIVER><PATIENT></PATIENT><LOCATION>20121009133435,00-1d-71-0a-71-80,-66</LOCATION><ROUTE></ROUTE><SITE></SITE><POWER>0,50</POWER></XML>
.. goes on
What my approach has been so far is to convert everything to CSV and store it in some sort of data structure and then query that data structure line by line and look if that number exists and if yes, write it to a file line by line. My function takes the path of the XML file and the number we are looking for in the XML data as parameters. I'm new to C# and I cannot understand how I would go about changing my function above. Any help will be appreciated!
EDIT:
Sample Input:
<XML><HEADER>1.0,770162,20121009133435,3,</HEADER>20121009133435,721,5,1,0,0,0,00:00,00:00,<EVENT>00032134826064957,4627,</EVENT><DRUG>1,1872161156,7,0,10000</DRUG><DOSE>1,0,5000000,0,10000000,0</DOSE><CAREAREA>1 </CAREAREA><ENCOUNTER></ENCOUNTER><ADVISORY>Keep it simple or spell
tham ALL out. For some reason
that is not the case
please press the on button
when trying to activate
device codes also available on
list</ADVISORY><CAREGIVER></CAREGIVER><PATIENT></PATIENT><LOCATION>20121009133435,00-1d-71-0a-
<XML><HEADER>1.0,770162,20121009133435,3,</HEADER>20121009133435,721,5,1,0,0,0,00:00,00:00,<EVENT>00032134826064957,4623,</EVENT><DRUG>1,1872161156,7,0,10000</DRUG><DOSE>1,0,5000000,0,10000000,0</DOSE><CAREAREA>1 </CAREAREA><ENCOUNTER></ENCOUNTER><ADVISORY>Keep it simple or spell
tham ALL out. For some reason
that is not the case
please press the on button
when trying to activate
device codes also available on
list</ADVISORY><CAREGIVER></CAREGIVER><PATIENT></PATIENT><LOCATION>20121009133435,00-1d-71-0a-
Required Output:
1.0,770162,20121009133435,3,,20121009133435,721,5,1,0,0,0,00:00,00:00,,00032134 26064957,4627,1,,1872161156,7,0,10000,1,0,5000000,0,10000000,0,1 ,,Keep it simple or spell
tham ALL out. For some reason
that is not the case
please press the on button
when trying to activate
device codes also available on
list,,,20121009133435,00-1d-71-0a-71-80,-66,,,0,50
The above will be the case if I call xmlToCSVfiltered(file, 4627);
Also note that, the output will be a single horizontal line as in CSV files but I can't really format it here for it to look like that.
I changed XmlDocumnet to XDocument so I can use Xml Linq. I also for testing used a StringReader to read the string instead of reading from a file. You can convert code back to your original File.ReadAlltext.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;
using System.IO;
using System.Text.RegularExpressions;
namespace ConsoleApplication1
{
class Program
{
const string FILENAME2 = #"c:\temp\test.txt";
static void Main(string[] args)
{
string input =
"<XML><HEADER>1.0,770162,20121009133435,3,</HEADER>20121009133435,721,5,1,0,0,0,00:00,00:00,<EVENT>00032134826064957,4627,</EVENT><DRUG>1,1872161156,7,0,10000</DRUG><DOSE>1,0,5000000,0,10000000,0</DOSE><CAREAREA>1 </CAREAREA><ENCOUNTER></ENCOUNTER><ADVISORY>Keep it simple or spell\n" +
"tham ALL out. For some reason \n" +
"that is not the case\n" +
"please press the on button\n" +
"when trying to activate\n" +
"device codes also available on\n" +
"list</ADVISORY><CAREGIVER></CAREGIVER><PATIENT></PATIENT><LOCATION>20121009133435,00-1d-71-0a-71-80,-66</LOCATION><ROUTE></ROUTE><SITE></SITE><POWER>0,50</POWER></XML>\n" +
"<XML><HEADER>2.0,773162,20121009133435,3,</HEADER>20121004133435,761,5,1,0,0,0,00:00,00:00,<EVENT>00032134826064957,4627,</EVENT><DRUG>1,18735166156,7,0,10000</DRUG><DOSE>1,0,5000000,0,10000000,0</DOSE><CAREAREA>1 </CAREAREA><ENCOUNTER></ENCOUNTER><ADVISORY>Keep it simple or spell\n" +
"tham ALL out. For some reason\n" +
"that is not the case\n" +
"please press the on button\n" +
"when trying to activate\n" +
"device codes also available on\n" +
"list</ADVISORY><CAREGIVER></CAREGIVER><PATIENT></PATIENT><LOCATION>20121009133435,00-1d-71-0a-71-80,-66</LOCATION><ROUTE></ROUTE><SITE></SITE><POWER>0,50</POWER></XML>\n";
xmlToCSVfiltered(input, 4627);
}
static public void xmlToCSVfiltered(string p, int e)
{
//string all_lines1 = File.ReadAllText(p);
StringReader reader = new StringReader(p);
string all_lines1 = reader.ReadToEnd();
all_lines1 = "<Root>" + all_lines1 + "</Root>";
XDocument doc_all = XDocument.Parse(all_lines1);
StreamWriter write_all = new StreamWriter(FILENAME2);
List<XElement> rows_all = doc_all.Descendants("XML").Where(x => x.Element("EVENT").Value.Split(new char[] {','}).Skip(1).Take(1).FirstOrDefault() == e.ToString()).ToList();
List<string[]> filtered = new List<string[]>();
foreach (XElement rowtemp in rows_all)
{
List<string> children_all = new List<string>();
foreach (XElement childtemp in rowtemp.Elements())
{
children_all.Add(Regex.Replace(childtemp.Value, "\\s+", " ")); // <------- Fixed the Bug , Advisories dont span
}
string.Join(",", children_all.ToArray());
//write_all.WriteLine(string.Join(",", children_all.ToArray()));
if (children_all.Contains(e.ToString()))
{
filtered.Add(children_all.ToArray());
write_all.WriteLine(children_all);
}
}
write_all.Flush();
write_all.Close();
foreach (var res in filtered)
{
Console.WriteLine(string.Join(",", res));
}
}
}
}
I have made some assumptions since it was not clear to me from the question
Assumptions
1. I am assuming you know that you need to check node event and you need to second position element from there.
2. You know the delimiter between the values in node. for eg. ',' here in events
public void xmlToCSVfiltered(string p, int e, string nodeName, char delimiter)
{
//get the xml node
XDocument xml = XDocument.Load(p);
//get the required node. I am assuming you would know. For eg. Event Node
var requiredNode = xml.Descendants(nodeName);
foreach (var node in requiredNode)
{
if (node == null)
continue;
//Also here, I am assuming you have the delimiter knowledge.
var valueSplit = node.Value.Split(delimiter);
foreach (var value in valueSplit)
{
if (value == e.ToString())
{
AddToCSV();
}
}
}
}
I am using the CSVHelper library, which can extract a list of objects from a CSV file with just three lines of code:
var streamReader = // Create a reader to your CSV file.
var csvReader = new CsvReader( streamReader );
List<MyCustomType> myData = csvReader.GetRecords<MyCustomType>();
However, by file has nonsense lines and I need to skip the first ten lines in the file. I thought it would be nice to use LINQ to ensure 'clean' data, and then pass that data to CsvFReader, like so:
public TextReader GetTextReader(IEnumerable<string> lines)
{
// Some magic here. Don't want to return null;
return TextReader.Null;
}
public IEnumerable<T> ExtractObjectList<T>(string filePath) where T : class
{
var csvLines = File.ReadLines(filePath)
.Skip(10)
.Where(l => !l.StartsWith(",,,"));
var textReader = GetTextReader(csvLines);
var csvReader = new CsvReader(textReader);
csvReader.Configuration.ClassMapping<EventMap, Event>();
return csvReader.GetRecords<T>();
}
But I'm really stuck into pushing a 'static' collection of strings through a stream like a TextReaer.
My alternative here is to process the CSV file line by line through CsvReader and examine each line before extracting an object, but I find that somewhat clumsy.
The StringReader Class provides a TextReader that wraps a String. You could simply join the lines and wrap them in a StringReader:
public TextReader GetTextReader(IEnumerable<string> lines)
{
return new StringReader(string.Join("\r\n", lines));
}
An easier way would be to use CsvHelper to skip the lines.
// Skip rows.
csvReader.Configuration.IgnoreBlankLines = false;
csvReader.Configuration.IgnoreQuotes = true;
for (var i = 0; i < 10; i++)
{
csvReader.Read();
}
csvReader.Configuration.IgnoreBlankLines = false;
csvReader.Configuration.IgnoreQuotes = false;
// Carry on as normal.
var myData = csvReader.GetRecords<MyCustomType>;
IgnoreBlankLines is turned off in case any of those first 10 rows are blank. IgnoreQuotes is turned off so you don't get any BadDataExceptions if those rows contain a ". You can turn them back on after for normal functionality again.
If you don't know the amount of rows and need to test based on row data, you can just test csvReader.Context.Record and see if you need to stop. In this case, you would probably need to manually call csvReader.ReadHeader() before calling csvReader.GetRecords<MyCustomType>().