Parsing XML file using SSIS / C#
operations like fetching record count from trailer , TIN from body and store into a variable or somewhere temporarily(your suggestions please) for further processing. I don't want to store it in a table.
Please find the sample xml mentioned below
<ACOParticipantData xmlns:xsi="">
<Header>
<HeaderCode>HDR_PFPRVDR</HeaderCode>
<FileCreationDate>20160101</FileCreationDate>
<ACOProgCode>21</ACOProgCode>
</Header>
<Participants>
<Participant>
<ACO_ID>V199</ACO_ID>
<TIN>123456789</TIN>
<Old_TIN>987654321</Old_TIN>
<Org_NPI>1234567890</Org_NPI>
<Ind_NPI>1234567890</Ind_NPI>
<CCN>123456</CCN>
<PRG_Eff_Dt>20160101</PRG_Eff_Dt>
<PRG_Term_Dt>20161231</PRG_Term_Dt>
</Participant>
</Participants>
<Trailer>
<TrailerCode>TRL_PFPRVDR</TrailerCode>
<FileCreationDate>20160101</FileCreationDate>
<RecordCount>1</RecordCount>
</Trailer>
</ACOParticipantData>
You need to get fist get list of Participants then fetch all Participants tin number into list like
Here i created console app for your demonstration purpose.
class Program
{
static void Main(string[] args)
{
XDocument doc = XDocument.Load(#"Path to your xml file");
List<long> tinList = new List<long>();
tinList = doc.Descendants("Participants").Elements().Elements("TIN").Select(x => (long)x).ToList();
foreach (long tin in tinList)
{
Console.WriteLine(tin);
}
Console.ReadLine();
}
}
Output: (For 2 Participants)
You need to create a class for each node and use XML deserialisation to create the object.
I had to remove the empty namespace as the deserialisation process requires a valid namespace.
Also you can change the type of the properties according to your needs.
using System;
using System.IO;
using System.Xml.Serialization;
using System.Linq;
public class Program
{
public class ACOParticipantData
{
public Header Header { get; set; }
public Participant[] Participants { get; set; }
}
public class Header
{
public string HeaderCode { get; set; }
public string FileCreationDate { get; set; }
public string ACOProgCode { get; set; }
}
public class Participant
{
public string ACO_ID { get; set; }
public string TIN { get; set; }
public string Old_TIN { get; set; }
public string Org_NPI { get; set; }
public string Ind_NPI { get; set; }
public string CCN { get; set; }
public string PRG_Eff_Dt { get; set; }
public string PRG_Term_Dt { get; set; }
}
public class Trailer
{
public string TrailerCode { get; set; }
public string FileCreationDate { get; set; }
public string RecordCount { get; set; }
}
public static void Main()
{
var xmlString = #"<ACOParticipantData>
<Header>
<HeaderCode>HDR_PFPRVDR</HeaderCode>
<FileCreationDate>20160101</FileCreationDate>
<ACOProgCode>21</ACOProgCode>
</Header>
<Participants>
<Participant>
<ACO_ID>V199</ACO_ID>
<TIN>123456789</TIN>
<Old_TIN>987654321</Old_TIN>
<Org_NPI>1234567890</Org_NPI>
<Ind_NPI>1234567890</Ind_NPI>
<CCN>123456</CCN>
<PRG_Eff_Dt>20160101</PRG_Eff_Dt>
<PRG_Term_Dt>20161231</PRG_Term_Dt>
</Participant>
<Participant>
<ACO_ID>V199</ACO_ID>
<TIN>123456780</TIN>
<Old_TIN>987654321</Old_TIN>
<Org_NPI>1234567890</Org_NPI>
<Ind_NPI>1234567890</Ind_NPI>
<CCN>123456</CCN>
<PRG_Eff_Dt>20160101</PRG_Eff_Dt>
<PRG_Term_Dt>20161231</PRG_Term_Dt>
</Participant>
</Participants>
<Trailer>
<TrailerCode>TRL_PFPRVDR</TrailerCode>
<FileCreationDate>20160101</FileCreationDate>
<RecordCount>1</RecordCount>
</Trailer>
</ACOParticipantData>";
var serializer = new XmlSerializer(typeof(ACOParticipantData));
ACOParticipantData obj = null;
using (var reader = new StringReader(xmlString))
{
obj = (ACOParticipantData)serializer.Deserialize(reader);
}
if (obj == null)
{
return;
}
foreach (var tin in obj.Participants.Select(x => x.TIN))
{
Console.WriteLine(tin);
}
}
}
Output:
123456789
123456780
Related
I have an XML like this:
<XmlSports CreateDate="2022-12-04T17:47:53.5569879Z">
<Sport Name="eSports" ID="2357">
<Event Name="NBA2K, NBA Blitz League" ID="66838" IsLive="false" CategoryID="9357">
<Match Name="LA Clippers (THA KID) Esports - Milwaukee Bucks (SHARPSHOOTER)" ID="2711992" StartDate="2022-12-04T17:36:00" MatchType="Live">
<Bet Name="Money Line" ID="43859970" IsLive="true">
<Odd Name="1" ID="297613016" Value="2.26"/>
<Odd Name="2" ID="297613021" Value="1.58"/>
</Bet>
<Bet Name="Spread" ID="43859969" IsLive="true">
<Odd Name="1" ID="297614398" Value="1.83" SpecialBetValue="2.5"/>
<Odd Name="2" ID="297614399" Value="1.90" SpecialBetValue="2.5"/>
</Bet>
<Bet Name="Total" ID="43859971" IsLive="true">
<Odd Name="Over" ID="297613741" Value="1.86" SpecialBetValue="140.5"/>
<Odd Name="Under" ID="297613740" Value="1.86" SpecialBetValue="140.5"/>
</Bet>
</Match>
</Event>
<Event Name="FIFA, GT League" ID="62647" IsLive="false" CategoryID="8212">
.
. **and so on, the content is too long to post it here**
.
</Sport>
</XmlSports>
and the goal is to deserialize it into C# classes.
My classes are as such:
using System.Xml.Serialization;
namespace BettingDataAPI.Models
{
[Serializable, XmlRoot("Sport")]
public class Sport
{
[XmlElement("ID")]
public int ID { get; set; }
[XmlElement("Name")]
public string Name { get; set; }
[XmlElement("Event")]
public List<Event> Events { get; set; }
}
}
also:
using System.Xml.Serialization;
namespace BettingDataAPI.Models
{
[XmlType("Event")]
public class Event
{
[XmlElement("ID")]
public int ID { get; set; }
[XmlElement("Name")]
public string Name { get; set; }
[XmlElement("CategoryID")]
public int CategoryID { get; set; }
[XmlElement("IsLive")]
public bool IsLive { get; set; }
[XmlElement("Match")]
public List<Match> Matches { get; set; }
}
}
I also have the classes: Odd, Match and Bet
This is my code:
private static async Task<Sport> DeserializeXMLToObject()
{
private static readonly string xmlUrl = #"*here is the url*";
XmlSerializer ser = new XmlSerializer(typeof(Sport));
//WebClient client = new WebClient();
HttpClient client = new HttpClient();
HttpResponseMessage response = client.GetAsync(xmlUrl).Result;
string xml = "";
if (response.IsSuccessStatusCode)
{
xml = response.Content.ReadAsStringAsync().Result;
}
Sport sport = new Sport();
using(TextReader reader = new StringReader(xml))
{
sport = (Sport)ser.Deserialize(reader);
}
return sport;
}
but on this line
sport = (Sport)ser.Deserialize(reader);
it gives me the following error:
InnerException = {"<XmlSports xmlns=''> was not expected."}
I think it is because I've had created a class XmlSports but I removed it because when I run code first migration with Entity Framework, it said that there was an error because it couldn't create a primary key for the table XmlSports, so I removed this class and made the Sport class the root class instead.
I tried to create an xsd file from this xml (I created a test.xml file and copied and pasted the content from the url) but it didn't happen because it said that: There was an error processing 'test.xml'.
I don't have experience with xml content and serialization and deserialization in general so it's pretty hard for me to understand what's wrong and how to fix it.
Any suggestions would be highly appreciated!
Thank you!
This should get you started. If you need more help ask
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Serialization;
namespace ConsoleApplication2
{
class Program
{
const string FILENAME = #"c:\temp\test.xml";
static void Main(string[] args)
{
XmlReader reader = XmlReader.Create(FILENAME);
XmlSerializer serializer = new XmlSerializer(typeof(XmlSports));
XmlSports sports = (XmlSports)serializer.Deserialize(reader);
}
}
public class XmlSports
{
public Sport Sport { get; set; }
}
public class Sport
{
[XmlAttribute("ID")]
public int ID { get; set; }
[XmlAttribute("Name")]
public string Name { get; set; }
[XmlElement("Event")]
public List<Event> Events { get; set; }
}
public class Event
{
[XmlAttribute("ID")]
public int ID { get; set; }
[XmlAttribute("Name")]
public string Name { get; set; }
[XmlAttribute("CategoryID")]
public int CategoryID { get; set; }
[XmlAttribute("IsLive")]
public bool IsLive { get; set; }
[XmlElement("Match")]
public List<Match> Matches { get; set; }
}
public class Match
{ }
}
I have the following XML;
<?xml version="1.0" encoding="UTF-8" ?>
<feedback>
<report_metadata>
<org_name>example.com</org_name>
</report_metadata>
</feedback>
and the following Feedback.cs class;
[XmlRoot("feedback", Namespace = "", IsNullable = false)]
public class Feedback
{
[XmlElement("report_metadata")]
public MetaData MetaData { get; set; }
}
[XmlType("report_metadata")]
public class MetaData
{
[XmlAttribute("org_name")]
public string Organisation { get; set; }
}
When I attempt to deserialize, the value for Organisation is null.
var xml = System.IO.File.ReadAllText("example.xml");
var serializer = new XmlSerializer(typeof(Feedback));
using (var reader = new StringReader(input))
{
var feedback = (Feedback)serializer.Deserialize(reader);
}
Yet, when I change Feedback.cs to the following, it works (obviously the property name has changed).
[XmlType("report_metadata")]
public class MetaData
{
//[XmlAttribute("org_name")]
public string org_name { get; set; }
}
I want the property to be Organisation, not org_name.
In the example XML file org_name is an XML element, not an XML attribute. Changing
[XmlAttribute("org_name")] to [XmlElement("org_name")] at the Organisation property will deserialize it as an element:
[XmlElement("org_name")]
public string Organisation { get; set; }
probably just typo
[XmlAttribute("org_name")]
public string Organisation { get; set; }
was supposed to be
[XmlElement("org_name")]
public string Organisation { get; set; }
Try to modify your Xml classes like
[XmlRoot(ElementName = "report_metadata")]
public class MetaData
{
[XmlElement(ElementName = "org_name")]
public string Organisation { get; set; }
}
[XmlRoot(ElementName = "feedback")]
public class Feedback
{
[XmlElement(ElementName = "report_metadata")]
public MetaData MetaData { get; set; }
}
Then you will get your desired output like
class Program
{
static void Main(string[] args)
{
Feedback feedback = new Feedback();
var xml = System.IO.File.ReadAllText(#"C:\Users\Nullplex6\source\repos\ConsoleApp4\ConsoleApp4\Files\XMLFile1.xml");
var serializer = new XmlSerializer(typeof(Feedback));
using (var reader = new StringReader(xml))
{
feedback = (Feedback)serializer.Deserialize(reader);
}
Console.WriteLine($"Organization: {feedback.MetaData.Organisation}");
Console.ReadLine();
}
}
Output:
I am still learning how to use LINQ and now I'm struggling with this situation.
I have this XML file (example)
<Results>
<PrxComissao>
<Id>0</Id>
<NumErro>0</NumErro>
<RetCode>0</RetCode>
<IdEvento>0</IdEvento>
<ExecutionTimeMilliseconds>63596143450994.227</ExecutionTimeMilliseconds>
<ExecutionTimeSeconds>63596143450.994225</ExecutionTimeSeconds>
<CodComissao>CFE</CodComissao>
<Montante>20.00</Montante>
<Percentagem>0.0000</Percentagem>
<MntMin>0.00</MntMin>
<MntMax>0.00</MntMax>
<Nome>CFE</Nome>
<Descricao>Custo Factura Estrangeiro</Descricao>
</PrxComissao>
<PrxComissao>
<Id>0</Id>
<NumErro>0</NumErro>
<RetCode>0</RetCode>
<IdEvento>0</IdEvento>
<ExecutionTimeMilliseconds>63596143450994.227</ExecutionTimeMilliseconds>
<ExecutionTimeSeconds>63596143450.994225</ExecutionTimeSeconds>
<CodComissao>CFE</CodComissao>
<Montante>20.00</Montante>
<Percentagem>0.0000</Percentagem>
<MntMin>13.00</MntMin>
<MntMax>123.00</MntMax>
<Nome>CFE</Nome>
<Descricao>www</Descricao>
</PrxComissao>
</Results>
And now what I want to do is get all the XML elements inside the "PrxComissao", and then assign them to my class. This is the code I was trying
XDocument xDoc = XDocument.Parse(resultado);
List<PrxComissao> lstPrxComissao = xDoc.Elements("Results")
.Elements("PrxComissao")
.Elements()
.Select(BL_CartaoCredito.Utils.Data.Converter.FromXElement<PrxComissao>)
.ToList();
ObjAuxResult = lstPrxComissao;
What I am trying to do with this Converter.FromXElement<PrxComissao> is get all that elements and assign them.
Here is my class
public class PrxComissao
{
public string CodComissao { get; set; }
public string Montante { get; set; }
public string Percentagem { get; set; }
public string MntMin { get; set; }
public string MntMax { get; set; }
public string Nome { get; set; }
public string Descricao { get; set; }
public string TipoImposto { get; set; }
public string ComFinanciamento { get; set; }
public string iActivo { get; set; }
public string UtlModificacao { get; set; }
public string DtModificacao { get; set; }
public string UtlCriacao { get; set; }
public string DtCriacao { get; set; }
}
public static T FromXElement<T>(XElement element) where T : class, new()
{
T value = new T();
foreach (var subElement in element.Elements())
{
var field = typeof(T).GetField(subElement.Name.LocalName);
field.SetValue(value, (string)subElement);
}
return value;
}
So now I have two problems. First, I can't get to the elements inside PrxComissao always returns me nothing and then is my LINQ Select correct ? Or is there a better way ?
Start with the Descendants assuming your convertor takes an XElement:
List<PrxComissao> lstPrxComissao = xDoc.Descendants()
.Elements("PrxComissao")
.Select(el => BL_CartaoCredito.Utils.Data.Converter.FromXElement<PrxComissao>(el))
.ToList();
and then (untested) ...
public static T FromXElement<T>(XElement element) where T : class, new()
{
var typeOfT = typeof(T);
T value = new T();
foreach (var subElement in element.Elements())
{
var prop = typeOfT.GetProperty(subElement.Name.LocalName);
if(prop != null)
{
prop.SetValue(value, subElement.Value);
}
}
return value;
}
Currently, your code passes individual child elements of <PrxComissao> to the converter method. I believe you want to pass XElement that references <PrxComissao> instead :
List<PrxComissao> lstPrxComissao =
xDoc.Elements("Results")
.Elements("PrxComissao")
.Select(o => BL_CartaoCredito.Utils
.Data
.Converter
.FromXElement<PrxComissao>(o)
)
.ToList();
Besides, your class uses properties instead of field, so the corresponding reflection method supposed to be used here is GetProperty(), not GetField().
I have figured out how to populate a custom class from XML data, but I ran into an issue along the way. Things were working perfectly with my existing method of populating data until I was thrown a bit of a curve ball. The new schema I was sent is similar to this:
<ITEM_REPLY>
<TRAN_ID>1320691307345</TRAN_ID>
<REPLY_CODE>0</REPLY_CODE>
<UNIT_PRICE>8.2784</UNIT_PRICE>
<SUP_LOCS>
<SUP_LOC>
<SUP_LOC_ID>001134</SUP_LOC_ID>
<COUNTRY_ID>USA</COUNTRY_ID>
<QTY_AVL>47.000</QTY_AVL>
<ITEM_UOM>EA</ITEM_UOM>
</SUP_LOC>
<SUP_LOC>
<SUP_LOC_ID>006817</SUP_LOC_ID>
<COUNTRY_ID>USA</COUNTRY_ID>
<QTY_AVL>20.000</QTY_AVL>
<ITEM_UOM>EA</ITEM_UOM>
</SUP_LOC>
</SUP_LOCS>
<MESSAGE />
<QTY_BREAKS />
</ITEM_REPLY>
Pretty standard XML schema, problem is I'm not sure how to populate my custom class with it. Here's what I do have:
static void Main(string[] args)
{
var order = ConvertXMLMessage<ItemReply>(request);
}
protected static T ConvertXMLMessage<T>(String xmlData) where T : class, new()
{
var xml = new XmlDocument();
xml.LoadXml(xmlData);
var serializer = new System.Xml.Serialization.XmlSerializer(typeof(T));
using (var xmlReader = new XmlNodeReader(xml.DocumentElement))
{
T work = (T)(serializer.Deserialize(xmlReader));
return work;
}
}
public class ItemReply
{
[XmlElement("ITEM_REPLY")]
public ItemAvlReply ITEM_REPLY { get; set; }
}
public class ItemAvlReply
{
[XmlElement("TRAN_ID")]
public string TRAN_ID { get; set; }
[XmlElement("REPLY_CODE")]
public string REPLY_CODE { get; set; }
[XmlElement("UNIT_PRICE")]
public string UNIT_PRICE { get; set; }
[XmlElement("SUP_LOCS")]
public SupplierLocations SUP_LOCS;
[XmlElement("MESSAGE")]
public string MESSAGE { get; set; }
[XmlElement("QTY_BREAKS")]
public string QTY_BREAKS { get; set; }
}
public class SupplierLocations
{
[XmlElement("SUP_LOC")]
public List<SupplierLocation> SUP_LOC;
}
public class SupplierLocation
{
[XmlElement("SUP_LOC_ID")]
public string SUP_LOC_ID { get; set; }
[XmlElement("COUNTRY_ID")]
public string COUNTRY_ID { get; set; }
[XmlElement("QTY_AVL")]
public string QTY_AVL { get; set; }
[XmlElement("ITEM_UOM")]
public string ITEM_UOM { get; set; }
}
This works perfectly minus the List<Item> part. I'm not overly experienced with LINQ and I'm not sure how to go about declaring a sub array in my class via this statement. I am also open to a different approach from creating the List<Item> part, I'm just not sure where to start otherwise. Is there a better approach for what I'm need to do? Is there an easy solution I am just unaware of in LINQ?
Here's a simple way to do it, assuming the example XML file you provided has typos. I assumed the OrderId has a closing tag, and that the closing tag for Items should be /Items.
Here's the version of the xml I used:
<Order>
<TransactionID>123</TransactionID>
<OrderID>1</OrderID>
<Items Number="2">
<Item>
<ItemName>Test</ItemName>
<Color>Red</Color>
</Item>
<Item>
<ItemName>Test1</ItemName>
<Color>Blue</Color>
</Item>
</Items>
</Order>
Here's the code to read/write the XML: (the xml variable is a String)
var order = ConvertXMLMessage<Order>(xml);
WriteXMLFile<Order>(order, #"test.xml");
Here's the ConvertXMLMessage and WriteXMLFile functions:
protected static T ConvertXMLMessage<T>(String xmlData) where T : class, new()
{
var xml = new XmlDocument();
xml.LoadXml(xmlData);
var serializer = new System.Xml.Serialization.XmlSerializer(typeof(T));
using (var xmlReader = new XmlNodeReader(xml.DocumentElement))
{
T work = (T)(serializer.Deserialize(xmlReader));
return work;
}
}
protected static void WriteXMLFile<T>(T item, String saveLocation) where T : class, new()
{
System.Xml.Serialization.XmlSerializer writer = new System.Xml.Serialization.XmlSerializer(typeof(T));
System.IO.StreamWriter file = new System.IO.StreamWriter(saveLocation);
writer.Serialize(file, item);
file.Close();
}
and here's the class structure:
public class Order
{
[XmlElement("TransactionID")]
public string TransactionId { get; set; }
[XmlElement("OrderID")]
public string OrderId { get; set; }
[XmlElement("Items")]
public ItemsContainer Items;
}
public class ItemsContainer
{
[XmlAttribute("Number")]
public Int32 Number { get; set; }
[XmlElement("Item")]
public List<Item> Items;
}
public class Item
{
[XmlElement("ItemName")]
public string ItemName { get; set; }
[XmlElement("Color")]
public string Color { get; set; }
}
As you'll notice I added some attributes to let the XML parser know how to handle the class when it's converting from/to the XML. I also added another small class called "ItemsContainer" just to hold the details on the Items tag. If you didn't need the "Number" attribute, then you could probably find a way to do away with this. However, this should get you in the right direction.
The example I provided is a simple version of how I usually handle the situation, obviously there's some improvements you can make depending on your needs.
Edit
I changed the Item class to use ItemName instead of TransactionId. It was an oversight on my part.
Edit 2
Here's the corrections you need to make to the newly posted code. The reason the Order class worked in the previous example was it matched the root XML element. You're new XML does align with the base class. So we need to add in a couple more attributes to make this work. You can also remove your ItemReply class. It's not needed.
So here's the new classes:
[XmlRoot("ITEM_REPLY")]
public class ItemAvlReply
{
[XmlElement("TRAN_ID")]
public string TRAN_ID { get; set; }
[XmlElement("REPLY_CODE")]
public string REPLY_CODE { get; set; }
[XmlElement("UNIT_PRICE")]
public string UNIT_PRICE { get; set; }
[XmlElement("SUP_LOCS")]
public SupplierLocations SUP_LOCS;
[XmlElement("MESSAGE")]
public string MESSAGE { get; set; }
[XmlElement("QTY_BREAKS")]
public string QTY_BREAKS { get; set; }
}
public class SupplierLocations
{
[XmlElement("SUP_LOC")]
public List<SupplierLocation> SUP_LOC;
}
public class SupplierLocation
{
[XmlElement("SUP_LOC_ID")]
public string SUP_LOC_ID { get; set; }
[XmlElement("COUNTRY_ID")]
public string COUNTRY_ID { get; set; }
[XmlElement("QTY_AVL")]
public string QTY_AVL { get; set; }
[XmlElement("ITEM_UOM")]
public string ITEM_UOM { get; set; }
}
Everything else should remain the same. The parsing/converting the XML to classes should work without any changes.
I have created a little logg program and with this i can save custom classes to xml, and convert them back from xml to a class. this works fine, but the problem is if i want to add one class to the list in the xml, i have to read all of them, add one class and rewrite all of them if i use this method, now i know i can manualy add a class by searching for elements and so on, but i wondered if i could do this in the way that i write all of them.
this is an example of the code that i ame using:
public static void Test()
{
List<LoggInformation> infos = new List<LoggInformation>();
infos.Add(new LoggInformation() { Level = BpuInterface.BpuInterface.BPUController.LoggLevel.Debug, Message = "error1" });
infos.Add(new LoggInformation() { Level = BpuInterface.BpuInterface.BPUController.LoggLevel.Error, Message = "error2" });
DataContractSerializer dd = new DataContractSerializer(typeof(List<LoggInformation>));
using (var writer = new StreamWriter(#"C://testLoggfile.xml"))
{
dd.WriteObject(writer.BaseStream, infos);
}
}
public static void AddOneItem()
{
//??????????
}
[DataContract]
public class LoggInformation
{
[DataMemberAttribute]
public BpuInterface.BpuInterface.BPUController.LoggLevel Level { get; set; }
[DataMemberAttribute]
public Source Source { get; set; }
[DataMemberAttribute]
public string ExceptionMessage { get; set; }
[DataMemberAttribute]
public string ExceptionStack { get; set; }
[DataMemberAttribute]
public string ThreadName { get; set; }
[DataMemberAttribute]
public System.Threading.ApartmentState ThreadApartmentState { get; set; }
[DataMemberAttribute]
public string Message { get; set; }
[DataMemberAttribute]
public DateTime DateTime { get; set; }
}
You can use this instead new StreamWriter(#"C://testLoggfile.xml"), true) will append line at the end of file.
public static void Test()
{
...
using (var writer = new StreamWriter(#"C://testLoggfile.xml"), true) // this will append line at the end of file.
{
dd.WriteObject(writer.BaseStream, infos);
}
}
...