Develop Reference

Extracting mixed-order data of type xml that I get from API

I am getting data with the help of api for credit card information. The data I got is as follows.
<<?xml version="1.0" encoding="ISO-8859-9"?>
<CC5Response>
<ProcReturnCode>00</ProcReturnCode>
<Response>Approved</Response>
<Extra>
<PAN1LABEL>PRICE</PAN1LABEL>
<PAN2INDEXORDER>1</PAN2INDEXORDER>
<PAN2STATUS>NO</PAN2STATUS>
<SAFEKEYLASTMODIFIED>11/22/333</SAFEKEYLASTMODIFIED>
<PAN2INDEXACCOUNTCLOSURE>00</PAN2INDEXACCOUNTCLOSURE>
<PAN1STATUS>NO</PAN1STATUS>
<PAN1INDEXACCOUNTCLOSURE>00</PAN1INDEXACCOUNTCLOSURE>
<PAN2EXPIRY>00.1111</PAN2EXPIRY>
<PAN1OWNER>JOHN DAVIS</PAN1OWNER>
<PAN2LABEL>PRODUCT BUY</PAN2LABEL>
<PAN2DESCRIPTION>DESCRIPION</PAN2DESCRIPTION>
<PAN1>1111 22** **** 3333</PAN1>
<PAN2>1111 22** **** 3333</PAN2>
<PAN2OWNER>JOHN DAVIS</PAN2OWNER>
<PAN1EXPIRY>02.2025</PAN1EXPIRY>
<NUMBEROFPANS>3</NUMBEROFPANS>
<PAN1INDEXORDER>1</PAN1INDEXORDER>
<PAN1DESCRIPTION>PRODUCT NO</PAN1DESCRIPTION>
</Extra>
</CC5Response>
I need to get the data with the whole word "PAN" in it. I'll add this to the user's saved card info screen. I tried a few times but it didn't work. Here are the codes i tried
List<string> list = new List<string>();
list.add(responseFromServer); //"responseFromServer" data from api
string[] getPanArray = list.ToArray();
List<string> panlist = new List<string>();
string pan = "PAN";
foreach (var item in getPanArray)
{
if (item.Contains(pan))
{
panlist.Add(item);
}
}
Latter
List<string> list = new List<string>();
list.Add(responseFromServer);
String[] test = list.ToArray();
String text = "PAN";
String[] result = test.Where(s =>
{
Boolean isInThere = true;
Char[] textArray = text.ToCharArray();
foreach (Char c in textArray)
{
if (!s.Contains(c))
isInThere = false;
}
return isInThere;
}).ToArray();

So, since responseFromServer is a string, first, parse the string into an XDocument.
using System.Xml.Linq;
...
var response = XDocument.Parse(responseFromServer);
Then you can use LINQ to filter out the elements you are interested in.
using System.Linq;
...
var justPans = response
.Element("CC5Response")
.Element("Extra")
.Elements().Where(e => e.Name.LocalName.Contains("PAN"));
Gives you an IEnumerable of XElement for just the PAN elements. Then you can extract the contexts of valid text elements.
var panContents = justPans
.Select(e => e.FirstNode)
.OfType<XText>()
.Select(t => t.Value)
.ToList();
This is demoed here.

That is a horribly formatted XML, and its contents do not make any sense. For example, it says it has 3 PANs:
<NUMBEROFPANS>3</NUMBEROFPANS>
But I only see two.
Furthermore, what is the purpose of having variable starting index and then have it the same for two entries?
<PAN1INDEXORDER>1</PAN1INDEXORDER>
In short, this is next to impossible to parse easily using direct deserialization into classes which is how you are supposed to parse XML in C#.
The data from the API should be reformatted by the host to look like this before they send it to you:
<Extra>
<Items>
<Item>
<INDEXORDER>1</INDEXORDER>
<LABEL>PRICE</LABEL>
<STATUS>NO</STATUS>
<INDEXACCOUNTCLOSURE>00</INDEXACCOUNTCLOSURE>
<OWNER>JOHN DAVIS</OWNER>
<PAN>1111 22** **** 3333</PAN>
<EXPIRY>02.2025</EXPIRY>
<DESCRIPTION>PRODUCT NO</DESCRIPTION>
</Item>
<Item>
<INDEXORDER>1</INDEXORDER>
<LABEL>PRODUCT BUY</LABEL>
<STATUS>NO</STATUS>
<INDEXACCOUNTCLOSURE>00</INDEXACCOUNTCLOSURE>
<OWNER>JOHN DAVIS</OWNER>
<PAN>1111 22** **** 3333</PAN>
<EXPIRY>00.1111</EXPIRY>
<DESCRIPTION>DESCRIPION</DESCRIPTION>
</Item>
</Items>
<SAFEKEYLASTMODIFIED>11/22/333</SAFEKEYLASTMODIFIED>
</Extra>
In order for it to be easily parsed. Then it is a simple matter of making model classes:
public class Item
{
public int IndexOrder;
public string Label;
public string Status;
public int IndexAccountClosure;
public string Owner;
public string PAN;
public string Expiry;
public string Description;
}
public class ExtraData
{
public Item[] Items;
public string SafeKeyLastModified;
}
public class CCSResponse
{
public int ProcReturnCode;
public string Response;
public ExtraData Extra;
}
And then letting XMLSerializer do its job:
string XMLTextFromAPI = YourAPICallHere();
CCSResponse data;
XmlSerializer serializer = new XmlSerializer(typeof(CCSResponse));
using (TextReader reader = new StringReader(XMLTextFromAPI)) {
data = (CCSResponse)serializer.Deserialize(reader);
}
Afterwards you can just access the data like this:
foreach (Item i in data.Extra.Items) {
// do whatever you need with those PANs
}
Anything other than that, and you are asking for trouble both in writing and in maintaining that code later.
TL;DR — sometimes a proper solution is not to accept to work with garbage data that you are given and ask them to change it, at least that's what I would do if I were in your place.
I am sure someone else will post an actual answer on how to parse that garbage data which will probably involve regex or LINQ in some way.

Match sections of a List, and Replace if both exist

I've got dates from separate countries within a single List<>. I'm trying to get two records that contain the same characters before the second comma, and replace BOTH of those items with a new one.
Example:
From This:
18/04/2014,Good Friday,England and Wales
18/04/2014,Good Friday,Scotland
Into this:
18/04/2014,Good Friday,"England, Wales and Scotland"
Please note there may be multiple scenarios within the list like the above example. I've managed to get everything before the second Comma with:
splitSubstring = line.Remove(line.LastIndexOf(','));
I've tried the below, but it's clearly flawed since it won't delete both the records even if it does find a match:
foreach (var line in orderedLines)
{
if (splitSubstring == line.Remove(line.LastIndexOf(',')))
{
//Replace if previous is match here
}
splitSubstring = line.Remove(line.LastIndexOf(','));
File.AppendAllText(correctFile, line);
}

I would suggest parsing it into a structure you can work with e.g.
public class HolidayInfo
{
public DateTime Date { get; set; }
public string Name { get; set; }
public string[] Countries { get; set; }
};
And then
string[] lines = new string[]
{
"18/04/2014,Good Friday,England and Wales",
"18/04/2014,Good Friday,Scotland"
};
// splits the lines into an array of strings
IEnumerable<string[]> parsed = lines.Select(l => l.Split(','));
// copy the parsed lines into a data structure you can write code against
IEnumerable<HolidayInfo> info = parsed
.Select(l => new HolidayInfo
{
Date = DateTime.Parse(l[0]),
Name = l[1],
Countries = l[2].Split(new[] {",", " and " }, StringSplitOptions.RemoveEmptyEntries)
});
...etc. And once you have it in a helpful data structure you can begin to develop the required logic. The above code is just an example, the approach is what you should focus on.

I ended up using LINQ to pull apart the List and .Add() them into another based on an if statement. LINQ made it nice and simple.
//Using LINQ to seperate the two locations from the list.
var seperateScotland = from s in toBeInsertedList
where s.HolidayLocation == scotlandName
select s;
var seperateEngland = from e in toBeInsertedList
where e.HolidayLocation == engAndWales
select e;
Thanks for pointing me to LINQ

Working with XML under C# - Difficulties

it's me again. This time, having some issues with XML. I had everything working in VB.NET (I'll show all the code I used later) but now I'm developing something else for VB application except I'm using C# for it. Part of this involves reading an XML and populating something specific into a ListBox and then being able to click on it and get the attributes for use in other controls (description loads to a text box, etc, you'll see).
I can't seem to figure out XML for the life of me in C# however. In VB, I did it like this:
Dim games() As String = xml...<episode>.Select(Function(n) n.Value).ToArray
AvailableEpisodes.DataSource = games
Where "AvailableEpisodes" is the ListBox I wish to populate. This displayed the "This is a test" term: This is a test
And then this is the SelectedIndexChanged code:
Dim node As XElement = xml...<episode>.First(Function(n) n.Value = AvailableEpisodes.Text)
DescriptionTextBox.Text = node.#Description
AuthorTextBox.Text = node.#Author
generatedDownloadLink = node.#DownloadLink
generatedTechName = node.#TechName
IconImage.ImageLocation = node.#IconLoc
What exactly would be the C# equivalent of this? I already tried copy-pasting (just figured I'd try it) and a couple code converters and none of them seem to work.
The goal of this application will be to be able to double click on the selected index and load these strings into another window (which I can work out by attaching the nodes to a string variable) I just need to get started.
Code I've tried:
using (XmlReader reader = XmlReader.Create(testXml))
{
while (reader.Read())
{
if ((reader.NodeType == XmlNodeType.Element) && reader.Name == "episode")
{
listBox1.DataSource = reader.GetAttribute("TechName").ToList();
}
}
}
But that literally just outputs this: http://imgur.com/Naeabf9.png
Any extra information I'll toss in an edit or a reply
Thanks in advanced,
Mike

Its easy: ... corresponds to Descendants. <foo> corresponds to element name. #bar corresponds to attribute name.
var xml = XDocument.Load(path_to_xml);
var node = xml.Descendants("episode")
.First(n => n.Value == AvailableEpisodes.Text);
DescriptionTextBox.Text = (string)node.Attribute("Description");
AuthorTextBox.Text = (string)node.Attribute("Author");
generatedDownloadLink = (string)node.Attribute("DownloadLink");
generatedTechName = (string)node.Attribute("TechName");
IconImage.ImageLocation = (string)node.Attribute("IconLoc");
Note - if there is no matching node, then First will throw exception. Usually you should use FirstOrDefault in such case and then check node for null.

Is "This is a test" supposed to be one single item in the ListBox? If it is, your logic in the C# code is wrong. Indeed, reader.GetAttribute("TechName").ToList() will return an array like this
["T","h","i", "s", " ", "i", "s", " ", "t", "e", "s", "t"]
Therefore, the text gets broken up into many items in the ListBox.
To read all nodes in the xml documents into ListBox, you have to create a temporary list to hold all the results read from the xml, then at the end bind the list to ListBox listBox1.DataSource = results

Why don't you just use XML Serialization?
Assuming your XML looks something like this:
<EpisodeData>
<Episodes>
<Episode Description="..." Author="..." DownloadLink="..." ... />
<Episode Description="..." Author="..." DownloadLink="..." ... />
<Episode Description="..." Author="..." DownloadLink="..." ... />
</Episodes>
</EpisodeData>
Create classes in C# that represents the data. Something like this:
[Serializable]
public class EpisodeData
{
[XmlArray("Episodes")]
[XmlArrayItem(ElementName = "Episode")]
List<Episode> Episodes { get; set; }
}
[Serializable]
public class Episode
{
[XmlAttribute]
public string Description { get; set; }
[XmlAttribute]
public string Author { get; set; }
[XmlAttribute]
public string DownloadLink { get; set; }
...
}
Then you can deserialize and use the data like this:
EpisodeData data;
XmlSerializer serializer = new XmlSerializer(typeof(EpisodeData));
using (StreamReader sr = new StreamReader(fileName))
{
data = (EpisodeData)serializer.Deserialize(sr);
}
// Assuming you only want to see the description. If you want something else
// you might want to use a DataGrid to bind to each property in Episode or
// override ToString in Episode.
List<string> descriptions = new List<string>();
foreach (Episode episode in data.Episodes)
{
descriptions.Add(episode.Description);
}
listBox1.DataSource = descriptions;
More info on XML Serialization here:
http://msdn.microsoft.com/en-us/library/58a18dwa(v=vs.110).aspx

selection using LINQ

I have a sample xml file that looks like this:
<Books>
<Category Genre="Fiction" BookName="book_name" BookPrice="book_price_in_$" />
<Category Genre="Fiction" BookName="book_name" BookPrice="book_price_in_$" />
<Category Genre="NonFiction" BookName="book_name" BookPrice="book_price_in_$" />
<Category Genre="Children" BookName="book_name" BookPrice="book_price_in_$" />
</Books>
I need to collect all book names and book prices and pass to some other method. Right now, i get all book names and book prices seperately into two different List<string> using the following command:
List<string>BookNameList = root.Elements("Category").Select(x => (string)x.Attribute("BookName")).ToList();
List<string>BookPriceList = root.Elements("Category").Select(x => (string)x.Attribute("BookPrice")).ToList();
I create a text file and send this back to the calling function (stroing these results in a text file is a requirement, the text file has two fields bookname and bookprice).
To write to text file is use following code:
for(int i = 0; i < BookNameList.Count; i++)
{
//write BookNameList[i] to file
// Write BookPriceList[i] to file
}
I somehow dont feel good about this approach. suppose due to any reason both lists of not same size. Right now i do not take that into account and i feel using foreach is much more efficient (I maybe wrong). Is it possible to read both the entries into a datastructure (having two attributes name and price) from LINQ? then i can easily iterate over the list of that datastructure with foreach.
I am using C# for programming.
Thanks,
[Edit]: Thanks everyone for the super quick responses, i choose the first answer which I saw.

Selecting:
var books = root.Elements("Category").Select(x => new {
Name = (string)x.Attribute("BookName"),
Price = (string)x.Attribute("BookPrice")
}).ToList();
Looping:
foreach (var book in books)
{
// do something with
// book.Name
// book.Price
}

I think you could make it more tidy by some very simple means.
A somewhat simplified example follows.
First define the type Book:
public class Book
{
public Book(string name, string price)
{
Name = name;
Price = price;
}
public string Name { get; set; }
public string Price { get; set; } // could be decimal if we want a proper type.
}
Then project your XML data into a sequence of Books, like so:
var books = from category in root.Elements("Category")
select new Book((string) x.Attribute("BookName"), (string) x.Attribute("BookPrice"));
If you want better efficiency I would advice using a XmlReader and writing to the file on every encountered Category, but it's quite involved compared to your approach. It depends on your requirements really, I don't think you have to worry about it too much unless speed is essential or the dataset is huge.
The streamed approach would look something like this:
using (var outputFile = OpenOutput())
using (XmlReader xml = OpenInput())
{
try
{
while (xml.ReadToFollowing("Category"))
{
if (xml.IsStartElement())
{
string name = xml.GetAttribute("BookName");
string price = xml.GetAttribute("BookPrice");
outputFile.WriteLine(string.Format("{0} {1}", name, price));
}
}
}
catch (XmlException xe)
{
// Parse error encountered. Would be possible to recover by checking
// ReadState and continue, this would obviously require some
// restructuring of the code.
// Catching parse errors is recommended because they could contain
// sensitive information about the host environment that we don't
// want to bubble up.
throw new XmlException("Uh-oh");
}
}
Bear in mind that if your nodes have XML namespaces you must register those with the XmlReader through a NameTable or it won't recognize the nodes.

You can do this with a single query and a foreach loop.
var namesAndPrices = from category in root.Elements("Category")
select new
{
Name = category.Attribute("BookName").Value,
Price = category.Attribute("BookPrice").Value
};
foreach (var nameAndPrice in namesAndPrices)
{
// TODO: Output to disk
}

To build on Jeff's solution, if you need to pass this collection into another function as an argument you can abuse the KeyValuePair data structure a little bit and do something along the lines of:
var namesAndPrices = from category in root.Elements("Category")
select new KeyValuePair<string, string>(
Name = category.Attribute("BookName").Value,
Price = category.Attribute("BookPrice").Value
);
// looping that happens in another function
// Key = Name
// Value = Price
foreach (var nameAndPrice in namesAndPrices)
{
// TODO: Output to disk
}

Parsing a CSV formatted text file

I have a text file that looks like this:
1,Smith, 249.24, 6/10/2010
2,Johnson, 1332.23, 6/11/2010
3,Woods, 2214.22, 6/11/2010
1,Smith, 219.24, 6/11/2010
I need to be able to find the balance for a client on a given date.
I'm wondering if I should:
A. Start from the end and read each line into an Array, one at a time.
Check the last name index to see if it is the client we're looking for.
Then, display the balance index of the first match.
or
B. Use RegEx to find a match and display it.
I don't have much experience with RegEx, but I'll learn it if it's a no brainer in a situation like this.

I would recommend using the FileHelpers opensource project:
http://www.filehelpers.net/
Piece of cake:
Define your class:
[DelimitedRecord(",")]
public class Customer
{
public int CustId;
public string Name;
public decimal Balance;
[FieldConverter(ConverterKind.Date, "dd-MM-yyyy")]
public DateTime AddedDate;
}
Use it:
var engine = new FileHelperAsyncEngine<Customer>();
// Read
using(engine.BeginReadFile("TestIn.txt"))
{
// The engine is IEnumerable
foreach(Customer cust in engine)
{
// your code here
Console.WriteLine(cust.Name);
// your condition >> add balance
}
}

This looks like a pretty standard CSV type layout, which is easy enough to process. You can actually do it with ADO.Net and the Jet provider, but I think it is probably easier in the long run to process it yourself.
So first off, you want to process the actual text data. I assume it is reasonable to assume each record is seperated by some newline character, so you can utilize the ReadLine method to easily get each record:
StreamReader reader = new StreamReader("C:\Path\To\file.txt")
while(true)
{
var line = reader.ReadLine();
if(string.IsNullOrEmpty(line))
break;
// Process Line
}
And then to process each line, you can split the string on comma, and store the values into a data structure. So if you use a data structure like this:
public class MyData
{
public int Id { get; set; }
public string Name { get; set; }
public decimal Balance { get; set; }
public DateTime Date { get; set; }
}
And you can process the line data with a method like this:
public MyData GetRecord(string line)
{
var fields = line.Split(',');
return new MyData()
{
Id = int.Parse(fields[0]),
Name = fields[1],
Balance = decimal.Parse(fields[2]),
Date = DateTime.Parse(fields[3])
};
}
Now, this is the simplest example, and doesn't account for cases where the fields may be empty, in which case you would either need to support NULL for those fields (using nullable types int?, decimal? and DateTime?), or define some default value that would be assigned to those values.
So once you have that you can store the collection of MyData objects in a list, and easily perform calculations based on that. So given your example of finding the balance on a given date you could do something like:
var data = customerDataList.First(d => d.Name == customerNameImLookingFor
&& d.Date == dateImLookingFor);
Where customerDataList is the collection of MyData objects read from the file, customerNameImLookingFor is a variable containing the customer's name, and customerDateImLookingFor is a variable containing the date.
I've used this technique to process data in text files in the past for files ranging from a couple records, to tens of thousands of records, and it works pretty well.

I think the cleanest way is to load the entire file into an array of custom objects and work with that. For 3 MB of data, this won't be a problem. If you wanted to do completely different search later, you could reuse most of the code. I would do it this way:
class Record
{
public int Id { get; protected set; }
public string Name { get; protected set; }
public decimal Balance { get; protected set; }
public DateTime Date { get; protected set; }
public Record (int id, string name, decimal balance, DateTime date)
{
Id = id;
Name = name;
Balance = balance;
Date = date;
}
}
…
Record[] records = from line in File.ReadAllLines(filename)
let fields = line.Split(',')
select new Record(
int.Parse(fields[0]),
fields[1],
decimal.Parse(fields[2]),
DateTime.Parse(fields[3])
).ToArray();
Record wantedRecord = records.Single
(r => r.Name = clientName && r.Date = givenDate);

Note that both your options will scan the file. That is fine if you only want to search in the file for 1 item.
If you need to search for multiple client/date combinations in the same file, you could parse the file into a Dictionary<string, Dictionary <date, decimal>> first.
A direct answer: for a one-off, a RegEx will probably be faster.

If you're just reading it I'd consider reading in the whole file in memory using StreamReader.ReadToEnd and then treating it as one long string to search through and when you find a record you want to look at just look for the previous and next line break and then you have the transaction row you want.
If it's on a server or the file can be refreshed all the time this might not be a good solution though.

If it's all well-formatted CSV like this then I'd use something like the Microsoft.VisualBasic.TextFieldParser class or the Fast CSV class over on code project to read it all in.
The data type is a little tricky because I imagine not every client has a record for every day. That means you can't just have a nested dictionary for your looksup. Instead, you want to "index" by name first and then date, but the form of the date record is a little different. I think I'd go for something like this as I read in each record:
Dictionary<string, SortedList<DateTime, double>>

hey, hey, hey!!! why not do it with this great project on codeproject Linq to CSV, way cool!
rock solid