I have a CSV file where the headers indicate in which order the information is going to appear in the detail lines. An example would be:
COLUMN1,COLUMN2,COLUMN3
VALUE1,VALUE2,VALUE3
but could also be
COLUMN2,COLUMN3,COLUMN1
VALUE2,VALUE3,VALUE1
The class would look like this
public class CSVImportLineItem
{
public string Column1 {get; set;}
public string Column2 {get; set;}
public string Column3 {get; set;}
}
Is there a way (using FileHelpers) to read the headers in and then determine the mapping to the properties based on the header order?
Any reason why you have to specifically use the FileHelpers library?
//Use File.ReadAllLines();
List<string> lines = new List<string>() { "Column1, Column2, Column3", "1,2,3" };
var cols = lines.First().Split(',');
List<CSVImportLineItem> imported = new List<CSVImportLineItem>();
var v = lines.Skip(1).ToList().Select(line =>
{
CSVImportLineItem item = new CSVImportLineItem();
var values = line.Split(',');
for (int i = 0; i < cols.Count(); i++)
{
item.GetType().GetProperty(cols[i].Trim()).SetValue(item, values[i], null);
}
return item;
}).ToList();
You can use the one of the ClassBuilder helper classes.
DelimitedClassBuilder cb = new DelimitedClassBuilder("LineItem");
// First field
cb.AddField("Column2", typeof(string));
// Second field
cb.AddField("Column1", typeof(string));
// Third field
cb.AddField("Column3", typeof(string));
engine = new FileHelperEngine(cb.CreateRecordClass());
LineItem[] lineItems = engine.ReadFile("FileIn.txt") as LineItem[];
You can change the order of the fields based on the contents of the first line (which you can read outside of FileHelpers).
See also, the answer here.
Related
I have DataTable object with test data:
DataTable testData = new DataTable();
I'd like to assign data to variables which are the same as column names. I can do it like this:
string foo = testData.Rows[1]["foo"].ToString();
string bar = testData.Rows[1]["bar"].ToString();
or:
string foo = testData.Rows[1][nameof(foo)].ToString();
string bar = testData.Rows[1][nameof(bar)].ToString();
But, I dont want to call variable name every time, I'd like to use somethink like this:
string foo = testData.Rows[1][nameof(this)].ToString();
string bar = testData.Rows[1][nameof(this)].ToString();
Is it possible?
Why do you considder
string foo = testData.Rows[1][nameof(foo)].ToString();
to me more elegant than
string anyName = testData.Rows[1]["foo"].ToString();
You´d have to provide the name anyway. However variable-names don´t mean anything and are just arbitrary to be more readable.
Instead of relying on variable-names why not create a list of names and access the rows by the elements wthin that list?
var myList = new List<string> {"foo", "bar", ... };
Now you can just loop your list and get the rows value:
foreach(var name in myList)
{
var a = testData.Rows[1][name].ToString();
// do something with a
}
You probably want a class to represent the data in the row. You could then populate an object with those properties, either using reflection or serialization. Here's a possibility using simple reflection:
class MyRow
{
public string foo { get; set; }
public string bar { get; set; }
}
var row = testData.Rows[1];
var myRow = new MyRow();
foreach (DataColumn col in testData.Columns)
{
var prop = typeof(MyRow).GetProperty(col.ColumnName);
prop.SetValue(myRow, (string)(row[col] ?? string.Empty), null);
}
You now have an object that has properties of foo and bar.
Or, using serialization instead, it looks like the DataTable serializes nicely into a collection of objects so you can serialize the whole table, then grab the record you want after you deserialize the table:
var tableJson = JsonConvert.SerializeObject(testData);
myRow = JsonConvert.DeserializeObject<MyRow[]>(tableJson)[1];
loop through columns:
foreach(DataColumn col in testData.Columns)
{
var colValue = testData.Rows[1][col.ColumnName];
}
I have thousands of lines of data in a text file I want to make easily searchable by turning it into something easier to search (I am hoping an XML or another type of large data structure, though I am not sure if it will be the best for what I have in mind).
The data looks like this for each line:
Book 31, Thomas,George, 32, 34, 154
(each book is not unique, they are indexes so book will have several different entries of whom is listed in it, and the numbers are the page they are listed)
So I am kinda of lost on how to do this, I would want to read the .txt file, trim out all the spaces and commas, I basically get how to prep the data for it, but how would I programmatically make that many elements and values in xml or populate some other large data structure?
If your csv file does not change too much and the structure is stable, you could simply parse it to a list of objects at startup
private class BookInfo {
string title {get;set;}
string person {get;set;}
List<int> pages {get;set;}
}
private List<BookInfo> allbooks = new List<BookInfo>();
public void parse() {
var lines = File.ReadAllLines(filename); //you could also read the file line by line here to avoid reading the complete file into memory
foreach (var l in lines) {
var info = l.Split(',').Select(x=>x.Trim()).ToArray();
var b = new BookInfo {
title = info[0],
person = info[1]+", " + info[2],
pages = info.Skip(3).Select(x=> int.Parse(x)).ToList()
};
allbooks.Add(b);
}
}
Then you can easily search the allbooks list with for instance LINQ.
EDIT
Now, that you have clarified your input, I adapted the parsing a little bit to better fit your needs.
If you want to search your booklist by either the title or the person more easily, you can also create a lookup on each of the properties
var titleLookup = allbooks.ToLookup(x=> x.title);
var personLookup = allbooks.ToLookup(x => x.person);
So personLookup["Thomas, George"] will give you a list of all bookinfos that mention "Thomas, George" and titleLookup["Book 31"] will give you a list of all bookinfos for "Book 31", ie all persons mentioned in that book.
If you want the CSV file to make easily searchable by turning it into something easier to search, you can convert it to DataTable.
if you want data , you can use LINQ to XML to search
The following class generates both DataTable or Xml data format. You can pass delimeter ,includeHeader or use the default:
class CsvUtility
{
public DataTable Csv2DataTable(string fileName, bool includeHeader = false, char separator = ',')
{
IEnumerable<string> reader = File.ReadAllLines(fileName);
var data = new DataTable("Table");
var headers = reader.First().Split(separator);
if (includeHeader)
{
foreach (var header in headers)
{
data.Columns.Add(header.Trim());
}
reader = reader.Skip(1);
}
else
{
for (int index = 0; index < headers.Length; index++)
{
var header = "Field" + index; // headers[index];
data.Columns.Add(header);
}
}
foreach (var row in reader)
{
if (row != null) data.Rows.Add(row.Split(separator));
}
return data;
}
public string Csv2Xml(string fileName, bool includeHeader = false, char separator = ',')
{
var dt = Csv2DataTable(fileName, includeHeader, separator);
var stream = new StringWriter();
dt.WriteXml(stream);
return stream.ToString();
}
}
example to use:
CsvUtility csv = new CsvUtility();
var dt = csv.Csv2DataTable("f1.txt");
// Search for string in any column
DataRow[] filteredRows = dt.Select("Field1 LIKE '%" + "Thomas" + "%'");
//search in certain field
var filtered = dt.AsEnumerable().Where(r => r.Field<string>("Field1").Contains("Thomas"));
//generate xml
var xml= csv.Csv2Xml("f1.txt");
Console.WriteLine(xml);
/*
output of xml for your sample:
<DocumentElement>
<Table>
<Field0>Book 31</Field0>
<Field1> Thomas</Field1>
<Field2>George</Field2>
<Field3> 32</Field3>
<Field4> 34</Field4>
<Field5> 154</Field5>
</Table>
</DocumentElement>
*/
I have code like below (I've generified and reduced it to represent just the issue at hand). The code works, that is it takes in a DataGridView.DataSource and ulitmately, using EPPlus, outputs the data to an Excel file. My question relates to covariance and how to use it, I think.
So you see it builds newList based on the type that it has found in the DataSource. Then a little further down it adds the data using the Properties, someClassObject.Name, .Address and .Phone that are unique to this type.
My problem is that there are about 75 different classes that could be passed in through the DataGridView parameter. Each class has its own unique properties (i.e. not necessarily Name, Address, Phone) though all of the objects in given DataGridView.DataSource are of the same class.
I could have a giant switch statement based on type.FullName and then each would have its own for loop to assign the Property values to the cell. That would work but would be incredibly cumbersome. Is there a better way to do this?
public void openExcelReport(ref DataGridView dataGridView, bool bolSave = false, bool bolOpen = true, string pageTitle = "EXPORTED DATA")
{
// put dataGridView.DataSource into a List
object firstItem = null;
var myDataSource = dataGridView.DataSource;
var myList = ((System.Windows.Forms.BindingSource)dataGridView.DataSource).List;
firstItem = ((System.Collections.IList)myList)[0];
var type = firstItem.GetType();
Type PROJECT1_TYPE = typeof(Project1.SomeClass);
Type PROJECT2_TYPE = typeof(Project2.SomeOtherClass); // many more of these
dynamic newList = null;
if (type.FullName.Equals(PROJECT1_TYPE.FullName))
{
newList = new List<Project1.SomeClass>();
foreach (Project1.SomeClass someClassObject in myList)
{
newList.Add(someClassObject);
}
}
ExcelPackage package = new ExcelPackage();
using ((package)) // use EPPlus
{
// Create the worksheet
ExcelWorksheet worksheet = package.Workbook.Worksheets.Add("Worksheet 1");
// Load the datatable into the sheet, starting from cell A1. Print the column names on row 1
System.Data.DataTable dataTable = new System.Data.DataTable();
dataTable.Columns.Add("Id");
dataTable.Columns.Add("FirstColumn", typeof(string));
dataTable.Columns.Add("SecondColumn", typeof(string));
dataTable.Columns.Add("ThirdColumn", typeof(string));
dataTable.Columns[0].AutoIncrement = true;
var column_id = 0;
foreach (Project1.SomeClass someClassObject in "FirstColumn")
{
DataRow dataRow = dataTable.NewRow();
dataRow["FirstColumn"] = someClassObject.Name;
dataRow["SecondColumn"] = someClassObject.Address;
dataRow["ThirdColumn"] = someClassObject.Phone
dataTable.Rows.Add(dataRow);
column_id += 1;
}
// worksheet is now populated, so save Excel File
...
}
Instead of doing the DataRow creation within this function, you could move it out to the class implementations using a common interface to enforce it, for instance:
public interface DataRowConvertable
{
DataRow GetDataRow();
}
public class SomeClass : DataRowConvertable
{
public SomeClass() { }
public SomeClass(string name, string address, string phone)
{
Name = name;
Address = address;
Phone = phone;
}
public string Name { get; set; }
public string Address { get; set; }
public string Phone { get; set; }
public DataRow GetDataRow()
{
DataRow row = GetDataTable().NewRow();
row["Name"] = this.Name;
row["Address"] = this.Address;
row["Phone"] = this.Phone;
return row;
}
public static DataTable GetDataTable()
{
DataTable table = new DataTable("SomeClassTable");
table.Columns.Add("Name", typeof(string));
table.Columns.Add("Address", typeof(string));
table.Columns.Add("Phone", typeof(string));
return table;
}
}
You could take it further, but this should give you a good alternative and a place to start. You can either leave the GetDataTable function public, and use that as well to create your table instance, or make it private and only use it internally. I would opt for the former and use that in your function to initialize the table before filling it. You could even get rid of the static modifier and add it to your interface, but I prefer the static usage of it in this instance since it is not reliant on the instance of the class and the data involved, only on the structure.
Either way, you could then change the code you have above to look like this:
ExcelWorksheet worksheet = package.Workbook.Worksheets.Add("Worksheet 1");
System.Data.DataTable dataTable = Project1.SomeClass.GetDataTable();
foreach (Project1.SomeClass someClassObject in myList)
{
dataTable.Rows.Add(someClassObject.GetDataRow());
}
If you need an incremented id column, you could easily add that in the GetDataTable/GetDataRow functions and update them just as you were above.
This is just a quick example, it could very likely be cleaned up and optimized some, but it still conveys the idea. Hope it helps you out some.
Hi i have this structure of txt file:
Lukas 1
Zdenek 3
Martin 2
Kate 1
And i need load this data...the name i need load to comboBox...and when i choose from ComboBox for example Lukas, i need to save Name Lukas to variable Name and number 1 to variable Number...
It is possible?
I have this code now...
using (StreamReader reader = new StreamReader(#"C:\Us...nka\example.txt"))
{
string data = "";
data = reader.ReadToEnd().Trim();
}
But i need read separately Name and separately Number...Have you any ideas? Thanks..
You can use File.ReadLines and String.Split:
var lines = File.ReadLines(#"C:\Us...nka\example.txt");
var data = lines.Select(l => l.Split());
I would use a class to store both properties:
public class Person
{
public int PersonID { get; set; }
public string PersonName { get; set; }
}
Now you can load the persons in a loop or with LINQ:
List<Person> allPersons = data
.Where(arr => arr.Length >= 2 && arr[1].Trim().All(Char.IsDigit))
.Select(arr => new Person
{
PersonName = arr[0].Trim(),
PersonID = int.Parse(arr[1].Trim())
})
.ToList();
Edit:
Yes thanks...but i cant load PersonsName to combobox
You can use a BindingSource for the ComboBox. Then set the DisplayMember and ValueMember properties accordingly:
var bindingSourcePersons = new BindingSource();
bindingSourcePersons.DataSource = allPersons;
personComboBox.DataSource = bindingSourcePersons.DataSource;
personComboBox.ValueMember = "PersonID";
personComboBox.DisplayMember = "PersonName";
First create a class like this:
public class Person {
public string Name {get;set;}
public int Number {get;set;}
}
then you can use Linq to convert the string you read like this:
var people = data
.Split(new {'\r','\n'}, StringSplitOptions.RemoveEmptyEntries)
.Select(d => new Person { Name = d.Split(' ')[0], Value = int.Parse(d.Split(' ')[1])})
.ToList();
Or better you could read your data line by line, like this:
var people = from l in File.ReadLines(#"C:\Us...nka\example.txt")
let parts = l.Split(' ')
select new Person {
Name = parts[0].Trim(),
Value = int.Parse(parts[1].Trim())
};
here is a pseudo:
while the reader is not EndOfStream
read current line
split the line that was just read into a string[] array, the separator being a space
first item in the array would be the name and the second item in the array would be the number.
then you add the item in the combo box. The combobox has an Items collection and an add method, which just takes a System.Object.
http://msdn.microsoft.com/en-us/library/aa983551(v=vs.71).aspx
Is there a built-in field attribute in the FileHelper library which will add a header row in the final generated CSV?
I have Googled and didn't find much info on it. Currently I have this:
DelimitedFileEngine _engine = new DelimitedFileEngine(T);
_engine.WriteStream
(HttpContext.Current.Response.Output, dataSource, int.MaxValue);
It works, but without a header.
I'm thinking of having an attribute like FieldTitleAttribute and using this as a column header.
So, my question is at which point do I check the attribute and insert header columns? Has anyone done something similar before?
I would like to get the headers inserted and use custom text different from the actual field name just by having an attribute on each member of the object:
[FieldTitleAttribute("Custom Title")]
private string Name
and maybe an option to tell the engine to insert the header when it's generated.
So when WriteStream or WriteString is called, the header row will be inserted with custom titles.
I have found a couple of Events for DelimitedFileEngine, but not what's the best way to detect if the current record is the first row and how to insert a row before this.
I know this is an old question, but here is an answer that works for v2.9.9
FileHelperEngine<Person> engine = new FileHelperEngine<Person>();
engine.HeaderText = engine.GetFileHeader();
Here's some code that'll do it: https://gist.github.com/1391429
To use it, you must decorate your fields with [FieldOrder] (a good FileHelpers practice anyway). Usage:
[DelimitedRecord(","), IgnoreFirst(1)]
public class Person
{
// Must specify FieldOrder too
[FieldOrder(1), FieldTitle("Name")]
string name;
[FieldOrder(2), FieldTitle("Age")]
int age;
}
...
var engine = new FileHelperEngine<Person>
{
HeaderText = typeof(Person).GetCsvHeader()
};
...
engine.WriteFile(#"C:\people.csv", people);
But support for this really needs to be added within FileHelpers itself. I can think of a few design questions off the top of my head that would need answering before it could be implemented:
What happens when reading a file? Afaik FileHelpers is currently all based on ordinal column position and ignores column names... but if we now have [FieldHeader] attributes everywhere then should we also try matching properties with column names in the file? Should you throw an exception if they don't match? What happens if the ordinal position doesn't agree with the column name?
When reading as a data table, should you use A) the field name (current design), or B) the source file column name, or C) the FieldTitle attribute?
I don't know if you still need this, but here is the way FileHelper is working :
To include headers of columns, you need to define a string with headers delimited the same way as your file.
For example with '|' as delimiter :
public const string HeaderLine = #"COLUMN1|COLUMN2|COLUMN3|...";
Then, when calling your engine :
DelimitedFileEngine _engine = new DelimitedFileEngine<T> { HeaderText = HeaderLine };
If you don't want to write the headers, just don't set the HeaderText attribute on the engine.
List<MyClass> myList = new List<MyClass>();
FileHelperEngine engine = new FileHelperEngine(typeof(MyClass));
String[] fieldNames = Array.ConvertAll<FieldInfo, String>(typeof(MyClass).GetFields(), delegate(FieldInfo fo) { return fo.Name; });
engine.HeaderText = String.Join(";", fieldNames);
engine.WriteFile(MapPath("MyClass.csv"), myList);
Just to include a more complete example, which would have saved me some time, for version 3.4.1 of the FileHelpers NuGet package....
Given
[DelimitedRecord(",")]
public class Person
{
[FieldCaption("First")]
public string FirstName { get; set; }
[FieldCaption("Last")]
public string LastName { get; set; }
public int Age { get; set; }
}
and this code to create it
static void Main(string[] args)
{
var people = new List<Person>();
people.Add(new Person() { FirstName = "James", LastName = "Bond", Age = 38 });
people.Add(new Person() { FirstName = "George", LastName = "Washington", Age = 43 });
people.Add(new Person() { FirstName = "Robert", LastName = "Redford", Age = 28 });
CreatePeopleFile(people);
}
private static void CreatePeopleFile(List<Person> people)
{
var engine = new FileHelperEngine<Person>();
using (var fs = File.Create(#"c:\temp\people.csv"))
using (var sw = new StreamWriter(fs))
{
engine.HeaderText = engine.GetFileHeader();
engine.WriteStream(sw, people);
sw.Flush();
}
}
You get this
First,Last,Age
James,Bond,38
George,Washington,43
Robert,Redford,28
I found that you can use the FileHelperAsyncEngine to accomplish this. Assuming your data is a list called "output" of type "outputData", then you can write code that looks like this:
FileHelperAsyncEngine outEngine = new FileHelperAsyncEngine(typeof(outputData));
outEngine.HeaderText = "Header1, Header2, Header3";
outEngine.BeginWriteFile(outputfile);
foreach (outputData line in output){
outEngine.WriteNext(line);
}
outEngine.Close();
You can simply use FileHelper's GetFileHeader function from base class
var engine = new FileHelperEngine<ExportType>();
engine.HeaderText = engine.GetFileHeader();
engine.WriteFile(exportFile, exportData);