I'm running into what appears to be unexpected behavior when utilizing the FieldNotInFile attribute in my mapping file. Please see below, abbreviated examples of what I have configured.
The mapping for the header record is defined separately to keep the option open for MasterDetail engine:
public class HeaderMapping
{
public string ID;
public DateTime RptDateFrom;
public DateTime RptDateTo;
public DateTime GenerationDate;
....
}
I would like to combine the values retrieved from the header into the final record result so they are specified with the FieldNotInFile attribute to be added later.
public class RecordMapping
{
// Values not in source
[FieldNotInFile()]
public string ID;
[FieldNotInFile()]
public DateTime RptDateFrom;
[FieldNotInFile()]
public DateTime RptDateTo;
[FieldNotInFile()]
public DateTime GenerationDate;
// Start values from source
public string RowPrefix;
public string Field1;
public string Field2;
public string Field3;
....
}
In the engine execution I have defined two instances. The first to capture the single header record and parse out its values. The AfterReadRecord event is used to stop the engine after the first line.
static void Main(string[] args)
{
// Extract the header
FileHelperEngine<HeaderMapping> headerEngine = new FileHelperEngine<HeaderMapping>();
headerEngine.AfterReadRecord +=
new FileHelpers.Events.AfterReadHandler<HeaderMapping>(AfterHeaderRead);
HeaderMapping[] headerRecord = headerEngine.ReadFile(source.FullName);
// Capture Values
companyId = headerRecord[0].ID;
rptDateFrom = headerRecord[0].RptDateFrom;
rptDateTo = headerRecord[0].RptDateTo;
generationDate = headerRecord[0].GenerationDate;
....
Next the record engine is created. The BeforeReadRecord event is used to insert the previously captured values into the placeholders signified in the RecordMapping with FieldNotInFile attributes.
....
// Extract the Records
FileHelperEngine<RecordMapping> recordEngine = new FileHelperEngine<RecordMapping>();
recordEngine.BeforeReadRecord +=
new FileHelpers.Events.BeforeReadHandler<RecordMapping>(BeforeThisRecord);
DataTable outputTable = recordEngine.ReadFileAsDT(source.FullName);
}
....
private static void BeforeThisRecord(EngineBase engine, BeforeReadEventArgs<RecordMapping> e)
{
e.Record.ID = companyId;
e.Record.RptDateFrom = rptDateFrom;
e.Record.RptDateTo = rptDateTo;
e.Record.GenerationDate = generationDate;
}
The outputTable result is not as expected. The fields marked as FieldNotInFile are completely omitted from the DataTable result. When debugging the process, the BeforeThisRecord method executes correctly and assigns the appropriate values but this is not reflected in the output. The DataTable columns are output as RowPrefix, Field1, Field2, etc. and not ID, RptDateFrom, RptDateTo, GenerationDate, RowPrefix, etc.
Strangely when I use the alternate method..
List <RecordMapping> recordList = recordEngine.ReadFileAsList(source.FullName);
The list items contain the RecordMapping objects with ALL of the correct values. It seems as though the DataTable translation of FieldNotInFile attributes is the culprit. Am I doing this wrong? Is this a bug?
You are correct that ReadFileAsDT() does not include the FieldNotInFile fields in the DataTable. It might be a bug, but honestly, I'm not sure how FieldNotInFile is supposed to be used - it's not in the documentation here.
I think you're better off using the Master Detail engine or alternatively just doing
RecordMapping[] recordMappings = recordEngine.ReadFile(source.FullName);
and then if you really need a DataTable, populate it yourself with something like:
DataTable outputTable = new DataTable(); // New data table.
outputTable.Columns.Add("ID", typeof(int)); // Add all columns.
outputTable.Columns.Add("RptDateFrom", typeof(DateTime));
outputTable.Columns.Add("RptDateTo", typeof(DateTime));
outputTable.Columns.Add("GenerationDate", typeof(DateTime));
outputTable.Columns.Add("RowPrefix", typeof(String));
outputTable.Columns.Add("Field1", typeof(String));
outputTable.Columns.Add("Field2", typeof(String));
outputTable.Columns.Add("Field3", typeof(String));
foreach (RecordMapping recordMapping in recordMappings)
{
outputTable.Rows.Add(
companyId,
rptDateFrom,
rptDateTo,
generationDate,
recordMapping.RowPrefix,
recordMapping.Field1,
recordMapping.Field2,
recordMapping.Field3)
}
Related
I have a DataTable with some attributes data get from a database, each data in that DataTable belong to a product, and each product could have more than one attribute.
So i have another DataTable which has all the products in a foreach by looping through each row i'm adding each product to it's List<Plu> like this:
var productAttr = new List<Plu.Attributi>();
foreach (DataRow rowPlu in dt.Rows)
{
try
{
int id = (int)rowPlu["ID_PLUREP"];
plu.Add(new Plu(
id,
(string)rowPlu["CODICE_PRP"],
(string)rowPlu[ESTESA],
(string)rowPlu[DESCR], (float)rowPlu["PRE_PRP"],
rowPlu.IsNull("IMG_IMG") ? null : (string)rowPlu["IMG_IMG"],
productAttr,
null,
(int)rowPlu["ID_MENU_PRP"]
));
}
catch
{
return plu;
}
}
For now the productAttr is empty but now i need to add to each product it's attributes, so with the following function i get a DataTable filled with data from database with all product attributes:
var attributi = Attributi(connection, idNegozio);
and then i was trying to do something like this inside the foreach
foreach (DataRow rowPlu in dt.Rows)
{
try
{
int id = (int)rowPlu["ID_PLUREP"];
plu.Add(new Plu(
id,
(string)rowPlu["CODICE_PRP"],
(string)rowPlu[ESTESA],
(string)rowPlu[DESCR], (float)rowPlu["PRE_PRP"],
rowPlu.IsNull("IMG_IMG") ? null : (string)rowPlu["IMG_IMG"],
from row in attributi.AsEnumerable() where row.Field<int>("ID_PLUREP_VAT") == id select row,
null,
(int)rowPlu["ID_MENU_PRP"]
));
}
catch
{
return plu;
}
}
But the LINQ returns a EnumerableRowCollection while i need a IEnumerable<Plu.Attribute>, so i was wondering if there is a lazy way to cast the .AsEnumerable to IEnumerable<Plu.Attrbute>...
The problem is, that the DataTable only knows which values are in the cells. It does not know what these values stand for. It does not know that the number in column 0 is in fact an Id. It doesn't know that the string in column 1 is the Name of a Customer, and the DateTime in column 2 is the Birthday of the Customer.
If you will be using the contents of this Datatable (or similar DataTables) for other queries in the future, you need some translation from DataRow to the items that they stand for.
Once you've got the translation from DataRow to Plu, you can convert your DataTable to an IEnumerable<Plu>, and do other LINQ processing on it.
Usage will be like:
DataTable table = ...
var mySelectedData = table.AsEnumerable().ToPlus()
.Where(plu => ...)
.Select(plu => new {...})
.ToList();
You need two extension methods: one that converts a DataRow to a Plu and one that converts a sequence of DataRows to a sequence of Plus. See extension methods demystified
public static Plu ToPlu(this DataRow row)
{
// TODO implement
}
public static IEnumerable<Plu> ToPlus(this IEnumerable<DataRow> dataRows)
{
// TODO: exception if null dataRows
return dataRows.Select(row => row.ToPlu());
}
If desired, create an extension method from DataTable to extract the Plus:
public static IEnumerable<Plu> ExtractPlus(this DataTable table)
{
// TODO: exception if table null
return table.AsEnumerable().ToPlus();
}
Usage:
DataTable table = ...
IEnumerable<Plu> plus = table.ExtractPlus();
I haven't got the faintest idea what a Plu is, and you forgot to mention the relevant properties of the Plu, so I'll give you an example of a table that contains Customers:
class Customer
{
public int Id {get; set;} // Id will be in column 0
public string Name {get; set;} // Name will be in column 1
...
}
public static Customer ToCustomer(this DataRow row)
{
return new Customer
{
Id = (int)row[0],
Name = (string)row[1],
};
}
If desired, instead of columnIndex you can use the name of the column.
So by only creating a ToPlu, and a one-liner method to convert sequences of DataRows to a sequence of Plus, you've extended LINQ with your methods to read your tables.
To be on the safe side, consider creating an extension method that converts a sequence of Plus to a DataTable. This way, the layout of the table is in one location: ToPlu(DataRow) and ToDataRow(Plu). Future changes in the table layout will be easier to manage, users of your DataTable will only think in sequences of Plus.
You can do something like below. If you want IEnumerable<Plu> you can remove the .ToList() from the end.
dt.AsEnumerable().Select(x => new Plu {
Id = x.Field<int>("ID_PLUREP"),
CodicePrep = x.Field<string>("CODICE_PRP"),
....
Attributes = attributi.AsEnumerable()
.Where(y => y.Field<int>("ID_PLUREP_VAT") == x.Field<int>("ID_PLUREP"))
.Select(z => new Attributi
{
....
}).ToList(),
....
}).ToList();
I'm using FastMember.ObjectReader to copy a list of structs to a DataTable, which I then use as the DataSource of a gridview:
struct Foo {
[DisplayName("title1")]
public string Bar { get; set; }
}
...
var rows = new List<Foo>();
rows.Add(new Foo { Bar = "somethingsomething" });
DataTable table = new DataTable();
using (var reader = ObjectReader.Create(rows)) {
table.Load(reader);
}
grid.DataSource = table.DefaultView;
If I select the list itself as the DataSource, the DisplayNames are used as column titles instead of the struct member name:
How can I recreate that when using FastMember.ObjectReader?
Oh, I see what you mean; you want the IDataReader to expose the [DisplayName] in the metadata; however, the primary way that is exposed is via GetSchemaTable(), and AFAIK there is no recognised key to represent [DisplayName]. It would be incorrect to pass that as the name, IMO.
Running a quick test:
var table = new DataTable();
table.Columns.Add("foo").Caption = "bar";
var schema = table.CreateDataReader().GetSchemaTable();
foreach(DataRow row in schema.Rows)
{
foreach(DataColumn col in schema.Columns)
{
Console.WriteLine($"{col.ColumnName}={row[col]}");
}
Console.WriteLine();
}
shows that indeed it is unlikely to expect it there:
ColumnName=foo
ColumnOrdinal=0
ColumnSize=-1
NumericPrecision=
NumericScale=
DataType=System.String
ProviderType=
IsLong=False
AllowDBNull=True
IsReadOnly=False
IsRowVersion=False
IsUnique=False
IsKey=False
IsAutoIncrement=False
BaseCatalogName=
BaseSchemaName=
BaseTableName=
BaseColumnName=foo
AutoIncrementSeed=0
AutoIncrementStep=1
DefaultValue=
Expression=
ColumnMapping=1
BaseTableNamespace=
BaseColumnNamespace=
this means that there isn't really anything I can suggest other than to manually populate the .Caption, perhaps use fast-member to get the data.
I have code like below (I've generified and reduced it to represent just the issue at hand). The code works, that is it takes in a DataGridView.DataSource and ulitmately, using EPPlus, outputs the data to an Excel file. My question relates to covariance and how to use it, I think.
So you see it builds newList based on the type that it has found in the DataSource. Then a little further down it adds the data using the Properties, someClassObject.Name, .Address and .Phone that are unique to this type.
My problem is that there are about 75 different classes that could be passed in through the DataGridView parameter. Each class has its own unique properties (i.e. not necessarily Name, Address, Phone) though all of the objects in given DataGridView.DataSource are of the same class.
I could have a giant switch statement based on type.FullName and then each would have its own for loop to assign the Property values to the cell. That would work but would be incredibly cumbersome. Is there a better way to do this?
public void openExcelReport(ref DataGridView dataGridView, bool bolSave = false, bool bolOpen = true, string pageTitle = "EXPORTED DATA")
{
// put dataGridView.DataSource into a List
object firstItem = null;
var myDataSource = dataGridView.DataSource;
var myList = ((System.Windows.Forms.BindingSource)dataGridView.DataSource).List;
firstItem = ((System.Collections.IList)myList)[0];
var type = firstItem.GetType();
Type PROJECT1_TYPE = typeof(Project1.SomeClass);
Type PROJECT2_TYPE = typeof(Project2.SomeOtherClass); // many more of these
dynamic newList = null;
if (type.FullName.Equals(PROJECT1_TYPE.FullName))
{
newList = new List<Project1.SomeClass>();
foreach (Project1.SomeClass someClassObject in myList)
{
newList.Add(someClassObject);
}
}
ExcelPackage package = new ExcelPackage();
using ((package)) // use EPPlus
{
// Create the worksheet
ExcelWorksheet worksheet = package.Workbook.Worksheets.Add("Worksheet 1");
// Load the datatable into the sheet, starting from cell A1. Print the column names on row 1
System.Data.DataTable dataTable = new System.Data.DataTable();
dataTable.Columns.Add("Id");
dataTable.Columns.Add("FirstColumn", typeof(string));
dataTable.Columns.Add("SecondColumn", typeof(string));
dataTable.Columns.Add("ThirdColumn", typeof(string));
dataTable.Columns[0].AutoIncrement = true;
var column_id = 0;
foreach (Project1.SomeClass someClassObject in "FirstColumn")
{
DataRow dataRow = dataTable.NewRow();
dataRow["FirstColumn"] = someClassObject.Name;
dataRow["SecondColumn"] = someClassObject.Address;
dataRow["ThirdColumn"] = someClassObject.Phone
dataTable.Rows.Add(dataRow);
column_id += 1;
}
// worksheet is now populated, so save Excel File
...
}
Instead of doing the DataRow creation within this function, you could move it out to the class implementations using a common interface to enforce it, for instance:
public interface DataRowConvertable
{
DataRow GetDataRow();
}
public class SomeClass : DataRowConvertable
{
public SomeClass() { }
public SomeClass(string name, string address, string phone)
{
Name = name;
Address = address;
Phone = phone;
}
public string Name { get; set; }
public string Address { get; set; }
public string Phone { get; set; }
public DataRow GetDataRow()
{
DataRow row = GetDataTable().NewRow();
row["Name"] = this.Name;
row["Address"] = this.Address;
row["Phone"] = this.Phone;
return row;
}
public static DataTable GetDataTable()
{
DataTable table = new DataTable("SomeClassTable");
table.Columns.Add("Name", typeof(string));
table.Columns.Add("Address", typeof(string));
table.Columns.Add("Phone", typeof(string));
return table;
}
}
You could take it further, but this should give you a good alternative and a place to start. You can either leave the GetDataTable function public, and use that as well to create your table instance, or make it private and only use it internally. I would opt for the former and use that in your function to initialize the table before filling it. You could even get rid of the static modifier and add it to your interface, but I prefer the static usage of it in this instance since it is not reliant on the instance of the class and the data involved, only on the structure.
Either way, you could then change the code you have above to look like this:
ExcelWorksheet worksheet = package.Workbook.Worksheets.Add("Worksheet 1");
System.Data.DataTable dataTable = Project1.SomeClass.GetDataTable();
foreach (Project1.SomeClass someClassObject in myList)
{
dataTable.Rows.Add(someClassObject.GetDataRow());
}
If you need an incremented id column, you could easily add that in the GetDataTable/GetDataRow functions and update them just as you were above.
This is just a quick example, it could very likely be cleaned up and optimized some, but it still conveys the idea. Hope it helps you out some.
I parse xml data from a file to a list of objects of this class:
public class VerificationSample
{
public double DesiredA { get; set; }
public double DesiredB { get; set; }
public double DesiredC { get; set; }
// ..
public void ReadXml(XmlReader reader)
{
reader.ReadToFollowing("VerificationSample");
this.DesiredA = (double)FileStructure.GetAttributeSafe(reader, "DesiredA", typeof(double));
this.DesiredB = (double)FileStructure.GetAttributeSafe(reader, "DesiredB", typeof(double));
this.DesiredC = (double)FileStructure.GetAttributeSafe(reader, "DesiredC", typeof(double));
this.ReferenceNumber = (int)FileStructure.GetAttributeSafe(reader, "ReferenceNumber", typeof(int));
// ..
}
}
I read this sample xml:
<VerificationSamples Count="4">
<VerificationSample DesiredA="0.000" DesiredB="50.000" DesiredC="0.000" ReferenceNumber="0"/>
<VerificationSample DesiredA="75.000" DesiredB="-3.000" DesiredC="0.000" ReferenceNumber="0"/>
<VerificationSample DesiredA="-30.000" DesiredB="0.000" DesiredC="0.000" ReferenceNumber="1"/>
<VerificationSample DesiredA="-2.000" DesiredB="-60.000" DesiredC="0.200" ReferenceNumber="1"/>
</VerificationSamples>
Using this routine:
public void LoadFromFile(string path)
{
XmlReader reader = XmlReader.Create(path);
// ..
reader.ReadToFollowing("VerificationSamples");
int count = (int)FileStructure.GetAttributeSafe(reader, "Count", typeof(int));
this.VerificationSamples.Clear();
for (int i = 0; i < count; i++)
{
VerificationSample newVerificationSample = new VerificationSample();
newVerificationSample.ReadXml(reader);
this.VerificationSamples.Add(newVerificationSample);
}
}
All is good so far and works like a charm. However, later when I execute another function where I try to access my VerificationSamples list to use some values of my objects, the order of the objects gets messed.. so first line in the xml (was the first object) but later it's the 3rd.. 2nd is the 1st.. 4th is the 2nd and so on (no obvious pattern). Does anyone imagine why???
Per this answer, List<T> guarantees both the order of insertion and the order of retrieval.
If the items are not coming out in the order in which they were added, there is some external force that is disrupting the order (sorting, perhaps).
If the file is not big,use XElement.
using System.Xml.Linq
XElement file=XElement.Load(#"your path");
foreach(var VerificationSample in file.Elements("VerificationSample"))
{
// do want you want XElement type VerificationSample in order at the file
}
This was happening due to a problem when retreiving data from the data base. It was fixed for now by adding a property in the xml OrderID that will be used as a reference for the order:
<VerificationSamples Count="4">
<VerificationSample DesiredA="0.000" DesiredB="50.000" DesiredC="0.000" ReferenceNumber="0" OrderID="1"/>
<VerificationSample DesiredA="75.000" DesiredB="-3.000" DesiredC="0.000" ReferenceNumber="0" OrderID="2"/>
<VerificationSample DesiredA="-30.000" DesiredB="0.000" DesiredC="0.000" ReferenceNumber="1" OrderID="3"/>
<VerificationSample DesiredA="-2.000" DesiredB="-60.000" DesiredC="0.200" ReferenceNumber="1" OrderID="4"/>
</VerificationSamples>
Anytime I retrieve the list from the data base, I just sort it by the OrderID. VerificationSamples.OrderBy(x => x.OrderID);
I have a list of DataSet.
for example:
List<DataSet> list = new List<DataSet>();
For my task, the number of DataSet in the list and the number of DataTable in each DataSet will be known at the run time.
Now I want to get those tables from the DataSets that contains a certain string in their names, for instance say 'Group1'.
I am trying with the following code:
var ds= from set in list from table in set
where li.Where(e=>e.Tables.Contains("Group")) select table;
But i am getting the error as 'An expression of type System.Data.DataSet is not allowed in a subsequent from clause in a query expression with source typeList'.
Please help me with the correct approach.
I've tried to replicate your data structure by creating another class. Hope this helps.
namespace TestCode
{
class Program
{
static void Main(string[] args)
{
var list = new List<TC> {new TC(2), new TC(2), new TC(3), new TC(4), new TC(5), new TC(2)};
var dt = list.Where( // Contains 3 elements
x => x.X == 2
);
//var ds = from set in list
// from table in set
// where li.Where(e => e.Tables.Contains("Group"))
// select table;
}
}
internal class TC
{
public int X { get; set; }
internal TC(int val)
{
X = val;
}
}
}
You original query is close. It just needs to be fleshed out a bit. First off it helps to declare the type in the from statement. Also specify you want the table collection from the set. The where clause should just need to examine the TableName property of the tables:
List<DataSet> list = new List<DataSet>();
var ds = from DataSet set in list
from DataTable table in set.Tables
where table.TableName.Contains("Group")
select table;
This gets the tables with the contained name:
var tables = list.SelectMany(x => x.Tables.Cast<DataTable>())
.Where(x => x.TableName.Contains("Group"));