I have a csv file with 8 columns, and I am trying to populate an object with 8 variables, each being a list to hold the columns in the csv file. Firstly, I am populating a DataTable with my csv data.
I am now trying to populate my object with the data from the DataTable
DataTable d = GetDataTableFromCSVFile(file);
CoolObject l = new CoolObject();
for (int i = 0; i < d.Rows.Count; i++)
{
l.column1[i] = d.Rows[i].Field<int>("column1"); <-- error here
}
And here is my CoolObject
public class CoolObject
{
public List<int> column1 { set; get; }
protected CoolObject()
{
column1 = new List<int>();
}
}
Unfortunately I am receiving an error on the highlighted line:
System.InvalidCastException: Specified cast is not valid
Why is this not allowed? How do I work around it?
Obviously you DataTable contains columns of type string, so do integer validation in GetDataTableFromCSVFile method, so consumers of this method don't need to worry about it.
Obviously you DataTable contains columns of type string, so do integer validation in GetDataTableFromCSVFile method, so consumers of this method don't need to worry about it.
private DataTable GetDataTableFromCSVFile()
{
var data = new DataTable();
data.Columns.Add("Column1", typeof(int));
// Read lines of file
// line is imaginery object which contains values of one row of csv data
foreach(var line in lines)
{
var row = data.NewRow();
int.TryParse(line.Column1Value, out int column1Value)
row.SetField("Column1", column1Value) // will set 0 if value is invalid
// other columns
}
return data;
}
Then another problem with your code, that you assugn new values to List<int> through index, where list is empty
l.column1[i] = d.Rows[i].Field<int>("column1");
Above line will throw exception because empty list doesn't have item on index i.
So you in the end your method will look
DataTable d = GetDataTableFromCSVFile(file);
CoolObject l = new CoolObject();
foreach (var row in d.Rows)
{
l.column1.Add(row.Field<int>("column1"));
}
In case you are using some third-party library for retrieving data from csv to DataTable - you can check if that library provide possibility to validate/convert string values to expected types in DataTable.
Sounds like someone didn't enter a number in one of the cells. You'll have to perform a validation check before reading the value.
for (int i = 0; i < d.Rows.Count; i++)
{
object o = d.rows[i]["column1"];
if (!o is int) continue;
l.column1[i] = (int)o;
}
Or perhaps it is a number but for some reason is coming through as a string. You could try it this way:
for (int i = 0; i < d.Rows.Count; i++)
{
int n;
bool ok = int.TryParse(d.rows[i]["column1"].ToString(), out n);
if (!ok) continue;
l.column1[i] = n;
}
Related
I'm trying to add values to a IExcelDataValidationList but the way I'm doing it right now is not very efficient.
I need to create many rows in my worksheet and populate a list for a certain cell for each of these rows. When I have many values to add to the list, this is taking forever.
for (var x = headerRowIndex + 1; x < 1000; x++)
{
var address = ExcelCellBase.TranslateFromR1C1($"R{x}C{colIndex}", 0, 0);
IExcelDataValidationList list = mainWorksheet.DataValidations.AddListValidation(address);
foreach (var e in values)
{
list.Formula.Values.Add(e);
}
}
You see how this can take a long time if values contain a lot of options.
Here's what I tried:
List<string> validationValues = new List<string>();
validationValues = values.ToList();
for (var x = headerRowIndex + 1; x < 1000; x++)
{
var address = ExcelCellBase.TranslateFromR1C1($"R{x}C{colIndex}", 0, 0);
var list = mainWorksheet.DataValidations.AddListValidation(address);
((List<string>)list.Formula.Values).AddRange(validationValues);
}
So I'm trying to add all values to the list at once. This compiles fine, but I'm getting this exception:
System.InvalidCastException: 'Unable to cast object of type
'DataValidationList' to type
'System.Collections.Generic.List`1[System.String]'.'
I've tried casting directly to DataValidationList but it's defined at private and only accessible by EPPlus itself.
Any ideas ?
Instead of creating IExcelDataValidationList per cell, create one for the whole column (address can include range of cells):
var address = ExcelCellBase.GetAddress(headerRowIndex + 1, colIndex, headerRowIndex + 1000, colIndex, true); // whole column
IExcelDataValidationList list = mainWorksheet.DataValidations.AddListValidation(address);
foreach (var e in values)
{
list.Formula.Values.Add(e);
}
I am making a program in Visual Studio where you can read in an excel file in a specific format and where my program converts the data from the excel file in a different format and stores it in a database table.
Below you can find a part of my code where something strange happens
//copy schema into new datatable
DataTable _longDataTable = _library.Clone();
foreach (DataRow drlibrary in _library.Rows)
{
//count number of variables in a row
string check = drlibrary["Check"].ToString();
int varCount = check.Length - check.Replace("{", "").Length;
int count_and = 0;
if (check.Contains("and") || check.Contains("or"))
{
count_and = Regex.Matches(check, "and").Count;
varCount = varCount - count_and;
}
//loop through number of counted variables in order to add rows to long datatable (one row per variable)
for (int i = 1; i <= varCount; i++)
{
var newRow = _longDataTable.NewRow();
newRow.ItemArray = drlibrary.ItemArray;
string j = i.ToString();
//fill variablename with variable number
if (i < 10)
{
newRow["VariableName"] = "Variable0" + j;
}
else
{
newRow["VariableName"] = "Variable" + j;
}
}
}
When varCount equals 1, I get the following error message when running the program after inserting an excel file
The source contains no DataRows.
I don't know why I can't run the for loop with just one iteration. Anyone who can help me?
I have a data row which I would like to make into an object array since to add it to my data table I need an object array. What I have made up to now is this...
data_table.Rows.Add(data_row.ToArray<object>());
but this does not work since this does not give an object array, at least thats what my compiler tells me
You can use ItemArray property on DataRow type.
object[] arr = data_row.ItemArray;
You could always make an extension method to DataRow like this:
public static class DataRowExtension
{
// Class for: Conversion to object[]
public static object[] ToObjectArray(this DataRow dataRow)
{
// Identifiers used are:
int columnCount = dataRow.Table.Columns.Count;
object[] objectArray = new object[columnCount];
// Check the row is not empty
if (columnCount == 0)
{
return null;
}
// Go through the row to add each element to the array
for (int i = 0; i < columnCount; i++)
{
objectArray[i] = dataRow[i];
}
// Return the object array
return objectArray;
}
}
Extension methods are great.
how can we read all lines from notepad comma separated and updated specific column conditionally, its successfully iterating no error is coming but not to update the value in C#
string[] existingLines = File.ReadAllLines(filepath);
foreach (var row in existingLines)
{
row.Split(Text_Separator)[0] = "Test Data";
}
var newdata = existingLines;
You'll need to use a for loop to modify an item while iterating over it. I believe you'll get an error if using foreach. This is because you are exposing an enumerator which is read-only.
This will iterate through each row, modify a column and replace the current row.
string[] existingLines = File.ReadAllLines(filepath);
foreach (var i = 0; i < existingLines.Length; i++)
{
// retrieve row by index
var row = existingLines[i];
// split into array of columns
var columns = row.Split(Text_Separator);
// update column
columns[0] = "Test Data";
// create row from array of columns
var updatedRow = string.Join(Text_Separator, columns);
// update row in array of rows
existingLines[i] = updatedRow;
}
var newdata = existingLines;
String.Split method will create a new instance of string array which is not referred to the existingLines variable. Hence, updating the value returned from Split does not reflect on existingLines
String[] stringArray = splitStrings;
if( arrIndex!= maxItems) {
stringArray = new String[arrIndex];
for( int j = 0; j < arrIndex; j++) {
stringArray[j] = splitStrings[j];
}
}
return stringArray;
https://github.com/Microsoft/referencesource/blob/master/mscorlib/system/string.cs
I have a list of DataTables like
List<DataTable> a = new List<DataTable>();
I want to make a deep copy of this list (i.e. copying each DataTable). My code currently looks like
List<DataTable> aCopy = new List<DataTable>();
for(int i = 0; i < a.Rows.Count; i++) {
aCopy.Add(a[i].Copy());
}
The performance is absolutely terrible, and I am wondering if there is a known way to speed up such a copy?
Edit: do not worry about why I have this or need to do this, just accept that it is part of a legacy code base that I cannot change
if you have to copy a data table it is essentially an N time operation. If the data table is very large and causing a large amount of allocation you may be able to speed up the operation by doing a section at a time, but you are essentially bounded by the work set.
You can try the following - it gave me a performance boost, although your mileage might vary! I've adapted it to your example to demonstrate how to copy a datatable using an alternative mechanism - clone the table, then stream the data in. You could easily put this in an extension method.
List<DataTable> aCopy = new List<DataTable>();
for(int i = 0; i < a.Rows.Count; i++) {
DataTable sourceTable = a[i];
DataTable copyTable = sourceTable.Clone(); //Clones structure
copyTable.Load(sourceTable.CreateDataReader());
}
This was many times faster (around 6 in my use case) than the following:
DataTable copyTable = sourceTable.Clone();
foreach(DataRow dr in sourceTable.Rows)
{
copyTable.ImportRow(dr);
}
Also, If we look at what DataTable.Copy is doing using ILSpy:
public DataTable Copy()
{
IntPtr intPtr;
Bid.ScopeEnter(out intPtr, "<ds.DataTable.Copy|API> %d#\n", this.ObjectID);
DataTable result;
try
{
DataTable dataTable = this.Clone();
foreach (DataRow row in this.Rows)
{
this.CopyRow(dataTable, row);
}
result = dataTable;
}
finally
{
Bid.ScopeLeave(ref intPtr);
}
return result;
}
internal void CopyRow(DataTable table, DataRow row)
{
int num = -1;
int newRecord = -1;
if (row == null)
{
return;
}
if (row.oldRecord != -1)
{
num = table.recordManager.ImportRecord(row.Table, row.oldRecord);
}
if (row.newRecord != -1)
{
if (row.newRecord != row.oldRecord)
{
newRecord = table.recordManager.ImportRecord(row.Table, row.newRecord);
}
else
{
newRecord = num;
}
}
DataRow dataRow = table.AddRecords(num, newRecord);
if (row.HasErrors)
{
dataRow.RowError = row.RowError;
DataColumn[] columnsInError = row.GetColumnsInError();
for (int i = 0; i < columnsInError.Length; i++)
{
DataColumn column = dataRow.Table.Columns[columnsInError[i].ColumnName];
dataRow.SetColumnError(column, row.GetColumnError(columnsInError[i]));
}
}
}
It's not surprising that the operation will take a long time; not only is it row by row, but it also does additional validation.
You should specify the capacity of the list otherwise it will have to grow internally to accommodate the data. See here for the detailed explanation.
List<DataTable> aCopy = new List<DataTable>(a.Count);
I found following approach much more efficient than other ways of filtering records like LINQ, provided your search criteria is simple:
public static DataTable FilterByEntityID(this DataTable table, int EntityID)
{
table.DefaultView.RowFilter = "EntityId = " + EntityID.ToString();
return table.DefaultView.ToTable();
}