Modifying DataTable with multiple foreach loops - c#

First thanks for taking the time to help if you can, now to the question. I am modifying an existing Data table to get a desired output.
I have written the code already to either do nothing, delete, or add column/row data. I viewed the Data table using dataset result viewer and it comes out as desired. But here in lies the problem, the data rows and data column's are in foreach loops, and I am now getting "collection was modified enumeration operation may not execute.".
I did everything in one Data Table, unaware that it would make the table immutable or un-modifiable if that is the correct term. I copied the table, and in a for loop, tried to iterate through it backwards as a suggestion from StackOverflow. But it did not work. So, I think I might have to either make a new method and create a new data Table within it, then set the columns, instantiate the Data Table and loop through the original table. Then add the rows to the new table and return that table.
But I am lost on how to even begin this process, I am a bit puzzled on what is the best way to do so and could use some advice. Here is My code if it helps out. I have a lot of small extension methods I am using to do the bulk of the work. Thank you for any advice on the matter!
internal void ParseNames(DataTable ConvertExcelToDataTable)
{
try
{
if (ConvertExcelToDataTable != null && ConvertExcelToDataTable.Rows != null)
{
foreach(DataRow dr in ConvertExcelToDataTable.Select())
{
foreach(DataColumn col in ConvertExcelToDataTable.Columns)
{
ConvertExcelToDataTable.AcceptChanges(); //initial DataTable Save
dr.Table.Columns.RemoveItem("LOB");
dr.Table.Columns.RemoveItem("UMID");
dr.Table.Columns.RemoveItem("MBR_SEX");
dr.Table.Columns.RemoveItem("DOS");
dr.Table.Columns.RemoveItem("Measures");
dr.Table.Columns.RemoveItem("ProjectName");
dr.Table.Columns.RemoveItem("Project Name");
dr.Table.Columns.RemoveItem("MemberKey");
dr.Table.Columns.RemoveItem("MSR_KEY");
dr.Table.Columns.RemoveItem("PRV_LAST");
dr.Table.Columns.RemoveItem("PRV_FIRST");
dr.Table.Columns.RemoveItem("PRV_MI");
dr.Table.Columns.RemoveItem("LocationName");
dr.Table.Columns.RemoveItem("Request ID");
dr.Table.Columns.RemoveItem("Measure(s)");
dr.Table.Columns.ReplaceItem("", "3");
dr.Table.Columns.ReplaceItem(" ", "3");
dr.Table.Columns.RenamingColumnValue("PERS_FIRST_NAME", "First_Name");
dr.Table.Columns.RenamingColumnValue("FirstName", "First_Name");
dr.Table.Columns.RenamingColumnValue("MBR_FIRST", "First_Name");
dr.Table.Columns.RenamingColumnValue("MBR_LAST", "Last_Name");
dr.Table.Columns.RenamingColumnValue("LastName", "Last_Name");
dr.Table.Columns.RenamingColumnValue("Gender", "Gender");
dr.Table.Columns.RenamingColumnValue("MBR_SEX", "Gender");
dr.Table.Columns.RenamingColumnValue("PERS_MID_INIT", "Middle_Initial");
dr.Table.Columns.RenamingColumnValue("MBR_MI", "Middle_Initial");
dr.Table.Columns.RenamingColumnValue("BIRTHDATE", "Date_of_Birth");
dr.Table.Columns.RenamingColumnValue("PatientDOB", "Date_of_Birth");
dr.Table.Columns.RenamingColumnValue("DOB", "Date_of_Birth");
dr.Table.Columns.RemoveSpecialCharacters("(");
dr.Table.Columns.RemoveSpecialCharacters(")");
dr.Table.Columns.RemoveSpecialCharacters(",");
dr.Table.Columns.RemoveSpecialCharacters("=");
dr.Table.Columns.RemoveSpecialCharacters("!");
dr.Table.Columns.RemoveSpecialCharacters("-");
dr.Table.Columns.SplitPatientData("Patient Name");
dr.Table.Columns.SplitPatientData("PatientName");
dr.Table.Columns.SplitPatientData("Patient, Name");
dr.Table.Columns.RenamingColumnValue("Patient", "First_Name");
dr.Table.Columns.RenamingColumnValue("Name", "Last_Name");
SplitPatientRowData(ConvertExcelToDataTable);
dr.Table.Columns.RemoveItem("Patient Name");
ReverseRowsInDataTable(ConvertExcelToDataTable);
ConvertExcelToDataTable.AcceptChanges();
}
}
}
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
throw;
}
}

What you're doing is basically converting DataTable into another DataTable. Instead of modifying the existing one in place you can create a new DataTable and return it from the method. This way you won't be modifying the DataTable while iterating.
internal DataTable ParseNames(DataTable ConvertExcelToDataTable)
{
if (ConvertExcelToDataTable == null && ConvertExcelToDataTable.Rows == null)
{
return null;
}
var output = new DataTable();
// Add columns here
try
{
foreach(DataRow dr in ConvertExcelToDataTable.Select())
{
// Add rows here
}
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
throw;
}
return output;
}

Related

The property 'ID" is part of the object's key on INSERT

We have to transfer data from one database to another. So I tried to write a program, which reads tables from the old database, create Entities and store them afterwards in the new database. At the beginning it worked very good. I tried to read only one table and transfer it to the new one. Now i receive the following error:
"The property 'Id' is part of the object's key information and cannot
be modified.
No I dont get rid of that error. Even if I try to get back to the first implementation (which worked like a charm).Here I have the definition of the Table:
Table definition
And here the code:
class MappingUtility
{
public static IEnumerable<Nation> MapNation(DataTable table, IModelFactoryService service)
{
IEnumerable<DataRow> rows = table.AsEnumerable();
Nation nat = service.Create<Nation>();
foreach(var nation in rows)
{
nat.Id = (System.Guid)nation.ItemArray[0];
nat.HVCode = (string)nation.ItemArray[1];
nat.Kurzbezeichung = (string)nation.ItemArray[2];
nat.KFZ = (string)nation.ItemArray[3];
nat.iso_a2 = (string)nation.ItemArray[4];
nat.iso_a3 = (string)nation.ItemArray[5];
nat.iso_n3 = (string)nation.ItemArray[6];
nat.Vollbezeichung = (string)nation.ItemArray[7];
nat.Updated = DateTime.Now;
nat.Created = DateTime.Now;
yield return nat;
}
}
}
using (var da = new SqlDataAdapter("SELECT * FROM NATION", "....."))
{
var table = new DataTable();
da.Fill(table);
using (var context = new DAtenbankContext())
{
int i = 0;
foreach (var nation in MappingUtility.MapNation(table, ef6))
{
Debug.WriteLine(i++);
if (context.Nation.Where(p => p.Id == nation.Id).FirstOrDefault() == null)
{
try
{
context.Entry(nation).State = EntityState.Added;
context.SaveChanges();
}
catch(DbEntityValidationException ex)
{
Debug.WriteLine("");
}
catch (DbUpdateException ex)
{
Debug.WriteLine("There where some duplicate columns in the old table.");
Debug.WriteLine(ex.StackTrace);
}
}
}
}
}
Note: The id is not autogenerated. If I try to create only one Nation at a time i can insert it. Even with this for loop I insert one nation, at the second iteration I get the error.
I suspect that you're operating on the same instance of Nation with every iteration of the loop. It appears that you only ever create one instance and then modify it over time. Entity Framework is trying to track that instance, so modifying the key is confusing it.
Move the instantiation into the loop so that you're creating new instances:
IEnumerable<DataRow> rows = table.AsEnumerable();
foreach(var nation in rows)
{
Nation nat = service.Create<Nation>();
// ...
yield return nat;
}

C# Best way to remove all datatables apart from one

I want to delete all datatables in a dataset apart from one. I've tried this :-
foreach (DataTable table in DtSet.Tables)
{
if (table.TableName != "tblAccounts")
{
DtSet.Tables.Remove(table);
}
}
but I get a
"Collection was modified; enumeration operation may not execute."
error
.
You cannot modify a collection during enumeration. But you could use a for-loop:
for (int i = DtSet.Tables.Count - 1; i >= 0; i--)
{
var table = DtSet.Tables[i];
if (table.TableName != "tblAccounts")
DtSet.Tables.Remove(table);
}
Just throwing this out there as it is untested but has no loops.
var accounts = DtSet.Tables["tblAccounts"];
DtSet.Tables.Clear();
DtSet.Tables.Add(accounts);
You'll get this anytime you try to modify a collection during a foreach. It has to do with the way the collection is iterated through.
You can do it a couple ways - one way is to determine the tables you want to remove while you're in the loop, then afterward, go back and remove them. I think the following will work:
var tablesToRemove = new List<DataTable>();
foreach (DataTable table in DtSet.Tables)
{
if (table.TableName != "tblAccounts")
{
tablesToRemove.Add(table);
}
}
foreach (DataTable table in tablesToRemove)
{
DtSet.Tables.Remove(table);
}

Query - row based table

I have no control over how the data is saved in this table. However, I have to query the table and combine the data for similar pn_id column as one row/record.
For instance current data structure is as follows,
Here we have same pn_id repeated with different question ids. This should have been really saved as one pn_id and then each question as a separate column, per my opinion. However, I have to retrieve the below data as one record like this this..
Any idea how this can be done?
Thanks
Here's some pseudocode for the transform algorithm. Note that it requires scanning the entire data set twice; there are a few other opportunities to improve the efficiency, for example, if the input data can be sorted. Also, since it's pseudocode, I haven't added handling for null values.
var columnNames = new HashSet<string> { "pn_id" };
foreach (var record in data)
columnNames.Add(record.question_id.ToString());
var table = new DataTable();
foreach (var name in columnNames)
table.Columns.Add(new DataColumn(name, typeof(string)));
foreach (var record in data)
{
var targetRecord = CreateNewOrGetExistingRecord(table, record.pn_id);
targetRecord[record.question_id.ToString()] = record.char_value ?? record.date_value.ToString();
}
And here's a sketch of the helper method:
DataRow CreateNewOrGetExistingRecord(DataTable table, object primaryKeyValue)
{
var result = table.Find(primaryKeyValue);
if (result != null)
return result;
//add code here to create a new row, add it to the table, and return it to the caller
}
the structure is fine. Wouldn't make sense to have one columns per question because you would have to add a new column every time a new question were added.
Your problem can easily be solved with PIVOT. Take a look at this link for explanation

Obtaining a MySql tables PrimaryKey in C# using OdbcConntion (System.Data.Odbc)

I am trying to retrieve the Primary Key of a table in a MySQL database using C-Sharp (C#) and running into problem.
I looked at the various MetaData collections offered and the corresponding columns, however none of them offer a primary key. The "Tables" and "Indexes" collection seem the most promising. INTERESTINGLY, OdbcConnection.GetSchema() has a PrimaryKey property/method, however there is no case where the PrimaryKey property yields something other than a null.
Indexes and Tables really did seem like the obvious choice. Yes, the tables in the database have a primary key and the database works.
Here is some code, although for this question none seem really necessary. I chose "Tables" for the purpose of this sample, but one can simply change to "Indexes" (or anything else). Obviously, COLUMN_NAME exists for Tables. I just have that there for whatever, playing.
public String GetPrimaryKey(String strTable)
{
try
{
String strPrimaryKey = null;
String[] strRestricted = new String[4] { null, null, strTable, null };
DataTable oSchema = null;
// Make sure that there is a connection.
if (ConnectionState.Open != this.m_oConnection.State)
this.m_oConnection.Open();
// DATABASE: Get the schema
oSchema = this.m_oConnection.GetSchema("Tables", strRestricted);
// Extract the information related to the primary column, in the format "{System.Data.DataColumn[0]}"
DataColumn[] oPrimaryKeys = oSchema.PrimaryKey;
// Extract: Column Names
foreach (DataRow oRow in oSchema.Rows)
{
// Get the column name.
String strColumnName = oRow["COLUMN_NAME"].ToString();
}
return strPrimaryKey;
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
return null;
}
In doing my research, I found it interesting that I could not find any posts from anyone using the GetSchema().PrimaryKey property.
So how can I identify the primary key?
Thanks in advance.
Your comment was the magic key. I did not know that the old interface was deprecated. Finding the right code was a bit of a challenge, as there is sadly no "COLUMN_NAME" on the Indexes collection or "PRIMRY" on the Columns collections, so I have to go through twice, but still, the new version is far better.
public String GetPrimaryKey(String strTable)
{
try
{
Boolean bIsPrimary = false;
String strIndexName = null;
String strColumnName = null;
String[] strRestricted = new String[4] { null, null, strTable, null };
DataTable oSchemaIndexes = null;
DataTable oSchemaIndexColumns = null;
// Make sure that there is a connection.
if (ConnectionState.Open != this.m_oConnection.State)
this.m_oConnection.Open();
// DATABASE: Get the schemas needed.
oSchemaIndexes = this.m_oConnection.GetSchema("Indexes", strRestricted);
oSchemaIndexColumns = this.m_oConnection.GetSchema("IndexColumns", strRestricted);
// Get the index name for the primary key.
foreach (DataRow oRow in oSchemaIndexes.Rows)
{
// If we have a primary key, then we found what we want.
strIndexName = oRow["INDEX_NAME"].ToString();
bIsPrimary = (Boolean)oRow["PRIMARY"];
if (true == bIsPrimary)
break;
}
// If no primary index, bail.
if (false == bIsPrimary)
return null;
// Get the corresponding column name.
foreach (DataRow oRow in oSchemaIndexColumns.Rows)
{
// Get the column name.
if (strIndexName == (String)oRow["INDEX_NAME"])
{
strColumnName = (String)oRow["COLUMN_NAME"];
break;
}
}
return strColumnName;
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
return null;
}

How to fill a DataTable with the result of a LINQ?

I have a method that calls a stored procedure. It uses the employee number as a parameter to retrieve the data of a particular employee and then fills the data table with the result.
protected DataTable CreateDT(string empNo)
{
DataTable dataTable = null;
try
{
SqlCommand cmd = new SqlCommand("FIND_EMPLOYEE_BY_EMPNO", pl.ConnOpen());
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("#EMPNO", (object)empNo));
SqlDataAdapter da = new SqlDataAdapter(pl.cmd);
dataTable = new DataTable("dt");
da.Fill(dt);
}
catch (Exception x)
{
MessageBox.Show(x.GetBaseException().ToString(), "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
finally
{
pl.MySQLConn.Close();
}
return dt;
}
What I'm trying to accomplish is convert this code to LINQ, but I don't know how get the result and fill it to my data table. See below:
alt text http://a.imageshack.us/img706/9017/testki.jpg
protected DataTable CreateDT(string empNo)
{
DataTable dataTable = null;
try
{
DataClasses1DataContext dataClass1 = new DataClasses1DataContext();
// I tried to cast it to DataTable, but it doesn't work...
dataTable = (DataTable)dataClass1.findEmployeeByID(empNo);
}
catch (Exception x)
{
MessageBox.Show(x.GetBaseException().ToString(), "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
finally
{
pl.MySQLConn.Close();
}
return dt;
}
Please guide me how to properly do this... Thanks in advance..
Why exactly do you need to fill a data table? Most bindable controls that use a data table accept any ienumerable-based object, which the collection result of standard LINQ produces.
You're having to refactor the code anyways to use the LINQ objects, so you might as well go ahead and change it all the way. You'll be happier in the long run as LINQ is much easier to use than ado.net.
But to answer the question, you would have to iterate through the list and insert each element into the datatable. Something like thus (code sample found at this article):
public DataTable LINQToDataTable<T>(IEnumerable<T> varlist)
{
DataTable dtReturn = new DataTable();
// column names
PropertyInfo[] oProps = null;
if (varlist == null) return dtReturn;
foreach (T rec in varlist)
{
// Use reflection to get property names, to create table, Only first time, others
will follow
if (oProps == null)
{
oProps = ((Type)rec.GetType()).GetProperties();
foreach (PropertyInfo pi in oProps)
{
Type colType = pi.PropertyType;
if ((colType.IsGenericType) && (colType.GetGenericTypeDefinition()
==typeof(Nullable<>)))
{
colType = colType.GetGenericArguments()[0];
}
dtReturn.Columns.Add(new DataColumn(pi.Name, colType));
}
}
DataRow dr = dtReturn.NewRow();
foreach (PropertyInfo pi in oProps)
{
dr[pi.Name] = pi.GetValue(rec, null) == null ?DBNull.Value :pi.GetValue
(rec,null);
}
dtReturn.Rows.Add(dr);
}
return dtReturn;
}
findEmployeeByID will most likely return IEnumerable<Employee>. Considering that you are switching to using LINQ, you should actually take advantage of strongly typed data and use it across you application. So, change the return type of CreateDT function and adjust the rest of the code accordingly(I assume that the stored procedure returns at most one result):
protected Employee CreateDT(string empNo)
{
try
{
DataClasses1DataContext dataClass1 = new DataClasses1DataContext();
// I tried to cast it to DataTable, but it doesn't work...
return dataClass1.findEmployeeByID(empNo).FirstOrDefault();
}
catch (Exception x)
{
MessageBox.Show(x.GetBaseException().ToString(), "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
finally
{
//might need to dispose the context here
}
return null;
}
Usage:
var employee = CreateDT(1234);
//You can now access members of employee in a typesafe manner
string name = employee.Name;
EDIT Updated code - this is how you can rewrite the old DataTable code:
protected void RetrieveEmployee(string empNo) {
Employee emp = CreateDT(empNo);// <---- Here
txtEmployeeNo.Text = emp.EmployeeNo;
txtLastName.Text = emp.LastName;
//....
}
Note the absence of array indices and late bound column specifiers - ie dt[0]["EmployeeNo"] became emp.EmployeeNo - must safer, faster and easier to read.
Why does it need to return a DataTable? One of the big advantages of LINQ is that you can work with strong-typed collections, rather than string-keyed DataTables.
protected IEnumerable<Employee> GetEmployee(string empNo)
{
try
{
DataClasses1DataContext dataClass1 = new DataClasses1DataContext();
// I tried to cast it to DataTable, but it doesn't work...
return dataClass1.findEmployeeByID(empNo);
}
catch (Exception x)
{
MessageBox.Show(x.GetBaseException().ToString(), "Error",
MessageBoxButtons.OK, MessageBoxIcon.Error);
return null;
}
finally
{
pl.MySQLConn.Close();
}
}
Some other points:
I would remove the exception handling from this method, do it a higher-level place. If you're using WinForms (which you seem to be) I would just let the exception bubble all the way to the default WinForms exception handler (add a handler to the Application.ThreadException)
I would also make the DataContext class a member variable, rather than creating and destroying one each call. The advantage here is that if you do multiple updates on the same instance, you can call one Save() and apply them all - which results in a single call to the server, rather than one call to the server for each one.

Categories

Resources