Large sorted datatable "wraps" around - c#

I am attempting to remove duplicates from a .NET datatable consisting of more than 50,000 rows. My approach is simple: I want to sort the datatable alphabetically, then scan through looking for rows that are the same as the row above it.
The problem I'm having is that the datatable "wraps" around when sorted. I use this to sort it:
myDataTable.DefaultView.Sort = "name";
When I view the datatable using the debugger, it is sorted alphabetically in chunks, like so:
Aardvark
Apple
Banana
...(20,000 rows later)...
Aardvark
Angle
Boat
Obviously this ruins my attempt to find duplicates. Is this some sort of optimization behavior of the framework when dealing with large tables? What is going on here?
Solution:
Here is what I was doing..
myDataTable.DefaultView.Sort = "name";
for (int i =0; i< myDataTable.DefaultView.Table.Rows.Count; i++)
{
var thisRow = myDataTable.DefaultView.Table.Rows[i];
var prevRow = myDataTable.DefaultView.Table.Rows[i-1];
}
Here is what I should have been doing:
myDataTable.DefaultView.Sort="name";
var myNewDatatable = myDataTable.DefaultView.ToTable();
for (int i =0; i< myNewDatatable.Rows.Count; i++)
{
var thisRow = myNewDatatable.Rows[i];
var prevRow = myNewDatatable.Rows[i-1];
}

myDataTable.DefaultView.Sort = "name";
Here you're sorting the DataView for the DataTable and not the DataTable.
So you have to either use the DataView (myDataTable.DefaultView)
or getting the DataRow's of the DataTable sorted by name
DataRow[] sorted = myDataTable.Select("", "name");

You can use this overload to accomplish this.
DataTable uniqueTable = myDataTable.DefaultView.ToTable("UniqueStuff", true, "SomeCol", "AnotherCol", "YetAnotherCol");

Related

Iterating through 2 lists at a time using while

var itemAmountList = _itemAmountvalue.Split(',');
var listItemDescriptionValues = _itemDescriptionvalue.Split(',');
IEnumerator enum1 = itemAmountList.GetEnumerator();
IEnumerator enum2 = listItemDescriptionValues.GetEnumerator();
DataTable dtCustomInvoiceDetail = new DataTable();
dtCustomInvoiceDetail.Columns.Add("purchaseOrderID");
dtCustomInvoiceDetail.Columns.Add("itemAmount");
dtCustomInvoiceDetail.Columns.Add("itemDescription");
while ((enum1.MoveNext()) && (enum2.MoveNext()))
{
var description = enum1.Current;
var amount = enum2.Current;
DataRow dr = dtCustomInvoiceDetail.NewRow();
dr["itemDescription"] = description;
dr["itemAmount"] = amount;
dr["purchaseOrderID"] = strPoid;
}
Above is the code that I am using to iterate through a list and populate a datatable.
I do not see any rows added to the datatable
Verified the data and looks good
it iterates through the loop the no of times as required
But still do not see any data in the datatable
In your code
DataRow dr = dtCustomInvoiceDetail.NewRow();
does not actually add a row to the table, it only creates a table row with the correct columns. In order to add the row you need to call
dtCustomInvoiceDetail.Rows.Add(dr)
at the end
the problem is that you are not adding the generated row to datatable.
Datatable.NewRow :
You must use the NewRow method to create new DataRow objects with
the same schema as the DataTable. After creating a DataRow, you
can add it to the DataRowCollection, through the DataTable
object's Rows property
you should add the row to your datatable like this
dtCustomInvoiceDetail.Rows.Add(dr);
also note that you can convert the whole enumerator logic to a simple for loop, it has more readability and less overhead
for (int i = 0; i < Math.Min(itemAmountList.Count, listItemDescriptionValues.Count); i++)
{
var description = itemAmountList[i];
var amount = listItemDescriptionValues[i];
DataRow dr = dtCustomInvoiceDetail.NewRow();
dr["itemDescription"] = description;
dr["itemAmount"] = amount;
dr["purchaseOrderID"] = strPoid;
}
also it seems that there is an inconsistency between names and variable assignments. you assign a value from listItemDescriptionValues to amount variable and from itemAmountList to description. Please double check it.

Merging DataTables with Condition

I would like to merge DataTables in a list using sum or average depending on conditions. For example:
private DataTable getData(List<DataTable> datas, string[] KeyColumnNames, string valueCol)
{
List<DataTable> dataTables = datas;
//if datas has 3 dataTables in it : dt1, dt2, dt3
// then I want to create another dataTable dtAll which will have the sum of
// valueCol of all three datatables for the row which will be conditioned using
// KeyColumnNames (can be multiple Primary keys)
return dataTable;
}
Consider all datatables to be exactly same but different values as they are tables from similar schemas but different data centers.
Thanks.
I would do
List<DataTable> dataTableList = dtList;
DataTable unionDataTable = new DataTable();
for(int i = 0; i < dtList.Count; i++)
unionDataTable = Union(unionDataTable, dtList[i], "UnionDataTable");
where the Union method is defined by something like the following
public static DataTable Union(DataTable First, DataTable Second, string strReturnedTableName)
{
// Result table.
DataTable table = new DataTable(strReturnedTableName);
// Build new columns.
DataColumn[] newcolumns = new DataColumn[First.Columns.Count];
for (int i = 0; i < First.Columns.Count; i++)
newcolumns[i] = new DataColumn(First.Columns[i].ColumnName, First.Columns[i].DataType);
// Add new columns to result table.
table.Columns.AddRange(newcolumns);
table.BeginLoadData();
// Load data from first table.
foreach (DataRow row in First.Rows)
table.LoadDataRow(row.ItemArray, true);
// Load data from second table.
foreach (DataRow row in Second.Rows)
table.LoadDataRow(row.ItemArray, true);
table.EndLoadData();
return table;
}
I hope this helps.
Edit. You can pass an Action in to the method and use this in the foreach blocks to do what you want.
Your question is way too vague to provide any actual code but you want to do a LINQ Join. It works about the same as a SQL join. This question shows joins in both query and method syntax How to do a join in linq to sql with method syntax? also you can look at the msdn docs here; http://msdn.microsoft.com/en-us/library/bb311040.aspx

Sorting Datarows using any dataColumn

I have a following datatable with 3 columns.
I want to sort the datatable rows on the basis of 2nd column.(with header E)
Is there any built in function do this?
i tried..
dt.DefaultView.Sort = dt.Columns[1].ColumnName;
dt.AcceptChanges();
Probably the easiest (and most powerful) way is to use Linq-To-DataSet and Enumerable.OrderBy:
var tblOrdered = tbl.AsEnumerable()
.OrderBy(row => row.Field<int>("E"))
.CopyToDataTable();
If you cannot use Linq, you can use DataView.Sort:
DataView view = tbl.DefaultView;
view.Sort = "E ASC";
But that uses a DataView, the original DataTable is unsorted.
So if you need the table, you could use DataView.ToTable:
tbl = view.ToTable();
You can sort a datable by column
dt.DefaultView.Sort = "E Asc";
Else this way
string[] names = new string[dt.Rows.Count];
for (int i = 0; i < dt.Rows.Count; i++)
{
names[i] = dt.Rows[i]["E"].ToString();
}
Array.Sort(names);
for (int i = 0; i < dt.Rows.Count; i++)
{
dt.Rows[i]["E"] = names[i];
}
Also dt.AcceptChanges will commits all the changes made to the table. aah msdn also say that it must be called after attempt to update is made
Remarks from MSDN
When AcceptChanges is called, any DataRow object still in edit mode
successfully ends its edits. The DataRowState also changes: all Added
and Modified rows become Unchanged, and Deleted rows are removed.
The AcceptChanges method is generally called on a DataTable after you
attempt to update the DataSet using the DbDataAdapter.Update method.

How to filter a dataview from the ith row to jth row? (C#)

I know there's the RowFilter option to filter according to column value. Also there are methods to choose the top N rows. But how do I filter out and get rows from (I prefer getting it to the same DataView dv), say, position 10 to position 23?
Here's my requirement. I have a DataView dv which has 100 rows. I have a listbox with 10 items. When I choose first item in the listbox, I want first 10 rows of the dataview to be loaded (loading part is in my program, leave it to me), if I choose 2nd item in listbox then I want row11 to row20 to be loaded and so on. I can do the listbox part, but how to choose dataview values based on row number?
This is how my code looks:
DataSet ds = new DataSet();
DataTable dt = ds.Tables["words"];
DataView dv = new DataView(dt);
Now how to have a dataview from dv based on row position?
Thanks.
You can utilize the extension methods provided in Linq to get rows by position. For example:
// just setting up a table for the sample
DataTable table = new DataTable();
table.Columns.Add("ID", typeof(int));
for (int i = 1; i <= 100; i++)
{
table.Rows.Add(i);
}
// grabbing rows 11 through 20 using Linq
DataTable filtered = table.AsEnumerable().Skip(10).Take(10).CopyToDataTable();
The above works with .NET 3.5+, C# 3.0+. For something that works in older versions of C# and .NET, you can do it manually in just a little more code.
// starting by cloning 'table' (see code above)
DataTable filtered = table.Clone();
int skipRows = 10;
int selectRows = 10;
for (int index = skipRows;
index < skipRows + selectRows && index < table.Rows.Count;
index++)
{
filtered.Rows.Add(table.Rows[index].ItemArray);
}

Copy rows from one Datatable to another DataTable?

How can I copy specific rows from DataTable to another Datable in c#? There will be more than one row.
foreach (DataRow dr in dataTable1.Rows) {
if (/* some condition */)
dataTable2.Rows.Add(dr.ItemArray);
}
The above example assumes that dataTable1 and dataTable2 have the same number, type and order of columns.
Copy Specified Rows from Table to another
// here dttablenew is a new Table and dttableOld is table Which having the data
dttableNew = dttableOld.Clone();
foreach (DataRow drtableOld in dttableOld.Rows)
{
if (/*put some Condition */)
{
dtTableNew.ImportRow(drtableOld);
}
}
Try This
String matchString="ID0001"//assuming we have to find rows having key=ID0001
DataTable dtTarget = new DataTable();
dtTarget = dtSource.Clone();
DataRow[] rowsToCopy;
rowsToCopy = dtSource.Select("key='" + matchString + "'");
foreach (DataRow temp in rowsToCopy)
{
dtTarget.ImportRow(temp);
}
Check this out, you may like it (previously, please, clone table1 to table2):
table1.AsEnumerable().Take(recodCount).CopyToDataTable(table2,LoadOption.OverwriteChanges);
Or:
table1.AsEnumerable().Where ( yourcondition ) .CopyToDataTable(table2,LoadOption.OverwriteChanges);
Supported in: 4, 3.5 SP1, you can now just call a method on the object.
DataTable dataTable2 = dataTable1.Copy()
As a result of the other posts, this is the shortest I could get:
DataTable destTable = sourceTable.Clone();
sourceTable.AsEnumerable().Where(row => /* condition */ ).ToList().ForEach(row => destTable.ImportRow(row));
I've created an easy way to do this issue
DataTable newTable = oldtable.Clone();
for (int i = 0; i < oldtable.Rows.Count; i++)
{
DataRow drNew = newTable.NewRow();
drNew.ItemArray = oldtable.Rows[i].ItemArray;
newTable.Rows.Add(drNew);
}
I needed to copy rows from multiple tables with the same structure into a new table to be used as a datasource for datagridview:
// Generate DataTable[] alltables from multiple datatables
DataTable newTable = alltables[0].Clone();
foreach (DataTable dt in alltables)
{
for (int i = 0; i < dt.Rows.Count; i++)
newTable.Rows.Add(dt.Rows[i].ItemArray);
}
below sample would be the fastest way to copy one row.
each cell is being copied based on the column name.
in case you dont need a specific cell to copy then have a try catch or add if.
if your going to copy more than 1 row then loop the code below.
DataRow dr = dataset1.Tables[0].NewRow();
for (int i = 0; i < dataset1.Tables[1].Columns.Count; i++)
{
dr[dataset1.Tables[1].Columns[i].ColumnName] = dataset1.Tables[1].Rows[0][i];
}
datasetReport.Tables[0].Rows.Add(dr);
dataset1.Tables[1].Rows[0][i]; change the index 0 to your specified row index or you can use a variable if your going to loop or if its going to be logical
private void CopyDataTable(DataTable table){
// Create an object variable for the copy.
DataTable copyDataTable;
copyDataTable = table.Copy();
// Insert code to work with the copy.
}
To copy whole datatable just do this:
DataGridView sourceGrid = this.dataGridView1;
DataGridView targetGrid = this.dataGridView2;
targetGrid.DataSource = sourceGrid.DataSource;
For those who want single command SQL query for that:
INSERT INTO TABLE002
(COL001_MEM_ID, COL002_MEM_NAME, COL002_MEM_ADD, COL002_CREATE_USER_C, COL002_CREATE_S)
SELECT COL001_MEM_ID, COL001_MEM_NAME, COL001_MEM_ADD, COL001_CREATE_USER_C, COL001_CREATE_S
FROM TABLE001;
This query will copy data from TABLE001 to TABLE002 and we assume that both columns had different column names.
Column names are mapped one-to-one like:
COL001_MEM_ID -> COL001_MEM_ID
COL001_MEM_NAME -> COL002_MEM_NAME
COL001_MEM_ADD -> COL002_MEM_ADD
COL001_CREATE_USER_C -> COL002_CREATE_USER_C
COL002_CREATE_S -> COL002_CREATE_S
You can also specify where clause, if you need some condition.
There is better way to do this.
DataTable targetDataTable = new DataTable();
targetDataTable = changedColumnMetadata.AsEnumerable().Where(dataRow => entityName.Equals(dataRow["EntityName"])).CopyToDataTable();
Please try this and let me know in case of any issues.
You can do it calling the DataTable.Copy() method, for example:
DataSet ds = new DataSet();
System.Data.DataTable dt = new System.Data.DataTable();
dt = _BOSearchView.DS.Tables[BusLib.TPV.TableName.SearchView].Copy();
ds.Tables.Add(dt);
UltGrdSaleExcel.SetDataBinding(ds, dt.TableName, true);
use Merge, this will append all rows of newDataTable with oldDateTale
oldDateTale.Merge(newDataTable);

Categories

Resources