I have a data table like below:
field_1 field_1 field_2 field_2
1 2 3 4
Now I want to convert it to:
field_1 field_2
1 3
2 4
using c#.
I tried to use this code to add columns header, but i cannot import rows to it:
private DataTable test(DataTable dispTable)
{
DataTable outputTbl = new DataTable();
int index = 0;
// Get list values of datatable
var stringArr = dispTable.Rows[0].ItemArray.Select(x => x.ToString()).ToArray();
// Add column headers
outputTbl.Columns.Add("hidden_col");
foreach (DataColumn dc in dispTable.Columns)
{
if (index % 2 == 0) //dispTable always contain 1 pair of column with same name
{
// Create a new column
outputTbl.Columns.Add(dc.ColumnName);
}
// Moving to next col
index++;
}
return outputTbl;
}
How to do it?
Are you looking for something like this?
public void Test(DataTable indata)
{
DataTable outdata = new DataTable();
ArrayList columns = new ArrayList();
// create a outdatatable to contain only distinct coulmn names
foreach (DataColumn dc in indata.Columns)
{
if (!columns.Contains(dc.ColumnName))
{
columns.Add(dc.ColumnName);
outdata.Columns.Add(dc);
}
}
foreach (DataRow dr in indata.Rows)
{
DataRow outdr = outdata.NewRow();
outdr["field_1"] = dr["field_1"] + " " +dr["field_1"]; // dont think it is possible to have two coulmns of same name
outdr["field_2"] = dr["field_2"] + " " + dr["field_2"];
}
}
Related
I have 500 Columns in my DataTable and I want to remove all of them except for 25 columns.
Is there any way to do this faster to save time and lines of code?
This is what I already tried:
private static void DeleteUselessColumns()
{
//This is example data!
List<DataColumn> dataColumnsToDelete = new List<DataColumn>();
DataTable bigData = new DataTable();
bigData.Columns.Add("Harry");
bigData.Columns.Add("Konstantin");
bigData.Columns.Add("George");
bigData.Columns.Add("Gabriel");
bigData.Columns.Add("Oscar");
bigData.Columns.Add("Muhammad");
bigData.Columns.Add("Emily");
bigData.Columns.Add("Olivia");
bigData.Columns.Add("Isla");
List<string> columnsToKeep = new List<string>();
columnsToKeep.Add("Isla");
columnsToKeep.Add("Oscar");
columnsToKeep.Add("Konstantin");
columnsToKeep.Add("Gabriel");
//This is the code i want to optimize------
foreach (DataColumn column in bigData.Columns)
{
bool keepColumn = false;
foreach (string s in columnsToKeep)
{
if (column.ColumnName.Equals(s))
{
keepColumn = true;
}
}
if (!keepColumn)
{
dataColumnsToDelete.Add(column);
}
}
foreach(DataColumn dataColumn in dataColumnsToDelete)
{
bigData.Columns.Remove(dataColumn);
}
//------------------------
}
var columnsToKeep = new List<string>() { "Isla", "Oscar", "Konstantin", "Gabriel"};
var toRemove = new List<DataColumn>();
foreach(DataColumn column in bigData.Columns)
{
if (!columnsToKeep.Any(name => column.ColumnName == name ))
{
toRemove.Add(column);
}
}
toRemove.ForEach(col => bigData.Columns.Remove(col));
Test1...test9 same code could be made a loop. No need to add the columns to delete in a list, just delete them in the first while loop. As for performance, not sure how to improve it.
You could try to use a DataView that selects the desired columns then copy to table. You need to experiment.
if they have different names create an array of string
var columns = new string[] { "Harry", "Konstantin","John"};
var columnsToKeep = new string[] { "John", "Konstantin"};
var columnsToDelete = from item in columns
where !columnsToKeep.Contains(item)
select item;
or using lambda
var columnsToDelete = columns
.Where (i=> !columnsToKeep.Contains(i))
.ToList();
toDelete
Harry
How to melt a DataTable in C# (wide to long format) as Python Pandas.melt does? https://pandas.pydata.org/docs/reference/api/pandas.melt.html
Is there any method already implemented? If not, how the code for melting a DataTable would
For example:
I have one DataTable which is in wide format, that is that has one row per id and has as many columns as variables. I would like to transform this DataTable to long format that has as many rows as combinations of id with each variable column. You can see this example in top image.
Please, if there is not clear enough visit Pandas documentation, there is more clear. (https://pandas.pydata.org/docs/reference/api/pandas.melt.html)
Note: I would like a solution that is DataTable independent, that is, that the solution is able to take parameters as id_vars, value_vars, etc... like Pandas.melt does
Any help is appreciated.
I don't know that Melt method but according to docs it seems to be an unpivot method:
public static DataTable MeltTable(DataTable inputTable, string outputColumn, params string[] unpivotColumns)
{
DataTable resultTable = new DataTable();
DataColumn col = new DataColumn(outputColumn, inputTable.Columns[outputColumn].DataType);
resultTable.Columns.Add(col);
resultTable.Columns.Add("Variable");
resultTable.Columns.Add("Value");
foreach(string unpivotColumn in unpivotColumns)
{
foreach (DataRow row in inputTable.Rows)
{
resultTable.Rows.Add(row[outputColumn], unpivotColumn, row[unpivotColumn]);
}
}
return resultTable;
}
You use it in this way:
DataTable table = new DataTable();
table.Columns.Add("Name");
table.Columns.Add("Course");
table.Columns.Add("Age", typeof(int));
table.Rows.Add("Tim", "Masters", 47);
table.Rows.Add("Bob", "Graduate", 19);
table.Rows.Add("Sheila", "Graduate", 20);
DataTable resultTable = MeltTable(table, "Name", "Course", "Age");
Result:
Name Variable Value
Tim Course Masters
Bob Course Graduate
Sheila Course Graduate
Tim Age 47
Bob Age 19
Sheila Age 20
#TimSchmelter gave me the answer but I modified a little bit to be a more general solution. Here's the code:
public static List<string> GetDifferenceColumns(DataTable dt, List<string> diffCols)
{
string[] columns = GetColumnsList(dt).ToArray();
IEnumerable<string> differenceColumns = from column in columns.Except(diffCols.ToArray()) select column;
return differenceColumns.ToList();
}
public static DataTable Melt(DataTable dt, List<string> idCols = null, List<string> varCols = null)
{
string errorPrefixString = "Error in DataProcessing Melt Method:\n";
bool varsColsIsNull = (varCols == null || varCols.Count == 0);
bool idColsIsNull = (idCols == null || idCols.Count == 0);
string varsName = "Variable";
string valueName = "Value";
if (dt.Rows.Count == 0)
{
throw new Exception(errorPrefixString + "DataTable is empty");
}
if (varsColsIsNull && varsColsIsNull)
{
throw new Exception(errorPrefixString+"You should past at least varCols or idCols");
}
if (varsColsIsNull)
{
varCols = GetDifferenceColumns(dt, idCols);
}
if (idColsIsNull)
{
idCols = GetDifferenceColumns(dt, varCols);
}
DataTable resultTable = new DataTable();
// Creating final columns of resultTable
foreach (string id in idCols)
{
resultTable.Columns.Add(id);
}
resultTable.Columns.Add(varsName);
resultTable.Columns.Add(valueName);
// Populating resultTable with the new rows
// generated by unpivoting varCols
foreach (string varCol in varCols)
{
foreach (DataRow row in dt.Rows)
{
DataRow resultRow = resultTable.NewRow();
foreach(string id in idCols)
{
resultRow[id] = row[id]; // create id cols
}
resultRow[varsName] = varCol;
resultRow[valueName] = row[varCol];
resultTable.Rows.Add(resultRow);
}
}
return resultTable;
}
How to use it:
DataTable dt = new DataTable();
dt.Columns.Add("Name");
dt.Columns.Add("Course");
dt.Columns.Add("Age");
dt.Rows.Add("Tim", "Masters", 47);
dt.Rows.Add("Bob", "Graduate", 19);
dt.Rows.Add("Sheila", "Graduate", 20);
List<string> varCols = new List<string> { "Course", "Age" };
DataTable finalDataTable = Melt(dt, varCols: varCols);
this question has been asked but mine has a different approach link 1
l have a datatable with the following data
DataTable dtProduct = new DataTable();
dtProduct.Columns.Add("productId");
dtProduct.Columns.Add("name");
DataRow dataRow = dtProduct.NewRow();
dataRow["productId"] = 1;
dataRow["name"] = "Burger";
dtProduct.Rows.Add(dataRow);
DataRow dataRow2 = dtProduct.NewRow();
dataRow2["productId"] = 2;
dataRow2["name"] = "Chicken";
dtProduct.Rows.Add(dataRow2);
DataTable dtSales = new DataTable();
dtSales.Columns.Add("productId");
dtSales.Columns.Add("saleId");
dtSales.Columns.Add("month");
dtSales.Columns.Add("quantity");
DataRow dataSalesRow = dtSales.NewRow();
dataSalesRow["productId"] = 1;
dataSalesRow["saleId"] = "1";
dataSalesRow["month"] = "Jan";
dataSalesRow["quantity"] = 3433;
dtSales.Rows.Add(dataSalesRow);
DataRow drSales2 = dtSales.NewRow();
drSales2["productId"] = 1;
drSales2["saleId"] = "2";
drSales2["month"] = "Feb";
drSales2["quantity"] = 56;
dtSales.Rows.Add(drSales2);
DataRow drSales3 = dtSales.NewRow();
drSales3["productId"] = 1;
drSales3["saleId"] = "3";
drSales3["month"] = "Mar";
drSales3["quantity"] = 34522;
dtSales.Rows.Add(drSales3);
DataRow drSales4 = dtSales.NewRow();
drSales4["productId"] = 2;
drSales4["saleId"] = "4";
drSales4["month"] = "Feb";
drSales4["quantity"] = 345;
dtSales.Rows.Add(drSales4);
And another sample 2
DataTable dtStudents = new DataTable();
dtStudents.Columns.Add("studentId");
dtStudents.Columns.Add("fullname");
DataRow drStudentrow = dtStudents.NewRow();
drStudentrow["studentId"] = 1;
drStudentrow["fullname"] = "Bil";
dtStudents.Rows.Add(drStudentrow);
DataRow drStudentrow2 = dtStudents.NewRow();
drStudentrow2["studentId"] = 2;
drStudentrow2["fullname"] = "Paul";
dtStudents.Rows.Add(drStudentrow2);
DataTable dtStudentExam = new DataTable();
dtStudentExam.Columns.Add("studentId");
dtStudentExam.Columns.Add("subjectId");
dtStudentExam.Columns.Add("mark");
DataRow dataStudentExamRow = dtStudentExam.NewRow();
dataStudentExamRow["studentId"] = 1;
dataStudentExamRow["subjectId"] = "E123";
dataStudentExamRow["mark"] = 34;
dtStudentExam.Rows.Add(dataStudentExamRow);
DataRow dataStudentExamRow2 = dtStudentExam.NewRow();
dataStudentExamRow2["studentId"] = 2;
dataStudentExamRow2["subjectId"] = "E123";
dataStudentExamRow2["mark"] = 90;
dtStudentExam.Rows.Add(dataStudentExamRow2);
DataRow dataStudentExamRow3 = dtStudentExam.NewRow();
dataStudentExamRow3["studentId"] = 1;
dataStudentExamRow3["subjectId"] = "E155";
dataStudentExamRow3["mark"] = 78;
dtStudentExam.Rows.Add(dataStudentExamRow3);
DataRow dataStudentExamRow4 = dtStudentExam.NewRow();
dataStudentExamRow4["studentId"] = 1;
dataStudentExamRow4["subjectId"] = "E101";
dataStudentExamRow4["mark"] = 12;
dtStudentExam.Rows.Add(dataStudentExamRow4);
DataRow dataStudentExamRow5 = dtStudentExam.NewRow();
dataStudentExamRow5["studentId"] = 1;
dataStudentExamRow5["subjectId"] = "E234";
dataStudentExamRow5["mark"] = 42;
dtStudentExam.Rows.Add(dataStudentExamRow5);
DataTable products(dtProduct ) has a productId which is the key and another data table sales (dtSales) has sales for the product based on month . and has a column productId .
what l want to achieve is merge this two tables to this
Same Applies for students
l have a data table student (dtStudents) but l however l have student exam marks data tables (dtStudentExam) for each subject
and combine to something like this
So my question is can l do this dynamically . l tried the following below
public static System.Data.DataTable MergeRowsToColumns(DataTable rowDataTable, string rowDataTableIdColumnName, string friendlyName, DataTable columData)
{
List<string> columnsToAdd = new List<string>();
if (rowDataTable == null)
{
return null;
}
DataTable finalDataTable = new DataTable();
finalDataTable.Columns.Add(friendlyName.ToLower());
finalDataTable.Columns.Add(rowDataTableIdColumnName.ToLower());
foreach (DataColumn column in columData.Columns)
{
if (column.ColumnName.ToString() == rowDataTableIdColumnName.ToString())
{
continue;
}
else
{
finalDataTable.Columns.Add(column.ColumnName.ToString());
columnsToAdd.Add(column.ColumnName.ToString());
}
}
foreach (DataRow row in rowDataTable.Rows)
{
DataRow newRow = finalDataTable.NewRow();
newRow[rowDataTableIdColumnName] = row[rowDataTableIdColumnName];
newRow[friendlyName] = row[friendlyName];
foreach (DataRow columnRows in columData.Rows)
{
foreach (string column in columnsToAdd)
{
var value = columnRows[column].ToString();
newRow[column] = value;
}
}
finalDataTable.Rows.Add(newRow);
}
return finalDataTable;
}
Algorithm logic
Table 1 holds the key information
Table 2 holds the value data for table
using a column name from table one for id which will be the same on table 2
can l do it generically
in one function like
var dtMergedProducts = HelperDataTable.MergeRowsToColumns(dtProduct, "productId", "name", dtSales);
var dtStudents = HelperDataTable.MergeRowsToColumns(dtStudents, "studentId", "fullname", dtStudentExam);
Please am retrieving the data as is hence can not group by in table 2 and also table two does not have a friendly name that will be displayed.
Most of the data that is loaded is not more than 100 000
this is the link of the helper utility am making git hub
Edit One
DataTable 2 will have column value one for column and the other for value
For example
dtSales
1- month : shown vertically
2- quantity: will be cell value
Sample 2
dtStudentExam
1- subjectId : shown vertically
2- mark : will be the student mark
l got it working this morning
here if the method
public static System.Data.DataTable MergeRowsToColumns(DataTable rowDataTable, string rowDataTableIdColumnName, string friendlyName, DataTable dataAllValue , string dataColumnValue , string dataColumnKey)
{
List<string> columnsToAdd = new List<string>();
if (rowDataTable == null)
{
return null;
}
DataTable finalDataTable = new DataTable();
finalDataTable.Columns.Add(friendlyName.ToLower());
finalDataTable.Columns.Add(rowDataTableIdColumnName.ToLower());
foreach (DataRow row in dataAllValue.Rows)
{
if (row[rowDataTableIdColumnName].ToString() == rowDataTableIdColumnName.ToString())
{
continue;
}
else
{
var isExistColumnValue = columnsToAdd.Where(c => c == row[dataColumnKey].ToString()).FirstOrDefault();
if(isExistColumnValue == null)
{
columnsToAdd.Add(row[dataColumnKey].ToString());
finalDataTable.Columns.Add(row[dataColumnKey].ToString());
}
}
}
foreach (DataRow row in rowDataTable.Rows)
{
DataRow newRow = finalDataTable.NewRow();
newRow[rowDataTableIdColumnName] = row[rowDataTableIdColumnName];
newRow[friendlyName] = row[friendlyName];
foreach (DataRow columnRows in dataAllValue.Rows)
{
if(row[rowDataTableIdColumnName].ToString() != columnRows[rowDataTableIdColumnName].ToString())
{
continue;
}
var columnName = columnRows[dataColumnKey].ToString();
var columnValue = columnRows[dataColumnValue].ToString();
newRow[columnName] = columnValue;
}
finalDataTable.Rows.Add(newRow);
}
return finalDataTable;
}
it looked like l was failing on adding the column names before merging
you can just call like
var dt = HelperDataTable.MergeRowsToColumns(dtProduct, "productId", "name", dtSales , "quantity", "month");
var dt2 = HelperDataTable.MergeRowsToColumns(dtStudents, "studentId", "fullname", dtStudentExam , "mark", "subjectId");
I'm trying to link data from my db (sqlite) to my textboxes, but I don't know exactly how...
Here's what i'm trying:
private void btnUpdate_Click(object sender, EventArgs e)
{
con.Open();
adapter = new SQLiteDataAdapter("SELECT PRECO_PRODUTO FROM INGREDIENTES WHERE NOME_PRODUTO='"+listProdutos.SelectedIndex+"'", con);
adapter.Fill(populate);
List<DataTable> precoIng = new List<DataTable>();
precoIng.Add(populate.Tables[0]);
con.Close();
tbPrecoProduto.Text = precoIng[0]; //error here
}
I think I need a way to convert the SQLiteDataTable to a string? I'm new on db stuffs. This is my table and this is my form design, if helps...
It seems as if you're trying to set the text of the TextBox to a DataTable. This doesn't quite make sense, as the type DataTable isn't the same as a string. You'd first have to extract the data out of your table like that:
var builder = new StringBuilder ();
foreach (var rowObject in precoIng [0].Rows)
{
var row = (DataRow) rowObject;
var id = row ["ID_PRODUTO"].ToString ();
var name = row ["NOME_PRODUTO"].ToString ();
var price = row ["PRECO_PRODUTO"].ToString ();
var count = row ["QNT_PRODUTO"].ToString ();
builder.AppendLine ($"Id: {id}; Name: {name}; Price: {price}; Count: {count}");
}
tbPrecoProduto.Text = builder.ToString ();
You can of course shorten this if you only want for example the price:
var builder = new StringBuilder ();
foreach (var rowObject in precoIng [0].Rows)
{
var row = (DataRow) rowObject;
var price = row ["PRECO_PRODUTO"].ToString ();
builder.AppendLine (price);
}
tbPrecoProduto.Text = builder.ToString ();
And, btw, precoIng[0] will always be the same as populate.Tables[0].
Alternatively you could write an extension method like that:
public static string GetData (this DataTable table)
{
var builder = new StringBuilder ();
foreach (DataRow row in table.Rows)
foreach (DataColumn column in table.Columns)
builder.Append (column.ColumnName).Append (": ").Append (row [column]).Append ("; ");
return builder.ToString ();
}
And then call precoIng[0].GetData() to get the data as a string.
If you want to be able to filter the data to get printed, you could do something like this:
public static string GetData (this DataTable table, List <string> columns = null)
{
var builder = new StringBuilder ();
foreach (DataRow row in table.Rows)
if (columns == null)
foreach (DataColumn column in table.Columns)
builder.Append (column.ColumnName).Append (": ").Append (row [column]).Append ("; ");
else
foreach (var column in columns)
builder.Append (column).Append (": ").Append (row [column]).Append ("; ");
return builder.ToString ();
}
I do however feel as if you don't want the data of your whole table as a string, but rather want the data to certain ids or something. To help you with this, however, I'd need more precise information.
Ok, so it seems as if what you actually want is something like this:
var preco = precoIng.FirstOrDefault ()?.Rows.Cast <DataRow> ().FirstOrDefault ()? ["PRECO_PRODUTO"]?.ToString () ?? "";
I am importing data from csv file, sometimes there are column headers and some times not the customer chooses custom columns(from multiple drop downs)
my problem is I am able to change the columns type and name but when I want to import data row into cloned table it just adds rows but no data with in those rows. If I rename the column to old values it works, let's say column 0 name is 0 if I change that to something else which I need to it won't fill the row below with data but If I change zero to zero again it will any idea:
here is my coding:
#region Manipulate headers
DataTable tblCloned = new DataTable();
tblCloned = tblDataTable.Clone();
int i = 0;
foreach (string item in lstRecord)
{
if (item != "Date")
{
var m = tblDataTable.Columns[i].DataType;
tblCloned.Columns[i].DataType = typeof(System.String);
tblCloned.Columns[i].ColumnName = item;
}
else if(item == "Date")
{
//get the proper date format
//FillDateFormatToColumn(tblCloned);
tblCloned.Columns[i].DataType = typeof(DateTime);
tblCloned.Columns[i].ColumnName = item;
}
i++;
}
tblCloned.AcceptChanges();
foreach (DataRow row in tblDataTable.Rows)
{
tblCloned.ImportRow(row);
}
tblCloned.AcceptChanges();
#endregion
in the second foreach loop when it calls to import data to cloned table it adds empty rows.
After couple of tries I came up with this solution which is working:
foreach (DataRow row in tblDataTable.Rows)
{
int x = 0;
DataRow dr = tblCloned.NewRow();
foreach (DataColumn dt in tblCloned.Columns)
{
dr[x] = row[x];
x++;
}
tblCloned.Rows.Add(dr);
//tblCloned.ImportRow(row);
}
but I will accept Scottie's answer because it is less code after all.
Instead of
foreach (DataRow row in tblDataTable.Rows)
{
tblCloned.ImportRow(row);
}
try
foreach (DataRow row in tblDataTable.Rows)
{
tblCloned.LoadDataRow(row.ItemArray, true);
}