Develop Reference

C# Efficient de-duplication of single Datatable column's data

I have a Datatable with some data, example as below, and need to de-duplicate any names in the names field by appending [1], [2] etc.
Current code below, works but is slow on large tables.
Any tips on the most efficient way of doing this in C# would be appreciated.
Current Table sample:
- ID Name X Y
- 1 John 45 66
- 2 Paul 44 66
- 3 George 88 102
- 4 John 33 90
- 5 John 53 37
- 6 Paul 97 65
- 7 Ringo 01 87
- 8 Ringo 76 65​
Required Table sample:
- ID Name X Y
- 1 John[1] 45 66
- 2 Paul[1] 44 66
- 3 George 88 102
- 4 John[2] 33 90
- 5 John[3] 53 37
- 6 Paul[2] 97 65
- 7 Ringo[1] 01 87
- 8 Ringo[2] 76 65​
Current code below:
foreach (DataRow aRow in ds.Tables[0].Rows) // run through all
{
string aName = aRow["Name"].ToString();
DataRow[] FoundRows = ds.Tables[0].Select("Name = '" + aName +"'"); // Find all rows with same name
if (FoundRows.Length > 1) // As will always find itself
{
int i = 1;
foreach (DataRow row in FoundRows)
{
row["Name"] = row["Name"].ToString() + "[" + i + "]";
i++;
}
ds.Tables[0].AcceptChanges(); // Ensure the rows are updated before looping around.
}
}

Here is one approach
DataTable table = new DataTable();
//test data
table.Columns.Add("Name");
table.Columns.Add("X", typeof(int));
table.Rows.Add(new object[] { "john", 10 });
table.Rows.Add(new object[] { "paul", 44 });
table.Rows.Add(new object[] { "ringo", 312 });
table.Rows.Add(new object[] { "george", 30 });
table.Rows.Add(new object[] { "john", 100 });
table.Rows.Add(new object[] { "paul", 443 });
//converting DataTable to enumerable collection of rows and then grouping by name,
//skipping groups with only one row(such as george or ringo)
var groupedData = table.AsEnumerable().GroupBy(row => row[0].ToString()).Where(g => g.Count() > 1);
//iterate through each group of <string, DataRow>
foreach (var group in groupedData)
{
int counter = 1; //counter for "[x]" suffix
//iterate through all rows under one name, eg. John
foreach (var groupedItem in group)
{
//add [x]
groupedItem[0] = string.Format("{0} [{1}]", group.Key, counter);
counter++;
}
}
EDIT: simplified code and made it a bit more efficient, as suggested by AdrianWragg

Probably old good for loop updating the whole table in one pass will be the fastest approach:
var foundNames = new Dictionary<string, int>();
for (int rowInd = 0; rowInd < dataTable.Rows.Count; rowInd++)
{
// If name is not yet found in foundNames, then store its row
// index. Don't update the dataTable yet -- this is the only
// occurrence so far.
// The index is stored inverted to distinguish from count.
//
// If name is found in foundNames, retrieve the count.
// If count is inverted (non-positive), then we've encountered
// the name second time. In this case update the row with the
// first occurrence and the current row too. Store the count of 2.
//
// If count is positive, then it's third or even later occurrence.
// Update the current row only and store the incremented count.
var name = dataTable.Rows[rowInd]["Name"].ToString();
int count;
if (!foundNames.TryGetValue(name, out count))
foundNames.Add(name, -rowInd);
else
{
if (count <= 0)
{
dataTable.Rows[-count]["Name"] = name + "[1]";
count = 1;
}
count++;
dataTable.Rows[rowInd]["Name"] = name + "[" + count + "]";
foundNames[name] = count;
}
}

Check and count values of a table column and show them in a pie chart using SQLite database in C#.NET

I am trying to draw a simple pie chart where data will come from a SQLite database. I need to check a column of a table whether it has value 2 or 1 or 0. I have written 3 functions to check and count them. Table name is banana and column name is status. The table is like following
Id B M status
1 10 101 2
2 11 102 2
3 11 103 0
4 12 104 1
5 13 105 1
6 13 105 0
7 14 106 1
8 14 106 2
The functions are
public double countA()
{
return GetDoubleEntity("SELECT COUNT(*) FROM banana WHERE banana.status = '2';");
}
public double countB()
{
return GetDoubleEntity("SELECT COUNT(*) FROM banana WHERE banana.status = '1';");
}
public double countC()
{
return GetDoubleEntity("SELECT COUNT(*) FROM banana WHERE banana.status = '0';");
}
Then I have written following codes to draw the pie chart
double Total, APercentage, BPercentage, CPercentage;
Total= countA() + countB() + countC();
APercentage= (countA() * 100) / Total;
BPercentage= (countB() * 100) / Total;
CPercentage= (countC() * 100) / Total;
//Take legend
string[] xAxis = { "A", "B", "C" };
//Take Percentage Values
double[] PiePercentage = { APercentage, BPercentage, CPercentage};
Chart.Series[0].Label = "#PERCENT";
Chart.Series[0].LegendText = "#AXISLABEL";
Chart.Series[0].Points.DataBindXY(xAxis, PiePercentage);
Chart.Series[0].Points[0].Color = Color.Green;
Chart.Series[0].Points[1].Color = Color.Red;
Chart.Series[0].Points[2].Color = Color.Yellow;
But, it is not showing results. Could anyone give me any hints what is wrong with it ? Is it problem with SQL queries or chart functions which are creating the problems ?

How to change specific column data in the datatable according to some condition

I have the following case :
I add row by row to Datatable dtItems according to the user data entry through a button .
One of the columns in my data table is Hours , and i wanna to achieve the following conditions :
1- for each user the total hours is less than or equal 5.
2- the default :if the user enter one row then hours = 5
if he enters two rows then make the first one 4 and the second one is 1
if he enters three rows then make the first one is 3 and the second is 1 and the third is 1.
etc.
3-the maximum number of rows for each user is 5.
LIKE this:
user_id | name | hours
323 | jo | 3
323 | jo | 1
323 | jo | 1
324 | jack | 4
324 | jack | 1
DataTable dtItems = GetDataTable();
DataRow dr = dtItems.NewRow();
dr["emp_num"] = txt_EmpNum.Text.Trim();
dr["name"] = txt_EmpName.Text.Trim();
dr["hours"] = 5;
dtItems.Rows.Add(dr);
GV_Employee.DataSource = dtItems;
GV_Employee.DataBind();
Session["ItemDT"] = dtItems;

I assume that you don't know how to change the DataRow's Hour fields accordingly before you insert them into database.
The only what you need to know is the new hour of the first DataRow, the others get 1 hour:
var firstHour = 5 + 1 - dtItems.Rows.Count; //where 5 is your MaxCount
for (var i = 0; i < dtItems.Rows.Count; i++) {
if (i == 0)
dtItems.Rows[i]["hours"] = firstHour;
else
dtItems.Rows[i]["hours"] = 1;
}
To prevent users from inserting more than 5 rows, you only need to check for dtItems.Rows.Count < 5 before you insert the new.
Edit: If you need it to be calculated for every emp_num in the DataTable as commented:
var q = from r in dtItems.AsEnumerable()
group r by r["emp_num"];
foreach(var empGrp in q){
var rows=empGrp.ToList();
var firstHour = 5 + 1 - rows.Count;
for (var i = 0; i < rows.Count; i++){
if (i == 0)
rows[i]["hours"] = firstHour;
else
rows[i]["hours"] = 1;
}
}

Sort several duplicate values in DataTable to one row

I've imported a DataTable from a SQL Database using SqlDataAdapter and Fill-Method.
My datatable looks like this:
Timestamp(unix time) | Value
x | 10
x | 42
x | 643
y | 5
y | 9
y | 70
...and so on. The table contains a lot of values (1000+) but has always three rows with the same timestamp.
Now I want it to look like this:
Timestamp(unix time) | Value 1 | Value 2 | Value 3
x | 10 | 42 | 643
y | 5 | 9 | 70
How can I sort it this way?
(If there are more than three values, the programm should just insert the first three values it has found)
Thanks for any help!

Thanks for your approach! I solved it myself now.
This is how I've done it:
var grouped = from myRow in myDataTable.AsEnumerable()
group myRow by myRow.Field<int>("TIMESTAMP");
foreach (var timestamp in grouped)
{
string[] myRow = new string[5];
myRow[0] = timestamp.Key.ToString();
int i = 1;
foreach (var value in timestamp)
{
myRow[i] = value.Field<double>("VALUE").ToString();
i++;
if (i > 4)
break;
}
mySortedTable.Rows.Add(myRow);
}

I think this may also be solvable in SQL, but if you want to do it programmatically, I have tested the following in LinqPad:
void Main()
{
var list = new List<Tuple<string,int>> {
Tuple.Create("x", 10),
Tuple.Create("x", 42),
Tuple.Create("x", 643),
Tuple.Create("y", 5),
Tuple.Create("y", 9),
Tuple.Create("y", 70),
};
var result =
from grp in list.GroupBy(t => t.Item1)
let firstThree = grp.Select(t => t.Item2).Take(3).ToList()
select new {
Key = grp.Key,
Value1 = firstThree[0],
Value2 = firstThree[1],
Value3 = firstThree[2] };
foreach (var item in result)
Console.WriteLine(item);
}
It assumes that you have at least three elements, otherwise you'll get an out of range exception.
While the end result is an anonymous type, you could easily pipe the results of the operation into a DataRow instead.

Merge 2 DataTables

I have 2 Datatable with these fields:
DataTable1
Hour - Value1
0 - 34
1 - 22
2 - NULL
3 - 12
..
..
23 - 10
DataTable2
Hour - Value1
0 - NULL
1 - 22
2 - 35
3 - 11
..
..
23 - NULL
I need to populate an unique DataTable that has the same "Hour" fields (the hours of the day) and for the values has to get the values from DataTable1 if is not null. If the values in Datatable1 is null, I have to take the correspondet values from DataTable2.
If also the value from DataTable2 is null, I have to log the error.
For my example, I need to get:
DataTableResult
Hour - Value1
0 - 34
1 - 22
2 - 35
3 - 12
..
..
23 - 10
How can I get this?

This is question follows a simple conditional logic you would need to process each element in a foreach statement.
You want to process this so every time an element in a row has null.
You go to that row in datatable 2 and check if that has a value if so this becomes the new value in
datatable 1.
If this does not then throw an error.
I have provided this link how to compare but really you don't need this as all you are doing is testing for null in a field in a row.

Using Linq to objects and assuming dataTable1 and dataTable2 have the same columns:
var hoursMap1 = dataTable1.Rows.Cast<DataRow>().ToDictionary(row => row[0]);
var hoursMap2 = dataTable2.Rows.Cast<DataRow>().ToDictionary(row => row[0]);
var resultTable = new DataTable();
// Clone the column from table 1
for (int i = 0; i < dataTable1.Columns.Count; i++)
{
var column = dataTable1.Columns[i];
resultTable.Columns.Add(column.ColumnName, column.ColumnType);
}
foreach (var entry in hoursMap1)
{
int hours = entry.Key;
DataRow row1 = entry.Value;
DataRow row2 = null;
if (!hoursMap2.TryGetValue(hours, out row2))
{
// Hours in table 1 but not table 2, handle error
}
var fields = new object[resultTable.Columns.Count];
int fieldIndex = 0;
fields[fieldIndex++] = hours;
for (int i = 1; i < row1.ItemsArray.Length; i++)
{
var field1 = row1.ItemsArray[i];
var field2 = row2.ItemsArray[i];
var newField = field1 ?? field2;
if (newField == null)
{
// Field neither in table 1 or table 2, handle error
}
fields[fieldIndex++] = newField;
}
resultTable.Rows.Add(fields);
}