I have a DataTable in C# with columns defined as follows:
DataTable dt = new DataTable();
dt.Columns.Add("OrgName", typeof(string));
dt.Columns.Add("OrgExId", typeof(string));
dt.Columns.Add("UserName", typeof(string));
dt.Columns.Add("UserExId", typeof(string));
dt.Columns.Add("UserEmail", typeof(string));
"UserName", "UserExId", and "UserEmail" are all unique and they are grouped by "OrgName" and "OrgExId"
I want to write a LINQ query to make a new DataTable that contains unique "OrgExId's" and "OrgName's"
This is as far as I got:
var results = from row in dt.AsEnumerable()
group row by row["OrgExId"] into orgs
select orgs;
Specifically in this query, I don't understand how I am supposed to select the rows from the original DataTable. Visual Studio says orgs is of the type `IGrouping, but I have never really seen this type before and am not sure how to manipulate it.
Is this a key value pair?
Sorry about that all. I did not specify my end result.
I want to end up with a DataTable with two columns, distinct "OrgExId" and "OrgName". (There is a one to one relationship between "OrgExId" and "OrgName")
All you really need is a Distinct clause
var output = dt.AsEnumerable()
.Select(x => new {OrgExId = x["OrgExId"], OrgName = x["OrgName"]})
.Distinct();
You can then iterate over this and build a DataTable or whatever you need.
UPDATE: You asked for the output to be a DataTable and the above solution didn't quite sit well with me since it requires extra work. To make this more efficient you could do a custom equality comparer.
Your linq looks like this...
// This returns a DataTable
var output = dt.AsEnumerable()
.Distinct(new OrgExIdEqualityComparer())
.CopyToDataTable();
And your comparer looks like this...
public class OrgExIdEqualityComparer : IEqualityComparer<DataRow>
{
public bool Equals(DataRow x, DataRow y)
{
return x["OrgExId"].Equals(y["OrgExId"]);
}
public int GetHashCode(DataRow obj)
{
return obj["OrgExId"].GetHashCode();
}
}
Use Key property of IGrouping:
var results = from row in dt.AsEnumerable()
group row by new {
row.GetField<string>("OrgExId"),
row.GetField<string>("UserName")
} into orgs
select orgs.Key;
It will give you collection of anonymous types. To get DataTable you can simply iterate over results and add it into DataTable.
DataTable dt = new DataTable();
dt.Columns.Add("OrgName", typeof(string));
dt.Columns.Add("OrgExId", typeof(string));
dt.Columns.Add("UserName", typeof(string));
dt.Columns.Add("UserExId", typeof(string));
dt.Columns.Add("UserEmail", typeof(string));
// put some data for testing purpose
var id = Guid.NewGuid().ToString();
for (var i = 0; i < 10; i++)
dt.Rows.Add(id, i.ToString(), "user_name", Guid.NewGuid().ToString());
var x = dt.Rows.Cast<DataRow>().Select(x => x.Field<string>("UserName")).Distinct();
Console.WriteLine(x);
Related
Suppose I have a call log DataTable where each row represents a call placed with the following columns:
AccountNumber1, AccountNumber2, AccountListDate, AccountDisposition
I want to GroupBy column AccountNumber1 and want a new DataTable with the same columns + 1 additional column NumCalls which will be the count of calls for each AccountNumber1.
New DataTable after GroupBy:
AccountNumber1, AccountNumber2, AccountListDate, AccountDisposition, NumCalls
So far I have the following:
table.AsEnumerable()
.GroupBy(x => x.Field<int>("AccountNumber1"))
.Select(x => new { x.Key.AccountNumber1, NumCalls = x.Count() })
.CopyToDataTable()
Which gives me a DataTable with just two columns AccountNumber1 and NumCalls. How do I get the other columns as I described above?? I would appreciate any help. Thank you.
There's no magic, you need to use a loop and initialize the new table with the new column:
DataTable tblResult = table.Clone();
tblResult.Columns.Add("NumCalls", typeof(int));
var query = table.AsEnumerable().GroupBy(r => r.Field<string>("AccountNumber1"));
foreach (var group in query)
{
DataRow newRow = tblResult.Rows.Add();
DataRow firstOfGroup = group.First();
newRow.SetField<string>("AccountNumber1", group.Key);
newRow.SetField<string>("AccountNumber2", firstOfGroup.Field<string>("AccountNumber2"));
newRow.SetField<DateTime>("AccountListDate", firstOfGroup.Field<DateTime>("AccountListDate"));
newRow.SetField<string>("AccountDisposition", firstOfGroup.Field<string>("AccountDisposition"));
newRow.SetField<int>("NumCalls", group.Count());
}
This takes arbitrary values from the first row of each group which seems to be desired.
I am trying to understand LINQ method notations better.
Say I have a DataTable:
DataTable table = new DataTable("Products");
table.Columns.Add("ProductID", typeof(int));
table.Columns.Add("ProductName", typeof(string));
table.Columns.Add("Category", typeof(string));
table.Columns.Add("UnitPrice", typeof(decimal));
table.Columns.Add("UnitsInStock", typeof(int));
Assume that variable products is being loaded from the DataTable("Products")
var products = testDS.Tables["Products"].AsEnumerable();
So I know I can do queries like:
var productNameGroups = words4.GroupBy(x => x.Field<string>("ProductName").Substring(0, 1)).Select(x => new { FirstLetter = x.Key, Words = x});
var productGroups = products.GroupBy(p => p.Field<string>("Category")).Select(x => new { Category = x.Key, Products = x });
I'm having trouble grasping the x.Key in the Select method. I'm not sure how its set or when I can/cant use it.
The x.Key is specific to processing results of a GroupBy method.
When you do this
var res = someData.GroupBy(item => item.Property);
the result is an IEnumerable of IGrouping<K,V> - key/value pairs, where the Key property represents the value of Property on which the items are grouped.
Since in your case the grouping is done on the string representing the first letter of ProductName or ProductCategory, that is what you get when you reference x.Key in each of the groups returned by the query.
I have a DataTable with 22 columns and one of the columns I have is called "id". I would like to query this column and keep all the distinct values in a list. The table can have between 10 and a million rows.
What is the best method to do this? Currently I am using a for loop to go though the column and compare the values and if the values are the same then the it goes to the next and when not the same it adds the id to the array. But as the table can have 10 to a million rows is there a more efficient way to do this! How would I go about doing this more efficiently?
Method 1:
DataView view = new DataView(table);
DataTable distinctValues = view.ToTable(true, "id");
Method 2:
You will have to create a class matching your datatable column names and then you can use the following extension method to convert Datatable to List
public static List<T> ToList<T>(this DataTable table) where T : new()
{
List<PropertyInfo> properties = typeof(T).GetProperties().ToList();
List<T> result = new List<T>();
foreach (var row in table.Rows)
{
var item = CreateItemFromRow<T>((DataRow)row, properties);
result.Add(item);
}
return result;
}
private static T CreateItemFromRow<T>(DataRow row, List<PropertyInfo> properties) where T : new()
{
T item = new T();
foreach (var property in properties)
{
if (row.Table.Columns.Contains(property.Name))
{
if (row[property.Name] != DBNull.Value)
property.SetValue(item, row[property.Name], null);
}
}
return item;
}
and then you can get distinct from list using
YourList.Select(x => x.Id).Distinct();
Please note that this will return you complete Records and not just ids.
This will retrun you distinct Ids
var distinctIds = datatable.AsEnumerable()
.Select(s=> new {
id = s.Field<string>("id"),
})
.Distinct().ToList();
dt- your data table name
ColumnName- your columnname i.e id
DataView view = new DataView(dt);
DataTable distinctValues = new DataTable();
distinctValues = view.ToTable(true, ColumnName);
All credit to Rajeev Kumar's answer, but I received a list of anonymous type that evaluated to string, which was not as easy to iterate over. Updating the code as below helped to return a List that was more easy to manipulate (or, for example, drop straight into a foreach block).
var distinctIds = datatable.AsEnumerable().Select(row => row.Field<string>("id")).Distinct().ToList();
Try this:
var idColumn="id";
var list = dt.DefaultView
.ToTable(true, idColumn)
.Rows
.Cast<DataRow>()
.Select(row => row[idColumn])
.ToList();
Sorry to post answer for very old thread. my answer may help other in future.
string[] TobeDistinct = {"Name","City","State"};
DataTable dtDistinct = GetDistinctRecords(DTwithDuplicate, TobeDistinct);
//Following function will return Distinct records for Name, City and State column.
public static DataTable GetDistinctRecords(DataTable dt, string[] Columns)
{
DataTable dtUniqRecords = new DataTable();
dtUniqRecords = dt.DefaultView.ToTable(true, Columns);
return dtUniqRecords;
}
Note: Columns[0] is the column on which you want to perform the DISTINCT query and sorting
DataView view = new DataView(DT_InputDataTable);
DataTable distinctValues = new DataTable();
view = new DataView(DT_InputDataTable) { Sort = DT_InputDataTable.Columns[0].ToString() };
distinctValues = view.ToTable(true, DT_InputDataTable.Columns[0].ToString());
I have a DataTable which looks like this:
ID Name DateBirth
.......................
1 aa 1.1.11
2 bb 2.3.11
2 cc 1.2.12
3 cd 2.3.12
Which is the fastest way to remove the rows with the same ID, to get something like this (keep the first occurrence, delete the next ones):
ID Name DateBirth
.......................
1 aa 1.1.11
2 bb 2.3.11
3 cd 2.3.12
I don't want to double pass the table rows, because the row number is big.
I want to use some LinQ if possible, but I guess it will be a big query and I have to use a comparer.
You can use LINQ to DataTable, to distinct based on column ID, you can group by on this column, then do select first:
var result = dt.AsEnumerable()
.GroupBy(r => r.Field<int>("ID"))
.Select(g => g.First())
.CopyToDataTable();
I was solving the same situation and found it quite interesting and would like to share my finding.
If rows are to be distinct based on ALL COLUMNS.
DataTable newDatatable = dt.DefaultView.ToTable(true, "ID", "Name", "DateBirth");
The columns you mention here, only those will be returned back in newDatatable.
If distinct based on one column and column type is int then I would prefer LINQ query.
DataTable newDatatable = dt.AsEnumerable()
.GroupBy(dr => dr.Field<int>("ID"))
.Select(dg => dg).Take(1)
.CopyToDataTable();
If distinct based on one column and column type is string then I would prefer loop.
List<string> toExclude = new List<string>();
for (int i = 0; i < dt.Rows.Count; i++)
{
var idValue = (string)dt.Rows[i]["ID"];
if (toExclude.Contains(idValue))
{
dt.Rows.Remove(dt.Rows[i]);
i--;
}
toExclude.Add(glAccount);
}
Third being my favorite.
I may have answered few things which are not asked in the question. It was done in good intent and with little excitement as well.
Hope it helps.
you can try this
DataTable uniqueCols = dt.DefaultView.ToTable(true, "ID");
Not necessarily the most efficient approach, but maybe the most readable:
table = table.AsEnumerable()
.GroupBy(row => row.Field<int>("ID"))
.Select(rowGroup => rowGroup.First())
.CopyToDataTable();
Linq is also more powerful. For example, if you want to change the logic and not select the first (arbitrary) row of each id-group but the last according to DateBirth:
table = table.AsEnumerable()
.GroupBy(row => row.Field<int>("ID"))
.Select(rowGroup => rowGroup
.OrderByDescending(r => r.Field<DateTime>("DateBirth"))
.First())
.CopyToDataTable();
Get a record count for each ID
var rowsToDelete =
(from row in dataTable.AsEnumerable()
group row by row.ID into g
where g.Count() > 1
Determine which record to keep (don't know your criteria; I will just sort by DoB then Name and keep first record) and select the rest
select g.OrderBy( dr => dr.Field<DateTime>( "DateBirth" ) ).ThenBy( dr => dr.Field<string>( "Name" ) ).Skip(1))
Flatten
.SelectMany( g => g );
Delete rows
rowsToDelete.ForEach( dr => dr.Delete() );
Accept changes
dataTable.AcceptChanges();
Heres a way to achive this,
All you need to use moreLinq library use its function DistinctBy
Code:
protected void Page_Load(object sender, EventArgs e)
{
var DistinctByIdColumn = getDT2().AsEnumerable()
.DistinctBy(
row => new { Id = row["Id"] });
DataTable dtDistinctByIdColumn = DistinctByIdColumn.CopyToDataTable();
}
public DataTable getDT2()
{
DataTable dt = new DataTable();
dt.Columns.Add("Id", typeof(string));
dt.Columns.Add("Name", typeof(string));
dt.Columns.Add("Dob", typeof(string));
dt.Rows.Add("1", "aa","1.1.11");
dt.Rows.Add("2", "bb","2.3.11");
dt.Rows.Add("2", "cc","1.2.12");
dt.Rows.Add("3", "cd","2.3.12");
return dt;
}
OutPut: As what you expected
For moreLinq sample code view my blog
I'm new to LINQ, so I'm sure there's an error in my logic below.
I have a list of objects:
class Characteristic
{
public string Name { get; set; }
public string Value { get; set; }
public bool IsIncluded { get; set; }
}
Using each object in the list, I want to build a query in LINQ that starts with a DataTable, and filters it based on the object values, and yields a DataTable as the result.
My Code so far:
DataTable table = MyTable;
// Also tried: DataTable table = MyTable.Clone();
foreach (Characteristic c in characteristics)
{
if (c.IsIncluded)
{
var q = (from r in table.AsEnumerable()
where r.Field<string>(c.Name) == c.Value
select r);
table = rows.CopyToDataTable();
}
else
{
var q = (from r in table.AsEnumerable()
where r.Field<string>(c.Name) != c.Value
select r);
table = q.CopyToDataTable();
}
}
UPDATE
I was in a panicked hurry and I made a mistake; my DataTable was not empty, I just forgot to bind it to the DataGrid. But also, Henk Holterman pointed out that I was overwriting my result set each iteration, which was a logic error.
Henk's code seems to work the best so far, but I need to do more testing.
Spinon's answer also helped bring clarity to my mind, but his code gave me an error.
I need to try to understand Timwi's code better, but in it's current form, it did not work for me.
NEW CODE
DataTable table = new DataTable();
foreach (Characteristic c in characteristics)
{
EnumerableRowCollection<DataRow> rows = null;
if (c.IsIncluded)
{
rows = (from r in MyTable.AsEnumerable()
where r.Field<string>(c.Name) == c.Value
select r);
}
else
{
rows = (from r in MyTable.AsEnumerable()
where r.Field<string>(c.Name) != c.Value
select r);
}
table.Merge(rows.CopyToDataTable());
}
dataGrid.DataContext = table;
The logic in your posting is wonky; here is my attempt of what I think you are trying to achieve.
DataTable table = MyTable.AsEnumerable()
.Where(r => characteristics.All(c => !c.IsIncluded ||
r.Field<string>(c.Name) == c.Value))
.CopyToDataTable();
If you actually want to use the logic in your posting, change || to ^, but that seems to make little sense.
You overwrite the table variable for each characteristic, so in the end it only holds the results from the last round, and that that apparently is empty.
What you could do is something like:
// untested
var t = q.CopyToDataTable();
table.Merge(t);
And I suspect your query should use MyTable as the source:
var q = (from r in MyTable.AsEnumerable() ...
But that's not entirely clear.
If you are trying to just insert the rows into your table then try calling the CopyToDataTable method this way:
q.CopyToDataTable(table, LoadOption.PreserveChanges);
This way rather than reassigning the table variable you can just update it with the new rows that are to be inserted.
EDIT: Here is an example of what I was talking about:
DataTable table = new DataTable();
table.Columns.Add("Name", typeof(string));
table.Columns.Add("Value", typeof(string));