Develop Reference

Search DataTable in C#

I have a DataTable with 36000 columns minimum, 20000 rows min. I have a hardcoded list of column names which I need to find in the datatable and perform some calculation on the values for each column.
e.g Sum of all rows for a particular column
I was thinking of using a simple foreach to find the column name and datatable.compute method to perform my calculation.
Is there a better way of achieving this? Any help is greatly appreciated.
Thank you.
I am using .Net 4.6 and VS2015

You can compare the performance difference between the Datatable.Compute and Foreach.
// Datatable.Compute sample
DataTable table;
table = dataSet.Tables["Orders"];
// Declare an object variable.
object sumObject;
sumObject = table.Compute("Sum(Total)", "EmpID = 5");
Below is a sample of foreach
// Foreach through DataTable
double sum = 0;
dt = ds.Tables["Orders"];
foreach (DataRow dr in dt.Rows)
{
sum += System.Convert.ToDouble(dr["Total"]);
}

For more performance its good to use parallel methods.
This link has a sample for you to use parallel and it is very Good to use it every where is possible.
At this example you see how much will improve your speed.

Querying inside a Dataset C#

I have an ADO.NET dataset which is set by a certain query,
say
SELECT ID,USER,PRODUCT,COUNT FROM PRODUCTION
Without using a where clause I need to derive some results from the dataset. Say I want to get the User and Product count of the user who has the maximum product count. (And I want to do it by using the existing dataset. I can't derive this from dataset.)
Any idea of a way to query inside the dataset? Since there are Datatables my thought was there is some way to query it.

Traditional SQL queries cannot be applied to the DataSet. The following is possible, however:
Filter rows using DataTable.Select. See here for detailed information about expressions in DataTables.
Calculate totals etc. using DataTable.Compute.
If these two don't do the trick, there's always LINQ.
Quick-and-dirty LINQ example: (which doesn't return a DataTable, but a list containing an anonymous type):
var joinedResult = dataTable1
// filtering:
.Select("MyColumn = 'value'")
// joining tables:
.Join(
dataTable2.AsEnumerable(),
row => row.Field<long>("PrimaryKeyField"),
row => row.Field<long?>("ForeignKeyField"),
// selecting a custom result:
(row1, row2) => new { AnotherColumn = row1.Field<string>("AnotherColumn") });
AsEnumerable converts a DataTable into an IEnumerable on which LINQ queries can be performed. If you are new to LINQ, check out this introduction.

Yes, you can use DataTable.Select method.
DataTable table = DataSet1.Tables["Orders"];
// Presuming the DataTable has a column named Date.
string expression;
expression = "Date > #1/1/00#";
DataRow[] foundRows;
// Use the Select method to find all rows matching the filter.
foundRows = table.Select(expression);
// Print column 0 of each returned row.
for(int i = 0; i < foundRows.Length; i ++)
{
Console.WriteLine(foundRows[i][0]);
}
Also see this link.

You can do cross-table queries of a Dataset object using LINQ to DataSet:
msdn.microsoft.com/en-us/library/bb386969.aspx

Problem with DataTable.Select

Is there a way to find a value in DataTable use method Select:
items.Select("Number like '%10%'");
But Number type is Int, and this code doesn`t work...

Look at the T-SQL CAST and CONVERT functions, they might help you here. Basically, in your query, use either CAST or CONVERT (not sure which is best suited for you) to change the int into a string that you can perform a LIKE on.

For those reading in the future, this might help:
DataRow[] matchingRows = a.Select("Convert(Age, 'System.String') like '2%' "); // cut and paste from my code
So in your code it would be:
DataRow[] matchingRows = a.Select("Convert(Number, 'System.String') like '10%' "); // where Number is the name of your column.

If you really want to have such behaviour, then it would be better just add to your table additional computed column like "NumberText" with type String, which will store the same values as for column Number. Then you can execute your expression: items.Select("NumberText like '%10%'");

You can use linq the following way:
var matchingRows =
from item in items.AsEnumerable()
where item.Field<string>("Number").Contains("10")
select item;
or if Number is actually a number:
var matchingRows =
from item in items.AsEnumerable()
where item.Field<double>("Number").ToString().Contains("10")
select item;

If memory serves you need to escape a single quote by doubling it. Ie:
items.Select("Number like ''%10%''");

You can use DataView to filter values in DataTable
DataView dv = new DataView(dtSample);
dv.RowFilter = "Number like '%10%'";

C# DataTable LINQ & GROUP BY

I have a DataTable with 20 columns (I only need 3 of them.) I need to perform the following query on it and then save the results as an array. I've done some searching, but I can't figure out how to perform the mathematical operation. I know LINQ should be used, but I'm not getting anywhere. Any help is greatly appreciated!
SELECT DISTINCT columnZ, (columnX + columnY) / 2 FROM DataTable
*EDIT - corrected SQL statement

Answering your last comment (I suggest you update the question):
var result =
(from row in dataTable.AsEnumerable()
let average = ((double)row["columnX"] + (double)row["columnY"])/2
select new
{
ColumnZ = (string)row["columnZ"],
Average = average
}).Distinct();
Use your actual data types.

C#: retrieve the first n records from a DataTable

I have a DataTable that contains 2000 records.
How would you retrieve the first 100 records in the DataTable?

If it implements IEnumerable<T>:
var first100 = table.Take(100);
If the type in question only implements IEnumerable, you can use the Cast extention method:
var first100 = table.Cast<Foo>().Take(100);

This works for DB2.
select * from table
fetch first 100 rows only;

and for mysql: select * from table limit 100

You could use something like this, but restrict the foreach loop to 100 records.

And to make the list full, here is the statement for MS SQL:
Select top 5 * from MyTable2
And some other methods with MS SQL can be found here.

To get a list of the top n records in C# using the 2.0 framework:
DataTable dt = new DataTable();
var myRows = new List<DataRow>();
//no sorting specified; take straight from the top.
for (int i = 0; i < 100; i++)
{
myRows.Add(dt.Rows[i]);
}