I have a DataTable with about 50,000 rows that I'm using DataTable.Select to retrieve rows from. The Select requires multiple AND conditions including a wildcard match. I've played around and found that by doing the same Select in multiple steps, the execution time can be greatly reduced, but changing the order of the AND statements doesn't affect it.
//This takes ~ 750 ms
DataRow[] results = myDataTable.Select("Field1 LIKE '*" + term1 + "*'" +
"AND Field2 = '" + term2 + "'" +
"AND Field3 = '" + term3 + "'");
//This also takes ~750 ms
DataRow[] results2 = myDataTable.Select("Field3 = '" + term3 + "'" +
"AND Field2 = '" + term2 + "'" +
"AND Field1 LIKE '*" + term1 + "*'");
//This takes 0.415 ms
DataTable table1 = myDataTable.Select("Field3 = '" + term3+ "'").CopyToDataTable();
DataTable table2 = table1.Select("Field2 = '" + term2 + "'").CopyToDataTable();
DataRow [] results3 = table2.Select("Field1 LIKE '*" + term1 + "*'");
My question is, is there a way to always make the SELECT operation evaluate AND conditions in a left-to-right order so that number of records searched would be reduced between steps? It would seem this could be a good time saver. Thank you for your ideas.
You could use Linq (note that the slowest condition is the last):
IEnumerable<DataRow> rows = myDataTable.AsEnumerable()
.Where(r => r.Field<string>("Field2") == term2
&& r.Field<string>("Field3") == term3
&& r.Field<string>("Field1").Contains(term1));
Use CopyToDataTable if you want to create a new DataTable from the result, ToArray to create a DataRow[] or leave it and use foreach to enumerate the result without creating a new collection.
Nope.. a datatable does a select row by row.
You can write your code a bit shorter:
DataRow [] results = myDataTable
.Select("Field3 = '" + term3+ "'").CopyToDataTable()
.Select("Field2 = '" + term2 + "'").CopyToDataTable()
.Select("Field1 LIKE '*" + term1 + "*'");
Or... if you make an extra extension method, like this:
static public DataRow[] Select(this IEnumerable rows, string filter)
{
return rows.CopyToDataTable().Select(filter);
}
Using this extension makes your code even shorter:
DataRow [] results = myDataTable
.Select("Field3 = '" + term3+ "'")
.Select("Field2 = '" + term2 + "'")
.Select("Field1 LIKE '*" + term1 + "*'");
Or:
static public DataRow[] Select(this DataTable dt, string firstFilter, params string[] filters)
{
DataRow[] result = dt.Select(firstFilter);
foreach(string filter in filters)
result = result.CopyToDataTable().Select(filter);
return result;
}
Opening up the way to:
DataRow [] results = myDataTable.Select(
"Field3 = '" + term3+ "'",
"Field2 = '" + term2 + "'",
"Field1 LIKE '*" + term1 + "*'");
Related
I have this row filter text:
"[add1] = '" + startAddress+ "%" + middle+ "%" + end +"'"
which fails, but if I put the % at the beginning or end it's OK.
Is there any way to achieve the same result (i.e. "any" string in the middle of the names)?
Full statement is:
dv = new DataView(MyDataTable, "[add1] = '" + startAddress+ "%" + middle+ "%" + end + + "'", "", DataViewRowState.CurrentRows);
Here i want filteration like "%something%someone%" How can i achieve this? Even using linq is also ok with me
I have this row filter text: "[Name 1] = '" + forename + "%" + surname + "'" which fails, but if I put the % at the beginning or end it's OK. Is there any way to achieve the same result (i.e. "any" string in the middle of the names)?
Full statement is:
dv = new DataView(MyDataTable,
"[Name 1] = '" + forename + "%" + surname + "'",
"", DataViewRowState.CurrentRows);
Just a free thougth, try:
dv = new DataView(MyDataTable,
"[Name 1] = '" + forename + "%' AND [Name 1] = '%" + surname + "'",
"", DataViewRowState.CurrentRows);
EDIT: some documentation:
"A wildcard is allowed at the start and end of a pattern, or at the end of a pattern, or at the start of a pattern. [...] Wildcard characters are not allowed in the middle of a string. For example, 'te*xt' is not allowed."
My guess would be it's for performance reasons (?)
I have data set that is being filled from sql query, like this
cmd_sql.CommandText = " SELECT BrDok " +
" FROM ordersstavke " +
" WHERE SifParFil = '" + rw_mat["sifskl_kor"] + "'";
MySqlDataAdapter sql_adapter = new MySqlDataAdapter(cmd_sql);
DataSet ds_dok = new DataSet("ordersstavke");
sql_adapter.Fill(ds_dok);
Now I want to extract value from data set for sql update, like this one
myQuery = "UPDATE ordersstavke " +
"SET BrDok = '" + rw_mat["brdok"] + "', " +
"SifParFil = '" + rw_mat["sifskl_kor"] + "', " +
"WHERE BrDok = " + ds_dok.Tables["ordersstavke"].Rows[0]["BrDok"] + "'";
I tried this ds_dok.Tables["ordersstavke"].Rows[0]["BrDok"] but I got an error,
I was thinking to do something like this
string BrDok;
BrDok = ds_dok.["BrDok"].ToString();
But nothing, how to extract that BrDok or just put it into procedure?
Thanks infront!
Make it
DataSet ds_dok = new DataSet("ordersstavke");
sql_adapter.Fill(ds_dok,"BrDok");
Then use
ds_dok.Tables["BrDok"].Rows[0]["BrDok"].ToString()
Try this
ds_dok.Tables[0].Rows[0]["BrDok"]
If you provide a string argument for the dataset class, then it will be the dataset name and not the datatable name. And there is no table in the database with name you provided for a dataset, so give it while filling the dataset. Write some thing like below.
DataSet ds_dok = new DataSet();
sql_adapter.Fill(ds_dok,"ordersstavke");
and you can write all the remaining code as it is in your code part.
And your second update query has some syntax error, see it like below
myQuery = "UPDATE ordersstavke " + "SET BrDok = '" + rw_mat["brdok"] + "', "
+ "SifParFil = '" + rw_mat["sifskl_kor"] + "', " + "WHERE BrDok
= '" + ds_dok.Tables["ordersstavke"].Rows[0]["BrDok"] + "'";
You forgot to put an starting inverted comma at the where clause.
Just a small hint to the sql-command. You should use sql-parameters to prefent sql-injection.
I have to update table on SQL Server but first i have to check for existing data in table so if data is there update it, if not make a new insert:
cmd_sql.CommandText = " SELECT BrDok as id_dok " +
" FROM ordersstavke " +
" WHERE SifParFil = '" + rw_mat["sifskl_kor"] + "'" +
" AND DokumentTip = '" + rw_mat["vrst_dok"] + "'";
MySqlDataAdapter sql_adapter = new MySqlDataAdapter(cmd_sql);
DataSet dt_dok = new DataSet("DOK_MAT_EXCHANGE");
sql_adapter.Fill(dt_dok);
if (dt_dok.Tables["DOK_MAT_EXCHANGE"].Rows.Count == 0)
{
myQuery = " INSERT INTO ordersstavke (BrDok, DocumentTip, SifParFil) " +
" VALUES ('" + rw_mat["brdok"] + "', '" +
rw_mat["vrst_dok"] + "', '" +
rw_mat["sifskl_kor"] + "')";
}
else
{
UPDATE DATA
}
But I have an error in the code, the error is here if (dt_dok.Tables["DOK_MAT_EXCHANGE"].Rows.Count == 0)
Object reference not set to an instance of an object.
The problem is in this if statement...
DOK_MAT_EXCHANGE is the name of the DataSet, not of the first table.
You should test with
if (dt_dok.Tables[0].Rows.Count == 0)
Also, I think is better to use a syntax like this to discover how many records are presents
cmd_sql.CommandText = "SELECT COUNT(BrDok) as id_dok " +
" FROM ordersstavke " +
" WHERE SifParFil = ?p1 " +
" AND DokumentTip = ?p2";
cmd_sql.Parameters.AddWithValue("?p1", rw_mat["sifskl_kor"] );
cmd_sql.Parameters.AddWithValue("?p2", rw_mat["vrst_dok"] );
int rowCount = (Int32)cmd_sql.ExecuteScalar();
change
DataSet dt_dok = new DataSet("DOK_MAT_EXCHANGE");
to
DataSet dt_dok = new DataSet("ordersstavke ");
and
if (dt_dok.Tables["DOK_MAT_EXCHANGE"].Rows.Count == 0)
to
if (dt_dok.Tables["ordersstavke "].Rows.Count == 0)
Accessing the first table via the dataset name is incorrect, that's for setting the XML.
Instead use
dt_dok.Tables[0].Rows.Count;
That being said, you're better off writing this as a single SQL statement instead of a separate select && insert. This way you're not going to the DB multiple times.
var sql = #"if exists(select * from ordersstavke where SifParFil = ? and DokumentTip = ?)
then
-- do insert statement
else
-- do update
end if";
This might also be better done with a stored proc, so you don't have as much SQL code in C#. It's easier to manage multiple operations that way.
And for crying out loud, use SqlParameters, not string concatenation! That's just asking for trouble!
Ok, thanks guys, I wrote it like this
if (ds_dok.Tables[0].Rows.Count <= 0)
{
myQuery = " INSERT INTO ordersstavke (BrDok, " +
" SifParFil) " +
" VALUES ('" + rw_mat["brdok"] + "', '" +
rw_mat["sifskl_kor"] + "')";
}
else if (ds_dok.Tables[0].Rows.Count >= 1)
{
myQuery = "UPDATE ordersstavke " +
"SET BrDok = '" + rw_mat["brdok"] + "', " +
"SifParFil = '" + rw_mat["sifskl_kor"] + "', " +
"WHERE BrDok = " + ds_dok.Tables["ordersstavke"].Rows[0]["BrDok"].ToString() + "'";
}
But there is a small problem in the section update: s_dok.Tables["ordersstavke"].Rows[0]["BrDok"].ToString(), here it give me that loving error : Object reference not set to an instance of an object.
Maybe the update on MySQL goes on different way, I'm referencing on example on sql server and there update goes differently
Okay, so in the past few weeks I've probably written about 40 select statements. So, I know how to do it. And I've just written another one, but this time I need to use ComboBox values to match against, and it keeps resulting in the names of the column (the right column, mind you), instead of what's inside the column.
string st = "SELECT '" + txtchange.Text + "'
FROM mysql_9269_dbase." + pages.Text + "";
MySql.Data.MySqlClient.MySqlCommand cd = new MySql.Data.MySqlClient.MySqlCommand(st, msc);
cd.CommandType = CommandType.Text;
MySql.Data.MySqlClient.MySqlDataReader msdr = cd.ExecuteReader();
while(msdr.Read())
{
txt.Text = msdr[0].ToString();
}
Now, why is it returning the column name instead of the content of that column?
Lose the single quotes.
Change
"SELECT '" + txtchange.Text + "' "
to
"SELECT " + txtchange.Text + " "
In sql you can do it like this.
string query = "Execute("+"'SELECT " + txtchange.Text + " FROM mysql_9269_dbase." + pages.Text + "')";