Having problems with Visual Fox Pro ODBC drivers (and OLE DB) the code I'm using to dynamically get all the column names for a DBF table works fine for small datasets. But if I try getting the table schema using the below code for a large table then it can take 60 seconds to get the schema.
OdbcConnection conn = new OdbcConnection(#"Dsn=Boss;sourcedb=u:\32BITBOSS\DATA;sourcetype=DBF;exclusive=No;backgroundfetch=Yes;collate=Machine;null=Yes;deleted=Yes");
OdbcCommand cmd = new OdbcCommand("Select * from " + listBox1.SelectedItem.ToString(), conn);
conn.Open();
try
{
OdbcDataReader reader = cmd.ExecuteReader();
DataTable dt = reader.GetSchemaTable();
listBox2.Items.Clear();
foreach (DataRow row in dt.Rows)
{
listBox2.Items.Add(row[0].ToString());
}
conn.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
conn.Close();
}
Now I get that this is due to the Select command dragging all 500,000 records or so into the data reader before it can then extract the column names. But visual fox pro is pretty rubbish at limiting the record return.
I ideally want to limit the return to just 1 record, but on the assumption I don't know any of the column names to utilise a WHERE clause this doesn't work.
VFP can utilise the TOP SQL command, but to do that you need an ORDER BY to go with it, and since I don't have the any of the column names to utilise, this doesn't work either.
So am a bit stumped trying to think of a clever way to dynamically get the table schema without the slow down in VFP.
I don't think database conversions on the fly will be any faster. Anyone got any clever ideas (apart from changing the database system, it's inherited in the job and I currently have to work with it :) )?
Change this line:
OdbcCommand cmd = new OdbcCommand("Select * from " + listBox1.SelectedItem.ToString(), conn);
To this line:
OdbcCommand cmd = new OdbcCommand(string.Format("Select * from {0} where 1 = 0", listBox1.SelectedItem.ToString()), conn);
This will bring back zero records, but fill the schema. Now that you have the schema you can build a filter further.
Related
private void btn_view_Click(object sender, EventArgs e)
{
con.Open();
OleDbDataAdapter da = new OleDbDataAdapter("Select * from tbl_emp", con);
DataSet ds = new DataSet();
da.Fill(ds);
dgv_emptable.DataSource = ds.Tables[0];
con.Close();
}
private void btn_insert_Click(object sender, EventArgs e)
{
con.Open();
OleDbCommand cmd = new OleDbCommand();
cmd.CommandText = "Insert into tbl_emp(emp_id,emp_name,emp_surname,designation_id,dept_id) Values(" + txt_id.Text + " , '" + txt_name.Text + "','" + txt_phone.Text + "'," + cmb_desigid.SelectedValue + ",'" + cmb_deptid.SelectedValue.ToString() +"')";
cmd.Connection = con;
cmd.ExecuteNonQuery();
MessageBox.Show("Record inserted");
con.Close();
}
private void Form1_Load(object sender, EventArgs e)
{
con.Open();
OleDbDataAdapter da = new OleDbDataAdapter("Select * from tbl_designation", con);
DataSet ds = new DataSet();
da.Fill(ds);
cmb_desigid.DataSource = ds.Tables[0];
cmb_desigid.DisplayMember = "designation_type";
cmb_desigid.ValueMember = "designation_id";
con.Close();
con.Open();
OleDbDataAdapter db = new OleDbDataAdapter("Select * from tbl_dept",con);
DataSet dm = new DataSet();
db.Fill(dm);
cmb_deptid.DataSource = dm.Tables[0];
cmb_deptid.DisplayMember = "dept_name";
cmb_deptid.ValueMember = "dept_id";
con.Close();
}
I have bound my database and I am writing an insert query to insert data in table but I get the same error at cmd.ExecuteNonQuery
no given parameters are given for required parameters.
I have checked thoroughly but can't seem to find the error
I have used textbox for emp_id,emmp_name,emp_surname,and two combo boxes for designation_id and dept_id.
the dept_id and designation_id are foreign key in tbl_emp. and i also have used the combo box property.So can anyone please tell what the error is and also if i have writtern the combo box code properly...
You need to get in the habit of using "parameterized queries" - those won't just protect your code from the #1 vulnerability out there - SQL injection - they'll also solve a lot of thorny issues with adding quotes etc. to string values.
Try this code:
private void btn_insert_Click(object sender, EventArgs e)
{
// define the insert query - OleDB uses unnamed, positional parameters
string insertQuery = "INSERT INTO tbl_emp (emp_id, emp_name, emp_surname, designation_id, dept_id) " +
"VALUES (?, ?, ?, ?, ?)";
// create command
OleDbCommand cmd = new OleDbCommand(insertQuery, con);
// define parameters - in the proper order! - and set their values
// The "names" like "#emp_id" that I'm using here are just to make it easier for you to grasp which parameter
// corresponds to which columns being inserted - you could also name them "p1", "p2" etc. - not very intuitive, though ...
// Check the *assumptions* I made for the datatypes - not sure if those are
// really what you have - adapt as needed
cmd.Parameters.Add("#emp_id", OleDbType.Int).Value = Convert.ToInt32(txt_id.Text);
cmd.Parameters.Add("#emp_name", OleDbType.VarChar, 100).Value = txt_name.Text;
cmd.Parameters.Add("#emp_surname", OleDbType.VarChar, 100).Value = txt_phone.Text
cmd.Parameters.Add("#designation_id", OleDbType.Int).Value = cmb_desigid.SelectedValue;
cmd.Parameters.Add("#dept_id", OleDbType.Int).Value = cmb_deptid.SelectedValue;
// open connection, execute query, close connection
con.Open();
cmd.ExecuteNonQuery();
con.Close();
MessageBox.Show("Record inserted");
}
As a general side note: if you're only ever interested in a single DataTable being returned from a query - I'd strongly recommend using this code (instead of what you have now):
private void Form1_Load(object sender, EventArgs e)
{
con.Open();
OleDbDataAdapter da = new OleDbDataAdapter("Select * from tbl_designation", con);
// define and use a "DataTable" - not a "DataSet" (which is overkill for just a single table of data)
DataTable dt = new DataTable();
da.Fill(dt);
cmb_desigid.DataSource = dt;
cmb_desigid.DisplayMember = "designation_type";
cmb_desigid.ValueMember = "designation_id";
con.Close();
I mentioned in the comments that you can get VS to do all this for you, in less time, and more securely/reliably than a human could do in a day. Writing db access code is boring and annoying, here's how you hand it off:
add a new dataset to the project, just like you would add a form or class or any other thing. Call it something sensible, not dataset1
open the server explorer window, and add a connection to your access db
drag the db into the dataset. Thoroughly read the long message box that pops up. No one reads this, and they should read it. It solves a lot of confusion later on when the build process is overwriting the database the exe is saving in, and it looks like your app never saves any data. Click yes
drag some tables out of the server explorer and into the dataset. Not the appearance of a datatabke with all the same columns as your db table and a tableadapter. This thing is NOT your database table, it is a strongly typed client side datatable which is a better version of what you're doing in your code above with weakly typed datasets and datatables. A tableadapter is a better version of a dataadapter designed to work with the better datatable it is visually attached to
switch to the forms designed
open the data sources window from the view menu, other windows submenu
drag one of the nodes out of data sources and onto the form
Many things appear, a data grid view, binding source, navigator, dataset, tableadapter, manager. Don't delete stuff until you understand how it all works because it will teach you a lot. Run the program
This app will work, load data, save data and you didn't so far write any code at all. VS wrote all the code for you and you can read it if you want, it's there in the .Designer.cs files on disk
Run the app, add some rows, change stuff, click save, close the app. Don't run the app again yet, but instead go into the bin/debug folder and open that db on there, in access. See your data you added/changed
Now close access and build the project again, now open the same bin/debug db in access.. see the data has gone? The build process copied the blank db from the project over the top of the db the exe altered when it ran. Make sure you grok what is happening here every time you build or you'll be very confused as to why your app "isn't saving" (it is, but the changes are being wiped by the build process)
Some other things you need to know about tableadapters:
they can have more than one select command- just right click them in the dataset designer and add another query. Use parameters, like SELECT * FROM t WHERE id = #id and give the command a sensible name like FillById. The tableadapter will gain a method myTabkeAdapter.FillById(someDatatableHere, 1234) to fill that datatable with row ID 1234
they have an Update method that takes a datatable. This is NOT JUST for running update queries. Update scans the whole passed on datatable looking for rows that need to be inserted updated or deleted and executes the relevant sql. When you change a datatable row, the change is tracked by the RowState property. If the rowstate is Added, and insert will be run by the table adapter, to insert the row. If the rowstate is Modified, an Update will be run. If the rowstate is deleted, a delete will be run. Microsoft should have called Update something else, like Save, because it causes confusion often
I have a sqlite database consist of 50 columns and more than 1.2 million rows. I'm using System.Data.Sqlite to work with Visual Studio 2013 C# language.
I used a very simple code to retrieve my data from database but it is taking too much time.
private SQLiteConnection sqlite;
public MySqlite(string path)
{
sqlite = new SQLiteConnection("Data Source="+path+"\\DBName.sqlite");
}
public DataTable selectQuery(string query)
{
SQLiteDataAdapter ad;
DataTable dt = new DataTable();
try
{
SQLiteCommand cmd;
sqlite.Open();
cmd = sqlite.CreateCommand();
cmd.CommandText = query; //set the passed query
ad = new SQLiteDataAdapter(cmd);
ad.Fill(dt); //fill the datasource
}
catch (SQLiteException ex)
{
//exception code here.
}
sqlite.Close();
return dt;
}
And, the select statement is:
select * from table
As I told you, it is a very simple code.
I want to know how to boost select operation performance to get the appropriate result. for this code the process takes up to 1 minute which I want to get to less than 1 second.
and another thing is that there seems to be some tags for configuring sqlite database but I don't know where to apply them. could some one tell me how to configure sqlite database with System.Data.Sqlite;
Consider narrowing your result set by getting necessary columns or paging.
I have Handheld device that connect to Sql Server database, read the Sql server data and get it on the SQL Compact database that is located on device. This is my code:
public void InsertData() // Function insert data into SQL commapct database
{
dt = new DataTable();
dt = SqlServer_GetData_dt("Select id_price, price, id_item from prices", SqlCeConnection); // get data form sql server
if (dt.Rows.Count > 0)
{
for (int i = 0; i < dt.Rows.Count; i++)
{
string sql = "";
sql = "insert into prices" +
" ( id_prices, price,id_item) values('"
+ dt.Rows[i]["id_price"].ToString().Trim() + "', '"
+ dt.Rows[i]["price"].ToString().Trim() + "', '"
+ dt.Rows[i]["id_item"].ToString().Trim() + "')";
obj.SqlCE_WriteData_bit(sql, connection.ConnectionString);//insert into sql compact
}
}
}
public DataTable SqlServer_GetData_dt(string query, string conn)
{
try
{
DataTable dt = new DataTable();
string SqlCeConnection = conn;
SqlConnection sqlConnection = new SqlConnection(SqlCeConnection);
sqlConnection.Open();
{
SqlDataReader darSQLServer;
SqlCommand cmdCESQLServer = new SqlCommand();
cmdCESQLServer.Connection = sqlConnection;
cmdCESQLServer.CommandType = CommandType.Text;
cmdCESQLServer.CommandText = query;
darSQLServer = cmdCESQLServer.ExecuteReader();
dt.Load(darSQLServer);
sqlConnection.Close();
}
return dt;
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
DataTable dt = new DataTable();
return dt;
}
}
public object SqlCE_WriteData_bit(string query, string conn)
{
try
{
string SqlCeConnection = conn;
SqlCeConnection sqlConnection = new SqlCeConnection(SqlCeConnection);
if (sqlConnection.State == ConnectionState.Closed)
{
sqlConnection.Open();
}
SqlCeCommand cmdCESQLServer = new SqlCeCommand();
cmdCESQLServer.Connection = sqlConnection;
cmdCESQLServer.CommandType = CommandType.Text;
cmdCESQLServer.CommandText = query;
object i = cmdCESQLServer.ExecuteScalar();
sqlConnection.Close();
return i;
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
return 0;
}
}
This is all work fine but the problem is that all this work very slow. I have 20 000 row that's need to be inserted into SQL compact database.
Is there any way for faster insert?
Thanks.
Aside from the obvious poor usage of the Connection for every call, you can greatly improve things by also eliminating the query processor altogether. That means don't use SQL. Instead open the destination table with TableDirect and a SqlCeResultset. The iterate through the source data (a DataTable is a bad idea, but that's a completely different thing) and use a series of CreateRecord, SetValues and Insert.
A pretty good example can be found here (though again, I'd use SetValues to set the entire row, not each individual field).
Reuse your connection and don't create a new connection for every INSERT statement.
Instead of passing a connection string to your SqlCE_WriteData_bit method, create the connection once in the InsertData method, and pass the connection object to SqlCE_WriteData_bit.
Put all the data into a DataTable and then use a SqlCeDataAdapter to save all the data with one call to Update. You may have to fiddle with the UpdateBatchSize to get the best performance.
Looking more closely, I see that you already have a DataTable. Looping through it yourself is therefore ludicrous. Just note that, as you have it, the RowState of every DataRow will be Unchanged, so they will not be inserted. I think that you can call DataTable.Load such that all RowState values are left as Added but, if not, then use a SqlDataAdapter instead, set AcceptChangesDuringFill to false and call Fill.
Is there any way for faster insert?
Yes, but it probably won't be "acceptably fast enough" when we're talking about inserting 20k rows.
The problem I can see is that you are opening a connection for every single row you are retrieving from SqlServer_GetData_dt, that is, you open a connection to insert data 20k times...opening a connection is an expensive transaction. You should build the whole query using a StringBuilder object and then execute all the insert statements in one batch.
This will bring some performance gains but don't expect it to solve you're problem, inserting 20k rows will still take some time, specially if indexes need to be re-built. My suggestion is that you should thoroughly analyse your requirements and be a bit smarter about how you approach it. Options are:
bundle a pre-populated database if possible so your app doesn't have to suffer the population process performance penalties
if not, run the insert process in the background and access the data only when the pre-population is finished
Sorry I know Title is really confusing but I couldn't figure out what exactly to put down.
Basically I created a Grid View which queries database and displays data. It works perfectly, no complain, however what I have right now is,
but what I want is,
Question: I am not sure how can I do this, can someone just point me out in right direction please ?
I think I will going to use nested gridviews.
Try to change your SELECT Query like below... It will you to get the Expected Result...
SQL Fiddle : http://www.sqlfiddle.com/#!3/00b5f/15
I have named the Table as Fruits
SELECT CrateTitle,CrateDescription,CrateID,
stuff(
(
SELECT ','+ [FruitTitle] FROM fruits WHERE CrateID = t.CrateID FOR XML path('')
),1,1,'') Types_of_Fruits_in_Crate
FROM (SELECT DISTINCT CrateTitle,CrateDescription,CrateID FROM fruits )t
OR
CREATE a PROC
*Place this Query in that Proc*
*Call that Proc*
*assign that Result set to GridView*
You can Assign he Stored Proc Result set to GridView by using the Below Code :
DataTable dt = new DataTable();
SqlConnection connection = new SqlConnection("Your Connection String");
try
{
connection.Open();
string spName = "YOURStoredProcudureName";
SqlCommand sqlCmd = new SqlCommand(spName, connection);
SqlDataAdapter sqlDa = new SqlDataAdapter(sqlCmd);
sqlCmd.CommandType = CommandType.StoredProcedure;
sqlDa.Fill(dt);
if (dt.Rows.Count > 0)
{
//display the DataTable to a Data control like GridView for example
GridView1.DataSource = dt;
GridView1.DataBind();
}
}
catch (System.Data.SqlClient.SqlException ex)
{
string msg = "Fetch Error:";
msg += ex.Message;
throw new Exception(msg);
}
finally
{
connection.Close();
}
This is more an sql (or whatever langue your database engine uses) problem than a c# problem although one solution from c# would be (though it may be a bit of extra work) to use a html literal to draw you table at run time
the other option would be to change your sql but without more information i can't say if you could perhaps use a group by on changeID or a pivot table
I am developing a windows mobile app. Right now I am just testing that it can correctly query the local SQL Server CE database. It works fine until I put a WHERE statement in.
Here is my code:
private void buttonStart_Click(object sender, EventArgs e)
{
System.Data.SqlServerCe.SqlCeConnection conn = new System.Data.SqlServerCe.SqlCeConnection(
("Data Source=" + (System.IO.Path.Combine(System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().GetName().CodeBase), "ElectricReading.sdf") + ";Max Database Size=2047")));
try
{
// Connect to the local database
conn.Open();
System.Data.SqlServerCe.SqlCeCommand cmd = conn.CreateCommand();
SqlCeParameter param = new SqlCeParameter();
param.ParameterName = "#Barcode";
param.Value = "%" + textBarcode.Text.Trim() + "%";
// Insert a row
cmd.CommandText = "SELECT * FROM Main2 WHERE Reading LIKE #Barcode";
cmd.Parameters.Add(param);
cmd.ExecuteNonQuery();
DataTable data = new DataTable();
using (SqlCeDataReader reader = cmd.ExecuteReader())
{
if (reader.Read())
{
data.Load(reader);
}
}
if (data != null)
{
this.dataGrid1.DataSource = data;
}
}
finally
{
conn.Close();
}
The database contains this data:
Okay so you can see I changed the WHERE statement to use the Reading column just for testing purposes. When I enter "111" into the textbox and run --> it returns only the row where reading ="1111" and not the row that contains "111".
If I enter "1111" it does not return any data.
If I enter "1" it will return both the "1111" row and the "111" row which is the correct behavior.
However if I enter "11" it once again only returns the "1111" row.
Any other data entry of 2's or 9's attempting to return those rows does not work.
I'm not sure what is going on? This does not make any sense. It is not behaving like I would expect in any way shape or form. I know this must be a little confusing to read. I hope it makes enough sense to get some answers. Please help!
NOTE: I added the "%" before and after the text in an attempt to get better results. This is not desired.
EDIT <<<-----------------------I did have Reading = #Barcode, I just accidently typed Location for this question, that is not the problem.
Firstly, some things to note:
1) As other commentators have noted, use the Reading column, not the Location column. I know you have mentioned you are testing, but swapping around column names and then changing code isn't the easiest way to troubleshoot these things. Try to only change one thing at a time.
2) If Reading is numeric, you are going to have to convert the column value first.
So your query becomes:
"SELECT * FROM Main2 WHERE CONVERT(varchar, Reading) LIKE #Barcode";
Also see How to use parameter with LIKE in Sql Server Compact Edition for more help with working with parameters in SqlServerCE.
3) Set a parameter type on your SqlCEParameter. I've linked to the appropriate page in the code example below.
4) You are using ExecuteNonQuery for no reason. Just get rid of it in this context. It's for when you want to make a change to the database (like an insert, update, delete) or execute something (like a stored proc that can also insert, update, delete etc) that returns no rows. You've probably cut and paste this code from another place in your app :-)
5) Use using on disposable objects (see example below). This will make managing your connection lifecycle much simpler. It's also more readable (IMO) and will take care of issues when exceptions occur.
6) Use the using statement to import the BCL (Base Class Libraries) into your current namespace:
Add the following using statements to the top of your class (.cs). This will make using all of the .Net classes a lot simpler (and is much easier to read and less wear on your keyboard ;-)
using System.Data.SqlServerCe;
using System.IO;
using System.Reflection;
A more complete example would look like the following
private void buttonStart_Click(object sender, EventArgs e)
{
using(SqlCeConnection conn = new SqlCeConnection(
("Data Source=" + (Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().GetName().CodeBase), "ElectricReading.sdf") + ";Max Database Size=2047"))))
{
// Connect to the local database
conn.Open();
using(SqlCeCommand cmd = conn.CreateCommand())
{
SqlCeParameter param = new SqlCeParameter();
param.ParameterName = "#Barcode";
param.DBType = DBType.String; //Intellisense is your friend here but See http://msdn.microsoft.com/en-US/library/system.data.sqlserverce.sqlceparameter.dbtype(v=VS.80).aspx for supported types
param.Value = "%" + textBarcode.Text.Trim() + "%";
// SELECT rows
cmd.CommandText = "SELECT * FROM Main2 WHERE CONVERT(varchar, Reading) LIKE #Barcode";
cmd.Parameters.Add(param);
//cmd.ExecuteNonQuery(); //You don't need this line
DataTable data = new DataTable();
using (SqlCeDataReader reader = cmd.ExecuteReader())
{
data.Load(reader); //SqlCeDataReader does not support the HasRows property.
if(data.Rows.Count > 0)
{
this.dataGrid1.DataSource = data;
}
}
}
}
}
Intellisense should be able to clean up any errors with the above but feel free to ask for more help.
Finally, you also might be able to set the data source of the grid directly to a datareader, try it!
using (SqlCeDataReader reader = cmd.ExecuteReader())
{
dataGrid1.DataSource = reader;
}
You can then get rid of the DataTable.
Change the following line:
cmd.CommandText = "SELECT * FROM Main2 WHERE Location LIKE #Barcode";
to
cmd.CommandText = "SELECT * FROM Main2 WHERE Reading LIKE #Barcode";
You are comparing the wrong columns.