Develop Reference

Why Inserting records in MS access using C# takes alot of time C#?

My data is composed of 33133 rows and about 45 columns. I am inserting data from datagridview into MS Access database(accdb). I am using local database file.
I tried inserting 33133 rows for just 13 columns and it took literally 15-20 minutes to get into database. I am using high end PC. I can't imagine how much time it will take for 45 columns. I searched around and read various answers regarding bulkcopy using DAO but it doesn't serve my purpose and to be honest the solutions were out of my understanding.
Why its taking so long? Am i doing something wrong? If yes, please guide me.
here is my simple code(I am using query from table adapter):
for (int i = 0; i < dataGridView1.Rows.Count; i++)
{
this.architecture_TowerBTableAdapter.InsertQueryTest(Convert.ToString(dataGridView1.Rows[i].Cells[0].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[1].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[2].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[3].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[4].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[5].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[6].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[7].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[8].Value),
Convert.ToString(dataGridView1.Rows[i].Cells[9].Value),
Convert.ToInt32(dataGridView1.Rows[i].Cells[10].Value),
Convert.ToInt32(dataGridView1.Rows[i].Cells[11].Value),
Convert.ToInt32(dataGridView1.Rows[i].Cells[12].Value));
}
EDIT
In comments, John suggested to try Datatable binding. The datagridview populates with data but database is still empty. Here is my code.
string PathConn = "Provider = Microsoft.Jet.OLEDB.4.0;Data Source = '" + tb_TableBPath.Text + "' ; Extended Properties = \"Excel 8.0; HDR = YES;\";";
OleDbConnection conn = new OleDbConnection(PathConn);
OleDbDataAdapter myDataAdapter = new OleDbDataAdapter("Select * from [" + tb_SheetB.Text + "$]", conn);
DataTable dt = new DataTable();
myDataAdapter.Fill(dt);
dataGridView1.DataSource = dt;
this.architectureTowerBBindingSource.DataSource = dt;
this.architectureTowerBBindingSource.EndEdit();
this.architecture_TowerBTableAdapter.Update(this.pCLiveDB_AykonDataSet.Architecture_TowerB);

The problem is OFTEN that the conneciton is not held open.
You also don't mention if a network of some type is involved (or is this local file??).
But, batch out the row adding to a data table.
You can use say this code approach:
(send out say 1000 rows each time.
// we have a table called rstFrom - 20,000 rows for this test
const int BatchSize = 1000;
int RowCount = 0;
strSQL = "SELECT * FROM tblBigTo WHERE ID = 0";
cmdSQL.CommandText = strSQL;
DataTable rstToTable = new DataTable();
rstToTable.Load(cmdSQL.ExecuteReader()); // load table structure (no reocrds
OleDbDataAdapter da = new OleDbDataAdapter(cmdSQL);
OleDbCommandBuilder daU = new OleDbCommandBuilder(da);
// proces all from rows
Debug.Print("start trans");
DateTime T = DateTime.Now;
foreach (DataRow FromRow in rstFrom.Rows)
{
DataRow MyNewRow = rstToTable.NewRow();
MyNewRow["FirstName"] = FromRow["FirstName"];
MyNewRow["LastName"] = FromRow["LastName"];
MyNewRow["City"] = FromRow["City"];
// etc. more columns
rstToTable.Rows.Add(MyNewRow);
RowCount += 1;
if (RowCount >= BatchSize)
{
da.Update(rstToTable);
rstToTable.Rows.Clear();
RowCount = 0;
Debug.Write("out " + BatchSize.ToString() + Convert.ToString((char)10));
}
}
// write out any rows left in buffer
if (rstToTable.Rows.Count > 0)
{
da.Update(rstToTable);
rstToTable.Rows.Clear();
RowCount = 0;
}
Debug.Print((DateTime.Now - T).Seconds.ToString());
So,, what we do in above, is create a data table. We pull zero rows from target (to get table schema).
We then just loop each "from row". In my example, my from rows are from a source table, but that would be the grid in your case.
If no network is involved, this should not take more then say about 1 minute to run.

how to get the top 10 of the day C#

I have a table in SQL server:
| date_Tim| Machine|Case_wrong|
|:---------|:--------:|:----------:|
|07/03/21 16:53:PM|Test1|1|
|07/03/21 16:58:PM|Test1|1|
|07/03/21 16:59:PM|Test1|1|
|07/03/21 16:58:PM|Test2|1|
|07/03/21 16:59:PM|Test2|1|
|07/03/21 17:00:PM|Test2|1|
|07/03/21 17:01:PM|Test3|1|
|08/03/21 16:58:PM|Test3|1|
|08/03/21 16:58:PM|Test2|1|
I want to sum column machine All machine from date 07/03/22 and fill to chart
I try code
private void loadchart()
{
var Today = DateTime.Now.ToString("dd/MM/yy");
ChartTop.Series[0].Points.Clear();
ChartTop.ChartAreas["ChartArea1"].AxisX.Interval = 1;
string constring = ConfigurationManager.ConnectionStrings["Connstring"].ConnectionString;
SqlConnection con = new SqlConnection(constring);
SqlCommand sqlCmd = new SqlCommand();
sqlCmd.Connection = con;
sqlCmd.CommandType = CommandType.Text;
SqlDataAdapter sqlDataAdap = new SqlDataAdapter(sqlCmd);
try
{
sqlCmd.CommandText = sqlCmd.CommandText = "SELECT TOP 10 Machine, Sum(Case_wrong) as Case_wrong FROM tbl_Count_" + cbWorkcell.Text + " group by Machine order by SUM(Case_wrong)";
DataSet dtRecord = new DataSet();
sqlDataAdap.Fill(dtRecord);
ChartTop.DataSource = dtRecord;
//set the member of the chart data source used to data bind to the X-values of the series
ChartTop.Series["Series1"].XValueMember = "Machine";
//set the member columns of the chart data source used to data bind to the X-values of the series
ChartTop.Series["Series1"].YValueMembers = "Case_wrong";
// ChartTop.Series[0].ChartType = SeriesChartType.Pie;
con.Close();
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
it works but it takes all the data in my table for calculation. is there a way to filter the data by date and sum there?
Please help me! Thanks.

What appears is that date_Tim is a datetime feild , if thats the case then you need a where clause added to your query , I am assuming that the date you mentioned (07/03/22) is DD/MM/YY based on that the query needs to be updated to add the clause as below
"SELECT TOP 10 Machine, Sum(Case_wrong) as Case_wrong FROM tbl_Count_" + cbWorkcell.Text + " WHERE date_Tim >='2022-03-07 00:00:00.000' group by Machine order by SUM(Case_wrong)"
Please note #madreflection has pointed out critical errors, please fix them more details about SQL injection here
Edit 1:
In case you are just looking for a specific day
"SELECT TOP 10 Machine, Sum(Case_wrong) as Case_wrong FROM tbl_Count_" + cbWorkcell.Text + " WHERE CAST(date_Tim AS DATE) ='2022-03-07' group by Machine order by SUM(Case_wrong)"
In case you are looking for the current date
"SELECT TOP 10 Machine, Sum(Case_wrong) as Case_wrong FROM tbl_Count_" + cbWorkcell.Text + " WHERE CAST(date_Tim AS DATE) = GETDATE() group by Machine order by SUM(Case_wrong)"

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version in C#

I am having a problem with my C# code, I put the query in C# code it returns the error as below:
Additional information: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ':= (0 + 1) as numA, ID, UID, File, StartDate, EndDate FROM OEE_PROD.thermalValue' at line 1.
I run the query in MySql database it work and can get the data.
This is my code:
public void exportCSVBtn_Click(object sender, EventArgs e)
{
string conn = ConfigurationManager.ConnectionStrings["constr"].ConnectionString;
using (MySqlConnection con = new MySqlConnection(conn))
{
string a = (string)Session["Userid"];
using (MySqlCommand cmd = new MySqlCommand("select A.*, timediff(B.StartDate, A.EndDate) from (select #rownumA:= (#rownumA + 1) as numA, ID, UID, File, StartDate, EndDate FROM OEE_PROD.thermalValue where File = '" + ddrKdf.SelectedItem.Text + "'AND UserID='" + a + "' order by ID) A LEFT JOIN(select #rownumB:= (#rownumB + 1) as numB, ID as BID, StartDate, EndDate FROM OEE_PROD.thermalValue where File = '" + ddrKdf.SelectedItem.Text + "'AND UserID='" + a + "'order by ID) B ON B.numB = (A.numA + 1)"))
{
using (MySqlDataAdapter sda = new MySqlDataAdapter())
{
cmd.Connection = con;
sda.SelectCommand = cmd;
cmd.Parameters.AddWithValue("#rownumB", 0);
cmd.Parameters.AddWithValue("#rownumA", 0);
using (DataTable dt = new DataTable())
{
sda.Fill(dt);
//Build the CSV file data as a Comma separated string.
string csv = string.Empty;
foreach (DataColumn column in dt.Columns)
{
//Add the Header row for CSV file.
csv += column.ColumnName + ',';
}
//Add new line.
csv += "\r\n";
foreach (DataRow row in dt.Rows)
{
foreach (DataColumn column in dt.Columns)
{
//Add the Data rows.
csv += row[column.ColumnName].ToString().Replace(",", ";") + ',';
}
//Add new line.
csv += "\r\n";
}
//Download the CSV file.
Response.Clear();
Response.Buffer = true;
Response.AddHeader("content-disposition", "attachment;filename=KDFExport_" + DateTime.Now + ".csv");
Response.Charset = "";
Response.ContentType = "application/text";
Response.Output.Write(csv);
Response.Flush();
Response.End();
}
}
}
}
}

The error message is caused by you treating #rownumA and #rownumB mysql user defined variables as a C# query parameter and provide 0 as its value with the following lines:
cmd.Parameters.AddWithValue("#rownumB", 0);
cmd.Parameters.AddWithValue("#rownumA", 0);
This means that #rownumA:= (#rownumA + 1) mysql expression becomes 0:= (0 + 1), which is obviously not correct.
If you want to use sql user defined variables, then add the following parameter to the connection string of your .Net connector connection:
Allow User Variables=True
or
AllowUserVariables=True
This option was added to connector v5.2.2
This way you can remove the parameter assignment lines from your C# code and the mysql variables do not get substituted.
However, the file and userid field values in the where clause indeed should be supplied via parameters and not through string concatenation!

Let's structure this an alternative way: run the two queries separately, download their data in order, then write the CSV using both results. This in contrast to getting mysql to join the data on a fake number created by row order.
using (MySqlDataAdapter daA = new MySqlDataAdapter ("
SELECT ID, UID, File, StartDate, EndDate
FROM OEE_PROD.thermalValue
WHERE File = #file
ORDER BY ID", connstr
))
using (MySqlDataAdapter daB = new MySqlDataAdapter ("
SELECT StartDate
FROM OEE_PROD.thermalValue
WHERE File = #file AND UserID = #userID
ORDER BY ID", connstr
))
{
DataTable dtA = new DataTable();
DataTable dtB = new DataTable();
daA.SelectCommand.Parameters.AddWithValue("#file", ddrKdf.SelectedItem.Text);
daB.SelectCommand.Parameters.AddWithValue("#file", ddrKdf.SelectedItem.Text);
daB.SelectCommand.Parameters.AddWithValue("#userID", a);
daA.Fill(dtA);
daB.Fill(dtB);
for(int i = 0; i < dtA.Rows.Count; i++)
{
string csvTimeDiffCol = "";
if(i+1 < dtB.Rows.Count){
DateTime st = (DateTime)dtA.Rows[i]["StartDate"];
DateTime ed = (DateTime)dtB.Rows[i+1]["EndDate"];
//now you can choose how do you want to represent this timespan?
csvTimeDiffCol = (ed - st).ToString();
}
//now write the row to CSV, including the csvTimeDiffCol string
}
}
Notes:
I've effectively rewritten your code here from what I could guess at your algorithm from what I saw in your code: "get db to prepare a superset and a subset of records, ordered by their ID, join them together on row position with an off-by-one offset" - I personally think this is wonky but I can't fault the logic because it's not mine
We download the two result sets then step over them in order, taking relevant rows
I've no idea what your timediff or dates looks like; I've assumed that they're Dates and you're looking for the difference in hours minutes etc between them, hence I cast them to DateTime in the c# and then used x-y to turn them into a TimeSpan; you'll probably need to format this
I haven't written your csv code in, because that part is unchanged. There are libraries to help with that though; you don't need to roll your own
This algorithm may not be perfect/may need debugging. I don't present it as a "paste this and I did your work for you" - I'm presenting it as something to get you to think about the problem another way

Bulk Update in SQL Server from C#

I have to update multiple records in SQL Server table from C#. Below are the steps I have to follow and below is the code.
The code is working but the process is taking much longer than expected.
I need a quick way to update 10000 records, not sure whether Bulk Copy would work for Update.
I have seen the other answers which has Bulk insert to temp and then update..But that update has a single statement and here I need to update the records in DB based on Excel data and for this I have to loop each excel record.So how can I achieve faster update.
1) Read the Excel Data and copied the data into a data table
string strDirectory = string. Empty;
strDirectory = System.IO.Directory.GetCurrentDirectory() + "\\" + "Filename.xlsx";
string Connection String = "Provider=Microsoft.ACE.OLEDB.12.0; Data Source = " + strDirectory + "; Extended Properties = \"Excel 12.0;HDR=YES;IMEX=1\"";
using (OleDbConnection conn = new OleDbConnection(Connection String))
{
conn.Open();
DataTable schemaTable = conn.GetOleDbSchemaTableOleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
DataRow schemaRow = schemaTable. Rows[0];
string sheet = schemaRow["TABLE_NAME"].ToString();
string query = "SELECT * FROM [" + sheet + "]";
OleDbDataAdapter daexcel = new OleDbDataAdapter(query, conn);
daexcel.Fill(dt);
conn.Close();
}
2) Doing some manipulations on datatable data before updating into table.
string strsqlst = string. Empty;
using (SqlConnection sqlConn = new SqlConnection(Connectionstring))
{
sqlConn.Open();
SqlCommand cmd;
StringBuilder sb = new StringBuilder();
sb.AppendLine("DataTable content:");
foreach (DataRow row in dt.Rows)
{
if (row.ItemArray[0].ToString() == "")
break;
strsqlst = "Update table Set col1= " + row.ItemArray[4].ToString() + " ,col2= " + row.ItemArray[5].ToString() + " where <Condition>'";
cmd = new SqlCommand(strsqlst, sqlConn);
cmd.CommandType = CommandType.Text;
cmd.ExecuteNonQuery();
}
sqlConn.Close();
}

The SqlCommand can be a whole SQL batch and is not limited to a single statement. So you can create a single large batch with 10,000 UPDATE statements, or divide it into for example 20 batches of 500 each.
In other words, you can create a single command with CommandText like this:
UPDATE [T] SET Col1='Value1', Col2='Value2' WHERE [Id] = 1;
...
UPDATE [T] SET Col1='Value999', Col2='Value1000' WHERE [Id] = 500;
That said, you should use parameters for all data values (to ensure SQL injection is not possible).
If you want to handle any errors (updates failing due to invalid data) you will need something a bit more sophisticated.

Getting multiple records from SQL Server database with C#

Can someone tell me why this is not working, please? This will show me the first database record only that matches the WHERE statement and I need it to display all of them that match. Everything works except it only shows the first record that matches. Thank you in advance.
using (connection = new SqlConnection(connectionString))
using (SqlDataAdapter adapter = new SqlDataAdapter("SELECT * FROM members WHERE Month = '" + currentMonth + "' AND Day = '" + currentDay +"'", connection))
{
DataTable membersTable = new DataTable();
DataSet info = new DataSet();
adapter.Fill(membersTable);
adapter.Fill(info);
var rows = info.Tables[0].Rows;
foreach (DataRow row in rows)
{
nameEmployee = info.Tables[0].Rows[0]["Name"].ToString();
MessageBox.Show (nameemployee);
}
}

See the following line:
nameEmployee = info.Tables[0].Rows[0]["Name"].ToString();
You are within a foreach loop and have the "current" row in the row variable. That line should read:
nameEmployee = row["Name"].ToString();