Develop Reference

C# - How To Loop Through A Table To Update Each Row MySQL

I'm working on a project where I have to update a table in MySql by looping through it and update every row (one by one) in C#. NOTE: I have to update the value's of only one column in my table.
Now, I know this question has been asked (many times) before but after spending a lot of time searching around the internet I haven't found a solution for my problem.
More Explanation:
So I already know a few things that are necessary to make this work:
I know how to establish a (working) SSH connection.
I know how to establish a (working) MySql connection to my database.
I know how to run/execute a SELECT and UPDATE Query/Command in C#.
So the only thing that I need to make this whole thing work, is the loop itself.
Now, during my research I came across some answers that suggested using a Reader. I don't know if this is what I need and (if yes) how to use it correctly.
Below is the code that I have so far (for establishing the connections and executing query's).
My Code:
class ReaderDemo1
{
public static void Update()
{
Console.WriteLine("[Core] Opening Connection To Database...");
Database.openStockConn(Settings.databaseName, Settings.databaseUsername, Settings.databasePassword, Settings.sshHost, Settings.sshUsername, Settings.sshPassword);
if (Database.stockConn.State != ConnectionState.Open)
{
Database.openStockConn(Settings.databaseName, Settings.databaseUsername, Settings.databasePassword, Settings.sshHost, Settings.sshUsername, Settings.sshPassword);
}
Console.WriteLine("[Core] database connection is now open!\n");
MySqlCommand cmd = new MySqlCommand("SELECT value FROM catalog_product_entity_decimal", Database.stockConn);
try
{
Console.WriteLine("[Price] Updating Prices...");
MySqlCommand command = new MySqlCommand("UPDATE catalog_product_entity_decimal SET value= 1112 WHERE value_id= 4063", Database.stockConn);
command.ExecuteNonQuery();
Console.WriteLine("[Price] Prices Have Been Updated!");
}
catch
{
Console.WriteLine("Updating Failed!");
}
finally
{
if (Database.stockConn != null)
{
Database.stockConn.Close();
}
}
}
}
Just to give some extra context:
Database.cs is where I create the connections (SSH and MySql)
Settings.cs is where I have all the Login data for the SSH and MySql connections.
If you guys would want to know, I have already attempted a few things myself regarding the loop but (as I already mentioned) these attempts weren't successful.
My Attempt With Reader:
using (var reader = command.ExecuteReader())
{
var indexOfValue = reader.GetOrdinal("value");
while (reader.Read())
{
var price1 = reader.GetValue(indexOfValue);
Console.WriteLine("Executing Update Command...");
MySqlCommand cmd = new MySqlCommand("UPDATE catalog_product_entity_decimal SET value= 1222 WHERE entity_id= 759 AND entity_id= 839 AND entity_id= 881", con);
cmd.ExecuteNonQuery();
Console.WriteLine("Update Command Executed!");
}
}
As said, the code above didn't work how I wanted it to work (probably because it isn't actually doing anything now). Just to let you guys know, I am not stuck at an error. I'm just stuck on how to do this.
I hope one of you guys can help me with this or point me in the rigth direction and if you think that my question is a duplicate of another one, just tell me and I will look into it! :) Thanks in advance.
Kind regards,
LKS
EDIT:
In case you guys wanted to know, this is what my table looks like.
The table contains about 5600 rows, so these are just the top rows.

Like Frederiks answer, the thing that you are trying to do now is Looping over an table and execute the same Query over and over again.
Even if you manage to get your code working it would be very slow (because you have about 5600 rows to update).
So my suggestion is that you create an new table with the new value's in it (so the one's you wanted to have after your loop). Then just run a single update command to update your old table with values from your new table.
This option probably takes a few seconds and it will be done, so its much faster! :)
The Query you need should look something like this:
UPDATE old_table
INNER JOIN new_table
USING (column) --> // if you want to update a specific column
EDIT:
In addition/ update to my answer, this is how you can update your table more accurate:
UPDATE old_table
INNER JOIN new_table ON old_table.value_id = new_table.value_id // Use this to set your columns with unique values's
SET old_table.value = new_table.value // For the column with the value's you want to update
So, in the above code you update your old_table with the value's from your new_table. In this example you only update the value's from only one column (which you wanted).
You can expand the query for a different result.

I cannot think of any reason that you would want to read one line at a time and write one line at a time in a loop, but this is how you could do it.
Use one SqlCommand to read and a separate SqlCommand to update. Change the parameters in the loop for each update.
public static void Update()
{
Database.openStockConn(Settings.databaseName, Settings.databaseUsername, Settings.databasePassword, Settings.sshHost, Settings.sshUsername, Settings.sshPassword);
if (Database.stockConn.State != ConnectionState.Open)
{
Database.openStockConn(Settings.databaseName, Settings.databaseUsername, Settings.databasePassword, Settings.sshHost, Settings.sshUsername, Settings.sshPassword);
}
Console.WriteLine("[Core] database connection is now open!\n");
MySqlCommand cmd1 = new MySqlCommand("SELECT value FROM catalog_product_entity_decimal", Database.stockConn);
MySqlCommand cmd2 = new MySqlCommand("UPDATE catalog_product_entity_decimal SET value= 1112 WHERE value_id= 4063", Database.stockConn);
try
{
Console.WriteLine("[Price] Updating Prices...");
using (var reader = cmd1.ExecuteReader())
{
var indexOfValue = reader.GetOrdinal("value");
while (reader.Read())
{
var price1 = reader.GetValue(indexOfValue);
Console.WriteLine("Executing Update Command...");
cmd2.ExecuteNonQuery();
Console.WriteLine("Update Command Executed!");
}
}
}
catch
{
Console.WriteLine("Updating Failed!");
}
finally
{
if (Database.stockConn != null)
{
Database.stockConn.Close();
}
}
}

SQL is set based, which means that there is almost always a better approach then looping over a table.
That is also the case with your problem. You want to avoid looping over an entire table to update record-by-record, since this is terribly slow.
It is not obvious to me on what you're trying to achieve, since your code sample loops over a table and then inside your loop, you execute the same statement over and over again, but you're always updating the exact same records ?
If you want to update records in a table with information coming from another table, you might want to have a look at the UPDATE statement together with the JOIN clause as explained here.

You can try this change , by having the count of rows and then updating the records.
MySqlCommand cmd = new MySqlCommand("SELECT count(*) FROM catalog_product_entity_decimal", Database.stockConn);
try
{
var reader = cmd .ExecuteReader();
if(reader.Read()){
var totalRows=reader.GetInt32(0);
Console.WriteLine("[Price] Updating Prices...");
while(totalRows-->0){
MySqlCommand command = new MySqlCommand("UPDATE catalog_product_entity_decimal SET value= 1112 WHERE value_id= 4063", Database.stockConn);
command.ExecuteNonQuery();
}
Console.WriteLine("[Price] Prices Have Been Updated!");
}
}
catch
{
Console.WriteLine("Updating Failed!");
}
finally
{
if (Database.stockConn != null)
{
Database.stockConn.Close();
}
}

Procedure Returning Blank Data Table from ASP.NET

I'm having issues with a stored procedure I'm attempting to run from an ASP.NET console application.
This is how I'm attempting to get the results in the code:
public static DataTable PerformStoredProcedure(int PracticeID, string CommandName)
{
DataTable dt = new DataTable();
try
{
using (SqlConnection conn = new SqlConnection(System.Configuration.ConfigurationManager.AppSettings["DbConnString"].ToString()))
{
conn.Open();
using (SqlCommand cmd = conn.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = CommandName;
cmd.Parameters.AddWithValue("#practiceID", PracticeID);
using (SqlDataAdapter da = new SqlDataAdapter(cmd))
{
da.Fill(dt);
}
}
for (int i = 0; i < dt.Rows.Count; i++)
{
for (int j = 0; j < dt.Rows[i].ItemArray.Length; j++)
{
Console.Write(dt.Rows[i].ItemArray[j] + ",");
}
Console.WriteLine();
}
}
}
After the procedure is run I get returned with a blank data set with no errors. However, when I run the procedure in SQL Server Management Studio I works just fine.
You may notice I don't use the new SqlCommand() constructor when initiating the SqlCommand cmd. That is because the query is enormous and takes a while to run so I figured it was a timeout issue.
Following the suggestion here, I made the following class in order to adjust the execution timeout limit.
class CommandFactory
{
public static int CommandTimeout
{
get
{
int commandTimeout = 0;
var configValue = ConfigurationManager.AppSettings["commandTimeout"];
if (int.TryParse(configValue, out commandTimeout))
return commandTimeout;
return 120;
}
}
Figured I'd include that just to avoid confusion about whether the SqlCommand was being initialized correctly. After that adjustment, I began getting the issue where the command returns a blank data table. I assumed it could have to do with the minimum capacity of the data table dt, so I adjusted that as you can see in the original code.
I've done additional research and made the suggested fixes that are suggested to no avail:
1. Set nocount on
2. Use data adapter instead of data reader
3. This is the most detailed answer. There are several suggestions made with in the forum but the eventual resolution has to do with views being used by the database as a default and blocking access to desired data. I have already confirmed with our DBA that we do not do this.
Like I mentioned before, the query encompassed in the stored procedure is enormous and I haven't included it here because of its sheer size and because this post is long enough as it is. Again, I did confirm that the procedure is working in SSMS.
I also haven't included an example result set because the data contains restricted information.
I appreciate any help you can give.
UPDATE:
I added another method that calls a stored procedure to see if the problem was particular to the particular procedure being called. That second stored procedure also returned a blank data table so the problem is not specific to either procedure. (I also confirmed the second procedure was working and returning rows in SSMS)
That led me to believe there was something wrong with the app.config file for that project. I then added a third method that called a stored procedure on another project in the solution. However, even that returned a blank data table.
Figured I'd provide clues as I happen upon them.

You're quite close. Here are a few pointers:
The SqlConnection is never opened. You need to call .Open() in order
for the connection to be established.
If you set
a breakpoint on the .Fill() line and hover over the DataTable
object, it will appear as if the DataTable is empty (as you've seen), but .Fill() actually fills the Rows of the DataTable for you
automatically (see MSDN). I bet that if you loop through the Rows, you'll find
that your query has actually been returning data all along.
You could use using statements which automatically call .Dispose() for you as well as close the SqlConnection.
Here's an updated version of your method:
public static DataTable getAllModifiedChargesToday(int practiceID)
{
DataTable dt = new DataTable();
try
{
using (SqlConnection conn = new SqlConnection(System.Configuration.ConfigurationManager.AppSettings["DbConnString"].ToString()))
{
conn.Open();
using (SqlCommand cmd = conn.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "ch_getAllModifiedChargesToday";
cmd.Parameters.AddWithValue("#practiceID", practiceID);
using (SqlDataAdapter da = new SqlDataAdapter(cmd))
{
da.Fill(dt);
}
}
}
}
catch (Exception ex)
{
LogError(ex);
}
return dt;
}
And here's a test you can run to view the data returned:
for (int i = 0; i < dt.Rows.Count; i++)
{
for (int j = 0; j < dt.Rows[i].ItemArray.Length; j++)
{
Console.Write(dt.Rows[i].ItemArray[j] + ",");
}
Console.WriteLine();
}
Also, you mentioned setting a timeout... I usually set my in my connection string, like so:
"Server=SERVER;Database=DATABASE;User ID=USER;Password=PASSWORD;Trusted_Connection=False;Asynchronous Processing=true;Timeout=60;"
You don't need that extra CommandFactory class at all, and you don't need to set a MinimumCapacity (MSDN) unless you are really trying to optimize performance and have a guesstimate of how many records will be returned. This property isn't used very often.

Reading existing rows from a database using C#

The application I am developing is meant to be a quick and easy tool to import data to our main app. So the user loads in a CSV, meddles with the data a little and pushes it up to the database.
Before the data is pushed to the database, I have a verification check going on which basically says, "Does a customer exist in the database with the same name, account and sort codes? If so, put their guid (which is known already) into a list."
The problem is, the result variable is always 0; this is despite the fact that there is duplicate test data already in my database which should show a result. Added to that, using SQL Profiler, I can't see a query actually being executed against the database.
I'm sure that the ExecuteScalar() is what I should be doing, so my attention comes to the Parameters I'm adding to the SqlCommand... but I'll be blowed if I can figure it... any thoughts?
string connectionString = Generic.GetConnectionString("myDatabase");
using (SqlConnection conn = new SqlConnection(connectionString))
{
conn.Open();
SqlCommand check = new SqlCommand("select COUNT(*) from Customers C1 INNER JOIN CustomerBank B1 ON C1.Id = B1.CustomerId WHERE C1.Name = #Name AND B1.SortCode = #SortCode AND B1.AccountNumber = #AccountNumber", conn);
foreach (DataRow row in importedData.Rows)
{
check.Parameters.Clear();
check.Parameters.Add("#Name", SqlDbType.NVarChar).Value = row["Name"].ToString();
check.Parameters.Add("#SortCode", SqlDbType.NVarChar).Value = row["SortCode"].ToString();
check.Parameters.Add("#AccountNumber", SqlDbType.NVarChar).Value = row["AccountNumber"].ToString();
Object result = check.ExecuteScalar();
int count = (Int32)result;
if (count > 0)
{
DuplicateData.Add((Guid)row["BureauCustomerId"]);
}
}
}
Clarification: importedData is a DataTable of the user's data held in this C# application. During the ForEach loop, each row has various columns, a few of those being Name, SortCode and AccountNumber. The values seem to get set in the parameters (but will verify now).

Most efficient way to populate List<int> via ADO.NET call to stored procedure

I’m calling a stored procedure via ADO.NET as shown (in simplified form) below. The database is a MySQL database.
The stored procedure returns a list of ID values that correspond to calls that need to have their call times rescheduled.
The C# code stores them in a list.
My question is: Is there a more efficient way to get the values into the C# list, instead of using the DataReader as I’m doing?
I don’t know yet if the way I’m doing it is too inefficient for our application (that will be determined during testing), I’m just looking for a faster strategy, if one exists.
I looked at using a DataSet but, from what I’ve read, that could be slower if the list of ID-s is large (which it could be).
Also, from what I’ve read, LINQ might be slower, as well.
I only need to store the list into callsToRescheduleList; ie, I don’t need to do any random access of the ID-s, so those features of the DataSet are not needed.
I’m just looking for the fastest way to get the data into the list.
Any suggestions?
The C# code:
private void GetCallsToRescheduleList()
{
MySqlCommand cmd = new MySqlCommand
(
"`phytel`.`spPhy_GetCallsToRescheduleListPreviousDays`",
(MySqlConnection) DatabaseConnection, workerTransaction
);
cmd.CommandType = CommandType.StoredProcedure;
MySqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
while( reader.Read())
{
callsToRescheduleList.Add(reader.GetInt32(0));
}
}
}
The MySQL stored procedure:
CREATE PROCEDURE `spPhy_GetCallsToRescheduleListPreviousDays` ()
BEGIN
SELECT
id
FROM callrequest
WHERE
dialerCampaignId = 'CATH001’
AND
status = 'SCHEDULED’
;
END

AFAIK that's about as good as it gets - the select is narrow, and use of the reader ordinal overload in the tight loop is good.
Some checks:
Ensure that there is an index (key) on (dialerCampaignId, status) in MySql - I'm assuming there is better selectivity on dialerCampaignId here.
If there is a chance that your List<Int> consumer won't iterate the full list every time, as an alternative to rolling the data up into a list, consider also using an enumerable and yield return:
public IEnumerable<int> GetCallsToRescheduleListPreviousDays()
{
using (var cmd = new MySqlCommand
(
"`phytel`.`spPhy_GetCallsToRescheduleListPreviousDays`",
(MySqlConnection) DatabaseConnection, workerTransaction
)
{
cmd.CommandType = CommandType.StoredProcedure;
using (var reader = cmd.ExecuteReader())
{
if (reader.HasRows)
{
while( reader.Read())
{
yield return reader.GetInt32(0);
}
}
}
}
}
Also remember to Dispose of IDisposable resources like Commands and Readers

Problems when adding data into database (More data/parameters added)

I have a database which i insert some data into. The problem i have is that it will add more data then it should.
When the logfordb.pos is increasing it will add all prevoius data one more time and next time it will add even more.
Here is the code... with some reduction of conn string and parametres.
public void WriteToDatabase(DatabaseLog logfordb)
{
conn.Open();
var cmd = new SqlCommand();
cmd.CommandText = "Insert INTO [X] (Y, Z, C " + "Values (#val1, #val2, #val3";
for (int index = 0; index < logfordb.ListofDB[logfordb.pos].X.Count; index++)
{
cmd.Connection = conn;
cmd.Parameters.Clear();
if (logfordb.ListofDB[logfordb.pos].X[index].Details != null)
{
cmd.Parameters.AddWithValue("#val1", logfordb.ListofDB[logfordb.pos].X[index].Details.Age);
cmd.Parameters.AddWithValue("#val2", logfordb.ListofDB[logfordb.pos].X[index].Details.Name);
cmd.Parameters.AddWithValue("#val3", logfordb.ListofDB[logfordb.pos].X[index].Details.Etc);
cmd.ExecuteNonQuery();
}
}
conn.Close();
logfordb.pos += 1;
}

It seems like you want to change the value of logfordb.pos I think you want to pass logfordb by ref. I suspect the value isn't changing and the second time you call this function you end up adding the same data again because the value outside of this scope isn't being changed just a local copy that the function sees (and then exists destroying references to the changes).

Your code iterates through the entire logfordb every single time this is called.
You have three options:
At the end of this method, clear the logfordb entirely. (probably makes the most sense)
Add a starting position parameter, and let the calling code decide the position to start pushing the logfordb records into the database.
Add a "Stored" property to each record inside logfordb so that you can set it when that individual record has been stored and not push that record again.
Out of those, option 1 is probably your safest bet.