Develop Reference

Collection was modified; enumeration operation may not execute (Multi Threading with Locks)

So I know this question has been asked here before, but the situation here is a little different.
I have a service Application that spawns worker threads. The main service thread is organized as so:
public void PollCrunchFilesTask()
{
try
{
var stuckDeletedServTableFiles = MaintenanceDbContext.stuckDeletedServTableFiles;
var stuckErrorStatusFiles = MaintenanceDbContext.stuckErrorStatusFiles;
while (_signalPollAutoEvent.WaitOne())
{
try
{
Poll();
lock (stuckDelLock)
{
if(stuckDeletedServTableFiles.Count > 0)
MaintenanceDbContext.DeleteFilesToBeDeletedInServiceTable(stuckDeletedServTableFiles);
}
lock (errorStatusLock)
{
if (stuckErrorStatusFiles.Count > 0)
MaintenanceDbContext.UpdateStuckErrorServiceLogEntries(stuckErrorStatusFiles);
}
}
catch (Exception ex)
{
}
}
}
catch (Exception ex)
{
}
}
Inside Poll you have this logic:
public void Poll()
{
try
{
if (ProducerConsumerQueue.Count() == 0 && ThreadCount_Diff_ActiveTasks > 0)
{
var dequeuedItems = MetadataDbContext.UpdateOdfsServiceEntriesForProcessingOnPollInterval(ThreadCount_Diff_ActiveTasks);
var handlers = Producer.GetParserHandlers(dequeuedItems);
foreach (var handler in handlers)
{
ProducerConsumerQueue.EnqueueTask(handler.Execute, CancellationTokenSource.Token);
}
}
}
catch (Exception ex)
{
}
}
That ProducerConsumerQueue.EnqueueTask(handler.Execute, CancellationTokenSource.Token); launches 4 worker threads and inside any one of these threads, the following function is called at any time:
public static int DeleteServiceEntry(string logFileName)
{
int rowsAffected = 0;
var stuckDeletedServTableFiles = MaintenanceDbContext.stuckDeletedServTableFiles;
try
{
string connectionString = GetConnectionString();
throw new Exception($"Testing Del HashSet");
using (SqlConnection connection = new SqlConnection())
{
//Attempt some query
}
}
catch (Exception ex)
{
lock (stuckDelLock)
{
stuckDeletedServTableFiles.Add(logFileName);
}
}
return rowsAffected;
}
Now I am testing the stuckDeletedServTableFiles hashset which is only called when there is an exception during a query. this is why I purposefully throw an exception. That hashset is the one that is operated on on the main service thread in the function DeleteFilesToBeDeletedInServiceTable(); who's excerpt is defined below:
public static int DeleteFilesToBeDeletedInServiceTable(HashSet<string> stuckDeletedServTableFiles)
{
int rowsAffected = 0;
string logname = String.Empty; //used to collect error log
var removedHashset = new HashSet<string>();
try
{
var dbConnString = MetadataDbContext.GetConnectionString();
string serviceTable = Constants.SERVICE_LOG_TBL;
using (SqlConnection connection = new SqlConnection(dbConnString))
{
SqlCommand cmd = new SqlCommand();
cmd.CommandType = CommandType.Text;
cmd.CommandText = $"DELETE FROM {serviceTable} WHERE LOGNAME = #LOGNAME";
cmd.Parameters.Add("#LOGNAME", SqlDbType.NVarChar);
cmd.Connection = connection;
connection.Open();
foreach (var logFname in stuckDeletedServTableFiles)
{
cmd.Parameters["#LOGNAME"].Value = logFname;
logname = logFname;
int currRowsAffected = cmd.ExecuteNonQuery();
rowsAffected += currRowsAffected;
if (currRowsAffected == 1)
{
removedHashset.Add(logFname);
Logger.Info($"Removed Stuck {logFname} Marked for Deletion from {serviceTable}");
}
}
Logger.Info($"Removed {rowsAffected} stuck files Marked for Deletion from {serviceTable}");
}
stuckDeletedServTableFiles.ExceptWith(removedHashset);
}
catch (Exception ex)
{
}
return rowsAffected;
}
Given that the hashset stuckDeletedServTableFiles is able to be accessed by multiple threads at the same time including the main service thread, I put a lock on the mainservice thread before it is operated on by DeleteFilesToBeDeletedInServiceTable() and in the function DeleteServiceEntry(). I am new to C# but I assumed this would be enough? I assume since a lock is called on the main service thread for the function DeleteFilesToBeDeletedInServiceTable(), that lock would prevent anything from using the hashset since its being operated on by the function. Why am I getting this error?
Note I am not modifying the Hashset in the forloop. I only do it after the loop is done. I am getting this error while I loop through the Hashset. I suppose because another thread is attempting to modify it. The question then is, why is a thread able to modify the hashSet when I have a lock on the function that calls it on the service level?

For Now, I fixed it by surrounding the for loop in DeleteFilesToBeDeletedInServiceTable() with a lock and also calling the same lock on the statement stuckDeletedServTableFiles.ExceptWith(removedHashset); inside that function. I am not sure if there are costs to this but it seems it will work. And giving how in practice how infrequent this issue will occur, I suppose it wont' cost much. Especially since the times that function is called, we are not doing intensive things. The file crunching would have finished by then and the main caller is utilizing another thread

There is already an open DataReader associated with this Command which must be closed first - AGAIN

I have a client running a web site portal which uses some legacy code that is periodically causing an issue. The system will work for days even weeks and then the worker process will die and no longer serve any data and you have to perform an IISRESET to get it working again
I have found numerous postings about this error and in my mind none of the solutions or explanations fit my code.
Here is the method in question that causes my error
/// <summary>
/// Returns Data from Database using Table Name using a field list (if supplied)
/// Otherwise will return all fields.
/// </summary>
/// <param name="TableName">The TableName rquired</param>
/// <param name="WHERE">Where clause if required</param>
/// <param name="FieldNames">String array of required field names (if any)</param>
/// <returns>Dictionary List of results.</returns>
public List<Dictionary<string, object>> Select(string TableName, string WHERE, string[] FieldNames, int TopRecords = -1, string OrderBy = null)
{
string query = string.Empty;
string sFieldNames = string.Empty;
if (FieldNames.Length > 0)
{
sFieldNames = string.Join(", ", FieldNames);
query = string.Format("SELECT {2}{0} FROM {1} ", sFieldNames, TableName, TopRecords > -1 ? "TOP (" + TopRecords + ") " : "");
if (!string.IsNullOrEmpty(WHERE))
{
query += string.Format(" WHERE {0}", WHERE);
}
}
else
{
// Select ALL fields
query = string.Format("SELECT {1}* FROM {0} ", TableName, TopRecords > -1 ? "TOP (" + TopRecords + ") " : "");
if (!string.IsNullOrEmpty(WHERE))
{
query += string.Format(" WHERE {0}", WHERE);
}
}
if (!string.IsNullOrEmpty(OrderBy))
{
query += " ORDER BY " + OrderBy;
}
//Open connection
if (this.OpenConnection() == true)
{
//System.Diagnostics.Debug.WriteLine( "SQL : " + query );
//Create Command
using (SqlCommand cmd = new SqlCommand(query, DBConnection))
{
//Create a data reader and Execute the command
//Read the data and store them in the list
List<Dictionary<string, object>> ResultsSet = null;//Create a list to store the result
using (SqlDataReader dataReader = cmd.ExecuteReader())
{
ResultsSet = new List<Dictionary<string, object>>();
while (dataReader.Read())
{
Dictionary<string, object> ROW = new Dictionary<string, object>();
for (int i = 0; i < dataReader.FieldCount; i++)
{
if (dataReader[i].GetType().ToString() == "System.Byte[]")
{
ROW.Add(dataReader.GetName(i), (byte[])dataReader[i]);
}
else
{
ROW.Add(dataReader.GetName(i), dataReader[i] + string.Empty);
}
}
ResultsSet.Add(ROW);
}
dataReader.Close(); //close Data Reader
cmd.Dispose(); // Only added today - have to wait for some time to see if it fails
}
return ResultsSet;
}
}
else
{
return null;
}
}
Many solutions state you cannot re-use a connection to perform updates but this method does not. I am pretty sure its obvious and it is only fetching the data from the database and no updates are performed.
I don't want to use MARS unless I have absolutely no choice.
Looking for pointers as to what I might have missed
Connection string
<add name="APP.Properties.Settings.DB" connectionString="server=trs-app;User Id=username;password=xxx;Persist Security Info=False;database=TRailS;Pooling=True" providerName="System.Data.SqlClient"/>
OpenConnection Method
//open connection to database
public bool OpenConnection()
{
try
{
if (DBConnection.State != ConnectionState.Open)
{
while (DBConnection.State == ConnectionState.Connecting)
{
// Do Nothing
}
DBConnection.Open();
System.Diagnostics.Debug.WriteLine("SQL Connection Opened");
}
return true;
}
catch (SqlException ex)
{
switch (ex.Number)
{
case 0:
LastError = "Cannot connect to server. Contact administrator";
return false;
case 1045:
LastError = "Invalid username/password, please try again";
return false;
}
LastError = "Unknown Error : " + ex.Message;
return false;
}
}
Just spotted this in the DAL Class - is this the cause!!!
private static SqlConnection DBConnection;
Solution might be to remove static from Sqlconnection variable (DBConnection) and implement the IDisposable pattern in the DAL class as suggested
protected virtual void Dispose(bool disposing)
{
if (disposing == true)
{
DBConnection.Close(); // call close here to close connection
}
}
~MSSQLConnector()
{
Dispose(false);
}

The code you have here ... while not perfect (see Zohar's comment - it is actively dangerous, in fact), does treat the SqlDataReader correctly - it is making use of using, etc. So: if this is throwing this error, there are three possibilities:
of the code we can see, one of the operations inside the while (dataReader.Read()) has side-effects that is causing another SQL operation to be executed on the same connection; frankly I suspect this is unlikely based on the code shown
there is code that we can't see that already has an open reader before this method is called
this could be because some code higher in the call-stack is doing another similar while (dataReader.Read()) {...} - typical in "N+1" scenarios
or it could be because something that happened earlier (but no-longer in the same call-stack) executed a query, and left the reader dangling
your this.OpenConnection() is sharing a connection between different call contexts without any consideration of what is going on (a textbook example of this would be a static connection, or a connection on some kind of "provider" that is shared between multiple call contexts)
2 and 3 are the most likely options. Unfortunately, diagnosing and fixing that requires a lot of code that we can't see.

After reading all the comments and reviewing the code in light of the information provided I have refactored the DAL class to ensure every method used in the class is now set to a using statement to create the connection. I understand IIS will handle the connection pool for this
I am also closing the db connection in code too (I know its not required with a using statement but its just for neatness.
I have a small monitoring application that will periodically refresh the login page of the portal to watch for an outage and log then perform an IISReset so I can monitor if the problem goes away after all.
Thanks for all the input.
//Open connection
using (SqlConnection DBConnection = new SqlConnection(connectionString))
{
DBConnection.Open();
//System.Diagnostics.Debug.WriteLine( "SQL : " + query );
//Create Command
using (SqlCommand cmd = new SqlCommand(query, DBConnection))
{
//Create a data reader and Execute the command
//Read the data and store them in the list
List<Dictionary<string, object>> ResultsSet = null;//Create a list to store the result
using (SqlDataReader dataReader = cmd.ExecuteReader())
{
ResultsSet = new List<Dictionary<string, object>>();
while (dataReader.Read())
{
Dictionary<string, object> ROW = new Dictionary<string, object>();
for (int i = 0; i < dataReader.FieldCount; i++)
{
if (dataReader[i].GetType().ToString() == "System.Byte[]")
{
ROW.Add(dataReader.GetName(i), (byte[])dataReader[i]);
}
else
{
ROW.Add(dataReader.GetName(i), dataReader[i] + string.Empty);
}
}
ResultsSet.Add(ROW);
}
}
DBConnection.Close();
return ResultsSet;
}
}
}

Exception handling quandry

I am throwing a new exception when a database row is not found.
Class that was called:
public ProfileBO retrieveProfileByCode(string profileCode)
{
return retrieveSingleProfile("profile_code", profileCode);
}
private ProfileBO retrieveSingleProfile(string termField, string termValue)
{
ProfileBO profile = new ProfileBO();
//Query string is temporary. Will make this a stored procedure.
string queryString = " SELECT * FROM GamePresenterDB.gp.Profile WHERE " + termField + " = '" + termValue + "'";
using (SqlConnection connection = new SqlConnection(App.getConnectionString()))
{
connection.Open();
SqlCommand command = new SqlCommand(queryString, connection);
SqlDataReader reader = command.ExecuteReader();
if (reader.Read())
{
profile = castDataReadertoProfileBO(reader, profile);
}
else
{
// No record was selected. log it and throw the exception (We'll log it later, for now just write to console.)
Console.WriteLine("No record was selected from the database for method retrieveSingleProfile()");
throw new InvalidOperationException("An exception occured. No data was found while trying to retrienve a single profile.");
}
reader.Close();
}
return profile;
}
However, when I catch the exception in the calling class, 'e' is now null. What am I doing wrong? I believe this works fine in Java, so C# must handle this differently.
Calling class:
private void loadActiveProfile()
{
try
{
ProfileBO profile = profileDAO.retrieveProfileByCode(p.activeProfileCode);
txtActiveProfileName.Text = profile.profile_name;
}
catch (InvalidOperationException e)
{
}
}

Now all the code has been put in the question, you can move the try catch outside of your 'loadActiveProfile' method and place it into 'retrieveSingleProfile'.
private void loadActiveProfile()
{
ProfileBO profile = profileDAO.retrieveProfileByCode(p.activeProfileCode);
txtActiveProfileName.Text = profile.profile_name;
}
removed the try catch^
private ProfileBO retrieveSingleProfile(string termField, string termValue)
{
try {
ProfileBO profile = new ProfileBO();
//Query string is temporary. Will make this a stored procedure.
string queryString = " SELECT * FROM GamePresenterDB.gp.Profile WHERE " + termField + " = '" + termValue + "'";
using (SqlConnection connection = new SqlConnection(App.getConnectionString()))
{
connection.Open();
SqlCommand command = new SqlCommand(queryString, connection);
SqlDataReader reader = command.ExecuteReader();
if (reader.Read())
{
profile = castDataReadertoProfileBO(reader, profile);
}
else
{
// No record was selected. log it and throw the exception (We'll log it later, for now just write to console.)
Console.WriteLine("No record was selected from the database for method retrieveSingleProfile()");
throw new InvalidOperationException("An exception occured. No data was found while trying to retrienve a single profile.");
}
reader.Close();
}
return profile;
}
catch(InvalidOperationException e)
{
}
}
Added try catch in the correct place.

You need to step into the catch block for e to be set to the thrown InvalidOperationException:
catch (System.InvalidOperationException e)
{
int breakPoint = 0; //<- set a breakpoint here.
//Either you reach the breakpoint and have an InvalidOperationException, or you don't reach the breakpoint.
MessageBox.Show(e.Message);
}
Also make sure that the InvalidOperationException you throw is actually a System.InvalidOperationException and not some custom type of yours called "InvalidOperationException".

Like #Clemens said, you need to show all the relevant code.
As a quick test, this works just fine:
class Program
{
static void Main(string[] args)
{
try
{
Console.WriteLine("Throwing error");
ThrowException();
}
catch (InvalidOperationException e)
{
Console.WriteLine(e.Message);
}
Console.ReadKey(true);
}
static void ThrowException()
{
throw new InvalidOperationException("Blah blah blah");
}
}

Closing MySql datareader connection

So this is a little bit code-ceptionlike.
I have a function that is checking the last ID in a table, this function is called within another function. At the end of that function, I have another function that's opening another datareader.
Error:
There is already an open Datareader associated with this connection which must be closed first.
getLastIdfromDB()
public string getLastIdFromDB()
{
int lastIndex;
string lastID ="";
var dbCon = DB_connect.Instance();
if (dbCon.IsConnect())
{
MySqlCommand cmd2 = new MySqlCommand("SELECT ID FROM `competitor`", dbCon.Connection);
try
{
MySqlDataReader reader = cmd2.ExecuteReader();
while (reader.Read())
{
string item = reader2["ID"].ToString();
lastIndex = int.Parse(item);
lastIndex++;
lastID = lastIndex.ToString();
}
}
catch (Exception ex)
{
MessageBox.Show("Error:" + ex.Message);
}
}
return lastID;
}
This function is later-on used in this function:
private void addPlayerBtn_Click(object sender, EventArgs e)
{
ListViewItem lvi = new ListViewItem(getLastIdFromDB());
.........................................^
... HERE
...
... irrelevant code removed
.........................................
var dbCon = DB_connect.Instance();
if (dbCon.IsConnect())
{
MySqlCommand cmd = new MySqlCommand("INSERT INTO `competitor`(`ID`, `Name`, `Age`) VALUES(#idSql,#NameSql,#AgeSql)", dbCon.Connection);
cmd.Parameters.AddWithValue("#idSql", getLastIdFromDB());
cmd.Parameters.AddWithValue("#NameSql", playerName.Text);
cmd.Parameters.AddWithValue("#AgeSql", playerAge.Text);
try
{
cmd.ExecuteNonQuery();
listView1.Items.Clear();
}
catch (Exception ex)
{
MessageBox.Show("Error:" + ex.Message);
dbCon.Connection.Close();
}
finally
{
updateListView();
}
}
}
What would be the best way for me to solve this problem and in the future be sure to close my connections properly?
UPDATE: (per request, included DB_connect)
class DB_connect
{
private DB_connect()
{
}
private string databaseName = "simhopp";
public string DatabaseName
{
get { return databaseName; }
set { databaseName = value; }
}
public string Password { get; set; }
private MySqlConnection connection = null;
public MySqlConnection Connection
{
get { return connection; }
}
private static DB_connect _instance = null;
public static DB_connect Instance()
{
if (_instance == null)
_instance = new DB_connect();
return _instance;
}
public bool IsConnect()
{
bool result = true;
try
{
if (Connection == null)
{
if (String.IsNullOrEmpty(databaseName))
result = false;
string connstring = string.Format("Server=localhost; database={0}; UID=root;", databaseName);
connection = new MySqlConnection(connstring);
connection.Open();
result = true;
}
}
catch (Exception ex)
{
Console.Write("Error: " + ex.Message);
}
return result;
}
public void Close()
{
connection.Close();
}
}
}

You are trying to have multiple open readers on the same connection. This is commonly called "MARS" (multiple active result sets). MySql seems to have no support for it.
You will have to either limit yourself to one open reader at a time, or use more than one connection, so you can have one connection for each reader.
My suggestion would be to throw away that singleton-like thingy and instead use connection pooling and proper using blocks.

As suggested by Pikoh in the comments, using the using clause indeed solved it for me.
Working code-snippet:
getLastIdFromDB
using (MySqlDataReader reader2 = cmd2.ExecuteReader()) {
while (reader2.Read())
{
string item = reader2["ID"].ToString();
lastIndex = int.Parse(item);
lastIndex++;
lastID = lastIndex.ToString();
}
}

Your connection handling here is not good. You need to ditch the DB_connect. No need to maintain a single connection - just open and close the connection each time you need it. Under the covers, ADO.NET will "pool" the connection for you, so that you don't actually have to wait to reconnect.
For any object that implements IDisposable you need to either call .Dispose() on it in a finally block, or wrap it in a using statement. That ensures your resources are properly disposed of. I recommend the using statement, because it helps keep the scope clear.
Your naming conventions should conform to C# standards. Methods that return a boolean should be like IsConnected, not IsConnect. addPlayerBtn_Click should be AddPlayerButton_Click. getLastIdFromDB should be GetlastIdFromDb or getLastIdFromDatabase.
public string GetLastIdFromDatabase()
{
int lastIndex;
string lastID ="";
using (var connection = new MySqlConnection(Configuration.ConnectionString))
using (var command = new MySqlCommand("query", connection))
{
connection.Open();
MySqlDataReader reader = cmd2.ExecuteReader();
while (reader.Read())
{
string item = reader2["ID"].ToString();
lastIndex = int.Parse(item);
lastIndex++;
lastID = lastIndex.ToString();
}
}
return lastID;
}
Note, your query is bad too. I suspect you're using a string data type instead of a number, even though your ID's are number based. You should switch your column to a number data type, then select the max() number. Or use an autoincrementing column or sequence to get the next ID. Reading every single row to determine the next ID and incrementing a counter not good.

Is there a way to use DBMS_Alert to notify a Winform application of a database change

I am trying to get a winform app to refresh an embedded browser on a database change using Oracle 10g. The only problem is that I am not allowed to use Database Change Notification. I am curious if anyone has a way of using the built-in package of DBMS_Alert and have had some action happen to a winform app on a database change.
Thanks, Andrew

food for thought...
if you are using ODP, you could use Oracle Advanced Queuing/Streams
and here.
this way your form app can subscribe to a queue and be notified of a change.
This may, however, be massive overkill for your application if you just want to add a new PO # into a drop down!
I have used streams before and it works as expected, but it had a nice level of research and trial & error to get things to click.

I had to do it like this for it to work. It holds the window in lock until an event occurs i know, but at least it works with DBMS_Alert. I set this code inside a timer:
OracleConnection conn = new OracleConnection(ConnectionString);
conn.Open();
OracleCommand cmd = new OracleCommand("DECLARE\n" +
"MESSAGE VARCHAR2(1800) := null;\n" +
"STATUS INTEGER;\n" +
"BEGIN\n" +
"DBMS_ALERT.REGISTER('ALERT');\n" +
"DBMS_ALERT.WAITONE('ALERT', MESSAGE, STATUS);\n" +
"DBMS_ALERT.REMOVE('ALERT');\n" +
"END;", conn);
cmd.ExecuteNonQuery();
wbMain.Refresh();
conn.Dispose();
This gives me what I need. I don't know if there is a better way to do it, but this is the only solution I could come up with.

It is better to use without timer. The below code sample is with background thread
Here is the code snippet
privateThread DBMSAlertThread;
private void DBMSAlert(bool Register)
{
try
{
string sSql;
if (Register)
sSql = "call dbms_alert.register('XYZ')";
else
sSql = "call dbms_alert.remove('XYZ')";
dbmsAlert = new OracleCommand();
dbmsAlert.CommandText = sSql;
dbmsAlert.ExecuteNonQuery();
if (Register) //start the background thread
{
DBMSAlertThread = new Thread(AlertEvent);
DBMSAlertThread.IsBackground = true;
DBMSAlertThread.Start();
}
}
catch (Exception LclExp)
{
//Show error or capture in eventlog
}
}
private void AlertEvent(object sender)
{
while (true)
{
string Message = "";
int Status = -1;
bool bStatus;
OracleParameter param;
try
{
OracleCommand dbmsAlert = new OracleCommand();
dbmsAlertScan.SQL.Add("call dbms_alert.WaitOne('XYZ', :Message, :Status, 0)"); //Last parameter indicate wait time
param = new OracleParameter("Message", OracleDbType.Varchar2, ParameterDirection.Output);
dbmsAlert.Parameters.Add(param);
param = new OracleParameter("Status", OracleDbType.Varchar2, ParameterDirection.Output);
dbmsAlert.Parameters.Add(param);
OracleParameter.ExceuteNonQuery();
Message = dbmsAlert.Parameters["Message"].Value.ToString();
bStatus = int.TryParse(dbmsAlert.Parameters["Status"].Value.ToString(), out Status);
if (Status == 0) //0 = Alert Received, 1 = Timed out
{
//notify or do ur stuff
}
}
catch (Exception Exp)
{
//raise an error
}
}
}