The easiest way to illustrate my question is with this C# code:
using (SqlCommand cmd = new SqlCommand("SELECT * FROM [tbl]", connectionString))
{
using (SqlDataReader rdr = cmd.ExecuteReader())
{
//Somewhere at this point a concurrent thread,
//or another process changes the [tbl] table data
//Begin reading
while (rdr.Read())
{
//Process the data
}
}
}
So what would happen with the data in rdr in such situation?
I actually tested this. Test code:
using (SqlConnection conn = new SqlConnection(ConfigurationManager.ConnectionStrings["test"].ConnectionString))
{
conn.Open();
using (SqlCommand comm = new SqlCommand("select * from test", conn))
{
using (var reader = comm.ExecuteReader())
{
int i = 0;
while (reader.Read())
{
if ((string)reader[1] == "stop")
{
throw new Exception("Stop was found");
}
}
}
}
}
To test, I initialized the table with some dummy data (making sure that no row with the value 'stop' was included). Then I put a break point on the line int i = 0;. While the execution was halted on the break point, I inserted a line in the table with the 'stop' value.
The result was that depending on the amount of initial rows in the table, the Exception was thrown/not thrown. I did not try to pin down where exactly the row limit was. For ten rows, the Exception was not thrown, meaning the reader did not notice the row added from another process. With ten thousand rows, the exception was thrown.
So the answer is: It depends. Without wrapping the command/reader inside a Transaction, you cannot rely on either behavior.
Obligatory disclaimer: This is how it worked in my environment...
EDIT:
I tested using a local Sql server on my dev machine. It reports itself as:
Microsoft SQL Server 2008 R2 (SP1) - 10.50.2550.0 (X64)
Regarding transactions:
Here's code where I use a transaction:
using (SqlConnection conn = new SqlConnection(ConfigurationManager.ConnectionStrings["test"].ConnectionString))
{
conn.Open();
using (var trans = conn.BeginTransaction())
using (SqlCommand comm = new SqlCommand("select * from test", conn, trans))
{
using (var reader = comm.ExecuteReader())
{
int i = 0;
while (reader.Read())
{
i++;
if ((string)reader[1] == "stop")
{
throw new Exception("Stop was found");
}
}
}
trans.Commit();
}
}
In this code, I create the transaction without explicitly specifying an isolation level. That usually means that System.Data.IsolationLevel.ReadCommitted will be used (I think the default isolation level can be set in the Sql Server settings somewhere). In that case the reader behaves the same as before. If I change it to use:
...
using (var trans = conn.BeginTransaction(System.Data.IsolationLevel.Serializable))
...
the insert of the "stop" record is blocked until the transaction is comitted. This means that while the reader is active, no changes to underlying the data is allowed by Sql Server.
Related
Sometimes SqlDataReader throws InvalidOperationException on .GetValue().
(Invalid attempt to read when no data is present).
Similar questions on stack doesn't have .Read() in code, but mine has.
Table1 always contains data, it has rows updated/inserted by other apps, but rows are never deleted.
using (var con = new SqlConnection(s))
{
using (var command = new SqlCommand($"SELECT COL1 FROM TABLE1", con))
{
con.Open();
using (var reader = command.ExecuteReader())
{
if (!reader.HasRows)
throw new InvalidOperationException();
reader.Read();
var globalTimestamp = (byte[])reader.GetValue(0); //Sometimes throws
}
}
}
This code is used in multithreaded apps.
I reproduced this in Parrallel.ForEach:
(using connectionstring with MaxPoolSize 1)
Parallel.For(0, 10, (_, __) =>
{
using (var con = new SqlConnection(s))
{
using (var command = new SqlCommand(/*same query*/, con))
{
con.Open();
var reader = command.ExecuteReader();
{
if (!reader.HasRows)
throw new InvalidOperationException();
reader.Read();
var globalTimestamp = (byte[])reader.GetValue(0); //threw once, after timeout in another thread.
}
}
}
});
I suspect problems with leaked SqlReader in another threads, that corrupt underlying sqlconnection. But i don't know if it is a problem here.
Maybe some of you had sililar issues, and have any ideas what can be done to avoid this problem.
(SqlReader has _sharedState._dataReady = false, and loaded _metadata)
I'm trying to figure out how to setup my database access correctly while using the SqlClient hitting a Microsoft SQL Server. For the most part it is working, but there's a particular scenario that is giving me trouble. Namely: attempting to simultaneously use two connections in the same thread; one with an open data reader and the other performing a delete operation.
The following code demonstrates my conundrum:
public class Database {
...
internal SqlConnection CreateConnection() => new SqlConnection(connectionString);
...
}
public IEnumerable<Model> GetModel() {
var cmd = new SqlCommand() { ... };
using(var conn = db.CreateConnection()) {
conn.Open();
cmd.Connection = conn;
using(var reader = cmd.ExecuteReader()) {
while(reader.Read()) {
var m = new Model();
// deserialization logic
yield return m;
}
}
}
}
public void Delete(int id) {
var cmd = new SqlCommand() { ... }
using(var conn = db.CreateConnection()) {
conn.Open(); // throwing the error here
cmd.Connection = conn;
cmd.ExecuteNonQuery();
}
}
Application Code:
using(var scope = new TransactionScope()) {
var models = GetModels();
foreach(var m in models) {
Delete(m.Id); // throws an exception
}
scope.Complete();
}
For whatever reason, an exception is thrown by the above code while trying to execute the Delete operation:
quote
System.Transactions.TransactionAbortedException: The transaction has aborted. ---> System.Transactions.TransactionPromotionException: Failure while attempting to promote transaction. ---> System.Data.SqlClient.SqlException: There is already an open DataReader associated with this Command which must be closed first. ---> System.ComponentModel.Win32Exception: The wait operation timed out
quote
Now, I have confirmed that if I either set MultipleActiveResultSets=true or Pooling=false on the ConnectionString, that then the above application code will work without error. However, it doesn't seem like I should need to set either of those. If I open two connections simultaneously, should they not be separate connections? Why then am I getting an error from the Delete connection saying that there's an open DataReader?
Please help <3
By far the easiest fix here is to simply load all the models outside the transaction before you go deleting any. Eg
var models = GetModels().ToList();
using(var scope = new TransactionScope()) {
foreach(var m in models) {
Delete(m.Id); // throws an exception
}
scope.Complete();
}
Even fetching the models inside the transaction shold work
using(var scope = new TransactionScope()) {
var models = GetModels().ToList();
foreach(var m in models) {
Delete(m.Id); // throws an exception
}
scope.Complete();
}
so long as you don't leave the connection open during the iteration. If you allow the connection in GetModels() to close, it will be returned to the connection pool, and be available for use for subsequent methods that are enlisted in the same transaction.
In the current code the connection in GetModels() is kept open during the foreach loop and Delete(id) has to open a second connection and try to create a distributed transaction, which is failing.
Without MultipleActiveResultsets, the GetModels connection can't be promoted to a distributed transaction in the middle of returning query results. Setting pooling=false will not make this error go away.
Here's a simplified repro to play with:
using Microsoft.Data.SqlClient;
using System.Collections.Generic;
using System.Transactions;
namespace SqlClientTest
{
class Program
{
static void Main(string[] args)
{
Setup();
var topt = new TransactionOptions();
topt.IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted;
using (new TransactionScope(TransactionScopeOption.Required, topt ))
{
foreach (var id in GetIds())
{
Delete(id);
}
}
}
static string constr = #"server=.;database=tempdb;Integrated Security=true;TrustServerCertificate=true;";
public static void Setup()
{
using (var con = new SqlConnection(constr))
{
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "drop table if exists ids; select object_id id into ids from sys.objects";
cmd.ExecuteNonQuery();
}
}
public static IEnumerable<int> GetIds()
{
using (var con = new SqlConnection(constr))
{
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "select object_id id from sys.objects";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
yield return reader.GetInt32(0);
}
}
}
}
public static void Delete(int id)
{
using (var con = new SqlConnection(constr))
{
con.Open();
var cmd = con.CreateCommand();
cmd.CommandText = "insert into ids(id) values (#id)";
cmd.Parameters.Add(new SqlParameter("#id", id));
cmd.ExecuteNonQuery();
}
}
}
}
And here's what Profiler shows when run:
The main reason here as far as I understand is your yielding iteration.
So the DB connection has not yet called disposed as it's still being used in your iteration (foreach). If for example, you called .ToList() at that point it should return all the entries and then dispose of the connection.
See here for a better explanation on how yield works in an iteration: https://stackoverflow.com/a/58449809/3329836
For some reason, HasRows is false despite the connection and query seeming valid. There is a record in the table.
There are no errors when opening the connection, so I assume it's valid.
I can't see why HasRows would be false.
string connectionString = #"Data Source=(LocalDB)\MSSQLLocalDB;AttachDbFilename=C:\Projects\app\app.mdf;Integrated Security=True;Connect Timeout=30";
string sql = #"SELECT * from SavedSettings";
using (var con = new SqlConnection(connectionString))
using (var cmd = new SqlCommand(sql, con))
{
con.Open();
using (var reader = cmd.ExecuteReader())
{
if (reader.HasRows)
{
while (reader.Read())
{
if (reader["ignoreCase"].ToString() == "1")
{
ignoreCase.Checked = true;
}
else
{
ignoreCase.Checked = false;
}
}
}
}
}
Don't always trust the HasRows. There are times when it has data but returns false. It's been a little wonky since it was introduced. Try this instead:
DataTable dt=new DataTable();
dt.Load(cmd.ExecuteReader());
if (dt.Rows.Count > 0)
{
// Has rows.
}
else {
// No row.
}
OR you don't even need to check if (reader.HasRows) anymore. If your table has no row, then the program will bypass the while (reader.Read()) with no problem. Just use:
while (reader.Read())
{
if (reader["ignoreCase"].ToString() == "1")
{
ignoreCase.Checked = true;
}
else
{
ignoreCase.Checked = false;
}
}
Finally, restart Visual Studio and reconnected to the database then selected Show Table Data.
Disconnected then reconnected to the database and opened the data viewer and it shows my changes. I'm using Visual Studio 2017.
Refreshing the database and refreshing the data viewer did nothing.
Incidentally, it's still not syncing (see below), and I have to keep connecting/reconnecting to see the changes. I'm going to use a different editor.
Before reconnecting (includeSubFolders field shows on left, not right):
After reconnecting:
I am getting Database is locked exception from SQLite for some queries only.
Below is my code:
When I execute any select statement it works fine.
When I am executing any write statement on Jobs Table it also works fine.
This works fine:
ExecuteNonQuery("DELETE FROM Jobs WHERE id=1");
But the same way if I am executing queries for Employees table it is throwing an exception that database is locked.
This throws Exception:
ExecuteNonQuery("DELETE FROM Employees WHERE id=1");
Below are my functions:
public bool OpenConnection()
{
if (Con == null)
{
Con = new SQLiteConnection(ConnectionString);
}
if (Con.State == ConnectionState.Closed)
{
Con.Open();
//Cmd = new SQLiteCommand("PRAGMA FOREIGN_KEYS=ON", Con);
//Cmd.ExecuteNonQuery();
//Cmd.Dispose();
//Cmd=null;
return true;
}
if (IsConnectionBusy())
{
Msg.Log(new Exception("Connection busy"));
}
return false;
}
public Boolean CloseConnection()
{
if (Con != null && Con.State == ConnectionState.Open)
{
if (Cmd != null) Cmd.Dispose();
Cmd = null;
Con.Close();
return true;
}
return false;
}
public Boolean ExecuteNonQuery(string sql)
{
if (sql == null) return false;
try
{
if (!OpenConnection())
return false;
else
{
//Tx = Con.BeginTransaction(IsolationLevel.ReadCommitted);
Cmd = new SQLiteCommand(sql, Con);
Cmd.ExecuteNonQuery();
//Tx.Commit();
return true;
}
}
catch (Exception exception)
{
//Tx.Rollback();
Msg.Log(exception);
return false;
}
finally
{
CloseConnection();
}
}
This is the Exception:
At line 103 : Cmd.ExecuteNonQuery();
Exception Found:
Type: System.Data.SQLite.SQLiteException
Message: database is locked
database is locked
Source: System.Data.SQLite
Stacktrace: at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt)
at System.Data.SQLite.SQLiteDataReader.NextResult()
at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave)
at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior)
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery()
at TimeSheet6.DbOp.ExecuteNonQuery(String sql) in d:\Projects\C# Applications\Completed Projects\TimeSheet6\TimeSheet6\DbOp.cs:line 103
Somewhere along the way a connection is getting left open. Get rid of OpenConnection and CloseConnection and change ExecuteNonQuery to this:
using (SQLiteConnection c = new SQLiteConnection(ConnectionString))
{
c.Open();
using (SQLiteCommand cmd = new SQLiteCommand(sql, c))
{
cmd.ExecuteNonQuery();
}
}
Further, change the way you read data to this:
using (SQLiteConnection c = new SQLiteConnection(ConnectionString))
{
c.Open();
using (SQLiteCommand cmd = new SQLiteCommand(sql, c))
{
using (SQLiteDataReader rdr = cmd.ExecuteReader())
{
...
}
}
}
Do not attempt, to manage connection pooling on your own like you are here. First, it's much more complex than what you have coded, but second, it's handled already inside the SQLiteConnection object. Finally, if you're not leveraging using, you're not disposing these objects properly and you end up with issues like what you're seeing now.
You can use 'using' statement as below, that will make sure connection & command disposed correctly even in exception
private static void ExecuteNonQuery(string queryString)
{
using (var connection = new SQLiteConnection(
ConnectionString))
{
using (var command = new SQLiteCommand(queryString, connection))
{
command.Connection.Open();
command.ExecuteNonQuery();
}
}
}
You should close your DataReader before attempting to write any data to the database. Use:
dr.Close();
after you finish using the DataReader.
In my case it was very stupid of me, I was making changes in SQLite browser and did not click on write changes, which locked the DB to be modified by the services. After I clicked the Write changes button, all the post request worked as expected.
A lot of helpful posts here for folks that may have forgotten to clean up a dangling connection, but there is another way this can happen: SQLite does not support concurrent INSERTs; if you issue two INSERTs at the same time the will be processed in serial. When the INSERTs are quick this is fine, but if an INSERT takes longer than the timeout the second INSERT can fail with this message.
I had this happen when I used a long running transaction to accumulate a bunch of INSERTs into one big commit. Basically I locked the database from any other activity during the transaction. Switching to journal_mode=WAL will allow concurrent writes and reads, but not concurrent writes.
I got rid of the long running transaction and let each INSERT autocommit, and that solved my problem.
Mine was caused by not closing a SqliteDataReader when calling HasRows().
I had this:
using (SQLiteConnection connection = new SQLiteConnection(DbPath))
{
connection.Open();
string sql = $"SELECT * FROM ...";
using (SQLiteCommand command = new SQLiteCommand(sql, connection))
{
return command.ExecuteReader().HasRows;
}
connection.Close();
}
But needed to put a using around the ExecuteReader like so:
using (SQLiteDataReader reader = command.ExecuteReader())
{
return command.ExecuteReader().HasRows;
}
Even though the DbConnection was being disposed and re-created each time the db was still being kept locked by the reader.
I was also getting the same error here:
if (new basics.HindiMessageBox(HMsg, HTitle).ShowDialog()==true)
{
SQLiteConnection m_dbConnection = new SQLiteConnection(MainWindow.con);
m_dbConnection.Open();
sql = "DELETE FROM `users` WHERE `id`=" + SelectedUser.Id;
command = new SQLiteCommand(sql, m_dbConnection);
command.ExecuteNonQuery();
m_dbConnection.Close();
LoadUserDG();
}
but when I just changed SQLiteConnection declaration location
public partial class User : Window
{
SQLiteCommand command;
string sql;
AddUser AddUserObj;
List<basics.users> usersList;
basics.users SelectedUser;
SQLiteConnection m_dbConnection;
// ...
private void DeleteBtn_Click(object sender, RoutedEventArgs e)
{
// ...
if (new basics.HindiMessageBox(HMsg, HTitle).ShowDialog()==true)
{
m_dbConnection = new SQLiteConnection(MainWindow.con);
m_dbConnection.Open();
sql = "DELETE FROM `users` WHERE `id`=" + SelectedUser.Id;
command = new SQLiteCommand(sql, m_dbConnection);
command.ExecuteNonQuery();
m_dbConnection.Close();
LoadUserDG();
}
}
Everything is fine now.
I hope this may work for you, too.
If someone can say how this happened, I would like to know the details to improve my knowledge, please.
I had the same issue when loading a lot of data to different tables from multiple threads.
When trying to do the inserts I was getting database locked because the program was doing too many insert too fast and SQLite didn't have time to complete each transaction before another one came.
The insert are done through threading because I didn't want the interface to be locked and wait for the insert to be done.
My solution is to use BlockingCollection with ThreadPool.QueueUserWorkItem.
This allows me to free the interface while doing the inserts.
All the insert are queued and executed in FIFO (First In First Out) order.
Now the database is never locked while doing any SQL transaction from any thread.
public class DatabaseQueueBus
{
private BlockingCollection<TransportBean> _dbQueueBus = new BlockingCollection<TransportBean>(new ConcurrentQueue<TransportBean>());
private CancellationTokenSource __dbQueueBusCancelToken;
public CancellationTokenSource _dbQueueBusCancelToken { get => __dbQueueBusCancelToken; set => __dbQueueBusCancelToken = value; }
public DatabaseQueueBus()
{
_dbQueueBusCancelToken = new CancellationTokenSource();
DatabaseQueue();
}
public void AddJob(TransportBean dto)
{
_dbQueueBus.Add(dto);
}
private void DatabaseQueue()
{
ThreadPool.QueueUserWorkItem((param) =>
{
try
{
do
{
string job = "";
TransportBean dto = _dbQueueBus.Take(_dbQueueBusCancelToken.Token);
try
{
job = (string)dto.DictionaryTransBean["job"];
switch (job)
{
case "SaveClasse":
//Save to table here
break;
case "SaveRegistrant":
//Save Registrant here
break;
}
}
catch (Exception ex)
{//TODO: Handle this exception or not
}
} while (_dbQueueBusCancelToken.Token.IsCancellationRequested != true);
}
catch (OperationCanceledException)
{
}
catch (Exception ex)
{
}
});
}
}
The inserts are done this way, but without the queuing I was still getting the lock issue.
using (SQLiteConnection c = new SQLiteConnection(BaseDal.SQLiteCon))
{
c.Open();
using (SQLiteCommand cmd = new SQLiteCommand(sql, c))
{
cmd.ExecuteNonQuery();
}
c.Close();
}
I normally use DataSet because It is very flexible. Recently I am assigned code optimization task , To reduce hits to the database I am changing two queries in a procedure. one Query returns the count and the other returns the actual data. That is , My stored procedure returns two tables. Now, I know how to read both tables using DataSets, But I need to read both tables using DataReader. In search of that I found This.
I follow the article and wrote my code like this:
dr = cmd.ExecuteReader();
while (dr.Read())
{
}
if (dr.NextResult()) // this line throws exception
{
while (dr.Read())
{
But I am getting an exception at dt.NextResult. Exception is :
Invalid attempt to call NextResult when reader is closed.
I also googled above error , but still not able to solve the issue.
Any help will be much appreciated. I need to read multiple tables using datareader, is this possible?
Try this because this will close connection ,data reader and command once task get over , so that this will not give datareader close exception
Also do check like this if(reader.NextResult()) to check there is next result,
using (SqlConnection connection = new SqlConnection("connection string here"))
{
using (SqlCommand command = new SqlCommand
("SELECT Column1 FROM Table1; SELECT Column2 FROM Table2", connection))
{
connection.Open();
using (SqlDataReader reader = command.ExecuteReader())
{
while (reader.Read())
{
MessageBox.Show(reader.GetString(0), "Table1.Column1");
}
if(reader.NextResult())
{
while (reader.Read())
{
MessageBox.Show(reader.GetString(0), "Table2.Column2");
}
}
}
}
}
I have tried to reproduce this issue (also because i haven't used multiple tables in a reader before). But it works as expected, hence i assume that you've omitted the related code.
Here's my test code:
using (var con = new SqlConnection(Properties.Settings.Default.ConnectionString))
{
using (var cmd = new SqlCommand("SELECT TOP 10 * FROM tabData; SELECT TOP 10 * FROM tabDataDetail;", con))
{
int rowCount = 0;
con.Open();
using (IDataReader rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
String object1 = String.Format("Object 1 in Row {0}: '{1}'", ++rowCount, rdr[0]);
}
if (rdr.NextResult())
{
rowCount = 0;
while (rdr.Read())
{
String object1 = String.Format("Object 1 in Row {0}: '{1}'", ++rowCount, rdr[0]);
}
}
}
}
}
I built on Pranay Rana's answer because I like keeping it as small as possible.
string rslt = "";
using (SqlDataReader dr = cmd.ExecuteReader())
{
do
{
while (dr.Read())
{
rslt += $"ReqID: {dr["REQ_NR"]}, Shpr: {dr["SHPR_NR"]}, MultiLoc: {dr["MULTI_LOC"]}\r\n";
}
} while (dr.NextResult());
}
The question is old but I find the answers are not correct.
Here's how I do it:
List<DataTable> dataTables = new();
using IDataReader dataReader = command.ExecuteReader();
do
{
DataTable dataTable = new();
dataTable.Load(dataReader);
dataTables.Add(dataTable);
}
while (!dataReader.IsClosed);