I'm using System.Data.SQLite, and I have some code where I'm wrapping a series of commands in a SQLite transaction.
Is it required to set the Transaction property to the transaction instance? It seems as though the command automatically picks it up from the connection.
The reason this is important to me is determining what I have to pass down to helper functions--i.e. just a reference to the connection or to the transaction as well.
Edit:
Here's a simplified example.
private void ExecuteSql(SQLiteConnection conn, IEnumerable<string> sqlStatements)
{
using (var trans = conn.BeginTransaction())
{
try
{
foreach (var sql in sqlStatements) ExecuteSql(conn, sql);
}
catch
{
trans.Rollback();
throw; // pass up to higher level
}
trans.Commit();
}
}
private void ExecuteSql(SQLiteConnection conn, string sqlStatement)
{
using (var cmd = new SQLiteCommand(conn)
{
//Transaction = trans, -- necessary?
CommandText = sqlStatement,
})
{
cmd.ExecuteNonQuery();
}
}
No, you don't need to set the Transaction property manually.
Everything you do with the connection while the transaction is open will be tied to that transaction. However, this might not hold true for other database engines (although in general I think it does), so be careful if ever you change your DB.
I find the ADO.NET interfaces convoluted for no good reason; there's lots of redundancy, and providers often have to jump through hoops to get optimal performance with certain parts of the interface (e.g. the IDataParameter/IDataParameterCollection monstrosity).
Related
I have the following code:
public void Execute(string Query, params SqlParameter[] Parameters)
{
using (var Connection = new SqlConnection(Configuration.ConnectionString))
{
Connection.Open();
using (var Command = new SqlCommand(Query, Connection))
{
if (Parameters.Length > 0)
{
Command.Parameters.Clear();
Command.Parameters.AddRange(Parameters);
}
Command.ExecuteNonQuery();
}
}
}
The method may be called 2 or 3 times for different queries but in same manner.
For example:
Insert an Employee
Insert Employee Certificates
Update Degree of Employee on another table [ Fail can cause here. for example ]
If Point [3] fails, all already committed commands shouldn't execute and must be rolled back.
I know I can put SqlTransaction above and use Commit() method. But what about 3rd point if failed? I think point 3 only will rollback and other point 1,2 will not? How to solve this and what approach should I do??
Should I use SqlCommand[] arrays? What I should I do?
I only find similar question but in CodeProject:
See Here
Without changing your Execute method you can do this
var tranOpts = new TransactionOptions()
{
IsolationLevel = IsolationLevel.ReadCommitted,
Timeout = TransactionManager.MaximumTimeout
};
using (var tran = new TransactionScope(TransactionScopeOption.Required, tranOpts)
{
Execute("INSERT ...");
Execute("INSERT ...");
Execute("UPDATE ...");
tran.Complete();
}
SqlClient will cache the internal SqlConnection that is enlisted in the Transaction and reuse it for each call to Execute. So you even end up with a local (not distributed) transaction.
This is all explained in the docs here: System.Transactions Integration with SQL Server
There are a few ways to do it.
The way that probably involves changing the least code and involves the least complexity is to chain multiple SQL statements into a single query. It's perfectly fine to build a string for the Query argument that runs more than one statement, including BEGIN TRANSACTION, COMMIT, and (if needed) ROLLBACK. Basically, keep a whole stored procedure in your C# code. This also has the nice benefit of making it easier to use version control with your procedures.
But it still feels kind of hackish.
One way to reduce that effect is marking the Execute() method private. Then, have an additional method in the class for each query. In this way, the long SQL strings are isolated, and when you're using the database it feels more like using a local API. For more complicated applications, this might instead be a whole separate assembly with a few types managing logical functional areas, where the core methods like Exectue() are internal. This is a good idea anyway, regardless of how you end up supporting transactions.
And speaking of procedures, stored procedures are also a perfectly fine way to handle this. Have one stored procedure to do all the work, and call it when ready.
Another option is overloading the method to accept multiple queries and parameter collections:
public void Execute(string TransactionName, string[] Queries, params SqlParameter[][] Parameters)
{
using (var Connection = new SqlConnection(Configuration.ConnectionString))
using (var Transaction = new SqlTransaction(TransactionName))
{
connection.Transaction = Transaction;
Connection.Open();
try
{
for (int i = 0; i < Queries.Length; i++)
{
using (var Command = new SqlCommand(Queries[i], Connection))
{
command.Transaction = Transaction;
if (Parameters[i].Length > 0)
{
Command.Parameters.Clear();
Command.Parameters.AddRange(Parameters);
}
Command.ExecuteNonQuery();
}
}
Transaction.Commit();
}
catch(Exception ex)
{
Transaction.Rollback();
throw; //I'm assuming you're handling exceptions at a higher level in the code
}
}
}
Though I'm not sure how the params keyword works with an array of arrays... I've just not tried that option, but something along these lines would work. The weakness here is also that it's not trivial to have a later query depend on a result from an earlier query, and even queries with no parameter would still need a Parameters array as a placeholder.
A final option is extending the type holding your Execute() method to support transactions. The trick here is it's common (and desirable) to have this type be static, but supporting transactions requires re-using common connection and transaction objects. Given the implied long-running nature of a transaction, you have to support more than one at a time, which means both instances and implementing IDisposable.
using (var connection = new SqlConnection(Configuration.ConnectionString))
{
SqlCommand command = connection.CreateCommand();
SqlTransaction transaction;
connection.Open();
transaction = connection.BeginTransaction("Transaction");
command.Connection = connection;
command.Transaction = transaction;
try
{
if (Parameters.Length > 0)
{
command.Parameters.Clear();
command.Parameters.AddRange(Parameters);
}
command.ExecuteNonQuery();
transaction.Commit();
}
catch (Exception e)
{
try
{
transaction.Rollback();
}
catch (Exception ex2)
{
//trace
}
}
}
I'm designing a small desktop app that fetches data from SQL server. I used BackgroundWorker to make the query execute in background. The code that fetches data generally comes down to this:
public static DataTable GetData(string sqlQuery)
{
DataTable t = new DataTable();
using (SqlConnection c = new SqlConnection(GetConnectionString()))
{
c.Open();
using (SqlCommand cmd = new SqlCommand(sqlQuery))
{
cmd.Connection = c;
using (SqlDataReader r = cmd.ExecuteReader())
{
t.Load(r);
}
}
}
return t;
}
Since query can take up 10-15 minutes I want to implement cancellation request and pass it from GUI layer to DAL. Cancellation procedure of BackroundWorker won't let me cancel SqlCommand.ExecuteReader() beacuse it only stops when data is fetched from server or an exception is thrown by Data Provider.
I tried to use Task and async/await with SqlCommand.ExecuteReaderAsync(CancellationToken) but I am confused where to use it in multi-layer app (GUI -> BLL -> DAL).
Have you tried using the SqlCommand.Cancel() method ?
Aproach: encapsulate that GetData method in a Thread/Worker and then when you cancel/stop that thread call the Cancel() method on the SqlCommand that is being executed.
Here is an example on how to use it on a thread
using System;
using System.Data;
using System.Data.SqlClient;
using System.Threading;
class Program
{
private static SqlCommand m_rCommand;
public static SqlCommand Command
{
get { return m_rCommand; }
set { m_rCommand = value; }
}
public static void Thread_Cancel()
{
Command.Cancel();
}
static void Main()
{
string connectionString = GetConnectionString();
try
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
Command = connection.CreateCommand();
Command.CommandText = "DROP TABLE TestCancel";
try
{
Command.ExecuteNonQuery();
}
catch { }
Command.CommandText = "CREATE TABLE TestCancel(co1 int, co2 char(10))";
Command.ExecuteNonQuery();
Command.CommandText = "INSERT INTO TestCancel VALUES (1, '1')";
Command.ExecuteNonQuery();
Command.CommandText = "SELECT * FROM TestCancel";
SqlDataReader reader = Command.ExecuteReader();
Thread rThread2 = new Thread(new ThreadStart(Thread_Cancel));
rThread2.Start();
rThread2.Join();
reader.Read();
System.Console.WriteLine(reader.FieldCount);
reader.Close();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
static private string GetConnectionString()
{
// To avoid storing the connection string in your code,
// you can retrieve it from a configuration file.
return "Data Source=(local);Initial Catalog=AdventureWorks;"
+ "Integrated Security=SSPI";
}
}
You can only do Cancelation checking and Progress Reporting between Distinct lines of code. Usually both require that you disect the code down to the lowest loop level, so you can do both these things between/in the loop itterations. When I wrote my first step into BGW, I had the advantage that I needed to do the loop anyway so it was no extra work. You have one of the worse cases - pre-existing code that you can only replicate or use as is.
Ideal case:
This operation should not take nearly as long is it does. 5-10 minutes indicates that there is something rather wrong with your design.
If the bulk of the time is transmission of data, then you are propably retreiving way to much data. Retrieving everything to do filtering in the GUI is a very common mistake. Do as much filtering in the query as possible. Usign a Distributed Database might also help with transmission performance.
If the bulk of the time is processing as part of the query operation (complex Conditions), something in your general approach might have to change. There are various ways to trade off complex calculation with a bit of memory on the DBMS side. Views afaik can cache the results of operations, while still maintaining transactional consistency.
But it really depends what your backend DB/DBMS and use case are. A lot of the use SQL as Query Language. So it does not allow us to predict wich options you have.
Second best case:
The second best thing if you can not cut it down, would be if you had the actually DB access code down to the lowest loop and would do progress reporting/cancelation checking on it. That way you could actually use the existing Cancelation Token System inherent in BGW.
Everything else
Using any other approach to Cancelation is really a fallback. I wrote a lot on why it is bad, but felt that this might work better if I focus on the core issue - likely something wrong in design of he DB and/or Query. Because those might well eliminate the issue altogether.
As learning exercise and before trying to use any ORM (like EF) I want to build a personal project using ADO.NET and stored procedures.
Because I don't want my code to become a mess over time, I want to use some patterns like the repository and UoW patterns.
I've got almost everything figured it out, except for the transaction handling.
To somehow 'simulate' a UoW, I used this class provided by #jgauffin, but what's stopping me from using that class is that every time you create a new instance of that class (AdoNetUnitOfWork) you're automatically beginning a transaction and there a lot of cases where you only need to read data.
In this regard this is what I found in one of the SQL books I've been reading:
Executing a SELECT statement within a transaction can create locks on the referenced tables, which can in turn block other users or sessions from performing work or reading data
This is the AdoNetUnitOfWork class:
public class AdoNetUnitOfWork : IUnitOfWork
{
public AdoNetUnitOfWork(IDbConnection connection, bool ownsConnection)
{
_connection = connection;
_ownsConnection=ownsConnection;
_transaction = connection.BeginTransaction();
}
public IDbCommand CreateCommand()
{
var command = _connection.CreateCommand();
command.Transaction = _transaction;
return command;
}
public void SaveChanges()
{
if (_transaction == null)
throw new InvalidOperationException("Transaction have already been commited. Check your transaction handling.");
_transaction.Commit();
_transaction = null;
}
public void Dispose()
{
if (_transaction != null)
{
_transaction.Rollback();
_transaction = null;
}
if (_connection != null && _ownsConnection)
{
_connection.Close();
_connection = null;
}
}
}
And this is how I want to use the UoW in my repositories:
public DomainTable Get(int id)
{
DomainTable table;
using (var commandTable = _unitOfWork.CreateCommand())
{
commandTable.CommandType = CommandType.StoredProcedure;
//This stored procedure contains just a simple SELECT statement
commandTable.CommandText = "up_DomainTable_GetById";
commandTable.Parameters.Add(commandTable.CreateParameter("#pId", id));
table = ToList(commandTable).FirstOrDefault();
}
return table;
}
I know I can tweak this code a bit so that the transaction would be optional, but since I trying to make this code as platform independent as possible and as far as I know in other persistence frameworks like EF you don't have to manage transactions manually, the question is, will I be creating some kind of bottleneck by using this class as it is, that is, with transactions always being created?
It all depends on the transaction isolation level. Using the default isolation level (ie. read committed) then your SELECT should occur no performance penalty if is wrapped in a transaction. SQL Server internally wraps statements in a transaction anyway if one is not already started, so your code should behave almost identical.
However, I must ask you why not use the built-in .Net TransactionScope? This way your code will interact much better with other libraries and frameworks, since TransactionScope is universally used. If you do decide to switch to this I must warn you that, by default, TransactionScope uses SERIALIZABLE isolation level and this does result in performance penalties, see using new TransactionScope() Considered Harmful.
I have a static helper method for bulk inserting Entity Framework objects. It works and my unit tests pass but it looks wrong and I can't help but think there must be a better way to do it.
public static void BulkInsert<T>(Entities db, IList<T> list)
{
SqlTransaction transaction = db.Database.CurrentTransaction != null ? db.Database.CurrentTransaction.UnderlyingTransaction as SqlTransaction : null;
if (db.Database.Connection.State != ConnectionState.Open)
{
db.Database.Connection.Open();
}
using (var bulkCopy = new SqlBulkCopy((db.Database.Connection) as SqlConnection, SqlBulkCopyOptions.Default, transaction))
{
//fill a datatable and write to server, this bit works
}
}
It could be called inside a DbContextTransaction and/or the DbContext.Database might have done something already so the connection might be open. If you don't explicitly open the connection before creating the bulk copy you get an error:
System.InvalidOperationException: WriteToServer requires an open and available Connection
which is odd, I'd have thought that sqlBulkCopy would have opened the connection if it had to.
So my question is am I going about this the right way?
I have used your approach successfully myself.
My feeling is that you should close it again if you opened it.
I would additionally let SqlBulkCopy handle the transaction itself. I don't think enlisting SqlBulkCopy into any ambient transaction is a great idea unless your particular scenario absolutely demands it.
public static void BulkInsert<T>(PandaDataContext db, IList<T> list)
{
var mustOpen = db.Database.Connection.State != ConnectionState.Open;
try
{
if (mustOpen)
db.Database.Connection.Open();
using (var bulkCopy = new SqlBulkCopy((db.Database.Connection) as SqlConnection, SqlBulkCopyOptions.Default))
{
}
}
finally
{
if (mustOpen)
db.Database.Connection.Close();
}
}
I have a web application that issues requests to 3 databases in the DAL. I'm writing some integration tests to make sure that the overall functionality round trip actually does what i expect it to do. This is completely separate from my unit tests, just fyi.
The way I was intending to write these tests were something to the effect of this
[Test]
public void WorkflowExampleTest()
{
(using var transaction = new TransactionScope())
{
Presenter.ProcessWorkflow();
}
}
The Presenter in this case has already been set up. The problem comes into play inside the ProcessWorkflow method because it calls various Repositories which in turn access different databases, and my sql server box does not have MSDTC enabled, so I get an error whenever I try to either create a new sql connection, or try to change a cached connection's database to target a different one.
For brevity the Presenter resembles something like:
public void ProcessWorkflow()
{
LogRepository.LogSomethingInLogDatabase();
var l_results = ProcessRepository.DoSomeWorkOnProcessDatabase();
ResultsRepository.IssueResultstoResultsDatabase(l_results);
}
I've attempted numerous things to solve this problem.
Caching one active connection at all times and changing the target database
Caching one active connection for each target database (this was kind of useless because pooling should do this for me, but I wanted to see if I got different results)
Adding additional TransactionScopes inside each repository so that they have their own transactions using the TransactionScopeOption "RequiresNew"
My 3rd attempt on the list looks something like this:
public void LogSomethingInLogDatabase()
{
using (var transaction =
new TransactionScope(TransactionScopeOption.RequiresNew))
{
//do some database work
transaction.Complete();
}
}
And actually the 3rd thing I tried actually got the unit tests to work, but all the transactions that completed actually HIT my database! So that was an utter failure, since the entire point is to NOT effect my database.
My question therefore is, what other options are out there to accomplish what I'm trying to do given the constraints I've laid out?
EDIT:
This is what "//do some database work" would look like
using (var l_context = new DataContext(TargetDatabaseEnum.SomeDatabase))
{
//use a SqlCommand here
//use a SqlDataAdapter inside the SqlCommand
//etc.
}
and the DataContext itself looks something like this
public class DataContext : IDisposable
{
static int References { get; set; }
static SqlConnection Connection { get; set; }
TargetDatabaseEnum OriginalDatabase { get; set; }
public DataContext(TargetDatabaseEnum database)
{
if (Connection == null)
Connection = new SqlConnection();
if (Connection.Database != DatabaseInfo.GetDatabaseName(database))
{
OriginalDatabase =
DatabaseInfo.GetDatabaseEnum(Connection.Database);
Connection.ChangeDatabase(
DatabaseInfo.GetDatabaseName(database));
}
if (Connection.State == ConnectionState.Closed)
{
Connection.Open() //<- ERROR HAPPENS HERE
}
ConnectionReferences++;
}
public void Dispose()
{
if (Connection.State == ConnectionState.Open)
{
Connection.ChangeDatabase(
DatabaseInfo.GetDatabaseName(OriginalDatabase));
}
if (Connection != null && --ConnectionReferences <= 0)
{
if (Connection.State == ConnectionState.Open)
Connection.Close();
Connection.Dispose();
}
}
}
Set Enlist=false on connection string to avoid auto enlistment on transaction.
Manually enlist connection as participants in transaction scope. (http://msdn.microsoft.com/en-us/library/ms172153%28v=VS.80%29.aspx)
Ok, I found a way around this issue. The only reason I'm doing it this way is because I couldn't find ANY other way to fix this problem, and because it's in my integration tests, so I'm not concerned about this having adverse effects in production code.
I had to add a property to my DataContext to act as a flag to keep track of whether or not to dispose of the connection object when my DataContext is being disposed. This way, the connection is kept alive throughout the entire transaction scope, and therefore no longer bothers DTC
Here's sample of my new Dispose:
internal static bool SupressConnectionDispose { get; set; }
public void Dispose()
{
if (Connection.State == ConnectionState.Open)
{
Connection.ChangeDatabase(
DatabaseInfo.GetDatabaseName(OriginalDatabase));
}
if (Connection != null
&& --ConnectionReferences <= 0
&& !SuppressConnectionDispose)
{
if (Connection.State == ConnectionState.Open)
Connection.Close();
Connection.Dispose();
}
}
this allows my integration tests to take the form of:
[Test]
public void WorkflowExampleTest()
{
(using var transaction = new TransactionScope())
{
DataContext.SuppressConnectionDispose = true;
Presenter.ProcessWorkflow();
}
}
I would not recommend utilizing this in production code, but for integration tests I think it is appropriate. Also keep in mind this only works for connections where the server is always the same, as well as the user.
I hope this helps anyone else who runs into the same problem I had.
If you don't want to use MSDTC you can use SQL transactions directly.
See SqlConnection.BeginTransaction().