We have a system that has both a .NET and SQLServer component. The .NET component is responsible to start the process and enclose the calls to stored procedures in a transaction.
That process is done using mostly stored procedures on the database, which all write to our own log table.
Now, the problem is that the transaction, being started in the .NET code, will be enclosing all others that are used in the database code, and if one of the child transactions rolls back the log table is gone.
Is there any way to keep the logs, rolling back everything else?
Thanks in advance.
Use a table variable to store the error: this is unaffected by transactions
Or use SET XACT_ABORT ON in the procs which implicitly rolls back and log after error
Is Transaction SavePoints can help you?
This is exactly data base level solution:
Savepoints offer a mechanism to roll back portions of transactions.
You create a savepoint using the SAVE TRANSACTION savepoint_name
statement. Later, you execute a ROLLBACK TRANSACTION savepoint_name
statement to roll back to the savepoint instead of rolling back to the
start of the transaction.
Full description see here: Transaction Savepoints.
Also see good (as for me) An example of Stored Procedure template
Also you can reffer to SqlTransaction.Save Method for a C# solution, hope this is exatcly what you are looking for:
Creates a savepoint in the transaction that can be used to roll back a
part of the transaction, and specifies the savepoint name.
Related
In our dev team we have an interesting discussion regarding opening transactions during reads in Entity Framework.
Case is this: we have unit of work in MVC app which spans action methods - simple open EF transaction before executing action and commits after no error appears during execution. This is fine and maybe some of you use an UoW pattern with EF in that way.
Interesting part is what about actions that performs only reads (no modification of any entity for example get by id). Should transaction be opened also for reads? What would be the difference in approach when we don't open transaction and during read there is active transaction on same table we read data not using tran? Suppose that we have set default transaction isolation level to read committed.
I was pro opening transaction which keeps reads consistent but there are things against like transactions slowdown reads (which is true but I don't know how much).
What are your thoughts? I know that some of u will answer as old architects saying "it depends" but I need strong arguments not hate :)
For SQL Server at READ COMMITTED isolation there is no difference between a SELECT inside a transaction and one outside a transaction.
With legacy READ COMMITTED the S locks are released at the end of each query even in a transaction.
With READ COMMITTED SNAPSHOT (which is the default for EF Code First) there are no S locks taken, and row versions provide only a statement-level point-in-time view of the database.
At SNAPSHOT isolation, the whole transaction would see the database at a single point-in-time, still with no locking.
Recently we faced quite an interesting issue that has to do with SQL transactions timeout. The statement that timed out does not really matter for the sake of question, but it was single INSERT statement w/o explicit transaction with client generated GUID as a key:
INSERT MyTable
(id, ...)
VALUES (<client-app-generated-guid>, ...)
We also have a retry policies in-place, so that if command fails with SqlException, then it will be retried. SQL Server (Azure SQL) did not behave normally one day and we faced a lot of strange PK violation errors during retries. They were caused by retrying actually successfully committed on the SQL Server transaction (so that causes insert with already taken ID). I understand that SQL timeout it's purely client side concept, so if Client thinks that SqlCommand failed - it might or might not mean it.
I suspect that Client explicit transaction control via for instance wrapping statements with TransactionScope as shown bellow will fix 99% of such troubles -- because Commit is actually quite fast&cheap operation. However, I still see the caveat there -- the timeout also can happen on Committing stage. The application again can be in conditions where it's impossible to guess whether transaction really committed or not (to figure out necessity of retry).
The question is how to write code in bulletproof (to such kind of troubles) and generic fashion and do a retry only when it's positively clear that transaction was not committed.
using (var trx = new TransactionScope())
using (var con = GetOpenConnection(connectionString))
{
con.Execute("<some-non-idempotent-query>");
// what if Complete() times out?!
// to retry or not to retry?!
trx.Complete();
}
The problem is that the Exception does not mean that the transaction failed. For any compensating action (like retrying) you need to have a definite way of telling if it failed. There are scalability issues with what I will suggest, but its the technique that is the important thing, the scalability issues can be solved in other ways.
My solution;
the last INSERT before COMMIT is to write a Guid to a tracking table.
if an exception occurs, that indicates a network failure, SELECT ##TRANCOUNT. If it indicates you are still in a transaction (is greater than 0)(which probably should never happen, but its worth checking) then you can happily resubmit your COMMIT
If ##TRANCOUNT returns 0 then you are no longer in a transaction. Selecting your Guid from the tracking table will tell you whether your COMMIT was successful.
If your commit was not successful (##TRANCOUNT ==0 and your Guid is not present in the tracking table) then resubmit your entire batch from the BEGIN TRANSACTION onwards.
The general approach is: try to read back what you just tried to insert.
If you can read back the ID that you tried to insert, then previous transaction committed successfully, no need to retry.
If you can't find the ID that you tried to insert, then you know that your attempt to insert has failed, so you should retry.
I'm afraid there is no way to have a completely generic pattern that would work for any SQL statement. Your "checking" code needs to know what to look for.
If it is INSERT with ID - then you are looking for that ID.
If it is some UPDATE, then the check would be custom and depend on the nature of that UPDATE.
If it is DELETE, then the check consists of trying to read what was meant to be deleted.
Actually, here is a generic pattern: any data modification batch that has one or multiple INSERT, UPDATE, DELETE statements should have one more INSERT statement within that transaction that inserts some GUID (some ID of the data modifying transaction itself) into a dedicated audit table. Then your checking code tries to read that same GUID from that dedicated audit table. If GUID is found, then you know that previous transaction committed successfully. If GUID is not found, then you know that previous transaction was rolled back and you can retry.
Having this dedicated audit table unifies/standardize the checks. The checks no longer depend on internals and details of your data changing code. Your data modification code and verification code depend on the same agreed interface - audit table.
I have a C# program that will run nightly. It reads in records from flat files and does many different inserts and updates to a SQL Server database depending on the contents of each record.
Despite using transaction blocks and Try...Catch in stored procedures, and Try...Catch in C#, I really need these updates to be all or nothing.
I cannot take snapshots of db since we are using SQL Server 2008 R2 Express edition.
Can anyone please explain a good method (if there is one) for returning the SQL Server database to the state it was in before the nightly C# job ran, if the C# program catches an error? Basically, if the C# program catches an error, I want it to stop running, and I want to "rollback" the database to before the C# program did any changes. I need this to be an automated solution, not a manual restore from a backup.
In a sense, I want a Transaction block around the C# program that could rollback everything the C# program did.
The database is in Multi_User mode with a full recovery model (with nightly scheduled db and log backups). There is a web app tied to this database, but end users should really never have need to access this web app in the middle of the night.
I have searched everywhere for solutions, but no luck. Perhaps not using right keywords.
I look forward to feedback.
Thanks in advance,
David
I've gotten around this in the past by reading the file, inserting into a staging table, and then calling a single stored procedure to to the actual inserts into the production tables from the staging table.
If there is an error reading the file, truncate the staging table (or use an import identifier of some sort). If there is an error in the data, it should be easy to know that on the stored procedure side and easy to use one or two set based operations wrapped in a transaction.
This has the added benefit of being able to look at the data easily in the staging table if there was a data issue, possibly manually fixing it, and running the import stored procedure again.
I'm sorry, but that's what I thought database transactions were for. First you connect to the database, next you start a database transaction, and then you use the same connection & transaction for every operation:
using ( SqlConnection conn = new SqlConnection( . . . ) ) {
using ( SqlTransaction tran = new conn.BeginTransaction() ) {
try {
// Data processing operation #1
using ( SqlCommand command = new SqlCommand( ". . .", conn, tran ) {
// Your processing code here
}
// Repeat the pattern used for Data Processing Operation #1 for all other operations
// Commit the transaction if everything completed without error.
tran.Commit();
} catch ( SqlException ) {
// An error occurred. Roll the transaction back.
tran.Rollback();
// Other error handling code if needed.
}
}
}
Since there's only one transaction, everything is rolled back to the initial state if anything goes wrong. If you need to cancel the operation for other reasons, you call tran.Rollback() & return. This is exactly why transactions were invented; I don't understand why they won't do the job for you.
I got into an application which is using .NET/C# as front end and SQL Server 2008 as back end. I found that ALWAYS transactions are handled in the c# code. Seems its an unwritten rule for this project that we shouldn't use Transactions within stored procedure.
I personally feel that transactions should be handled within Stored Procedure as it would give more control over the code! We might have lot of validations happening within the script all this while we don't need a open transaction. We need to open a transaction just before we do a Insert/Update/Delete and can close it asap.
Looking for answers which would help me understand the best practice for handling transactions and when exactly do we need to opt for Transactions within Stored Proc / C#.
There isn't a hard and fast rule, but I see several reasons to control transactions from the business tier:
Communication across data store boundaries. Transactions don't have to be against a RDBMS; they can be against a variety of entities.
The ability to rollback/commit transactions based on business logic that may not be available to the particular stored procedure you are calling.
The ability to invoke an arbitrary set of queries within a single transaction. This also eliminates the need to worry about transaction count.
Personal preference: c# has a more elegant structure for declaring transactions: a using block. By comparison, I've always found transactions inside stored procedures to be cumbersome when jumping to rollback/commit.
We might have lot of validations happening within the script all this
while we don't need a open transaction. We need to open a transaction
just before we do a Insert/Update/Delete and can close it asap.
This may or may not be a problem depending on how many transactions are being opened (it's not clear if this is a single job, or a procedure which is run with high concurrency). I would suggest looking at what locks are being placed on objects, and how long those locks are being held.
Keep in mind that validation possibly should lock; what if the data changes between the time you validated it and the time the action occurs?
If it is a problem, you could break the offending procedure into two procedures, and call one from outside of a TransactionScope.
Greetings
I stumbled onto a problem today that seems sort of impossible to me, but its happening...I'm calling some database code in c# that looks something like this:
using(var tran = MyDataLayer.Transaction())
{
MyDataLayer.ExecSproc(new SprocTheFirst(arg1, arg2));
MyDataLayer.CallSomethingThatEventuallyDoesLinqToSql(arg1, argEtc);
tran.Commit();
}
I've simplified this a bit for posting, but whats going on is MyDataLayer.Transaction() makes a TransactionScope with the IsolationLevel set to Snapshot and TransactionScopeOption set to Required. This code gets called hundreds of times a day, and almost always works perfectly. However after reviewing some data I discovered there are a handful of records created by "SprocTheFirst" but no corresponding data from "CallSomethingThatEventuallyDoesLinqToSql". The only way that records should exist in the tables I'm looking at is from SprocTheFirst, and its only ever called in this one function, so if its called and succeeded then I would expect CallSomethingThatEventuallyDoesLinqToSql would get called and succeed because its all in the same TransactionScope. Its theoretically possible that some other dev mucked around in the DB, but I don't think they have. We also log all exceptions, and I can find nothing unusual happening around the time that the records from SprocTheFirst were created.
So, is it possible that a transaction, or more properly a declarative TransactionScope, with Snapshot isolation level can fail somehow and only partially commit?
We have spotted the same issue. I have recreated it here - https://github.com/DavidBetteridge/MSMQStressTest
For us we see the issue when reading from the queue rather than writing to it. Our solution was to change the isolation level of the first read in the subscriber to be serialised.
no, but snapshot isolation level isn't the same as serializable.
snapshoted rows are stored in the tempdb until the row commits.
so some other transaction can read the old data just fine.
at least that's how i understood your problem. if not please provide more info like a grapf of the timeline or something similar.
Can you verify that CallSomethingThatEventuallyDoesLinqToSQL is using the same Connection as the first call? Does the second call read data that the first filed into the db... and if it is unable to "see" that data would cause the second to skip a few steps and not do it's job?
Just because you have it wrapped in a .NET transaction doesn't mean the data as seen in the db is the same between connections. You could for instance have connections to two different databases and want to rollback both if one failed, or file data to a DB and post a message to MSMQ... if MSMQ operation failed it would roll back the DB operation too. .NET transaction would take care of this multi-technology feature for you.
I do remember a problem in early versions of ADO.NET (maybe 3.0) where the pooled connection code would allocate a new db connection rather than use the current one when a .NET level TransactionScope was used. I believe it was fully implemented with 3.5 (I may have my versions wrong.. might be 3.5 and 3.5.1). It could also be caused by the MyDataLayer and how it allocates connections.
Use SQL Profiler to trace these operations and make sure the work is being done on the same spid.
It sounds like your connection may not be enlisted in the transaction. When do you create your connectiion object? If it occurs before the TransactionScope then it will not be enlisted in the transaction.