UPDATE based on accepted answer:
bool success = false;
using (var bulkCopy = new SqlBulkCopy(connection)) //using!
{
connection.Open();
//explicit isolation level is best-practice
using (var tran = connection.BeginTransaction(IsolationLevel.ReadCommitted))
{
bulkCopy.DestinationTableName = "table";
bulkCopy.ColumnMappings...
using (var dataReader = new ObjectDataReader<SomeObject>(paths))
{
bulkCopy.WriteToServer(dataReader);
success = true;
}
tran.Commit(); //commit, will not be called if exception escapes
}
}
return success;
I use BulkCopy class for large insert and it works fine.
After execute WriteToServer and saving data to database I wan't to know are all data saved successfully so I can return true/false because I need to save all or nothing?
var bulkCopy = new SqlBulkCopy(connection);
bulkCopy.DestinationTableName = "table";
bulkCopy.ColumnMappings...
using (var dataReader = new ObjectDataReader<SomeObject>(paths))
{
try
{
bulkCopy.WriteToServer(dataReader);
}
catch(Exception ex){ ... }
}
If the call to WriteToServer completed without exceptions, all rows were saved and are on disk. This is just the standard semantics for SQL Server DML. Nothing special with bulk copy.
Like all other DML, SqlBulkCopy is all-or-nothing as well. Except if you configure a batch size which you did not.
using (var bulkCopy = new SqlBulkCopy(connection)) //using!
{
connection.Open();
//explicit isolation level is best-practice
using (var tran = connection.BeginTransaction(IsolationLevel.ReadCommitted))
{
bulkCopy.DestinationTableName = "table";
bulkCopy.ColumnMappings...
using (var dataReader = new ObjectDataReader<SomeObject>(paths))
{
//try
//{
bulkCopy.WriteToServer(dataReader, /*here you set some options*/);
//}
//catch(Exception ex){ ... } //you need no explicit try-catch here
}
tran.Commit(); //commit, will not be called if exception escapes
}
}
I've added you sample code that I aligned with best-practices.
There is no direct way of identifying if the process was completed successfully or not, other than to look for/catch any exceptions raised by WriteToServer() method.
An alternative approach might be to check the number of records in the database, and then check the number of records after the process completes - The difference being the number that were inserted. Comparing this value against the numbers of records to be inserted could give an idea of failure or success. However, this is not fool proof particularly if there are other process inserting/deleting records.
However, these techniques in conjunction with TransactionScope - http://msdn.microsoft.com/en-us/library/system.transactions.transactionscope.aspx - or something similar should achieve what you require.
EDIT
By Default each insert operation is processed as a batch; if the operation fails in a particular batch then that batch is rolled back, not any inserted before it.
However, If an internal transaction is applied to the bulk operation than a failure in any row can roll back the entire result set. For Example;
using (SqlBulkCopy bulkCopy =
new SqlBulkCopy(connectionString, SqlBulkCopyOptions.KeepIdentity
| SqlBulkCopyOptions.UseInternalTransaction))
{
bulkCopy.BatchSize = 10;
bulkCopy.DestinationTableName = "dbo.BulkCopyDemoMatchingColumns";
try
{
bulkCopy.WriteToServer(reader);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
bulkCopy.Close();
}
}
An error in any of the above operation would cause the entire operation to roll back. See more details on this at http://msdn.microsoft.com/en-us/library/tchktcdk.aspx.
From the docs for this function, the following snippet suggests that you should catch any exceptions that are thrown and otherwise you can take it that the operation was successful.
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(connectionString))
{
bulkCopy.DestinationTableName = "dbo.BulkCopyDemoMatchingColumns";
try
{
// Write from the source to the destination.
bulkCopy.WriteToServer(reader);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
finally
{
// Close the SqlDataReader. The SqlBulkCopy
// object is automatically closed at the end
// of the using block.
reader.Close();
}
}
If you want to be super-sure, execute a query against the database to check the rows are there, after the bulk copy completes.
Related
The idea : I am trying to run an insertion to 2 databases using 2 different dbContext, the goal is to allow a role back on the insertion from both QBs in case of an exception from ether one of the insertions.
My code:
using (var db1 = new DbContext1())
{
db1.Database.Connection.Open();
using (var trans = db1.Database.Connection.BeginTransaction())
{
//do the insertion into db1
db1.SaveChanges();
using (var db2 = new DbContext2())
{
//do the insertions into db2
db2.SaveChanges();
}
trans.Commit();
}
}
On the first call to save changes: db1.SaveChanges(); I get an invalid operation exception : sqlconnection does not support parallel transactions
I tried figuring out what does it exactly mean, why does it happen and how to solve it but haven't been able to achieve that.
So my questions are:
What does it exactly mean? and why do I get this exception?
How can I solve it?
Is there a way to use the begin transaction is a different way that won't cause this error?
Also, is this the proper way to use begin transaction or should I do something different?
***For clarification, I am using the db1.Database.Connection.Open(); because otherwise I get an "connection is close" error.
Instead of trying to strech your connection and transaction across two DbContext, you may go for handling your connection and transaction outside of your DbContext, something like this :
using (var conn = new System.Data.SqlClient.SqlConnection("yourConnectionString"))
{
conn.Open();
using (var trans = conn.BeginTransaction())
{
try
{
using (var dbc1 = new System.Data.Entity.DbContext(conn, contextOwnsConnection: false))
{
dbc1.Database.UseTransaction(trans);
// do some work
// ...
dbc1.SaveChanges();
}
using (var dbc2 = new System.Data.Entity.DbContext(conn, contextOwnsConnection: false))
{
dbc2.Database.UseTransaction(trans);
// do some work
// ...
dbc2.SaveChanges();
}
trans.Commit();
}
catch
{
trans.Rollback();
}
}
}
I found out that I was simply abusing the syntax, so to help anyone who may stumble upon this question this is the proper way to do this:
using (var db1 = new DbContext1())
{
using (var trans = db1.Database.BeginTransaction())
{
try
{
//do the insertion into db1
db1.SaveChanges();
using (var db2 = new DbContext2())
{
//do the insertions into db2
db2.SaveChanges();
}
trans.Commit();
}
catch (Exception e)
{
trans.Rollback();
}
}
}
I'm using nested transactions by using IDBConecction interface on c#. I have to methods that insert data into 2 different tables, but when it comes to the second insert the first insert transaction locks the second one causing a timeout exception.
public void FirstInsert()
{
using (var cn = new Connection().GetConnection())
{
cn.Open();
using (var tran = cn.BeginTransaction())
{
try
{
//1st insert
SecondInsert() //calling second insert method
tran.Commit();
}
catch
{
tran.Rollback();
}
}
}
}
public void SecondInsert()
{
using (var cn = new Connection().GetConnection())
{
cn.Open();
using (var tran = cn.BeginTransaction())
{
try
{
//2nd insert, this one fails
tran.Commit();
}
catch
{
tran.Rollback();
}
}
}
}
When I check on SqlServer fisrt insert has the SPID 56, then when the second insert is being performed with SPID 57, and I use
exec sp_who2
In the column "BlkBy" for SPID 57 it says it is blocked by SPID 56.
How can I overcome these problem?
Use one connection for both operations. This likely involves passing the connection object around.
Usually, the connection+transaction per request pattern solves this issue well. Opening a connection in all kinds of methods is a code smell. It shows that the infrastructure fails to handle that.
You are doing correct but there is no need of a separate connection object and transaction object in your second method since call to the secondinsert() is already inside transaction scope. Your code can simply be
public void FirstInsert(){
using (var cn = new Connection().GetConnection()){
cn.Open();
using (var tran = cn.BeginTransaction()){
try{
//1st insert
SecondInsert() //calling second insert method
tran.Commit();
}
catch{
tran.Rollback();
}
}
}
}
public void SecondInsert(){
//perform second insert operation
}
}
}
I am inserting values into two tables using two stored procedure , and the data in both tables are linked to each other so i want if any error occurs in second stored procedure ,the data entered via 1st stored procedure should get roll backed.
I am using Sql server 2008 as my back end and ASP.net (c#) as front end
use need to use TransactionScope as below
using(var tran = new TransactionScope())
{
//calling stored procedures here
tran.Complete();
}
when an exception occurs the control will go out from the using and thus transaction will rollback
if you are using entity framework you can use it.
using (var dataContext = new SchoolMSDbContext())
{
using (var trans = dataContext.Database.BeginTransaction(IsolationLevel.ReadCommitted))
{
try
{
// your query
trans.Commit();
}
catch (Exception ex)
{
trans.Rollback();
Console.WriteLine(ex.InnerException);
}
}
}
Or you can try this
using (var dataContext = new SchoolMSDbContext())
{
using (var transaction = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
try
{
//your query
transaction.Complete();
}
catch (Exception ex)
{
transaction.Dispose();
Console.WriteLine(ex.InnerException);
}
}
}
for this you'll have to this references.
System.Transactions
for more information check this links
https://msdn.microsoft.com/en-us/data/dn456843.aspx
https://msdn.microsoft.com/en-us/library/2k2hy99x(v=vs.110).aspx
I have a windows services which is performing lot of Data processing. At some point, my service is crashed on closing a SQLConnection. When I comment the Close connection method call, Service is working consistently without crashing.
What could be the problem ? Below is a code snippet
private void DeleteTempTable()
{
_logger.Info("DeleteTempTable");
try
{
foreach (KeyValuePair<string, string> item in _stables)
{
string dropSql = string.Empty;
dropSql = string.Format("DROP TABLE [{0}];", item.Value);
SqlConnection oConnDropTables = new SqlConnection(connectionString);
oConnDropTables.Open();
if (!string.IsNullOrEmpty(dropSql))
{
using (SqlCommand cmd = new SqlCommand(dropSql, oConnDropTables))
{
cmd.ExecuteNonQuery();
}
}
if (oConnDropTables != null && oConnDropTables.State == ConnectionState.Open)
oConnDropTables.Close();
oConnDropTables = null;
}
}
catch (Exception ex)
{
_logger.Error("Error " + ex.Message);
throw ex;
}
}
When I comment the Close connection, service is working without crashing. Also it is not caught in the catch block. Also Connection is not Null and connectionstate is open only..
What I have tried:
1) Put "using" construct for connection - Didn't help
2) catching SQLException to check if anything I get- Didn't help
Removing that Close() should not make any problems go away, and frankly I don't believe it has. Since you don't yet understand the problem, it is premature to assume that a random code change has fixed it. Specifically:
with the Close(), it is returning the connection to the pool each time; when you call Open(), it will get back the same connection from the pool (cleansed, except for a few minor things)
without the Close(), the previous connection will be left to be garbage collected, which can cause either the connection-pool to become saturated, or the database-server's connection count to saturate; basically - bad things
What I suspect is happening is that you had some random error, that you now aren't seeing, by random. For example, network connectivity, or the unpredictable ordering of a Dictionary<,> (which means you don't know what order the tables are being DROPped, which is very important if there are foreign keys between them).
The only major problem with the current code is that it isn't using using. There are some redundant lines, though. This would be better:
foreach (var item in _stables)
{
var dropSql = string.Format("DROP TABLE [{0}];", item.Value);
using(var oConnDropTables = new SqlConnection(connectionString))
using (var cmd = new SqlCommand(dropSql, oConnDropTables))
{
oConnDropTables.Open();
cmd.ExecuteNonQuery();
}
}
or (preferable):
using(var oConnDropTables = new SqlConnection(connectionString))
{
oConnDropTables.Open();
foreach (var item in _stables)
{
var dropSql = string.Format("DROP TABLE [{0}];", item.Value);
using (var cmd = new SqlCommand(dropSql, oConnDropTables))
{
cmd.ExecuteNonQuery();
}
}
}
The issue is the creation of a new connection object each time the loop runs. When you close a SQL Connection, it is not actually closed but its returned to the app pool ready to be re-used. There is a limited number of connections you can open in SQL at once.
Try moving the SQLConnection object out of the loop and just execute commands in the loop and close the connection after the loop finishes.
private void DeleteTempTable()
{
_logger.Info("DeleteTempTable");
try
{
using(SqlConnection oConnDropTables = new SqlConnection(connectionString))
{
oConnDropTables.Open();
foreach (KeyValuePair<string, string> item in _stables)
{
string dropSql = string.Empty;
dropSql = string.Format("DROP TABLE [{0}];", item.Value);
if (!string.IsNullOrEmpty(dropSql))
{
using (SqlCommand cmd = new SqlCommand(dropSql, oConnDropTables))
{
cmd.ExecuteNonQuery();
}
}
}
}
}
catch (Exception ex)
{
_logger.Error("Error " + ex.Message);
throw ex;
}
}
Im making a system which should be running 24/7, with timers to control it. There are many calls to the database, and at some point, two methods are trying to open a connection, and one of them will fail. I've tried to make a retry method, so my methods would succeed. With the help from Michael S. Scherotter and Steven Sudit's methods in Better way to write retry logic without goto, does my method look like this:
int MaxRetries = 3;
Product pro = new Product();
SqlConnection myCon = DBcon.getInstance().conn();
string barcod = barcode;
string query = string.Format("SELECT * FROM Product WHERE Barcode = #barcode");
for (int tries = MaxRetries; tries >= 0; tries--) //<-- 'tries' at the end, are unreachable?.
{
try
{
myCon.Open();
SqlCommand com = new SqlCommand(query, myCon);
com.Parameters.AddWithValue("#barcode", barcode);
SqlDataReader dr = com.ExecuteReader();
if (dr.Read())
{
pro.Barcode = dr.GetString(0);
pro.Name = dr.GetString(1);
}
break;
}
catch (Exception ex)
{
if (tries == 0)
Console.WriteLine("Exception: "+ex);
throw;
}
}
myCon.Close();
return pro;
When running the code, the program stops at the "for(.....)", and the exception: The connection was not closed. The connection's current state is open... This problem was the reason why I'm trying to make this method! If anyone knows how to resovle this problem, please write. Thanks
You do
myCon.Open();
inside the for loop, but
myCon = DBcon.getInstance().conn();
outside of it. This way you try to open the same connection multiple times. If you want to protect against loss of DB connection you need to put both inside teh loop
You should move the call to myCon.Open outside the for statement or wrap myCon.Open() checking the connection state before re-opening the connection:
if (myCon.State != ConnectionState.Open)
{
myCon.Open();
}
Edited for new information
How about using Transactions to preserve data integrity, getting on-the-fly connections for multiple access and wrapping them in Using statements to ensure connections are closed? eg
Using (SqlConnection myCon = new SqlConnection('ConnectionString'))
{
myCon.Open();
var transaction = myCon.BeginTransaction();
try
{
// ... do some DB stuff - build your command with SqlCommand but use your transaction and your connection
var sqlCommand = new SqlCommand(CommandString, myCon, transaction);
sqlCommand.Parameters.Add(new Parameter()); // Build up your params
sqlCommand.ExecuteNonReader(); // Or whatever type of execution is best
transaction.Commit(); // Yayy!
}
catch (Exception ex)
{
transaction.RollBack(); // D'oh!
// ... Some logging
}
myCon.Close();
}
This way even if you forget to Close the connection, it will still be done implicitly when the connection gets to the end of its Using statement.
Have you tried adding
myCon.Close();
Into a Finally block. It looks like it is never being hit if you have an exception. I would highly recommend that you wrap the connection, command object etc in Using statements. This will ensure they are disposed of properly and the connection is closed.