Develop Reference

timeout when deleting user inside of transactionScope

Background
We are trying to archive old user data to keep our most common tables smaller.
Issue
Normal EF code for removing records works for our custom tables. The AspNetUsers table is a different story. It appears that the way to do it is using _userManager.Delete or _userManager.DeleteAsync. These work without trying to do multiple db calls in one transaction. When I wrap this in a transactionScope, it times out. Here is an example:
public bool DeleteByMultipleIds(List<string> idsToRemove)
{
try
{
using (var scope = new TransactionScope())
{
foreach (var id in idsToRemove)
{
var user = _userManager.FindById(id);
//copy user data to archive table
_userManager.Delete(user);//causes timeout
}
scope.Complete();
}
return true;
}
catch (TransactionAbortedException e)
{
Logger.Publish(e);
return false;
}
catch (Exception e)
{
Logger.Publish(e);
return false;
}
}
Note that while the code is running and I call straight to the DB like:
DELETE
FROM ASPNETUSERS
WHERE Id = 'X'
It will also time out. This SQL works before the the C# code is executed. Therefore, it appears that more than 1 db hit seems to lock the table. How can I find the user(db hit #1) and delete the user (db hit #2) in one transaction?

For me, the problem involved the use of multiple separate DbContexts within the same transaction. The BeginTransaction() approach did not work.
Internally, UserManager.Delete() is calling an async method in a RunSync() wrapper. Therefore, using the TransactionScopeAsyncFlowOption.Enabled parameter for my TransactionScope did work:
using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
_myContext1.Delete(organisation);
_myContext2.Delete(orders);
_userManager.Delete(user);
scope.Complete();
}

Advice from microsoft is to use a different API when doing transactions with EF. This is due to the interactions between EF and the TransactionScope class. Implicitly transaction scope is forcing things up to serializable, which causes a deadlock.
Good description of an EF internal API is here: MSDN Link
For reference you may need to look into user manager if it exposes the datacontext and replace your Transaction scope with using(var dbContextTransaction = context.Database.BeginTransaction()) { //code }
Alternatively, looking at your scenario, you are actually quite safe in finding the user ID, then trying to delete it and then just catching an error if the user has been deleted in the fraction of a second between finding it and deleting it.

How can I close connection string object in Entity Framework

Today I got an exception when I run my application from the below code
dbContext.ManageTasks.Where(x => x.IsDeleted == false).ToList();
Error is
Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
I got a solution from this discussion : How can I solve a connection pool problem between ASP.NET and SQL Server?
Solution is : if I close current connection string object, then this error will be gone.
SqlConnection myConnection = new SqlConnection(ConnectionString);
myConnection.Open();
// some code
myConnection.Close();
The above code for ADO.NET, not Entity Framework.
How can I close the connection string object in Entity Framework?

I'm a bit unsure as to why you would need the use of "SqlConnection". DbContext is a disposable class. I found the safest way to clean up is by using the "using" tag. E.g.
using(DbContext _context = new MyContext())
{
\\Your Work here
}
This will automatically call _context.Dispose() when it reaches the closing curly bracket which in effect closes your connection.
The alternative would be:
DbContext _context = new MyContext();
_context.ManageTasks.Where(x => x.IsDeleted == false).ToList();
_context.Dispose();
This does mean it's possible to miss the dispose call if an exception occurs.
Hope this helps.
Useful Link
In order to handle transaction scope and to help dispose your objects, you may want to look at the repository pattern and the unitOfWork pattern. Or both.
Further to this, I know use DependencyInjection to handle a lot of connections. I just ensure they are Scoped (per web request) :)

You are probably keeping a global variable dbContext somewhere in your class. You should not do that!
You could use the IDisposable interface that DbContext implements and use using statements. You initialize a new DbContext whenever you need one:
using (YourDbContext dbContext = ...)
{
// do some actions
}
// context will be closed
The benefit of using using is that it will close and dispose, even if the code inside will throw an exception.

TransactionScope and Entity Framework

I need to wrap some pieces of code around a TransactionScope. The code inside this using statement calls a managed C++ library, which will call some unmanaged code. I do also want to update my database, which is using Entity Framework.
Here comes the problem, when doing SaveChanges on the DbContext inside the TransactionScope I always get some sort of Timeout exception in the database layer. I've googled this, and it seems to be a fairly common problem but I haven't found any applicable answers to my problem. This is a snippet of my code
using (var transactionScope = new TransactionScope())
{
try
{
//Do call to the managed C++ Library
using (var dbContext = _dbContextFactory.Create())
{
//doing some CRUD Operations on the DbContext
//Probably some more dbContext related stuff
dbContext.SaveChanges(); //Results with a timeout
}
}
catch (Exception)
{
transactionScope.Dispose();
throw;
}
}
I'm using Entity Framework 6.1.3 so I can access the BeginTransaction on the database, but I also need wrap the C++ calls inside a TransactionScope.
Any suggestions?

You will need to pass in TransactionScopeOptions defining your timeout (How long to keep the transaction open). An example for the absolute upper limit of timeouts would be:
TransactionOptions to = new TransactionOptions();
to.IsolationLevel = IsolationLevel.ReadCommitted;
to.Timeout = TransactionManager.MaximumTimeout;
using (TransactionScope ts = new TransactionScope(TransactionScopeOption.Required, to)) { }
You should definitely be aware of the impact of such a long-running transaction, this isn't typically required, and I'd highly recommend using something below MaximumTimeout which reflects how long you expect it to run. You should do your best to keep the time period for which the transaction is held as small as possible, doing any processing that doesn't have to be a single transaction outside the transaction scope.
It's also worth noting that depending on the underlying database it can enforce it's own limitations on transaction durations if configured to do so.

How can I use Sql CE 4 databases for functional tests

Due to the potential differences between Linq-to-Entities (EF4) and Linq-to-Objects, I need to use an actual database to make sure my query classes retrieve data from EF correctly. Sql CE 4 seems to be the perfect tool for this however I have run into a few hiccups. These tests are using MsTest.
The problem I have is if the database doesn't get recreated (due to model changes), data keeps getting added to the database after each test with nothing getting rid of the data. This can potentially cause conflicts in tests, with more data being returned by queries than intended.
My first idea was to initialize a TransactionScope in the TestInitialize method, and dispose the transaction in TestCleanup. Unfortunately, Sql CE4 does not support transactions.
My next idea was to delete the database in TestCleanup via a File.Delete() call. Unfortunately, this seems to not work after the first test is run, as the first test's TestCleanup seems to delete the database, but every test after the first does not seem to re-create the database, and thus it gives an error that the database file is not found.
I attempted to change TestInitialize and TestCleanup tags to ClassInitialize and ClassCleanup for my testing class, but that errored with a NullReferenceException due to the test running prior to ClassInitialize (or so it appears. ClassInitialize is in the base class so maybe that's causing it).
I have run out of ways to effectively use Sql CE4 for testing. Does anyone have any better ideas?
Edit: I ended up figuring out a solution. In my EF unit test base class I initiate a new instance of my data context and then call context.Database.Delete() and context.Database.Create(). The unit tests run a tad slower, but now I can unit test effectively using a real database
Final Edit: After some emails back and forth with Microsoft, it turns out that TransactionScopes are now allowed in SqlCE with the latest release of SqlCE. However, if you are using EF4 there are some limitations in that you must explicitly open the database connection prior to starting the transaction. The following code shows a sample on how to successfully use Sql CE for unit/functional testing:
[TestMethod]
public void My_SqlCeScenario ()
{
using (var context = new MySQLCeModelContext()) //ß derived from DbContext
{
ObjectContext objctx = ((IObjectContextAdapter)context).ObjectContext;
objctx.Connection.Open(); //ß Open your connection explicitly
using (TransactionScope tx = new TransactionScope())
{
var product = new Product() { Name = "Vegemite" };
context.Products.Add(product);
context.SaveChanges();
}
objctx.Connection.Close(); //ß close it when done!
}
}

In your TestInitialize you should do the following:
System.Data.Entity.Database.DbDatabase.SetInitializer<YourEntityFrameworkClass>(
new System.Data.Entity.Database.DropCreateDatabaseAlways<YourEntityFrameworkClass>());
This will cause entity framework to always recreate the database whenever the test is run.
Incidentally you can create an alternative class that inherits from DropCreateDatabaseAlways. This will allow you to seed your database with set data each time.
public class DataContextInitializer : DropCreateDatabaseAlways<YourEntityFrameworkClass> {
protected override void Seed(DataContext context) {
context.Users.Add(new User() { Name = "Test User 1", Email = "test#test.com" });
context.SaveChanges();
}
}
Then in your Initialize you would change the call to:
System.Data.Entity.Database.DbDatabase.SetInitializer<YourEntityFrameworkClass>(
new DataContextInitializer());

I found the approach in the "final edit" works for me as well. However, it's REALLY annoying. It's not just for testing, but any time you want to use TransactionScope with Entity Framework and SQL CE. I want to code once and have my app support both SQL Server and SQL CE, but anywhere I use transactions I have to do this. Surely the Entity Framework team should have handled this for us!
In the meantime, I took it one step farther to make it a little cleaner in my code. Add this block to your data context (whatever class you derive from DbContext):
public MyDataContext()
{
this.Connection.Open();
}
protected override void Dispose(bool disposing)
{
if (this.Connection.State == ConnectionState.Open)
this.Connection.Close();
base.Dispose(disposing);
}
private DbConnection Connection
{
get
{
var objectContextAdapter = (IObjectContextAdapter) this;
return objectContextAdapter.ObjectContext.Connection;
}
}
This makes it a lot cleaner when you actually use it:
using (var db = new MyDataContext())
{
using (var ts = new TransactionScope())
{
// whatever you need to do
db.SaveChanges();
ts.Complete();
}
}
Although I suppose that if you design your app such that all changes are committed in a single call to SaveChanges(), then the implicit transaction would be good enough. For the testing scenario, we want to roll everything back instead of calling ts.Complete(), so it's certainly required there. I'm sure there are other scenarios where we need the transaction scope available. It's a shame it isn't supported directly by EF/SQLCE.

passing DB Connection object to methods

Was wondering if it is recomended to pass a database connection object around(to other modules) or let the method (in the other module) take care of setting it up. I am leaning toward letting the method set it up as to not have to check the state of the connection before using it, and just having the caller pass any needed data to the calling method that would be needed to setup the connection.

Personally I like to use tightly scoped connections; open them late, use them, and close them (in a "using" block, all within the local method). Connection pooling will deal with re-using the connection in most cases, so there is no real overhead in this approach.
The main advantage in passing connections used to be so that you could pass the transaction around; however, TransactionScope is a simpler way of sharing a transaction between methods.
Since the classes are implementation specific, I'd write each to open it's own native transaction. Otherwise, you can use the ado.net factory methods to create the appropriate type from the config file (the provider name).

Personally, I like storing a stack of my current open connection and transactions on top of the Thread Local Storage using SetData and GetData. I define a class that manages my connections to the database and allow it to use the dispose pattern. This saves me the need to pass connections and transactions around, which is something that I think clutters and complicates the code.
I would strongly recommend against leaving it up to the methods to open connections every time they need data. It will leads to a really bad situation where it is both hard to manage transactions throughout the application and too many connections are opened and closed (I know about connection pooling, it is still more expensive to look up a connection from the pool than it is to reuse an object)
So I end up having something along these lines (totally untested):
class DatabaseContext : IDisposable {
List<DatabaseContext> currentContexts;
SqlConnection connection;
bool first = false;
DatabaseContext (List<DatabaseContext> contexts)
{
currentContexts = contexts;
if (contexts.Count == 0)
{
connection = new SqlConnection(); // fill in info
connection.Open();
first = true;
}
else
{
connection = contexts.First().connection;
}
contexts.Add(this);
}
static List<DatabaseContext> DatabaseContexts {
get
{
var contexts = CallContext.GetData("contexts") as List<DatabaseContext>;
if (contexts == null)
{
contexts = new List<DatabaseContext>();
CallContext.SetData("contexts", contexts);
}
return contexts;
}
}
public static DatabaseContext GetOpenConnection()
{
return new DatabaseContext(DatabaseContexts);
}
public SqlCommand CreateCommand(string sql)
{
var cmd = new SqlCommand(sql);
cmd.Connection = connection;
return cmd;
}
public void Dispose()
{
if (first)
{
connection.Close();
}
currentContexts.Remove(this);
}
}
void Test()
{
// connection is opened here
using (var ctx = DatabaseContext.GetOpenConnection())
{
using (var cmd = ctx.CreateCommand("select 1"))
{
cmd.ExecuteNonQuery();
}
Test2();
}
// closed after dispose
}
void Test2()
{
// reuse existing connection
using (var ctx = DatabaseContext.GetOpenConnection())
{
using (var cmd = ctx.CreateCommand("select 2"))
{
cmd.ExecuteNonQuery();
}
}
// leaves connection open
}

For automated testing purposes, it's usually easier to pass it in. This is called dependency injection.
When you need to write tests, you can create a mock database connection object and pass that instead of the real one. That way, your automated tests won't rely on an actual database that needs to be repopulated with data every time.

I personally work to centralize my data access as much as possible, however, if not possible I ALWAYS open a new connection in the other classes, as I find that there are too many other things that can get in the way when passing the actual connection object.

Here is a little more insight into this problem. I have a class that manages db connections, and have 2 classes that implement an interface. One of the classes is for SQL and the other is of OLAP. The manager is the one that knows which connection to use, so it could pass the exact connection to the type, or the type can create his own connection.

You can pass connection objects without any problem (for instance Microsoft Enterprise Library allows static method calls passing in a connection) or you could manage it externally its up to your design, there are not direct technical tradeoffs.
Be careful for portability not to pass an specific connection if your solution will be ported to other databases (meaning don´t pass a SqlConnection it you plan to work with other databases)

Setting up the connection is potentially expensive and potentially adds a round trip. So, again, potentially, the better design is to pass the connection object.
I say potentially, because if you are a Microsoft ADO app, you are probably using a connection pool....

I would suggest that you distinguish between the connection object and its state (open, closed).
You can have a single method (or property) that reads the connection string from web.config. Using the same version of the connection string every time ensures that you will benefit from connection pooling.
Call that method when you need to open a connection. At the very last moment, after setting up all of the SqlCommand properties, open the connection, use it, and then close it. In C#, you can use the using statement to make sure the connection is closed. If not, be sure to close the connection in a finally block.

I would use the web.config
<configuration>
<connectionStrings>
<add name="conn1" providerName="System.Data.SqlClient" connectionString="string here" />
<add name="conn2" providerName="System.Data.SqlClient" connectionString="string here" />
</connectionStrings>
</configuration>
Then you can reference it from anywhere in the application