Develop Reference

TransactionScope and Entity Framework

I need to wrap some pieces of code around a TransactionScope. The code inside this using statement calls a managed C++ library, which will call some unmanaged code. I do also want to update my database, which is using Entity Framework.
Here comes the problem, when doing SaveChanges on the DbContext inside the TransactionScope I always get some sort of Timeout exception in the database layer. I've googled this, and it seems to be a fairly common problem but I haven't found any applicable answers to my problem. This is a snippet of my code
using (var transactionScope = new TransactionScope())
{
try
{
//Do call to the managed C++ Library
using (var dbContext = _dbContextFactory.Create())
{
//doing some CRUD Operations on the DbContext
//Probably some more dbContext related stuff
dbContext.SaveChanges(); //Results with a timeout
}
}
catch (Exception)
{
transactionScope.Dispose();
throw;
}
}
I'm using Entity Framework 6.1.3 so I can access the BeginTransaction on the database, but I also need wrap the C++ calls inside a TransactionScope.
Any suggestions?

You will need to pass in TransactionScopeOptions defining your timeout (How long to keep the transaction open). An example for the absolute upper limit of timeouts would be:
TransactionOptions to = new TransactionOptions();
to.IsolationLevel = IsolationLevel.ReadCommitted;
to.Timeout = TransactionManager.MaximumTimeout;
using (TransactionScope ts = new TransactionScope(TransactionScopeOption.Required, to)) { }
You should definitely be aware of the impact of such a long-running transaction, this isn't typically required, and I'd highly recommend using something below MaximumTimeout which reflects how long you expect it to run. You should do your best to keep the time period for which the transaction is held as small as possible, doing any processing that doesn't have to be a single transaction outside the transaction scope.
It's also worth noting that depending on the underlying database it can enforce it's own limitations on transaction durations if configured to do so.

Correct Implementation of Transient Fault Handling (Azure)

For the past day or so I've been trying to implement Transient Fault Handling on an Azure SQL database. Although I have a working connection to the DB, I'm not convinced that it's handling the transient faults as expected.
So far my approach has involved
public static void SetRetryStratPol()
{
const string defaultRetryStrategyName = "default";
var strategy = new Incremental(defaultRetryStrategyName, 3, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(2));
var strategies = new List<RetryStrategy> { strategy };
var manager = new RetryManager(strategies, defaultRetryStrategyName);
RetryManager.SetDefault(manager);
retryPolicy = new RetryPolicy<SqlDatabaseTransientErrorDetectionStrategy>(strategy);
retryPolicy.Retrying += (obj, eventArgs) =>
{
var msg = String.Format("Retrying, CurrentRetryCount = {0} , Delay = {1}, Exception = {2}", eventArgs.CurrentRetryCount, eventArgs.Delay, eventArgs.LastException.Message);
System.Diagnostics.Debug.WriteLine(msg);
};
}
I call that method from the Global.asax's, Application_Start(). [retryPolicy is a global static variable on a static class which also includes this next method.]
I then have a method
public static ReliableSqlConnection GetReliableConnection()
{
var conn = new ReliableSqlConnection("Server=...,1433;Database=...;User ID=...;Password=...;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;", retryPolicy);
conn.Open();
return conn;
}
I then use this method
using (var conn = GetReliableConnection())
using (var cmd = conn.CreateCommand())
{
cmd.CommandText = "SELECT COUNT(*) FROM ReliabilityTest";
result = (int) cmd.ExecuteScalarWithRetry();
return View(result);
}
So far, this works. Then, in order to test the retry policy, I've tried using a wrong username (a suggestion from here).
But when I step through that code the cursor immediately jumps to my catch statement with
Login failed for user '[my username]'.
I would have expected that this exception only be caught after several seconds, but no delay is incurred at all.
Furthermore, I've also tried with the Entity Framework, following exactly this post, but get the same result.
What have I missed? Is there a configuration step or am I incorrectly inducing a transient fault?

Transient Fault Handling block is for handling transient errors. Failed login because of incorrect username/password is certainly not one of them. From this web page: http://msdn.microsoft.com/en-us/library/dn440719%28v=pandp.60%29.aspx:
What Are Transient Faults?
When an application uses a service, errors can occur because of
temporary conditions such as intermittent service,
infrastructure-level faults, network issues, or explicit throttling by
the service; these types of error occur more frequently with
cloud-based services, but can also occur in on-premises solutions. If
you retry the operation a short time later (maybe only a few
milliseconds later) the operation may succeed. These types of error
conditions are referred to as transient faults. Transient faults
typically occur very infrequently, and in most cases, only a few
retries are necessary for the operation to succeed.
You may want to check the source code for this application block (http://topaz.codeplex.com/) and see what error codes returned from SQL databases are considered transient errors and are thus retried.
You can always extend the functionality and include failed login as one of the transient error to test your code.
UPDATE
Do take a look at the source code here: http://topaz.codeplex.com/SourceControl/latest#source/Source/TransientFaultHandling.Data/SqlDatabaseTransientErrorDetectionStrategy.cs. This is where the retry magic happens. What you could do is create a class (let's call it CustomSqlDatabaseTransientErrorDetectionStrategy) and copy the entire code from the link to this class). Then for testing purpose, you can add login failed scenario as one of the transient error and use this class in your application instead of SqlDatabaseTransientErrorDetectionStrategy.

Nested Transaction Scope for transactional Operation

I've got a doubt about transactionscope because I'd like to make a transactional operation where first I perform some CRUD operations (a transaction which inserts and updates some rows on the DataBase) and I got a result from the whole transaction (an XML).
After I got the XML I send the XML to a Web Service which my customer exposes to integrate my system with.
The point is, let's imagine that one day the WS that my customer exposes falls down due to a weekly or monthly support task that its IT Area perform, so everymoment I perform the whole thing It performs the DB operation but of course It will throw an exception at the moment that I try to call the WS.
After Searching on the Internet I started to think of Transaction Scope. My Data Access Method which is on my Data Access Layer already has a TransactionScope where I perform insert, update, delete, etc.
The following Code is what I'd like to try:
public void ProcessSomething()
{
using (TransactionScope mainScope = new TransactionScope())
{
FooDAL dl = new FooDAL();
string message = dl.ProcessTransaction();
WSClientFoo client = new WSClientFoo();
client.SendTransactionMessage(message);
mainScope.Complete();
}
}
public class FooDAL
{
public string ProcessTransaction()
{
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Required, new TransactionOptions(){ IsolationLevel = IsolationLevel.ReadCommitted}))
{
///Do Insert, Update, Delete and According to the Operation Generates a message
scope.Complete();
}
return transactionMessage;
}
}
The question is, is it correct to use TransactionScope to handle what I want to do ?
Thanks a lot for your time :)

TransactionScopeOption.Required in your FooDAL.ProcessTransaction method means in fact: if there is a transaction available, reuse it in this scope; otherwise, create a new one.
So in short: yes, this is the correct way of doing this.
But be advised that if you don't call scope.Complete() in FooDAL.ProcessTransaction, a call to mainScope.Complete() will crash with a 'TransactionAbortedException' or something like that, which makes sense: if a nested scope decides that the transaction cannot be committed the outer scope should not be able to commit it.

Transaction deadlocks, how to design properly?

So I'm working on this Entity Framework project that'll be used as kind of a DAL and when running stress tests (starting a couple of updates on entities through Thread()'s) and I'm getting these:
_innerException = {"Transaction (Process ID 94) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction."}
Here's some example of how I implemented my classes' methods:
public class OrderController
{
public Order Select(long orderID)
{
using (var ctx = new BackEndEntities())
{
try
{
var res = from n in ctx.Orders
.Include("OrderedServices.Professional")
.Include("Agency")
.Include("Agent")
where n.OrderID == orderID
select n;
return res.FirstOrDefault();
}
catch (Exception ex)
{
throw ex;
}
}
}
public bool Update(Order order)
{
using (var ctx = new BackEndEntities())
{
try
{
order.ModificationDate = DateTime.Now;
ctx.Orders.Attach(order);
ctx.SaveChanges();
return true;
}
catch (Exception ex)
{
throw ex;
}
}
}
}
and:
public class AgentController
{
public Agent Select(long agentID)
{
using (var ctx = new BackEndEntities())
{
try
{
var res = from n in ctx.Agents.Include("Orders")
where n.AgentID == agentID
select n;
return res.FirstOrDefault();
}
catch (Exception ex)
{
throw ex;
}
}
}
public bool Update(Agent agent)
{
using (var ctx = new BackEndEntities())
{
try
{
agent.ModificationDate = DateTime.Now;
ctx.Agents.Attach(agent);
ctx.ObjectStateManager.ChangeObjectState(agent, System.Data.EntityState.Modified);
ctx.SaveChanges();
return true;
}
catch (Exception ex)
{
throw ex;
}
}
}
}
Obviously, the code here probably could be better but I'm rather of an EF newbie. But I think my problem is rather a design problem with the context.
I remember someone here mentioning that if my context is NOT shared, I won't run into these deadlock issues.
This does not seem 'shared' to me as I do a using new BackEndEntities() in each method, so what do I have to change to make it more robust ?
This DAL will be used in a web service exposed on the internet (after code review of coure) so I have no control on how much it'll be stressed and lots of different instances might want to update the same entity.
Thanks!

The reason for thouse deadlocks isn't your code but due to EF that is using SERIALIZABLE for default TransactionScope isolation level.
SERIALIZABLE is the most restricted locking possible, this means that you are by default opting into the most restrictive isolation level, and you can expect a lot of locking!
The solution is to specify another TransactionScope depending on the action you want to perform. You can surround your EF actions with something like this:
using (var scope = new TransactionScope(TransactionScopeOption.Required, new
TransactionOptions { IsolationLevel= IsolationLevel.Snapshot }))
{
// do something with EF here
scope.Complete();
}
Read more on this issue:
http://blogs.msdn.com/b/diego/archive/2012/04/01/tips-to-avoid-deadlocks-in-entity-framework-applications.aspx
http://blogs.u2u.be/diederik/post/2010/06/29/Transactions-and-Connections-in-Entity-Framework-40.aspx
http://blog.aggregatedintelligence.com/2012/04/sql-server-transaction-isolation-and.html
https://serverfault.com/questions/319373/sql-deadlocking-and-timing-out-almost-constantly

Deadlock freedom is a pretty hard problem in a big system. It has nothing to do with EF by itself.
Shortening the lifetime of your transactions reduces deadlocks but it introduces data inconsistencies. In those places where you were deadlocking previously you are now destroying data (without any notification).
So choose your context lifetime and your transaction lifetime according to the logical transaction, not according to physical considerations.
Turn on snapshot isolation. This takes reading transactions totally out of the equation.
For writing transactions you need to find a lock ordering. Often it is the easiest way to lock pessimistically and at a higher level. Example: Are you always modifying data in the context of a customer? Take an update lock on that customer as the first statement of your transactions. That provides total deadlock freedom by serializing access to that customer.

The context is what gives entity its ability to talk to the database, without a context there's no concept of what goes where. Spinning up a context, therefore, is kind of a big deal and it occupies a lot of resources, including external resources like the database. I believe your problem IS the 'new' command, since you would have multiple threads attempting to spin up and grab the same database resource, which definitely would deadlock.
Your code as you've posted it seems to be an anti-pattern. The way it looks, you have your Entity Context spinning up and going out of scope relatively quickly, while your repository CRUD objects seem to be persisting for a much longer time.
The way the companies I have implemented Entity for have traditionally done it exactly the opposite way - the Context is created and is kept for as long as the assembly has need of database, and the repository CRUD objects are created and die in microseconds.
I cannot say where you got your assertion of the context not being shared from so I dunno what circumstances that was said under, but it is absolutely true that you should not share the context across assemblies. Among the same assembly I cannot see any reason why you wouldn't with how many resources it takes to start up a context, and how long it takes to do so. The Entity Context is quite heavy, and if you were to make your current code work by going single-threaded I suspect you would see some absolutely atrocious performance.
So what I would recommend instead is to refactor this so you have Create(BackEndEntites context) and Update(BackEndEntities context), then have your master thread (the one making all these child threads) create and maintain a BackEndEntities context to pass along to its children. Also be sure that you get rid of your AgentControllers and OrderControllers the instant you're done with them and never, ever, ever reuse them outside of a method. Implementing a good inversion of control framework like Ninject or StructureMap can make this a lot easier.

How can I use Sql CE 4 databases for functional tests

Due to the potential differences between Linq-to-Entities (EF4) and Linq-to-Objects, I need to use an actual database to make sure my query classes retrieve data from EF correctly. Sql CE 4 seems to be the perfect tool for this however I have run into a few hiccups. These tests are using MsTest.
The problem I have is if the database doesn't get recreated (due to model changes), data keeps getting added to the database after each test with nothing getting rid of the data. This can potentially cause conflicts in tests, with more data being returned by queries than intended.
My first idea was to initialize a TransactionScope in the TestInitialize method, and dispose the transaction in TestCleanup. Unfortunately, Sql CE4 does not support transactions.
My next idea was to delete the database in TestCleanup via a File.Delete() call. Unfortunately, this seems to not work after the first test is run, as the first test's TestCleanup seems to delete the database, but every test after the first does not seem to re-create the database, and thus it gives an error that the database file is not found.
I attempted to change TestInitialize and TestCleanup tags to ClassInitialize and ClassCleanup for my testing class, but that errored with a NullReferenceException due to the test running prior to ClassInitialize (or so it appears. ClassInitialize is in the base class so maybe that's causing it).
I have run out of ways to effectively use Sql CE4 for testing. Does anyone have any better ideas?
Edit: I ended up figuring out a solution. In my EF unit test base class I initiate a new instance of my data context and then call context.Database.Delete() and context.Database.Create(). The unit tests run a tad slower, but now I can unit test effectively using a real database
Final Edit: After some emails back and forth with Microsoft, it turns out that TransactionScopes are now allowed in SqlCE with the latest release of SqlCE. However, if you are using EF4 there are some limitations in that you must explicitly open the database connection prior to starting the transaction. The following code shows a sample on how to successfully use Sql CE for unit/functional testing:
[TestMethod]
public void My_SqlCeScenario ()
{
using (var context = new MySQLCeModelContext()) //ß derived from DbContext
{
ObjectContext objctx = ((IObjectContextAdapter)context).ObjectContext;
objctx.Connection.Open(); //ß Open your connection explicitly
using (TransactionScope tx = new TransactionScope())
{
var product = new Product() { Name = "Vegemite" };
context.Products.Add(product);
context.SaveChanges();
}
objctx.Connection.Close(); //ß close it when done!
}
}

In your TestInitialize you should do the following:
System.Data.Entity.Database.DbDatabase.SetInitializer<YourEntityFrameworkClass>(
new System.Data.Entity.Database.DropCreateDatabaseAlways<YourEntityFrameworkClass>());
This will cause entity framework to always recreate the database whenever the test is run.
Incidentally you can create an alternative class that inherits from DropCreateDatabaseAlways. This will allow you to seed your database with set data each time.
public class DataContextInitializer : DropCreateDatabaseAlways<YourEntityFrameworkClass> {
protected override void Seed(DataContext context) {
context.Users.Add(new User() { Name = "Test User 1", Email = "test#test.com" });
context.SaveChanges();
}
}
Then in your Initialize you would change the call to:
System.Data.Entity.Database.DbDatabase.SetInitializer<YourEntityFrameworkClass>(
new DataContextInitializer());

I found the approach in the "final edit" works for me as well. However, it's REALLY annoying. It's not just for testing, but any time you want to use TransactionScope with Entity Framework and SQL CE. I want to code once and have my app support both SQL Server and SQL CE, but anywhere I use transactions I have to do this. Surely the Entity Framework team should have handled this for us!
In the meantime, I took it one step farther to make it a little cleaner in my code. Add this block to your data context (whatever class you derive from DbContext):
public MyDataContext()
{
this.Connection.Open();
}
protected override void Dispose(bool disposing)
{
if (this.Connection.State == ConnectionState.Open)
this.Connection.Close();
base.Dispose(disposing);
}
private DbConnection Connection
{
get
{
var objectContextAdapter = (IObjectContextAdapter) this;
return objectContextAdapter.ObjectContext.Connection;
}
}
This makes it a lot cleaner when you actually use it:
using (var db = new MyDataContext())
{
using (var ts = new TransactionScope())
{
// whatever you need to do
db.SaveChanges();
ts.Complete();
}
}
Although I suppose that if you design your app such that all changes are committed in a single call to SaveChanges(), then the implicit transaction would be good enough. For the testing scenario, we want to roll everything back instead of calling ts.Complete(), so it's certainly required there. I'm sure there are other scenarios where we need the transaction scope available. It's a shame it isn't supported directly by EF/SQLCE.