I am relatively new to Entity Framework and have limited knowledge on entity frameworks/sqlservers connection pooling functionality. I have to implement a multithreaded operation that includes database changes.
The MainMethod looks for new database entries and queues up the tasks using ThreadPool.QueueUserWorkItem().
I am having database operations in the child threads too. For that I am creating the dbcontext object in the child thread.
I would like to know if the way I have implemented will have any performance issues. Will my child threads be creating new connection strings or reusing available ones. I have read that Entity framework makes use of the sql server pooling functionality but don't know how exactly.
public class DataProcessor
{
public void MainMethod(DataUow dataUow) //Entry point (Main thread)DataUow is the repository class
{
while (true)
{
try
{
var toDoMessageList = dataUow.MessageDumps.GetAll().ToList(); //Get Records
if (toDoMessageList.Count() > 0)
{
foreach (var message in toDoMessageList)
{
ThreadPool.QueueUserWorkItem(RunThread, message);
message.ThreadHandlingStatus = 2; //Do changes to message
dataUow.MessageDumps.Update(message); //and update
}
dataUow.Commit();
}
else
{
Thread.Sleep(500);
}
}
catch (Exception ex)
{
//Do Something
}
}
}
private void RunChildThread(object messageDataSent)
{
using (var dataContext = new DataUow(new RepositoryProvider(new RepositoryFactories())))
{
try
{
//Some Long Task Including Database changes
dataContext.Commit(); //Repository method that contains the entity framework .SaveChanges()
}
catch (Exception ex)
{
//Do Something
}
}
}
}
DataUow is the repository pattern class.
Thank you.
Related
Let's say I have an Entity which has a concurrency token column configured in EF core. When an exception occours because data the changed in the database, I'd like to retry the whole transaction from a clean context.
All of the examples use the following pattern:
using (var context = new PersonContext())
{
try
{
//Business logic
}
catch (DbUpdateConcurrencyException ex)
{
//Reload/merge entries in ex.Entries, etc...
}
}
The example is working, but how do you handle this scenario when the DbContext is registered as a scoped service, and it's injected into the repositories and you have a more complex scenario. I think it would be easier to retry the whole business process than handling the conflicts.
public class SomeService {
public SomeService (ISomeRepository repo)
{
}
public Task SomeMethod()
{
try
{
//Business logic
}
catch (DbUpdateConcurrencyException ex)
{
//clean context, restart SomeMethod()
}
}
}
What would be a clean way to handle the retry process?
You don't need to restart the whole DbContext.
Just refresh the entity you are modifying.
You can replace the original database value of the entity with it's current value of the database like this:
public class SomeService {
public SomeService (ISomeRepository repo)
{
}
public Task SomeMethod(Dto dto)
{
try
{
//Business logic
}
catch (DbUpdateConcurrencyException ex)
{
var entry = e.Entries.FirstOrDefault(x => x.Entity is <your entity type> ); // You can also check the Id.
if(null == entry)
throw e;
var dbValue = entry.GetDatabaseValues();
if (null == dbValue)
throw new Exception("The entity is deleted from the database in the meanwhile.");
entry.OriginalValues.SetValues(dbValue);
SomeMethod(dto);
}
}
}
EF checks the optimistic concurrency token with the original database values.
I am using this basic structure:
using(IDataContext ctx = DataContext.Instance())
{
try{
ctx.BeginTransaction();
ctx.Commit();
}
catch(Exception)
{
ctx.RollbackTransaction();
throw;
}
}
What I would like to do is to nest transactions so that I can build using functional programming.
Very simplified version:
public void DoSomething(){
using(IDataContext ctx = DataContext.Instance())
{
try{
ctx.BeginTransaction();
// make a change to the data
ctx.Commit();
}
catch(Exception)
{
ctx.RollbackTransaction();
throw;
}
}
}
public void CallingFunction(){
using(IDataContext ctx = DataContext.Instance())
{
try{
ctx.BeginTransaction();
//do some stuff
DoSomething();
//do some other stuff
ctx.Commit();
}
catch(Exception)
{
ctx.RollbackTransaction();
throw;
}
}
}
So I want to be able to have multiple 'CallingFunctions' that all call DoSomething(), but if there is an exception thrown in the code that comes in CallingFunction after DoSomething, then I want DoSomething to roll back also.
DoSomething may be in the same class as CallingFunction or it may be in another class.
Surely this is possible, but I haven't been able to find the answer.
Thank you for your assistance.
After checking my code, I realised that it is using DataContext from the DotNetNuke.Data namespace. Perhaps a DNN expert can assist?
You should inject the context in all the functions that need to use it, instead of having create their own. That way all changes before commit are on the same transaction and you can rollback everything if anything errors.
I'm trying to create dependent transactions in Entity Framework Core (v. 2.1).
The idea is to create "children" transactions inside a "father" transaction.
If one of the childs fails, so will have the father, too.
I've tried with TransactionScope, but I've got the following error.
Warning as error exception for warning
'Microsoft.EntityFrameworkCore.Database.Transaction.AmbientTransactionWarning':
An ambient transaction has been detected. Entity Framework Core does
not support ambient transactions. See
http://go.microsoft.com/fwlink/?LinkId=800142 To suppress this
Exception use the DbContextOptionsBuilder.ConfigureWarnings API.
ConfigureWarnings can be used when overriding the
DbContext.OnConfiguring method or using AddDbContext on the
application service provider.
The followings are controller's methods, the dbcontext is created and initialized in the constructor of the controller
Here's my code:
public bool father()
{
using (var transaction = new TransactionScope(TransactionScopeOption.Required))
{
try
{
//Do Stuff with the dbcontext
//(read, as "var x1 = _dbcontext.Database.Table1; etc...")...
child();
transaction.Complete();
//Do Stuff...
}
catch
{
//Do Stuff...
transaction.Dispose();
//Do Stuff...
}
}
}
}
public bool child()
{
using (var transaction2 = new TransactionScope(TransactionScopeOption.Required))
{
try
{
//Do other stuff with the dbcontext
//(read, as "var x2 = _dbcontext.Database.Table2; etc...")...
transaction2.Complete();
//Do Stuff...
}
catch
{
//Do Stuff...
transaction2.Dispose();
//Do Stuff...
}
}
}
}
How can I make it work?
I have a method which internally performs different sub-operations in an order and at failure of any of the sub operation i want to Rollback the entire operation.
My issue is the sub-operations are not all database operations. These are mainly system level changes like adding something in windows registry, creating a folder at a specified path and setting permissions etc. the sub-operations can be more than this.
want to do somthing like this;
CreateUser(){
CreateUserFtpAccount();
CreateUserFolder();
SetUserPermission();
CreateVirtualDirectoryForUser();
....
....
....
and many more
}
if last operation fails, i want to roll back all previous operations.
So, what is the standard way to do this? is there a design pattern do handle this?
Note: i'm using C#.net
Here's one way to do it:
Using the command pattern, you can create undoable actions. With each operation, you register the related commands, so that you can undo the executed commands when a fail condition occurs.
For example, this might all belong in a transaction-like context object that implements IDisposable and put in a using block. The undoable actions would be registered to this context object. On dispose, if not committed, "undo" is carried out for all registered commands. Hope it helps. The downside is you may have to convert some methods to classes. This might be a necessary evil though.
Code sample:
using(var txn = new MyTransaction()) {
txn.RegisterCommand(new CreateUserFtpAccountCommand());
txn.RegisterCommand(new CreateUserFolderCommand());
txn.RegisterCommand(new SetUserPermissionCommand());
txn.RegisterCommand(new CreateVirtualDirectoryForUserCommand());
txn.Commit();
}
class MyTransaction : IDisposable {
public void RegisterCommand(Command command){ /**/ }
public void Commit(){ /* Runs all registered commands */ }
public void Dispose(){ /* Executes undo for all registered commands */ }
}
class UndoableCommand {
public Command(Action action) { /**/ }
public void Execute() { /**/ }
public void Undo{ /**/ }
}
Update:
You mentioned that you have hundreds of such reversible operations. In this case, you can take a more functional approach and get rid of UndoableCommand completely. You would register delegates instead, like this:
using(var txn = new MyTransaction()) {
txn.Register(() => ftpManager.CreateUserAccount(user),
() => ftpManager.DeleteUserAccount(user));
txn.Register(() => ftpManager.CreateUserFolder(user, folder),
() => ftpManager.DeleteUserFolder(user, folder));
/* ... */
txn.Commit();
}
class MyTransaction : IDisposable {
public void Register(Action operation, Action undoOperation){ /**/ }
public void Commit(){ /* Runs all registered operations */ }
public void Dispose(){ /* Executes undo for all registered and attempted operations */ }
}
As a note, you'd need to be careful with closures with this approach.
I think your best bet would be to encapsulate the execution and reversal of each step of the process. It will make a lot more easily read code than nested try-catch blocks. Something like:
public interface IReversableStep
{
void DoWork();
void ReverseWork();
}
public void DoEverything()
{
var steps = new List<IReversableStep>()
{
new CreateUserFTPAccount(),
new CreateUserFolder(),
...
}
var completed = new List<IReversableStep>();
try
{
foreach (var step in steps)
{
step.DoWork();
completed.Add(step);
}
}
catch (Exception)
{
//if it is necessary to undo the most recent actions first,
//just reverse the list:
completed.Reverse();
completed.ForEach(x => x.ReverseWork());
}
}
Both NTFS and the Registry support enrollment in KTM and MS DTC transactions (and by extension, TransactionScope). However, the transactional file system has been deprecated due to complexity, and may not be in some future version of windows.
Example of using transactional file system and registry from C#
Transactional NTFS
If not everything fits in a transaction, I would certainly look to the command history patterns presented in other answers to this question.
I'm not aware of any standard pattern for this type of thing but I'd probably do it with nested try/catch blocks myself - with appropriate code for the rollback of non-database operations in the catch block. Use a TransactionScope to ensure all the database operations are transactionactional.
eg:
using (TransactionScope scope)
{
try
{
DoOperationOne();
try
{
DoOperationTwo();
DoDataBaseOperationOne(); // no need for try/catch surrounding as using transactionscope
try
{
DoOperationThree();
}
catch
{
RollBackOperationThree();
throw;
}
}
catch
{
RollBackOperationTwo();
throw;
}
}
catch
{
RollbackOperationOne();
throw;
}
scope.Complete(); // the last thing that happens, and only if there are no errors!
}
In Visual Studio 2008, C#, Entity Framework 3.5, MS Test, I have two tests:
[TestMethod]
public void Test1()
{
using (Entities context = new Entities())
{
using (TransactionHelper transaction = new TransactionHelper(context))
{
// Stuff
transaction.Rollback();
}
}
}
And:
[TestMethod]
public void Test2()
{
using (Entities context = new Entities())
{
using (TransactionHelper transaction = new TransactionHelper(context))
{
// Stuff
transaction.Rollback();
}
}
}
When running them individually, they both pass. But when I run all tests, it says the following for the second test:
System.InvalidOperationException: The relationship between the two
objects cannot be defined because they are attached to different
ObjectContext objects..
I do not understand this at all. Nowhere in the codepath do I create a new Entities object. I checked several times (and seeing as how the code works in indivual test runs and production use, that must be true). And I even wrapped the Entities in a using statement.
Is there some weird retention in the TestContext perhaps?
For reference, the TransactionHelper (simplified):
public class TransactionHelper : IDisposable
{
private global::System.Data.Objects.ObjectContext _context;
private ConnectionState _initialConState;
private DbTransaction _transaction;
public TransactionHelper(global::System.Data.Objects.ObjectContext context)
{
_context = context;
_initialConState = context.Connection.State;
if (_initialConState != ConnectionState.Open)
{
_context.Connection.Open(); // open connection if not already open
}
_transaction = _context.Connection.BeginTransaction(IsolationLevel.ReadUncommitted);
}
public void Rollback()
{
_transaction.Rollback();
}
public void Commit()
{
_transaction.Commit();
}
public void Dispose()
{
_transaction.Dispose();
if (_initialConState != ConnectionState.Open)
{
_context.Connection.Close(); // only close connection if not initially open
}
}
}
It makes no difference if I use isolation level ReadCommitted.
Since unit tests should ideally be totally independent of each other (in terms of state, order, etc), Visual Studio and other test frameworks such as NUnit speed up the execution process by running them in parallel. Shared static resources usually turned out to be the culprit for me in cases like yours.
It might be that these simple objects you mention (e.g some lookup values) get assigned to a property of an entity object, and then in the second test you assign the same simple object to an entity of another context, which would raise an error like you mentioned.
If this is the case then you might try mocking the cache in your tests (or maybe minimize the use of static).