Good morning!
I have long running async transcation with fails with some different errors.
THere is method with run the transaction
public T PerformInTransaction<T>(Func<ITransaction> beginTransaction, Func<ITransaction, T> func)
{
try
{
AzureDbConfiguration.SuspendExecutionStrategy = true;
using (var transaction = beginTransaction())
{
return func(transaction);
}
}
finally
{
AzureDbConfiguration.SuspendExecutionStrategy = false;
}
}
The ITranscation interface is implemented as default of EF DbContextTransaction class as below:
public class Transaction : ITransaction
{
private readonly DbContextTransaction _transaction;
public Transaction(DbContextTransaction transaction)
{
_transaction = transaction;
}
public Task CommitAsync()
{
this.Commit();
return Task.CompletedTask;
}
.... more methods
}
I have preety long operation running in transcation where inside I have loop with another async methods and each method might call SaveChanges(). Snippet basically looks like
await _unitOfWork.PerformInTransaction(async transaction =>
{
foreach()
{
if(await Action())
_unitOfWork.SaveChanges()
else
transaction.Rollback()
}
transaction.Commit()
})
The errors I have
System.InvalidOperationException: Invalid attempt to call IsDBNull
when reader is closed. at
System.Data.SqlClient.SqlDataReader.CheckHeaderIsReady(Int32
columnIndex, Boolean permitAsync, String methodName) at
System.Data.SqlClient.SqlDataReader.IsDBNull(Int32 i) at
lambda_method(Closure , Shaper ) at
System.Data.Entity.Core.Common.Internal.Materialization.Coordinator1.ReadNextElement(Shaper
shaper) at
System.Data.Entity.Core.Common.Internal.Materialization.Shaper1.SimpleEnumerator.d__4.MoveNext()
or another
System.Data.Entity.Core.EntityException: The underlying provider
failed on Commit. ---> System.ArgumentNullException: Value cannot be
null. Parameter name: connection at
System.Data.Entity.Utilities.Check.NotNull[T](T value, String
parameterName) at
System.Data.Entity.Infrastructure.Interception.DbTransactionInterceptionContext.WithConnection(DbConnection
connection)
Do I do something wrong with async transcations? Is it related to some timouts ?
Thanks in advance
Related
Is it possible to use EF Core interceptors for catching database connection issues, like invalid username/password, update them by using an API call and re-try the same query?
Perhaps perform a check once in your app before running current code, here is a language extension.
public static class Extensions
{
public static async Task<(bool success, Exception exception)> CanConnectAsync(this DbContext context)
{
try
{
var result = await Task.Run(async () => await context.Database.CanConnectAsync());
return (result, null);
}
catch (Exception localException)
{
return (false, localException);
}
}
}
Usage
public static async Task Demo()
{
await using var cn = new CustomerContext();
var (success, exception) = await cn.CanConnectAsync();
if (success)
{
// connected
}
else
{
// examine exception variable
}
}
Has ToListAsync been depricated in EntityFrameWorkCore on EF core 6?
If I use the ToList() method it works, but if I add ToListAsync I get the following erro:
'IOrderedEnumerable' does not contain a definition for 'ToListAsync' and no accessible extension method 'ToListAsync' accepting a first argument of type 'IOrderedEnumerable' could be found (are you missing a using directive or an assembly reference?
Sample of the code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using Microsoft.EntityFrameworkCore;
using System.Threading.Tasks;
namespace RainbowOF.Repositories.Common
{
public class AppRepository<TEntity> : IAppRepository<TEntity> where TEntity : class
{
#region Privates
private ApplicationDbContext _Context { get; set; } // = null;
private DbSet<TEntity> _Table = null;
private ILoggerManager _Logger { get; }
private IAppUnitOfWork _AppUnitOfWork { get; set; }
#endregion
#region Initialise
public AppRepository(ApplicationDbContext dbContext, ILoggerManager logger, IAppUnitOfWork unitOfWork)
{
_Context = dbContext;
_Table = dbContext.Set<TEntity>();
_Logger = logger;
_AppUnitOfWork = unitOfWork;
}
#endregion
public IEnumerable<TEntity> GetAllOrderBy(Func<TEntity, object> orderByExpression, bool sortDesc = false)
{
_Logger.LogDebug($"Getting all records in Table of type: {typeof(TEntity)} order by {orderByExpression}");
if (_AppUnitOfWork.DBTransactionIsStillRunning())
_Logger.LogDebug("Second transaction started before current transaction completed!");
try
{
var _result = sortDesc
? _Table.OrderByDescending(orderByExpression).ToList()
: _Table.OrderBy(orderByExpression).ToList();
return _result;
}
catch (Exception ex)
{
_AppUnitOfWork.LogAndSetErrorMessage($"!!!Error Getting all (async): {ex.Message} - Inner Exception {ex.InnerException}");
#if DebugMode
throw; // #Debug?
#endif
}
return null;
}
public async Task<IEnumerable<TEntity>> GetAllAsync()
{
_Logger.LogDebug($"Getting all records (async) in Table of type: {typeof(TEntity)}");
if (_AppUnitOfWork.DBTransactionIsStillRunning())
_Logger.LogDebug("Second transaction started before current transaction completed!");
try
{
var _result = await _Table.ToListAsync();
return _result;
}
catch (Exception ex)
{
_AppUnitOfWork.LogAndSetErrorMessage($"!!!Error Getting all (async): {ex.Message} - Inner Exception {ex.InnerException}");
#if DebugMode
throw; // #Debug?
#endif
}
return null;
}
//--> the above works fine, below gives the error.
public async Task<IEnumerable<TEntity>> GetAllOrderByAsync(Func<TEntity, object> orderByExpression, bool sortDesc = false)
{
_Logger.LogDebug($"Getting all records (async) in Table of type: {typeof(TEntity)} order by {orderByExpression}");
if (_AppUnitOfWork.DBTransactionIsStillRunning())
_Logger.LogDebug("Second transaction started before current transaction completed!");
try
{
var _result = sortDesc
? await _Table.OrderByDescending(orderByExpression).ToListAsync()
: await _Table.OrderBy(orderByExpression).ToListAsync();
return _result;
}
catch (Exception ex)
{
_AppUnitOfWork.LogAndSetErrorMessage($"!!!Error Getting all (async) order by: {ex.Message} - Inner Exception {ex.InnerException}");
#if DebugMode
throw; // #Debug?
#endif
}
return null;
}
I have tried adding in the above.
I Google and a few tickets said Async is added in 6, and some suggested adding "using System.Data.Entity;" to the usings.
If I add using System.Data.Entity; then that error goes away, but then I get conflicts with EntityFrameworkCore.
The project has been upgraded to .net 6 in VS2022. It was working in .net 5
Perhaps I am confusing technologies.
OrderBy<TSource,TKey>(IEnumerable<TSource>, Func<TSource,TKey>) which accepts Func is a method defined for IEnumerable which does have async version of ToList. You need OrderBy<TSource,TKey>(IQueryable<TSource>, Expression<Func<TSource,TKey>>)) which is defined for IQueryable which is extended with ToListAsync (you also want it cause IQueryable overloads are actually translated into SQL while IEnumerable ones will fetch all the data to the client side and sort it on there). So change your method signature to accept expression of func (Expression<Func<TEntity, object>>) not just func:
public async Task<IEnumerable<TEntity>> GetAllOrderByAsync(Expression<Func<TEntity, object>> orderByExpression, bool sortDesc = false)
{
...
}
Extra info:
Exrpression vs Func
IQueryable vs IEnumerable
I am developing a .Net core application with Hangfire and facing the below exception
A second operation started on this context before a previous operation completed. Any instance members are not guaranteed to be thread safe.
I have used Hangfire for scheduling the jobs with 1 hour interval. I am facing the above issue when the new process/job gets started before the earlier job has finished its process.
How can we implement multiple Hangfire processes/jobs(multiple workers) to work (in parallel) to accomplish the task. (Resolved now, by using the default AspNetCoreJobActivator)
var scopeFactory = serviceProvider.GetService<IServiceScopeFactory>();
if (scopeFactory != null)
GlobalConfiguration.Configuration.UseActivator(new AspNetCoreJobActivator(scopeFactory));
Now, I am getting the following exception in CreateOrderData.cs:-
/*System.InvalidOperationException: An exception has been raised that
is likely due to a transient failure. If you are connecting to a SQL
Azure database consider using SqlAzureExecutionStrategy. --->
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred
while updating the entries. See the inner exception for details. --->
System.Data.SqlClient.SqlException: Transaction (Process ID 103) was
deadlocked on lock resources with another process and has been chosen
as the deadlock victim. Rerun the transaction. */
I am scheduling the hangfire cron job as below:-
RecurringJob.AddOrUpdate<IS2SScheduledJobs>(x => x.ProcessInputXML(), Cron.MinuteInterval(1));
Startup.cs
public void ConfigureServices(IServiceCollection services)
{
string hangFireConnection = Configuration["ConnectionStrings:HangFire"];
GlobalConfiguration.Configuration.UseSqlServerStorage(hangFireConnection);
var config = new AutoMapper.MapperConfiguration(cfg =>
{
cfg.AddProfile(new AutoMapperProfileConfiguration());
);
var mapper = config.CreateMapper();
services.AddSingleton(mapper);
services.AddScoped<IHangFireJob, HangFireJob>();
services.AddScoped<IScheduledJobs, ScheduledJobs>();
services.AddScoped<BusinessLogic>();
services.AddHangfire(opt =>
opt.UseSqlServerStorage(Configuration["ConnectionStrings:HangFire"]));
services.AddEntityFrameworkSqlServer().AddDbContext<ABCContext>(options =>
options.UseSqlServer(Configuration["ConnectionStrings:ABC"]));
}
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory, IServiceProvider serviceProvider)
{
GlobalConfiguration.Configuration.UseActivator(new HangFireActivator(serviceProvider));
//hangFireJob.Jobs();
// add NLog to ASP.NET Core
loggerFactory.AddConsole(Configuration.GetSection("Logging"));
loggerFactory.AddDebug();
loggerFactory.AddNLog();
// app.UseCors("AllowSpecificOrigin");
foreach (DatabaseTarget target in LogManager.Configuration.AllTargets.Where(t => t is DatabaseTarget))
{
target.ConnectionString = Configuration.GetConnectionString("Logging");
}
LogManager.ReconfigExistingLoggers();
}
Hangfire.cs
public class HangFireJob : IHangFireJob
{
private ABCContext _abcContext;
private IScheduledJobs scheduledJobs;
public HangFireJob(ABCContext abcContext, IScheduledJobs scheduledJobs)
{
_abcContext = abcContext;
this.scheduledJobs = scheduledJobs;
}
public void Jobs()
{
RecurringJob.AddOrUpdate<IScheduledJobs>(x => x.ProcessInputXML(), Cron.HourInterval(1));
}
}
ScheduledJobs.cs
public class S2SScheduledJobs : IS2SScheduledJobs
{
private BusinessLogic _businessLogic;
public ScheduledJobs(BusinessLogic businessLogic)
{
_businessLogic = businessLogic;
}
public async Task<string> ProcessInputXML()
{
await _businessLogic.ProcessXML();
}
}
BusinessLogic.cs
public class BusinessLogic
{
private ABCContext _abcContext;
public BusinessLogic(ABCContext abcContext) : base(abcContext)
{
_abcContext = abcContext;
}
public async Task ProcessXML()
{
var batchRepository = new BatchRepository(_abcContext);
var unprocessedBatchRecords = await BatchRepository.GetUnprocessedBatch();
foreach (var batchRecord in unprocessedBatchRecords)
{
try
{
int orderId = await LoadDataToOrderTable(batchRecord.BatchId);
await UpdateBatchProcessedStatus(batchRecord.BatchId);
if (orderId > 0)
{
await CreateOrderData(orderId);
}
}
catch(Exception ex)
{
}
}
}
CreateOrderData.cs
public async Task<int> CreateOrderData(int orderId)
{
try
{
await OrderRepo.InsertOrder(order);
await _abcContext.SaveChangesAsync();
}
catch(Exception ex)
{
/*System.InvalidOperationException: An exception has been raised that is likely due to a transient failure. If you are connecting to a SQL Azure database consider using SqlAzureExecutionStrategy. ---> Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while updating the entries. See the inner exception for details. ---> System.Data.SqlClient.SqlException: Transaction (Process ID 103) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction. */
}
}
InsertOrder.cs
public async Task InsertOrder(Order o)
{
// creation of large number of entites(more than 50) to be inserted in the database
woRepo.Insert(p);
poRepo.Insert(y);
//and many more like above
Insert(order);
}
Insert.cs
public virtual void Insert(TEntity entity)
{
entity.ObjectState = ObjectState.Added;
if (entity is IXYZEntity xyzEntity)
{
xyzEntity.CreatedDate = DateTime.Now;
xyzEntity.UpdatedDate = xyzEntity.CreatedDate;
xyzEntity.CreatedBy = _context.UserName ?? string.Empty;
xyzEntity.UpdatedBy = _context.UserName ?? string.Empty;
}
else if (entity is IxyzEntityNull xyzEntityNull)
{
xyzEntityNull.CreatedDate = DateTime.Now;
xyzEntityNull.UpdatedDate = xyzEntityNull.CreatedDate;
xyzEntityNull.CreatedBy = _context.UserName;
xyzEntityNull.UpdatedBy = _context.UserName;
}
_dbSet.Add(entity);
_context.SyncObjectState(entity);
}
LoadDataToOrder.cs
public async Task<int> LoadDataToOrder(int batchId)
{
// using (var unitOfWork = new UnitOfWork(_abcContext))
// {
var orderRepo = new OrderRepository(_abcContext);
Entities.Order order = new Entities.Order();
order.Guid = Guid.NewGuid();
order.BatchId = batchId;
order.VendorId = null;
orderRepo.Insert(order);
//unitOfWork.SaveChanges();
await _abcContext.SaveChangesAsync();
return order.OrderId;
//
}
}
HangfireActivator.cs
public class HangFireActivator : Hangfire.JobActivator
{
private readonly IServiceProvider _serviceProvider;
public HangFireActivator(IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
}
public override object ActivateJob(Type type)
{
return _serviceProvider.GetService(type);
}
}
Please advise.
Thanks.
Following solutions worked for the 2 problems:
Implementation of multiple Hangfire processes/jobs(multiple workers) to work (in parallel).
Answer: This got resolved when I used the built-in AspNetCoreJobActivator instead that's available out of the box, i.e. removed the HangfireActivator class and removed the call to the UseActivator method.
var scopeFactory = serviceProvider.GetService<IServiceScopeFactory>();
if (scopeFactory != null)
GlobalConfiguration.Configuration.UseActivator(new AspNetCoreJobActivator(scopeFactory));
SqlAzureExecutionStrategy Exception in CreateOrder.cs (transaction was deadlocked)
Answer: Resolved this issue by retrying the query automatically when deadlock occurs.
Thanks odinserj for the suggestions.
Kindly consider the following code:
public class UnitOfWork
{
private readonly Context _context;
public IEmployeeRepository Employees { get; private set; }
public UnitOfWork(Context context)
{
_context = context;
Employees = new EmployeeRepository(_context);
}
public int SaveChanges()
{
return _context.SaveChanges();
}
}
public class Program
{
try
{
using (var unitOfWork = new UnitOfWork(new Context()))
{
var employee = unitOfWork.Employees.GetById(1);
employee.Name = "John";
unitOfWork.SaveChanges();
ComputeSalary(employee);
}
}
catch (Exception ex)
{
// Exception logic goes here...
}
private void ComputeSalary(Employee employee)
{
int x = 1/0; // This will throw an exception.
return;
}
}
The Context class inherits from DbContext of Entity Framework. So in the ComputeSalary() function, you will see that an exception will be thrown since division of zero is not allowed. This will cause the code to exit the using block, thereby, disposing unitOfWork. My question is, will the changes performed by SaveChanges() still propagate to the database even though an exception occurred in the ComputeSalary() function?
If you don't have a TransactionScope or something like that, the changes will be committed.
PS: Not sure, but think if your using EF Core, it will commit to DataBase event if you have a TransactionScope. See https://github.com/aspnet/EntityFrameworkCore/issues/5595
We need to ensure that EF Core-based code has performed a specific kind of database-level operation under test (for example, any command execution or any transaction commit).
Let's suppose a real database should be triggered and we can't isolate it by DbContext mocking. How could it look:
[Fact]
public async Task Test()
{
using (var context = new FooContext())
{
//context-related code to produce queries and commands
}
Assert.True(/* any context-related transaction has been committed */);
}
Is it possible?
EF Core doesn't provide its own tracing mechanism. However, it logs a lot of database interaction events. We could gather these log messages and check their EventId to determine did a specific operation occur or not. Here is the list of relational events used by EF Core:
EF Core 1.1.2: RelationalEventId enum.
EF Core 2.0.0 preview 1: RelationalEventId class (breaking change!).
All we need to do is create a fake logger and pass it to context:
[Fact]
public async Task TransactionCommit_Logger_ContainsEvent()
{
var logger = new FakeLogger();
var factoryMock = new Mock<ILoggerFactory>();
factoryMock.Setup(f => f.CreateLogger(It.IsAny<string>()))
.Returns(logger);
using (var context = new FooContext(factoryMock.Object))
{
using (var transaction = await context.Database.BeginTransactionAsync())
{
transaction.Commit();
}
}
Assert.True(logger.Events.Contains((int)RelationalEventId.CommittingTransaction));
}
FakeLogger adds a logged event id to the Events list.
public class FakeLogger : ILogger
{
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception exception,
Func<TState, Exception, string> formatter)
{
Events.Add(eventId.Id);
}
public List<int> Events { get; set; } = new List<int>();
public bool IsEnabled(LogLevel logLevel) => true;
public IDisposable BeginScope<TState>(TState state) => null;
}
Call UseLoggerFactory to attach a factory instance to a context:
public class FooContext : FooParentContext
{
private readonly ILoggerFactory _loggerFactory;
public FooContext() { }
public FooContext(ILoggerFactory loggerFactory)
{
_loggerFactory = loggerFactory;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseLoggerFactory(_loggerFactory);
}
}
P.S. You could go even deeper and analyze log message or even raw SQL produced by EF.