I'm trying to insert some potentially poor quality data in to the system and I need a "per row" report of what happened so i've ben trying to do this ...
public async Task<IEnumerable<Result<Invoice>>> AddAllAsync(IEnumerable<Invoice> invoices, Guid bucketId)
{
var results = new List<Result<Invoice>>();
log.Debug(invoices.ToJson());
foreach (var invoice in invoices)
{
try
{
results.Add(new Result<Invoice> { Success = true, Item = await AddAsync(invoice, bucketId), Message = "Imported Successfullly" });
await SaveChangesAsync();
}
catch (Exception ex)
{
results.Add(new Result<Invoice> { Message = ex.Message, Item = invoice });
}
}
return results;
}
My problem is that after a single add call fails that attempted add is left in the change tracker so calling add again with a different item raises the exception again for the first item.
Is there a way to (without rebuilding the context) do "batch inserts" and getting details on a per row / entity level of all the issues not just the first?
Related
I have an issue, I want continue to insert data after the exception was raised by SQL Server.
I got an Unique Index on 3 different columns in table to detect duplicates.
For example I am trying to insert 2 rows, the first one is an duplicate, the second one is not.
When the duplicate is detected it goes in the catch, then I'm doing nothing, but when it comes on the second row which is not an duplicate, an exception is raised again for the previous row.
This is my code:
public async Task<IEnumerable<Result>> Handle(NewResultCommandDTO requests, CancellationToken cancellationToken)
{
var results = new List<Result>();
...
for (var j = 0; j < resultDetails.Count(); j++)
{
var rd = resultDetails.ElementAt(j);
var newResult1 = new Result
{
AthleteFEIID = rd.AthleteFEIID,
CompetitionCode = competition.CompetitionCode,
HorseId = horse.Id,
};
results.Add(newResult1);
try
{
await _resultsService.AddResultAsync(newResult1);
await _resultsService.CompleteAsync();
}
catch (Exception ex)
{
var x = ex;
}
}
}
public async Task AddResultAsync(Result result)
{
Context.Results.AddAsync(result);
}
public async Task CompleteAsync()
{
await Context.SaveChangesAsync().ConfigureAwait(false);
}
Thank you for your help !
await _resultsService.CompleteAsync(); is throwing the sql exception statement.
await _resultsService.AddResultAsync(newResult1); statement is already adding the entity in the db context. Even if the next statement throws the exception and it goes to catch block, the duplicated entity is still added in the context. So, when you are adding the next entity in the context and trying to save it, it is throwing exception because of the previous duplicated entity which is not removed from the context.
One solution is to remove the duplicated entity from the context when it goes to catch block.
try
{
await _resultsService.AddResultAsync(newResult1);
await _resultsService.CompleteAsync();
}
catch (Exception ex) {
var x = ex;
_resultsService.RemoveResult(newResult1);
}
public void RemoveResult(Result result)
{
Context.Results.Remove(result);
}
Another solution is to check if the duplication already exists in the table before adding it. For that you will have to write a get method using the unique indexed columns.
All,
I need to show all validation errors within EF Core. So for example
I get data by doing this :
foreach (var item in FRFT)
{
var FinancialReportingFrameworkType = new FinancialReportingFrameworkType { FinancialReportingFrameworkTypeId = item.Frfid, Acronym = item.Frf1, Name = item.Frfdescription };
_clDBContext.FinancialReportingFrameworkType.Add(FinancialReportingFrameworkType);
}
Then I want to push this data into another table but there may be some vlaidation errors. I need to track the errors.
If I do a try .. catch block, I only recieve the first error but there maybe an error on every single row I push into the databse.
_clDBContext.SaveChanges();
}
catch (DbUpdateException e)
{
if (e.InnerException != null)
{
// to be chnged to an POST HTML error code
Console.WriteLine(e.InnerException.Message);
return new OkObjectResult(e.InnerException.Message);
}
I need to do some pre-Save Changes valdiation.
Any ideas ?
I'm testing out a bulk import using EF Core and trying to save asynchronously.
I'm trying to add 100 entities at a time and then asynchronously save and repeat until they are all saved. I occasionally get a PK error because it tries to add two entities with the same id to the database. None of the entities being added have an id set, the ids are sequences that are auto generated.
The code:
public async Task<bool> BulkAddAsync(IEnumerable<VehicleCatalogModel> models)
{
_dbContext.ChangeTracker.AutoDetectChangesEnabled = false;
try
{
var entities = models.Select(ToEntity);
_dbContext.Set<VehicleCatalog>().AddRange(entities);
await _dbContext.SaveChangesAsync();
}
catch (Exception ex)
{
_logger.LogError(0, ex, "An Error occurred during the import");
return false;
}
return true;
}
I am calling the method in an xunit test that generates a list of test data and calls the import
var result = manager.BulkAddAsync(modelsToAdd.AsEnumerable());
var counter = 0;
while (!result.IsCompleted && counter < 10)
{
Thread.Sleep(6000);
counter++;
}
Assert.True(result.IsCompleted && result.Result);
It should hold up the execution after 100 entities have been added until they are saved, and then add more but I am still occasionally getting this error. Is there something else I need to add to get this to work correctly? Or a better method of bulk insert?
I am using an Entity Framework 6.1 Model from Database 'wizard' setup.
When I create a Business object from my context and then try to add for attachment and then SaveChanges() nothing happens. Is there a tracing mode? or something I can turn on to see what is really happened under the covers.
Simple example:
var fb = _context.Business.Create();
//fb.Id exists and is an int but it is auto incr in the db
fb.Name = ub.ACCOUNT_NAME;
fb.ServiceManager = ub.SERVICE_MANAGER;
fb.AccountManager = ub.ACCOUNT_MANAGER;
fb.SalesPerson = ub.SALESPERSON;
fb.Created = DateTime.UtcNow;
fb.Updated = DateTime.UtcNow;
_context.Add(fb);
_context.SaveChanges();
The best way I have found to catch EF errors is by overriding the SaveChange method like below. If you have a centered place to recover logs (like log4net), the function will be able to insert it there.
public partial class Business
{
/// <summary>Override the SaveChange to return better error messages</summary>
public override int SaveChanges()
{
try {
return base.SaveChanges();
}
catch (System.Data.Entity.Validation.DbEntityValidationException ex) {
// Retrieve the error messages as a list of strings.
var errorMessages = ex.EntityValidationErrors
.SelectMany(x => x.ValidationErrors)
.Select(x => x.ErrorMessage);
// Join the list to a single string.
var fullErrorMessage = string.Join("; ", errorMessages);
// Combine the original exception message with the new one.
var exceptionMessage = string.Concat(ex.Message, " The validation errors are: ", fullErrorMessage);
// Add some logging with log4net here
// Throw a new DbEntityValidationException with the improved exception message.
throw new System.Data.Entity.Validation.DbEntityValidationException(exceptionMessage, ex.EntityValidationErrors);
}
}
Have you tried checking for any validation errors?
Here is the try block and validation method I am using in one of my new classes, so treat it as a code sample and not a 100% tested solution as I am still putting together some unit tests:
public List<string> ValidationErrorList = new List<string>();
try
{
_context.SaveChanges();
}
catch (Exception)
{
GetErrors(_context);
}
private void GetErrors(System.Data.Entity.DbContext context)
{
IEnumerable<System.Data.Entity.Validation.DbEntityValidationResult> ve;
ve = context.GetValidationErrors();
ValidationErrorList.Clear();
foreach (var vr in ve)
{
if (vr.IsValid == false)
{
foreach (var e in vr.ValidationErrors)
{
var errorMessage = e.PropertyName.Trim() + " : " +
e.ErrorMessage;
ValidationErrorList.Add(errorMessage);
}
}
}
}
While the above sample only calls the GetErrors method when an exception is triggered, you might also want to try calling it right after the SaveChanges() to see if there are validation errors that are not throwing an exception.
Have you tried creating a new Business object and adding it in? instead of creating one first?
var fb = new Business();
//fb.Id exists and is an int but it is auto incr in the db
fb.Name = ub.ACCOUNT_NAME;
fb.ServiceManager = ub.SERVICE_MANAGER;
fb.AccountManager = ub.ACCOUNT_MANAGER;
fb.SalesPerson = ub.SALESPERSON;
fb.Created = DateTime.UtcNow;
fb.Updated = DateTime.UtcNow;
_context.Business.Add(fb);
_context.SaveChanges();
At work we are looking to move to an ORM (still using an access database with ADO!) I started building with entity framework and everything was going smoothly until I separated it off into it's own .dll (so we could have the website/crm/production/barcoding systems all using the same database logic).
The issue comes with handling the DbEntityValidationExceptions, my initial test code (which worked)
public override int SaveChanges(System.Data.Objects.SaveOptions options)
{
try{return base.SaveChanges(options);}
catch (DbEntityValidationException dbEx)
{
foreach (var validationErrors in dbEx.EntityValidationErrors)
{
foreach (var validationError in validationErrors.ValidationErrors)
{
Trace.TraceInformation("Class: {0}, Property: {1}, Error: {2}",
validationErrors.Entry.Entity.GetType().FullName,
validationError.PropertyName,
validationError.ErrorMessage);
}
}
//handle here
throw;
}
}
but this doesn't get caught now and i'm left handling a generic threadException. Is there any way to access the original data (cast the threadException?) or is there a different approach I should take. I think I just need a push in the right direction and I can figure something out.
Regards, Pete
UPDATE:
Hmm bit of an issue calling the GetValidationErrors. I notice that my database Context has the baseClass of ObjectContext and not DbContext (So I can't call the ValidateEntity). I'm using Entity Framework 5 with default code generation enabled - using a database first approach if that helps.
check for validation errors before calling SaveChanges. Something like...
var errors = context.GetValidationErrors();
if(errors.Any())
{
//handle validation errors
}
else
{
context.SaveChanges();
}
from memory, so the exact syntax may not be correct.
this is what I ended up going with in the end (i'll probably end up fleshing it out as I learn more about EF)
public List<DbEntityValidationException> vErrors = new List<DbEntityValidationException>();
public int DbChanges = 0;
public bool SaveChanges()
{
try
{
this.vErrors = (List<DbEntityValidationException>)base.GetValidationErrors();
if (this.vErrors.Count == 0)
{
this.DbChanges = base.SaveChanges();
return true;
}
}
catch (Exception Ex)
{
this.vErrors.Add(new DbEntityValidationException(string.Format("General Error: {0}", Ex.GetType().ToString())));
}
return false;
}
and from code
using(Db db = new Db())
{
//changes
if(db.SaveChanges)
{
//some message using db.DbChanges
}
else
{
//handle errors in db.vErrors
}