Develop Reference

Audit Entity Framework deleted entries preserving previous value

I've been struggling for a while with a problem that consists on auditing generically database entities when they're saved. I have a project that uses EF 6 and it was required to me to create a "non-invasive" method to audit entities when they're added, modified or deleted. I have to store a JSON of the inserted entity, modified entity or deleted entity without interfering with the normal flow. The project has a Database First implementation.
My solution was simple, add a partial class of any entity that the rest of the programmers want to audit implementing IAudit which is basically an empty interface to get all changes from entities that implement it.
public interface IAudit {}
I have a Currencies entity that just implement it without any other code (I could do something else in the future but I don't need it)
public partial class Currencies : IAudit
I override the SaveChanges method to look for entities to audit
public override int SaveChanges()
{
ChangeTracker.DetectChanges();
// This linq looks for new entities that were marked for audit
CreateAuditLog(System.Data.Entity.EntityState.Added);
CreateAuditLog(System.Data.Entity.EntityState.Modified);
CreateAuditLog(System.Data.Entity.EntityState.Deleted);
return base.SaveChanges();
}
The solution calls 3 times the CreateAuditLog because in the near future I need to implement a configuration to audit whatever the user decides, might be from a database configuration that is activated/deactivated by users.
Everything worked perfectly, I was able to get saved entities in the specified state:
private void CreateAuditLog(System.Data.Entity.EntityState state)
{
var auditedEntities = ChangeTracker.Entries<IAudit>()
.Where(p => p.State == state)
.Select(p => p.Entity);
... some code that do something else
foreach (var auditedEntity in auditedEntities)
{
... some information I required to add
strJSON = JsonConvert.SerializeObject(auditedEntity, new EFNavigationPropertyConverter());
... some code to save audit information
}
}
The problem is I lose every value in the Deleted state, I only get the ID, there's no information in the properties except the ID and there is no any possibility of extract it in any way. I looked for every single solution in StackOverflow and other websites and there is nothing to recover the original information.
How can I get the previous deleted values to store them in the same way I'm storing Added and Modified entities?

It took me a couple of days to figure it out. Might be the solution is a bit complex but I tried several less complex options with not a good result.
First, as I'm just auditing Delete in a different way I separated Deleted state from Added and Modified that work well with no change. Deleted state is a particular case and I treat it like that.
First, I needed to obtain the original values from the database. In the Deleted state they're gone, there's not any possibility of recovering them from the entity. It's possible to obtain them with the following code:
var databaseValues = this.Entry(auditedEntity).GetDatabaseValues();
The result is just a collection of DB property values (DbPropertyValues). If I can get the original values I set the original values from the deleted entity:
dbEntityEntry.OriginalValues.SetValues(databaseValues);
This line just fills the entity original values, it doesn't modify the current value at all. It's useful to do it that way because it takes some code to check every property and set it ourselves, it's an interesting shortcut.
Now, the problem is I don't have the entity to serialize, so I need a new one which in my case I create by reflection because I don't know the type (I receive entities that implement IAudit)
Type type = auditedEntity.GetType();
var auditDeletedEntity = Activator.CreateInstance(type);
This is the entity I will serialize to store the audit later.
Now, the complex part, I need to get the entity properties and fill them by reflection from the original values set in the entity:
foreach (var propertyInfo in type.GetProperties())
{
if (!propertyInfo.PropertyType.IsArray && !propertyInfo.PropertyType.IsGenericType)
{
var propertyValue = originalValues.GetValue<object>(propertyInfo.Name);
auditDeletedEntity.GetType().InvokeMember(propertyInfo.Name,
BindingFlags.Instance | BindingFlags.Public | BindingFlags.SetProperty,
Type.DefaultBinder, auditDeletedEntity, new[] { propertyValue });
}
}
I had to check generic and array types to avoid following EF relations that are not going to work with this method and I also don't need (I need the object not the whole tree)
After that I simply need to serialize the audited deleted entity:
strJSON = JsonConvert.SerializeObject(auditDeletedEntity, new EFNavigationPropertyConverter());
The code looks like this:
string strJSON = string.Empty;
if (state == System.Data.Entity.EntityState.Deleted)
{
var databaseValues = this.Entry(auditedEntity).GetDatabaseValues();
// Get original values from the database (the only option, in the delete method they're lost)
DbEntityEntry dbEntityEntry = this.Entry(auditedEntity);
if (databaseValues != null)
{
dbEntityEntry.OriginalValues.SetValues(databaseValues);
var originalValues = this.Entry(auditedEntity).OriginalValues;
Type type = auditedEntity.GetType();
var auditDeletedEntity = Activator.CreateInstance(type);
// Get properties by reflection
foreach (var propertyInfo in type.GetProperties())
{
if (!propertyInfo.PropertyType.IsArray && !propertyInfo.PropertyType.IsGenericType)
{
var propertyValue = originalValues.GetValue<object>(propertyInfo.Name);
auditDeletedEntity.GetType().InvokeMember(propertyInfo.Name,
BindingFlags.Instance | BindingFlags.Public | BindingFlags.SetProperty,
Type.DefaultBinder, auditDeletedEntity, new[] { propertyValue });
}
}
strJSON = JsonConvert.SerializeObject(auditDeletedEntity, new EFNavigationPropertyConverter());
}
}
else
{
strJSON = JsonConvert.SerializeObject(auditedEntity, new EFNavigationPropertyConverter());
}
Might be there's a better way but I seriously spent a good amount of time looking for options and I couldn't find anything better.
Any suggestion or optimization is appreciated.

Failure to attach a detached entity (entity with the same key is already in the context)

I'm using Entity Framework 6, Code First approach. I'll try to present my problem with a simple piece of code:
public void ViewEntity(MyEntity Entity) // Want to read properties of my entity
{
using (var Db = new MyDbContext())
{
var DummyList = Db.MyEntities.ToList(); // Iteration on this DbSet
Db.MyEntities.Attach(Entity); // Exception
}
}
The exception message is: Attaching an entity of type 'MyProgram.MyEntity' failed because another entity of the same type already has the same primary key value.
From what I've read on MSDN it's an expected behaviour. But what I want on that last line is to first check if there is an entity with the same key already attached to a context; if it is, use it instead, and only otherwise attach my entity to context.
But I've failed to find a way to do so. There are many utility methods on ObjectContext instance (for example GetObjectByKey). I can't test them all 'cause they all ultimately need a qualifiedEntitySetName, and I don't have any in my real imlpementation, because this method should be on an abstract class and it should work for all entity types. Calling Db.Entity(this) is no use, there is no EntityKey which would have EntitySetName.
So all of this became complex really fast. And in my terms I just want to check if the object is already in "cache" (context), use it, otherwise use my object and attach it to this context.
To be clear, I have a detached object from a TreeNode.Tag in the first place, and I just want to use it again, or if it's impossible; if there already is one in the context), use that one instead. Maybe I'm missing some crucial concepts of EF6, I'm just starting out with EF.

I've found a solution for me. As I guessed correctly ObjectContext.GetObjectByKey method does what I need, but first I needed to construct qualifiedEntitySetName, and I found a way to do so. A tad bit cumbersome (using reflection, iterating properties of MyDbContext), but does not compare to a headache of a problem I made out of all this. Just in case, here's the patch of code that is a solution for me:
public SdsAbstractObject GetAttachedToContext()
{
var ObjContext = (SdsDbContext.Current as IObjectContextAdapter).ObjectContext;
var ExistingItem = ObjContext.GetObjectByKey(GetEntityKey()) as SdsAbstractObject;
if (ExistingItem != null)
return ExistingItem;
else
{
DbSet.Attach(this);
return this;
}
}
public EntityKey GetEntityKey()
{
string DbSetName = "";
foreach (var Prop in typeof(SdsDbContext).GetProperties())
{
if (Prop.PropertyType.IsGenericType
&& Prop.PropertyType.GenericTypeArguments[0] == ObjectContext.GetObjectType(GetType()))
DbSetName = Prop.Name;
}
if (String.IsNullOrWhiteSpace(DbSetName))
return null;
else
return new EntityKey("SdsDbContext." + DbSetName, "Id", Id);
}

An Entity can be in one of five stages : Added, Unchanged, Modified, Deleted, Detached.
public void ViewEntity(MyEntity entity) // Want to read properties of my entity
{
using (var Db = new MyDbContext())
{
var DummyList = Db.MyEntities.ToList(); // Iteration on this DbSet
// Set the Modified state of entity or you can write defensive code
// to check it before set the state.
if (Db.Entry(entity).State == EntityState.Modified) {
Db.Entry(entity).State = EntityState.Modified
}
// Attached it
Db.MyEntities.Attach(Entity);
Db.SaveChanges();
}
}
Since EF doesn't know which properties are different from those in the database, it will update them all.

When using Attach in EF, is it possible to not overwrite properties I haven't Updated?

I'm updating an existing entity by attaching it to my data context like this:
var updatedDocumentState = new AccDocumentState()
{
Id = accDocumentState.Id,
IsDocumentary = accDocumentState.IsDocumentary,
IsEditable = accDocumentState.IsEditable,
IsRecursive = accDocumentState.IsRecursive,
Title = accDocumentState.Title,
Reportable = accDocumentState.Reportable,
};
context.AccDocumentStates.Attach(updatedDocumentState);
context.ObjectStateManager.ChangeObjectState(updatedDocumentState, System.Data.EntityState.Modified);
flag = context.SaveChanges() > 0;
And this works, however after saving the attached entity, the properties of the existing entity which i didn't update, but i want to keep as they were, are overwritten and given null values. How can I attach my entity and keep the properties of the existing entity which i have not updated?

EF has an Object Data change tracker. Is enabled via proxies
Tracking changes in Poco entries
Essentially You/find Read the Object/Poco entity first.
Change only those properties you want. And save.
Only the changed properties are updated.
If you are not using autoDetectChnages
this.Configuration.AutoDetectChangesEnabled = false; ////<<<<<<<<< Default true
Then you would Call Detect Changes before Saving.
But either way the concept is based around a Read first to get entity.
Make the necessary changes and save.
Only the actually changes are sent back to Db.
eg:
var mypoco = Context.Set<TPoco>.Find(1);
myPoco.propertyXyz = "changed";
// normally not required by default, But incase your are not using tracking proxies , tell ef heads Up
// Context.Context.ChangeTracker.DetectChanges(); // uncomment when needed
Context.SaveChanged();
Only the actually changes are sent to DB.
Whilst the POST from Rameez is correct, it does not indicate why setting the whole entry as changed is desirable nor why do that ? Why link the State entry post from documentation ?
Context.Entry(poco).State = state; // why do this ? or the objectContext equivalent
This will result in an UPdate Set for all values going to Database on SaveChanges
Since ALL fields will be treated as changed. This is NOT a good way to use EF.
It is important to know about the auto detect changes in EF.
See Automatic detect changes
and Entity states and SaveChanges

As per msdn When you change the EntityState of an entity object entry to Modified, all the properties of the object are marked as modified, regardless of the current or original values. http://msdn.microsoft.com/en-us/library/system.data.objects.objectstatemanager.changeobjectstate.aspx
Hence I think all other properties get set to null as the object that you create will have other properties as null or their default values.
Below is the modified code.
var updatedDocumentState = context.AccDocumentStates.First(a => a.Id== accDocumentState.Id);
updatedDocumentState.IsDocumentary = accDocumentState.IsDocumentary,
updatedDocumentState.IsEditable = accDocumentState.IsEditable,
updatedDocumentState.IsRecursive = accDocumentState.IsRecursive,
updatedDocumentState.Title = accDocumentState.Title,
updatedDocumentState.Reportable = accDocumentState.Reportable,
flag = context.SaveChanges() > 0;

As a workaround to your problem, create a model for just the fields you are updating. Assuming this is a common scenario and warrants the extra model to avoid an extra call to the db.
With the new, minimized model, pointing to the same table, but with only the required properties, it will work as you want. Of course, nothing changed on the EF side, but it will only update the properties it knows about.
While I agree this is not how EF was designed, I too feel frustrated with the extra DB calls to do an update or delete. This solution helps with that.

Try this. Maybe works as you need:
var updatedDocumentState = context.AccDocumentStates.Find(accDocumentState.Id)
{
IsDocumentary = accDocumentState.IsDocumentary,
IsEditable = accDocumentState.IsEditable,
IsRecursive = accDocumentState.IsRecursive,
Title = accDocumentState.Title,
Reportable = accDocumentState.Reportable,
};
flag = context.SaveChanges() > 0;

I've had luck with the following. First I created an extension method to unset the IsModified flag for any property that is not in the set of properties that I want to restrict updates to:
public static void RestrictModifiedProps<ENT>(this DbContext context, ENT entity, IEnumerable<string> restrictedPropNames)
where ENT : class
{
//Grab the meta entry that knows whether the entity/properties have been updated
var entry = context.Entry(entity);
if (entry == null) return;
//loop over properties, only allow properties in the
// restrictedPropNames list to be modified
foreach (var propName in entry.CurrentValues.PropertyNames)
{
var prop = entry.Property(propName);
if (!prop.IsModified) continue;
prop.IsModified = restrictedPropNames.Any(O => O == propName);
}
}
In my case, I am accepting the entity's property values from a json post to an MVC action. So, I want to find out what properties were posted and created a (couple) extension methods for the controller:
public static JObject JsonPostData(this Controller cntrlr)
{
//ensure we're at the start of the input stream
Stream req = cntrlr.Request.InputStream;
req.Seek(0, SeekOrigin.Begin);
//read in any potential json
string json = d2s.SafeTrim(new StreamReader(req).ReadToEnd());
if (string.IsNullOrWhiteSpace(json)
|| !json.StartsWith("{")
|| !json.EndsWith("}"))
return null;
//try to deserialize it
return JsonConvert.DeserializeObject(json) as JObject;
}
public static IEnumerable<JProperty> JsonPostProperties(this Controller cntrlr)
{
JObject jObj = cntrlr.JsonPostData();
if (jObj == null) return null;
return jObj.Properties();
}
public static IEnumerable<string> JsonPostPropNames(this Controller cntrlr)
{
IEnumerable<JProperty> jProps = cntrlr.JsonPostProperties();
if (jProps == null) return null;
return jProps.Select(O => O.Name);
}
In the action, we get:
[HttpPost, ActionName("Edit")]
public virtual ActionResult Edit_Post(ENT obj)
{
...code...
Ctxt.Set<ENT>().Attach(obj);
Ctxt.Entry(obj).State = EntityState.Modified;
Ctxt.RestrictModifiedProps(obj, this.JsonPostPropNames());
...code...
}

If you are just excluding one or two properties, like say you never wanted to allow updates to the Title property (in your example), just unset IsModified on target properties after you set the object state to modified:
context.AccDocumentStates.Attach(updatedDocumentState);
context.ObjectStateManager.ChangeObjectState(updatedDocumentState, System.Data.EntityState.Modified);
context.Entry(updatedDocumentState).Property("Title").IsModified = false;
flag = context.SaveChanges() > 0;
Also FYI - Default MVC5 projects in VS use this line to set the object's modified property:
context.Entry(updatedDocumentState).State = System.Data.EntityState.Modified;

A generic way to save an entity in Entity Framework

I am trying to write a GenericEFRepository which will be used by other Repositories. I have a Save method as below.
public virtual void Save(T entity) // where T : class, IEntity, new() And IEntity enforces long Id { get; set; }
{
var entry = _dbContext.Entry(entity);
if (entry.State != EntityState.Detached)
return; // context already knows about entity, don't do anything
if (entity.Id < 1)
{
_dbSet.Add(entity);
return;
}
var attachedEntity = _dbSet.Local.SingleOrDefault(e => e.Id == entity.Id);
if (attachedEntity != null)
_dbContext.Entry(attachedEntity).State = EntityState.Detached;
entry.State = EntityState.Modified;
}
You can find the problem in comments of below code
using (var uow = ObjectFactory.GetInstance<IUnitOfWork>()) // uow is implemented like EFUnitOfWork which gives the DbContext instance to repositories in GetRepository
{
var userRepo = uow.GetRepository<IUserRepository>();
var user = userRepo.Get(1);
user.Name += " Updated";
userRepo.Save(user);
uow.Save(); // OK only the Name of User is Updated
}
using (var uow = ObjectFactory.GetInstance<IUnitOfWork>())
{
var userRepo = uow.GetRepository<IUserRepository>();
var user = new User
{
Id = 1,
Name = "Brand New Name"
};
userRepo.Save(user);
uow.Save();
// NOT OK
// All fields (Name, Surname, BirthDate etc.) in User are updated
// which causes unassigned fields to be cleared on db
}
The only solution I can think of is creating Entities via repository like userRepo.CreateEntity(id: 1) and repository will return an Entity which is attached to DbContext. But this seems error prone, still any developer may create an entity using new keyword.
What are your solution suggestions about this particular problem?
Note: I already know about cons and pros of using a GenericRepository and an IEntity interface. So, "Don't use a GenericRepository, don't use an IEntity, don't put a long Id in every Entity, don't do what you are trying to do" comments will not help.

Yes it is error prone but simply that is the problem with EF and repositories. You must either create entity and attach it before you set any data you want to update (Name in your case) or you must set modified state for each property you want to persist instead of whole entity (as you can imagine again developer can forget to do that).
The first solution leads to special method on your repository doing just this:
public T Create(long id) {
T entity = _dbContext.Set<T>().Create();
entity.Id = id;
_dbContext.Set<T>().Attach(entity);
return entity;
}
The second solution needs something like
public void Save(T entity, params Expression<Func<T, TProperty>>[] properties) {
...
_dbContext.Set<T>().Attach(entity);
if (properties.Length > 0) {
foreach (var propertyAccessor in properties) {
_dbContext.Entry(entity).Property(propertyAccessor).IsModified = true;
}
} else {
_dbContext.Entry(entity).State = EntityState.Modified;
}
}
and you will call it like:
userRepository(user, u => u.Name);

This is kind of a fundamental problem of this approach because you expect the repository to magically know which fields you changed and which ones you didn't. Using null as a signal for "unchanged" does not work in case null is a valid value.
You'd need to tell the repository which fields you want to have written, for example sending a string[] with the field names. Or one bool for each field. I do not think this is a good solution.
Maybe you can invert the control flow like this:
var entity = repo.Get(1);
entity.Name += "x";
repo.SaveChanges();
That would allow change tracking to work. It is closer to how EF wants to be used.
Alternative:
var entity = repo.Get(1);
entity.Name += "x";
repo.Save(entity);

While the other two answers provide good insight into how perhaps you can avoid this issue I think its worth pointing out a couple of things.
What you are trying to do (ie a proxy entity update) is extremely EF-centeric and IMO actually doesn't make sense outside of the EF context and hence it doesnt make sense that a generic repository would be expected to behave in this way.
You actually haven't even gotten the flow quite right for EF, if you attach an object with a few fields already set EF will conciser what you told it to be the current DB state unless you modify a value or set a modified flag. To do what you are attempting without a select you would normally attach an object without the name and then set the name after attaching the ID object
Your approach is normally used for performance reasons, I would suggest that by abstracting over the top of an existing framework you are almost always going to suffer some logical performance degradation. If this is a big deal maybe you shouldn't be using a repository? The more you add to your repository to cater to performance concerns the more complex and restrictive it becomes and the harder it gets to provide more than one implementation.
All that being said I do think you can handle this particular case in a generic situation.
This is one possible way you could do it
public void UpdateProperty(Expression<Func<T,bool>> selector, FunctionToSetAProperty setter/*not quite sure of the correct syntax off the top of my head*/)
{
// look in local graph for T and see if you have an already attached version
// if not attach it with your selector value set
// set the property of the setter
}
Hope this makes some sense, I'm not by my dev box atm so I cant really do a working sample.
I think this is a better approach for a generic repository as it allows you to implement this same behavior in multiple different ways, the abovc may work for EF but there will be different methods if you have an in memory repository (for example). This approach allows you to implement different implementations that fulfill the intent rather than restrict your repository to only act like EF.

How to make NHibernate considered property to always be dirty when using dynamic update or insert?

I am looking for help on an issue with NHibernate which has been bugging me for a while now. Long story short:
I’m looking for a way to, in the first level cache, “reset” a property on an entity each time I do an update or an insert.
What I want to achieve is that the property in question will always be considered to be dirty by NHibernate when using dynamic update or insert.
The backstory for this is that I know that, if the transaction was successful, the column that I want to “reset” will be set to Null in the database by a trigger. On the flip side, the first level cache does not know this, and thus NHibernate will think that the property was not updated when I set it to the same value as I did on the previous update/insert. The catch is that my trigger is dependent on this value being set. The resulting mess is that if I want to use dynamic update or insert I’m only able to update/insert an entity once without “refreshing” it afterwards (which I really don’t want to do).
Tips or help would be much appreciated, because I’ve really hit a wall here

NHibernate provides many places for extension. Among them is the Session IInterceptor. There is documentation with many details:
http://nhibernate.info/doc/nh/en/index.html#objectstate-interceptors
In this case, we can create our custom one, which will be observing our entity (for example Client) and a property which must be updated every time (for example Code). So our implementation could look like this:
public class MyInterceptor : EmptyInterceptor
{
public override int[] FindDirty(object entity, object id, object[] currentState, object[] previousState, string[] propertyNames, NHibernate.Type.IType[] types)
{
var result = new List<int>();
// we do not care about other entities here
if(!(entity is Client))
{
return null;
}
var length = propertyNames.Length;
// iterate all properties
for(var i = 0; i < length; i++)
{
var areEqual = currentState[i].Equals(previousState[i]);
var isResettingProperty = propertyNames[i] == "Code";
if (!areEqual || isResettingProperty)
{
result.Add(i); // the index of "Code" property will be added always
}
}
return result.ToArray();
}
}
NOTE: This is just an example! Apply your own logic for checking the dirty properties.
And we have to wrap Session this way:
var interceptor = new MyInterceptor()
_configuration.SetInterceptor(interceptor);
And this is it. While Client is marked as dynamic-update, the property Code will always be set as dirty
<class name="Client" dynamic-update="true" ...