I have a situation where i have a counter field in a table named Profile, and on form submit, i will retrieve the counter field and +1 to the counter and update profile table. The incremented counter will be stored in a variable where i will then use to create new records in another table [Bidder]. The problem is when there are multiple form submit at the same time, duplicate record values will be created in the Bidder table
Profile profile = db.Profile.Where(w => w.TenderId == tender_Id && w.IsDeleted == false).FirstOrDefault();
int submission = profile.TotalSubmission + 1;
if (profile != null) {
profile.TotalSubmission = submission;
profile.ModifiedBy = user_id;
profile.ModifiedOn = DateTime.Now;
db.Entry(profile).State = EntityState.Modified;
db.SaveChanges();
}
bid.ROId = string.Format("RO{0}", submission);
db.Entry(bid).State = EntityState.Modified;
db.SaveChanges();
How do i prevent duplicate ROId to be created?
The uniqueness should be enforced using a unique index or a unique constraint.
You can create these using code first (from MSDN):
public class User
{
public int UserId { get; set; }
[Index(IsUnique = true)]
public string Username { get; set; }
public string DisplayName { get; set; }
}
or directly via the database.
The counter should be protected using optimistic concurrency:
public class MyEntity
{
[Key]
public Guid Id { get; set; }
// Add a timestamp property to your class
[Timestamp]
[Required]
[DatabaseGenerated(DatabaseGeneratedOption.Computed)]
[ConcurrencyCheck]
public byte[] VersionTimestamp { get; set; }
public int Counter { get; set; }
}
If you try to update the row with the VersionTimestamp after it has been changed without re-reading it from the database, you'll get an OptimisiticConcurrencyException e.g. in this test scenario:
// Read the entity
MyEntity entity;
using (var context = new MyContext())
{
entity = context.MyEntities.Single(e => e.Id == id1);
}
// Read and update the entity
using (var context = new MyContext())
{
var entity2 = context.MyEntities.Single(e => e.Id == id1);
entity2.Counter++;
context.SaveChanges();
}
// Try to update stale data
// - an OptimisticConcurrencyException will be thrown
using (var context = new MyContext())
{
entity.Counter++;
context.SaveChanges();
}
If you are using SQL Server 2012 or newer, you can use a Sequence to accomplish this. You would also want to enforce uniqueness through a unique constraint.
public partial class YourEfContext : DbContext
{
.... (other EF stuff) ......
// get your EF context
public int GetNextSequenceValue()
{
var rawQuery = Database.SqlQuery<int>("SELECT NEXT VALUE FOR dbo.SomeSequence;");
var task = rawQuery.SingleAsync();
int nextVal = task.Result;
return nextVal;
}
}
Another option, if you don't have a version that supports sequences, is to use a stored procedure on the database to issue Id numbers. The stored proc can work in conjunction with an ID table, which it can place an explicit lock on. This means you can request an id from the proc, it can lock the table, read the current number, increment it, store it back in the table, release the lock, and return the id. You would need to call your proc from code to get the new id to assign. The lock on the db side ensures that you are only ever assigned unique values. As long as your id column is only ever given a value assigned by the proc, you will have unique values. You will still be able to assign arbitrary numbers though, which could include duplicates, that can be solved with a unique constraint.
None of this in Entity-Framework specific, though you can still access all this through entity-framework in one way or another.
You can not rely only on entity framework for your solution. Only the database has a full picture of the stored data. Your different entity context instances don't even know if other instances exist, so coordinating sequence numbers on a global scale is extremely difficult on EF level.
Depending on the frequency of conflicts, two options come to my mind to enforce the uniqueness of the sequence number:
Unique constraint
Stored procedure for writing the data
Unique constraint
You can create a UNIQUE constraint over the ProfileId and Sequence columns. When you store the data with a duplicate sequence number, you will get an exception. Either the exception itself or one of its inner exceptions will be an SqlException. You can examine the error number of that exception and if it's error number 2627 (if your DBMS is SQL Server; if it is not, check for the similar error in your DBMS), you know it's a unique key constraint violation. In this case you get the current sequence number from the DB and write the data again with a new sequence. You have to repeat that until the insert was successful.
In case you're using SQL server, you can selectively handle a UNIQUE KEY constraint violation like this (using C# 6.0 exception filters):
private bool IsUniqueKeyViolation(Exception exception) {
Exception currentException = exception;
while (currentException != null) {
SqlException sqlException = exception as SqlException;
if (sqlException != null) {
return sqlException.Errors.Cast<SqlError>().Any(error => error.Number == 2627);
}
currentException = currentException.InnerException;
}
return false;
}
//...
//...Code to set up the POCOs before Save...
while(true) {
try {
context.SaveChanges();
}
catch(Exception exc) when (IsUniqueKeyViolation(exc)) {
//...Code to update the sequence number...
continue;
}
break;
}
This solution is only practical if the number of conflicts is expected to be small. If the number of conflicts is large, you will see a lot of unsuccessful UPDATE requests to the DB, which can become a performance issue.
EDIT:
As some other answers suggested, you could also use optimistic concurrency with a timestamp column. As long as you only update the DB from your own code, this works fine. However, a UNIQUE KEY constraint will protect the integrity of your data also from changes that don't originate from your application (like migration scripts etc.). Optimistic concurrency does not give you the same guarantee.
Stored procedure
You can create a stored procedure that will set the new sequence number from the last existing number in the same INSERT or UPDATE statement. The stored procedure can return the new sequence number back to the client and you can process it accordingly.
Since this solution will always update the DB in a single statement, it works well for a larger amount of conflicting updates. The disadvantage is that you have to write a part of your program logic in SQL on the DB level.
Related
I have a Comment table which can be linked to many different entities that have comments, but for reasons, I have not linked those tables. Instead Comment contains TableReferenceId and EntryReferenceId. TableReferenceId is just an int that we can check in the app layer as to which entity/table that comment refers to, and EntryReferenceId is an int that refers to a particular entry in said entity/table to which the comment belongs.
Querying such comments by table and entry reference would be fine, but when inserting bulk data, I am drawing a blank. For example if I have Vehicle entity and a Vehicle can have many comments, when inserting the data, how would I link them since I don't have a VehicleId yet? Is this doable or is it better to just go many-to-many route for each of the tables that link to comments?
If you can avoid this situation, then you should try to, or you should try to avoid supporting a bulk insert. If you must do this though, then either of the following patterns may work for you.
Perform the Bulk Insert in 2 stages, before the normal import, maintain a map or dictionary of records and the comments that they are linked to, then after the first call to SaveChanges() the IDs will be available to insert.
You could store the mapped comments inside an unbound collection on the entity, after SaveChanges() if there are any entries in this collection, they should be inserted using the new record's Id.
Lets look at the first option:
var mappedComments = new Dictionary<Vehicle,Comment[]>();
// bulk processing, however you choose to do it
// importantly for each item, capture the record reference and the comments
foreach(var item in source)
{
Vehicle newItem;
... construct/parse the new Entity object
List<Comment> newComments = new List<Comment>();
... parse the comments records
// store the map
mappedComments.Add(newItem, newComments.ToArray());
// Add the entity to the context?
db.AddToVehicles(newItem);
}
db.SaveChanges();
foreach(var mapEntry in mappedComments)
{
var newVehicle = mapEntry.Key;
// replace this with your actual logic of course...
int vehicleTableReferenceId = db.TableReferences.Single(x => x.TableName == nameof(Vehicle));
foreach(var comment in mappEntry.Value)
{
comment.TableReferenceId = vehicleTableReferenceId;
comment.EntityReferenceId = newVehicle.Id; // the Id that is now populated
db.AddToComments(comment);
}
}
db.SaveChanges();
If you have a lot Entity types that exhibit this linking behaviour, then you could build this functionality into the Entities themselves, by embedding the mapped comments within the entity itself.
Define an Interface that describes an object that has a weak reference to these Comments
public interface ICommentsToInsert
{
// Only necessary if your convention is NOT to use a common name for the PK
int Id { get; }
ICollection<Comment> CommentsToInsert { get;set;}
}
Implement this interface and add an unmapped collection property to the entities to store the Comment Entries to insert against each record.
partial class Vehicle : ICommentsToInsert
{
[NotMapped]
int ICommentsToInsert.Id { get => Vehicle_Id; }
[NotMapped]
public ICollection<Comment> CommentsToInsert { get;set; } = new HashSet<Comment>();
}
In your bulk logic, add the Comment records into the Vehicle.CommentsToInsert collection, I'll leave that to you...
Override SaveChanges() to detect entities that have comments and re-process them after the save operation.
In this example I am storing the EntityState for all modified entries before the save, this is overkill for this particular example, but you only lose this state information during the save, keeping a record of it becomes useful for a whole range of other applications for post-processing logic.
public override int SaveChanges()
{
var beforeStates = BeforeSaveChanges();
int result = base.SaveChanges();
if (AfterSaveChanges(beforeStates);
result += base.SaveChanges();
return results;
}
private Dictionary<DbEntityEntry, EntityState> BeforeSaveChanges()
{
var beforeSaveChanges = new Dictionary<DbEntityEntry, EntityState>();
foreach( var entry in this.ChangeTracker.Entries())
{
//skip unchanged entries!
if (entry.State == EntityState.Unchanged)
continue;
// Today, only cache the ICommentsToInsert records...
if (entry.Entity is ICommentsToInsert)
beforeSaveChanges.Add(entry, entry.State);
}
return beforeSaveChanges;
}
private bool AfterSaveChanges(Dictionary<DbEntityEntry, EntityState> statesBeforeSaveChanges)
{
bool moreChanges = false;
foreach (var entry in statesBeforeChanges)
{
if (entry.Key.Entity is ICommentsToInsert hasComments)
{
if(hasComments.CommentsToInsert.Any())
{
moreChanges = true;
// Get the Id to the TableReference, based on the name of the Entity type
// you would normally cache this type of lookup, rather than hitting the DB every time
int tableReferenceId = db.TableReferences.Single(x =
> x.TableName == entry.Key.Entity.GetType().Name);
foreach (var comment in hasComments.CommentsToInsert)
{
comment.TableReferenceId = tableReferenceId;
comment.EntityReferenceId = hasComments.Id;
db.AddToComments(comment);
}
}
}
}
return moreChanges;
}
You can further evolve this by implementing DbTransaction scopes to rollback the whole lot if things fail, this code itself is para-phrased from my common routines that I use in production code, so whilst it may not work as is, the concept has served me well in many projects.
Cross posted on MSDN
We use manual code first Entity Framework for System.Data.Sqlite
So when creating a new entity, we manually create the table, the c# object, and add the DbSet to our context.
I have a very simple object created and I am getting this error when trying to query the context. I have checked the column names and data types many times but I don't see a mismatch here. Also there are no foreign key relationships defined even though the fields have id in the name. Just a standalone table.
The weirdest part is that I can add a new entity to the context, save changes, and it will be persisted to the db. However in the next line when trying to retrieve the entities I get The entity set is not defined in the entity container error. I have also noticed that if I hover over an instantiated context, all the other db sets will have the EF SQL such as SELECT EXTENT1.myCol as myCol, but the department_resources set just says {System.Data.Entity.DbSet<department_resource>}.
Any ideas on what the issue is here?
Below are excerpts of my files:
DDL
CREATE TABLE department_resources (
dep_res_key VARCHAR PRIMARY KEY,
department_id INT NOT NULL,
resource_id INT NOT NULL);
department_resource.cs
[Table("department_resources")]
public class department_resource
{
[Key]
public string dep_res_key { get; set; }
public int department_id { get; set; }
public int resource_id { get; set; }
}
MyContext.cs
public class MyContext : DbContext
{
public DbSet<department_resource> department_resources { get; set; }
}
Sample Usage
using (MyContext db = new MyContext())
{
db.department_resources.Add(new department_resource()
{ dep_res_key = "anID",
resource_id = 22,
department_id = 23 }); // Works
db.SaveChanges(); // Also works. Even persists to db
var foo = from r in db.department_resources
select r.resource_id; // Doesn't work. Will error as soon as I try to use foo. Like assigning to a combo box item source. Or even just enumerating the results
var bar = db.department_resources; // Also doesn't work.
}
The issue is with deferred execution. Although you're assigning foo and bar within your using block, they're not getting evaluated until they're actually used, after MyContext has been disposed.
You need to force them to evaluated within your using block. e.g. by converting the results to lists.
Also I notice you've declared them as vars within your using block. They'll need to be defined outside of it to be able to use them outside (perhaps you just did this in your sample to simplify?)
List<int> foo;
List<department_resource> bar;
using (MyContext db = new MyContext())
{
db.department_resources.Add(new department_resource()
{ dep_res_key = "anID",
resource_id = 22,
department_id = 23 }); // Works
db.SaveChanges(); // Also works. Even persists to db
foo = (from r in db.department_resources
select r.resource_id).ToList();
bar = db.department_resources.ToList();
}
From MSDN
the query variable itself only stores the query commands. The actual
execution of the query is deferred until you iterate over the query
variable in a foreach statement. This concept is referred to as
deferred execution
Forcing Immediate Execution
Queries that perform aggregation functions over a range of source elements must first iterate over
those elements. Examples of such queries are Count, Max, Average, and
First. These execute without an explicit foreach statement because the
query itself must use foreach in order to return a result. Note also
that these types of queries return a single value, not an IEnumerable
collection.
You can also force execution by putting the foreach loop immediately
after the query expression. However, by calling ToList or ToArray you
also cache all the data in a single collection object.
This is the same of the entity that I am planning to save in the Azure Table Storage (ATS):
public class CarEntity : TableEntity
{
public CarEntity(string objPartitionKey, string objRowKey)
{
this.PartitionKey = objPartitionKey;
this.RowKey = objRowKey;
}
public string TableName
{
get { return "EntityTableName"; }
}
public string Property1 { get; set; }
// and this goes on
public string Property60 { get; set; }
}
Not all properties are required. Population of records depend on the selections that the user would be saving (e.g this is a CarEntity - if the user ordered wheels, properties WheelSize and WheelQuantity would be populated, if the user asks for repainting, RepaintingColor would be populated and so, on).
Assuming that there are 60 properties in this entity, not all properties gets saved in ATS. Despite a property being defined and no error being returned, that data doesn't get saved in the table. I know that there's a limit of 1MB per entity but considering the computations that we have done, it is kinda far from the 1MB limit.
Any help why columns don't appear even if the properties are saved accordingly? My save function is defined as follows:
public static CarEntity CarInsertOrReplace(CarEntity entity)
{
if (entity == null)
{
throw new ArgumentNullException("entity");
}
var table = SetupTable(entity.TableName);
table.CreateIfNotExists();
TableOperation insertOrMergeOperation = TableOperation.InsertOrReplace(entity);
TableResult result = table.Execute(insertOrMergeOperation);
CarEntity objEntity = result.Result as CarEntity;
return objEntity;
}
Sounds like the properties for your Entity vary based on the usage. What's probably happening is that Azure Table Storage is only creating columns for properties that are not null (have a value set). So you are will only see columns created for properties that have been set.
It sounds as if Table Storage is performing as you require but not necessarily as you expect. As answered by #Paul Fryer ATS will not store null values and as you do not (going by your quoted code) initialise the CarEntity properties they will be null by default. Therefore only properties set by the user will be saved to the table.
https://msdn.microsoft.com/en-us/library/azure/hh452242.aspx #Remarks:
If the Insert Or Replace Entity operation is used to replace an
entity, any properties from the previous entity will be removed if the
new entity does not define them. Properties with a null value will
also be removed.
Also, from your code
TableResult result = table.Execute(insertOrMergeOperation);
CarEntity objEntity = result.Result as CarEntity;
result will contain the TableOperation not a copy of the full entity in case that was what you were expecting.
This scenario might be the difference between using, for example, a SQL table where fields that are not given a value have a database default or default to null, against the Azure table model where fields not given a value do not exist. You just need to be aware of that difference when reading/writing to the storage chosen.
If you require all fields to be persisted to the table then provide a default value for each property, e.g. string.Empty
If strings are null or empty then it doesn't save the property at all. You're not doing anything wrong, you just have to consider this in your design when you're working with them.
If you're using TableEntity then it does the null/empty check for you. If you're using DynamicTableEntity then you have to do the check yourself.
I am new to entity framework and I have been searching a while for an answer to this question and I can't find anything that directally addresses this.
Here is the problem. I have a table in Oracle. In this table there are 2 fields(there are more but not important to this question). Card_Transaction_Id and Parent_Card_Transaction_ID. The Parent_Card_Transaction_Id field is constrained by the Card_Transaction_Id field and I am using a Oracle sequence via a trigger to populate the Card_Transaction_Id field.
In my code, I am using Entity Framework(Version 5) to connect using the Code First Approach.
The issue is when I try to create a new record. I need to know what the next sequence value is in order to populate the Parent_Card_Transaction_Id. My mapping for card transactions:
public class CardTransactionMap : EntityTypeConfiguration<CardTransaction>
{
public CardTransactionMap(string schema)
{
ToTable("CARD_TRANSACTION", schema);
// Mappings & Properties
// Primary Key
HasKey(t => t.CardTransactionId);
Property(t => t.CardTransactionId)
.HasColumnName("CARD_TRANSACTION_ID")
.HasDatabaseGeneratedOption(DatabaseGeneratedOption.Identity);
Property(t => t.ParentCardTransactionId)
.HasColumnName("PARENT_CARD_TRANSACTION_ID");
Property(t => t.CardProfileId)
.HasColumnName("CARD_PROFILE_ID");
}
}
The question is - is there any way to get the next sequence number before I save the record?
My current work arround is to use the following method:
public static decimal GetNextCardTransactionSequenceValue()
{
using (var context = new Context(new OracleConnectionFactory().GetConnection()))
{
var sequence = context.Database.SqlQuery<int>("select card_transaction_id from card_transaction").ToList();
return sequence[0];
}
}
Using that method, I get the next value and then just populate my table. This works but I don't like doing it this way. I feel that there must be a better way to do it.
Thanks in advance.
You have to do this by navigation properties.
By fetching the next value from a sequence before actually using it in the same session you create yourself a concurrency issue: another user can increment the index (by an insert) in the time between drawing its next value and assigning it to the child record. Now the child will belong to the other user's record!
If your CardTransaction class has a parent reference like this:
int ParentCardTransaction { get; set; }
[ForeignKey("ParentCardTransaction")]
CardTransaction ParentCardTransaction { get; set; }
you can create a parent and child in one go and call SaveChanges without worrying about setting FK values yourself:
var parent = new CardTransaction { ... };
var child = new CardTransaction { ParentCardTransaction = parent, ... };
SaveChanges();
Now EF wil fetch the new CardTransactionId from the parent and assign it to the FK of the child. So generating and getting the parent Id happens all in one session, so it is guaranteed to be the same value.
Apart from preventing concurrency issues, of course it is much easier anyway to let EF do the heavy lifting of getting and assiging key values.
Create a Stored Procedure or Query that will return you the next Value from the Table here is an Example
SELECT NVL(MAX(card_transaction_id + 1), 0) AS MAX_VAL
FROM card_transaction T;
or Create a Trigger - for OracleDB
Change your table definition to this :
CREATE TABLE t1 (c1 NUMBER GENERATED BY DEFAULT ON NULL AS IDENTITY,
c2 VARCHAR2(10));
as per the information in the link i provided in the comment.
after the update ef will automatically query the value for the id that is inserted, there is no need to fill in the id before the insert. ef will generate an insert sql query without id.
In the database I'm working on the company keeps track of order numbers for each regional office. I have a table named [Company] that has a single record for each regional office. This table has a field that holds the next available order number for that office. All orders are created in a single [Order] table. I realized it would be possible for more than one user at a given office to ask for a new order at the same time. I have everything set up for optimistic concurrency. The [Company] table has a column named RowModTracker used for concurrency purposes (see images at bottom for all the properties in EF and SQL).
In my code below, I get the record for the regional office, increment the field that has the order number and put a copy in a local variable. Then I immediately save off the table. I would expect that if two users hit this method at the same time the first user to save off will get a valid order number. I would expect the second user to save off would get an optimistic concurrency error and null should be returned (which causes a prompt that asks them to try again). The users tell me that a few times in the last eight months they have indeed gotten orders with the same order number. No messages and no errors happen until later in the process when they go to save the order and the unique constraint on the order number is violated.
I obviously don’t understand something fundamental about EF concurrency checking and I’m hoping someone can show me the light. I wrote a test version of GetNextOrderNumberAndUpdate where I wrapped a second using context around the first and it did indeed generate the expected concurrency error. It just doesn't seem to happen when two different clients do it.
private int? TryToGetNextOrderNumber()
{
int? order_number;
bool keep_trying = true;
while (keep_trying)
{
order_number = _entityManager.GetNextOrderNumberAndUpdate();
if (order_number != null) return order_number;
keep_trying = KeepTryingForOrderNumberPrompt();
}
return null;
}
public int? GetNextOrderNumberAndUpdate()
{
try
{
using (var db_context = new HotEntities(EntityConnectionString))
{
var hss_company = db_context.Companies.Find(HssCompanyInUse.HssCompanyId);
int? next_order_num = ++hss_company.NextOrderNum;
db_context.SaveChanges();
return next_order_num;
}
}
catch (DbUpdateConcurrencyException)
{
return null;
}
catch (Exception ex)
{
return null; //actual code puts out an alert as well
}
}
I am using Entity Framework 5 and SQL Server 2008 R2 with .NET 4.5.
Edit
The test version looked like this, I've elided all the try/catch stuff it was the same as above
//Test force a failure
using (HotEntities test_context = new HotEntities(EntityConnectionString))
{
var test_hss_company = test_context.Companies.Find(HssCompanyInUse.HssCompanyId);
int? test_next_order_num = ++test_hss_company.NextOrderNum;
//======= Normal Code ============
using (var db_context = new HotEntities(EntityConnectionString))
{
var hss_company = db_context.Companies.Find(HssCompanyInUse.HssCompanyId);
var next_order_num = ++hss_company.NextOrderNum;
db_context.SaveChanges();
return next_order_num;
}
//======= End Normal Code ============
test_context.SaveChanges(); //DbUpdateConcurrencyException exception would get thrown here
} //end for test using