I'm using Entity Framework v6 and I'm trying to make sure that I can perform an atomic Insert or Select the record if it doesn't already exist so that in a farm of servers (or multiple threads) I can guarantee that I don't get a unique key constraint violation.
I have a simple example with a table like this and a corresponding simple Model with the 2 properties.
CREATE TABLE [dbo].[NewItem]
(
[ID] [int] IDENTITY(1,1) NOT NULL,
[Name] [varchar](40) NOT NULL,
CONSTRAINT [PK_NewItem] PRIMARY KEY CLUSTERED ([ID] ASC),
CONSTRAINT [IX_NewItem] UNIQUE NONCLUSTERED ( [Name] ASC)
)
When I write my Entity Framework code, I cannot guarantee that the object doesn't get inserted after the .Any returns false.
using (var myContext = new MyContext())
{
using (var transaction = myContext.Database.BeginTransaction())
{
if (!myContext.NewItems.Any(item => item.Name == identifier))
{
var newItem = new NewItem { Name = identifier };
myContext.NewItems.Add(newItem);
try
{
var result = myContext.SaveChanges();
}
catch (Exception ex)
{
Debug.Print(ex.ToString());
throw;
}
}
transaction.Commit();
}
}
If I was to perform this same code using a direct SQL statement via a SqlCommand object, this is what I would use and as far as I can tell it always works.
using (var connection = new SqlConnection(ConfigurationManager.ConnectionStrings["Test"].ConnectionString))
{
using (var cmd = connection.CreateCommand())
{
cmd.CommandText = "IF (NOT EXISTS(SELECT ID FROM NewItem WHERE Name = #name)) " +
" BEGIN " +
" INSERT INTO NewItem VALUES (#name) " +
" SELECT SCOPE_IDENTITY() AS ID " +
" END " +
" ELSE " +
" BEGIN " +
" SELECT ID FROM NewItem WHERE Name = #name " +
" END";
var nameParam = new SqlParameter("#name", System.Data.SqlDbType.VarChar, 40);
nameParam.Value = Name;
cmd.Parameters.Add(nameParam);
connection.Open();
var result = cmd.ExecuteScalar();
connection.Close();
Id = Convert.ToInt32(result);
}
}
Is there some way with Entity Framework that I can perform the same operation as my raw SqlCommand so that if the record doesn't exist it gets inserted and if it already exists, I just get the one that is there in an atomic operation?
To demonstrate this is some of the threading code that I've used to simulate multiple servers.
private static ExecutionMode mode;
private static ManualResetEventSlim wait;
private static string identifier;
private enum ExecutionMode
{
None = 0,
Entityframework,
RawSql
}
static void Main(string[] args)
{
//Comment out the relevant item to test
//mode = ExecutionMode.RawSql;
mode = ExecutionMode.Entityframework;
wait = new ManualResetEventSlim(false);
identifier = Guid.NewGuid().ToString();
var threads = new List<Thread>();
for (var i = 0; i < 20; i++)
{
var t = new Thread(RunCreate);
threads.Add(t);
t.Start();
}
wait.Set();
for (var i = 0; i < 20; i++)
{
var t = threads[i];
t.Join();
}
}
private static void RunCreate()
{
wait.Wait();
switch (mode)
{
case ExecutionMode.Entityframework:
{
//perform the Entityframework code above
}
break;
case ExecutionMode.RawSql:
{
var i = new NewItem { Name = identifier };
i.InsertOrSelect(); //This is just using the code for the raw SQL Statements
}
break;
}
}
Exceptions experienced with the Entityframework, displaying ex.Message looping through the exception->innerexception until innerexception is null
With Default Transaction Isolation Level
BeginTransaction()
A first chance exception of type 'System.Data.Entity.Infrastructure.DbUpdateException'
occurred in EntityFramework.dll
An error occurred while updating the entries.
See the inner exception for details.
An error occurred while updating the entries.
See the inner exception for details.
Violation of UNIQUE KEY constraint 'IX_NewItem'. Cannot insert duplicate key in object 'dbo.NewItem'.
The duplicate key value is (f32c6a59-1462-49c3-85e2-5126c96ad484).
With Serializable tranaction Isolation
BeginTransaction(System.Data.IsolationLevel.Serializable)
A first chance exception of type 'System.Data.Entity.Infrastructure.DbUpdateException'
occurred in EntityFramework.dll
Transaction (Process ID 59) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Neither of these are guaranteed to work. The piece that you are missing is Transaction Isolation Level.
The SQL Server default isolation level is READ COMMITTED. This has only minimal locking protections. It means that it is possible for another command to insert the new item after your IF condition is evaluated but before the new row is inserted. The raw SQL server command is much less likely to encounter this scenario because it is processed without a round-trip between the commands: there is less time for another server to interfere.
If you set the isolation level to Serializable then you are guaranteed to either insert the item or select the existing item. In Entity Framework your code will look like this:
using (var myContext = new MyContext())
{
using (var transaction = myContext.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
{
if (!myContext.NewItems.Any(item => item.Name == identifier))
{
var newItem = new NewItem { Name = identifier };
myContext.NewItems.Add(newItem);
try
{
var result = myContext.SaveChanges();
}
catch (Exception ex)
{
Debug.Print(ex.ToString());
throw;
}
}
transaction.Commit();
}
}
Related
Is this a good way to set the transaction level to serializable? If I set a lower transaction level, I will get duplicate orders. What is better approach? I assume this method will be invoked frequently by different users.
Here is my code
using (var tran = context.Database.BeginTransaction(IsolationLevel.Serializable))
{
try
{
var lastDocument = context.Documents.OrderByDescending(x => x.Id).FirstOrDefault();
int order = 1;
if (lastDocument != null)
{
order = lastDocument.Order + 1;
}
var document = new Document
{
CreatedDate = DateTimeOffset.UtcNow,
Name = Guid.NewGuid().ToString(),
Order = order
};
context.Documents.Add(document);
context.SaveChanges();
tran.Commit();
}
catch (Exception ex)
{
tran.Rollback();
}
}
I am trying to load EOD stock data into a table using this method:
public async Task<long> BulkInsertEodData(IEnumerable<EodData> records, string symbol)
{
var recordsProcessed = 0L;
using (var conn = await OpenConnection())
using (var trans = conn.BeginTransaction())
using (var comm = _factory.CreateCommand())
{
try
{
comm.Connection = conn;
comm.Transaction = trans;
comm.CommandText = INSERT_EOD;
var ps = AddParametersToInsertEodQuery(comm);
foreach (var p in ps) comm.Parameters.Add(p);
comm.Prepare();
foreach (var record in records)
{
comm.Parameters["#date_id"].Value = record.DateId;
comm.Parameters["#symbol"].Value = symbol.ToUpper();
comm.Parameters["#eod_close"].Value = record.EodClose;
comm.Parameters["#eod_high"].Value = record.EodHigh;
comm.Parameters["#eod_low"].Value = record.EodLow;
comm.Parameters["#eod_volume"].Value = record.EodVolume;
comm.Parameters["#eod_open"].Value = record.EodOpen;
comm.Parameters["#eod_split"].Value = record.EodSplit;
comm.Parameters["#eod_dividend"].Value = record.EodDividend;
comm.Parameters["#last_modified"].Value = DateTime.UtcNow;
await comm.ExecuteNonQueryAsync();
recordsProcessed++;
}
trans.Commit();
}
catch (Exception ex)
{
_logger.LogError(ex, "BulkInsertEodData(IEnumerable<EodData>)");
trans.Rollback();
}
}
return recordsProcessed;
}
The query text is as follows:
INSERT INTO public.eod_datas(
date_id,
stock_id,
eod_open,
eod_close,
eod_low,
eod_high,
eod_volume,
eod_dividend,
eod_split,
last_modified_timestamp
)
values
#date_id,
(select s.id from stocks s where s.symbol = #symbol limit 1),
#eod_open,
#eod_clos,
#eod_low,
#eod_high,
#eod_volume,
#eod_dividend,
#eod_split,
current_timestamp
on conflict (date_id, stock_id)
do update set
eod_open = #eod_open,
eod_close = #eod_close,
eod_low = #eod_low,
eod_high = #eod_high,
eod_volume = #eod_volume,
eod_dividend = #eod_dividend,
eod_split = #eod_split,
last_modified_timestamp = current_timestamp;
It's not my first rodeo with prepared statements, but I am doing a couple things different this time around (.NET Core, using DbProviderFactory) and I'm getting odd results.
The first couple of times through this method, I get an error to the effect of Npgsql.PostgresException (0x80004005): 42601: syntax error at or near "$1" which is fairly mystifying in itself but the most mysterious of all is that the error actually goes away after a couple of method calls, and I start getting Npgsql.PostgresException (0x80004005): 26000: prepared statement "_p1" does not exist consistently afterwords.
Can someone explain this behavior? What am I doing wrong? Where do I get more details into what "$1" is all about?
You are missing brackets around values being inserted. Postgres sadly will not tell you that it is expecting bracket before $1.
Correct syntax:
values (
#date_id,
(select s.id from stocks s where s.symbol = #symbol limit 1),
#eod_open,
#eod_clos,
#eod_low,
#eod_high,
#eod_volume,
#eod_dividend,
#eod_split,
current_timestamp)
on conflict (date_id, stock_id)
In my EF code and SQL Server, I try to insert a user data while this user is not exist, but EF inserted it 1000 times, just 1 time in SQL Server.
EF code:
static void Main(string[] args)
{
using (var db = new MyDbContext())
{
for (var i = 0; i < 1000; i++)
{
var count = db.Users.Count(f => f.Name == "Test");
if (count > 0) continue;
db.Users.Add(new User
{
Name = "Test",
Gender = "Male",
Phone = "1111111111",
CreateTime = DateTime.Now
});
}
try
{
db.SaveChanges();
//1000 rows effected
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
Console.ReadKey();
}
T-SQL code:
declare #i int = 0
begin tran
while(#i < 1000)
begin
if not exists (select 1 from [dbo].[User] where Name = 'Test')
insert into [dbo].[User] values('Test','Male','1111111111',getdate())
set #i = #i + 1
end
if(##ERROR > 0)
rollback tran
else
commit tran
Even if I use a transaction wrap the EF code (but call SaveChanges once), the result is same as 1000 rows affected.
But when I use this code, the result is the same as in SQL Server:
using (var db = new MyDbContext())
{
using (var trans = db.Database.BeginTransaction(ReadCommitted))
{
for (var i = 0; i < 1000; i++)
{
var count = db.Users.Count(f => f.Name == "Test");
if (count > 0) continue;
db.Users.Add(new User
{
Name = "Test",
Gender = "Male",
Phone = "1111111111",
CreateTime = DateTime.Now
});
//1 row effected as sql does
db.SaveChanges();
}
try
{
trans.Commit();
}
catch (Exception ex)
{
trans.Rollback();
Console.WriteLine(ex.Message);
}
}
}
Please tell me why transaction is different between EF and SQL
db.Users.Add does not insert. It makes for insertion when SaveChanges is called (what did you think SaveChanges did or why it was required if not for writing out changes that were previously not written?). That's why db.Users.Count(f => f.Name == "Test") always returns zero.
This has nothing to do with transactions.
(Btw, you probably should use Any instead of Count here).
I recently created a light weight ORM tool in C# and posted it on Github. One of the things my ORM does is manage the opening and closing of connections for you along with opening and disposing of a sql transaction. The way I do this is I open the connection and begin the sql transaction in the constructor of my context class and close the connection and dispose the transaction on the dispose method of my context class. The code below are some methods from my tool:
public class ADOCRUDContext : IDisposable
{
internal IDbConnection sqlConnection;
internal IDbTransaction sqlTransaction;
internal string suffix = "_";
public ADOCRUDContext(string connectionString)
{
sqlConnection = new SqlConnection(connectionString);
sqlConnection.Open();
sqlTransaction = sqlConnection.BeginTransaction();
}
public void Commit()
{
sqlTransaction.Commit();
}
/// <summary>
/// Disposes transaction and closes sql connection
/// </summary>
public void Dispose()
{
sqlTransaction.Dispose();
sqlConnection.Close();
}
public void Insert<T>(T item)
{
string tableName = ObjectTypeHelper.GetTableName<T>(item);
// Properties of object that are not primary keys
PropertyInfo[] modelProperties = ObjectTypeHelper.GetModelProperties<T>(item, false);
// Properties of object that are also primary keys
PropertyInfo[] primaryKeyProperties = ObjectTypeHelper.GetPrimaryKeyProperties<T>(item);
if (modelProperties == null || modelProperties.Count() == 0)
{
// Moves composite keys to model properties if they are the only properties of the table/object
if (primaryKeyProperties.Count() > 0)
modelProperties = primaryKeyProperties;
else
throw new Exception("Class has no properties or are missing Member attribute");
}
// Generates insert statement
StringBuilder query = new StringBuilder();
query.Append("insert into " + tableName + "(" + String.Join(",", modelProperties.Select(x => x.Name)) + ") ");
query.Append("values (" + String.Join(", ", modelProperties.Select(x => "#" + x.Name+suffix)) + ") ");
query.Append("select #primaryKey = ##identity");
try
{
// Generates and executes sql command
SqlCommand cmd = GenerateSqlCommand<T>(item, query.ToString(), modelProperties, null, ADOCRUDEnums.Action.Insert, suffix);
cmd.ExecuteNonQuery();
// Loads db generated identity value into property with primary key attribute
// if object only has 1 primary key
// Multiple primary keys = crosswalk table which means property already contains the values
if (primaryKeyProperties != null && primaryKeyProperties.Count() == 1)
primaryKeyProperties[0].SetValue(item, cmd.Parameters["primaryKey"].Value as object);
}
catch (Exception ex)
{
sqlTransaction.Rollback();
throw new Exception(ex.Message);
}
}
Below is an example of how you use my tool:
public void AddProduct(Product p)
{
using (ADOCRUDContext context = new ADOCRUDContext(connectionString))
{
context.Insert<Product>(p);
context.Commit();
}
}
public void UpdateProduct(int productId)
{
using (ADOCRUDContext context = new ADOCRUDContext(connectionString))
{
Product p = this.GetProductById(productId);
p.Name = "Basketball";
context.Update<Product>(p);
context.Commit();
}
}
Currently if I do something like:
public void DoSomething(Product p)
{
using (ADOCRUDContext context = new ADOCRUDContext(connectionString))
{
this.AddProduct(p);
p.Name = "Test";
throw new Exception("Blah");
}
}
Even though I threw an exception on DoSomething, the product still got added to the database since the inner transaction was committed. Is there a way to get the inner transaction (AddProduct) to fail if the outer transaction fails?
I want to input data into my table (sql 2008) using linq to sql:
public static bool saveEmail(Emailadressen email)
{
TBL_Emailadressen saveMail = new TBL_Emailadressen();
destil_loterijDataContext db = new destil_loterijDataContext();
saveMail.naam = email.naam;
saveMail.emailadres = email.emailadres;
saveMail.lotnummer = email.lotnummer;
try
{
saveMail.naam = email.naam;
saveMail.lotnummer = email.lotnummer;
saveMail.emailadres = email.emailadres;
db.TBL_Emailadressens.InsertOnSubmit(saveMail);
return true;
}
catch (Exception ex)
{
Console.WriteLine("Opslaan niet gelukt!" + ex.ToString());
return false;
}
}
For some reason nothing is being added to this table.
My table has the following fields:
ID (Auto incr int)
Naam (varchar50)
lotnummer (varchar50)
emailadres (varchar50)
My object im trying to save (saveMail) always has an ID = 0 , and i don't know why. I think that is preventing me from saving to the DB?
Be sure to call SubmitChanges on your DataContext-derived class:
using(var dc = new MyDataContext())
{
saveEmail(new Emailadressen(...));
dc.SubmitChanges();
}