I developed a multithread application which insert some data inside a database.
Suppose I have the following situation:
public void Foo()
{
Task.Factory.StartNew(() => AddTeams());
Task.Factory.StartNew(() => AddTable());
}
as you can see I call two different methods which insert data into a db, the problem is that each method need to check if a specific record exists in a particular table:
public void AddTeams()
{
//Pseudo code:
//Check if a team with id 1249 exist in the table team
//if not exist in the table team, insert the team with id 1249
//then insert the record attached with an `FK` to the team table.
}
the same thing happen to AddTable, so sometimes I get this error:
'Duplicate entry '1249' for key 'PRIMARY
because the checking fails on the AddTable method, the reason of the failure is the parallelization that I used, for summarize: a time problem.
How can I manage this?
The only way that come to my mind is to handle the exception, but I don't like this approach.
You simply need to not parallelize the query to get the team and create it if necessary. You don't want or need to do it twice, there's only ever a need to do it once to begin with. So get or create the team, get the primary key of that team, and then pass that to those two methods to each create a related object, and that can be done in parallel.
Related
When using OrmLite to add an entry into the database there seems to be two ways of doing it:
dbConn.Insert(customer);
and
dbConn.Save(customer);
When using Insert() the AutoIncrement ID field does not get updated, but when using Save() it does.
If you use:
dbConn.LastInsertId();
It will return the correct ID if Save() is used but the wrong ID if Insert() is used.
Why do these two methods exist? Using Insert() will add an entry to the database with the correct ID, it is just not reflected in the POCO model. It seems strange that there are no exceptions thrown and no indication of a problem, when you can end up using the wrong ID and get hard to track bugs if you are not aware of this.
It's already mentioned in the documentations:
Save and SaveAll will Insert if no record with Id exists, otherwise it Updates. Both take multiple items, optimized to perform a single read to check for existing records and are executed within a sinlge transaction.
For Insert operation, it's pretty straighforward, there's nothing special in that functions.
So, Use Insert when you know there's no duplicated key in your records, otherwise use save method.
If you want to retrieve the LastInsertId, with Insert method, you need to specify the [AutoIncrement] on your POCO.
UPDATE: When you set your IDbConnectionFactory using Singelton With Lazy< T > implementation, it seems the LastInsertId will not working. so just use lock or static implementation.
I've shown three programmers this problem and we're all stumped. I call a Sql Server stored procedure in a foreach loop and the result always is the same as the first call. Even if I hard code parameters (removing the loop) only the first result is assigned to all subsequent calls.
The stored procedure is called by an Entity Framework function import (EF4 database first using the designer). The calling code lives in a repository that is a class library. The repository is called by a separate Asp.net webforms project. The problem code looks like this:
IEnumerable<WorkOrder> orders = _context.GetWorkOrders(UserName, workOrder, customerCode).ToList();
OrderStatus lastStatus = new OrderStatus();
foreach (Order order in orders)
{
lastStatus = _context.GetOrderStatus(order.OrderNumber).FirstOrDefault();
order.LastOrderStatus = lastStatus.OrderStatus;
}
As you can see this is pretty basic stuff. Depending on the order numbers passed in I always get the result of the first order number in the loop. I've turned off Ajax (part of the Telerik controls I use) because that has caused baffling errors for me in the past. I really hope you can suggest a way to debug this problem! Thanks in advance.
EDIT: Daniel J.G.'s comment led me to this possible solution. Now I need to figure out how to apply Ladislav Mrnka's answer..."Try to call ExecuteFunction directly with MergeOption.OverwriteChanges."
I'm answering my own question (since no one else has after a few days). The problem is caused by the Entity Framework database first designer. It generates code that caches the first stored procedure result causing the bad results in subsequent calls.
As I mentioned in the edit to my question the fix involves replacing the default MergeOption parameter used by ExecuteFunction. You need to use MergeOption.OverwriteChanges instead of the default (which I believe is MergeOption.PreserveChanges).
You could change that parameter in the generated code but your changes would be lost each time the designer is rebuilt. Instead I simply copied the generated code to my repository class, changed the MergeOption to OverwriteChanges, and stopped using the generated code. The end result looks like this:
IEnumerable<WorkOrder> orders = _context.GetWorkOrders(UserName, workOrder, customerCode).ToList();
OrderStatus lastStatus = new OrderStatus();
foreach (Order order in orders)
{
ObjectParameter workOrderParameter;
if (wo.WorkOrder != null)
{
workOrderParameter = new ObjectParameter("WorkOrder", order.WorkOrder);
}
else
{
workOrderParameter = new ObjectParameter("WorkOrder", typeof(global::System.String));
}
lastStatus = _context.ExecuteFunction<OrderStatus>("GetOrderStatus", MergeOption.OverwriteChanges, workOrderParameter).FirstOrDefault();
if (status != null)
{
order.LastOrderStatus = status.OrderStatus;
}
}
I also see that there is a way you can modify the T4 template to make the generated code use the correct MergeOption parameter. I haven't tried it though. If you're interested take a look here.
I'm back with a second answer to my own question. Be sure the Entity Key is truly a unique identifier for each Entity!
In my case, the OrderStock Entity was missing the OrderID (along with StockID) as the Entity Key. Typically the designer culls the primary key fields from the database but I have a unique situation (where my entity is based on a view). Since I left off OrderID from the Entity Key I saw duplicate rows for a single OrderStock Entity.
When I marked OrderID Entity Key = True the duplicate problem went away.
I'm currently creating a method on my data access class that is going to insert an entity object to the database and I was expecting to get the latest inserted ID afterwards... and I've already done that but then I was wondering what would happend if the method somehow gets invoked twice at the same time, would it return the wrong ID?
So as a work around to that I decided to Lock the table on my datacontext:
lock(dataContext.Persons)
{
InsertOnSubmit(person);
dataContext.SubmitChanges();
}
but I do feel like this is inappropriate... I mean, the table isnt big and it wont take long to the datacontext to submit changes... So my question is, what kind of trouble would I run into by locking it like that?
PS: Let me know if my question was not clear enough and I'll edit it!
There is no need to lock your table.
LINQ-to-SQL will automatically populate your ID field with the identity from the database upon your call to .SubmitChanges() when you insert your person.
The caveat is that both your database and L2S entity must define your ID field as an identity. On your Person entity, you should have that field defined as the primary key, IsDbGenerated=true, UpdateCheck=never, and have the correct database type.
Once you submit, you should be able to simply retrieve the ID from your Person entity:
dataContext.Persons.InsertOnSubmit(person);
dataContext.SubmitChanges();
var id = person.ID; // now has the database generated identity.
Im not sure if this is part of your concerns, but if you use TransactionScope you can pass in a TransactionOption that will deal with multiple transactions.
Check out TransactionOption.Isolationlevel
Currently, I'm using the following code to perform an 'upsert' in CRM
try
{
crm.Create(c);
}
catch (SoapException)
{
crm.Update(c);
}
Am I right in assuming that this type of updating will wipe all existing information from my business entity? And if so, how can I get the entity to update from the repository? Do I need fetch, retrieve, or something like that?
Thanks.
Using exception handling for flow control is bad 99.99% of the time; in this case among other things because you never know what the actual reason for your SoapException is.
A much cleaner way would be to check whether your record's ID field has a value; if so, do an Update, if not, do a Create (and maybe add the resulting ID to your object if you use it further). (We've seen a Create on a record with an ID actually do an update years ago, but we've never been able to reproduce it.)
Other than that, #ckeller is perfectly right; an attribute that is null in your object (because it wasn't in the ColumnSet when retrieving from the database or hasn't been set otherwise) will not be touched in an Update.
How can I do these 2 scenarios.
Currently I am doing something like this
public class Repository
{
private LinqtoSqlContext dbcontext = new LinqtoSqlContext();
public void Update()
{
// find record
// update record
// save record ( dbcontext.submitChanges()
}
public void Insert()
{
// make a database table object ( ie ProductTable t = new ProductTable() { productname
="something"}
// insert record ( dbcontext.ProductTable.insertOnSubmit())
// dbcontext.submitChanges();
}
}
So now I am trying to load an XML file what has tons of records. First I validate the records one at a time. I then want to insert them into the database but instead of doing submitChanges() after each record I want to do a mass submit at the end.
So I have something like this
public class Repository
{
private LinqtoSqlContext dbcontext = new LinqtoSqlContext();
public void Update()
{
// find record
// update record
}
public void Insert()
{
// make a database table object ( ie ProductTable t = new ProductTable() { productname
="something"}
// insert record ( dbcontext.ProductTable.insertOnSubmit())
}
public void SaveToDb()
{
dbcontext.submitChanges();
}
}
Then in my service layer I would do like
for(int i = 0; i < 100; i++)
{
validate();
if(valid == true)
{
update();
insert()
}
}
SaveToDb();
So pretend my for loop is has a count for all the record found in the xml file. I first validate it. If valid then I have to update a table before I insert the record. I then insert the record.
After that I want to save everything in one go.
I am not sure if I can do a mass save when updating of if that has to be after every time or what.
But I thought it would work for sure for the insert one.
Nothing seems to crash and I am not sure how to check if the records are being added to the dbcontext.
The simple answer is: you do not. Linq2Sql is a lot of things - it is not a replacement for bulk upload / bulk copy. You will be a LOT more efficient using the ETL route:
Generate a flat file (csv etc.) with the new data
Load it into the database using bulk load mechanisms
If the data is updating etc. - load it into temporary tables and use the MERGE command to merge it into the main table.
Linq2Sql will by design always suck in mass insert scenarios. ORM's just are not ETL tools.
Linq2SQL (as has been noted) does not handle this well by default, but luckily there are some solutions out there. here's one i used for a website when i wanted to do some bulk deletes. It worked well for me and due to its use of extension methods it was basically indistinguishable from regular Lin2SQL methods.
I haven't really "released" this project yet, but it's a T4-based repository system that extends Linq To SQL and implements a bunch of batch operations (delete, update, create csv, etc.): http://code.google.com/p/grim-repo/. You can check out the source code and implement it however you see fit.
Also, this link has some great source code for batch operations: http://www.aneyfamily.com/terryandann/post/2008/04/Batch-Updates-and-Deletes-with-LINQ-to-SQL.aspx
And, also, I know it's tempting, but don't crap on the elderly. Try performing batch operations with DataAdapters/ADO.net: http://davidhayden.com/blog/dave/archive/2006/01/05/2665.aspx. It's faster, but inevitably hairier.
Finally, if you have an XML file, you can create a stored procedure that takes advantage of SQL server's built-in sproc, sp_xml_preparedocument. Check out how to use it here: http://msdn.microsoft.com/en-us/library/ms187367.aspx
Even when you add multiple records to the DataContext before calling SubmitChanges, LINQ2SQL will loop through and insert them one by one. You can verify this by implementing one of the partial methods on an entity class ("InsertMyObject(MyObject instance)"). It will be called for each pending row individually.
I don't see anything wrong with your plan -- you say it works, but you just don't know how to verify it? Can't you simply look in the database to check if the records got added?
Another way to see what records are pending in the DataContext and have not yet been added is to call GetChangeSet() on the data context and then refer to the "Inserts" property of the returned object to get a list of rows that will be inserted when SubmitChanges is called.