Related
I am currently getting this error:
System.Data.SqlClient.SqlException: New transaction is not allowed because there are other threads running in the session.
while running this code:
public class ProductManager : IProductManager
{
#region Declare Models
private RivWorks.Model.Negotiation.RIV_Entities _dbRiv = RivWorks.Model.Stores.RivEntities(AppSettings.RivWorkEntities_connString);
private RivWorks.Model.NegotiationAutos.RivFeedsEntities _dbFeed = RivWorks.Model.Stores.FeedEntities(AppSettings.FeedAutosEntities_connString);
#endregion
public IProduct GetProductById(Guid productId)
{
// Do a quick sync of the feeds...
SyncFeeds();
...
// get a product...
...
return product;
}
private void SyncFeeds()
{
bool found = false;
string feedSource = "AUTO";
switch (feedSource) // companyFeedDetail.FeedSourceTable.ToUpper())
{
case "AUTO":
var clientList = from a in _dbFeed.Client.Include("Auto") select a;
foreach (RivWorks.Model.NegotiationAutos.Client client in clientList)
{
var companyFeedDetailList = from a in _dbRiv.AutoNegotiationDetails where a.ClientID == client.ClientID select a;
foreach (RivWorks.Model.Negotiation.AutoNegotiationDetails companyFeedDetail in companyFeedDetailList)
{
if (companyFeedDetail.FeedSourceTable.ToUpper() == "AUTO")
{
var company = (from a in _dbRiv.Company.Include("Product") where a.CompanyId == companyFeedDetail.CompanyId select a).First();
foreach (RivWorks.Model.NegotiationAutos.Auto sourceProduct in client.Auto)
{
foreach (RivWorks.Model.Negotiation.Product targetProduct in company.Product)
{
if (targetProduct.alternateProductID == sourceProduct.AutoID)
{
found = true;
break;
}
}
if (!found)
{
var newProduct = new RivWorks.Model.Negotiation.Product();
newProduct.alternateProductID = sourceProduct.AutoID;
newProduct.isFromFeed = true;
newProduct.isDeleted = false;
newProduct.SKU = sourceProduct.StockNumber;
company.Product.Add(newProduct);
}
}
_dbRiv.SaveChanges(); // ### THIS BREAKS ### //
}
}
}
break;
}
}
}
Model #1 - This model sits in a database on our Dev Server.
Model #1 http://content.screencast.com/users/Keith.Barrows/folders/Jing/media/bdb2b000-6e60-4af0-a7a1-2bb6b05d8bc1/Model1.png
Model #2 - This model sits in a database on our Prod Server and is updated each day by automatic feeds. alt text http://content.screencast.com/users/Keith.Barrows/folders/Jing/media/4260259f-bce6-43d5-9d2a-017bd9a980d4/Model2.png
Note - The red circled items in Model #1 are the fields I use to "map" to Model #2. Please ignore the red circles in Model #2: that is from another question I had which is now answered.
Note: I still need to put in an isDeleted check so I can soft delete it from DB1 if it has gone out of our client's inventory.
All I want to do, with this particular code, is connect a company in DB1 with a client in DB2, get their product list from DB2 and INSERT it in DB1 if it is not already there. First time through should be a full pull of inventory. Each time it is run there after nothing should happen unless new inventory came in on the feed over night.
So the big question - how to I solve the transaction error I am getting? Do I need to drop and recreate my context each time through the loops (does not make sense to me)?
After much pulling out of hair I discovered that the foreach loops were the culprits. What needs to happen is to call EF but return it into an IList<T> of that target type then loop on the IList<T>.
Example:
IList<Client> clientList = from a in _dbFeed.Client.Include("Auto") select a;
foreach (RivWorks.Model.NegotiationAutos.Client client in clientList)
{
var companyFeedDetailList = from a in _dbRiv.AutoNegotiationDetails where a.ClientID == client.ClientID select a;
// ...
}
As you've already identified, you cannot save from within a foreach that is still drawing from the database via an active reader.
Calling ToList() or ToArray() is fine for small data sets, but when you have thousands of rows, you will be consuming a large amount of memory.
It's better to load the rows in chunks.
public static class EntityFrameworkUtil
{
public static IEnumerable<T> QueryInChunksOf<T>(this IQueryable<T> queryable, int chunkSize)
{
return queryable.QueryChunksOfSize(chunkSize).SelectMany(chunk => chunk);
}
public static IEnumerable<T[]> QueryChunksOfSize<T>(this IQueryable<T> queryable, int chunkSize)
{
int chunkNumber = 0;
while (true)
{
var query = (chunkNumber == 0)
? queryable
: queryable.Skip(chunkNumber * chunkSize);
var chunk = query.Take(chunkSize).ToArray();
if (chunk.Length == 0)
yield break;
yield return chunk;
chunkNumber++;
}
}
}
Given the above extension methods, you can write your query like this:
foreach (var client in clientList.OrderBy(c => c.Id).QueryInChunksOf(100))
{
// do stuff
context.SaveChanges();
}
The queryable object you call this method on must be ordered. This is because Entity Framework only supports IQueryable<T>.Skip(int) on ordered queries, which makes sense when you consider that multiple queries for different ranges require the ordering to be stable. If the ordering isn't important to you, just order by primary key as that's likely to have a clustered index.
This version will query the database in batches of 100. Note that SaveChanges() is called for each entity.
If you want to improve your throughput dramatically, you should call SaveChanges() less frequently. Use code like this instead:
foreach (var chunk in clientList.OrderBy(c => c.Id).QueryChunksOfSize(100))
{
foreach (var client in chunk)
{
// do stuff
}
context.SaveChanges();
}
This results in 100 times fewer database update calls. Of course each of those calls takes longer to complete, but you still come out way ahead in the end. Your mileage may vary, but this was worlds faster for me.
And it gets around the exception you were seeing.
EDIT I revisited this question after running SQL Profiler and updated a few things to improve performance. For anyone who is interested, here is some sample SQL that shows what is created by the DB.
The first loop doesn't need to skip anything, so is simpler.
SELECT TOP (100) -- the chunk size
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
FROM [dbo].[Clients] AS [Extent1]
ORDER BY [Extent1].[Id] ASC
Subsequent calls need to skip previous chunks of results, so introduces usage of row_number:
SELECT TOP (100) -- the chunk size
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
FROM (
SELECT [Extent1].[Id] AS [Id], [Extent1].[Name] AS [Name], row_number()
OVER (ORDER BY [Extent1].[Id] ASC) AS [row_number]
FROM [dbo].[Clients] AS [Extent1]
) AS [Extent1]
WHERE [Extent1].[row_number] > 100 -- the number of rows to skip
ORDER BY [Extent1].[Id] ASC
We have now posted an official response to the bug opened on Connect. The workarounds we recommend are as follows:
This error is due to Entity Framework creating an implicit transaction during the SaveChanges() call. The best way to work around the error is to use a different pattern (i.e., not saving while in the midst of reading) or by explicitly declaring a transaction. Here are three possible solutions:
// 1: Save after iteration (recommended approach in most cases)
using (var context = new MyContext())
{
foreach (var person in context.People)
{
// Change to person
}
context.SaveChanges();
}
// 2: Declare an explicit transaction
using (var transaction = new TransactionScope())
{
using (var context = new MyContext())
{
foreach (var person in context.People)
{
// Change to person
context.SaveChanges();
}
}
transaction.Complete();
}
// 3: Read rows ahead (Dangerous!)
using (var context = new MyContext())
{
var people = context.People.ToList(); // Note that this forces the database
// to evaluate the query immediately
// and could be very bad for large tables.
foreach (var person in people)
{
// Change to person
context.SaveChanges();
}
}
Indeed you cannot save changes inside a foreach loop in C# using Entity Framework.
context.SaveChanges() method acts like a commit on a regular database system (RDMS).
Just make all changes (which Entity Framework will cache) and then save all of them at once calling SaveChanges() after the loop (outside of it), like a database commit command.
This works if you can save all changes at once.
Just put context.SaveChanges() after end of your foreach(loop).
Making your queryable lists to .ToList() and it should work fine.
FYI: from a book and some lines adjusted because it's still valid:
Invoking SaveChanges() method begins a transaction which automatically rolls back all changes persisted to the database if an exception occurs before iteration completes; otherwise the transaction commits. You might be tempted to apply the method after each entity update or deletion rather than after iteration completes, especially when you're updating or deleting massive numbers of entities.
If you try to invoke SaveChanges() before all data has been processed, you incur a "New transaction is not allowed because there are other threads running in the session" exception. The exception occurs because SQL Server doesn't permit starting a new transaction on a connection that has a SqlDataReader open, even with Multiple Active Record Sets (MARS) enabled by the connection string (EF's default connection string enables MARS)
Sometimes its better to understand why things are happening ;-)
Always Use your selection as List
Eg:
var tempGroupOfFiles = Entities.Submited_Files.Where(r => r.FileStatusID == 10 && r.EventID == EventId).ToList();
Then Loop through the Collection while save changes
foreach (var item in tempGroupOfFiles)
{
var itemToUpdate = item;
if (itemToUpdate != null)
{
itemToUpdate.FileStatusID = 8;
itemToUpdate.LastModifiedDate = DateTime.Now;
}
Entities.SaveChanges();
}
I was getting this same issue but in a different situation. I had a list of items in a list box. The user can click an item and select delete but I am using a stored proc to delete the item because there is a lot of logic involved in deleting the item. When I call the stored proc the delete works fine but any future call to SaveChanges will cause the error. My solution was to call the stored proc outside of EF and this worked fine. For some reason when I call the stored proc using the EF way of doing things it leaves something open.
We started seeing this error "New transaction is not allowed because there are other threads running in the session" after migrating from EF5 to EF6.
Google brought us here but we are not calling SaveChanges() inside the loop. The errors were raised when executing a stored procedure using the ObjectContext.ExecuteFunction inside a foreach loop reading from the DB.
Any call to ObjectContext.ExecuteFunction wraps the function in a transaction. Beginning a transaction while there is already an open reader causes the error.
It is possible to disable wrapping the SP in a transaction by setting the following option.
_context.Configuration.EnsureTransactionsForFunctionsAndCommands = false;
The EnsureTransactionsForFunctionsAndCommands option allows the SP to run without creating its own transaction and the error is no longer raised.
DbContextConfiguration.EnsureTransactionsForFunctionsAndCommands Property
Here are another 2 options that allow you to invoke SaveChanges() in a for each loop.
The first option is use one DBContext to generate your list objects to iterate through, and then create a 2nd DBContext to call SaveChanges() on. Here is an example:
//Get your IQueryable list of objects from your main DBContext(db)
IQueryable<Object> objects = db.Object.Where(whatever where clause you desire);
//Create a new DBContext outside of the foreach loop
using (DBContext dbMod = new DBContext())
{
//Loop through the IQueryable
foreach (Object object in objects)
{
//Get the same object you are operating on in the foreach loop from the new DBContext(dbMod) using the objects id
Object objectMod = dbMod.Object.Find(object.id);
//Make whatever changes you need on objectMod
objectMod.RightNow = DateTime.Now;
//Invoke SaveChanges() on the dbMod context
dbMod.SaveChanges()
}
}
The 2nd option is to get a list of database objects from the DBContext, but to select only the id's. And then iterate through the list of id's (presumably an int) and get the object corresponding to each int, and invoke SaveChanges() that way. The idea behind this method is grabbing a large list of integers, is a lot more efficient then getting a large list of db objects and calling .ToList() on the entire object. Here is an example of this method:
//Get the list of objects you want from your DBContext, and select just the Id's and create a list
List<int> Ids = db.Object.Where(enter where clause here)Select(m => m.Id).ToList();
var objects = Ids.Select(id => db.Objects.Find(id));
foreach (var object in objects)
{
object.RightNow = DateTime.Now;
db.SaveChanges()
}
If you get this error due to foreach and you really need to save one entity first inside loop and use generated identity further in loop, as was in my case, the easiest solution is to use another DBContext to insert entity which will return Id and use this Id in outer context
For example
using (var context = new DatabaseContext())
{
...
using (var context1 = new DatabaseContext())
{
...
context1.SaveChanges();
}
//get id of inserted object from context1 and use is.
context.SaveChanges();
}
I was also facing same issue.
Here is the cause and solution.
http://blogs.msdn.com/b/cbiyikoglu/archive/2006/11/21/mars-transactions-and-sql-error-3997-3988-or-3983.aspx
Make sure before firing data manipulation commands like inserts, updates, you have closed all previous active SQL readers.
Most common error is functions that read data from db and return values.
For e.g functions like isRecordExist.
In this case we immediately return from the function if we found the record and forget to close the reader.
So in the project were I had this exact same issue the problem wasn't in the foreach or the .toList() it was actually in the AutoFac configuration we used.
This created some weird situations were the above error was thrown but also a bunch of other equivalent errors were thrown.
This was our fix:
Changed this:
container.RegisterType<DataContext>().As<DbContext>().InstancePerLifetimeScope();
container.RegisterType<DbFactory>().As<IDbFactory>().SingleInstance();
container.RegisterType<UnitOfWork>().As<IUnitOfWork>().InstancePerRequest();
To:
container.RegisterType<DataContext>().As<DbContext>().As<DbContext>();
container.RegisterType<DbFactory>().As<IDbFactory>().As<IDbFactory>().InstancePerLifetimeScope();
container.RegisterType<UnitOfWork>().As<IUnitOfWork>().As<IUnitOfWork>();//.InstancePerRequest();
I know it is an old question but i faced this error today.
and i found that, this error can be thrown when a database table trigger gets an error.
for your information, you can check your tables triggers too when you get this error.
I needed to read a huge ResultSet and update some records in the table.
I tried to use chunks as suggested in Drew Noakes's answer.
Unfortunately after 50000 records I've got OutofMemoryException.
The answer Entity framework large data set, out of memory exception explains, that
EF creates second copy of data which uses for change detection (so
that it can persist changes to the database). EF holds this second set
for the lifetime of the context and its this set thats running you out
of memory.
The recommendation is to re-create your context for each batch.
So I've retrieved Minimal and Maximum values of the primary key- the tables have primary keys as auto incremental integers.Then I retrieved from the database chunks of records by opening context for each chunk. After processing the chunk context closes and releases the memory. It insures that memory usage is not growing.
Below is a snippet from my code:
public void ProcessContextByChunks ()
{
var tableName = "MyTable";
var startTime = DateTime.Now;
int i = 0;
var minMaxIds = GetMinMaxIds();
for (int fromKeyID= minMaxIds.From; fromKeyID <= minMaxIds.To; fromKeyID = fromKeyID+_chunkSize)
{
try
{
using (var context = InitContext())
{
var chunk = GetMyTableQuery(context).Where(r => (r.KeyID >= fromKeyID) && (r.KeyID < fromKeyID+ _chunkSize));
try
{
foreach (var row in chunk)
{
foundCount = UpdateRowIfNeeded(++i, row);
}
context.SaveChanges();
}
catch (Exception exc)
{
LogChunkException(i, exc);
}
}
}
catch (Exception exc)
{
LogChunkException(i, exc);
}
}
LogSummaryLine(tableName, i, foundCount, startTime);
}
private FromToRange<int> GetminMaxIds()
{
var minMaxIds = new FromToRange<int>();
using (var context = InitContext())
{
var allRows = GetMyTableQuery(context);
minMaxIds.From = allRows.Min(n => (int?)n.KeyID ?? 0);
minMaxIds.To = allRows.Max(n => (int?)n.KeyID ?? 0);
}
return minMaxIds;
}
private IQueryable<MyTable> GetMyTableQuery(MyEFContext context)
{
return context.MyTable;
}
private MyEFContext InitContext()
{
var context = new MyEFContext();
context.Database.Connection.ConnectionString = _connectionString;
//context.Database.Log = SqlLog;
return context;
}
FromToRange is a simple structure with From and To properties.
Recently I faced the same issue in my project so posting my experience and it might help some on the same boat as i was. The issue was due to i am looping through the results of EF select query (results are not retrieved into memory).
var products = (from e in _context.Products
where e.StatusId == 1
select new { e.Name, e.Type });
foreach (var product in products)
{
//doing some insert EF Queries
//some EF select quries
await _context.SaveChangesAsync(stoppingToken); // This code breaks.
}
I have updated my Products select query to bring the results into LIST rather than IQueryable (This seems to be opening the reader throughout for each loop and hence save was failing).
var products = (from e in _context.Products
where e.StatusId == 1
select new { e.Name, e.Type })**.ToList()**; //see highlighted
The code below works for me:
private pricecheckEntities _context = new pricecheckEntities();
...
private void resetpcheckedtoFalse()
{
try
{
foreach (var product in _context.products)
{
product.pchecked = false;
_context.products.Attach(product);
_context.Entry(product).State = EntityState.Modified;
}
_context.SaveChanges();
}
catch (Exception extofException)
{
MessageBox.Show(extofException.ToString());
}
productsDataGrid.Items.Refresh();
}
In my case, the problem appeared when I called Stored Procedure via EF and then later SaveChanges throw this exception. The problem was in calling the procedure, the enumerator was not disposed. I fixed the code following way:
public bool IsUserInRole(string username, string roleName, DataContext context)
{
var result = context.aspnet_UsersInRoles_IsUserInRoleEF("/", username, roleName);
//using here solved the issue
using (var en = result.GetEnumerator())
{
if (!en.MoveNext())
throw new Exception("emty result of aspnet_UsersInRoles_IsUserInRoleEF");
int? resultData = en.Current;
return resultData == 1;//1 = success, see T-SQL for return codes
}
}
I am much late to the party but today I faced the same error and how I resolved was simple. My scenario was similar to this given code I was making DB transactions inside of nested for-each loops.
The problem is as a Single DB transaction takes a little bit time longer than for-each loop so once the earlier transaction is not complete then the new traction throws an exception, so the solution is to create a new object in the for-each loop where you are making a db transaction.
For the above mentioned scenarios the solution will be like this:
foreach (RivWorks.Model.Negotiation.AutoNegotiationDetails companyFeedDetail in companyFeedDetailList)
{
private RivWorks.Model.Negotiation.RIV_Entities _dbRiv = RivWorks.Model.Stores.RivEntities(AppSettings.RivWorkEntities_connString);
if (companyFeedDetail.FeedSourceTable.ToUpper() == "AUTO")
{
var company = (from a in _dbRiv.Company.Include("Product") where a.CompanyId == companyFeedDetail.CompanyId select a).First();
foreach (RivWorks.Model.NegotiationAutos.Auto sourceProduct in client.Auto)
{
foreach (RivWorks.Model.Negotiation.Product targetProduct in company.Product)
{
if (targetProduct.alternateProductID == sourceProduct.AutoID)
{
found = true;
break;
}
}
if (!found)
{
var newProduct = new RivWorks.Model.Negotiation.Product();
newProduct.alternateProductID = sourceProduct.AutoID;
newProduct.isFromFeed = true;
newProduct.isDeleted = false;
newProduct.SKU = sourceProduct.StockNumber;
company.Product.Add(newProduct);
}
}
_dbRiv.SaveChanges(); // ### THIS BREAKS ### //
}
}
I am a little bit late, but I had this error too. I solved the problem by checking what where the values that where updating.
I found out that my query was wrong and that there where over 250+ edits pending. So I corrected my query, and now it works correct.
So in my situation: Check the query for errors, by debugging over the result that the query returns. After that correct the query.
Hope this helps resolving future problems.
My situation was similar others above. I had an IQueryable which I was doing a foreach on. This in turn called a method with SaveChanges(). Booom exception here as there was already a transaction open from the query above.
// Example:
var myList = _context.Table.Where(x => x.time == null);
foreach(var i in myList)
{
MyFunction(i); // <<-- Has _context.SaveChanges() which throws exception
}
Adding ToList() to the end of the query was the solution in my case.
// Fix
var myList = _context.Table.Where(x => x.time == null).ToList();
Most of answers related with loops. But my problem was different. While i was trying to use multiple dbcontext.Savechanges() command in same scope, i got the error many times.
In my case for ef core 3.1 using
dbcontext.Database.BeginTransaction()
and
dbcontext.Database.CommitTransaction();
has fixed the problem. Here is my entire Code :
public IActionResult ApplyForCourse()
{
var master = _userService.GetMasterFromCurrentUser();
var trainee = new Trainee
{
CourseId = courseId,
JobStatus = model.JobStatus,
Gender = model.Gender,
Name = model.Name,
Surname = model.Surname,
Telephone = model.Telephone,
Email = model.Email,
BirthDate = model.BirthDate,
Description = model.Description,
EducationStatus = EducationStatus.AppliedForEducation,
TraineeType = TraineeType.SiteFirst
};
dbcontext.Trainees.Add(trainee);
dbcontext.SaveChanges();
dbcontext.Database.BeginTransaction();
var user = userManager.GetUserAsync(User).Result;
master.TraineeId = trainee.Id;
master.DateOfBirth = model.BirthDate;
master.EducationStatus = trainee.EducationStatus;
user.Gender = model.Gender;
user.Email = model.Email;
dbcontext.Database.CommitTransaction();
dbcontext.SaveChanges();
return RedirectToAction("Index", "Home");
}
}
I have a Entity called "Client", and another Entity called "Card".
A Client may have many Cards.
My Client Entity looks like this:
public class Client{
public virtual ICollection<Card> Cards {get; set;}
}
Now I want to show the Client data in a DataGrid in WPF, and I want to get Cards Count data,so I add a property to Client Entity, which like this:
public class Client{
public virtual ICollection<Card> Cards {get; set;}
public int CardCount
{
return Cards.Count;
}
}
And then I query the data with Linq and Bind to view
var query = from n in db.Clients select n;
When I run the Application, I got a Exception just right on the return Cards.Count; line;
System.ObjectDisposedException
The ObjectContext instance has been disposed and can no longer be used for operations that require a connection.
So how could I correctly get the cards count?
There is a way simpler method than the other answers here show. Please also realize that solutions such as
var client = db.Clients.FirstOrDefault(c=> c.Id = someid); //get a client
if (client != null)
{
cardCount = client.Cards.Count;
}
will cause an issue called Select N+1 problem. Read up on it if interested, but in a nutshell, it means the following:
Because you are not only interested in one exact client, but you want to display N clients, you need to do one (1) query to get just the clients. Then, by doing the FirstOrDefault stuff, you are actually doing one (1) extra db roundtrip to the database per Client record, which results in an additional N * 1 = N roundtrips. What this means that, if you were to just query the Clients without any related data, you could get however many client records you like, in just one query. But by fetching related data to each of them one-by-one, you are doing way too many db roundtrips.
Here is a way to solve this issue, by using joins and projections. You can get all the data you need in a single DB access:
using (var context = GetDbContext())
{
return context.Clients.Select(cli => new YourViewModel
{
Name = cli.FullName,
// Other prop setters go here
CardCount = cli.Cards.Count
}).Skip((page - 1) * pageSize).Take(pageSize).ToList();
}
You might be wondering, what's the difference afterall? Well, here, you are not working with materialized objects, as others call them here, but with a DbContext. By applying the proper LINQ operators to it (note, that this works not just with DbContext, but also with any IQueryable (well obviously not if you call AsQueryable() on an already in-memory collection but whatever)), LINQ to Entities can construct a proper SQL to join the tables and project the results and therefore you fetch all required data in one go. Note that LINQ to Entities IS ABLE to translate the cli.Cards.Count into a proper SQL Count statement.
You can get the count without loading the entities like this:
using (var context = new MyContext())
{
var client = context.Client.Find(clientId);
// Count how many cards the client has
var cardsCount = context.Entry(client)
.Collection(b => b.Cards)
.Query()
.Count();
}
More information on MSDN page.
You get an ObjectDisposedException if you do not materialize the retreived query. In the following case, the query gets executed only when you Access the first time the list from GetNonMaterialized and not before leaving the method. Fact of this the db is disposed because of lost of scope.
public IEnumerable<Client> GetNonMaterialized()
{
return from n in db.Clients select n;
}
In the following case the query is executed before leaving the method.
public IEnumerable<Client> GetMaterialized()
{
return (from n in db.Clients select n).ToList();
}
Always be sure that the query is executed before exiting the scope of a ObjectContext.
If you want to know whether the query is executed and when enalbe Logging of EF.
How can I turn off Entity Framework 6.1 logging?
I'm having an issue with Entity Framework, when I execute SaveChanges, the context has quite a few objects associated with it, some of the objects are updated and some are added, afterwards I want to use the Id's of all these items (the Id's for the added items are only assigned on insert in the database). After the save changes a list of all the objects is empty.
I've seen samples on the site where the object is updated after the save so I suspect it might be how I'm getting the list of objects in the first place
Here's my code:
// Lots of processing to create or update objects
using (var localContext = this.context)
{
var updatedObjects = localContext.ChangeTracker.Entries().Where(e => e.Entity is GenerationEvent && (e.State == EntityState.Modified || e.State == EntityState.Added));
var updatedEvents = updatedObjects.Select(e => (GenerationEvent)e.Entity);
// The list has 5 items in at this point
localContext.SaveChanges();
// This list is now empty
DoSomethingWithList(updatedEvents);
}
Thanks in advance for any help.
The variable updatedEvents is a Linq query. Linq queries aren't executed immediately. By the time it is executed in your code it won't find any updated object anymore. Putting .ToList() after the Linq query will execute it immediately.
var updatedEvents = updatedObjects.Select(e => (GenerationEvent)e.Entity).ToList();
first your "using" statement is odd.
it should be
using (var context = new MyContext()) //passing in optional connection string.
{
}
Then the way you access your entities seem odd or i have no clue what you are doing there...
var updatedObjects = localContext.ChangeTracker.Entries().Where(e => e.Entity is GenerationEvent && (e.State == EntityState.Modified || e.State == EntityState.Added));
var updatedEvents = updatedObjects.Select(e => (GenerationEvent)e.Entity);
Seems like you are asking the context for all items which are considered "Add" or "Updated"
Then you are accepting the changes to the context. eg SaveChanges().
I fully expect "updatedEvents" to be empty, after save-changes is called.
Change you stuff... to something like
using (var context = new MyContext()) //passing in optional connection string.
{
LIst<EntityType> listOfChangedEntities = //ToDo:either passed in or generated
context.EntityType.AddRandge(listOfChangedEntities);
context.SaveChanges();
//after SaveChanges has been done all the entities in the
//listOfChangedEntities will now have there id's
//for update, fetch the entities... change them and Update them
}
I suspect that you are trying to create some sort of generic code to handle any type of Entity without specifying its type. Your code is not suited for this as it is, if this is what you are trying to do, I would modify the question to ask what you are trying to achieve. But the above is the Normal way of getting the Id's of the entities which have been inserted.
The other examples you are passably talking about is where they use foreign keys and navigation properties to automatically associate related entities, but your code looks way off from that.
UPDATE
routine
public static DoWork()
{
var context = new MyContext();
List<GenerationEvent > internalEntityType = new List<GenerationEvent ();
foreach(var item in SomeList)
{
var newItemEntity = new GenerationEvent();
newItemEntity.Name = "Test";
context.GenerationEvent.Add(newItemEntity);
//add to internal list
internalEntityType.Add(newItemEntity )
}
context.SaveChanges();
var first_id = internalEntityType.FirstOrDefault().Id;
//first_id will not be 0 it will be the Id the database gave it.
}
I am creating small application in which i have used LINQ To SQL to perform all operation to database.
Now here i am giving the small part of my database structure please take a look.
So update language detail i am getting the object of login using the datacontext something like this.
XVDataContext Context = new XVDataContext ();
var myQuery = from objLogIn in Context.GetTable<LogIn>() where objLogIn.Emp_Id == nEmpId select objLogIn;
In nEmpId i will always have some value.
So it is not creating any problem in fact i am getting the required record from DB and storing it in objUser object using the following code.
LogIn objUser = myQuery.First<LogIn>();
Now to update LanguageDetail i am executing following code but it throws Exception when i execute SubmitChanges line.
Here is the code that i am executing to update.
LanguageDetail obj = new LanguageDetail();
foreach (string sLanguages in TextBoxLanguagesKnown.Text.Split('\n'))
{
obj.Emp_Id = objUser.Emp_Id;
obj.Language = sLanguages.Trim();
}
objUser.LanguageDetails[0] = obj;
Context.SubmitChanges();
I already read following links.
cannot add an entity with a key that is already in use
LINQ To SQL exception with Attach(): Cannot add an entity with a key that is alredy in use
Cannot add an entity with a key that is already in use (LINQ)
By reading the above links i found that i am doing some mistake in ID fields but still i am unable to resolve.
Please tell me the clear understanding of raising this issue and how can i resolve this.
EDIT:
I simply want to update LanguageDetail table.
When i try to add new object using following code it still throws exception.
objUser.LanguageDetail.Add(obj);
You might want to add / remove languages for specific user by using following code.
var languages = TextBoxLanguagesKnown.Text.Split('\n');
// Removes deleted languages (first find all language details that are missing from the UI).
var deletedLanguages = objUser.LanguageDetails.Where(ld => !languages
.Any(l => ld.Language == l.Trim())).ToArray();
foreach(var deletedLanguage in deletedLanguages)
{
objUser.LanguageDetails.Remove(deletedLanguage);
Context.LanguageDetails.DeleteOnSubmit(deletedLanguage);
}
// Adds new languages (then adds new language details that are not found in the database).
var newLanguages = languages.Where(l => !objUser.LanguageDetails
.Any(ld => ld.Language == l.Trim())).ToArray();
foreach (string newLanguage in newLanguages)
{
var languageDetail = new LanguageDetail
{
Emp_Id = objUser.Emp_Id,
Language = newLanguage.Trim()
};
objUser.LanguageDetails.Add(languageDetail);
}
Context.SubmitChanges();
From my understanding you want to update the LanguageDetail entity in your database. In order to do so you have to do one of the following:
Retrieve the original LanguageDetail object based on its id, and update that object instead of creating a new one and assigning it the id of an existing object.
Attach the newly created object to your context instead of just giving a reference to it to your LanguageDetails collection.
The exception you are seeing happens because the way linq to sql behaves is that it threats the obj as a new object that you want to insert and because of that it tries to insert it into the language details table.
Modifying your code like that should work:
Context.LanguageDetails.Attach(obj);
objUser.Employee_LanguageDetails[0] = obj;
How can I quickly remove all rows in the table using Entity Framework?
I am currently using:
var rows = from o in dataDb.Table
select o;
foreach (var row in rows)
{
dataDb.Table.Remove(row);
}
dataDb.SaveChanges();
However, it takes a long time to execute.
Are there any alternatives?
For those that are googling this and ended up here like me, this is how you currently do it in EF5 and EF6:
context.Database.ExecuteSqlCommand("TRUNCATE TABLE [TableName]");
Assuming context is a System.Data.Entity.DbContext
Edit:
Currently in net6.0 (dotnet 6 core) you can do the following:
context.Database.ExecuteSqlRaw("TRUNCATE TABLE [TableName]");
Or use the Async overload:
await context.Database.ExecuteSqlRawAsync("TRUNCATE TABLE [TableName]");
These are extension methods coming from Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions
If you're having issues with foreign keys (in MySql), you might have to do the following (Doing the SET FOREIGN_KEY_CHECKS = 0; part in a separate call does not seem to work for me)
context.Database.ExecuteSqlRaw("SET FOREIGN_KEY_CHECKS = 0; TRUNCATE TABLE [TableName];");
So if you want to truncate your entire database (Possibly for unittesting reasons) - you can do the following:
var tableNames = context.Model.GetEntityTypes()
.Select(t => t.GetTableName())
.Distinct()
.ToList();
foreach (var tableName in tableNames)
{
context.Database.ExecuteSqlRaw($"SET FOREIGN_KEY_CHECKS = 0; TRUNCATE TABLE `{tableName}`;");
}
Warning: The following is only suitable for small tables (think < 1000 rows)
Here is a solution that uses entity framework (not SQL) to delete the rows, so it is not SQL Engine(R/DBM) specific.
This assumes that you're doing this for testing or some similar situation.
Either
The amount of data is small or
The performance doesn't matter
Simply call:
VotingContext.Votes.RemoveRange(VotingContext.Votes);
Assuming this context:
public class VotingContext : DbContext
{
public DbSet<Vote> Votes{get;set;}
public DbSet<Poll> Polls{get;set;}
public DbSet<Voter> Voters{get;set;}
public DbSet<Candidacy> Candidates{get;set;}
}
For tidier code you can declare the following extension method:
public static class EntityExtensions
{
public static void Clear<T>(this DbSet<T> dbSet) where T : class
{
dbSet.RemoveRange(dbSet);
}
}
Then the above becomes:
VotingContext.Votes.Clear();
VotingContext.Voters.Clear();
VotingContext.Candidacy.Clear();
VotingContext.Polls.Clear();
await VotingTestContext.SaveChangesAsync();
I recently used this approach to clean up my test database for each testcase run (it´s obviously faster than recreating the DB from scratch each time, though I didn´t check the form of the delete commands that were generated).
Why can it be slow?
EF will get ALL the rows (VotingContext.Votes)
and then will use their IDs (not sure exactly how, doesn't matter), to delete them.
So if you're working with serious amount of data you'll kill the SQL server process (it will consume all the memory) and same thing for the IIS process since EF will cache all the data same way as SQL server. Don't use this one if your table contains serious amount of data.
Using SQL's TRUNCATE TABLE command will be the fastest as it operates on the table and not on individual rows.
dataDb.ExecuteStoreCommand("TRUNCATE TABLE [Table]");
Assuming dataDb is a DbContext (not an ObjectContext), you can wrap it and use the method like this:
var objCtx = ((System.Data.Entity.Infrastructure.IObjectContextAdapter)dataDb).ObjectContext;
objCtx.ExecuteStoreCommand("TRUNCATE TABLE [Table]");
var all = from c in dataDb.Table select c;
dataDb.Table.RemoveRange(all);
dataDb.SaveChanges();
using (var context = new DataDb())
{
var ctx = ((System.Data.Entity.Infrastructure.IObjectContextAdapter)context).ObjectContext;
ctx.ExecuteStoreCommand("DELETE FROM [TableName] WHERE Name= {0}", Name);
}
or
using (var context = new DataDb())
{
context.Database.ExecuteSqlCommand("TRUNCATE TABLE [TableName]");
}
You can do that without Foreach
dataDB.Table.RemoveRange(dataDB.Table);
dataDB.SaveChanges();
This will remove all rows
This avoids using any sql
using (var context = new MyDbContext())
{
var itemsToDelete = context.Set<MyTable>();
context.MyTables.RemoveRange(itemsToDelete);
context.SaveChanges();
}
context.TableName.RemoveRange(context.TableName);
context.SaveChanges();
I came across this question when I had to deal with a particular case: fully updating of content in a "leaf" table (no FKs pointing to it). This involved removing all rows and putting new rows information and it should be done transactionally (I do not want to end up with an empty table, if inserts fails for whatever reason).
I have tried the public static void Clear<T>(this DbSet<T> dbSet) approach, but new rows are not inserted. Another disadvante is that the whole process is slow, as rows are deleted one by one.
So, I have switched to TRUNCATE approach, since it is much faster and it is also ROLLBACKable. It also resets the identity.
Example using repository pattern:
public class Repository<T> : IRepository<T> where T : class, new()
{
private readonly IEfDbContext _context;
public void BulkInsert(IEnumerable<T> entities)
{
_context.BulkInsert(entities);
}
public void Truncate()
{
_context.Database.ExecuteSqlCommand($"TRUNCATE TABLE {typeof(T).Name}");
}
}
// usage
DataAccess.TheRepository.Truncate();
var toAddBulk = new List<EnvironmentXImportingSystem>();
// fill toAddBulk from source system
// ...
DataAccess.TheRepository.BulkInsert(toAddBulk);
DataAccess.SaveChanges();
Of course, as already mentioned, this solution cannot be used by tables referenced by foreign keys (TRUNCATE fails).
EF Core 7.0 solves this problem once and for all by adding bulk update and delete semantics:
await dataDB.Table.ExecuteDeleteAsync();
Note that this syntax immediately executes the underlying (SQL) command to delete the data from the table. It does not fiddle around with tracking the entity, marking it for deletion, and waiting for UpdateDatabase to execute the transaction against the database.
Also note that multiple ExecuteDelete and ExecuteUpdate commands will not be contained in a single transaction by default. However, the DbContext transaction APIs can be used in the normal way to wrap these commands in a transaction.
If
using(var db = new MyDbContext())
{
await db.Database.ExecuteSqlCommandAsync(#"TRUNCATE TABLE MyTable"););
}
causes
Cannot truncate table 'MyTable' because it is being referenced by a FOREIGN KEY constraint.
I use this:
using(var db = new MyDbContext())
{
await db.Database.ExecuteSqlCommandAsync(#"DELETE FROM MyTable WHERE ID != -1");
}
var data = (from n in db.users select n);
db.users.RemoveRange(data);
db.SaveChanges();
The following works on SQLite database (using Entity Framework).
It seems that the fastest way to clear all the db tables is using context.Database.ExecuteSqlCommand("some SQL"), as some comments above highlighted as well. Here I am going to show how to reset the 'index' count of tables too.
context.Database.ExecuteSqlCommand("delete from TableA");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableA'");//resets the autoindex
context.Database.ExecuteSqlCommand("delete from TableB");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableB'");//resets the autoindex
context.Database.ExecuteSqlCommand("delete from TableC");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableC'");//resets the autoindex
One important point is that if you use foreign keys in your tables, you must first delete the child table before the parent table, so the sequence (hierarchy) of tables during deletion is important, otherwise a SQLite exception may occur.
Note: var context = new YourContext()
If you wish to clear your entire database.
Because of the foreign-key constraints it matters which sequence the tables are truncated. This is a way to bruteforce this sequence.
public static void ClearDatabase<T>() where T : DbContext, new()
{
using (var context = new T())
{
var tableNames = context.Database.SqlQuery<string>("SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE' AND TABLE_NAME NOT LIKE '%Migration%'").ToList();
foreach (var tableName in tableNames)
{
foreach (var t in tableNames)
{
try
{
if (context.Database.ExecuteSqlCommand(string.Format("TRUNCATE TABLE [{0}]", tableName)) == 1)
break;
}
catch (Exception ex)
{
}
}
}
context.SaveChanges();
}
}
usage:
ClearDatabase<ApplicationDbContext>();
remember to reinstantiate your DbContext after this.
In EFCore (version i am using is 3.1) you can use the following to remove all rows -
context.Database.ExecuteSqlRaw("TRUNCATE TABLE [TableName]");
This works for me... EF v3.1.5
context.ModelName.RemoveRange(context.ModelName.ToList());
context.SaveChanges();
This works Properly in EF 5:
YourEntityModel myEntities = new YourEntityModel();
var objCtx = ((IObjectContextAdapter)myEntities).ObjectContext;
objCtx.ExecuteStoreCommand("TRUNCATE TABLE [TableName]");
Delete all records. Do not reset the primary index like "truncate".
/// <summary>
/// SET - DELETE all record by table - no truncate - return deleted records
/// </summary>
public static int setListDelAllMYTABLE()
{
// INIT
int retObj = 0;
using (MYDBEntities ctx = new MYDBEntities())
{
// GET - all record
var tempAllRecord = ctx.MYTABLE.ToList();
// RESET
ctx.MYTABLE.RemoveRange(tempAllRecord);
// SET - final save
retObj += ctx.SaveChanges();
}
// RET
return retObj;
}
If MVC, you can do:
public async Task<IActionResult> DeleteAll()
{
var list = await _context.YourClass.ToListAsync();
_context.YourClass.RemoveRange(list);
await _context.SaveChangesAsync();
return RedirectToAction(nameof(Index));
}
Make sure when you are trying to delete parent all children will cascade on delete. Or children have nullable foreign key.
Here is a variation on the popular solution by Ron that avoids the use of hardcoded string table names by taking advantage of another popular solution on stack overflow for determining the underlying table name for an entity framework class.
With these extension methods the solution looks like this:
_dbContext.TruncateTable<TheTableName>();
(use this.TruncateTable<... if you're editing code within an EF DBContext class or partial class file)
And here's the extension class:
public static class EntityFrameworkExtensions
{
private static string ParseTableNameFromSQL(string sql)
{
Regex regex = new Regex("FROM (?<table>.*) AS");
Match match = regex.Match(sql);
string table = match.Groups["table"].Value;
return table;
}
public static string GetTableName<T>(this IObjectContextAdapter context) where T : class =>
ParseTableNameFromSQL(context.ObjectContext.CreateObjectSet<T>().ToTraceString());
public static void TruncateTable<T>(this DbContext dbContext) where T : class =>
dbContext.Database.ExecuteSqlCommand($"TRUNCATE TABLE {dbContext.GetTableName<T>()}");
public static void DeleteAllTableRows<T>(this DbContext dbContext) where T : class =>
dbContext.Database.ExecuteSqlCommand($"DELETE FROM {dbContext.GetTableName<T>()}");
}
The last extension method DeleteAllTableRows is a useful alternative if your table cannot be truncated (e.g. due to foreign key references) - this is still much faster than the Entity Framework RemoveAll alternative.
Works for EF Core 3
public static class EntityExtensions
{
public static async Task ClearAsync<T>(this DbSet<T> dbSet) where T : class
{
var command = dbSet.CreateDbCommand();
command.CommandText = $"TRUNCATE TABLE {dbSet.EntityType.GetSchema()}.{dbSet.EntityType.GetTableName()}";
await command.ExecuteNonQueryAsync();
}
}
but please note that dbSet.CreateDbCommand is an extension
My solution, mixing my ideas, some answers (Ron one from this thread, but also this and this for reflection) and trying to cover some different conditions.
It is based on EF6, but it should work fine, just fixing some extensions like GetTableName<TEntity>.
My solution:
uses extensions, so you only need DbSet to launch
has a row count threshold, to decide between RemoveRange or SQL execution, to avoid perfomance issues
the SQL version is based on DELETE instead of TRUNCATE, to avoid foreign key issues (it has to fit your requirements, of course)
has a parameter to save changes inline
private const int RangeLimit = 100;
private static void ClearTable<TEntity>(this DbSet<TEntity> dataSet, bool saveChanges = true) where TEntity : class
{
DbContext context = null;
if (dataSet.Count() > RangeLimit)
{
context = dataSet.GetContext();
context.Database.ExecuteSqlCommand($"DELETE FROM [{context.GetTableName<TEntity>()}]");
}
else
{
dataSet.RemoveRange(dataSet);
}
if (!saveChanges)
{
return;
}
if (context == null)
{
context = dataSet.GetContext();
}
context.SaveChanges();
}
private static DbContext GetContext<TEntity>(this DbSet<TEntity> dbSet)
where TEntity : class
{
var internalSet = dbSet
.GetType()
.GetField("_internalSet", BindingFlags.NonPublic | BindingFlags.Instance)
?.GetValue(dbSet);
var internalContext = internalSet?.GetType().BaseType
?.GetField("_internalContext", BindingFlags.NonPublic | BindingFlags.Instance)
?.GetValue(internalSet);
return (DbContext)internalContext?.GetType()
.GetProperty("Owner", BindingFlags.Instance | BindingFlags.Public)
?.GetValue(internalContext, null);
}
public static string GetTableName<TEntity>(this DbContext context) where TEntity : class
{
return (context as IObjectContextAdapter).ObjectContext.CreateObjectSet<TEntity>().EntitySet.Name;
}
All you have to do, with a database table named Entries, is:
databaseContext.Entries.ClearTable();
if you want to save changes, or if you don't want:
databaseContext.Entries.ClearTable(false);
It is based on reflection, to simplify code. It has some performance tradeoff, of course, but reflection happens once for each table, hence should be completely acceptable in these conditions.
It is a very clean solution.
_context.RemoveRange(_context.ModelName);
_context.SaveChanges();
There are several issues with pretty much all the answers here:
1] Hard-coded sql. Will brackets work on all database engines?
2] Entity framework Remove and RemoveRange calls. This loads all entities into memory affected by the operation. Yikes.
3] Truncate table. Breaks with foreign key references and may not work accross all database engines.
Use https://entityframework-plus.net/, they handle the cross database platform stuff, translate the delete into the correct sql statement and don't load entities into memory, and the library is free and open source.
Disclaimer: I am not affiliated with the nuget package. They do offer a paid version that does even more stuff.