Azure storage table delete row by row key - c#

I am trying to delete row from azure storage filter by only rowkey value.
But I dont see any overload for delete operation where we can filter with only rowkey. Is there any alternative option to delete row from azure storage table for records with specific rowkey?
RemoveEntityByRowKey('123456');
public static void RemoveEntityByRowKey(string myRowKey)
{
try
{
CloudTable table = _tableClient.GetTableReference("MyAzureTable");
TableOperation delteOperation = TableOperation.Delete(myRowKey);
table.Execute(delteOperation);
}
catch (Exception ex)
{
LogError.LogErrorToAzure(ex);
throw;
}
}

In order to delete an entity, you would need both PartitionKey and RowKey (Delete Entity REST API). So what you would need to do is first fetch the entity with matching RowKey. Once you have fetched this entity, you should be able to call TableOperation.Delete as mentioned in the answers.
However, fetching entity by RowKey is not recommended because it will do a Full Table Scan. It may not be a problem if your table size is small but would be an issue where your table contains large number of entities. Furthermore, a RowKey is unique in a Partition i.e. in a table there can be only one entity with a PartitionKey/RowKey combination. In other words, you can potentially have entities with same RowKey in different Partitions. So when you fetch entities by RowKey only, you may get more than one entity back. You need to ensure that you're deleting correct entity.

If you know the PartitionKey as well as the RowKey, you don't need to retrieve the entire entity to delete it. You could adapt your code as follows:
public static void RemoveEntityByRowKey(string myRowKey)
{
try
{
var entity = new YourEntity
{
PartitionKey = "partition",
RowKey = myRowKey,
ETag = "*"
}
CloudTable table = _tableClient.GetTableReference("MyAzureTable");
TableOperation delteOperation = TableOperation.Delete(entity);
table.Execute(delteOperation);
}
catch (Exception ex)
{
LogError.LogErrorToAzure(ex);
throw;
}
}

If you're targeting .NET Core, you'll need to use the ExecuteQuerySegmentedAsync method to execute the filter condition query. ExecuteQuery is deprecated.
var cloudTableClient = _cloudStorageAccount.CreateCloudTableClient();
var myTable = cloudTableClient.GetTableReference("MyTable");
var query = new TableQuery<MyEntity>().Where(TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.Equal, "myRowKey"));
var segment = await myTable.ExecuteQuerySegmentedAsync(query, null);
var myEntities = segment.Results;

By row do you mean a record ?
TableOperation.Delete accepts a Table Entity. See here: https://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storage.table.tableoperation.delete.aspx
In order to delete that entity, you must first retrieve it by specifying its Partition key and/or Row key.
Look into TableQuery class here https://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storage.table.tablequery_methods.aspx
Once you retrieve it, pass it to Delete method.

There is no method
TableOperation.Delete(String rowKey),
only method
public static TableOperation delete(final TableEntity entity)
in TableOperation. For details, see Get started with Azure Table storage using .NET

With reference to Franks pointers, I am posting answer so that it would be useful to others who face the similar issue.
RemoveEntityByRowKey('123456');
public static void RemoveEntityByRowKey(string myRowKey)
{
try
{
CloudTable table = _tableClient.GetTableReference("MyAzureTable");
TableQuery<myEntity> query = new TableQuery<myEntity>()
.Where(TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.Equal, myRowKey));
foreach (var item in table.ExecuteQuery(query))
{
var oper = TableOperation.Delete(item);
table.Execute(oper);
}
}
catch (Exception ex)
{
LogErrorToAzure(ex);
throw;
}
}

mpl is my table row entity and is required to delete the record from the db.
I have added this answer to show an async (with result check)
if (result)
{
//delete the lead from the storage table
TableOperation delRow = TableOperation.Delete(mpl);
TableResult tr = await table.ExecuteAsync(delRow);
if (((int)tr.HttpStatusCode) < 400)
log.LogInformation($"Table Record: {mpl.RowKey} deleted");
else
log.LogError($"Table Record: {mpl.RowKey} NOT deleted");
}

The Windows.Azure.Storage namespace is now deprecated in favour of the Azure.Data.Tables namespace. As such the TableOperation.Delete method is also deprecated. You should now use a TableClient and it's DeleteEntity method:
TableClient tableClient = new TableClient(connectionString, Table);
tableClient.DeleteEntity(PartitionKey, RowKey);
There is also the async version DeleteEntityAsync if you wish too.

Related

How to delete all entities with a timestamp more than 1 day old from Azure Storage Table?

Azure storage tables all have a timestamp column. Based on documentation here the listed way to delete from a storage table is to select an entity then delete it.
Does anyone know how to delete any entity from a storage table based on a datetime comparison on the timestamp value using code?
EDIT:
Based on the advice given I wrote the following code. However, it throws a Bad Request exception on my table.ExecuteQuery(rangeQuery) call. Any advice?
StorageCredentials creds = new StorageCredentials(logAccountName, logAccountKey);
CloudStorageAccount account = new CloudStorageAccount(creds, useHttps: true);
CloudTableClient client = account.CreateCloudTableClient();
CloudTable table = client.GetTableReference(LogTable);
TableQuery<CloudQuerySummary> rangeQuery = new TableQuery<CloudQuerySummary>()
.Where(TableQuery.GenerateFilterCondition("Timestamp", QueryComparisons.LessThan
, DateTime.Now.AddHours(- DateTime.Now.Hour).ToString()));
TableOperation deleteOperation;
// Loop through the results, displaying information about the entity.
foreach (CloudQuerySummary entity in table.ExecuteQuery(rangeQuery))
{
deleteOperation = TableOperation.Delete(entity);
table.Execute(deleteOperation);
}
EDIT 2
Here is the final working code for anyone who chooses to copy/reference it.
public void DeleteLogsNotFromToday()
{
StorageCredentials creds = new StorageCredentials(logAccountName, logAccountKey);
CloudStorageAccount account = new CloudStorageAccount(creds, useHttps: true);
CloudTableClient client = account.CreateCloudTableClient();
CloudTable table = client.GetTableReference(LogTable);
TableQuery<CloudQuerySummary> rangeQuery = new TableQuery<CloudQuerySummary>()
.Where(TableQuery.GenerateFilterConditionForDate("Timestamp", QueryComparisons.LessThan
, DateTime.Now.AddHours(-DateTime.Now.Hour)));
try
{
TableOperation deleteOperation;
// Loop through the results, displaying information about the entity.
foreach (CloudQuerySummary entity in table.ExecuteQuery(rangeQuery))
{
deleteOperation = TableOperation.Delete(entity);
table.Execute(deleteOperation);
}
}
catch (Exception ex)
{
throw;
}
}
You will have to do a partition scan to do that, as entities are only indexed on their PartitionKey and RowKey.
In the tutorial link you posted, look at the section Retrieve a range of entities in a partition. Once you get the entities you want to delete, you will then execute a table operation to delete them.
If you don't want to delete them one by one, you can create a batch delete operation (provided all entities to delete have the same partition key). The link above also instructs how to construct a batch operation.
Alternatively, if you do not want to do a table scan, you should store the date reference (for instance, storing the date in milliseconds as the RowKey) and then use that to filter the entities you need to delete based on a date-time comparison (something similar to THIS)
UPDATE: I think the problem is in this line:
DateTime.Now.AddHours(- DateTime.Now.Hour).ToString()
As from the documentation:
The Timestamp property is a DateTime value that is maintained on the
server side to record the time an entity was last modified
You are trying to compare a DateTime property with a String. I'm no C# expert, but that does not look to me as a valid comparison.
If you use Slazure this kind of job becomes easier, the following code should also work with the Light(free) edition.
using SysSurge.Slazure;
using SysSurge.Slazure.Linq;
using SysSurge.Slazure.Linq.QueryParser;
namespace TableOperations
{
public class LogOperations
{
public static void DeleteOldLogEntities()
{
// Get a reference to the table storage, example just uses the development storage
dynamic storage = new QueryableStorage<DynEntity>("UseDevelopmentStorage=true");
// Get a reference to the table named "LogTable"
QueryableTable<DynEntity> logTable = storage.LogTable;
var query = logTable.Where("Timestamp > #0", DateTime.UtcNow.AddDays(-1));
// Delete all returned log entities
foreach (var entity in query)
logTable.Delete(entity.PartitionKey, entity.RowKey);
}
}
}
Full disclosure: I coded Slazure.

DbUpdateConcurrencyException for tables with foreign keys

Update: Ahh! Dumb error: I had two instances of a Repository available, and, our implemntation requires that we provide a connection string for each Repo, so the two Repos were pointing to different databases, and I was adding an entity from one DB to another, and its id probably could not be found in the updated DB.
I'm running .Net MVC 4.0, with EF 5. We implement a repository pattern. In the following Delete method of a WebApi controller I have the following code:
int userID = UserHelper.GetCurrentUserID();
DateTime now = DateTime.UtcNow;
ExhibitLinkRepository el = new ExhibitLinkRepository();
el.setCase = caseID;
ExhibitLink link = el.All.SingleOrDefault(l => l.id == id);
//Mark Link Deleted
link.usermodified_id = userID;
link.datetimemodified = now;
link.deleted_flag = true;
exhibitLinkRepository.InsertOrUpdate(link);
exhibitLinkRepository.SaveChanges();
Where exhibitLinkRepository.InsertOrUpdate:
public void InsertOrUpdate(ExhibitLink exhibitLink)
{
if (exhibitLink.id == default(int))
{
// New entity
context.ExhibitLinks.Add(exhibitLink);
}
else
{
// Existing entity
context.Entry(exhibitLink).State = EntityState.Modified;
}
}
When I invoke context.SaveChanges() I get the dreaded:
Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. Refresh ObjectStateManager entries.
Now, the ExhibitLink table has foreign key constraints as follows:
There happen to be triggers on the underlying DB for some of these related tables, but disabling them did not change the outcome.
I just don't get it. Any ideas?

Entity Framework. Delete all rows in table

How can I quickly remove all rows in the table using Entity Framework?
I am currently using:
var rows = from o in dataDb.Table
select o;
foreach (var row in rows)
{
dataDb.Table.Remove(row);
}
dataDb.SaveChanges();
However, it takes a long time to execute.
Are there any alternatives?
For those that are googling this and ended up here like me, this is how you currently do it in EF5 and EF6:
context.Database.ExecuteSqlCommand("TRUNCATE TABLE [TableName]");
Assuming context is a System.Data.Entity.DbContext
Edit:
Currently in net6.0 (dotnet 6 core) you can do the following:
context.Database.ExecuteSqlRaw("TRUNCATE TABLE [TableName]");
Or use the Async overload:
await context.Database.ExecuteSqlRawAsync("TRUNCATE TABLE [TableName]");
These are extension methods coming from Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions
If you're having issues with foreign keys (in MySql), you might have to do the following (Doing the SET FOREIGN_KEY_CHECKS = 0; part in a separate call does not seem to work for me)
context.Database.ExecuteSqlRaw("SET FOREIGN_KEY_CHECKS = 0; TRUNCATE TABLE [TableName];");
So if you want to truncate your entire database (Possibly for unittesting reasons) - you can do the following:
var tableNames = context.Model.GetEntityTypes()
.Select(t => t.GetTableName())
.Distinct()
.ToList();
foreach (var tableName in tableNames)
{
context.Database.ExecuteSqlRaw($"SET FOREIGN_KEY_CHECKS = 0; TRUNCATE TABLE `{tableName}`;");
}
Warning: The following is only suitable for small tables (think < 1000 rows)
Here is a solution that uses entity framework (not SQL) to delete the rows, so it is not SQL Engine(R/DBM) specific.
This assumes that you're doing this for testing or some similar situation.
Either
The amount of data is small or
The performance doesn't matter
Simply call:
VotingContext.Votes.RemoveRange(VotingContext.Votes);
Assuming this context:
public class VotingContext : DbContext
{
public DbSet<Vote> Votes{get;set;}
public DbSet<Poll> Polls{get;set;}
public DbSet<Voter> Voters{get;set;}
public DbSet<Candidacy> Candidates{get;set;}
}
For tidier code you can declare the following extension method:
public static class EntityExtensions
{
public static void Clear<T>(this DbSet<T> dbSet) where T : class
{
dbSet.RemoveRange(dbSet);
}
}
Then the above becomes:
VotingContext.Votes.Clear();
VotingContext.Voters.Clear();
VotingContext.Candidacy.Clear();
VotingContext.Polls.Clear();
await VotingTestContext.SaveChangesAsync();
I recently used this approach to clean up my test database for each testcase run (it´s obviously faster than recreating the DB from scratch each time, though I didn´t check the form of the delete commands that were generated).
Why can it be slow?
EF will get ALL the rows (VotingContext.Votes)
and then will use their IDs (not sure exactly how, doesn't matter), to delete them.
So if you're working with serious amount of data you'll kill the SQL server process (it will consume all the memory) and same thing for the IIS process since EF will cache all the data same way as SQL server. Don't use this one if your table contains serious amount of data.
Using SQL's TRUNCATE TABLE command will be the fastest as it operates on the table and not on individual rows.
dataDb.ExecuteStoreCommand("TRUNCATE TABLE [Table]");
Assuming dataDb is a DbContext (not an ObjectContext), you can wrap it and use the method like this:
var objCtx = ((System.Data.Entity.Infrastructure.IObjectContextAdapter)dataDb).ObjectContext;
objCtx.ExecuteStoreCommand("TRUNCATE TABLE [Table]");
var all = from c in dataDb.Table select c;
dataDb.Table.RemoveRange(all);
dataDb.SaveChanges();
using (var context = new DataDb())
{
var ctx = ((System.Data.Entity.Infrastructure.IObjectContextAdapter)context).ObjectContext;
ctx.ExecuteStoreCommand("DELETE FROM [TableName] WHERE Name= {0}", Name);
}
or
using (var context = new DataDb())
{
context.Database.ExecuteSqlCommand("TRUNCATE TABLE [TableName]");
}
You can do that without Foreach
dataDB.Table.RemoveRange(dataDB.Table);
dataDB.SaveChanges();
This will remove all rows
This avoids using any sql
using (var context = new MyDbContext())
{
var itemsToDelete = context.Set<MyTable>();
context.MyTables.RemoveRange(itemsToDelete);
context.SaveChanges();
}
context.TableName.RemoveRange(context.TableName);
context.SaveChanges();
I came across this question when I had to deal with a particular case: fully updating of content in a "leaf" table (no FKs pointing to it). This involved removing all rows and putting new rows information and it should be done transactionally (I do not want to end up with an empty table, if inserts fails for whatever reason).
I have tried the public static void Clear<T>(this DbSet<T> dbSet) approach, but new rows are not inserted. Another disadvante is that the whole process is slow, as rows are deleted one by one.
So, I have switched to TRUNCATE approach, since it is much faster and it is also ROLLBACKable. It also resets the identity.
Example using repository pattern:
public class Repository<T> : IRepository<T> where T : class, new()
{
private readonly IEfDbContext _context;
public void BulkInsert(IEnumerable<T> entities)
{
_context.BulkInsert(entities);
}
public void Truncate()
{
_context.Database.ExecuteSqlCommand($"TRUNCATE TABLE {typeof(T).Name}");
}
}
// usage
DataAccess.TheRepository.Truncate();
var toAddBulk = new List<EnvironmentXImportingSystem>();
// fill toAddBulk from source system
// ...
DataAccess.TheRepository.BulkInsert(toAddBulk);
DataAccess.SaveChanges();
Of course, as already mentioned, this solution cannot be used by tables referenced by foreign keys (TRUNCATE fails).
EF Core 7.0 solves this problem once and for all by adding bulk update and delete semantics:
await dataDB.Table.ExecuteDeleteAsync();
Note that this syntax immediately executes the underlying (SQL) command to delete the data from the table. It does not fiddle around with tracking the entity, marking it for deletion, and waiting for UpdateDatabase to execute the transaction against the database.
Also note that multiple ExecuteDelete and ExecuteUpdate commands will not be contained in a single transaction by default. However, the DbContext transaction APIs can be used in the normal way to wrap these commands in a transaction.
If
using(var db = new MyDbContext())
{
await db.Database.ExecuteSqlCommandAsync(#"TRUNCATE TABLE MyTable"););
}
causes
Cannot truncate table 'MyTable' because it is being referenced by a FOREIGN KEY constraint.
I use this:
using(var db = new MyDbContext())
{
await db.Database.ExecuteSqlCommandAsync(#"DELETE FROM MyTable WHERE ID != -1");
}
var data = (from n in db.users select n);
db.users.RemoveRange(data);
db.SaveChanges();
The following works on SQLite database (using Entity Framework).
It seems that the fastest way to clear all the db tables is using context.Database.ExecuteSqlCommand("some SQL"), as some comments above highlighted as well. Here I am going to show how to reset the 'index' count of tables too.
context.Database.ExecuteSqlCommand("delete from TableA");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableA'");//resets the autoindex
context.Database.ExecuteSqlCommand("delete from TableB");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableB'");//resets the autoindex
context.Database.ExecuteSqlCommand("delete from TableC");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableC'");//resets the autoindex
One important point is that if you use foreign keys in your tables, you must first delete the child table before the parent table, so the sequence (hierarchy) of tables during deletion is important, otherwise a SQLite exception may occur.
Note: var context = new YourContext()
If you wish to clear your entire database.
Because of the foreign-key constraints it matters which sequence the tables are truncated. This is a way to bruteforce this sequence.
public static void ClearDatabase<T>() where T : DbContext, new()
{
using (var context = new T())
{
var tableNames = context.Database.SqlQuery<string>("SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE' AND TABLE_NAME NOT LIKE '%Migration%'").ToList();
foreach (var tableName in tableNames)
{
foreach (var t in tableNames)
{
try
{
if (context.Database.ExecuteSqlCommand(string.Format("TRUNCATE TABLE [{0}]", tableName)) == 1)
break;
}
catch (Exception ex)
{
}
}
}
context.SaveChanges();
}
}
usage:
ClearDatabase<ApplicationDbContext>();
remember to reinstantiate your DbContext after this.
In EFCore (version i am using is 3.1) you can use the following to remove all rows -
context.Database.ExecuteSqlRaw("TRUNCATE TABLE [TableName]");
This works for me... EF v3.1.5
context.ModelName.RemoveRange(context.ModelName.ToList());
context.SaveChanges();
This works Properly in EF 5:
YourEntityModel myEntities = new YourEntityModel();
var objCtx = ((IObjectContextAdapter)myEntities).ObjectContext;
objCtx.ExecuteStoreCommand("TRUNCATE TABLE [TableName]");
Delete all records. Do not reset the primary index like "truncate".
/// <summary>
/// SET - DELETE all record by table - no truncate - return deleted records
/// </summary>
public static int setListDelAllMYTABLE()
{
// INIT
int retObj = 0;
using (MYDBEntities ctx = new MYDBEntities())
{
// GET - all record
var tempAllRecord = ctx.MYTABLE.ToList();
// RESET
ctx.MYTABLE.RemoveRange(tempAllRecord);
// SET - final save
retObj += ctx.SaveChanges();
}
// RET
return retObj;
}
If MVC, you can do:
public async Task<IActionResult> DeleteAll()
{
var list = await _context.YourClass.ToListAsync();
_context.YourClass.RemoveRange(list);
await _context.SaveChangesAsync();
return RedirectToAction(nameof(Index));
}
Make sure when you are trying to delete parent all children will cascade on delete. Or children have nullable foreign key.
Here is a variation on the popular solution by Ron that avoids the use of hardcoded string table names by taking advantage of another popular solution on stack overflow for determining the underlying table name for an entity framework class.
With these extension methods the solution looks like this:
_dbContext.TruncateTable<TheTableName>();
(use this.TruncateTable<... if you're editing code within an EF DBContext class or partial class file)
And here's the extension class:
public static class EntityFrameworkExtensions
{
private static string ParseTableNameFromSQL(string sql)
{
Regex regex = new Regex("FROM (?<table>.*) AS");
Match match = regex.Match(sql);
string table = match.Groups["table"].Value;
return table;
}
public static string GetTableName<T>(this IObjectContextAdapter context) where T : class =>
ParseTableNameFromSQL(context.ObjectContext.CreateObjectSet<T>().ToTraceString());
public static void TruncateTable<T>(this DbContext dbContext) where T : class =>
dbContext.Database.ExecuteSqlCommand($"TRUNCATE TABLE {dbContext.GetTableName<T>()}");
public static void DeleteAllTableRows<T>(this DbContext dbContext) where T : class =>
dbContext.Database.ExecuteSqlCommand($"DELETE FROM {dbContext.GetTableName<T>()}");
}
The last extension method DeleteAllTableRows is a useful alternative if your table cannot be truncated (e.g. due to foreign key references) - this is still much faster than the Entity Framework RemoveAll alternative.
Works for EF Core 3
public static class EntityExtensions
{
public static async Task ClearAsync<T>(this DbSet<T> dbSet) where T : class
{
var command = dbSet.CreateDbCommand();
command.CommandText = $"TRUNCATE TABLE {dbSet.EntityType.GetSchema()}.{dbSet.EntityType.GetTableName()}";
await command.ExecuteNonQueryAsync();
}
}
but please note that dbSet.CreateDbCommand is an extension
My solution, mixing my ideas, some answers (Ron one from this thread, but also this and this for reflection) and trying to cover some different conditions.
It is based on EF6, but it should work fine, just fixing some extensions like GetTableName<TEntity>.
My solution:
uses extensions, so you only need DbSet to launch
has a row count threshold, to decide between RemoveRange or SQL execution, to avoid perfomance issues
the SQL version is based on DELETE instead of TRUNCATE, to avoid foreign key issues (it has to fit your requirements, of course)
has a parameter to save changes inline
private const int RangeLimit = 100;
private static void ClearTable<TEntity>(this DbSet<TEntity> dataSet, bool saveChanges = true) where TEntity : class
{
DbContext context = null;
if (dataSet.Count() > RangeLimit)
{
context = dataSet.GetContext();
context.Database.ExecuteSqlCommand($"DELETE FROM [{context.GetTableName<TEntity>()}]");
}
else
{
dataSet.RemoveRange(dataSet);
}
if (!saveChanges)
{
return;
}
if (context == null)
{
context = dataSet.GetContext();
}
context.SaveChanges();
}
private static DbContext GetContext<TEntity>(this DbSet<TEntity> dbSet)
where TEntity : class
{
var internalSet = dbSet
.GetType()
.GetField("_internalSet", BindingFlags.NonPublic | BindingFlags.Instance)
?.GetValue(dbSet);
var internalContext = internalSet?.GetType().BaseType
?.GetField("_internalContext", BindingFlags.NonPublic | BindingFlags.Instance)
?.GetValue(internalSet);
return (DbContext)internalContext?.GetType()
.GetProperty("Owner", BindingFlags.Instance | BindingFlags.Public)
?.GetValue(internalContext, null);
}
public static string GetTableName<TEntity>(this DbContext context) where TEntity : class
{
return (context as IObjectContextAdapter).ObjectContext.CreateObjectSet<TEntity>().EntitySet.Name;
}
All you have to do, with a database table named Entries, is:
databaseContext.Entries.ClearTable();
if you want to save changes, or if you don't want:
databaseContext.Entries.ClearTable(false);
It is based on reflection, to simplify code. It has some performance tradeoff, of course, but reflection happens once for each table, hence should be completely acceptable in these conditions.
It is a very clean solution.
_context.RemoveRange(_context.ModelName);
_context.SaveChanges();
There are several issues with pretty much all the answers here:
1] Hard-coded sql. Will brackets work on all database engines?
2] Entity framework Remove and RemoveRange calls. This loads all entities into memory affected by the operation. Yikes.
3] Truncate table. Breaks with foreign key references and may not work accross all database engines.
Use https://entityframework-plus.net/, they handle the cross database platform stuff, translate the delete into the correct sql statement and don't load entities into memory, and the library is free and open source.
Disclaimer: I am not affiliated with the nuget package. They do offer a paid version that does even more stuff.

bulk delete in entity framework

I'd like to bulk delete records from a table using linq.
There's a post that describes how to do it:
Bulk-deleting in LINQ to Entities
var query = from c in ctx.Customers
where c.SalesPerson.Email == "..."
select c;
query.Delete();
But the function "Delete" doesn't exist in my var variable.
Furthermore, the function "SubmitChanges" doesn't exist on my context.
There is an interesting NuGet package that lets you do batch deletes and updates:
There is no currently supported bulk delete baked into Entity Framework. Its actually one of the features being discussed on codeplex now EF is open-source.
EntityFramework.Extended provides batch delete support (you can find this in nuget) however my experience is that it has some performance issues.
This code adds a simple extension method to any DbContext that will bulk delete all data in any table referenced within the entity framework query you provide. It works by simply extracting all table names involved in the query, and attempting to delete the data by issuing a "DELETE FROM tablename" SQL query, which is common across most types of database.
To use, simply do this:
myContext.BulkDelete(x => x.Things);
which will delete everything in the table linked to the Things entity store.
The code:
using System.Linq;
using System.Text.RegularExpressions;
namespace System.Data.Entity {
public static class DbContextExtensions {
/// <summary>
/// Efficiently deletes all data from any database tables used by the specified entity framework query.
/// </summary>
/// <typeparam name="TContext">The DbContext Type on which to perform the delete.</typeparam>
/// <typeparam name="TEntity">The Entity Type to which the query resolves.</typeparam>
/// <param name="ctx">The DbContext on which to perform the delete.</param>
/// <param name="deleteQuery">The query that references the tables you want to delete.</param>
public static void BulkDelete<TContext, TEntity>(this TContext ctx, Func<TContext, IQueryable<TEntity>> deleteQuery) where TContext : DbContext {
var findTables = new Regex(#"(?:FROM|JOIN)\s+(\[\w+\]\.\[\w+\])\s+AS");
var qry = deleteQuery(ctx).ToString();
// Get list of all tables mentioned in the query
var tables = findTables.Matches(qry).Cast<Match>().Select(m => m.Result("$1")).Distinct().ToList();
// Loop through all the tables, attempting to delete each one in turn
var max = 30;
var exception = (Exception)null;
while (tables.Any() && max-- > 0) {
// Get the next table
var table = tables.First();
try {
// Attempt the delete
ctx.Database.ExecuteSqlCommand(string.Format("DELETE FROM {0}", table));
// Success, so remove table from the list
tables.Remove(table);
} catch (Exception ex) {
// Error, probably due to dependent constraint, save exception for later if needed.
exception = ex;
// Push the table to the back of the queue
tables.Remove(table);
tables.Add(table);
}
}
// Error error has occurred, and cannot be resolved by deleting in a different
// order, then rethrow the exception and give up.
if (max <= 0 && exception != null) throw exception;
}
}
}
I do it like this which seems to work fine. Please let know if there is a reason why this is bad practice in any way.
var customersToDelete = await ctx.Customers.Where(c => c.Email == "...").ToListAsync();
foreach (var customerToDelete in customersToDelete)
{
ctx.Customers.Remove(customerToDelete);
}
await ctx.SaveChangesAsync();
I was experiencing the same problem with EF executing thousands of DELETE queries after SaveChanges call. I wasn't sure that EntityFramework.Extensions commercial library would help me so I decided to implement bulk DELETE myself and came up with something similar to BG100's solution!
public async Task<List<TK>> BulkDeleteAsync(List<TK> ids)
{
if (ids.Count < 1) {
return new List<TK>();
}
// NOTE: DbContext here is a property of Repository Class
// SOURCE: https://stackoverflow.com/a/9775744
var tableName = DbContext.GetTableName<T>();
var sql = $#"
DELETE FROM {tableName}
OUTPUT Deleted.Id
// NOTE: Be aware that 'Id' column naming depends on your project conventions
WHERE Id IN({string.Join(", ", ids)});
";
return await #this.Database.SqlQuery<TK>(sql).ToListAsync();
}
If you have something like generic repository that should work for you just fine. At least you could try to fit it into your EF infrastructure.
I also tweaked it a bit more and was able to execute queries on multiple chunks of entities. It would help you if there are any restrictions of query size in your DB.
const int ChunkSize = 1024;
public async Task<List<TK>> BulkDeleteAsync(List<TK> ids)
{
if (ids.Count < 1) {
return new List<TK>();
}
// SOURCE: https://stackoverflow.com/a/20953521/11344051
var chunks = ids.Chunks(ChunkSize).Select(c => c.ToList()).ToList();
var tableName = DbContext.GetTableName<T>();
var deletedIds = new List<TK>();
foreach (var chunk in chunks) {
var sql = $#"
DELETE FROM {tableName}
OUTPUT Deleted.Id
WHERE Id IN({string.Join(", ", chunk)});
";
deletedIds.AddRange(DbContext.Database.SqlQuery<TK>(sql).ToListAsync());
}
return deletedIds;
}

EF4: Get EntitySet specified as string

I have a database with many single isolated tables, and I need to fill their contents to their Entities.
Currently I have for each and every table something like this:
try
{
using(DBContext context = new DBContext())
{
var vehicleTypes = context.VehicleTypes;
return vehicleTypes;
}
}
catch(Exception ex)
{
//handle error
}
What I'm seeking for is best described as something like this:
var vehicleTypes = context.GetEntitySet(VehicleEntity);
var buildingTypes = context.GetEntitySet(BuildingEntity);
where VehicleEntity, BuildingEntity (...) are entities from entity model.
I know I don't have this option explicitly, but something in that similar way would be nice. Extension method is also an option...
I'm working with EntityFramework 4.0, POCO Self Tracking Entities (without proxy).
Thanks
edit: My latest try was this:
public static IEnumerable<TEntity> GetTableContent<TEntity>() where TEntity:class
{
try
{
using (var context = new DBEntities())
{
var result = context.ExecuteStoreQuery<TEntity>("SELECT * FROM " + typeof(TEntity).Name); //the table names correspond to Entity type class names
return result;
}
}
catch (Exception ex)
{
//handle error...
}
}
but I get the error:
The data reader is incompatible with the specified 'DBEntities.MyEntity'. A member of the type, 'Id', does not have a corresponding column in the data reader with the same name.
and that's true - the names don't match - but I figured it would've mapped the table from query based on edmx definition? How can I get this right?
Start with code which actually works before trying to make it "generic"
result = context.ExecuteStoreQuery<Something>("SELECT SOMETHING_ID AS ID, ..., FROM SOMETHING...");

Categories

Resources