Related
I only need it to work for SQL Server. This is an example. The question is about a general approach.
There is a nice extension method from https://entityframework-extensions.net called WhereBulkContains. It is, sort of, great, except that the code of the methods in this library is obfuscated and they do not produce valid SQL when .ToQueryString() is called on IQueryable<T> with these extension methods applied.
Subsequently, I can't use such methods in production code as I am not "allowed" to trust such code due to business reasons. Sure, I can write tons of tests to ensure that WhereBulkContains works as expected, except that there are some complicated cases where the performance of WhereBulkContains is well below stellar, whereas properly written SQL works in a blink of an eye. And (read above), since the code of this library is obfuscated, there is no way to figure out what's wrong there without spending a large amount of time. We would've bought the license (as this is not a freeware) if the library weren't obfuscated. All together that basically kills the library for our purposes.
This is where it gets interesting. I can easily create and populate a temporary table, e.g. (I have a table called EFAgents with an int PK called AgentId in the database):
private string GetTmpAgentSql(IEnumerable<int> agentIds) => #$"
drop table if exists #tmp_Agents;
create table #tmp_Agents (AgentId int not null, primary key clustered (AgentId asc));
{(agentIds
.Chunk(1_000)
.Select(e => $#"
insert into #tmp_Agents (AgentId)
values
({e.JoinStrings("), (")});
")
.JoinStrings(""))}
select 0 as Result
";
private const string AgentSql = #"
select a.* from EFAgents a inner join #tmp_Agents t on a.AgentID = t.AgentId";
where GetContext returns EF Core database context and JoinStrings comes from Unity.Interception.Utilities and then use it as follows:
private async Task<List<EFAgent>> GetAgents(List<int> agentIds)
{
var tmpSql = GetTmpAgentSql(agentIds);
using var ctx = GetContext();
// This creates a temporary table and populates it with the ids.
// This is a proprietary port of EF SqlQuery code, but I can post the whole thing if necessary.
var _ = await ctx.GetDatabase().SqlQuery<int>(tmpSql).FirstOrDefaultAsync();
// There is a DbSet<EFAgent> called Agents.
var query = ctx.Agents
.FromSqlRaw(AgentSql)
.Join(ctx.Agents, t => t.AgentId, a => a.AgentId, (t, a) => a);
var sql = query.ToQueryString() + Environment.NewLine;
// This should provide a valid SQL; https://entityframework-extensions.net does NOT!
// WriteLine - writes to console or as requested. This is irrelevant to the question.
WriteLine(sql);
var result = await query.ToListAsync();
return result;
}
So, basically, I can do what I need in two steps:
using var ctx = GetContext();
// 1. Create a temp table and populate it - call GetTmpAgentSql.
...
// 2. Build the join starting from `FromSqlRaw` as in example above.
This is doable, half-manual, and it is going to work.
The question is how to do that in one step, e.g., call:
.WhereMyBulkContains(aListOfIdConstraints, whateverElseIsneeded, ...)
and that's all.
I am fine if I need to pass more than one parameter in each case in order to specify the constraints.
To clarify the reasons why do I need to go into all these troubles. We have to interact with a third party database. We don't have any control of the schema and data there. The database is large and poorly designed. That resulted in some ugly EFC LINQ queries. To remedy that, some of that ugliness was encapsulated into a method, which takes IQueryable<T> (and some more parameters) and returns IQueryable<T>. Under the hood this method calls WhereBulkContains. I need to replace this WhereBulkContains by, call it, WhereMyBulkContains, which would be able to provide correct ToQueryString representation (for debugging purposes) and be performant. The latter means that SQL should not contain in clause with hundreds (and even sometimes thousands) of elements. Using inner join with a [temp] table with a PK and having an index on the FK field seem to do the trick if I do that in pure SQL. But, ... I need to do that in C# and effectively in between two LINQ method calls. Refactoring everything is also not an option because that method is used in many places.
Thanks a lot!
I think you really want to use a Table Valued Parameter.
Creating an SqlParameter from an enumeration is a little fiddly, but not too difficult to get right;
CREATE TYPE [IntValue] AS TABLE (
Id int NULL
)
private IEnumerable<SqlDataRecord> FromValues(IEnumerable<int> values)
{
var meta = new SqlMetaData(
"Id",
SqlDbType.Int
);
foreach(var value in values)
{
var record = new SqlDataRecord(
meta
);
record.SetInt32(0, value);
yield return record;
}
}
public SqlParameter ToIntTVP(IEnumerable<int> values){
return new SqlParameter()
{
TypeName = "IntValue",
SqlDbType = SqlDbType.Structured,
Value = FromValues(values)
};
}
Personally I would define a query type in EF Core to represent the TVP. Then you can use raw sql to return an IQueryable.
public class IntValue
{
public int Id { get; set; }
}
modelBuilder.Entity<IntValue>(e =>
{
e.HasNoKey();
e.ToView("IntValue");
});
IQueryable<IntValue> ToIntQueryable(DbContext ctx, IEnumerable<int> values)
{
return ctx.Set<IntValue>()
.FromSqlInterpolated($"select * from {ToIntTVP(values)}");
}
Now you can compose the rest of your query using Linq.
var ids = ToIntQueryable(ctx, agentIds);
var query = ctx.Agents
.Where(a => ids.Any(i => i.Id == a.Id));
I would propose to use linq2db.EntityFrameworkCore (note that I'm one of the creators). It has built-in temporary tables support.
We can create simple and reusable function which filters records of any type:
public static class HelperMethods
{
private class KeyHolder<T>
{
[PrimaryKey]
public T Key { get; set; } = default!;
}
public static async Task<List<TEntity>> GetRecordsByIds<TEntity, TKey>(this IQueryable<TEntity> query, IEnumerable<TKey> ids, Expression<Func<TEntity, TKey>> keyFunc)
{
var ctx = LinqToDBForEFTools.GetCurrentContext(query) ??
throw new InvalidOperationException("Query should be EF Core query");
// based on DbContext options, extension retrieves connection information
using var db = ctx.CreateLinqToDbConnection();
// create temporary table and BulkCopy records into that table
using var tempTable = await db.CreateTempTableAsync(ids.Select(id => new KeyHolder<TKey> { Key = id }), tableName: "temporaryIds");
var resultQuery = query.Join(tempTable, keyFunc, t => t.Key, (q, t) => q);
// we use ToListAsyncLinqToDB to avoid collission with EF Core async methods.
return await resultQuery.ToListAsyncLinqToDB();
}
}
Then we can rewrite your function GetAgents to the following:
private async Task<List<EFAgent>> GetAgents(List<int> agentIds)
{
using var ctx = GetContext();
var result = await ctx.Agents.GetRecordsByIds(agentIds, a => a.AgentId);
return result;
}
I am trying to query a database using Entity Framework and I need to make several slightly different queries on the same set of tables. There are a load of navigation properties I need to add and it seems logical to me that I should be able to define the "base query" (i.e the one with all the navigation properties" and then further filter this as required and execute the query.
Some code may help explain further. This is what I am calling my "base query"
private static IEnumerable<HelpdeskTicket> GetAll()
{
IEnumerable<HelpdeskTicket> Tickets;
using (ItManagement_Entities db = new ItManagement_Entities())
{
Tickets = db.HelpdeskTickets.Include("CreatedByPerson")
.Include("HelpdeskCategory")
.Include("HelpdeskPriority")
.Include("HelpdeskStatus");
}
return Tickets;
}
As an example, some of the queries I need to perform are open tickets, recently closed tickets, my tickets, yada yada yada.
My thinking is to have methods similar to the following to do the filtering bit I need without having to define all the .Include()'s again.
public static List<HelpdeskTicketModel> GetAllTickets()
{
List<HelpdeskTicketModel> Tickets = new List<HelpdeskTicketModel>();
GetAll().OrderByDescending(t => t.TicketId)
.ToList()
.ForEach(t => Tickets.Add(HelpdeskTicketModel.Map(t)));
return Tickets;
}
public static List<HelpdeskTicketModel> GetRecentlyClosedTickets()
{
List<HelpdeskTicketModel> Tickets = new List<HelpdeskTicketModel>();
GetAll().Where(t => t.HelpdeskStatus.IsClosedStatus)
.OrderByDescending(t => t.ClosedTime)
.ToList()
.ForEach(t => Tickets.Add(HelpdeskTicketModel.Map(t)));
return Tickets;
}
//And so on...
When I try this I get a System.InvalidOperationException exception thrown complaining that The operation cannot be completed because the DbContext has been disposed, which makes sense really because my query was in a different context, in the GetAll method.
Question is, how do I go about doing what I want?
You may try something similar to the Template Method pattern, where each particular method calls some base, private methods that do the common work and then each one adds its particular bits of the query. Something like that may come handy as a starting point:
// Here you define common parts applicable to all methods, or at least shared among some of them
private static IQueryable<HelpdeskTicket> BuildBaseQuery(this IQueryable<HelpdeskTicket> query)
{
return query.Include("CreatedByPerson")
.Include("HelpdeskCategory")
.Include("HelpdeskPriority")
.Include("HelpdeskStatus");
}
// Here are the particular methods, they create a query, call helper methods for the common bits and add their specifics
public static List<HelpdeskTicketModel> GetAllTickets()
{
List<HelpdeskTicketModel> Tickets = new List<HelpdeskTicketModel>();
using (ItManagement_Entities db = new ItManagement_Entities())
{
Tickets = db.HelpdeskTickets.BuildBaseQuery()
.OrderByDescending(t => t.TicketId)
.ToList()
.ForEach(t => Tickets.Add(HelpdeskTicketModel.Map(t)));
}
return Tickets;
}
How can I quickly remove all rows in the table using Entity Framework?
I am currently using:
var rows = from o in dataDb.Table
select o;
foreach (var row in rows)
{
dataDb.Table.Remove(row);
}
dataDb.SaveChanges();
However, it takes a long time to execute.
Are there any alternatives?
For those that are googling this and ended up here like me, this is how you currently do it in EF5 and EF6:
context.Database.ExecuteSqlCommand("TRUNCATE TABLE [TableName]");
Assuming context is a System.Data.Entity.DbContext
Edit:
Currently in net6.0 (dotnet 6 core) you can do the following:
context.Database.ExecuteSqlRaw("TRUNCATE TABLE [TableName]");
Or use the Async overload:
await context.Database.ExecuteSqlRawAsync("TRUNCATE TABLE [TableName]");
These are extension methods coming from Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions
If you're having issues with foreign keys (in MySql), you might have to do the following (Doing the SET FOREIGN_KEY_CHECKS = 0; part in a separate call does not seem to work for me)
context.Database.ExecuteSqlRaw("SET FOREIGN_KEY_CHECKS = 0; TRUNCATE TABLE [TableName];");
So if you want to truncate your entire database (Possibly for unittesting reasons) - you can do the following:
var tableNames = context.Model.GetEntityTypes()
.Select(t => t.GetTableName())
.Distinct()
.ToList();
foreach (var tableName in tableNames)
{
context.Database.ExecuteSqlRaw($"SET FOREIGN_KEY_CHECKS = 0; TRUNCATE TABLE `{tableName}`;");
}
Warning: The following is only suitable for small tables (think < 1000 rows)
Here is a solution that uses entity framework (not SQL) to delete the rows, so it is not SQL Engine(R/DBM) specific.
This assumes that you're doing this for testing or some similar situation.
Either
The amount of data is small or
The performance doesn't matter
Simply call:
VotingContext.Votes.RemoveRange(VotingContext.Votes);
Assuming this context:
public class VotingContext : DbContext
{
public DbSet<Vote> Votes{get;set;}
public DbSet<Poll> Polls{get;set;}
public DbSet<Voter> Voters{get;set;}
public DbSet<Candidacy> Candidates{get;set;}
}
For tidier code you can declare the following extension method:
public static class EntityExtensions
{
public static void Clear<T>(this DbSet<T> dbSet) where T : class
{
dbSet.RemoveRange(dbSet);
}
}
Then the above becomes:
VotingContext.Votes.Clear();
VotingContext.Voters.Clear();
VotingContext.Candidacy.Clear();
VotingContext.Polls.Clear();
await VotingTestContext.SaveChangesAsync();
I recently used this approach to clean up my test database for each testcase run (it´s obviously faster than recreating the DB from scratch each time, though I didn´t check the form of the delete commands that were generated).
Why can it be slow?
EF will get ALL the rows (VotingContext.Votes)
and then will use their IDs (not sure exactly how, doesn't matter), to delete them.
So if you're working with serious amount of data you'll kill the SQL server process (it will consume all the memory) and same thing for the IIS process since EF will cache all the data same way as SQL server. Don't use this one if your table contains serious amount of data.
Using SQL's TRUNCATE TABLE command will be the fastest as it operates on the table and not on individual rows.
dataDb.ExecuteStoreCommand("TRUNCATE TABLE [Table]");
Assuming dataDb is a DbContext (not an ObjectContext), you can wrap it and use the method like this:
var objCtx = ((System.Data.Entity.Infrastructure.IObjectContextAdapter)dataDb).ObjectContext;
objCtx.ExecuteStoreCommand("TRUNCATE TABLE [Table]");
var all = from c in dataDb.Table select c;
dataDb.Table.RemoveRange(all);
dataDb.SaveChanges();
using (var context = new DataDb())
{
var ctx = ((System.Data.Entity.Infrastructure.IObjectContextAdapter)context).ObjectContext;
ctx.ExecuteStoreCommand("DELETE FROM [TableName] WHERE Name= {0}", Name);
}
or
using (var context = new DataDb())
{
context.Database.ExecuteSqlCommand("TRUNCATE TABLE [TableName]");
}
You can do that without Foreach
dataDB.Table.RemoveRange(dataDB.Table);
dataDB.SaveChanges();
This will remove all rows
This avoids using any sql
using (var context = new MyDbContext())
{
var itemsToDelete = context.Set<MyTable>();
context.MyTables.RemoveRange(itemsToDelete);
context.SaveChanges();
}
context.TableName.RemoveRange(context.TableName);
context.SaveChanges();
I came across this question when I had to deal with a particular case: fully updating of content in a "leaf" table (no FKs pointing to it). This involved removing all rows and putting new rows information and it should be done transactionally (I do not want to end up with an empty table, if inserts fails for whatever reason).
I have tried the public static void Clear<T>(this DbSet<T> dbSet) approach, but new rows are not inserted. Another disadvante is that the whole process is slow, as rows are deleted one by one.
So, I have switched to TRUNCATE approach, since it is much faster and it is also ROLLBACKable. It also resets the identity.
Example using repository pattern:
public class Repository<T> : IRepository<T> where T : class, new()
{
private readonly IEfDbContext _context;
public void BulkInsert(IEnumerable<T> entities)
{
_context.BulkInsert(entities);
}
public void Truncate()
{
_context.Database.ExecuteSqlCommand($"TRUNCATE TABLE {typeof(T).Name}");
}
}
// usage
DataAccess.TheRepository.Truncate();
var toAddBulk = new List<EnvironmentXImportingSystem>();
// fill toAddBulk from source system
// ...
DataAccess.TheRepository.BulkInsert(toAddBulk);
DataAccess.SaveChanges();
Of course, as already mentioned, this solution cannot be used by tables referenced by foreign keys (TRUNCATE fails).
EF Core 7.0 solves this problem once and for all by adding bulk update and delete semantics:
await dataDB.Table.ExecuteDeleteAsync();
Note that this syntax immediately executes the underlying (SQL) command to delete the data from the table. It does not fiddle around with tracking the entity, marking it for deletion, and waiting for UpdateDatabase to execute the transaction against the database.
Also note that multiple ExecuteDelete and ExecuteUpdate commands will not be contained in a single transaction by default. However, the DbContext transaction APIs can be used in the normal way to wrap these commands in a transaction.
If
using(var db = new MyDbContext())
{
await db.Database.ExecuteSqlCommandAsync(#"TRUNCATE TABLE MyTable"););
}
causes
Cannot truncate table 'MyTable' because it is being referenced by a FOREIGN KEY constraint.
I use this:
using(var db = new MyDbContext())
{
await db.Database.ExecuteSqlCommandAsync(#"DELETE FROM MyTable WHERE ID != -1");
}
var data = (from n in db.users select n);
db.users.RemoveRange(data);
db.SaveChanges();
The following works on SQLite database (using Entity Framework).
It seems that the fastest way to clear all the db tables is using context.Database.ExecuteSqlCommand("some SQL"), as some comments above highlighted as well. Here I am going to show how to reset the 'index' count of tables too.
context.Database.ExecuteSqlCommand("delete from TableA");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableA'");//resets the autoindex
context.Database.ExecuteSqlCommand("delete from TableB");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableB'");//resets the autoindex
context.Database.ExecuteSqlCommand("delete from TableC");
context.Database.ExecuteSqlCommand("delete from sqlite_sequence where name='TableC'");//resets the autoindex
One important point is that if you use foreign keys in your tables, you must first delete the child table before the parent table, so the sequence (hierarchy) of tables during deletion is important, otherwise a SQLite exception may occur.
Note: var context = new YourContext()
If you wish to clear your entire database.
Because of the foreign-key constraints it matters which sequence the tables are truncated. This is a way to bruteforce this sequence.
public static void ClearDatabase<T>() where T : DbContext, new()
{
using (var context = new T())
{
var tableNames = context.Database.SqlQuery<string>("SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE' AND TABLE_NAME NOT LIKE '%Migration%'").ToList();
foreach (var tableName in tableNames)
{
foreach (var t in tableNames)
{
try
{
if (context.Database.ExecuteSqlCommand(string.Format("TRUNCATE TABLE [{0}]", tableName)) == 1)
break;
}
catch (Exception ex)
{
}
}
}
context.SaveChanges();
}
}
usage:
ClearDatabase<ApplicationDbContext>();
remember to reinstantiate your DbContext after this.
In EFCore (version i am using is 3.1) you can use the following to remove all rows -
context.Database.ExecuteSqlRaw("TRUNCATE TABLE [TableName]");
This works for me... EF v3.1.5
context.ModelName.RemoveRange(context.ModelName.ToList());
context.SaveChanges();
This works Properly in EF 5:
YourEntityModel myEntities = new YourEntityModel();
var objCtx = ((IObjectContextAdapter)myEntities).ObjectContext;
objCtx.ExecuteStoreCommand("TRUNCATE TABLE [TableName]");
Delete all records. Do not reset the primary index like "truncate".
/// <summary>
/// SET - DELETE all record by table - no truncate - return deleted records
/// </summary>
public static int setListDelAllMYTABLE()
{
// INIT
int retObj = 0;
using (MYDBEntities ctx = new MYDBEntities())
{
// GET - all record
var tempAllRecord = ctx.MYTABLE.ToList();
// RESET
ctx.MYTABLE.RemoveRange(tempAllRecord);
// SET - final save
retObj += ctx.SaveChanges();
}
// RET
return retObj;
}
If MVC, you can do:
public async Task<IActionResult> DeleteAll()
{
var list = await _context.YourClass.ToListAsync();
_context.YourClass.RemoveRange(list);
await _context.SaveChangesAsync();
return RedirectToAction(nameof(Index));
}
Make sure when you are trying to delete parent all children will cascade on delete. Or children have nullable foreign key.
Here is a variation on the popular solution by Ron that avoids the use of hardcoded string table names by taking advantage of another popular solution on stack overflow for determining the underlying table name for an entity framework class.
With these extension methods the solution looks like this:
_dbContext.TruncateTable<TheTableName>();
(use this.TruncateTable<... if you're editing code within an EF DBContext class or partial class file)
And here's the extension class:
public static class EntityFrameworkExtensions
{
private static string ParseTableNameFromSQL(string sql)
{
Regex regex = new Regex("FROM (?<table>.*) AS");
Match match = regex.Match(sql);
string table = match.Groups["table"].Value;
return table;
}
public static string GetTableName<T>(this IObjectContextAdapter context) where T : class =>
ParseTableNameFromSQL(context.ObjectContext.CreateObjectSet<T>().ToTraceString());
public static void TruncateTable<T>(this DbContext dbContext) where T : class =>
dbContext.Database.ExecuteSqlCommand($"TRUNCATE TABLE {dbContext.GetTableName<T>()}");
public static void DeleteAllTableRows<T>(this DbContext dbContext) where T : class =>
dbContext.Database.ExecuteSqlCommand($"DELETE FROM {dbContext.GetTableName<T>()}");
}
The last extension method DeleteAllTableRows is a useful alternative if your table cannot be truncated (e.g. due to foreign key references) - this is still much faster than the Entity Framework RemoveAll alternative.
Works for EF Core 3
public static class EntityExtensions
{
public static async Task ClearAsync<T>(this DbSet<T> dbSet) where T : class
{
var command = dbSet.CreateDbCommand();
command.CommandText = $"TRUNCATE TABLE {dbSet.EntityType.GetSchema()}.{dbSet.EntityType.GetTableName()}";
await command.ExecuteNonQueryAsync();
}
}
but please note that dbSet.CreateDbCommand is an extension
My solution, mixing my ideas, some answers (Ron one from this thread, but also this and this for reflection) and trying to cover some different conditions.
It is based on EF6, but it should work fine, just fixing some extensions like GetTableName<TEntity>.
My solution:
uses extensions, so you only need DbSet to launch
has a row count threshold, to decide between RemoveRange or SQL execution, to avoid perfomance issues
the SQL version is based on DELETE instead of TRUNCATE, to avoid foreign key issues (it has to fit your requirements, of course)
has a parameter to save changes inline
private const int RangeLimit = 100;
private static void ClearTable<TEntity>(this DbSet<TEntity> dataSet, bool saveChanges = true) where TEntity : class
{
DbContext context = null;
if (dataSet.Count() > RangeLimit)
{
context = dataSet.GetContext();
context.Database.ExecuteSqlCommand($"DELETE FROM [{context.GetTableName<TEntity>()}]");
}
else
{
dataSet.RemoveRange(dataSet);
}
if (!saveChanges)
{
return;
}
if (context == null)
{
context = dataSet.GetContext();
}
context.SaveChanges();
}
private static DbContext GetContext<TEntity>(this DbSet<TEntity> dbSet)
where TEntity : class
{
var internalSet = dbSet
.GetType()
.GetField("_internalSet", BindingFlags.NonPublic | BindingFlags.Instance)
?.GetValue(dbSet);
var internalContext = internalSet?.GetType().BaseType
?.GetField("_internalContext", BindingFlags.NonPublic | BindingFlags.Instance)
?.GetValue(internalSet);
return (DbContext)internalContext?.GetType()
.GetProperty("Owner", BindingFlags.Instance | BindingFlags.Public)
?.GetValue(internalContext, null);
}
public static string GetTableName<TEntity>(this DbContext context) where TEntity : class
{
return (context as IObjectContextAdapter).ObjectContext.CreateObjectSet<TEntity>().EntitySet.Name;
}
All you have to do, with a database table named Entries, is:
databaseContext.Entries.ClearTable();
if you want to save changes, or if you don't want:
databaseContext.Entries.ClearTable(false);
It is based on reflection, to simplify code. It has some performance tradeoff, of course, but reflection happens once for each table, hence should be completely acceptable in these conditions.
It is a very clean solution.
_context.RemoveRange(_context.ModelName);
_context.SaveChanges();
There are several issues with pretty much all the answers here:
1] Hard-coded sql. Will brackets work on all database engines?
2] Entity framework Remove and RemoveRange calls. This loads all entities into memory affected by the operation. Yikes.
3] Truncate table. Breaks with foreign key references and may not work accross all database engines.
Use https://entityframework-plus.net/, they handle the cross database platform stuff, translate the delete into the correct sql statement and don't load entities into memory, and the library is free and open source.
Disclaimer: I am not affiliated with the nuget package. They do offer a paid version that does even more stuff.
I have two tables Studies and Series. Series are FK'd back to Studies so one Study contains a variable number of Series.
Each Series item has a Deleted column indicating it has been logically deleted from the database.
I am trying to implement a Deleted property in the Study class that returns true only if all the contained Series are deleted.
I am using O/R Designer generated classes, so I added the following to the user modifiable partial class for the Study type:
public bool Deleted
{
get
{
var nonDeletedSeries = from s in Series
where !s.Deleted
select s;
return nonDeletedSeries.Count() == 0;
}
set
{
foreach (var series in Series)
{
series.Deleted = value;
}
}
}
This gives an exception "The member 'PiccoloDatabase.Study.Deleted' has no supported translation to SQL." when this simple query is executed that invokes get:
IQueryable<Study> dataQuery = dbCtxt.Studies;
dataQuery = dataQuery.Where((s) => !s.Deleted);
foreach (var study in dataQuery)
{
...
}
Based on this http://www.foliotek.com/devblog/using-custom-properties-inside-linq-to-sql-queries/, I tried the following approach:
static Expression<Func<Study, bool>> DeletedExpr = t => false;
public bool Deleted
{
get
{
var nameFunc = DeletedExpr.Compile();
return nameFunc(this);
}
set
{ ... same as before
}
}
I get the same exception when a query is run that there is no supported translation to SQL. (
The logic of the lambda expression is irrelevant yet - just trying to get past the exception.)
Am I missing some fundamental property or something to allow translation to SQL? I've read most of the posts on SO about this exception, but nothing seems to fit my case exactly.
I believe the point of LINQ-to-SQL is that your entities are mapped for you and must have correlations in the database. It appears that you are trying to mix the LINQ-to-Objects and LINQ-to-SQL.
If the Series table has a Deleted field in the database, and the Study table does not but you would like to translate logical Study.Deleted into SQL, then extension would be a way to go.
public static class StudyExtensions
{
public static IQueryable<study> AllDeleted(this IQueryable<study> studies)
{
return studies.Where(study => !study.series.Any(series => !series.deleted));
}
}
class Program
{
public static void Main()
{
DBDataContext db = new DBDataContext();
db.Log = Console.Out;
var deletedStudies =
from study in db.studies.AllDeleted()
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
}
}
This maps your "deleted study" expression into SQL:
SELECT t0.study_id, t0.name
FROM study AS t0
WHERE NOT EXISTS(
SELECT NULL AS EMPTY
FROM series AS t1
WHERE (NOT (t1.deleted = 1)) AND (t1.fk_study_id = t0.study_id)
)
Alternatively you could build actual expressions and inject them into your query, but that is an overkill.
If however, neither Series nor Study has the Deleted field in the database, but only in memory, then you need to first convert your query to IEnumerable and only then access the Deleted property. However doing so would transfer records into memory before applying the predicate and could potentially be expensive. I.e.
var deletedStudies =
from study in db.studies.ToList()
where study.Deleted
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
When you make your query, you will want to use the statically defined Expression, not the property.
Effectively, instead of:
dataQuery = dataQuery.Where((s) => !s.Deleted);
Whenever you are making a Linq to SQL query, you will instead want to use:
dataQuery = dataQuery.Where(DeletedExpr);
Note that this will require that you can see DeletedExpr from dataQuery, so you will either need to move it out of your class, or expose it (i.e. make it public, in which case you would access it via the class definition: Series.DeletedExpr).
Also, an Expression is limited in that it cannot have a function body. So, DeletedExpr might look something like:
public static Expression<Func<Study, bool>> DeletedExpr = s => s.Series.Any(se => se.Deleted);
The property is added simply for convenience, so that you can also use it as a part of your code objects without needing to duplicate the code, i.e.
var s = new Study();
if (s.Deleted)
...
I have two similar methods that basically does the same thing only with different objects.
What's the best way to make a generic method out of this if possible?
The two objects:
public class StoreObject {
int Key;
string Address;
string Country;
int Latitude;
int Longitude;
}
public class ProjectObject {
int ProjectKey;
string Address;
string Description;
}
The two methods that I potentially want to make into a generic:
public StoreObject GetStoreByKey(int key)
{
using (DBEntities dbe = new DBEntities())
{
StoreObject so = new StoreObject();
var storeObject = (from s in dbe.StoreTables
where s.Key == key
select s).First();
so.Key = storeObject.key;
so.Address = storeObject.address;
so.Country = storeObject.country;
so.Latitude = storeObject.latitude;
so.Longitude = storeObject.longitude;
return so;
}
}
public ProjectObject GetProjectByKey(int projectKey)
{
using (DBEntities dbe = new DBEntities())
{
ProjectObject po = new ProjectObject();
var projectObject = (from p in dbe.ProjectTables
where p.ProjectKey == projectKey
select p).First();
po.Key = projectObject.p_key;
po.Address = projectObject.p_address;
po.Description = projectObject.p_description;
return po;
}
}
I must note that:
- I have no control over how the table fields are named (ie. p_description).
- StoreTable in the DB, for example, may have other properties (like telephone, postal code, etc) but I'm only interested in showing what I've shown in the code.
- The same goes for the ProjectTable.
Well, the tricky part is that your entities have different properties, so using generics to populate the different properties within one method will not be worth it. But you can return the whole object and then just use the properties you are interested in.
public T GetEntityByKey<T>(int key)
{
using (DBEntities dbe = new DBEntities())
{
return = dbe.StoreTables.Set<T>.Find(new object[] {key});
}
}
And to use it
StoreObject so = GetEntityByKey<StoreObject>(123);
if(so != null)
{
int lat = so.Latitude;
}
You can indeed abstract out the type returned, and factor the using, but for the rest you'd need either a switch on the type requested or, reflection to pass in the fields to retrieve as parameters and the DB query to use.
The former would be bad practice and brings little to the equation, and the latter is costly and can get messy.
This is not really a good candidate for generics, unless you have many of such look-alike methods, in which case I'd go for the reflection approach.
HTH,
Bab.
It's very unlikely that this is your entire 'unit of work' and thus the use of a fresh DBEntities() context in each of these methods is probably the root of your problem here.
Creating a Repository class that includes an instance of the DBEntities class for a single web request (or whatever other unit of request you have in your application) and which has these methods in it would be a better approach to eliminating the duplicate code here. The scope of the using() is then outside these methods and hopefully tied to your web request or other unit of time.
As an option instead of creating a new class you could also extend the DBEntities partial class to include methods like these (assuming this is generated code).
You essentially have two different functionalities in each method:
Query an entity
Map that entity to another type
The first part has been addressed by Steve Mallory.
For the second part, you can use a mapper framework to handle copying values from one instance to another. Since the names of each type do not match, you'll need to tell it how to map names (in your example, adding "p_" and making it lowercase). One possibility would be Emit Mapper.
If you were to factor out all commonality, it would be something like:
public TResult GetById<TResult, TEntity>(int id)
{
using (DBEntities dbe = new DBEntities())
{
T result = dbe.StoreTables.Set<T>.Find(new object[] {key});
var mapper = ObjectMapperManager.DefaultInstance
.GetMapper<TEntity, TResult>(
new DefaultMapConfig().MatchMembers((m1, m2) => "p_" + m1.ToLower() == m2));
return mapper.Map(result);
}
}