I am having a strange performance issue with ef core 6 and MySlq and hoping you can help me spot the problem.
here's my setup.
Ef core 6/ Mysql
Table per Hierarchy approach. Here's the hierarchy:
public class RealEstate : Property{
}
Repository pattern with UnitOfWork. Here it is:
public interface IUnitOfWork
{
IDataAccessLayer<Property> PropertyRepository { get; }
IDataAccessLayer<RealEstate> RealEstateRepository { get; }
}
Here's my database context:
public class MeerkatContext : IdentityDbContext<AppUser>
{
public MeerkatContext(DbContextOptions<MeerkatContext> options) : base(options)
{
}
public DbSet<Property> Property { get; set; }
public DbSet<RealEstate> RealEstate { get; set; }
}
I have the following index defined on the "property" table
I have 1 million records in the table.
Here's the issue:
The following query take less than 1 seconds:
var count1 = await this._unitOfWork.PropertyRepository.CountAsync(x =>
x.CountryId == 1 && !x.IsBlocked && x.IsPublic);
this one takes 10 seconds:
var count2 = await this._unitOfWork.RealEstateRepository.CountAsync(x =>
x.CountryId == 1 && !x.IsBlocked && x.IsPublic);
I am stomped. Any help would be really appreciated.
edited to show query excution in MySql WorkBench
Thanks
Sind you probably want to search for un-blocked items, change the logic to help with the SQL:
AND NOT IsBlocked -- inefficient
AND IsBlocked = 0 -- efficient (using "=" instead of "NOT")
AND NotBlocked -- efficient (flip the name and logic)
In general: avoid OR and NOT when constructing SQL. (This is an oversimplification.)
Then add this 4-column index to the table:
INDEX(CountryId, IsBlocked [or NotBlocked], IsPublic, CategoryName)
Look through the rest of the schema for similar changes.
Related
I am already using transactions inside my repository functions in some cases because I sometimes need to insert data into two tables at once and I want the whole operation to fail if one of the inserts fails.
Now I ran into a situation where I had to wrap calls to multiple repositories / functions in another transaction, but when one of those functions already uses a transaction internally I will get the error The connection is already in a transaction and cannot participate in another transaction.
I do not want to remove the transaction from the repository function because this would mean that I have to know for which repository functions a transaction is required which I would then have to implement in the service layer. On the other hand, it seems like I cannot use repository functions in a transaction when they already use a transaction internally. Here is an example for where I am facing this problem:
// Reverse engineered classes
public partial class TblProject
{
public TblProject()
{
TblProjectStepSequences = new HashSet<TblProjectStepSequence>();
}
public int ProjectId { get; set; }
public virtual ICollection<TblProjectStepSequence> TblProjectStepSequences { get; set; }
}
public partial class TblProjectTranslation
{
public int ProjectId { get; set; }
public string Language { get; set; }
public string ProjectName { get; set; }
public virtual TblProject Project { get; set; }
}
public partial class TblProjectStepSequence
{
public int SequenceId { get; set; }
public int ProjectId { get; set; }
public int StepId { get; set; }
public int SequencePosition { get; set; }
public virtual TblStep Step { get; set; }
public virtual TblProject Project { get; set; }
}
// Creating a project in the ProjectRepository
public async Task<int> CreateProjectAsync(TblProject project, ...)
{
using (var transaction = this.Context.Database.BeginTransaction())
{
await this.Context.TblProjects.AddAsync(project);
await this.Context.SaveChangesAsync();
// Insert translations... (project Id is required for this)
await this.Context.SaveChangesAsync();
transaction.Commit();
return entity.ProjectId;
}
}
// Creating the steps for a project in the StepRepository
public async Task<IEnumerable<int>> CreateProjectStepsAsync(int projectId, IEnumerable<TblProjectStepSequence> steps)
{
await this.Context.TblProjectStepSequences.AddRangeAsync(steps);
await this.Context.SaveChangesAsync();
return steps.Select(step =>
{
return step.SequenceId;
}
);
}
// Creating a project with its steps in the service layer
public async Task<int> CreateProjectWithStepsAsync(TblProject project, IEnumerable<TblProjectStepSequence> steps)
{
// This is basically a wrapper around Database.BeginTransaction() and IDbContextTransaction
using (Transaction transaction = await transactionService.BeginTransactionAsync())
{
int projectId = await projectRepository.CreateProjectAsync(project);
await stepRepository.CreateProjectStepsAsync(projectId, steps);
return projectId;
}
}
Is there a way how I can nest multiple transactions inside each other without already knowing in the inner transactions that there could be an outer transaction?
I know that it might not be possible to actually nest those transactions from a technical perspective but I still need a solution which either uses the internal transaction of the repository or the outer one (if one exists) so there is no way how I could accidentally forget to use a transaction for repository functions which require one.
You could check the CurrentTransaction property and do something like this:
var transaction = Database.CurrentTransaction ?? Database.BeginTransaction()
If there is already a transaction use that, otherwise start a new one...
Edit: Removed the Using block, see comments. More logic is needed for Committing/Rollback the transcaction though...
I am answering the question you asked "How to nest transactions in EF Core 6?"
Please note that this is just a direct answer, but not an evaluation what is best practice and what not. There was a lot of discussion going around best practices, which is valid to question what fits best for your use case but not an answer to the question (keep in mind that Stack overflow is just a Q+A site where people want to have direct answers).
Having said that, let's continue with the topic:
Try to use this helper function for creating a new transaction:
public CommittableTransaction CreateTransaction()
=> new System.Transactions.CommittableTransaction(new TransactionOptions()
{
IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted
});
Using the Northwind database as example database, you can use it like:
public async Task<int?> CreateCategoryAsync(Categories category)
{
if (category?.CategoryName == null) return null;
using(var trans = CreateTransaction())
{
await this.Context.Categories.AddAsync(category);
await this.Context.SaveChangesAsync();
trans.Commit();
return category?.CategoryID;
}
}
And then you can call it from another function like:
/// <summary>Create or use existing category with associated products</summary>
/// <returns>Returns null if transaction was rolled back, else CategoryID</returns>
public async Task<int?> CreateProjectWithStepsAsync(Categories category)
{
using var trans = CreateTransaction();
int? catId = GetCategoryId(category.CategoryName)
?? await CreateCategoryAsync(category);
if (!catId.HasValue || string.IsNullOrWhiteSpace(category.CategoryName))
{
trans.Rollback(); return null;
}
var product1 = new Products()
{
ProductName = "Product A1", CategoryID = catId
};
await this.Context.Products.AddAsync(product1);
var product2 = new Products()
{
ProductName = "Product A2", CategoryID = catId
};
await this.Context.Products.AddAsync(product2);
await this.Context.SaveChangesAsync();
trans.Commit();
return catId;
}
To run this with LinqPad you need an entry point (and of course, add the NUGET package EntityFramework 6.x via F4, then create an EntityFramework Core connection):
// Main method required for LinqPad
UserQuery Context;
async Task Main()
{
Context = this;
var category = new Categories()
{
CategoryName = "Category A1"
// CategoryName = ""
};
var catId = await CreateProjectWithStepsAsync(category);
Console.WriteLine((catId == null)
? "Transaction was aborted."
: "Transaction successful.");
}
This is just a simple example - it does not check if there are any product(s) with the same name existing, it will just create a new one. You can implement that easily, I have shown it in the function CreateProjectWithStepsAsync for the categories:
int? catId = GetCategoryId(category.CategoryName)
?? await CreateCategoryAsync(category);
First it queries the categories by name (via GetCategoryId(...)), and if the result is null it will create a new category (via CreateCategoryAsync(...)).
Also, you need to consider the isolation level: Check out System.Transactions.IsolationLevel to see if the one used here (ReadCommitted) is the right one for you (it is the default setting).
What it does is creating a transaction explicitly, and notice that here we have a transaction within a transaction.
Note:
I have used both ways of using - the old one and the new one. Pick the one you like more.
Just don't call SaveChanges multiple times.
The problem is caused by calling SaveChanges multiple times to commit changes made to the DbContext instead of calling it just once at the end. It's simply not needed. A DbContext is a multi-entity Unit-of-Work. It doesn't even keep an open connection to the database. This allows 100-1000 times better throughput for the entire application by eliminating cross-connection blocking.
A DbContext tracks all modifications made to the objects it tracks and persists/commits them when SaveChanges is called using an internal transaction. To discard the changes, simply dispose the DbContext. That's why all examples show using a DbContext in a using block - that's actually the scope of the Unit-of-Work "transaction".
There's no need to "save" parent objects first. EF Core will take care of this itself inside SaveChanges.
Using the Blog/Posts example in the EF Core documentation tutorial :
public class BloggingContext : DbContext
{
public DbSet<Blog> Blogs { get; set; }
public DbSet<Post> Posts { get; set; }
public string DbPath { get; }
// The following configures EF to create a Sqlite database file in the
// special "local" folder for your platform.
protected override void OnConfiguring(DbContextOptionsBuilder options)
=> options.UseSqlServer($"Data Source=.;Initial Catalog=tests;Trusted_Connection=True; Trust Server Certificate=Yes");
}
public class Blog
{
public int BlogId { get; set; }
public string Url { get; set; }
public List<Post> Posts { get; } = new();
}
public class Post
{
public int PostId { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public int BlogId { get; set; }
public Blog Blog { get; set; }
}
The following Program.cs will add a Blog with 5 posts but only call SaveChanges once at the end :
using (var db = new BloggingContext())
{
Blog blog = new Blog { Url = "http://blogs.msdn.com/adonet" };
IEnumerable<Post> posts = Enumerable.Range(0, 5)
.Select(i => new Post {
Title = $"Hello World {i}",
Content = "I wrote an app using EF Core!"
});
blog.Posts.AddRange(posts);
db.Blogs.Add(blog);
await db.SaveChangesAsync();
}
The code never specifies or retrieves the IDs. Add is an in-memory operation so there's no reason to use AddAsync. Add starts tracking both the blog and the related Posts in the Inserted state.
The contents of the tables after this are :
select * from blogs
select * from posts;
-----------------------
BlogId Url
1 http://blogs.msdn.com/adonet
PostId Title Content BlogId
1 Hello World 0 I wrote an app using EF Core! 1
2 Hello World 1 I wrote an app using EF Core! 1
3 Hello World 2 I wrote an app using EF Core! 1
4 Hello World 3 I wrote an app using EF Core! 1
5 Hello World 4 I wrote an app using EF Core! 1
Executing the code twice will add another blog with another 5 posts.
PostId Title Content BlogId
1 Hello World 0 I wrote an app using EF Core! 1
2 Hello World 1 I wrote an app using EF Core! 1
3 Hello World 2 I wrote an app using EF Core! 1
4 Hello World 3 I wrote an app using EF Core! 1
5 Hello World 4 I wrote an app using EF Core! 1
6 Hello World 0 I wrote an app using EF Core! 2
7 Hello World 1 I wrote an app using EF Core! 2
8 Hello World 2 I wrote an app using EF Core! 2
9 Hello World 3 I wrote an app using EF Core! 2
10 Hello World 4 I wrote an app using EF Core! 2
Using SQL Server XEvents Profiler shows that these SQL calls are made:
exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [Blogs] ([Url])
VALUES (#p0);
SELECT [BlogId]
FROM [Blogs]
WHERE ##ROWCOUNT = 1 AND [BlogId] = scope_identity();
',N'#p0 nvarchar(4000)',#p0=N'http://blogs.msdn.com/adonet'
exec sp_executesql N'SET NOCOUNT ON;
DECLARE #inserted0 TABLE ([PostId] int, [_Position] [int]);
MERGE [Posts] USING (
VALUES (#p1, #p2, #p3, 0),
(#p4, #p5, #p6, 1),
(#p7, #p8, #p9, 2),
(#p10, #p11, #p12, 3),
(#p13, #p14, #p15, 4)) AS i ([BlogId], [Content], [Title], _Position) ON 1=0
WHEN NOT MATCHED THEN
INSERT ([BlogId], [Content], [Title])
VALUES (i.[BlogId], i.[Content], i.[Title])
OUTPUT INSERTED.[PostId], i._Position
INTO #inserted0;
SELECT [i].[PostId] FROM #inserted0 i
ORDER BY [i].[_Position];
',N'#p1 int,#p2 nvarchar(4000),#p3 nvarchar(4000),#p4 int,#p5 nvarchar(4000),#p6 nvarchar(4000),#p7 int,#p8 nvarchar(4000),#p9 nvarchar(4000),#p10 int,#p11 nvarchar(4000),#p12 nvarchar(4000),#p13 int,#p14 nvarchar(4000),#p15 nvarchar(4000)',#p1=3,#p2=N'I wrote an app using EF Core!',#p3=N'Hello World 0',#p4=3,#p5=N'I wrote an app using EF Core!',#p6=N'Hello World 1',#p7=3,#p8=N'I wrote an app using EF Core!',#p9=N'Hello World 2',#p10=3,#p11=N'I wrote an app using EF Core!',#p12=N'Hello World 3',#p13=3,#p14=N'I wrote an app using EF Core!',#p15=N'Hello World 4'
The unusual SELECT and MERGE are used to ensure IDENTITY values are returned in the order the objects were inserted, so EF Core can assign them to the object properties. After calling SaveChanges all Blog and Post objects will have the correct database-generated IDs
I am already using transactions inside my repository functions in some cases because I sometimes need to insert data into two tables at once and I want the whole operation to fail if one of the inserts fails.
Now I ran into a situation where I had to wrap calls to multiple repositories / functions in another transaction, but when one of those functions already uses a transaction internally I will get the error The connection is already in a transaction and cannot participate in another transaction.
I do not want to remove the transaction from the repository function because this would mean that I have to know for which repository functions a transaction is required which I would then have to implement in the service layer. On the other hand, it seems like I cannot use repository functions in a transaction when they already use a transaction internally. Here is an example for where I am facing this problem:
// Reverse engineered classes
public partial class TblProject
{
public TblProject()
{
TblProjectStepSequences = new HashSet<TblProjectStepSequence>();
}
public int ProjectId { get; set; }
public virtual ICollection<TblProjectStepSequence> TblProjectStepSequences { get; set; }
}
public partial class TblProjectTranslation
{
public int ProjectId { get; set; }
public string Language { get; set; }
public string ProjectName { get; set; }
public virtual TblProject Project { get; set; }
}
public partial class TblProjectStepSequence
{
public int SequenceId { get; set; }
public int ProjectId { get; set; }
public int StepId { get; set; }
public int SequencePosition { get; set; }
public virtual TblStep Step { get; set; }
public virtual TblProject Project { get; set; }
}
// Creating a project in the ProjectRepository
public async Task<int> CreateProjectAsync(TblProject project, ...)
{
using (var transaction = this.Context.Database.BeginTransaction())
{
await this.Context.TblProjects.AddAsync(project);
await this.Context.SaveChangesAsync();
// Insert translations... (project Id is required for this)
await this.Context.SaveChangesAsync();
transaction.Commit();
return entity.ProjectId;
}
}
// Creating the steps for a project in the StepRepository
public async Task<IEnumerable<int>> CreateProjectStepsAsync(int projectId, IEnumerable<TblProjectStepSequence> steps)
{
await this.Context.TblProjectStepSequences.AddRangeAsync(steps);
await this.Context.SaveChangesAsync();
return steps.Select(step =>
{
return step.SequenceId;
}
);
}
// Creating a project with its steps in the service layer
public async Task<int> CreateProjectWithStepsAsync(TblProject project, IEnumerable<TblProjectStepSequence> steps)
{
// This is basically a wrapper around Database.BeginTransaction() and IDbContextTransaction
using (Transaction transaction = await transactionService.BeginTransactionAsync())
{
int projectId = await projectRepository.CreateProjectAsync(project);
await stepRepository.CreateProjectStepsAsync(projectId, steps);
return projectId;
}
}
Is there a way how I can nest multiple transactions inside each other without already knowing in the inner transactions that there could be an outer transaction?
I know that it might not be possible to actually nest those transactions from a technical perspective but I still need a solution which either uses the internal transaction of the repository or the outer one (if one exists) so there is no way how I could accidentally forget to use a transaction for repository functions which require one.
You could check the CurrentTransaction property and do something like this:
var transaction = Database.CurrentTransaction ?? Database.BeginTransaction()
If there is already a transaction use that, otherwise start a new one...
Edit: Removed the Using block, see comments. More logic is needed for Committing/Rollback the transcaction though...
I am answering the question you asked "How to nest transactions in EF Core 6?"
Please note that this is just a direct answer, but not an evaluation what is best practice and what not. There was a lot of discussion going around best practices, which is valid to question what fits best for your use case but not an answer to the question (keep in mind that Stack overflow is just a Q+A site where people want to have direct answers).
Having said that, let's continue with the topic:
Try to use this helper function for creating a new transaction:
public CommittableTransaction CreateTransaction()
=> new System.Transactions.CommittableTransaction(new TransactionOptions()
{
IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted
});
Using the Northwind database as example database, you can use it like:
public async Task<int?> CreateCategoryAsync(Categories category)
{
if (category?.CategoryName == null) return null;
using(var trans = CreateTransaction())
{
await this.Context.Categories.AddAsync(category);
await this.Context.SaveChangesAsync();
trans.Commit();
return category?.CategoryID;
}
}
And then you can call it from another function like:
/// <summary>Create or use existing category with associated products</summary>
/// <returns>Returns null if transaction was rolled back, else CategoryID</returns>
public async Task<int?> CreateProjectWithStepsAsync(Categories category)
{
using var trans = CreateTransaction();
int? catId = GetCategoryId(category.CategoryName)
?? await CreateCategoryAsync(category);
if (!catId.HasValue || string.IsNullOrWhiteSpace(category.CategoryName))
{
trans.Rollback(); return null;
}
var product1 = new Products()
{
ProductName = "Product A1", CategoryID = catId
};
await this.Context.Products.AddAsync(product1);
var product2 = new Products()
{
ProductName = "Product A2", CategoryID = catId
};
await this.Context.Products.AddAsync(product2);
await this.Context.SaveChangesAsync();
trans.Commit();
return catId;
}
To run this with LinqPad you need an entry point (and of course, add the NUGET package EntityFramework 6.x via F4, then create an EntityFramework Core connection):
// Main method required for LinqPad
UserQuery Context;
async Task Main()
{
Context = this;
var category = new Categories()
{
CategoryName = "Category A1"
// CategoryName = ""
};
var catId = await CreateProjectWithStepsAsync(category);
Console.WriteLine((catId == null)
? "Transaction was aborted."
: "Transaction successful.");
}
This is just a simple example - it does not check if there are any product(s) with the same name existing, it will just create a new one. You can implement that easily, I have shown it in the function CreateProjectWithStepsAsync for the categories:
int? catId = GetCategoryId(category.CategoryName)
?? await CreateCategoryAsync(category);
First it queries the categories by name (via GetCategoryId(...)), and if the result is null it will create a new category (via CreateCategoryAsync(...)).
Also, you need to consider the isolation level: Check out System.Transactions.IsolationLevel to see if the one used here (ReadCommitted) is the right one for you (it is the default setting).
What it does is creating a transaction explicitly, and notice that here we have a transaction within a transaction.
Note:
I have used both ways of using - the old one and the new one. Pick the one you like more.
Just don't call SaveChanges multiple times.
The problem is caused by calling SaveChanges multiple times to commit changes made to the DbContext instead of calling it just once at the end. It's simply not needed. A DbContext is a multi-entity Unit-of-Work. It doesn't even keep an open connection to the database. This allows 100-1000 times better throughput for the entire application by eliminating cross-connection blocking.
A DbContext tracks all modifications made to the objects it tracks and persists/commits them when SaveChanges is called using an internal transaction. To discard the changes, simply dispose the DbContext. That's why all examples show using a DbContext in a using block - that's actually the scope of the Unit-of-Work "transaction".
There's no need to "save" parent objects first. EF Core will take care of this itself inside SaveChanges.
Using the Blog/Posts example in the EF Core documentation tutorial :
public class BloggingContext : DbContext
{
public DbSet<Blog> Blogs { get; set; }
public DbSet<Post> Posts { get; set; }
public string DbPath { get; }
// The following configures EF to create a Sqlite database file in the
// special "local" folder for your platform.
protected override void OnConfiguring(DbContextOptionsBuilder options)
=> options.UseSqlServer($"Data Source=.;Initial Catalog=tests;Trusted_Connection=True; Trust Server Certificate=Yes");
}
public class Blog
{
public int BlogId { get; set; }
public string Url { get; set; }
public List<Post> Posts { get; } = new();
}
public class Post
{
public int PostId { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public int BlogId { get; set; }
public Blog Blog { get; set; }
}
The following Program.cs will add a Blog with 5 posts but only call SaveChanges once at the end :
using (var db = new BloggingContext())
{
Blog blog = new Blog { Url = "http://blogs.msdn.com/adonet" };
IEnumerable<Post> posts = Enumerable.Range(0, 5)
.Select(i => new Post {
Title = $"Hello World {i}",
Content = "I wrote an app using EF Core!"
});
blog.Posts.AddRange(posts);
db.Blogs.Add(blog);
await db.SaveChangesAsync();
}
The code never specifies or retrieves the IDs. Add is an in-memory operation so there's no reason to use AddAsync. Add starts tracking both the blog and the related Posts in the Inserted state.
The contents of the tables after this are :
select * from blogs
select * from posts;
-----------------------
BlogId Url
1 http://blogs.msdn.com/adonet
PostId Title Content BlogId
1 Hello World 0 I wrote an app using EF Core! 1
2 Hello World 1 I wrote an app using EF Core! 1
3 Hello World 2 I wrote an app using EF Core! 1
4 Hello World 3 I wrote an app using EF Core! 1
5 Hello World 4 I wrote an app using EF Core! 1
Executing the code twice will add another blog with another 5 posts.
PostId Title Content BlogId
1 Hello World 0 I wrote an app using EF Core! 1
2 Hello World 1 I wrote an app using EF Core! 1
3 Hello World 2 I wrote an app using EF Core! 1
4 Hello World 3 I wrote an app using EF Core! 1
5 Hello World 4 I wrote an app using EF Core! 1
6 Hello World 0 I wrote an app using EF Core! 2
7 Hello World 1 I wrote an app using EF Core! 2
8 Hello World 2 I wrote an app using EF Core! 2
9 Hello World 3 I wrote an app using EF Core! 2
10 Hello World 4 I wrote an app using EF Core! 2
Using SQL Server XEvents Profiler shows that these SQL calls are made:
exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [Blogs] ([Url])
VALUES (#p0);
SELECT [BlogId]
FROM [Blogs]
WHERE ##ROWCOUNT = 1 AND [BlogId] = scope_identity();
',N'#p0 nvarchar(4000)',#p0=N'http://blogs.msdn.com/adonet'
exec sp_executesql N'SET NOCOUNT ON;
DECLARE #inserted0 TABLE ([PostId] int, [_Position] [int]);
MERGE [Posts] USING (
VALUES (#p1, #p2, #p3, 0),
(#p4, #p5, #p6, 1),
(#p7, #p8, #p9, 2),
(#p10, #p11, #p12, 3),
(#p13, #p14, #p15, 4)) AS i ([BlogId], [Content], [Title], _Position) ON 1=0
WHEN NOT MATCHED THEN
INSERT ([BlogId], [Content], [Title])
VALUES (i.[BlogId], i.[Content], i.[Title])
OUTPUT INSERTED.[PostId], i._Position
INTO #inserted0;
SELECT [i].[PostId] FROM #inserted0 i
ORDER BY [i].[_Position];
',N'#p1 int,#p2 nvarchar(4000),#p3 nvarchar(4000),#p4 int,#p5 nvarchar(4000),#p6 nvarchar(4000),#p7 int,#p8 nvarchar(4000),#p9 nvarchar(4000),#p10 int,#p11 nvarchar(4000),#p12 nvarchar(4000),#p13 int,#p14 nvarchar(4000),#p15 nvarchar(4000)',#p1=3,#p2=N'I wrote an app using EF Core!',#p3=N'Hello World 0',#p4=3,#p5=N'I wrote an app using EF Core!',#p6=N'Hello World 1',#p7=3,#p8=N'I wrote an app using EF Core!',#p9=N'Hello World 2',#p10=3,#p11=N'I wrote an app using EF Core!',#p12=N'Hello World 3',#p13=3,#p14=N'I wrote an app using EF Core!',#p15=N'Hello World 4'
The unusual SELECT and MERGE are used to ensure IDENTITY values are returned in the order the objects were inserted, so EF Core can assign them to the object properties. After calling SaveChanges all Blog and Post objects will have the correct database-generated IDs
I have a domain class like below :
public class Employee
{
public int EmployeeId { get; set; }
public int DeptId { get; set; }
}
public class Transaction
{
public int TRID { get; set; }
public int EmployeeId { get; set; }
public string Status { get; set; }
}
Now I want to get all employees from the EmployeeTable for DeptId = 100. I want to calculate Pending status for those employees whose transactions are pending.
So if employee records are found in Transactions table then just want to return a column saying whether employee has any pending transactions or not)
Query :
var t = (from e in _employeeRepository.GetAll() //returns IQueryable<Employee>
where e.DeptId == 100
from t in _transactionRepository.GetAll().Where(t => t.EmployeeId == e.EmployeeId)
select new
{
IsPendingTransaction = (t != null && t.Status != "Done") ? true : false,
}).ToList();
Error : LINQ to Entities does not recognize the method
'System.Linq.IQueryable`1[Transaction] GetAll()' method, and this
method cannot be translated into a store expression."}
Sql Query :
SELECT e.*
(CASE WHEN (t.EmployeeId is not null and t.Status <> 'Done')
THEN CAST(1 AS BIT)
ELSE CAST(0 AS BIT)
End) as IsPendingTransaction
FROM Employee e OUTER APPLY
(SELECT t.*
FROM Transactions t
WHERE e.EmployeeId = t.EmployeeId
) t
WHERE e.DeptId = 100;
The issue is that when you work within IQueryable, every statement inside that Linq expression must be understood by EF to be able to be translated to SQL.
Your first repository call returns an IQueryable<Employee> which you are trying to extend by telling it to join on some code called "_transactionRepository.GetAll()" EF doesn't know what this is, it doesn't correlate to mapped DbSets or properties on entities...
If your Transaction entity has a navigation property back to Employee (which it should) you should be able to accomplish what you want using just the TransactionRepository with something like:
var t = _transactionRepository.GetAll()
.Where(t => t.Employee.DeptId == 100)
.Select(t => new
{
IsPendingTransaction = (t != null && t.Status != "Done") ? true : false
}).ToList();
Using IQueryable<TEntity> in a repository pattern can be quite powerful, however I don't recommend adopting a Generic repository pattern as it just serves to fragment your thinking when working with entities and their relationships with one another, allowing EF to manage the resulting SQL without you resorting to pre-emptively trying to do the joining yourself, often causing conflicts with what EF is capable of working out itself.
Edit: Ok, from your description to get a list of employees with a flag if they have a pending transaction: That would be back at the Employee level with a query something like:
var employees = _employeeRepository.GetAll()
.Where(e => e.DeptId == 100)
.Select(e =>
{
Employee = e,
HasPendingTransaction = e.Transactions.Any(t => t.Status != "Done")
}).ToList();
Or projected to a ViewModel to embed the HasPendingTransaction alongside the Employee details:
var employees = _employeeRepository.GetAll()
.Where(e => e.DeptId == 100)
.Select(e => new EmployeeDetailsViewModel
{
EmployeeId = e.EmployeeId,
Name = e.Name,
// include other relevent details needed for the view...
HasPendingTransaction = e.Transactions.Any(t => t.Status != "Done")
}).ToList();
The advantage of projection is you can build more efficient / faster queries that reduce the amount of data sent over the wire and avoid issues like lazy load trips if you try to serialize entities to the view.
Fix Transaction class
public class Transaction
{
public int TRID { get; set; }
public string Status { get; set; }
public int EmployeeId { get; set; }
public virtual Employee Employee { get; set; }
}
It is not the best idea to have a separate repository for each entity since query usually consists from several entities. It is better to make a join using dbcontext then several repository queries as you trying to do. Don't try to create a base generic repository also. Sooner or later you will see that is is not very helpfull. So add in one of your repositories (probably EmployeeRepository) query like this
var employees= dbcontext.Transactions
.Where(t=> t.Employee.DeptId == 100 && t.EmployeeId==employeeId)
.Select (t=> new {
EmployeeName= t.Employee.Name,
IsPendingTransaction = (t.Status != null && t.Status != "Done") ? true : false}).ToList()
This should be really simple but I think I'm having possible issues with my model. I have been working with linq over a year and I should have this simple remove easily done. Please help! It's removing both records from the database when I only want one deleted
I have a database table with these properties.
Email, EmployeeName, StoreId
jsch#m.com,Joe Schneider,9
jsch#m.com,Joe Schneider,8
I need to delete Joe Schneider with storeId 9
So I run this simple query and remove process.
var temp2 = difference[i];
var PersonToRemove = db.Permissions.SingleOrDefault(s => s.EmployeeName == temp2 && s.StoreId == Persons.StoreId);
if (PersonToRemove.EmployeeName != null)
{
db.Permissions.Remove(PersonToRemove);
db.SaveChanges();
}
I am assuming one is going to say, hey your model is not right and don't put the name as a key, but I can't just be changing the model because other parts of the app are based on this model and would cause huge breaks. Could you give me advise how to edit the linq query to not delete both records?
model
[Table("Permissions")]
public class Permissions
{
[Key]
public String EmployeeName { get; set; }
public string Department { get; set; }
public int? StoreId { get; set; }
public String Email { get; set; }
}
[Table("Permissions")]
public class Permissions
{
[Key]
public String EmployeeName { get; set; }
}
The problem is here you are defining a primary key which has no length constraint on it. (MaxLength). This leads to EF generate a column with NVARCHAR(MAX). As mentioned here VARCHAR(MAX) columns are not allowed to be primary key. So correct definition should be like below
[Table("Permissions")]
public class Permissions
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.None)] <--
[MaxLength(255)] // <---
public String EmployeeName { get; set; }
}
Edit: You need to recreate the database in order to associated tables initialized with correct settings.
Edit 2 : Also you may need a DatabaseGenerated(DatabaseGeneratedOption.None) since its not identity column.
you can set Deleted state on individual entity like so:
var temp2 = difference[i];
var PersonToRemove = db.Permissions.SingleOrDefault(s => s.EmployeeName == temp2 && s.StoreId == Persons.StoreId);
if (PersonToRemove.EmployeeName != null)
{
db.Entry(PersonToRemove).State = EntityState.Deleted; // do this instead
db.SaveChanges();
}
EF should then figure out which entity you wanted to delete
UPD
I am assuming you are using EF6 and DB-first approach. I am also assuming you've got your DB context class set up with default convention model builder. It seems EF's default object tracking based on Key will not work as your key is not unique (this is a bigger problem, but I understand you're already aware of that).
You might try circumvent that convention by adding custom model builder configuration like so:
class MyDbContext : DbContext {
public virtual DbSet<Permissions> Permissions {get;set;}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<Permissions>().HasKey(p => new { p.EmployeeName, p.StoreId});
}
}
since you didn't share your DbContext definition this is just a snippet but hopefully gives you some ideas to explore.
this is the API reference: https://learn.microsoft.com/en-us/dotnet/api/system.data.entity.modelconfiguration.entitytypeconfiguration-1?view=entity-framework-6.2.0
I'm writing a ASP.NET Core Web API project. As a data source It will be using existing (and pretty big) database. But not entire database. The API will use only some of the tables and even in these tables it will not use all the columns.
Using Reverse engineering and scaffolding I was able to generate DbContext and Entity classes... and it got me thinking. There is a table with 30 columns (or more). I'm using this table, but I only need 5 columns.
My question is:
Is there any advantage of removing 25 unused columns from C# entity object? Does it really matter?
The advantage of leaving them there unused is that in case of someone wants to add new functionality that will need one of them, he will not need to go to the db and reverse engineer needed columns (there are there already).
The advantage of removing unused is... ?
EDIT: Here is the sample code:
public class FooContext : DbContext
{
public FooContext(DbContextOptions<FooContext> options)
: base(options)
{
}
public DbSet<Item> Items { get; set; }
}
[Table("item")]
public class Item
{
[Key]
[Column("itemID", TypeName = "int")]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
[Column("name", TypeName = "varchar(255)")]
public string Name { get; set; }
}
Sample usage:
public ItemDto GetItem(int id)
{
var item = _fooContext.Items.Where(i => i.Id == id).FirstOrDefault();
// Here I have item with two fields: Id and Name.
var itemDto = _mapper.Map<ItemDto>(item);
return itemDto;
}
Obviously I'm curious about more complex operations. Like... when item entity is being included by other entity. For example:
_foo.Warehouse.Include(i => i.Items)
or other more complex functions on Item entity
Your entity needs to match what's in the database, i.e. you need a property to match each column (neglecting any shadow properties). There's no choice here, as EF will complain otherwise.
However, when you actually query, you can select only the columns you actually need via something like:
var foos = await _context.Foos
.Select(x => new
{
Bar = x.Bar,
Baz = z.Baz
})
.ToListAsync();
Alternatively, if you don't need to be able to insert/update the table, you can instead opt to use DbQuery<T> instead of DbSet<T>. With DbQuery<T>, you can use anything class you want, and project the values however you like, via FromSql.