Both project A and project B are ASP NET Core 2.2 apps.
Project B uses Hangfire for background jobs and does very little else, and the fact that it uses Hangfire may not even be important (more on this at the bottom). Project A enqueues jobs on B's Hangfire.
Now, let's say I have my class representing a task, called Job. This is contained in project C, a plain old class library referenced by project B, and which in turns references other projects containing the entities it's working with.
Dependencies are to be injected into this class through the constructor:
public class Job
{
public Job(UserManager<ApplicationUser> userManager,
IThisRepository thisRepository,
IThatRepository thatRepository)
{
}
public void Execute(string userId)
{
// this is where the work gets done
}
}
and for the most part they do get injected: IThisRepository and IThatRepository are injected and they work... mostly.
In project B's Startup.cs, the one that is supposed to run this job, I manually and successfully registered those interfaces, along with the DbContext that they require a some other stuff.
UserManager was quite a bit harder to register manually because of all the parameters its constructor requires, so since I didn't really need it inside my job, I just decided to make a few changes.
Now, an example of the entities I'm working with is as follows:
public class Category
{
[Key]
public int Id { get; set; }
// several other properties of primitive types
public ApplicationUser User { get; set; }
[Required]
public string UserId { get; set; }
}
public class Dish
{
[Key]
public int Id { get; set; }
// several other properties of primitive types
public ApplicationUser User { get; set; }
[Required]
public string UserId { get; set; }
public Category Category { get; set; }
[Required]
public string CategoryId { get; set; }
}
now the problem is this: inside of Job I try to create a new Dish and associate it with both the user and the category. Since I just have the user id and I don't have access to UserManager, this is what I try to do:
// ...
var category = await categoryRepository.FindByUserAndCode(userId, "ABC");
// this is a category that is guaranteed to exist
var dish = new Dish();
dish.UserId = userId;
// notice there's no dish.User assignment, because I don't have an ApplicationUser object
dish.Category = category;
dishRepository.Upsert(dish); (which internally either creates a new entity or updates the existing one as appropriate)
and this is where it all breaks down, because it says that a category with the same Id I'm trying to insert is already present, so I'm trying to duplicate a primary key.
Since the category with code ABC for this user exists in the db, I thought it was odd.
Here's the thing: the instance of Category that the repository returns does have it's UserId property populated, but the User property is null.
I think this is what causes my problem: EF probably sees that the property is null and considers this object a new one.
I don't know why it comes up null (and it does even for other entities that all have a property referencing the user), but I tried to backtrack and, instead of using just the user id, I wanted to try to get Hangfire to instantiate Job injecting UserManager<ApplicationUser> into it, so at least I could get an instance of my user by its id.
It's worth noting that this works in other parts of project A, it's just that when I'm executing the background job something goes horribly wrong and I can't for the life of me figure out what it is.
However the dependencies of UserManager are many, and I fear I might be barking up the wrong tree or doing it completely wrong.
I said that the fact I'm using Hangfire might not matter because the assumption under which it operates is: just give me the name of your class, I'll take care of instantiating it as long as all the dependencies have been registered.
Anyone has done this before and can help shed some light?
You've included an absolute ton of information here that is entirely inconsequential to the problem at hand. What your issue boils down is simply the exception you're getting when attempting to add a dish: "a category with the same Id I'm trying to insert is already present, so I'm trying to duplicate a primary key."
This is most normally caused by attempting to use a detached entity as a relationship, i.e.:
dish.Category = category;
If category is detached from the context, then EF will attempt to create it because of this assignment, and since it already exists, that creation fails. We can't see what's going on in categoryRepository.FindByUserAndCode, but I'd imagine you're either calling AsNoTracking with the query, or are newing up an instance of Category manually yourself. In either case, that instance, then, is detached from the context. To attach it again, you simply need to do:
context.Attach(category);
However, you don't have direct access to your context here. This is yet one more reason that you should never use the repository pattern with EF. So much pain and suffering has been subjected on developers throughout the year by either bad advice or erroneously attempting to do things as they are used to.
EF is an ORM (object relational mapper), which is a fancy way of saying that it is itself a data layer. The DbContext is the unit of work and each DbSet is a repository... already. The repository pattern is for abstracting low-level database access (i.e. all the crud of constructing SQL strings, for example). EF is already a high-level abstraction, trying to cram it into another repository pattern layer only cripples it and leads to problems like what you're experiencing here.
Long and short, the issue is that category is detached. You need to either ensure that it never becomes detached in the first place (i.e. don't use AsNoTracking for example) or find a way to ensure that it's reattached later. However, your best bet here is to throw away all this repository garbage completely and just use the context directly. Choosing to use an ORM like EF is simply choosing to use a third-party DAL, rather than write your own. Writing your own, anyways, on top of that is just wrong. You use the built in routing framework in ASP.NET Core. You use the built in templating engine (i.e. Razor). Do you feel the need to put some abstraction around those? Of course not, so why is a DAL any different? If you simply must create an abstraction, then use a meaningful one such as CQRS, service layer, or microservices patterns.
Related
There seems to be a certain movement advocating that when we use EF Core we should avoid creating a Repository & Unit of work pattern because EF Core already implement those two and we can leverage this implicit implementation. That would be great because implementing those patterns is not always as straightforward as it would seem.
So here's the problem. When implementing repository the 'classic' way we have a place to put the code that builds our domain objects. Let me explain with an example; we have an Invoice and an InvoiceRow entities. Each Invoice has many InvoiceRows. I included only the navigational property for brevity.
public class Invoice
{
public int Id { get; set; }
public DateTime Date { get; set; }
public decimal Total { get; set; }
public List<InvoiceRow> InvoiceRows { get; }
}
public class InvoiceRow
{
public int Id { get; set; }
public Invoice Invoice { get; set; }
public decimal UnitPrice { get; set; }
public decimal RowPrice { get; set; }
}
Now, my business object is an Invoice with its rows, and this should be the only way to manipulate the invoices.
When using 'explicit' repository we would do something like:
public class InvoicesRepo
{
public AppDbContext AppDbContext { get; private set; }
public Invoice Find(int id)
{
return
AppDbContext.Invoices.Where(invoice => invoice.Id == id)
.Include(nameof(InvoiceRow))
.First();
}
}
This restricts the access to the Invoice to the method [InvoicesRepo].Find(id) that builds the invoice in the way that is expected by the domain logic code.
Is it possible to achieve this with bare EF Core? Maybe working with visibility of DbSets and/or additional features that I don't know? Since this seems to be quite a fundamental functionality of a full-blown repository, if it's not achievable, have I just destroyed the main argument of experts advocating for no (additional) repository when using EF Core?
Is it possible to achieve this with bare EF Core? Maybe working with visibility of DbSets and/or additional features that I don't know?
Sure, accepting that the DbContext is your repository doesn't mean you can't make design decisions and you have to have use the default DbContext design.
You can add reusable data access code to your DbContext for convenience and consistency, eg methods like:
public Invoice FindInvoice(int id)
{
this.Invoices.Where(invoice => invoice.Id == id)
.Include(nameof(InvoiceRow))
.First();
}
So for code that needs the standard shape of Invoice with InvoiceRows, they call this method. But for code that needs some nonstandard shape, they still can access the DbSets or IQueryable methods and construct a custom query.
You can even eliminate the DbSet properties, to more strongly guide users to use your custom methods, like:
public IQueryable<Invoice> Invoices => this.Invoices.Include(nameof(InvoiceRow));
Then to get Invoices without InvoiceRows a consumer would either add a custom projection to this, something like
db.Invoices.Where(i => i.CustomerID == custId).Select(i => new InvoiceDTO(i)).ToList();
or access the DbSet
var invoice = db.Set<Invoice>().Find(invoiceId);
And you can organize the methods on your DbContext by having it implement various interfaces.
Ok, this is just a preliminary answer based on many readings all around.
public class InvoiceQuery
{
public AppDbContext AppDbContext { get; private set; }
public int Id { get; set; }
public Invoice Execute()
{
return AppDbContext
.Invoices
.Include(nameof(Invoice.InvoiceRows))
.Where(invoice => invoice.Id == Id)
.FirstOrDefault();
}
}
My problem was that this is not substantially different from what we would have into a repository. That's from a practical point of view; from a theoretical point of view, this puts you on the right perspective while the repository is sort of misleading.
The reason why it's misleading is that there is no 1-1 association between entities and actions or queries that you can do on the database. Even in this case, Invoice is not just invoice, but is Invoice plus InvoiceRows. (By the way, I think that InvoiceQuery is a good name (and not InvoiceWithRowsQuery) because from a business logic point of view an invoice is a full-loaded invoice; an invoice with 0 rows is an empty invoice, not an partially-loaded invoice.)
So a query focuses on what you get, not on the entity you start from, because they can be more than one.
This "query" name is sort of counter-intuitive, because one would say that as you move towards the business logic, you stop seeing things like queries and you start seeing things like paramenters. And actually we have parameters, "query" is only the name of the container. Maybe we should call it Business Object Query, but it would be too long, but that's the meaning. So this query object is just a simple container for an EF Query; I will talk about this later on. In this class based implementation, besides being glorified as object, there is no addictional functionality. Maybe you would expect some additional functionality, instead you have something less. This missing thing is that they are no associated with an entity anymore. This is another counter-intuitive fact that proves that here we are dismantleing something: the contrived entity-repository view.
Having a so simple object poses a question. Why not a method on some related class? If we put it on the Entity, we go back to the repository-like organization, that seems flawed mainly because the missing 1-1 bla bla means that in many cases you couldn't tell which entity associate a query to. But some authors use a sort of 'container class' that is really no more than a container just it's not entitled to an entity (OrdersData for example). If you like me like classes that have a clear purpose, this sends you shivers down the spine. We started up saying that we have a solution that is better than the repository. That the repository has so many problems. We end up with a class that has "Data" in its name. How could a better solution be so lame? It feels like the repository alternative is just a bunch of stuff, not a steady and well designed as expected. On the other hand, this is just a way to collect queries (the same queries that you would have in a repository, the very same query that you would have in a query/command pattern...) under a name that is not that of an entity.
Yes, all this seems to boil down to a naming choice. Advocates of repository pattern and advocates of no-repository fiercely fighting each others. Then you look at the code, what it actually does. The code is the same and the names are all we are talking about. That is: if you look what repositoriests do in their repositories and non-repositoriests do in their whatever (not always clear) classes, they do exaclty the same things. But this is just an impression, I've to time-proof it.
In the repository model, having a repository for entity with its methods was reassuring. It provided a sort of scaffolding where to put all of the methods. Unfortunately that scaffolding seems to be too limitating, again because of the not 1-1 relation between entities and queries/commands. Yet we have to say that the root entities like Invoice (in DDD speaking Invoice are root, InvoiceRows are aggregate) would fit quite well in repository-style organization.
For this reason, in this query based solution the flatness of the Query collection is not fully satisfying, and this is another reason why I think I want to further elaborate on this topic.
My goal is async loading of related entities using DBContext.
Let imagine two projects. The first named MyApp.Domain and contains domain entities.
namespace MyApp.Domain
{
public class PlanPage
{
public Guid Id { get; set; }
}
}
namespace MyApp.Domain
{
public class PlanPageDay
{
public Guid Id { get; set; }
public Guid PlanPageId { get; set; }
}
}
The second project named MyApp.Infrastructure.EntityFramework and contains configuration of projection entities to database. It also contains class which extends domain entity and implements Entity framework specific logic.
namespace MyApp.Infrastructure.EntityFramework.Models
{
public class PlanPageEntity : PlanPage
{
private readonly ApplicationDbContext _applicationDbContext;
protected PlanPageEntity(ApplicationDbContext applicationDbContext)
{
_applicationDbContext = applicationDbContext;
}
public ICollection<PlanPageDay>? Days { get; set; }
public async Task<ICollection<PlanPageDay>> GetDays()
{
return Days ??= await _applicationDbContext.PlanPageDays
.Where(pd => pd.PlanPageId == Id)
.ToListAsync();
}
}
}
The purpose of this example is simple. We separate infrastructure code from domain code. Look how do we plan to use this concept:
// Entity initializing code. Placing somewhere in domain logic.
var plan = new PlanPage(/*some constructor arguments*/);
// Entity loading code. Placing somewhere in infrastructure implementation.
public async Task<PlanPage> GetPlanPage(Guid id)
{
return await _applicationDbContext.Set<PlanPageEntity>().FindAsync(id);
}
Note that we tell to Entity framework to use child class (PlanPageEntity) so it can handle all specific things that it can.
The question is: Is it possible to configure the EF so that it allows us to use this concept?
As requested here's a little more details for my opinion stated in the comments.
The main reason why I think your current approach is a bad idea is that it violates the separation of concerns design principle: when you are mixing domain models with data access models, you make your domain logic completely dependent on how you model the data in your database. This quickly limits your options because the database may have some restrictions on how you can model your data that doesn't fit well with the domain logic you want to implement as well as making maintenance difficult. E.g. if you decide to split up one DB table into two then you might have a big task ahead of you in order to make your domain logic work with those two new models/tables. Additionally, making performance optimizations in your database easily becomes a nightmare if not thought through ahead of time - and you shouldn't spend time thinking of optimizing your system before it's necessary.
I know this is a little abstract since I don't know much about your domain but I'm sure I could find more arguments against it.
Instead, separating data access models (and in general all external data models) from your domain models makes it much easier to maintain: if you need to make some changes to your database, you simply need to update the logic that maps the data from your data access models to your domain model - nothing in your domain logic needs to change.
In the examples you have given, you have already logically separated your domain models and data access models into two separate projects. So why not follow through with that thought and separate the two with a binding/mapping layer in-between?
Is it possible to configure the EF so that it allows us to use this concept?
Yes. Essentially you have DTO's, and your Entities derive from your DTOs. So when you fetch an Entity you can return it directly. But if you wouldn't be able to attach a non-Entity, so you'd have to map it. It's going to be inconvenient, and like 99.999% of bespoke entity and repository designs, will be ultimately a waste of time.
This is somewhat similar to the what EF already does for you. Start with persistence-ignorant Entity classes, and introduce persistence-aware runtime subtypes for scenarios that require them, which is basically just Lazy Loading.
For a project we are starting to look at persistence features and how we want to implement this. Currently we are looking at keeping Clean Architecture in mind, probably going for Onion Architecture. As such, we want to define a new outer layer which in which the persistence layer resides.
We're looking at various ORM solutions (we seem to be converging to Entity Framework) using SQLite as data store and we are hitting a snag: How should be manage ID's and deal with add/removal in some collection or move some instance between different collections.
In the core of our 'onion', we want to keep our POCO objects. As such, we do not want some kind of 'ID' property to be added in our business objects. Only inside the persistence layer do we want to have classes with object ID's. Because of this separation:
how should removing a business object from some collection cause a row to be deleted from the SQLite database?
More complex (at least I think it is), how should a POCO instance moved from 1 collection to another cause a foreign key of a SQLite databaserow to be changed? (Instead of removing the row and recreating it with the same values)
Looking around the internet I've yet to find an implementation somewhere that demonstrates a persistence layer in a Clean Architecture design. Plenty of high level diagrams and "depend only inward", but no source code examples to give a demonstration.
Some possible solutions that we came up with so far:
Have some lookup between POCO instances and their representative 'database model objects' (which have ID's etc) within the persistence layer. When saving the project state, business model objects will be matched with this database model objects and update the state for the matches accordingly. Then the object is persisted.
When loading a project, the persistence layer returns decorator objects of business objects that add an ID to the business object, which is only visible within the persistence layer by casting the objects to that decorator class. However, this prevents us from defining sealed POCO objects and seems to break the Clean Architecture design philosophy.
Option 1 seems costly in memory due to effectively doubling the business objects in memory. Option 2 seems the most elegant, but as I've written: it feels that it breaks Clean Architecture.
Are there better alternatives to there? Should we just go for Option 2 and take Clean Architecture more as guidelines than rule? Can someone point us to a working example in code (I did find a iOs example at https://github.com/luisobo/clean-architecture, but as I'm not literate in the language it cannot do much with it).
As others have mentioned in the comments, IDs are a natural part of applications and are usually required in other parts than persistence. So trying to avoid IDs at all costs is going to produce awkward designs.
Identity Design
However, identity design (where to use which IDs, what information to put in IDs, user defined vs system generated, etc.) is something that is very important and requires thought.
A good starting point to determine what requires an ID and what not is the Value Object / Entity distinction of domain-driven design.
Value objects are things that consist of other values and don't change - so you don't need an ID.
Entities have a lifecycle and change over time. So their value alone is not enough to identify them - they need an explicit ID.
As you see here, reasoning is very different from the technical point of view that you take in your question. This does not mean you should ignore constraints imposed by frameworks (e.g. entity framework), however.
If you want an in-depth discussion about identity design, I can recommend "Implementing DDD" by Vaughn Vernon (Section "Unique Identity" in Chapter 5 - Entities).
Note: I don't mean to recommend that you use DDD because of this. I just think that DDD has some nice guidelines about ID design. Whether or not to use DDD in this project is an entirely different question.
First of all, everything in the real world have ids. You have your social security number. Cars have their registration number. Items in shops have an EAN code (and a production identity). Without ids nothing in the world would work (a bit exaggerated, but hopefully you get my point).
It's the same with applications.
If your business objects do not have any natural keys (like a social security number) you MUST have a way to identify them. You application will otherwise fail as soon as you copy your object or transfer it over the process boundry. Because then it's a new object. It's like when you cloned the sheep Dolly. Is it the same sheep? No, it's Mini-Dolly.
The other part is that when you build complex structures you are violating the law of Demeter. For instance like:
public class ForumPost
{
public int Id { get; set; }
public string Title { get; set; }
public string Body { get; set; }
public User Creator { get; set; }
}
public class User
{
public string Id { get; set; }
public string FirstName { get; set; }
}
When you use that code and invoke:
post.User.FirstName = "Arnold";
postRepos.Update(post);
what do you expect to happen? Should your forum post repos suddenly be responsible of changes made in the user?
That's why ORMs are so sucky. They violate good architecture.
Back to ids. A good design is instead to use a user id. Because then we do not break law of Demeter and still got a good separation of concern.
public class ForumPost
{
public int Id { get; set; }
public string Title { get; set; }
public string Body { get; set; }
public int CreatorId { get; set; }
}
So the conclusion is:
Do not abandon ids, as it introduces complexity when trying to identify the real object from all the copies of it that you will get.
Using ids when referencing different entities helps you keep a good design with distinct responsibilities.
A project I'm working on has DbContext that tracks a lot of different Entities. Due to the large number of relationships involved, it takes a long time to query from the context the first time around while it generates its views. In order to reduce the startup time, and better organize contexts into functional areas, I'm looking for ways to split it apart.
These are some methods I've tried so far, and problems I've seen with them:
Create a new smaller Context with a subset of DbSets from the huge Context.
This doesn't help, since EF seems to crawl through all the navigation properties and include all related entities anyway (according to LINQPad at least, which shows all the entities related to the Context when it's expanded in the connection panel). We have a few top-level entities that are far reaching, so there are very few subsets that can be fully isolated without removing navigation properties and doing a good amount of refactoring.
Split Entities into classes that include navigation properties, and ones that are just db fields, like so:
public class PersonLight
{
public int Id { get; set; }
public string Name { get; set; }
public int JobId { get; set; }
}
public class Person : PersonLight
{
public Job Job { get; set; }
}
public class ContextLight : DbContext
{
public virtual DbSet<PersonLight> People { get; set; }
}
No dice here as well. Even though Person isn't used at all, EF (or again, possibly just LINQPad) includes Person despite the fact that it can't be used. I assume this is because EF supports inheritance patterns, so it ends crawling related entities in this direction as well.
Do the same as #2, but with PersonLight and Person in different projects (or use partial classes in different projects). This is the best option so far, but it would be nice to have PersonFields right next to Person for easy reference.
So my questions are:
Are there any better ways to do this that I'm missing?
Why, in #3, does putting them in different projects seem to separate them enough that EF doesn't try to include both? I've tried putting them in different namespaces, but that doesn't do the trick.
Thanks.
Options to speed things along:
Generated views
Bounded Contexts
Ironically IIS app pool only needs to generate the view once.
Command line based on my tests, generates the view each time.
Not sure what linqpad does.
BTW I didn't originally add this link since you tagged it EF6.
But in case others aren't on EF6. There are some performance improvements reported. More information here:
EF6 Ninja edition
I have these entities:
public class Company : PrimaryKey
{
public string Name {get;set;}
public virtual Account Account {get;set;}
}
public class Account
{
[Key]
public Guid CompanyId { get; set; }
public virtual Company Company {get;set;}
}
I use these configurations:
modelBuilder.Entity<Company>()
.HasOptional(c => c.Account)
.WithRequired(a => a.Company)
.WillCascadeOnDelete();
Now, I have two projects, one is a test bench project which is a Console Application with a DbContext and a Repository, the second is the full blown production project which is a MVC 4 in which I use Dependancy Injection to create a Repository .InTransientScope() which in turn loads a new context each time it is called.
Both have exactly the same contexts and repositories (the product obviously has Interfaces).
in the test bench when I call this:
_repository.GetById<Company>(id);
All of it properties are filled out, i.e. eager loading
in the production when I call the same line, nothing is loaded and its not loaded till I created another function which does this:
_dbContext.Companies.Include("Account").FirstOrDefault(x => x.Id.Equals(id));
Of which, when executed does provide all the Account information, but funnily bar any other navigation properties that Account contains!!! Even though I have disable LazyLoading, it still doesn't work.
This is surprising because both projects are fundamentally the same, bar the use of the IoC DI in one of them....
Why is this happening? How can I specify in a predominantly generic Repository to eager load all this information at the Controllers preference....?
Break Points
I set break points in both projects too look at the ADO.NET call to the database and the sql statement that was executed, in the test bench it did go off and call the information, in the production it did not show any joins or anything of that nature what so ever.
Other Things Tried
I tried accessing the navigation property directly when loading it from the database:
var acc = _repository.GetById<Company>(id).Account;
It still says null. So my repository/context is not even loading any related data when asked for it.... what is going on?!
Definitions
_repository.GetById<Company>(id);
is:
public T GetById<T>(Guid id)
{
return _dbContext.Set<T>().FirstOrDefault(x => x.Id.Equals(id));
}
It's working now, I have no idea why.. I haven't tampered with anything. The only thing that I have, was to put .InTransientScope() onto IDbContextFactory<MyContext>
I actually enabled all Lazy Loading everywhere I could, and it now works.... but it's strange that when I started on the Production project I never even tampered with Lazy Loading at all, but since I extented the Model and added modelBuilder stuff I have had to specifically tell it to Lazy Load.