Develop Reference

How do I use the UserManager from project A in project B?

Both project A and project B are ASP NET Core 2.2 apps.
Project B uses Hangfire for background jobs and does very little else, and the fact that it uses Hangfire may not even be important (more on this at the bottom). Project A enqueues jobs on B's Hangfire.
Now, let's say I have my class representing a task, called Job. This is contained in project C, a plain old class library referenced by project B, and which in turns references other projects containing the entities it's working with.
Dependencies are to be injected into this class through the constructor:
public class Job
{
public Job(UserManager<ApplicationUser> userManager,
IThisRepository thisRepository,
IThatRepository thatRepository)
{
}
public void Execute(string userId)
{
// this is where the work gets done
}
}
and for the most part they do get injected: IThisRepository and IThatRepository are injected and they work... mostly.
In project B's Startup.cs, the one that is supposed to run this job, I manually and successfully registered those interfaces, along with the DbContext that they require a some other stuff.
UserManager was quite a bit harder to register manually because of all the parameters its constructor requires, so since I didn't really need it inside my job, I just decided to make a few changes.
Now, an example of the entities I'm working with is as follows:
public class Category
{
[Key]
public int Id { get; set; }
// several other properties of primitive types
public ApplicationUser User { get; set; }
[Required]
public string UserId { get; set; }
}
public class Dish
{
[Key]
public int Id { get; set; }
// several other properties of primitive types
public ApplicationUser User { get; set; }
[Required]
public string UserId { get; set; }
public Category Category { get; set; }
[Required]
public string CategoryId { get; set; }
}
now the problem is this: inside of Job I try to create a new Dish and associate it with both the user and the category. Since I just have the user id and I don't have access to UserManager, this is what I try to do:
// ...
var category = await categoryRepository.FindByUserAndCode(userId, "ABC");
// this is a category that is guaranteed to exist
var dish = new Dish();
dish.UserId = userId;
// notice there's no dish.User assignment, because I don't have an ApplicationUser object
dish.Category = category;
dishRepository.Upsert(dish); (which internally either creates a new entity or updates the existing one as appropriate)
and this is where it all breaks down, because it says that a category with the same Id I'm trying to insert is already present, so I'm trying to duplicate a primary key.
Since the category with code ABC for this user exists in the db, I thought it was odd.
Here's the thing: the instance of Category that the repository returns does have it's UserId property populated, but the User property is null.
I think this is what causes my problem: EF probably sees that the property is null and considers this object a new one.
I don't know why it comes up null (and it does even for other entities that all have a property referencing the user), but I tried to backtrack and, instead of using just the user id, I wanted to try to get Hangfire to instantiate Job injecting UserManager<ApplicationUser> into it, so at least I could get an instance of my user by its id.
It's worth noting that this works in other parts of project A, it's just that when I'm executing the background job something goes horribly wrong and I can't for the life of me figure out what it is.
However the dependencies of UserManager are many, and I fear I might be barking up the wrong tree or doing it completely wrong.
I said that the fact I'm using Hangfire might not matter because the assumption under which it operates is: just give me the name of your class, I'll take care of instantiating it as long as all the dependencies have been registered.
Anyone has done this before and can help shed some light?

You've included an absolute ton of information here that is entirely inconsequential to the problem at hand. What your issue boils down is simply the exception you're getting when attempting to add a dish: "a category with the same Id I'm trying to insert is already present, so I'm trying to duplicate a primary key."
This is most normally caused by attempting to use a detached entity as a relationship, i.e.:
dish.Category = category;
If category is detached from the context, then EF will attempt to create it because of this assignment, and since it already exists, that creation fails. We can't see what's going on in categoryRepository.FindByUserAndCode, but I'd imagine you're either calling AsNoTracking with the query, or are newing up an instance of Category manually yourself. In either case, that instance, then, is detached from the context. To attach it again, you simply need to do:
context.Attach(category);
However, you don't have direct access to your context here. This is yet one more reason that you should never use the repository pattern with EF. So much pain and suffering has been subjected on developers throughout the year by either bad advice or erroneously attempting to do things as they are used to.
EF is an ORM (object relational mapper), which is a fancy way of saying that it is itself a data layer. The DbContext is the unit of work and each DbSet is a repository... already. The repository pattern is for abstracting low-level database access (i.e. all the crud of constructing SQL strings, for example). EF is already a high-level abstraction, trying to cram it into another repository pattern layer only cripples it and leads to problems like what you're experiencing here.
Long and short, the issue is that category is detached. You need to either ensure that it never becomes detached in the first place (i.e. don't use AsNoTracking for example) or find a way to ensure that it's reattached later. However, your best bet here is to throw away all this repository garbage completely and just use the context directly. Choosing to use an ORM like EF is simply choosing to use a third-party DAL, rather than write your own. Writing your own, anyways, on top of that is just wrong. You use the built in routing framework in ASP.NET Core. You use the built in templating engine (i.e. Razor). Do you feel the need to put some abstraction around those? Of course not, so why is a DAL any different? If you simply must create an abstraction, then use a meaningful one such as CQRS, service layer, or microservices patterns.

how DbContext.AttachRange() works in this scenario

I saw a book with some code like this:
public class Order
{
public int OrderID { get; set; }
public ICollection<CartLine> Lines { get; set; }
...
}
public class CartLine
{
public int CartLineID { get; set; }
public Product Product { get; set; }
public int Quantity { get; set; }
}
//Product class is just a normal class that has properties such as ProductID, Name etc
and in the order repository, there is a SaveOrder method:
public void SaveOrder(Order order)
{
context.AttachRange(order.Lines.Select(l => l.Product));
if (order.OrderID == 0)
{
context.Orders.Add(order);
}
context.SaveChanges();
}
and the book says:
when store an Order object in the database. When the user’s cart data is deserialized from the session store, the JSON package creates new objects that are not known to
Entity Framework Core, which then tries to write all the objects into the database. For the Product objects, this means that Entity Framework Core tries to write objects that have already been stored, which causes an error. To avoid this problem, I notify Entity Framework Core that the objects exist and shouldn’t be stored in the database unless they are modified
I'm confused, and have two questions:
Q1-why writing objects that have already been stored will cause an error, in the point of view of underlying database, it's just an update SQL statement that modify all columns to their current values?I know it does unnecessary works by changing nothing and rewrite everything, but it shouldn't throw any error in database level?
Q2-why we don't do the same thing to CartLine as:
context.AttachRange(order.Lines.Select(l => l.Product));
context.AttachRange(order.Lines);
to prevent CartLine objects stored in the database just as the way we do it to Product object?

Okay, so this is gonna be a long one:
1st Question:
In Entity Framework (core or "old" 6), there's this concept of "Change tracking". The DbContext class is capable of tracking all the changes you made to your data, and then applying it in the DB via SQL statements (INSERT, UPDATE, DELETE). To understand why it throws an error in your case, you first need to understand how the DbContext / change tracking actually works. Let's take your example:
public void SaveOrder(Order order)
{
context.AttachRange(order.Lines.Select(l => l.Product));
if (order.OrderID == 0)
{
context.Orders.Add(order);
}
context.SaveChanges();
}
In this method, you receive an Order instance which contains Lines and Products. Let's assume that this method was called from some web application, meaning you didn't load the Order entity from the DB. This is what's know as the Disconected Scenario
It's "disconnected" in the sense that your DbContext is not aware of their existence. When you do context.AttachRange you are literally telling EF: I'm in control here, and I'm 100% sure these entities already exist in the DB. Please be aware of them for now on!,
Let's use your code again: Imagine that it's a new Order (so it will enter your if there) and you remove the context.AttachRange part of the code. As soon as the code reaches the Add and SaveChanges these things will happen internally in the DbContext:
The DetectChanges method will be called
It will try to find all the entities Order, Lines and Products in its current graph
If it doesn't find them, they will be added to the "pending changes" as a new records to be inserted
Then you continue and call SaveChanges and it will fail as the book tells you. Why? Imagine that the Products selected were:
Id: 1, "Macbook Pro"
Id: 2, "Office Chair"
When the DbContext looked at the entities and didn't know about them, it added them to the pending changes with a state of Added. When you call SaveChanges, it issues the INSERT statements for these products based on their current state in the model. Since Id's 1 and 2 already exists in the database, the operation failed, with a Primary Key violation.
That's why you have to call Attach (or AttachRange) in this case. This effectively tells EF that the entities exist in the DB, and it should not try to insert them again. They will be added to the context with a state of Unchanged. Attach is often used in these cases where you didn't load the entities from the dbContext before.
2nd question:
This is hard for me to access because I don't know the context/model at that level, but here's my guess:
You don't need to do that with the Cartline because with every order, you probably want to insert new Order line. Think like buying stuff at Amazon. You put the products in the cart and it will generate an Order, then Order Lines, things that compose that order.
If you were then to update an existing order and add more items to it, then you would run into the same issue. You would have to load the existing CartLines prior to saving them in the db, or call Attach as you did here.
Hope it's a little bit clearer. I have answered a similar question where I gave more details, so maybe reading that also helps more:
How does EF Core Modified Entity State behave?

EF Core 2.0 ThenInclude() navigation not reachable

I'm trying to fetch entities which have children and grandchildren
The Entities are following code first conventions and are as follows
//This is the father class
public partial Class Solicitud{
[InverseProperty("Solicitud")]
public virtual ICollection<Operacion> Operaciones { get; set; }
//Other properties
}
//This is the child class
public partial Class Operacion{
[JsonIgnore] //This is so when serializing we don't get a circular reference
[InverseProperty("Operaciones")]
public virtual Solicitud Solicitud { get; set; }
public virtual Practica Practica { get; set; }
//Other Properties
}
//This is the grandchild class
public partial Class Practica
{
String Nombre;
//Other Properties
}
If I do
context.Solicitudes
.Include(w => w.Operaciones)
.Where(x => x.Profesional == profesional).OrderBy(something);
It works out ok, populating the "Operaciones" collections, and leaving the "Practica" property as null as expected.
The problem arises when I try to get the grandchildren, by use of
context.Solicitudes
.Include(w => w.Operaciones)
.ThenInclude(o => o.Practica)
.Where(x => x.Profesional == profesional);
There, it still populates Operaciones, but in each Operacion the property practica stays null, and I get the following message
warn: Microsoft.EntityFrameworkCore.Query[100106]
The Include operation for navigation '[w].Operaciones.Practica' is unnecessary and was ignored because the navigation is not reachable in the final query results. See https://go.microsoft.com/fwlink/?linkid=850303 for more information.
Which to me makes no sense because I could very well do
String something = solicitud.Operaciones.ElementAt(0).Practica.Nombre;
Is this a bug? Is there any way I can avoid using nested selects? The classes are really big in that they have a lot of Properties and it becomes difficult to mantain changes to the domain model using that approach.
Thanks.
Edit: edited title.

It seems that you need to start the query from the entity you want as a result. In your example Practica is not present in the result of your query because is nested (there is no direct path between your resulting query and Practica).
You could try to rewrite your query this way (and add a navigation property inside Practica if not already present):
context.Practicas
.Include(p => p.Operacion)
.ThenInclude(o => o.Solicitud)
.Where(p => p.Operacion.Solicitud.Profesional == profesional)
.ToList();

Well, I think this is actually a bug.
This database is running in SQL Server 2016 and migrated from an old, kind of unmaintained Visual Fox Pro database via an Integration Services package.
Somehow something went wrong while writing said package and the database ended up with rows that broke a foreign key restriction (specifically the one relating Operaciones to Practicas), once I deleted the rows that broke the restriction I ran the query again and it successfully populated every member it was supposed to.
I think this is a bug because the warning message was, in my opinion, a bit misleading. It said that I couldn't access Practica from Solicitud which is tecnically true because it could never get me any Practica as the database was broken, but not very accurate in why I couldn't get them.

Entity Framework is not loading the reference and collection properties lazily

I have Backpack and Book entities. Book references Backpack (one to many).
I am creating an instance of Backpack and bunch of Books. So in this case backpack has bunch of books. I am saving those entities to the db. I am verifying that those got saved to the db. When I try to load backpack it loads fine and all the properties are set except the navigation properties. I am also checking that LazyLoading is not disabled. My navigation properties has the virtual keyword.
I am not sure what I am doing wrong. If I try to Load the backpack with Include() it loads the books:
dbContext.Backpacks.Where(b=>b.Name!="").Include("Books").FirstOrDefault()
I am trying to figure out why it is not loading the books lazily? I have the same problem with loading the book. When I load the book, it doesn't have the backpack attached. I see that the BackpackId is there.
In my property getter/setter I have some logic that will be fired, but I am not sure how that could be a problem.

With the limited information at hand, I can see the following explanations for your problem.
Lazy Loading or Proxy Creation are disabled
Make sure that lazy loading and proxy creation are enabled, the first doesn't work without the latter.
dbContext.Configuration.ProxyCreationEnabled = true;
dbContext.Configuration.LazyLoadingEnabled = true;
(see this SO post for details)
Accessing the entities after disposing the context
Simplified, Lazy Loading works like this:
Whenever you retrieve an entity from the context, you actually get an object of an automatically created subclass of the class you expected that overrides your virtual navigation properties, which is the proxy.
Whenever you then access said navigation properties, the proxy will access the database and load the linked entities when needed.
This last step is, of course, only possible if the entity/proxy is still attached to the context and can therefore query the database to retrieve said objects.
using( var dbContext = new MyContext() )
{
dbContext.Configuration.ProxyCreationEnabled = true;
dbContext.Configuration.LazyLoadingEnabled = true;
// Note: You can modify or derive from `MyContext` so that this is
// done automatically whenever a new `MyContext` is instantiated
var backpack = dbContext.Backpacks.Where(b=>b.Name!="").FirstOrDefault();
// This should work
var firstBook = backpack.Books.FirstOrDefault();
}
// This will probably not, because the context was already disposed
var firstDrink = backpack.Drinks.FirstOrDefault();
I hope this helps, but feel free to provide more information if it doesn't

After a several days of debugging, finally figured out the problem. As people mentioned above, you have to enable the LazyLoading and ProxyCreating. I had the issues even after having the enabling the LazyLoading and ProxyCreating. Also make sure you declare your navigation properties as virtual, otherwise EF will not be able to load entities lazily.
So the issue I had was, EF wasn't crating Proxies because of my entity didn't have a public or protected constructor with NO parameters. After creating public (in my case protected) constructor without parameter it worked.
NOTE: Not having public/protected constructor without parameters will not affect the eager loading.
Here is a link that explains the requirements for the LazyLoading

Eager loading is achieved using the Include() method and as a result you are forcing eager loading by using Include("Books").
Change this:
dbContext.Backpacks.Where(b=>b.Name!="").Include("Books").FirstOrDefault()
to this:
dbContext.Backpacks.Where(b=>b.Name!="").FirstOrDefault()
You should now see that the Books are no longer being loaded eagerly.
Reference:
http://www.entityframeworktutorial.net/EntityFramework4.3/eager-loading-with-dbcontext.aspx
https://msdn.microsoft.com/en-us/library/jj574232(v=vs.113).aspx

Steps I had to do using .NET Core 3.1 and Microsoft.EntityFrameworkCore 3.1.5...
1) Add the Microsoft.EntityFrameworkCore.Proxies NuGet package
2) Configure your DbContext to UseLazyLoadingProxies (in Startup.cs)
public void ConfigureServices(IServiceCollection services)
{
...
services.AddDbContext<DataContext>(optionsBuilder =>
{
optionsBuilder
.UseLazyLoadingProxies() // <--- You need this bit
.UseSqlServer(Configuration.GetConnectionString("DefaultConnection"));
});
...
}
3) Mark all appropriate properties (those you want lazy loaded) as virtual
public class MyEntity
{
public virtual OtherEntity? { get; set; } // Lazy loaded coz `virtual`
public ICollection<OtherEntity> { get; set; } // NOT lazy loaded coz not `virtual`
}

Prevent cached objects to end up in the database with Entity Framework

We have an ASP.NET project with Entity Framework and SQL Azure.
A big part of our data only needs to be updated a few times a day, other data is very volatile.
The data that barely changes we cache in memory at startup, detach from the context and than use it mainly for reading, drastically lowering the amount of database requests we have to do.
The volatile data is requested everytime by a DbContext per Http request.
When we do an update to the cached data, we send a message to all instances to catch a fresh version of all the data from the SQL server.
So far, so good.
Until we introduced a bug that linked one of these 'cached' objects to the 'volatile' data, and did a SaveChanges.
Well, that was quite a mess.
The whole data tree was added again and again by every update, corrupting the whole database with a whole lot of duplicated data.
As a complete hack I added a completely arbitrary column with a UniqueConstraint and some gibberish data on one of the root tables; hopefully failing the SaveChanges() next time we introduce such a bug because it will violate the Unique Constraint.
But it is of course hacky, and I'm still pretty scared ;P
Are there any better ways to prevent whole tree's of cached objects ending up in the database?
More information
Project is ASP.NET MVC
I cache this data, because it is mainly read only, and this saves a tons of extra database calls per http request
This is in a high traffic website, with a lot of personal customized views. Having the POCO data in memory works really good for what I want. Except the problem I mentioned.
It is a bit more complicated, but a simplified version is that I cache the objects by a singleton: so i.e:
EntityCache.Instance.LolCats = new DbContext().LolCats.AsNoTracking().ToList();
This cache I dependency-inject into my controllers.

You can solve it like this:
1) Create an interface like this:
public interface IIsReadOnly
{
bool IsReadOnly { get; set; }
}
2) Implement this interface in all of the entities that can be cached. When you read and cache them, set the IsReadOnly property to true. This flag will be used when SaveChanges is invoked. Remember to decorate this property with the [NotMapped] attribute, or use any other mean to make EF ignore it.
public class ACacheableEntitySample
: IIsReadOnly
{
[NotMapped]
public bool IsReadOnly { get; set; }
// define the "regular" entity properties
}
NOTE: you can include the property directly in the class definition (if using Code First), or use partial classes (for Db First, Model First, or Code First).
NOTE: alternatively you can make EF ignore the IsReadOnly property using the Fluent API, or even better a custom convention (EF 6+)
3) Override your inherited DbContext.SaveChanges method. In the overridden method, review all the entries with pending changes, and if they are read only, change there state to Unchanged:
if (entry is IIsReadOnly) // if it's a cacheable entity
{
if (entry.IsReadOnly) // and it was marked as readonly when caching
{
// change the entry state to unchanged here, so that it's not updated
}
}
NOTE: This is sample code to explain what you need to do. In your final implementation you can do it with a simple LINQ sentence that get all the IIsReadOnly entities, which have the IsReadOnly set to true, and set their state to Unchanged.
You can use the IIsReadOnly entites in another DbContext and manipulate them in the usual way. For example if you get one of these entites, update it, and call SaveChanges, the changes will be saved because IsReadOnly will have the default false value. But you'll easily avoid saving changes of cached data accidentally, simply by setting the IsReadOnly property to true when caching.

Original answer deleted because it was a waste of time.
Your post and proceeding comments are a perfect example of the XY Problem.
You say:
I really need a solution for the problem, not for the architecture
What if the architecture is the problem?
The problem you presented
A caching solution you implemented that violates at least a half dozen best practices has (surprise!) blown up in your face. You've managed to stop it from blowing again up via a spectacular (not in a good way) hack but you want to know how to do it in a way that won't require such a spectacular hack.
The problem you had
You needed to cache some data because it was getting too expensive to hit the database for every request.
The answers that were offered
Use foreign keys instead of navigation properties
This is a perfectly valid answer and, surprise, a best practice. Navigation properties can change any time you regenerate the code in your Entity Data Model and are often ambiguous. With a bit of effort you could have used this and never had to worry about EF's handling of object relationships again.
Cache models instead of Entity objects
Another valid answer, and one that requires the least amount of actual work. MVC applications usually require some redundancy between viewmodels and entity objects and if you ever write a proper multi-tier application you'll practically drown in redundant objects. And nobody will accidentally add these objects to a DbContext ever again - because they can't.
Criticism
You have offered up very little useful information. From what I can tell your approach from the get-go was wrong.
Firstly, dumping whole tables into memory at App_Start is at best a temporary solution. If the table was too big to hit on every request, it's too big to hit on App_Start. What happens if something important breaks while people are using your application and you need to deploy a bug fix ASAP? What happens when your tables get really big and you start getting timeouts from EF while trying to dump them into memory? What happens if 95% of your users only really ever need 10% of that big table you've dumped into memory? Is the memory on your web/cache server going to be enough to accommodate the increasing size of your tables? For how long?
Secondly, no Entity object should remain anywhere after its originating DbContext is disposed. Entity objects behave in a convenient way while their DbContext is in scope and become troublesome POCOs when it's out of scope. I say troublesome because the 'magic' DbContext does with change tracking tends to fool people unfamiliar with the inner workings of EF into thinking that an Entity object is directly connected to a table row in the database. The problem you had illustrates this point perfectly.
Thirdly, it looks like you need to delete and re-dump a whole table to memory, even if you only update a single column in a single row. That's immensely wasteful to both the memory and CPU on your web server, and to your Azure SQL instance(s). What happens when a small bit of data comes in wrong and needs to be updated in a hurry? What if one of your nightly update jobs fails but you need fresh data in the morning?
You may not worry about any of this stuff now but your solution blowing up in your face should have at the very least raised some red flags. I've had to deal with as lot of caching in projects I've worked on in the past few years and everything I say here comes from experience.
Proposed solution - On-demand caching
If you've put a little effort into organizing your code, all of your CRUD operations on the database should be in specialized helper classes which I call repositories. Your controller calls its specialized repository (StuffController - StuffRepository), receives a model and binds that model to a view, kinda like this:
public class StuffController : Controller
{
private MyDbContext _db;
private StuffRepository _repo;
public StuffController()
{
_db = new MyDbContext();
_repo = new StuffRepository(_db);
}
// ...
public ActionResult Details(int id)
{
var model = _repo.ReadDetails(id);
// ...
return View(model);
}
protected override void Dispose(bool disposing)
{
_db.Dispose();
base.Dispose(disposing);
}
}
What on-demand caching would do is wrap that call to the repository in such a way that if the result of that method was already in the cache and it was not stale, it would return it from the cache. Otherwise it would hit the database.
Here's a simplified (and probably nonfunctional) example of a CacheWrapper class so you can understand what it does, using HttpRuntime.Cache:
public static class CacheWrapper
{
private static List<string> _keys = new List<string>();
public static List<string> Keys
{
get { lock(_keys) { return _keys.ToList(); } }
}
public static T Fetch<T>(string key, Func<T> dlgt, bool refresh = false) where T : class
{
var result = HttpRuntime.Cache.Get(key) as T;
if(result != null && !refresh) return result;
lock(HttpRuntime.Cache)
{
lock(_keys)
{
_keys.Add(key);
}
result = dlgt();
HttpRuntime.Cache.Add(key, result, /* some other params */);
}
return result;
}
}
And the new way to call things from the controller:
public ActionResult Details(int id)
{
var model = CacheWrapper.Fetch("StuffDetails_" + id, () => _repo.ReadDetails(id));
// ...
return View(model);
}
A slightly more complex version of this is in production on a public web application as we speak and working quite well.