Develop Reference

Calculated fields that improve performance but need to be maintained (EF)

I have this "1 to N" model:
class Reception
{
public int ReceptionId { get; set; }
public string Code { get; set; }
public virtual List<Item> Items { get; set; }
}
class Item
{
public int ItemId { get; set; }
public string Code { get; set; }
public int Quantity { get; set; }
public int ReceptionId { get; set; }
public virtual Reception Reception { get; set; }
}
And this action, api/receptions/list
public JsonResult List()
{
return dbContext.Receptions
.Select(e => new
{
code = e.Code,
itemsCount = e.Items.Count,
quantity = e.Items.Sum(i => i.Quantity)
}).ToList();
}
which returns a list of receptions, with their number of items:
[
{code:"1231",itemsCount:10,quantity:30},
{code:"1232",itemsCount:5,quantity:70},
{code:"1234",itemsCount:30,quantity:600},
...
]
This was working fine but I'm having too many Reception's and Item's thus the query is taking too long...
So I want to speed up by adding some persisted fields to Reception:
class Reception
{
public int ReceptionId { get; set; }
public string Code { get; set; }
public virtual List<Item> Items { get; set; }
public int ItemsCount { get; set; } // Persisted
public int Quantity { get; set; } // Persisted
}
With this change, the query ends up being this:
public JsonResult List()
{
return dbContext.Receptions
.Select(e => new
{
code = e.Code,
itemsCount = e.ItemsCount,
quantity = e.Quantity
}).ToList();
}
My question is:
What's the best way to maintain these two fields?
I will gain in performance but now I will need to be more careful with the creation of Item's
Today an Item can be created, edited and deleted:
api/items/create?receptionId=...
api/items/edit?itemId=...
api/items/delete?itemId=...
I also have a tool for importing receptions via Excel:
api/items/createBulk?...
Maybe tomorrow I will have more ways of creating Item's, so the question is how do I make sure that these two new fields, ItemsCount and Quantity will be up to date always?
Should I create a method within Reception like this?
class Reception
{
...
public void UpdateMaintainedFields()
{
this.Quantity = this.Items.Sum(e => e.Quantity);
this.ItemsCount = this.Items.Count();
}
}
And then REMEMBER to call it from all the previous URL's? (items/create, items/edit, ...)
Or maybe should I have a stored procedure in the database?
What is the common practice? I know there are calculated columns but these refer to fields of the same class. Also there are indexed views, but I'm not sure if they apply well to scenarios like this.

From your code it seems to me that you do not have a layer for business logic, and everything is implemented in the controllers, this causes the problem for you that when you would have a different way (and it seems, that you mean a different controller) you have to implement this logic again and it is easy to forget, and if you do not forget, you could forget to maintain later.
So I would recommend to have a layer for business logic (like adding new items) and use it from the controllers where you want to create items.
I would also recommend write the function UpdateMaintainedFields as you asked, but call it in the business logic layer after adding the items, not in the controllers!
You could write the logic on the database also (trigger) if you can accept that you can't write unit test.

Assuming the original query cannot be improved with the correct execution plan in SQLServer, the way to update these fields is via a trigger in the DB. When an insert occurs (or possible an update if your persisted fields change according to the data) then when an insert occurs to that table, the trigger is run. It would be responsible for updating all the rows with the new values.
Obviously your insert performance would drop, but your query performance would be that of a simple index and read of a single row. Obviously you wouldn't be able to use this trick if you were to return a subset of the table, as all the quantities would be fixed.
An alternative is to hold the count and quantity sums in a separate table, or in a dummy row that holds the summed quantities as its entry for quantity. YMMV.
PS I hate how what is a SQL question has been turned in one about C# code! Learn SQL and run the queries you need directly in the DB, that will show you much more about the performance and structure of what you're looking for than getting EF involved. /rant :)

You want to store the same information duplicitly, which can lead to inconsistencies. As an inspiration, indexes are also duplicating data. How do you update them? You don't. It is all fully transparent. And I would recommend the same approach here.
Make sum table, maintained by triggers. The table would not be included in any datacontext schema, only way to read it would be through non updateable views or stored procedures. Its name should evoke, that nobody should ever touch this table directly.
You can now access your data from various frameworks and do not worry about updating anything. Database would assure the precalculated sums are always correct, as long as you do not write to the sum table on your own. In fact you can add or remove this table any time and no application would even notice.

Entity Framework Code First Relationships; what am I missing?

I'm experimenting with EF5 Code First and I am using the models (show below).
When I look at the database that is created, I am confused because I do not see anything in the Track table that points to the Category table. Category has a FK pointing back to Track but that means that there are going to be duplicates of the categories?
A little background: I am trying to build a model that has tracks and every track can have 1 to N Categories. All of the categories are already defined, that is they are basically a lookup and I plan to create them in the seed method when database is created.
I think I am not understanding something obvious... When I query a track, how will I know what category it contains?
Thx
public class Track : IAuditInfo
{
public Int32 Id { get; set; }
public String Name { get; set; }
public String Description { get; set; }
public String Data { get; set; }
public DateTime CreatedOn { get; set; }
public DateTime ModifiedOn { get; set; }
public ICollection<Category> Categories { get; set; }
public Track()
{
Categories = new List<Category>();
}
}
public class Category
{
public Int32 Id { get; set; }
public Boolean IsVisible { get; set; }
public String DisplayName { get; set; }
}

Your current model is a one-to-many relationship between tracks and categories.
This usually implemented, as you have noted that entity framework does, using a foreign key on the many side (category) to the one side (track).
If I understand you correctly, what you want is a many-to-many relationship. Many tracks can be related to the same category, and a single track can belong to many categories.
To let entity framework understand that you want a many-to-many relationship you can simply add a ICollection property to your category class.
So both your classes should have a collection of the other class.
I.e. tracks have many categories and categories have many tracks.
For more information you can also see: http://msdn.microsoft.com/en-us/data/hh134698.a.nospx

Olav is right, your data model at the moment is not telling Entity Framework that there is a many-to-many relationship in there.
The simplest way to resolve this is to add
public virtual ICollection<Track> Tracks { get; set; }
to your Category class.
However... You may not want to pollute your domain model with artefacts that are not relevant to your domain. More importantly, when you do it this way, it is up to Entity Framework to figure out what to call the binding table. Prior to EF6 this naming is non deterministic (see http://entityframework.codeplex.com/workitem/1677), which may mean that two different machines compiling the same code will decide on different names for that table and cause some interesting migration problems in your production system.
The answer to both problems is to always explicitly manage many-to-many relationships with Fluent Configuration.
In your Data Context class, override the OnModelCreating, something like this:
public class MyDb : DbContext
{
public IDbSet<Track> Tracks { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<Track>()
.HasMany(t => t.Categories)
.WithMany()
.Map(c => c.ToTable("CategoriesForTracks"));
}
}
If you do this, you don't need to add a navigation property to your Category class, though you still can (if you do, you should use the overload for WithMany that allows you to specify a property).
Relationships between entities and how to map that to a relational database is inherently hard. For anything other than the simplest parent-child relationships you will want to use the fluent API to make sure you actually get what you want.
Morteza Manavi has a really good blog series describing relationships in EF Code First in exhaustive detail.
NOTE
You should usually make navigation properties virtual. So, you should change your Category class like this:
public virtual ICollection<Category> Categories { get; set; }
In theory, not making it virtual should just cause eager loading rather than lazy loading to happen. In practice I have always found lots of subtle bugs appearing when my navigation properties are not virtual.

Converting between an entity and it's surrogate type

I am having some problem about how to work with an entity say an EF entity and a surrogate type, which will be bound to the UI.
Suppose that I have following classes
// Db Entity
public class Car
{
public virtual int Id { get; set; }
public string ChassisNumber { get; set; }
public virtual string Brand { get; set; }
public virtual string Name { get; set; }
}
// Surrogate type that reflects some properties of Car entity
// This class will be bound to UI
public class SurrogateCar
{
public string Brand { get; set; }
public string Name { get; set; }
}
Now I will be getting List<Car> from db and want to create a List<SurrogateCar> that represents my entities. I can do this easily in many ways, one of them like this:
List<Car> cars = CarTable.GetMyCars(); // Just a dummy method, suppose it returns all entities from Db.
List<SurrogateCar> surrogates = new List<SurrogateCar>();
foreach (var car in cars)
{
surrogates.Add(new SurrogateCar { Brand = car.Brand, Name = car.Name });
}
or I can write a custom cast method. But what I worry about is the performance. This method will be called frequently, so creating a list and populating it one by one seems a potential problem to me.
Do you have any better ways to do this, or is it okay to use it like this?
Thanks.

If you have a web service, and that service is always going to return the SurrogateCar class, then you can write your entity query to return the class you want rather than getting the class you don't want:
var cars = from c in context.Cars where {your condition}
select new SurrogateCar
{
Brand=c.Brand,
Name=c.Name
};
If, on the other hand you need the list of cars all the time, then as Roger pointed out AutoMapper is great! You just call
CreateMap<Car, SurrogateCar>
then you just use Automapper to populate your new list:
surrogates.AddRange(Map<IEnumberable<Car>, IEnumerable<SurrogateCar>>(cars));

Don't worry about the performance until you've really measured that's your bottleneck! Most probably these mappings between different types aren't that slow.
There are tools out there, eg AutoMapper
http://automapper.org/
It's main purpose isn't performance though, but to potentially makes you write easier and less code.

I believe what you are really looking for is AutoMapper, it allows for seamless, easy code written around this situation. I would not worry too much about the performance unless you need to worry about it.
Here is a SO about mapping lists using automapper, also

ASP.NET MVC: Where should this business logic go?

I'm working on my first real MVC application and I'm trying to follow general OOP best practices. I'm refactoring some simple business logic that I had in a controller into my domain model. I've been doing some reading lately and it seems pretty clear that I should put the logic somewhere in a domain model entity class in order to avoid the "anemic domain model" anti-pattern.
The application will allow people to purchase leases for parking spaces. Rates are determined by the length of the spot and whether or not the customer is a member of the business park.
So I have entity classes in my domain model that look like this (simplified):
public class Customer
{
int ID { get; set; }
string Name { get; set; }
bool IsMember { get; set; }
}
public class ParkingSpace
{
int ID { get; set; }
int Length { get; set; }
}
public class ParkingSpaceLease
{
int ID { get; set; }
DateTime OpenDate { get; set; }
DateTime CloseDate { get; set; }
Customer Customer { get; set; }
ParkingSpace ParkingSpace { get; set; }
}
Edit: Just to clarify the LeaseQuote is not an entity class as it is just used to display the cost breakdown to perspective customers and is not persisted anywhere.
public class LeaseQuote
{
int SubTotal { get; set; }
int Discount { get; set; }
int Total { get; set; }
}
Now as a feature of the application I need to be able to generate quotes for different customer and parking space combinations. The quotes will normally be accessed outside the context of actually creating a lease such as when a customer calls up to inquire about a price.
So what is the best way to go about this? Does it make sense to instantiate a new ParkingSpaceLease object inside the controller just to call a GetQuote method on it?
var lease = new ParkingSpaceLease();
var quote = lease.GetQuote(length: 168, isMember: true);
return Json(quote);
Or should the LeaseQuote class have the method?
var leaseQuote = new LeaseQuote();
var quote = leaseQuote.GetQuote(length: 168, isMember: true);
return Json(quote);
It feels strange putting the logic in the actual ParkingSpaceLease class. I guess it feels kind of "heavy" to create a new lease object when I know that I'm not going to actually do anything with it other than access the GetQuote method which seems kind of like a separate service.
So where should the GetQuote method go and why should it go there?

It almost sounds like your LeaseQuote isn't an entity and more of a business level class. I mean, you're not storing it in the database anywhere, are you? And it's not a part of another data object.
When I see this
Now as a feature of the application I need to be able to generate quotes for different customer and parking space combinations. The quotes will normally be accessed outside the context of actually creating a lease such as when a customer calls up to inquire about a price.
I think of a method signature like this
public LeaseQuote GetQuote(Customer customer, ParkingSpace parkingSpace, int length)
But with that in mind, I'd probably also want to store information about the cost of the parking space within the ParkingSpace entity and (if applicable) the customer's discount in the Customer entity.
Where would this stuff go? In a model class (business model, not LINQ or Entity model) that accesses your entities and serves as a provider for your controller.
Now I know that's not using your models exactly as written. And it could just be personal bias. But when I think about data models and data entities, they should not have any addon methods outside of what's coming back from the database. They should just represent the data unaltered as it appears in the database. If you're acting on the data, that belongs in a tier above the data entities.
Update:
What I am curious about from your example is why one would want to pass the full Entity objects (Customer and Parking Space) versus just the properties needed to perform the calculation?
It depends on your standard of code. Exposing the entity itself could be dangerous if the consuming code manipulates the entity. I prefer passing the entity mainly because that's what I'm used to. But I'm also careful not to manipulate the entity on the way in. That, and I think the method signature reflects what the GetQuote method is focused on; it's related to a customer and a parking space.
I could also make the case that if more fields go into the Entity later that can effect the GetQuote method, then the method signature doesn't have to change. In this case, only the implementation for GetQuote has to change.
Short answer: Preference.

Just make GetQuote a static method in ParkingSpaceLease.

I think you may have your object model slightly askew, which would lead to your concern about the lease being the wrong place from which to get a quote. It seems to me that a lease would be wholly composed by the parking space which is being leased, and would be related only to the customer purchasing the lease. IOW:
public class ParkingSpace
{
int ID { get; set; }
int Length { get; set; }
IEnumerable<ParkingSpaceLease> Leases { get; set; }
LeaseQuote GetQuote(Customer customer/*, other relevant parameters */) { ... }
}
public class ParkingSpaceLease
{
int ID { get; set; }
DateTime OpenDate { get; set; }
DateTime CloseDate { get; set; }
Customer Customer { get; set; }
}
public class LeaseQuote
{
//Properties
ParkingSpaceLease GetLease();
}
EDIT I missed the part about the LeaseQuote being a separate class.

How do you break circular associations between entities?

my first time on the site so apologies if it's tagged incorrectly or been answered elsewhere...
I keep running into particular situation on my current project and I was wondering how you guys would deal with it. The pattern is: a parent with a collection of children, and the parent has one or more references to particular items in the child collection, normally the 'default' child.
A more concrete example:
public class SystemMenu
{
public IList<MenuItem> Items { get; private set; }
public MenuItem DefaultItem { get; set; }
}
public class MenuItem
{
public SystemMenu Parent { get; set; }
public string Name { get; set; }
}
To me this seems like a good clean way of modelling the relationship, but causes problems immediately thanks to the circular association, I can't enforce the relationship in the DB because of the circular foreign keys, and LINQ to SQL blows up due to the cyclic association. Even if I could bodge my way round this, it's clearly not a great idea.
My only idea currently is to have an 'IsDefault' flag on MenuItem:
public class SystemMenu
{
public IList<MenuItem> Items { get; private set; }
public MenuItem DefaultItem
{
get
{
return Items.Single(x => x.IsDefault);
}
set
{
DefaultItem.IsDefault = false;
value.DefaultItem = true;
}
}
}
public class MenuItem
{
public SystemMenu Parent { get; set; }
public string Name { get; set; }
public bool IsDefault { get; set; }
}
Has anyone dealt with something similar and could offer some advice?
Cheers!
Edit: Thanks for the responses so far, perhaps the 'Menu' example wasn't brilliant though, I was trying to think of something representative so I didn't have to go into the specifics of our not-so-self-explanatory domain model! Perhaps a better example would be a Company/Employee relationship:
public class Company
{
public string Name { get; set; }
public IList<Employee> Employees { get; private set; }
public Employee ContactPerson { get; set; }
}
public class Employee
{
public Company EmployedBy { get; set; }
public string FullName { get; set; }
}
The Employee would definitely need a reference to their Company, and each Company could only have one ContactPerson. Hope this makes my original point a bit clearer!

The trick to solving this is to realize that the parent does not need to know about all of the methods of the child, and that the child does not need to know all the methods of the parent. Therefore you can use the Interface Segregation Principle to decouple them.
In short, you create an interface for the parent that has only those methods that the child needs. You also create an interface for the child that has only those methods that the parent needs. Then you have the parent contain a list of the child interfaces, and you have the child point back to the parent interface. I call this the Flip Flob Pattern because the UML diagram has the geometry of an Eckles-Jordan flip-flop (Sue me, I'm an old hardware engineer!)
|ISystemMenu|<-+ +->|IMenuItem|
A 1 \ / * A
| \/ |
| /\ |
| / \ |
| / \ |
| / \ |
|SystemMenu| |MenuItem|
Notice that there is not cycle in this diagram. You cannot start at one class and follow the arrows back to your starting point.
Sometimes, in order to get the separation just right, you have to move some methods around. There might be code that you thought should have been in the SystemMenu that you move to the MenuItem, etc. But in general the technique works well.

Your solution seems quite reasonable.
Another thing to think about is that your objects in memory don't have to exactly match the database schema. In the database you can have the simpler schema with the child properties, but in memory you can optimize things and have the parent with references to the child objects.

I don't really see your problem. Clearly you're using C#, which holds objects as references not instances. This means it's perfectly fine to have cross-referencing, or even self-referencing.
in C++ and other languages where objects are more compositied then you can have problems, which are typically solved using references or pointers, but C# should be fine.
More than likely your problem is that you're trying to follow all references somehow, leading to a circular reference. LINQ uses lazy loading to address this issue. For instance, LINQ won't load the Company or the Employee until you reference it. You just need to avoid following such references further than one level.
However, you can't really add two tables as each others foreign key, otherwise you would never be able to delete any record, since deleting an employee would require deleting the company first, but you can't delete the company without deleting the employee. Typically, in this case, you would only use one as a real foreign key, the other would simply be a psuedo-FK (that is, one that is used as an FK but doesn't have constraints enabled). You have to decide which is the more important relationship.
In the company example, you would likely want to delete the employee but not the company, so make the company->employee FK the constraint relationship. This prevents you from deleting the company if there are employees, but you can delete employees without deleting the company.
Also, avoid creating new objects in the constructor in these situations. For instance, if your Employee object creates a new Company object, which includes a new employee ojbect created for the employee, it will eventually exhaust memory. Instead, pass the objects already created to the constructor, or set them after construction, possibly by using an initalization method.
For instance:
Company c = GetCompany("ACME Widgets");
c.AddEmployee(new Employee("Bill"));
then, in AddEmployee, you set the company
public void AddEmployee(Employee e)
{
Employees.Add(e);
e.Company = this;
}

Maybe a self-referential GoF Composite pattern is an order here. A Menu has a collection of leaf MenuItems, and both have a common interface. That way you can compose a Menu out of Menus and/or MenuItems. The schema has a table with a foreign key that points back to its own primary key. Works with walking menus that way, too.

In code, you need to have references both ways to reference things both ways. But in the database, you only need the reference one way to make things work. Because of the way joins work, you only need to have the foreign key in one of your tables. When you think about it, every foreign key in your database could be flipped around, and create and create a circular reference. Best to just pick one record, in this case probably the child with a foreign key to the parent, and just be done.

In a domain driven design sense way, you can choose to avoid bidirectional relations between entities where it's possible. Choose one "aggregate root" to hold the relations, and use the other entity only when navigation from the aggregate root. I try to avoid bidirectional relations where it's possible. Because of YAGNI, and it will make you ask the question "what was first, the chicken or the egg?" Sometimes you will still need bidirectional associations, then choose one of the solutions mentioned earlier.
/// This is the aggregate root
public class Company
{
public string Name { get; set; }
public IList<Employee> Employees { get; private set; }
public Employee ContactPerson { get; set; }
}
/// This isn't
public class Employee
{
public string FullName { get; set; }
}

You can enforce foreign keys in the database where two tables refer to each other. Two ways come to mind:
The default child column in the parent is initially null and is only updated once all the child rows have been inserted.
You defer constraint checking until commit time. This means you can insert first the parent with an initially broken reference to the child, then insert the child. One problem with deferred constraint checking is that you can end up with database exceptions being thrown at commit time which is often inconvenient in many db frameworks. Also, it means you need to know the primary key of the child before you insert it which may be awkward in your setup.
I've assumed here that the parent menu item lives in one table and the child in a different table but the same solution would work if they are both in the same table.
Many DBMS's support deferred constraint checking. Possibly yours does too although you don't mention which DBMS you are using

Thanks to all who answered, some really interesting approaches! In the end I had to get something done in a big hurry so this is what I came up with:
Introduced a third entity called WellKnownContact and corresponding WellKnownContactType enum:
public class Company
{
public string Name { get; set; }
public IList<Employee> Employees { get; private set; }
private IList<WellKnownEmployee> WellKnownEmployees { get; private set; }
public Employee ContactPerson
{
get
{
return WellKnownEmployees.SingleOrDefault(x => x.Type == WellKnownEmployeeType.ContactPerson);
}
set
{
if (ContactPerson != null)
{
// Remove existing WellKnownContact of type ContactPerson
}
// Add new WellKnownContact of type ContactPerson
}
}
}
public class Employee
{
public Company EmployedBy { get; set; }
public string FullName { get; set; }
}
public class WellKnownEmployee
{
public Company Company { get; set; }
public Employee Employee { get; set; }
public WellKnownEmployeeType Type { get; set; }
}
public enum WellKnownEmployeeType
{
Uninitialised,
ContactPerson
}
It feels a little cumbersome but gets around the circular reference issue, and maps cleanly onto the DB which saves trying to get LINQ to SQL to do anything too clever! Also allows for multiple types of 'well known contacts' which is definitely coming in the next sprint (so not really YAGNI!).
Interestingly, once I came up with the contrived Company/Employee example it made it MUCH easier to think about, in contrast to the fairly abstract entities that we're really dealing with.