Develop Reference

Calculated fields that improve performance but need to be maintained (EF)

I have this "1 to N" model:
class Reception
{
public int ReceptionId { get; set; }
public string Code { get; set; }
public virtual List<Item> Items { get; set; }
}
class Item
{
public int ItemId { get; set; }
public string Code { get; set; }
public int Quantity { get; set; }
public int ReceptionId { get; set; }
public virtual Reception Reception { get; set; }
}
And this action, api/receptions/list
public JsonResult List()
{
return dbContext.Receptions
.Select(e => new
{
code = e.Code,
itemsCount = e.Items.Count,
quantity = e.Items.Sum(i => i.Quantity)
}).ToList();
}
which returns a list of receptions, with their number of items:
[
{code:"1231",itemsCount:10,quantity:30},
{code:"1232",itemsCount:5,quantity:70},
{code:"1234",itemsCount:30,quantity:600},
...
]
This was working fine but I'm having too many Reception's and Item's thus the query is taking too long...
So I want to speed up by adding some persisted fields to Reception:
class Reception
{
public int ReceptionId { get; set; }
public string Code { get; set; }
public virtual List<Item> Items { get; set; }
public int ItemsCount { get; set; } // Persisted
public int Quantity { get; set; } // Persisted
}
With this change, the query ends up being this:
public JsonResult List()
{
return dbContext.Receptions
.Select(e => new
{
code = e.Code,
itemsCount = e.ItemsCount,
quantity = e.Quantity
}).ToList();
}
My question is:
What's the best way to maintain these two fields?
I will gain in performance but now I will need to be more careful with the creation of Item's
Today an Item can be created, edited and deleted:
api/items/create?receptionId=...
api/items/edit?itemId=...
api/items/delete?itemId=...
I also have a tool for importing receptions via Excel:
api/items/createBulk?...
Maybe tomorrow I will have more ways of creating Item's, so the question is how do I make sure that these two new fields, ItemsCount and Quantity will be up to date always?
Should I create a method within Reception like this?
class Reception
{
...
public void UpdateMaintainedFields()
{
this.Quantity = this.Items.Sum(e => e.Quantity);
this.ItemsCount = this.Items.Count();
}
}
And then REMEMBER to call it from all the previous URL's? (items/create, items/edit, ...)
Or maybe should I have a stored procedure in the database?
What is the common practice? I know there are calculated columns but these refer to fields of the same class. Also there are indexed views, but I'm not sure if they apply well to scenarios like this.

From your code it seems to me that you do not have a layer for business logic, and everything is implemented in the controllers, this causes the problem for you that when you would have a different way (and it seems, that you mean a different controller) you have to implement this logic again and it is easy to forget, and if you do not forget, you could forget to maintain later.
So I would recommend to have a layer for business logic (like adding new items) and use it from the controllers where you want to create items.
I would also recommend write the function UpdateMaintainedFields as you asked, but call it in the business logic layer after adding the items, not in the controllers!
You could write the logic on the database also (trigger) if you can accept that you can't write unit test.

Assuming the original query cannot be improved with the correct execution plan in SQLServer, the way to update these fields is via a trigger in the DB. When an insert occurs (or possible an update if your persisted fields change according to the data) then when an insert occurs to that table, the trigger is run. It would be responsible for updating all the rows with the new values.
Obviously your insert performance would drop, but your query performance would be that of a simple index and read of a single row. Obviously you wouldn't be able to use this trick if you were to return a subset of the table, as all the quantities would be fixed.
An alternative is to hold the count and quantity sums in a separate table, or in a dummy row that holds the summed quantities as its entry for quantity. YMMV.
PS I hate how what is a SQL question has been turned in one about C# code! Learn SQL and run the queries you need directly in the DB, that will show you much more about the performance and structure of what you're looking for than getting EF involved. /rant :)

You want to store the same information duplicitly, which can lead to inconsistencies. As an inspiration, indexes are also duplicating data. How do you update them? You don't. It is all fully transparent. And I would recommend the same approach here.
Make sum table, maintained by triggers. The table would not be included in any datacontext schema, only way to read it would be through non updateable views or stored procedures. Its name should evoke, that nobody should ever touch this table directly.
You can now access your data from various frameworks and do not worry about updating anything. Database would assure the precalculated sums are always correct, as long as you do not write to the sum table on your own. In fact you can add or remove this table any time and no application would even notice.

How do I best bind query results to an in-memory graph, rather than to a flat collection, with Neo4jClient?

I'm just starting to dabble in Neo4j and I haven't been able to find a good code sample that implements this simple use case.
I know how to fetch all nodes in an n-level relationship with a given node -- as a flat one-dimensional array or list of objects.
But what is the most elegant way to maintain the graph structure in memory? What's the best way to bind to a simple class like this:
public class Foo
{
public int Id { get; set; }
public string Name { get; set; }
public IEnumerable<Foo> Relationships { get; set; }
}
I can think of all sorts of cheesy hacks such as getting all objects, including relationships, from the db as a flat list -- and then re-creating the in-memory graph "manually" on the client... But please tell me there's a better way.

How to create dynamic Linq Select Expression with anonymous objects

Am using Entity Framework to run a query on a table. However, i need to get select columns only.
class MyEvent
{
public string Name { get; set; }
public int Id { get; set; }
virtual Stage EventStage { get; set; }
..... more columns .....
}
class Stage
{
public string Name { get; set; }
public string Location { get; set; }
..... more columns .....
}
I can write an IQueryable to return these as
dbContext.MyEvents
.Select(s =>
new {
Name = s.Name,
Id = s.Id,
EventStage = new
{
Name = s.EventStage.Name,
Id = s.EventStage.Id
}
}
)
.ToList();
This works as expected, giving me just those columns am interested in.
Now, I need to construct that 'Select' call dynamically using Expression tree, something like here.
How can I achieve that? Is it feasible to construct an anynomous object, like above, via expressions?
EDIT:
The use case for me is that I have a generic dB context class which takes a list of columns as strings to be fetched. In the past, we were returning all columns, ignoring that input list. So, now I need to dynamically generate the select statement to return only the required subset of columns, which can either be done via anonymous object or a dynamically created DTO.
Thanks

Maybe you can use something like the ToDynamic method from here:
https://gist.github.com/volak/20f453de023ff75edeb8
A possible usecase for this problem:
Let the user select the columns to display and query only those selected columns, so you don't query always the whole entity from the database.

Define a strongly typed object and return that. I would avoid using a dynamic object.
Note: you can't return an anonymous object.

Converting between an entity and it's surrogate type

I am having some problem about how to work with an entity say an EF entity and a surrogate type, which will be bound to the UI.
Suppose that I have following classes
// Db Entity
public class Car
{
public virtual int Id { get; set; }
public string ChassisNumber { get; set; }
public virtual string Brand { get; set; }
public virtual string Name { get; set; }
}
// Surrogate type that reflects some properties of Car entity
// This class will be bound to UI
public class SurrogateCar
{
public string Brand { get; set; }
public string Name { get; set; }
}
Now I will be getting List<Car> from db and want to create a List<SurrogateCar> that represents my entities. I can do this easily in many ways, one of them like this:
List<Car> cars = CarTable.GetMyCars(); // Just a dummy method, suppose it returns all entities from Db.
List<SurrogateCar> surrogates = new List<SurrogateCar>();
foreach (var car in cars)
{
surrogates.Add(new SurrogateCar { Brand = car.Brand, Name = car.Name });
}
or I can write a custom cast method. But what I worry about is the performance. This method will be called frequently, so creating a list and populating it one by one seems a potential problem to me.
Do you have any better ways to do this, or is it okay to use it like this?
Thanks.

If you have a web service, and that service is always going to return the SurrogateCar class, then you can write your entity query to return the class you want rather than getting the class you don't want:
var cars = from c in context.Cars where {your condition}
select new SurrogateCar
{
Brand=c.Brand,
Name=c.Name
};
If, on the other hand you need the list of cars all the time, then as Roger pointed out AutoMapper is great! You just call
CreateMap<Car, SurrogateCar>
then you just use Automapper to populate your new list:
surrogates.AddRange(Map<IEnumberable<Car>, IEnumerable<SurrogateCar>>(cars));

Don't worry about the performance until you've really measured that's your bottleneck! Most probably these mappings between different types aren't that slow.
There are tools out there, eg AutoMapper
http://automapper.org/
It's main purpose isn't performance though, but to potentially makes you write easier and less code.

I believe what you are really looking for is AutoMapper, it allows for seamless, easy code written around this situation. I would not worry too much about the performance unless you need to worry about it.
Here is a SO about mapping lists using automapper, also

Dynamic Expression for Ordering Child Collections with Entity Framework

I am new to EF. I am trying to get Entity Framework 4.2 to do a sort by a calculated property (not mapped).
Here is what my entity look like:
public class Site : Entity
{
public Site()
{
Equipments = new HashSet<Equipment>();
Forecasts = new HashSet<Forecast>();
}
[StringLength(8)]
public string Number { get; set; }
[StringLength(50)]
public string EquipmentShortCLLI { get; set; }
[StringLength(50)]
public string Location { get; set; }
public virtual Central Central { get; set; }
public virtual ICollection<Equipment> Equipments { get; set; }
public virtual ICollection<Forecast> Forecasts { get; set; }
#region Calculated Items
public bool IsEmbargo {
get { return Equipments.Count > 0 && Equipments.SelectMany(x => x.EquipmentDetails).Any(e => e.IsEmbargo); }
}
//...
public int PortsCapacity
{
get
{
return Equipments.Count > 0
? Equipments.SelectMany(x => x.Slots).Sum(x => x.PortsCapacity)
: 0;
}
}
#endregion
//...
By trying to order using any of my readonly properties I am getting the exception:
The specified type member 'PortsCapacity' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.
Which makes sense because EF is trying to build an sql orderby with a field that does not exist in the database (my understanding..).
Now, by using some dynamic linq code I was able to make this work for my many-to-one columns by passing "Central.SomeField" (as opposed to making a ReadOnly Property that returns Central.SomeField).
I.E.:
query.OrderBy("Central.SomeField");
However, I still face the same issue when it comes to a collection of items (Equipments). I am trying to make this as dynamic as possible by using a string coming from the client side and avoiding a long switch case, but at this point I will accept any ideas, so long as the sorting happens on the database side.
Edit 1:
Following what Ladislav Mrnka says, how would one execute an OrderBy clause on one-to-many child items using lambdas or expression?

I don't think that Dynamic Linq is capable of this. You need a real Linq subquery to compute aggregations on Equipements so it will simply not work. If the user selects ordering by IsEmbargo or PortsCapacity you must have some switch / if block to handle this case by appending special part of the query - no other way.