Develop Reference

Filtering/Sorting computed fields in Hot Chocolate

In my application I have the following DTO object which retrieves data, via EF Core, from SQL and computes a certain field:
public class MyDTO
{
public string MyDTOProperty { get; set ; }
public string MyDTOComputedField(){
...
}
}
My query method looks like:
public class MyQueries
{
...
[UseDbContext(typeof(ApiDbContext))]
[UseFiltering(typeof(MyFilter))]
[UseSorting]
public IQueryable<MyDTO> GetObject([ScopedService] ApiDbContext context){
var query = context.MyDB;
return query.Select(fea => new MyDTO(){
MyDTOProperty = fea.property
});
}
}
Filtering and sorting only seems to work on the properties with get and set method. My question is, how can I enable filtering and sorting on my computed fields such that the following GraphQL query would be possible:
{
Object(where: {MyDTOComputedField: {contains: "someSubString"}}, order: {MyDTOComputedField: ASC}){
MyDTOProperty
MyDTOComputedField
}
}
I already tried with defining my own filtering/sorting middleware, without any luck so far.

TL;DR; You are using Custom Resolver (HC feature), not Computed Column (T-SQL feature), which could not be translated to SQL by Entity Framework.
First thing first, this is not a Hot Chocolate problem, but Entity Framework problem.
[UseFiltering]
Use filtering is not a magic nor golden bullet. It is only middleware, which will generate argument where for your endpoint and then, at runtime, it will take this argument (in your case {MyDTOComputedField: {contains: "someSubString"}}), make Linq Expression from it and return input.Where(Expression).
And thats pretty much it.
(Of course, if you ever wrote string -> linq expression piece of code then you know, its not THAT simple, but good folks from HC did exactly that for us :) )
Something like
System.Linq.Expression<Func<MyDTO, bool>> where =
myDto => myDto.MyDTOComputedField.Contains("someSubString");
return input.Where(where);
(remember, every middleware in HC is just a piece of pipe - it have input, some process and output. Thats why order of middlewares matters. Btw, same with "order by", but it will return input.OrderBy(expression))
Now, because input is DbSet<MyDTO>, then nothing is executed "right away" but lazily - real work is done by Entity Framework - it take linq Expresion (.Where().Sort()), translate it to T-SQL and send it as query.
And there is your problem: Your MyDTO.MyDTOComputedField is not translateable to SQL.
Why its not translateable?
Because your MyDTOComputedField is not "computed column" but "custom resolver". It exists only in your app and SQL have no idea what it should contains. Maybe it is something trivial as a + b * 42 (then computed column would be great!) but maybe it is request to another server REST api (why not :) ) - we dont know.
Then why not execute part of query on server and rest locally?
Because this scale reeeeeeeeally badly. You did not show us implementation of MyDTO.MyDTOComputedField, so let assume it do something trivial. Like cast((a + b * 42) as nvarchar(max));. Meaning, it will always be some int but casted as nvarchar. Meaning, if you ask for Contains("someSubString") it will always have 0 results.
Ok, now imagine, your MyDTO table (btw, I expect MyDTO to be EF model even with DataTransferObject in name...) have 10.000.000 rows (in enterprise scale app its bussiness as usual :) ).
Because you are sane person (and because it will make this example much better to understand :) ), you add pagination. Lets say 100 items per page.
In this example, you expect EF to do select top 100 * from MyDto where MyDTOComputedField like '%someSubString%'.
But thats not gonna happen - sql have no idea what MyDTOComputedField is.
So it have two options, both bad: It will execute select top 100, then do filter locally - but there is zero result. So it will take another 100 and another 100 and another and another and (10.000.000/100 = 100.000 select query!) only to found that there is 0 result.
Another possible solution is, when EF found that some part of expression have to be executed locally, it will execute locally whole query. So it will select, fetch, materialise 10.000.000 entities in one go and THEN it will filter them just to see there is 0 result. Somehow better, but still bad.
You just DDoS yourself.
Btw, Option 2 was what Entity Framework before core (Classic?) did. And it was source of soooo much bugs, when you accidentally fetched whole table, that good folks from EF team drop support for it and now they throw
"The LINQ expression 'DbSet()\n .Where(f => new MyDTO{ \r\n id = f.i, \r\n }\r\n.MyDTOProperty == __p_3' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See go.microsoft.com/fwlink/?linkid=2101038 for more information."
Ok... But what to do?
I know there will never be many rows.
Maybe MyDTO is just some list which will never explode (lets say, for example, VAT rates - there is pretty much Standard, Zero, Reduced + some states have some more. So we hardly look at table greater then ~5 rows, more when your app is international, but still - a few.)
Then you dont have to be afraid of local execution. Just add ".ToArray()" or ".ToList()" on the end of your DbSet call. As your Exception told you.
But be aware that this can really bite you later, if not done carefully.
Computed Column
If your implementation of MyDTOComputedField is trivial, you can move it to database. Set EF ComputedColumn, do migration, drop your resolver and you are ready to go.
Database View
Another possible option is to make database view.
This is more robust solution then Computed Column (at least, you can optimalise your view really well (custom index(es), better joins, no inner query etc...), but it take more work & you have to know what you are doing. AFAIK EF cant generate view for you, you have to write it by hand.
Just make empty migration, add your view, EF entity (make sure to use ToView() and not ToTable()), drop your resolver and you are ready to go.
In both cases, your query (dto?) model will be different from mutation (domain?) model, but thats ok - you really do not want to let consumer of your api to even try to mutate your MyDTOComputedField anyway.
Its not possible to translate it to SQL
Maybe your custom resolver do something not really under your control / not doable in sql (= not doable in EF). For example, http call to another server. Then, its up to you to do it right within your business logic. Maybe add custom query argument. Maybe write your own implementation of [UseFiltering] (its not THAT hard - HotChocolate is open source with great licencing, so you can basically go and [ctrl] + [c] [ctrl] + [v] current implementation and add what you need to add.)
I can't advise you, i dont know your bussiness requirement for MyDTOComputedField.

Mark some child entities as "Never Load" in LINQ to Entities query

TL;DR
I want to write a query in LINQ to Entities and tell it that I'll never load the child entities of an entity. How do I do that without projecting?
Eg,
return (from a in this.Db.Assets
join at in this.Db.AssetTypes on a.AssetTypeId equals at.AssetTypeId
join ast in this.Db.AssetStatuses on a.AssetStatusId equals ast.AssetStatusId
select new {
a = a,
typeDesc = at.AssetTypeDesc,
statusDesc = ast.AssetStatusDesc
}).ToList().Select(anon => new AssetViewModel(anon.a, anon.typeDesc, anon.statusDesc)).ToList();
I want the entity called Asset pulled into a on the anonymous type I'm defining, and when I call ToList(), I don't want the Assets' children, Status and Type, to lazy load.
EDIT: After some random Visual Studio autcomplete investigation, much of this can be accomplished by turning off lazy loading in the DbContext:
this.Db.Configuration.LazyLoadingEnabled = false;
Unfortunately, if your work with the query results does have a few child tables, even with LazyLoadingEnabled turned off, things may still "work" for some subset of them iff the data for those children has already been loaded earlier in this DbContext -- that is, if those children have already had their context cached -- which can make for some surprising and temporarily confusing results.
That is to say, I want to explicitly load some children at query time and completely sever any relationship to other child entities.
Best would be some way to actively load some entities and to ignore the rest. That is, I could call ToList() and not have to worry about throwing off lots of db connections.
Context
I have a case where I'm hydrating a view model with the results of a LINQ to Entities query from an entity called Asset. The Asset table has a couple of child tables, Type and Status. Both Type and Status have Description fields, and my view model contains both descriptions in it. Let's pretend that's as complicated as this query gets.
So I'd like to pull everything from the Asset table joined to Type and Status in one database query, during which I pull the Type and Status descriptions. In other words, I don't want to lazy load that info.
WET (Woeful Entity reTranscription?)
What we're doing now, which does exactly what I want from a connection standpoint, is the usual .Select into the view model, with a tedious field matchup.
return (from a in this.Db.Assets
join at in this.Db.AssetTypes on a.AssetTypeId equals at.AssetTypeId
join ast in this.Db.AssetStatuses on a.AssetStatusId equals ast.AssetStatusId
select new AssetViewModel
{
AssetId = a.AssetId,
// *** LOTS of fields from Asset removed ***
AssetStatusDesc = ast.AssetStatusDesc,
AssetTypeDesc = at.AssetTypeDesc
}).ToList();
That's good in that the Status and Type child entities of Asset are never accessed, and there's no lazy load. The SQL is one join in one database hit for all the assets. Perfect.
The worry is all the repeated jive in // *** LOTS of fields from Asset removed ***. Currently, we've got that projection in every freakin query, which obviously isn't DRY. And it means that when the Asset table changes, it's rare that the new field is included in every projection (because humans), which stinks.
I don't see a quick way around the query, btw. If I want to do it in a single query, I have to have the joins. I could add wheres to it in separate methods, but I'm not sure how I'd skip the projection each time. Or I could add joins to the query in cascading methods, but then my projection is still "repository bound", which isn't best case if I'm using these sorts of queries elsewhere. But I'm betting I'm stupiding something here.
Dumb
When I tried adding a cast to my view model from asset and changing to something like this, which is beautiful from a code standpoint, though I get bitten by lazy loading -- two extra database hits per Asset, one for Status and one for Type.
return (from a in this.Db.Assets
select a).ToList().Select(asset => (AssetViewModel)asset).ToList();
Just as we would expect, since I'm using lines like...
AssetTypeDesc = a.AssetType.AssetTypeDesc,
... inside of the casting code. So that was dumb. Concise, reusable, but dumb. This is why we hate folks who use ORMs without checking the SQL. ;^)
Overly clever, sorta
But then I tried getting too clever, with a new constructor for the view model that took the asset entity & the two description values as strings, which ended up with the same lazy load issue (because, duh, the first ToList() before selecting the anonymous objects means we don't know how the Assets are going to be used, and we're stuck pulling back everything to be safe (I assume)).
//Use anon type to skirt "Only parameterless constructors
//and initializers are supported in LINQ to Entities,"
//issue.
return (from a in this.Db.Assets
join at in this.Db.AssetTypes on a.AssetTypeId equals at.AssetTypeId
join ast in this.Db.AssetStatuses on a.AssetStatusId equals ast.AssetStatusId
select new {
a = a,
typeDesc = at.AssetTypeDesc,
statusDesc = ast.AssetStatusDesc
}).ToList().Select(anon => new AssetViewModel(anon.a, anon.typeDesc, anon.statusDesc)).ToList();
If only there was some way to say, "cast these anonymous objects to a List, but don't lazy load the Asset's children while you're doing it." <<< That's my question, natch.
I've read some about DataLoadOptions.LoadWith(), which probably provides an okay solution, and I might end up just doing that, but that's not precisely what I'm asking. I think that's a global-esque setting (? I think just for the life of the data context, which should be the single controller interaction), which I might not necessarily want to set. I may also want ObjectTrackingEnabled = false, but I'm not grokking yet.
I also don't want to use an automapper.

Painfully, after some random Visual Studio autcomplete investigation, this might be as easy as turning off lazy loading in your DbContext:
this.Db.Configuration.LazyLoadingEnabled = false;
The wacky thing is that if your work with the query results does have a few child tables, even with LazyLoadingEnabled turned off, things may still "work" for some subset of them iff the data for those children has already been loaded earlier in this DbContext -- that is, if those children have already had their context cached -- which can make for some surprising and temporarily confusing results.
Better would be to be able to cherry pick what children are "lazy-loading eligible".
I may need to update the question to make it cover this variation of the original question.

Queryable Linq Query Differences In Entity Framework

I have a very simple many to many table in entity framework connecting my approvals to my transactions (shown below).
I am trying to do a query inside the approval object to count the amount of transactions on the approval, which should be relatively easy.
If I do something like this then it works super fast.
int count;
EntitiesContainer dbContext = new EntitiesContainer ();
var aCnt = from a in dbContext.Approvals
where a.id == id
select a.Transactions.Count;
count = aCnt.First();
However when I do this
count = Transactions.Count;
or this
count = Transactions.AsQueryable<Transaction>().Count();
its exceedingly slow. I have traced the sql running on the server and it does indeed seem to be trying to load in all the transactions instead of just doing the COUNT query on the collection of Transactions.
Can anyone explain to me why?
Additional :
Here is how the EF model looks in regards to these two classes
UPDATE :
Thanks for all the responses, I believe where I was going wrong was to believe that the collections attached to the Approval object would execute as IQueryable. I'm going to have to execute the count against the dbContext object.
Thanks everyone.

var aCnt = from a in dbContext.Approvals
where a.id == id
select a.Transactions.Count;
EF compiles query by itself, the above query will be compiled as select count transactions
Unlike,
count = Transactions.AsQueryable<Transaction>().Count();
count = Transactions.Count;
these will select all the records from transaction and then computes the count

When you access the a.Transactions property, then you load the list of transactions (lazy loading). If you want to get the Count only, then use something like this:
dbContext.Transactions.Where(t => t.Approvals.Any(ap => ap.Id == a.Id)).Count();
where a is given Approval.

Your first method allows the counting to take place on the database server level. It will ask the database not to return the records, but to return the amount of records found. This is the most efficient method.
This is not to say that other methods can't work as efficiently, but with the other two lines, you are not making it clear in the first place that you are retrieving transactions from a join on Approvals. Instead, in the other two lines, you take the Transactions collection just by itself and do a count on that, basically forcing the collection to be filled so it can be counted.

Your first snippet causes a query to be executed on the database server. It works that because the IQueryable instance is of type ObjectQuery provided by the Entity Framework which performs the necessary translation to SQL and then execution.
The second snippet illustrates working with IEnumerable instances. Count() works on them by, in worst case, enumerating the entire collection.
In the third snippet you attempt to make the IEnumerable an IQueryable again. But the Enumerable.AsQueryable method has no way of knowing that the IEnumerable it is getting "came" from Entity Framework. The best it can do is to wrap the IEnumerable in a EnumerableQuery instance which simply dynamically compiles the expression trees given to all LINQ query operators and executes them in memory.
If you need the count to be calculated by the database server, you can either formulate the requisite query manually (that is, write what you already did in snippet one), or use the method CreateSourceQuery available to you if you're not using Code First. Note that it will really be executed on the database server, so if you have modified the collection and have not yet saved changes, the result will be different to what would be returned by calling Count directly.

Data binding issues

I have started using the Entity Framework quite recently and think it is very good but I am a bit confused over a couple of things.
I'm writing a winforms-application where you are presented a list of persons in a list and if you click a particular person more information about that person appears in textboxes. Nothing fancy so far, but you are supposed to be able to edit the data about the person so data binding is nice here.
It works just fine but I am bit confused about what is the correct way to do it. First I did like this:
var query = _context.Person.Where(c => c.Id == chosenId);
this.personBindingSource.DataSource = query.ToList();
Then I read a bit and tried:
var local = _context.Person.Local;
IEnumerable<Customer> enumerable = local.Where(c => c.Id == chosenId);
this.personBindingSource.DataSource = enumerable.ToList();
This one seems to work fine as well.
Then I saw that someone suggested something like:
_context.Person.Load();
this.personBindingSource.DataSource = _context.Person.Local.ToBindingList();
I am a bit confused right now on what approach is the correct one and what is the difference between these three? Anyone can help me?

Depends on What You Want to do
I honestly never liked getting this answer because it seems to be the answer to everything but in this case it is really the only answer I can give
What the Local Property Does
Local gives you a reference to the current elements being tracked by the data context that hasn't been marked by delete, essentially you are asking for the current object that have already been loaded into memory by the context, you can read more about it here DbSet(TEntity).Local Property
What The Load Method Does
The Load method eagerly loads the targeted context, you can read more about it here.
DbExtensions.Load Method
What ToBindingList Does
Basically this is creating a two way binding between whatever entity you have created and the UI when you use a collection created using this method. That is that any changes in the UI should be automatically reflected in the related entities within this collection. You can read more about it using the following links
BindingList(T) Class DbExtensions.ToBindingList()
What Each Of Your Examples Do
First Example
var query = _context.Person.Where(c => c.Id == chosenId);
this.personBindingSource.DataSource = query.ToList();
Under the covers the following is going on
Creating a Query to be processed by the server with your linq expressionGetting the content from the database and creating a list around it
Here you are grabbing any of the people with Id of chosen Id from the database and then loading them into your application and creating a list
Second Example
var local = _context.Person.Local;
IEnumerable<Customer> enumerable = local.Where(c => c.Id == chosenId);
this.personBindingSource.DataSource = enumerable.ToList();
Under the covers the following is going on
Getting all of the currently tracked objects by the context object that have been hit by a query but have not been marked as deletedgetting all of the elements in local memory that have the Id chosen Id
Here you are grabbing any people that have already been loaded into the context this is not going to get all of your persisted data items, you must have hit them in other queries
Third Example
_context.Person.Load();
this.personBindingSource.DataSource = _context.Person.Local.ToBindingList();
Under the covers the following is going on
You are loading all of the people into local memoryYou create binding list (allows two way data binding between the objects and the UI) You bind the list to the UI element personBindingSource
Unless you want to load all of the items into memory this is probably not what you want to do, if the dataset ever grows large enough it will slow your program down and could possibly cause it to not work correctly (unlikely with case of person in most scenarios but is possible)
When You Should Use Them
FirstWhen you want to just get the data that matches your query into local memory and don't need a link between UI and the entitiesSecondWhen you have already run a query in the context and need to use it someplace else but don't need to rerun the query since it is currently in memoryThirdWhen you want to load all of the elements of an entity set into memory and create a two way databinding between them and a control

The only difference between the first and second example you have there is one is deferred and one is not. The third one is a little different as it creates a two way binding between your database and control datasource, effectively making the datacontext track changes to the list for you (add, & deletes).
In all your examples, so long as you keep you datacontext open changes to the objects themselves would be tracked.
As far as which way is correct, that depends on your application. Pick the approach that works best for you based on what you are trying to accomplish

How do I make Entity Framework not join tables?

Ok so I am testing out EF once again for performance and I just want a simple result back from my database.
Example
var jobsList = from j in mf.Jobs
where j.UserID == 1001 select new { Job = j };
This unfortunately joins my User object to this list, which I don't want EF to do. How do I tell EF not to join just because there is a relationship. Basically I just want a simple row from that table.
Or do I need to use a different type of retrieval. I am still using the basic type of database retrieval below and I feel there are better ways to handle db work by now.
SqlConnection myconnection = new SqlConnection();
Edit
Basically what I am saying in a more clearer context. Is that instead of only getting the following.
Job.JobID
Job.UserID
//Extra properties
I Get
Job.JobID
Job.UserID
Job.User
//Extra properties
That User object easily consumes way more memory than is needed, plus I don't need it.
My Solution
So I am still not believing in EF too much and here is why. I turned off LazyLoading and turned it on and didn't really notice too much of a performance difference there. I then compared the amount of data that my SqlConnection type method uses compared to my EF method.
I get back the exact same result set and here are the performance differences.
For my Entity Framework method I get back a list of jobs.
MyDataEntities mf = new MyDataEntities(); // 4MB for the connection...really?
mf.ContextOptions.LazyLoadingEnabled = false;
// 9MB for the list below
var test = from j in mf.Jobs
where j.UserID == 1031
select j;
foreach (Job job in test) {
Console.WriteLine(job.JobID);
}
For my SqlConnection method that executes a Stored Proc and returns a result set.
//356 KB for the connection and the EXACT same list.
List<MyCustomDataSource.Jobs> myJobs = MyCustomDataSource.Jobs.GetJobs(1031);
I fully understand that Entity Framework is doing way way more than a standard SqlConnection, but why all this hype if it is going to take at a minimum of 25x more memory for a result set. Just doesn't seem worth it.
My solution is not to go with EF after all.

The User property is part of the job class but wont be loaded until you access it (lazy loading). So it is not actually "joined".
If you only want the two specified columns you can write
var jobsList = from j in mf.Jobs
where j.UserID == 1001
select new {
Job.JobID,
Job.UserID
};

The most probable reason for this behavior is that you have LazyLoadingEnabled property set to true.
If this is the case, the User isn't recovered in the original query. But if you try to acces this property, even if you do it through an inspection while debugging, this will be loaded from the database. But only if you try to access it.
You can check this opening a a SQL Server Profiler, and seeing what commands are begin sent to the DB.
Your code is not using eager loading or explicit loading. So this must be the reason.

I think that EF don't know that you want one result only. Try something like this.
Job jobsItem = mf.Jobs.Single(j=>j.UserID==1001)
If you don't want to use lambas...
Job JobItem = (from j in mf.Jobs where j.UserID == 1001 select j).Single()
I haven't a compiler near right now, I hope the syntax is right. Use can use var instead of Job for your variable if you preffer. It has no effect but I think Job is more readable in this case.

User actually is not attached to context until you access User property of Job. Turn off Lazy Loading if you want to get a null for User.

Entity Framework does not support lazy loading of properties. However, it has table-splitting
Emphasized the properties. Of course, Entity Framework supports lazy loading of rows