Update
The crux of the issue is caused by using a nested field on a property inside the where clause; I've provided a simplified example (which fails with the same "cannot be translated" error) for illustration. For fuller picture, read the content under the Original heading.
The following clause (rest of the query was working before I started using this nested property)
where (r.Status.IntVal == (int)StatusEnum.Answered) // IntVal is just an int
gets translated at runtime to
.Where(ti => ti.Outer.Status.IntVal == 1) // right hand side translates as expected
So the question is: How do you write a proper ValueConverter (or what other step do you take) in order to properly read and evaluate a nested field like this in EntityFramework?
Orginal Post
I recently updated a field on one of my EF models (using EF Core v3.1.3) from an enum to a class which has a System.Enum field on it. Because I want the database / EF to treat that field as the underlying integer value (like it does with enum) I'm using .HasConversion() in the fluent API, but I'm getting the dreaded "could not be translated" error.
The following line is the code in the query which is causing the issue
...
where (r.Status.Value == StatusEnum.Answered || r.Status.Value == StatusEnum.AwaitingResponse)
...
which gets translated to
.Where(ti => (int)ti.Outer.Status.Value == 1 || (int)ti.Outer.Status.Value == 0)
The value it is trying to translate is of this type:
public class MyStatusClass<T> where T : Enum
{
public MyStatusClass(int i)
{
Value = (T)Enum.Parse(typeof(T), value.ToString());
IntVal = i // This was added to simplify problem, see update above
}
public T Value { get; internal set; } // This is the field that I need stored in the database
public int IntVal {get; internal set; } // This was added to simplify problem, see update above
// Some other methods in here; the reason I converted from an enum to this class
}
The conversion in my DbContext ModelBuilder looks like this
entity.Property(x => x.Status)
.HasConversion(
status => status.Value,
status => new StatusClass((int)status) // This is the part to focus on, as its' the part that tells EF how to read the property from the database
);
Related
I have a case where I need to send tens of thousands of ids to the graphql server in the filtering query.
The query now generated by the HT is something like this:
_dbContext.
Forms
.Where(c=>staticLiistOfIds.Contains(c.Id))
.Select(c=>new {C.Name,C.Age});
I have two problems with this:
slow performance
SQL Server Limit I guess is around 32K
I have found a Nuget library to convert this static list to a temp table,so now I want to override the HT middle to rewrite the above query generated to the following:
_dbContext.
Forms
.Where(c=>_dbContext..AsQueryableValues(staticLiistOfIds).Contains(c.Id))
.Select(c=>new {C.Name,C.Age});
This will create a temp table for this static list of ids so I will be able to solve the above two problems that I have.
So since i didn't get answers, I had to ask from the Slack of HotChocolate's Team and hopefully, they provided me with the documentation extending-filtering/extending-iqueryable:
in case the link was broken, here is
Extending IQueryable The default filtering implementation uses
IQueryable under the hood. You can customize the translation of
queries by registering handlers on the QueryableFilterProvider.
The following example creates a StringOperationHandler that supports
case-insensitive filtering:
// The QueryableStringOperationHandler already has an implemenation of CanHandle
// It checks if the field is declared in a string operation type and also checks if
// the operation of this field uses the `Operation` specified in the override property further
// below
public class QueryableStringInvariantEqualsHandler : QueryableStringOperationHandler
{
// For creating a expression tree we need the `MethodInfo` of the `ToLower` method of string
private static readonly MethodInfo _toLower = typeof(string)
.GetMethods()
.Single(
x => x.Name == nameof(string.ToLower) &&
x.GetParameters().Length == 0);
// This is used to match the handler to all `eq` fields
protected override int Operation => DefaultFilterOperations.Equals;
public override Expression HandleOperation(
QueryableFilterContext context,
IFilterOperationField field,
IValueNode value,
object parsedValue)
{
// We get the instance of the context. This is the expression path to the propert
// e.g. ~> y.Street
Expression property = context.GetInstance();
// the parsed value is what was specified in the query
// e.g. ~> eq: "221B Baker Street"
if (parsedValue is string str)
{
// Creates and returnes the operation
// e.g. ~> y.Street.ToLower() == "221b baker street"
return Expression.Equal(
Expression.Call(property, _toLower),
Expression.Constant(str.ToLower()));
}
// Something went wrong 😱
throw new InvalidOperationException();
}
}
This operation handler can be registered on the convention:
public class CustomFilteringConvention : FilterConvention
{
protected override void Configure(IFilterConventionDescriptor descriptor)
{
descriptor.AddDefaults();
descriptor.Provider(
new QueryableFilterProvider(
x => x
.AddDefaultFieldHandlers()
.AddFieldHandler<QueryableStringInvariantEqualsHandler>()));
}
}
// and then
services.AddGraphQLServer()
.AddFiltering<CustomFilteringConvention>();
To make this registration easier, Hot Chocolate also supports
convention and provider extensions. Instead of creating a custom
FilterConvention, you can also do the following:
services
.AddGraphQLServer()
.AddFiltering()
.AddConvention<IFilterConvention>(
new FilterConventionExtension(
x => x.AddProviderExtension(
new QueryableFilterProviderExtension(
y => y.AddFieldHandler<QueryableStringInvariantEqualsHandler>()))));
but I was suggested that doing this way(sending up to 100k list of string ids to the graphQL server) is not a good approach. so I decided to take another approach by writing a custom simple dynamic LINQ generates.
Thanks.
I am using HotChocolate library to work with GraphQL via .NET. I already can get all objects, that are stored in db, using this query:
query
{
news
{
title
description
}
}
But I need to have an opportunity, to select object with specific id like in this query:
query
{
news(id: 5)
{
title
description
}
}
But I'm getting the following exception
Unknown argument "id" on field "Query.news".
I use this code to get all news from database and return it to a client:
[UseDbContext(typeof(Context.Context))]
[UseFiltering]
[UseSorting]
public IQueryable<Entities.News> GetNews([ScopedService] Context.Context context)
{
return context.News;
}
Also I tried to get an object by id using this code:
[UseDbContext(typeof(Context.Context))]
[UseFiltering]
[UseSorting]
public Entities.News GetNews(int id, [ScopedService] Context.Context context)
{
return context.News.Find(id);
}
But I Ñ–till have the exception when trying to get it by id
With the UseFiltering attribute, this exposes a new schema "descriptor" of WHERE and you can then use this in your Graph QL query
query ($searchId: Int!)
{
news(where: {id: {eq: $searchId}})
{
title
description
}
}
variables {
"searchId" : 1
}
The short answer: no, you cannot proceed that way.
The explanation: the attribute [UseFiltering] can be applied to something that presents a collection of items only. In the second method with 'id', you are trying to return the single entity along with [UseFiltering]. It's not possible.
So, in total, you have the following options (which all do not conform to your needs exactly):
You follow the way in that answer - as for me, it is the most logically correct approach.
You can add the parameter 'id' to the method GetNews but still return IQueryable containing the single element or nothing - as for me, the most non-intuitive approach because the selection by single id is logically incompatible with filtering by WHERE, results ordering and pagination.
Create a separate method GetUserById returning a single element - as for me, the correct approach which is often used along with p.1.
I'm building a complicated Linq query and I'm running into the problem that "The specified type member 'JobRatio' is not supported in LINQ to Entities".
In my model, Jobs is an IEnumerable:
Job has the following calculated property:
public decimal JobRatio
{
get
{
return TotalValueApprovedQuotes == 0 ? 0 : ((TotalValueApprovedQuotes - TotalWorkOrders) / TotalValueApprovedQuotes);
}
}
When I try to set the where clauses from my query based on the user requirement:
if (model.ShowJobRatio)
{
jobs = jobs.Where(x => x.JobRatio < 0.3m);
}
I get the error that "The specified type member 'JobRatio' is not supported in LINQ to Entities"
While I realise I can change my IEnumerable to a List to resolve the issue, I actually have a number of these calculated properties to go through and would like to not run the query just yet. (as I'm still building the Where clauses)
One option would be to turn this type into a basic decimal that is precalculated, what are some strategies to address this? And if I were to calculate this, do I do that on the model? Or in the controller when TotalValueApprovedQuotes, TotalWorkOrders or TotalValueApprovedQuotes changes?
If this is code first, you can add a setter to the calculated property to have EF recognise it as a persisted field.
public decimal JobRatio
{
get { return ...; }
set { }
}
Doing this, EF will store the calculated value when you edit and save the entity but it will no be able to populate the value on load so you always have an in-memory up to date value.
However, if the dependency fields are to change outside of this control, your persisted value will be incorrect.
Note that while it's not pretty, ternary operators (expr ? expr : expr) are accepted in LINQ to Entities so you can use your calculation in place of x.JobRatio.
I followed the great advice here (Handling calculated properties with breezejs and web api) to allow Breeze to access my calculated properties which I have set up in a partial class on the server side:
public partial class EventPerson
{
[NotMapped]
public Decimal TotalAmountPaid
{
get
{
return this.EventPersonPayments.Sum(p => p.AmtPaid);
}
}
}
But for each EventPerson I retrieve, this value shows up as 0 unless I use .expand("EventPersonPayments") clientside or .Include("EventPersonPayments") serverside.
I don't want all the data in EventPersonPayments to be serialized and sent to the client; all I want is the summed value. Is this possible?
EDIT: If my calculated property is derived from other properties already in the entity, it works fine. For example:
public partial class EventPerson
{
[NotMapped]
public String DisplayName
{
get
{
return this.FirstName + " " + this.LastName;
}
}
}
returns the DisplayName in the JSON payload. The former type of calculated property always returns 0 or null unless I specifically load all the extra information.
I considered converting these into User Defined Functions in SQL Server, but I shouldn't have to throw out my C# code just to make it work the way it should.
One approach is to use a projection that incorporates both the entities being queried and some calculated properties as well. i.e. your server query might look like this:
[HttpGet]
public IQueryable<Object> CustomersAndFreightTotals(companyName) {
var stuff = ContextProvider.Context.Customers
.Where(c => c.CompanyName.StartsWith(companyName))
.Select(c => new { Customer = c, FreightTotal = c.Orders.Sum(o => o.Freight)) });
return stuff;
}
This query will load all of your customers that start with a specified company name but will also give you the "total freight" for all of the orders on each customer.
You would call this with code something like this:
var query = EntityQuery.from("CustomersAndFreightTotals")
.withParameters({ companyName: "C" });
myEntityManager.executeQuery(query).then(function(data) {
var results = data.results;
results.forEach(function (r) {
// note that each customer WILL also be added to the local entityManager
// because it is an entity, whereas the freightTotal is only available here.
var customer = r.Customer;
var freightTotal = r.FreightTotal;
// and if you wanted to hack the customer entity
// you could do this.
customer.freightTotal = freightTotal;
});
}
I came across this problem also, and there are a couple of other questions/answers that seem to point to what's going on:
My unmapped properties in breeze does not seems to work whith a projection
UnMapped property on the Angular/Breeze SPA template
From my understanding, to put it shortly, [NotMapped] prevents Breeze/Entity Framework from correctly wiring up to the field. Yet Json.NET will serialize the field and send it to Breeze, which will populate the field if you've manually set it up via the class's constructor, and the data has been retrieved by using expand for the other property which Entity Framework recognizes. This seems to be almost an accident you can get [NotMapped] fields to work on the client in this given case; the Breeze+Entity Framework does not seem to be designed for this case.
There is a suggestion at Breeze's User Voice that you could vote and comment on. I'm not sure that Breeze could solve this problem themselves without some work from the Entity Framework team, but at least it could put the issue on their radar.
I have two tables Studies and Series. Series are FK'd back to Studies so one Study contains a variable number of Series.
Each Series item has a Deleted column indicating it has been logically deleted from the database.
I am trying to implement a Deleted property in the Study class that returns true only if all the contained Series are deleted.
I am using O/R Designer generated classes, so I added the following to the user modifiable partial class for the Study type:
public bool Deleted
{
get
{
var nonDeletedSeries = from s in Series
where !s.Deleted
select s;
return nonDeletedSeries.Count() == 0;
}
set
{
foreach (var series in Series)
{
series.Deleted = value;
}
}
}
This gives an exception "The member 'PiccoloDatabase.Study.Deleted' has no supported translation to SQL." when this simple query is executed that invokes get:
IQueryable<Study> dataQuery = dbCtxt.Studies;
dataQuery = dataQuery.Where((s) => !s.Deleted);
foreach (var study in dataQuery)
{
...
}
Based on this http://www.foliotek.com/devblog/using-custom-properties-inside-linq-to-sql-queries/, I tried the following approach:
static Expression<Func<Study, bool>> DeletedExpr = t => false;
public bool Deleted
{
get
{
var nameFunc = DeletedExpr.Compile();
return nameFunc(this);
}
set
{ ... same as before
}
}
I get the same exception when a query is run that there is no supported translation to SQL. (
The logic of the lambda expression is irrelevant yet - just trying to get past the exception.)
Am I missing some fundamental property or something to allow translation to SQL? I've read most of the posts on SO about this exception, but nothing seems to fit my case exactly.
I believe the point of LINQ-to-SQL is that your entities are mapped for you and must have correlations in the database. It appears that you are trying to mix the LINQ-to-Objects and LINQ-to-SQL.
If the Series table has a Deleted field in the database, and the Study table does not but you would like to translate logical Study.Deleted into SQL, then extension would be a way to go.
public static class StudyExtensions
{
public static IQueryable<study> AllDeleted(this IQueryable<study> studies)
{
return studies.Where(study => !study.series.Any(series => !series.deleted));
}
}
class Program
{
public static void Main()
{
DBDataContext db = new DBDataContext();
db.Log = Console.Out;
var deletedStudies =
from study in db.studies.AllDeleted()
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
}
}
This maps your "deleted study" expression into SQL:
SELECT t0.study_id, t0.name
FROM study AS t0
WHERE NOT EXISTS(
SELECT NULL AS EMPTY
FROM series AS t1
WHERE (NOT (t1.deleted = 1)) AND (t1.fk_study_id = t0.study_id)
)
Alternatively you could build actual expressions and inject them into your query, but that is an overkill.
If however, neither Series nor Study has the Deleted field in the database, but only in memory, then you need to first convert your query to IEnumerable and only then access the Deleted property. However doing so would transfer records into memory before applying the predicate and could potentially be expensive. I.e.
var deletedStudies =
from study in db.studies.ToList()
where study.Deleted
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
When you make your query, you will want to use the statically defined Expression, not the property.
Effectively, instead of:
dataQuery = dataQuery.Where((s) => !s.Deleted);
Whenever you are making a Linq to SQL query, you will instead want to use:
dataQuery = dataQuery.Where(DeletedExpr);
Note that this will require that you can see DeletedExpr from dataQuery, so you will either need to move it out of your class, or expose it (i.e. make it public, in which case you would access it via the class definition: Series.DeletedExpr).
Also, an Expression is limited in that it cannot have a function body. So, DeletedExpr might look something like:
public static Expression<Func<Study, bool>> DeletedExpr = s => s.Series.Any(se => se.Deleted);
The property is added simply for convenience, so that you can also use it as a part of your code objects without needing to duplicate the code, i.e.
var s = new Study();
if (s.Deleted)
...