I'm attempting to build an expression tree to perform a LINQ to Entities query in .NET 4.0 with EF4. When I attempt to execute the query I've built, I get a NotSupportedException with the following message:
LINQ to Entities does not recognize the method
'System.Data.Objects.ObjectQuery`1[TestWpf.Customer]
Where(System.String, System.Data.Objects.ObjectParameter[])' method,
and this method cannot be translated into a store expression.
I'm querying against the Northwind database. My entities were generated from the database. In my code below, I have the query I'm trying to build in the method GetQuery1(), and I'm attempting to build it in the GetQuery2() method.
If I set a breakpoint and inspect the query1 variable, it's Expression property is:
Convert(value(System.Data.Objects.ObjectSet`1[TestWpf.Customer])).MergeAs(AppendOnly).Where(c => c.CompanyName.Contains("z"))
What is this Convert().MergeAs(AppendOnly) doing? I attempted to search on MSDN but couldn't locate what I need (at least I don't think I could find it...). Additionally, what am I doing wrong?
I think that perhaps I'm calling an incorrect Where() method, as Intellisense says there is another, which is an extension method. I have not tried to update the whereMethod variable to fetch that one, but I'm not sure how, either.
private static IQueryable<Customer> GetQuery1(NorthEntities context) {
return context.Customers.Where(c => c.CompanyName.Contains("z"));
}
private static IQueryable<Customer> GetQuery2(NorthEntities context) {
var custParam = Expression.Parameter(typeof(Customer), "c");
var custCollection = Expression.Constant(context.Customers);
var companyNamePropValue = Expression.Property(custParam, typeof(Customer).GetProperty("CompanyName"));
var containsParameter = Expression.Constant("z");
var containsMethod = Expression.Call(companyNamePropValue, typeof(string).GetMethod("Contains"), containsParameter);
var whereMethod = context.Customers.GetType().GetMethod("Where", new Type[] { typeof(string), typeof(ObjectParameter[]) });
var param2 = Expression.Constant(new ObjectParameter[] { });
var where = Expression.Call(custCollection, whereMethod, companyNamePropValue, param2);
return ((IQueryable<Customer>)context.Customers).Provider.CreateQuery<Customer>(where);
}
private static void Main(string[] args) {
using (var context = new NorthEntities()) {
var query1 = GetQuery1(context);
var query2 = GetQuery2(context);
foreach (var c in query1)
Console.WriteLine(c.CompanyName);
foreach (var c in query2)
Console.WriteLine(c.CompanyName);
}
Console.ReadLine();
}
To construct the specific query you're working with, try the following:
private static IQueryable<Customer> GetQuery2(NorthEntities context) {
IQueryable<Customer> customers = context.Customers;
var custParam = Expression.Parameter(typeof(Customer), "c");
var companyNamePropValue = Expression.Property(custParam, typeof(Customer).GetProperty("CompanyName"));
var containsParameter = Expression.Constant("z");
var containsCall = Expression.Call(companyNamePropValue, typeof(string).GetMethod("Contains"), containsParameter);
var wherePredicate = Expression.Lambda<Func<Customer, bool>>(containsCall, custParam);
return customers.Where(wherePredicate);
}
In general, to get access to the LINQ extension methods (e. g. Where), you'll have to look in the Queryable class:
var genericWhereMethod = typeof(Queryable).GetMethods()
.Single(m => m.Name == "Where"
// distinguishes between Where((T, int) => bool) and Where(T => bool)
&& m.GetParameters()[1].ParameterType
.GetGenericArguments()[0].GetGenericTypeDefinition() == typeof(Func<,>));
// make the Where method that applies to IQueryable<Customer>
var whereMethod = genericWhereMethod.MakeGenericMethod(typeof(Customer));
Related
I have many tables that I need to query data to see if a certain parameter(isCorrect) is yes. This field is present in all the tables.
I have built a dynamic query using generic type. But im getting execution period timeout when fetching the data.
Here is a small code snippet:
public bool isRecordCorrect<T>(Guid userID)
{
using (EmployeeEntities dbContext = new EmployeeEntities())
{
DbSet dbSet = dbContext.Set(typeof(T)) // T here is the table types for e.g employee, department
IQueryable<T> query = (IQueryable<T>)dbSet.AsQueryable();
var list = query
.Where(DynamicQuery<T>.FilterStatement(userID))
.FirstOrDefault();
return list == null ? false : true;
}
}
public class DynamicQuery<T>
{
public static Func<T,bool> FilterStatement(Guid userID)
{
var xParameter = Expression.Parameter(typeof(T), "o");
var prop = Expression.Property(xParameter, "IsCorrect");
var incorrect = Expression.Constant("N");
var equalIncorrect = Expression.Equal(prop,equalIncorrect);
var userIdProp = Expression.Property(xParameter, "userID");
var userId = Expression.Constant(userID);
var equaluserID = Expression.Equal(userIdProp, userId);
var andExpresion = Expression.And(equalIncorrect, equaluserId);
var lambda = Expression.Lambda<Func<T, bool>> (andExpresion, xParameter);
return lambda.Compile();
}
}
It is timing out for tables which have huge number of records but works okay for others.
To translate query on the server side, EF Core needs pure Expression. Compile and returning Func<,>, instead of Expression<Func<,>> forces to use Enumerable.Where extension, which just loads whole table in memory before filtering. Simple fix should be enough in companion with index on UserId (if it is not present).
public static Expression<Func<T,bool> FilterStatement(Guid userID)
{
var xParameter = Expression.Parameter(typeof(T), "o");
var prop = Expression.Property(xParameter, "IsCorrect");
var incorrect = Expression.Constant("N");
var equalIncorrect = Expression.Equal(prop,equalIncorrect);
var userIdProp = Expression.Property(xParameter, "userID");
var userId = Expression.Constant(userID);
var equaluserID = Expression.Equal(userIdProp, userId);
var andExpresion = Expression.And(equalIncorrect, equaluserId);
var lambda = Expression.Lambda<Func<T, bool>> (andExpresion, xParameter);
return lambda;
}
This question already has answers here:
Generate EF orderby expression by string [duplicate]
(6 answers)
Closed 7 years ago.
I am using .NET 4.51, EF 6
I make a number of calls to my repository layer where I need to do some basic ordering on a single field in either ascending or descending order such as:
The result of GetAllList() is a List<T>. Now unfortunately the Id field I have to sort by is not always called Id nor is the Text field. They can be other things such as MyId, SomeTextField and so on.
So I was wondering if there was a way I could do the OrderBy() and OrderByDescending() clauses by supplying a string for the field name something like:
_Repository.GetAllList().OrderBy(r => r."SomeTextField")
In this way I could move all this code to a common method.
Any pointers greatly appreciated.
This will work:
public static class LinqExtensions
{
private static PropertyInfo GetPropertyInfo(Type objType, string name)
{
var properties = objType.GetProperties();
var matchedProperty = properties.FirstOrDefault (p => p.Name == name);
if (matchedProperty == null)
throw new ArgumentException("name");
return matchedProperty;
}
private static LambdaExpression GetOrderExpression(Type objType, PropertyInfo pi)
{
var paramExpr = Expression.Parameter(objType);
var propAccess = Expression.PropertyOrField(paramExpr, pi.Name);
var expr = Expression.Lambda(propAccess, paramExpr);
return expr;
}
public static IEnumerable<T> OrderBy<T>(this IEnumerable<T> query, string name)
{
var propInfo = GetPropertyInfo(typeof(T), name);
var expr = GetOrderExpression(typeof(T), propInfo);
var method = typeof(Enumerable).GetMethods().FirstOrDefault(m => m.Name == "OrderBy" && m.GetParameters().Length == 2);
var genericMethod = method.MakeGenericMethod(typeof(T), propInfo.PropertyType);
return (IEnumerable<T>) genericMethod.Invoke(null, new object[] { query, expr.Compile() });
}
public static IQueryable<T> OrderBy<T>(this IQueryable<T> query, string name)
{
var propInfo = GetPropertyInfo(typeof(T), name);
var expr = GetOrderExpression(typeof(T), propInfo);
var method = typeof(Queryable).GetMethods().FirstOrDefault(m => m.Name == "OrderBy" && m.GetParameters().Length == 2);
var genericMethod = method.MakeGenericMethod(typeof(T), propInfo.PropertyType);
return (IQueryable<T>) genericMethod.Invoke(null, new object[] { query, expr });
}
}
Testing:
var r = new List<temp> {
new temp { a = 5 },
new temp { a = 1 },
new temp { a = 15 }
}.OrderBy("a");
Gives the correct result (1, 5, 15) - and will provide lazy execution for your use with EF
You will need to implement the overloads if needed.
Does it have to be a string? Why not just make a method that takes a Func key selector as a parameter.
public List<T> GetAllListOrdered<T,TProp>(SimpleOrderingDirectionEnum direction, Func<T,TProp> keySelector)
{
return direction == SimpleOrderingDirectionEnum.Ascending ? _Repository.GetAllList().OrderBy(keySelector).ToList() : _Repository.GetAllList().OrderByDescending(keySelector).ToList();
}
Then call it like
Func<ObjectToSortType, ObjectPropertyToSortBy> keySelector = r => r.Id;
GetAllListOrdered(SimpleOrderingDirectionEnum.Ascending, keySelector);
If the Rob's answer is not enough for you. Try Linq Dynamic. http://dynamiclinq.azurewebsites.net/
using System.Linq.Dynamic; //Import the Dynamic LINQ library
//The standard way, which requires compile-time knowledge
//of the data model
var result = myQuery
.Where(x => x.Field1 == "SomeValue")
.Select(x => new { x.Field1, x.Field2 });
//The Dynamic LINQ way, which lets you do the same thing
//without knowing the data model before hand
var result = myQuery
.Where("Field1=\"SomeValue\"")
.Select("new (Field1, Field2)");
Thanks to all. Rob, your solution was pretty close to what I ended up with.
Based on your insights I did some more searching and came across Marc Gravel's answer here Dynamic LINQ OrderBy on IEnumerable<T> (second post).
It added dynamic's as an additional bonus.
I'm in a situation where I need to specify queries on a EF-context at runtime. Our consultants configure these queries at a customer site for customer specific situations.
In order to facilitate that I was thinking of using linq to build the queries, based on a criteria-list which the consultants specify in a front-end of some kind (right now, winforms). The consultants basically specify a property from an object, specify the operator and then the value. For example: give me all clients where the [status] [equals] [1].
At the moment I have an expression builder which creates the where clause at runtime, and so far I can manage one group by clause. Where I'm running against the wall here, is when a consultant configures multiple group by clauses of different types (f.e. a string and a datetime property).
For example, I must be able to process this query: select bsn as a, dateofbirth as b from clients where status = 1 group by bsn, dateofbirth (where bsn = string and dateofbirth = datetime).
At the moment, this is the code which "glues" the query together:
public List<ClientV2> ExportClients(List<CriteriaV2> criteriaList)
{
var whereExpression = BuildWhereExpressionChain(criteriaList.Where(c => c.Operator != CriteriaOperatorV2.GROUPBY).ToList());
var groupByExpression = BuildGroupByExpression(criteriaList.Where(c => c.Operator == CriteriaOperatorV2.GROUPBY).ToList());
var sourceClients = _context.Clients.Where(whereExpression).GroupBy(groupByExpression).ToList();
IEnumerable<Client> resultClients = sourceClients.SelectMany(group => group);
return ClientToClientV2.MapList(resultClients.ToList());
}
This is the where-clause builder:
private Expression<Func<Client, bool>> BuildWhereExpressionChain(List<CriteriaV2> criteriaList)
{
var expressionList = new List<Expression<Func<Client, bool>>>();
var paramExp = Expression.Parameter(typeof(Client));
foreach (var crit in criteriaList)
{
var propertyItem = PropertyTranslator.GetPropertyItem(crit.Property);
if (propertyItem == null) throw new InvalidFilterCriteriaException("Property " + crit.Property + " niet toegestaan als filter criterium");
var propInfo = typeof(Client).GetProperty(propertyItem.InternalName);
var left = Expression.Property(paramExp, propInfo);
Expression right;
if (propInfo.PropertyType.IsEnum)
right = Expression.Constant(Enum.ToObject(propInfo.PropertyType, PropertyTranslator.TranslateEnum(propertyItem.Type, crit.Value)));
else if (propInfo.PropertyType == typeof(DateTime) || propInfo.PropertyType == typeof(DateTime?))
right = Expression.Constant(DateTime.Parse(crit.Value), propInfo.PropertyType);
else
right = Expression.Constant(crit.Value, typeof(string));
var exp = BuildExpression(left, right, crit.Operator);
expressionList.Add(Expression.Lambda<Func<Client, bool>>(exp, new ParameterExpression[] { paramExp }));
}
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.And(ex); });
return firstExpression;
}
And this is the part where I am stuck (it does work for one clause of type string):
private Expression<Func<Client, string>> BuildGroupByExpression(List<CriteriaV2> criteriaList)
{
var expressionList = new List<Expression<Func<Client, string>>>();
var paramExp = Expression.Parameter(typeof(Client));
foreach (var crit in criteriaList)
{
var propertyItem = PropertyTranslator.GetPropertyItem(crit.Property);
if (propertyItem == null) throw new InvalidFilterCriteriaException("Property " + crit.Property + " niet toegestaan als group by criterium");
var propInfo = typeof(Client).GetProperty(propertyItem.InternalName);
var body = Expression.Property(paramExp, propInfo);
var lambda = Expression.Lambda<Func<Client, string>>(body, paramExp);
expressionList.Add(lambda);
}
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.And(ex); });
return firstExpression;
}
Would it be possible to make the BuildGroupByExpression() in such way that it results in an expression which contains multiple clauses of different types that I can use directly in .GroupBy(expression);?
I'm not that much of an expert on linq, but I have a feeling that what I want could be possible. If I do stupid things here, please point them out and I'll work on it.
I dont think so. At least using the approach where the Expression is built they way you have. At least I ended up building expressions per type.
The main reason is
var lambda = Expression.Lambda<Func<Client, string>>(body, paramExp);
I dont know how you can make this Lambda defintion dynamic or generic.
There is a different approach, and a library that uses a string interpretation at runtime to build expressions. See
Install-Package System.Linq.Dynamic.Library
see also https://msdn.microsoft.com/en-US/vstudio/bb894665.aspx
I have this LINQ Expression that finds all the historical changes to a given customer's CreditBalance:
var history = GetHistory(id);
var changes = history.Where(x => history.Where(y => y.AuditId < x.AuditId)
.OrderByDescending(y => y.AuditId)
.Select(y => y.CreditBalance)
.FirstOrDefault() != x.CreditBalance);
This function works as expected. What I want to do is change this function to allow the user to query changes to any historical field. The way I chose to tackle this was with expression trees.
So far I have come up with this solution:
var history = GetHistory(id);
var c = Expression.Parameter(typeof(Customer_history), "c");
var d = Expression.Parameter(typeof(Customer_history), "d");
var cAudit = Expression.Property(c, typeof(Customer_history).GetProperty("AuditId"));
var dAudit = Expression.Property(d, typeof(Customer_history).GetProperty("AuditId"));
var whereBody = Expression.LessThan(dAudit, cAudit);
var whereLambda = Expression.Lambda(whereBody, d);
var where = Methods.QueryableWhere.MakeGenericMethod(typeof(Customer_history));
var whereCall = Expression.Call(null, where, **Expression.Constant(history)**, whereLambda);
var orderByLambda = Expression.Lambda(dAudit, d);
var orderBy = Methods.QueryableOrderByDescending.MakeGenericMethod(typeof(Customer_history), orderByLambda.Body.Type);
var orderByCall = Expression.Call(null, orderBy, whereCall, orderByLambda);
var dProp = Expression.Property(d, typeof(Customer_history).GetProperty(field));
var selectLambda = Expression.Lambda(dProp, d);
var select = Methods.QueryableSelect.MakeGenericMethod(typeof(Customer_history), selectLambda.Body.Type);
var selectCall = Expression.Call(null, select, orderByCall, selectLambda);
var firstOrDefault = Methods.QueryableFirstOrDefault.MakeGenericMethod(selectLambda.Body.Type);
var firstOrDefaultCall = Expression.Call(null, firstOrDefault, selectCall);
var cProp = Expression.Property(c, typeof(Customer_history).GetProperty(field));
var comparison = Expression.NotEqual(firstOrDefaultCall, cProp);
var lambda = Expression.Lambda<Func<Customer_history, bool>>(comparison, c);
var changes = history.Where(lambda);
The problem is, I get this exception when the query is executed:
Unable to create constant value of type
'Namespace.Customer_history'. Only primitive types or enumeration
types are supported in this context.
Now I am assuming that the issue is the Expression.Constant(history) statement based on the exception message. The problem is, I don't know how to rewrite it to allow the query provider to handle it appropriately. I know it works because of the original query, I just don't know how to do it in an expression tree.
Can anyone provide any direction?
As suspected, it seems that a ConstantExpression is not the way to obtain the value from a local variable.
What I needed to do was create a private class to store the variable in, and then I was able to access it with a field MemberExpression
private class ValueHolder<T>
{
public IQueryable<T> History;
}
Then in my method I was able to have the expression evaluated using this:
var valueHolder = new ValueHolder<T>
{
History = data
};
var c = Expression.Parameter(typeof(T), "c");
var constantEx = Expression.Constant(valueHolder);
var fieldEx = Expression.Field(constantEx, valueHolder.GetType().GetField("History"));
You can always try using dynamic linq, which allows you to use strings as expressions instead of lambda.
Example:
var query = history.Where("MyField = MyFilter");
https://www.nuget.org/packages/System.Linq.Dynamic.Library/
https://github.com/NArnott/System.Linq.Dynamic
I am having trouble creating an extension method for an IQueryable that will include the translation for a specified column in a Linq Query.
Suppose i have the query below.
I would like to call a method IncludeTranslation on the CFG_Article IQueryable specifying the column i want to get the translation for.
Could someone help me in the right direction.
var translations =
from t in UoW.CFG_TRANSLATION.GetAll()
select t;
var result = (
from a in UoW.CFG_ARTICLE.GetAll()
select new
{
a,
translation = translations
.Where(t=> t.TR_TEXT == a.AR_NAME).FirstOrDefault()
});
All i have come up so far is the code below but this does not compile.
public static IQueryable IncludeTranslation<T>(
this IQueryable<T> query,
Expression<Func<t, bool>> fieldToTranslate)
{
// this will get an IQueryable of CFG_TRANSLATION
var translations = GetTranslations();
var result = (
from q in query
select new
{
q,
translation = translations
.Where(t=> t.TR_TEXT == fieldToTranslate)
.FirstOrDefault()
});
// even better is to return all fields from query
// + the TR_TRANSLATION field from the translations table
return result;
}
Try this (I'm having to guess class names are CFG_ARTICLE & CFG_TRANSLATION - replace as required)
public static IQueryable IncludeTranslation(
this IQueryable<CGF_ARTICLE> query,
Func<CFG_ARTICLE, CFG_TRANSLATION, bool> fieldToTranslate)
{
var translations = GetTranslations();
var result =
from a in query
select new
{
a,
translation = translations
.Where(t => fieldToTranslate(a, t))
.FirstOrDefault()
};
return result;
}
calling like this
var result = query.IncludeTranslation(
(article, translation) => article.TR_TEXT == translation.AR_NAME);
I found an other way of returning the same result using a Generic Way.
public static IQueryable IncludeTranslation<S>(this IQueryable<S> source, Expression<Func<S, string>> keyField)
where S : class
{
IQueryable<CFG_TRANSLATION> translations = GetTranslations();
var trans = source.GroupJoin(translations, keyField, t => t.TR_TEXT, (s, t) => new { Source = s, Translations = t });
var result = trans.Select(t => new {
Source = t.Source,
Translation = t.Translations
.FirstOrDefault()
});
return result;
}
Maybe someone can use this as a sollution
this can be called as follow
var Result = QueryableTable.IncludeTranslation(t => t.FieldToTranslate);