I am trying to understand dynamic linq and expression trees. Very basically trying to do an equals supplying the column and value as strings. Here is what I have so far
private IQueryable<tblTest> filterTest(string column, string value)
{
TestDataContext db = new TestDataContext();
// The IQueryable data to query.
IQueryable<tblTest> queryableData = db.tblTests.AsQueryable();
// Compose the expression tree that represents the parameter to the predicate.
ParameterExpression pe = Expression.Parameter(typeof(tblTest), "item");
Expression left = Expression.Property(pe, column);
Expression right = Expression.Constant(value);
Expression e1 = Expression.Equal(left, right);
MethodCallExpression whereCallExpression = Expression.Call(
typeof(Queryable),
"Where",
new Type[] { queryableData.ElementType },
queryableData.Expression,
Expression.Lambda<Func<tblTest, bool>>(e1, new ParameterExpression[] { pe }));
// Create an executable query from the expression tree.
IQueryable<tblTest> results = queryableData.Provider.CreateQuery<tblTest>(whereCallExpression);
return results;
}
That works fine for columns in the DB. But fails for properties in my code eg
public partial class tblTest
{
public string name_test
{ get { return name; } }
}
Giving an error cannot be that it cannot be converted into SQL. I have tried rewriting the property as a Expression<Func but with no luck, how can I convert simple properties so they can be used with linq in this dynamic way?
Many Thanks
To use non-table properties, you'll need to first materialize the query and use LINQ to objects. I don't think you can query against both SQL and non-SQL properties at the same time for the reason that you state: non-SQL properties have no SQL translation. I suspect that if you do a ToList() before calling filterTest(), you'll find that your code works just fine for both types of properties. Unfortunately, this probably isn't what you want and, if your non-SQL property is derived from various SQL columns, you will need a way to generate an expression that matches the property definition instead.
Related
I have a search api which queries a given list of columns from a table or view using a given string. As such, I'm converting each column to string. This works well thus far, except for datetime. These columns are datetimeoffsets. I'm trying to convert these columns to MM-dd-yyyy so that if they search for a value, it will ignore the time values.
The error I'm getting when executing the query is
System.InvalidOperationException: 'The LINQ expression 'DbSet()
.Where(f => f.Id.ToString().Contains("4") | f.CreatedDate.ToString("MM-dd-yyyy").Contains("4"))' could not be translated. Additional information: Translation of method 'System.DateTimeOffset.ToString' failed.
So far I've got the following
public static IQueryable<T> FieldsContains<T>(this IQueryable<T> query, List<string> fields, string searchValue)
{
Expression predicate = null;
var parameterExpression = Expression.Parameter(typeof(T), "type");
foreach (var next in fields.Select(field =>
GetFieldContainsExpression<T>(parameterExpression, field, searchValue)))
{
predicate = predicate == null ? next : Expression.Or(predicate, next);
}
var lambda = Expression.Lambda<Func<T, bool>>(predicate, parameterExpression);
return query.Where(lambda);
}
private static Expression GetFieldContainsExpression<T>(ParameterExpression parameterExpression, string field,
string value)
{
var propertyType = typeof(T).GetProperty(field).PropertyType;
Expression propertyExpression = Expression.Property(parameterExpression, field);
var filterValue = Expression.Constant(value);
var method = typeof(string).GetMethod("Contains", new[] {typeof(string)});
if (propertyType == typeof(DateTimeOffset))
{
MethodInfo? toString = typeof(DateTimeOffset).GetMethod("ToString", new[] { typeof(string) });
MethodCallExpression call = Expression.Call(propertyExpression, toString, Expression.Constant("MM-dd-yyyy"));
return Expression.Call(call, method, filterValue);
}
return Expression.Call(propertyType != typeof(string) ? Expression.Call(propertyExpression, "ToString", Type.EmptyTypes) : propertyExpression, method, filterValue);
}
You have not shown the part that throws the exception. But from what you shared, You're using reflection to generate lambda. This will not work with EF. EF has to translate each of your conditional expressions to something that is reognized by your target database management system.
As stated in the error, EF does not know how to convert f.CreatedDate.ToString("MM-dd-yyyy") to something that is recognized by you dbms.
If you're trying to compare date parts only, you can either create 2 parameters, one for start of the day and one for the end of the day and check if your datetimeoffset column is between those 2 dates.
you can also investigate further to see if comparing date parts is supported by the EF provider. I've not tested this in EF core but it's probably supported when your dbms is SQL Server: f.CreatedDate.Date == myDateParam.Date.
Other workaround would be to fetch all data from database to memory (e.g. using ToList()) and perform your operartion on the retuned IEnumerable as opposed to IQueryable. This way reflection is also supported but it costs you performance degradations based on how big your data is before filtering.
I'd like to read the value of some of the properties in the expression tree some I can proceed to some calculations.
var products = db.Products
.Where(GetPredicate())
.ToList();
private Expression<Func<Product, bool>> GetPredicate()
{
ParameterExpression pe = Expression.Parameter(typeof(Product), "p");
Expression exp0 = Expression.Property(pe, "Price");
//I'd like to know the value of the 'Price'
// so I can do some calculation, then check whether
//this particular product meet the criteria...
Expression body = Expression.Constant(Result); //result is a boolean
var expr = Expression.Lambda<Func<Product, bool>>(body, new ParameterExpression[] { pe });
return expr;
}
You seem to be using a database query provider (LINQ2SQL, EF, etc.)
Before you try to use expressions to solve your problem you need to make sure the expression is understood by the query provider. In your case many of the Math methods can be converted to T-SQL valid statements. Also, if you are using Entity Framework you might want to leverage the System.Data.Objects.SqlClient.SqlFunctions class to create an expression and execute your logic on the SQL Server side against native T-SQL functions.
Now, something to understand on expression trees, is that values cannot be obtained from constructing expressions unless this is a LambdaExpression, once compiled you invoke it, in your case you can obtain the bool value.
If you need to work with the price value, you need to create more expressions representing the call to the other logic, in your example the expressions calling the static Sqrt method of the Math class.
private Expression<Func<Product, bool>> GetPredicate()
{
var pe = Expression.Parameter(typeof(Product), "p");
var price = Expression.Property(pe, "Price");
var priceDouble = Expression.Convert(price, typeof(double));
var sqrtMethod = typeof(Math).GetMethod("Sqrt");
var sqrtCall = Expression.Call(sqrtMethod, priceDouble);
var constant = Expression.Constant(4d);
var gtThan = Expression.GreaterThan(sqrtCall, constant);
var lambda = Expression.Lambda<Func<Product, bool>>(gtThan, pe);
return lambda;
}
As you can see all the logic is an expression and the provider can consume the whole expression and convert it to syntax that can be understood by the target process. The prior expression generates p => Math.Sqrt((double)p.Price) > 4d
How can I create a property selector for entity framework like this?
public static List<T> StandardSearchAlgorithm<T>(this IQueryable<T> queryable, Func<T, string> property, string query)
{
return queryable.Where(e => property(e).ToLower().IndexOf(query) > -1).ToList();
}
I want the calling code to be able to be clean and simple like this:
var usernameResults = _db.Users.StandardSearchAlgorithm(u => u.Username, query);
I get a "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities." error. I cannot work out how to get the expression built.
UPDATE:
Based on the answer by MBoros here is the code I ended up with. It works great.
The key to expression trees is to understand expression trees are all about breaking up what you normally write in code (like "e => e.Username.IndexOf(query)") into a series of objects: "e" gets its own object, "Username" its own object, "IndexOf()" its own object, the "query" constant its own object, and so on. The second key is to know that you can use a series of static methods on the Expression class to create various kinds of these objects, as shown below.
PropertyInfo pinfo = (PropertyInfo)((MemberExpression)property.Body).Member;
ParameterExpression parameter = Expression.Parameter(typeof(T), "e");
MemberExpression accessor = Expression.Property(parameter, pinfo);
ConstantExpression queryString = Expression.Constant(query, typeof(string));
ConstantExpression minusOne = Expression.Constant(-1, typeof(int));
MethodInfo indexOfInfo = typeof(string).GetMethod("IndexOf", new[] { typeof(string) }); // easiest way to do this
Expression indexOf = Expression.Call(accessor, indexOfInfo, queryString);
Expression expression = Expression.GreaterThan(indexOf, minusOne);
Expression<Func<T, bool>> predicate = Expression.Lambda<Func<T, bool>>(expression, parameter);
//return predicate.Body.ToString(); // returns "e => e.Username.IndexOf(query) > -1" which is exactly what we want.
var results = queryable.Where(predicate).ToList();
return results;
Now I have a real problem, but I will ask it in a separate question. My real query looks like this:
public static List<T> StandardSearchAlgorithm<T>(this IQueryable<T> queryable, Func<T, string> property, string query)
{
return queryable.Where(e => property(e).IndexOf(query) > -1).Select(e=> new { Priority = property(e).IndexOf(query), Entity = e } ).ToList();
}
So I need to build an expression that returns an Anonymous Type!! Or even if I create a class to help, I need to write an expression that returns a new object. But I will include this in a separate question.
You cannot invoke CLR delegates so simply in sql. But you can pass in the property selector as an Expression tree., so your signature would be:
public static List<T> StandardSearchAlgorithm<T>(this IQueryable<T> queryable, Expression<Func<T, string>> property, string query)
Calling would look the same. But now that you have an expression in your hand, you can have a look at this answer:
Pass expression parameter as argument to another expression
It gives you the tools to simply put an expression tree inside another one. In your case it would look like:
Expression<Func<T, bool>> predicate = e => property.AsQuote()(e).Contains(query);
predicate = predicate.ResolveQuotes();
return queryable.Where(predicate).ToList();
Once you are there, you still have the .ToLower().Contains() calls (use .Contains instead of the .IndexOf()> 1). This is actually tricky. Normally the db uses its default collation, so if it set to CI (case insensitive), then it will do the compare that way. If you don't have any constraints, and can adjust the db collation, I would go for that. In this case you can omit the .ToLower() call.
Otherwise check out this anser: https://stackoverflow.com/a/2433217/280562
I've been doing research on how to dynamically make queries for linq. I'm creating a UI for a web application which allows them to select properties from a DB and set where clauses to build reports dynamically. All of the questions I've seen on stack or other sites reference using Dynamic Linq and LinqKit to Solve the problem, example Here . However I can't find an solution to express the syntax.
// This attempts to grab out a title whose from the Database whose
// Copyright Date equals 2006
propName = "CopyrightDate";
Model.Titles.Where(t => t.GetType().GetProperty(propName).GetValue(t, null) == "2006");
I want to do something like this but in Linq to Entities. Linq to entities doesn't support reflection like that, I do not want to pull out all of the data and run Linq to Object the DB is too Large. Any Suggestions on how to write this in Dynamic Linq. Bonus points if you can cast the type to the property type so It can be evaultuated with standard operators (== , > , < , etc..).
What the LINQ-to-entities Where extension method wants, is an Expression<Func<T, bool>>. You can create such an expression like this:
// Create lambda expression: t => t.CopyrightDate == "2006"
ParameterExpression parameter = Expression.Parameter(typeof(T), "t");
var lambda = Expression.Lambda<Func<T, bool>>(
Expression.Equal(
Expression.Property(parameter, "CopyrightDate"),
Expression.Constant("2006")
),
parameter
);
where T is the type of your class containing the CopyrightDate property.
var result = context.Titles
.Where(lambda)
.ToList();
A somewhat more general solution is:
Expression<Func<T, bool>> Comparison<T>(ExpressionType comparisonType,
string propertyName, object compareTo)
{
ParameterExpression parameter = Expression.Parameter(typeof(T), "t");
MemberExpression property = Expression.Property(parameter, propertyName);
ConstantExpression constant = Expression.Constant(compareTo);
Expression body = Expression.MakeBinary(comparisonType, property, constant);
return Expression.Lambda<Func<T, bool>>(body, parameter);
}
Here is how you can rewrite your query using Dynamic Linq:
var propName = "CopyrightDate";
var propValue = "2006";
Model.Titles.Where(string.Format("{0}=#0", propName), propValue);
string.Format("{0}=#0", propName) produces the query string for the Where clause, which would be "CopyrightDate=#0" in this case. #0 specifies the parameter for the query, which becomes propValue.
I am trying to create a array of anonymous type from the result set using LINQ.
my code is
var data = (from s in sortedStudents
select new
{
//id = s.CurrencyId,
cell = new object[] { s.PropertyNumber.ToString(), s.Name,
s.Status.ToString(), "Hello" }
}).ToArray();
But I am getting an error as below, but I have not used any System.Int64 variable.
Unable to cast the type 'System.Int64' to type 'System.Object'. LINQ
to Entities only supports casting EDM primitive or enumeration types.
Any ideas what is wrong here.
After spending sometime I suspect that my logic for sorting the data is creating the problem, my code for sorting the date is
// Utility method to sort IQueryable given a field name as "string"
// May consider to put in a cental place to be shared
public IQueryable<T> SortIQueryable<T>(IQueryable<T> data, string fieldName, string sortOrder)
{
if (string.IsNullOrWhiteSpace(fieldName)) return data;
if (string.IsNullOrWhiteSpace(sortOrder)) return data;
var param = Expression.Parameter(typeof(T), "i");
Expression conversion = Expression.Convert(Expression.Property(param, fieldName), typeof(object));
var mySortExpression = Expression.Lambda<Func<T, object>>(conversion, param);
return (sortOrder == "desc") ? data.OrderByDescending(mySortExpression)
: data.OrderBy(mySortExpression);
}
Now when I my the column name whose type is system.int64, then its creating problem, as execution of query takes place actually when i do .ToArray to the results.
The problem is likely in your sorting logic, namely this line:
Expression conversion = Expression.Convert(Expression.Property(param, fieldName), typeof(object));
As the error message states, LINQ to Entities does not support this type of conversion. You will either need to execute the query beforehand with a .ToList() or reformulate your sorting logic.