I have a situation where I am using a linq provider that does not support the .Contains method to generate a WHERE IN clause in the query. I am looking for a way to generate the (Value = X OR Value = Y OR Value = Z) statement dynamically from the list of items to match. I haven't found a good example of building expression trees to do this.
Normally I would query this way:
var names = new string[] { "name1", "name2", "name3" }
var matches = query.Where(x => names.Contains(x.Name));
So far the closest thing I could find was to use the
Dynamic Linq Library and build a string to be interpreted but it feels a bit too hacky.
You don't even need an external library for something like this:
var names = new string[] { "name1", "name2", "name3" };
// Where MyClass is the type of your class
ParameterExpression par = Expression.Parameter(typeof(MyClass));
MemberExpression prop = Expression.Property(par, "Name");
Expression expression = null;
foreach (string name in names)
{
Expression expression2 = Expression.Equal(prop, Expression.Constant(name));
if (expression == null)
{
expression = expression2;
}
else
{
expression = Expression.OrElse(expression, expression2);
}
}
var query = ...; // Your query
if (expression != null)
{
// Where MyClass is the type of your class
var lambda = Expression.Lambda<Func<MyClass, bool>>(expression, par);
query = query.Where(lambda);
}
You can build a concatenation of Expression.OrElse, with the comparisons between the property Name and one of the strings.
In this particular case (3 strings), the resulting Expression, when looked from the debugger, is:
(((Param_0.Name == "name1") OrElse (Param_0.Name == "name2")) OrElse (Param_0.Name == "name3"))
Just improving the answer by xanatos:
var names = new string[] { "name1", "name2", "name3" };
var query = ...; // Your query
if (names.Any())
{
// Where MyClass is the type of your class
ParameterExpression par = Expression.Parameter(typeof(MyClass));
MemberExpression prop = Expression.Property(par, "Name");
var expression=names
.Select(v => Expression.Equal(prop, Expression.Constant(v)))
.Aggregate(Expression.OrElse);
// Where MyClass is the type of your class
var lambda = Expression.Lambda<Func<MyClass, bool>>(expression, par);
query = query.Where(lambda);
}
Related
The following code works for me if I need to filter a generic query for a single parameter, like Status=1.
public static IQueryable<T> FilterBy<T>(this IQueryable<T> query)
{
var propertyName = "Status";
var param_1 = "1";
//var param_2 = 2;
//var param_3 = "DE";
var parameterExp = Expression.Parameter(typeof(T), "type");
var propertyExp = Expression.Property(parameterExp, propertyName);
var converter = TypeDescriptor.GetConverter(propertyExp.Type);
var result = converter.ConvertFrom(param_1);
var value = Expression.Constant(result);
var equalBinaryExp = Expression.Equal(propertyExp, value);
query = query.Where(Expression.Lambda<Func<T, bool>>(equalBinaryExp, parameterExp));
return query;
}
This is like filtering a list with a value.
var list = new List<Employee> {
new Employee { Id = 1, Name = "Phil", Status=1, Country="DE" } ,
new Employee { Id = 2, Name = "Kristina", Status=2, Country="DE" },
new Employee { Id = 3, Name = "Mia", Status=1, Country="US" }
};
list = list.Where(x => (x.Status == 1)).ToList();
But how should I modify the method FilterBy, so that it will work for filtering multiple values? Like (Status=1 or Status=2) and Country="DE",
list = list.Where(x => (x.Status == 1 || x.Status == 2) && x.Country == "DE").ToList();
Thanks for your time.
The trick when writing lambdas by hand is to write them how you would in regular C#, and decompile them, perhaps using sharplab like this. If you look on the right, you can see that it is doing something like:
Expression.AndAlso(
Expression.OrElse(
Expression.Equal(idPropExp, Expression.Constant(1)),
Expression.Equal(idPropExp, Expression.Constant(2))
),
Expression.Equal(countryPropExp, Expression.Constant("DE"))
)
I try to perform a simple LIKE action on the database site, while having query building services based on generic types. I found out while debugging however, that performing EF.Functions.Like() with reflection does not work as expected:
The LINQ expression 'where __Functions_0.Like([c].GetType().GetProperty("FirstName").GetValue([c], null).ToString(), "%Test%")' could not be translated and will be evaluated locally..
The code that makes the difference
That works:
var query = _context.Set<Customer>().Where(c => EF.Functions.Like(c.FirstName, "%Test%"));
This throws the warning & tries to resolve in memory:
var query = _context.Set<Customer>().Where(c => EF.Functions.Like(c.GetType().GetProperty("FirstName").GetValue(c, null).ToString(), "%Test%"));
Does the Linq query builder or the EF.Functions not support reflections?
Sorry if the questions seem basic, it's my first attempt with .NET Core :)
In EF the lambdas are ExpressionTrees and the expressions are translated to T-SQL so that the query can be executed in the database.
You can create an extension method like so:
public static IQueryable<T> Search<T>(this IQueryable<T> source, string propertyName, string searchTerm)
{
if (string.IsNullOrEmpty(propertyName) || string.IsNullOrEmpty(searchTerm))
{
return source;
}
var property = typeof(T).GetProperty(propertyName);
if (property is null)
{
return source;
}
searchTerm = "%" + searchTerm + "%";
var itemParameter = Parameter(typeof(T), "item");
var functions = Property(null, typeof(EF).GetProperty(nameof(EF.Functions)));
var like = typeof(DbFunctionsExtensions).GetMethod(nameof(DbFunctionsExtensions.Like), new Type[] { functions.Type, typeof(string), typeof(string) });
Expression expressionProperty = Property(itemParameter, property.Name);
if (property.PropertyType != typeof(string))
{
expressionProperty = Call(expressionProperty, typeof(object).GetMethod(nameof(object.ToString), new Type[0]));
}
var selector = Call(
null,
like,
functions,
expressionProperty,
Constant(searchTerm));
return source.Where(Lambda<Func<T, bool>>(selector, itemParameter));
}
And use it like so:
var query = _context.Set<Customer>().Search("FirstName", "Test").ToList();
var query2 = _context.Set<Customer>().Search("Age", "2").ToList();
For reference this was the Customer I used:
public class Customer
{
[Key]
public Guid Id { get; set; }
public string FirstName { get; set; }
public int Age { get; set; }
}
Simple answer, no.
EntityFramework is trying to covert your where clause in to a SQL Query. There is no native support for reflection in this conversation.
You have 2 options here. You can construct your text outside of your query or directly use property itself. Is there any specific reason for not using something like following?
var query = _context.Set<Customer>().Where(c => EF.Functions.Like(c.FirstName, "%Test%"));
Keep in mind that every ExpresionTree that you put in Where clause has to be translated into SQL query.
Because of that, ExpressionTrees that you can write are quite limited, you have to stick to some rules, thats why reflection is not supported.
Image that instead of :
var query = _context.Set<Customer>().Where(c => EF.Functions.Like(c.GetType().GetProperty("FirstName").GetValue(c, null).ToString(), "%Test%"));
You write something like:
var query = _context.Set<Customer>().Where(c => EF.Functions.Like(SomeMethodThatReturnsString(c), "%Test%"));
It would mean that EF is able to translate any c# code to SQL query - it's obviously not true :)
I chucked together a version of the accepted answer for those using NpgSQL as their EF Core provider as you will need to use the ILike function instead if you want case-insensitivity, also added a second version which combines a bunch of properties into a single Where() clause:
public static IQueryable<T> WhereLike<T>(this IQueryable<T> source, string propertyName, string searchTerm)
{
// Check property name
if (string.IsNullOrEmpty(propertyName))
{
throw new ArgumentNullException(nameof(propertyName));
}
// Check the search term
if(string.IsNullOrEmpty(searchTerm))
{
throw new ArgumentNullException(nameof(searchTerm));
}
// Check the property exists
var property = typeof(T).GetProperty(propertyName);
if (property == null)
{
throw new ArgumentException($"The property {typeof(T)}.{propertyName} was not found.", nameof(propertyName));
}
// Check the property type
if(property.PropertyType != typeof(string))
{
throw new ArgumentException($"The specified property must be of type {typeof(string)}.", nameof(propertyName));
}
// Get expression constants
var searchPattern = "%" + searchTerm + "%";
var itemParameter = Expression.Parameter(typeof(T), "item");
var functions = Expression.Property(null, typeof(EF).GetProperty(nameof(EF.Functions)));
var likeFunction = typeof(NpgsqlDbFunctionsExtensions).GetMethod(nameof(NpgsqlDbFunctionsExtensions.ILike), new Type[] { functions.Type, typeof(string), typeof(string) });
// Build the property expression and return it
Expression selectorExpression = Expression.Property(itemParameter, property.Name);
selectorExpression = Expression.Call(null, likeFunction, functions, selectorExpression, Expression.Constant(searchPattern));
return source.Where(Expression.Lambda<Func<T, bool>>(selectorExpression, itemParameter));
}
public static IQueryable<T> WhereLike<T>(this IQueryable<T> source, IEnumerable<string> propertyNames, string searchTerm)
{
// Check property name
if (!(propertyNames?.Any() ?? false))
{
throw new ArgumentNullException(nameof(propertyNames));
}
// Check the search term
if (string.IsNullOrEmpty(searchTerm))
{
throw new ArgumentNullException(nameof(searchTerm));
}
// Check the property exists
var properties = propertyNames.Select(p => typeof(T).GetProperty(p)).AsEnumerable();
if (properties.Any(p => p == null))
{
throw new ArgumentException($"One or more specified properties was not found on type {typeof(T)}: {string.Join(",", properties.Where(p => p == null).Select((p, i) => propertyNames.ElementAt(i)))}.", nameof(propertyNames));
}
// Check the property type
if (properties.Any(p => p.PropertyType != typeof(string)))
{
throw new ArgumentException($"The specified properties must be of type {typeof(string)}: {string.Join(",", properties.Where(p => p.PropertyType != typeof(string)).Select(p => p.Name))}.", nameof(propertyNames));
}
// Get the expression constants
var searchPattern = "%" + searchTerm + "%";
var itemParameter = Expression.Parameter(typeof(T), "item");
var functions = Expression.Property(null, typeof(EF).GetProperty(nameof(EF.Functions)));
var likeFunction = typeof(NpgsqlDbFunctionsExtensions).GetMethod(nameof(NpgsqlDbFunctionsExtensions.ILike), new Type[] { functions.Type, typeof(string), typeof(string) });
// Build the expression and return it
Expression selectorExpression = null;
foreach (var property in properties)
{
var previousSelectorExpression = selectorExpression;
selectorExpression = Expression.Property(itemParameter, property.Name);
selectorExpression = Expression.Call(null, likeFunction, functions, selectorExpression, Expression.Constant(searchPattern));
if(previousSelectorExpression != null)
{
selectorExpression = Expression.Or(previousSelectorExpression, selectorExpression);
}
}
return source.Where(Expression.Lambda<Func<T, bool>>(selectorExpression, itemParameter));
}
I have written a ToList(); extension Method to convert a DataTable to List. This just works under some circumstances but we have much old code which uses DataTables and sometimes it's needed. My Problem is that this method works with reflection what is ok but not that performant. I need about 1,2sek for 100.000 DataRows.
So i decided to build this with Expression Trees. At first i want to replace the Setter Call of Properties. Up to this time i could easily get the value:
var exactType = Nullable.GetUnderlyingType(propType) ?? propType;
var wert = Convert.ChangeType(zeile[spaltenname], exactType);
and set it:
propertyInfo.SetValue(tempObjekt, wert, null);
Now i searched StackOverflow and found this:
var zielExp = Expression.Parameter(typeof(T));
var wertExp = Expression.Parameter(propType);
var propertyExp = Expression.Property(zielExp, matchProp);
var zuweisungExp = Expression.Assign(propertyExp, wertExp);
var setter = Expression.Lambda<Action<T, int>>(zuweisungExp, zielExp, wertExp).Compile();
setter(tempObjekt, wert);
My big Problem is that the Lambda Action expects an integer. But i need this expecting the type of my Property. I have the Type of my Property via PropertyInfo. But can't get this to work. Thought i can easily make:
Action<T, object>
but this results in following excepion:
ArgumentException The ParameterExpression from Type "System.Int32"
cannot be used as Delegateparameter from Type "System.Object".
Someone out there knows a possible solution?
Instead of the generic Expression.Lambda method you can use this overload which takes a type:
public static LambdaExpression Lambda(
Type delegateType,
Expression body,
params ParameterExpression[] parameters
)
Then you can use the Type.MakeGenericType method to create the type for your action:
var actionType = typeof(Action<,>).MakeGenericType(typeof(T), proptype);
var setter = Expression.Lambda(actionType, zuweisungExp, zielExp, wertExp).Compile();
Edit following the comments regarding performance:
You can also just build the expression runtime to map the DataTable to your class of type T with a select, so there's only need to use reflection once, which should greatly improve performance. I wrote the following extension method to convert a DataTable to List<T> (note that this method will throw a runtime exception if you don't plan to map all datacolumns to a property in the class, so be sure to take care of that if that might happen):
public static class LocalExtensions
{
public static List<T> DataTableToList<T>(this DataTable table) where T : class
{
//Map the properties in a dictionary by name for easy access
var propertiesByName = typeof(T)
.GetProperties(BindingFlags.Public | BindingFlags.Instance)
.ToDictionary(p => p.Name);
var columnNames = table.Columns.Cast<DataColumn>().Select(dc => dc.ColumnName);
//The indexer property to access DataRow["columnName"] is called "Item"
var property = typeof(DataRow).GetProperties().First(p => p.Name == "Item"
&& p.GetIndexParameters().Length == 1
&& p.GetIndexParameters()[0].ParameterType == typeof(string));
var paramExpr = Expression.Parameter(typeof(DataRow), "r");
var newExpr = Expression.New(typeof(T));
//Create the expressions to map properties from your class to the corresponding
//value in the datarow. This will throw a runtime exception if your class
//doesn't contain properties for all columnnames!
var memberBindings = columnNames.Select(columnName =>
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
//Datarow["columnName"] is of type object, cast to the right type
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
});
var initExpr = Expression.MemberInit(newExpr, memberBindings);
var func = Expression.Lambda<Func<DataRow, T>>(initExpr,paramExpr).Compile();
return table.Rows.Cast<DataRow>().Select(func).ToList();
}
}
Then I wrote a small testclass and some code which creates a datatable of 1,000,000 rows that get mapped to a list. Building the expression + converting to a list now only takes 486ms on my pc (granted it is a very small class of course):
class Test
{
public string TestString { get; set; }
public int TestInt { get; set; }
}
class Program
{
static void Main()
{
DataTable table = new DataTable();
table.Columns.Add(new DataColumn("TestString", typeof(string)));
table.Columns.Add(new DataColumn("TestInt", typeof(int)));
for(int i = 0; i < 1000000; i++)
{
var row = table.NewRow();
row["TestString"] = $"String number: {i}";
row["TestInt"] = i;
table.Rows.Add(row);
}
var stopwatch = Stopwatch.StartNew();
var myList = table.DataTableToList<Test>();
stopwatch.Stop();
Console.WriteLine(stopwatch.Elapsed.ToString());
}
}
I think I've understood you correctly. I cannot translate your variables so I'm taking my best guess here based on what I'm seeing in your question:
For an Action<object,object> where the first parameter is the Entity itself and the second is the type of the property you can use
var instance = Expression.Parameter(typeof(object), "i");
var argument = Expression.Parameter(typeof(object), "a");
var convertObj = Expression.TypeAs(instance, propertyInfo.DeclaringType);
var convert = Expression.Convert(argument, propertyInfo.PropertyType);
var setterCall = Expression.Call(convertObj, propertyInfo.GetSetMethod(), convert);
var compiled = ((Expression<Action<object, object>>) Expression.Lambda(setterCall, instance, argument)).Compile();
If you know T (ie, the type of the Entity), you can do this instead:
var instance = Expression.Parameter(typeof(T), "i");
var argument = Expression.Parameter(typeof(object), "a");
var convert = Expression.Convert(argument, propertyInfo.PropertyType);
var setterCall = Expression.Call(instance , propertyInfo.GetSetMethod(), convert);
var compiled = ((Expression<Action<T, object>>) Expression.Lambda(setterCall, instance, argument)).Compile();
I comment here because I do not have the necessary reputation to comment on the response of #Alexander Derek
var memberBindings = columnNames.Select(columnName =>
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
//Datarow["columnName"] is of type object, cast to the right type
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
});
in order to avoid runtime exception i added a try-catch and .where()
var memberBindings = columnNames.Select(columnName =>
{
try
{
var pi = propertiesByName[columnName];
var indexExpr = Expression.MakeIndex(paramExpr, property,
new[] { Expression.Constant(columnName) });
var convert = Expression.Convert(indexExpr, pi.PropertyType);
return Expression.Bind(pi, convert);
}
catch(Exception e)
{
return null;
}
});
var initExpr = Expression.MemberInit(newExpr, memberBindings.Where(obj => obj != null));
I'm trying to generate a LINQ OrderBy clause using lambda expressions with an input of the column name of an entity as a string (in the "sortOn" variable below).
The code below works fine for a sortOn value like "Code" generating the lambda
p => p.Code
But I would also like to sort on a child entity, where the lambda might be
p => p.Category.Description
So in this instance, I would just like to set sortOn = "Category.Description" and have the correct lamdba expression generated.
Is this possible? Any suggestions about the best way to do this would be welcomed.
This code works fine for the simple case:
var param = Expression.Parameter(typeof (Product), "p");
var sortExpression = Expression.Lambda<Func<Product, object>>(
Expression.Property(param, sortOn), param);
if (sortAscending ?? true)
{
products = products.OrderBy(sortExpression);
}
else
{
products = products.OrderByDescending(sortExpression);
}
The use-case for this problem is displaying a grid of data and being able to sort the data, simply by passing the column name to be sorted on back to the server. I'd like to make the solution generic, but have started using a particular type (Product in the example) for now.
This will generate proper lambda expression:
var sortOn = "Category.Description";
var param = Expression.Parameter(typeof(Product), "p");
var parts = sortOn.Split('.');
Expression parent = param;
foreach (var part in parts)
{
parent = Expression.Property(parent, part);
}
var sortExpression = Expression.Lambda<Func<Product, object>>(parent, param);
Here is an extension OrderBy method which works for any number of nested parameters.
public static IQueryable<T> OrderBy<T>(this IQueryable<T> query, string key, bool asc = true)
{
try
{
string orderMethodName = asc ? "OrderBy" : "OrderByDescending";
Type type = typeof(T);
Type propertyType = type.GetProperty(key)?.PropertyType; ;
var param = Expression.Parameter(type, "x");
Expression parent = param;
var keyParts = key.Split('.');
for (int i = 0; i < keyParts.Length; i++)
{
var keyPart = keyParts[i];
parent = Expression.Property(parent, keyPart);
if (keyParts.Length > 1)
{
if (i == 0)
{
propertyType = type.GetProperty(keyPart).PropertyType;
}
else
{
propertyType = propertyType.GetProperty(keyPart).PropertyType;
}
}
}
MethodCallExpression orderByExpression = Expression.Call(
typeof(Queryable),
orderMethodName,
new Type[] { type, propertyType },
query.Expression,
CreateExpression(type, key)
);
return query.Provider.CreateQuery<T>(orderByExpression);
}
catch (Exception e)
{
return query;
}
}
The CreateExpression method which is used in my solution is defined in this post.
The usage of the OrderBy extension method is as follows.
IQueryable<Foo> q = [Your database context].Foos.AsQueryable();
IQueryable<Foo> p = null;
p = q.OrderBy("myBar.name"); // Ascending sort
// Or
p = q.OrderBy("myBar.name", false); // Descending sort
// Materialize
var result = p.ToList();
The type Foo and its properties are also taken from the same post as method CreateExpression.
Hope you find this post helpful.
You can use the Dynamic LINQ Query Library to do this easily. Assuming you have an IQueryable<T> implementation of Product, you can easily do:
IQueryable<Product> products = ...;
// Order by dynamically.
products = products.OrderBy("Category.Description");
The blog post has a link to the libary, and you'll have to build/include the project in your solution yourself, but it works very well, and the parsing is very robust. It prevents you from having to write the parsing code yourself; even for something so simple, if the requirements expand, the library has you covered, whereas a homegrown solution does not.
It also has a number of other dynamic operators (Select, Where, etc.) so you can perform other dynamic operations.
There's no magic under the hood, it just parses the strings you pass it and then creates the lambda expressions based on the parsing results.
If you don't need expressions, how about:
products = products.Orderby(p1 => p1.Code).ThenBy(p2 => p2.Category.Description)
Hi you can also create an extension method like which can sort to any depth not only just child
public static IEnumerable<TSource> CustomOrderBy<TSource, TKey>(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
List<string> list=new List<string>();
List<TSource> returnList=new List<TSource>();
List<int> indexList = new List<int>();
if (source == null)
return null;
if (source.Count() <= 0)
return source;
source.ToList().ForEach(sc=>list.Add(keySelector(sc).ToString())); //Extract the strings of property to be ordered
list.Sort(); //sort the list of strings
foreach (string l in list) // extract the list of indexes of source according to the order
{
int i=0;
//list.ForEach(l =>
foreach (var s in source.ToList())
{
if (keySelector(s).ToString() == l)
break;
i++;
}
indexList.Add(i);
}
indexList.ForEach(i=>returnList.Add(source.ElementAt(i))); //rearrange the source according to the above extracted indexes
return returnList;
}
}
public class Name
{
public string FName { get; set; }
public string LName { get; set; }
}
public class Category
{
public Name Name { get; set; }
}
public class SortChild
{
public void SortOn()
{
List<Category> category = new List<Category>{new Category(){Name=new Name(){FName="sahil",LName="chauhan"}},
new Category(){Name=new Name(){FName="pankaj",LName="chauhan"}},
new Category(){Name=new Name(){FName="harish",LName="thakur"}},
new Category(){Name=new Name(){FName="deepak",LName="bakseth"}},
new Category(){Name=new Name(){FName="manish",LName="dhamaka"}},
new Category(){Name=new Name(){FName="arev",LName="raghaka"}}
};
var a = category.CustomOrderBy(s => s.Name.FName);
}
}
Its custom method and right now it works only for string property only however it can be reactified using generics to work for any primitive type. I hope this will help.
I was trying to generate a simple Lambda Expression at runtime with no luck... something like this:
var result = queryableData.Where(item => item.Name == "Soap")
Here is my example class and a fixture queryable:
public class Item
{
public int Id { get; set; }
public string Name { get; set; }
}
IQueryable<Item> queryableData = ...;
Then I generate a lambda expression at runtime correct code follows:
//"item" in "item =>..."
var item = Expression
.Parameter(typeof(Item), "item");
//property of my item, this is "item.Name"
var prop = Expression
.Property(item, "Name");
//then "Soap" in '... => item.Name=="Soap"'
var value = Expression.Constant("Soap");
//equality expression "==" in my primer
var equals = Expression.Equal(prop, value);
//then lambda
var lambda = Expression.Lambda<Func<Item, bool>>(equals, item);
//and here are the results
var results = queryableData.Where(lambda);
Big thanks to dtb for advice!
In the following query
var result = query.Where(item => item.Name == "Soap")
the lambda expression is
item => item.Name == "Soap"
You only need to construct this part, not the Where call which accepts an expression tree.
The expression tree for the lambda expression looks like this:
Lambda
/ \
Equal Parameter
/ \ item
Property \
"Name" Constant
| "Soap"
Parameter
item
In code:
var item = Expression.Parameter(typeof(Item), "item");
var prop = Expression.Property(item, "Name");
var soap = Expression.Constant("Soap");
var equal = Expression.Equal(prop, soap);
var lambda = Expression.Lambda<Func<Item, bool>>(equal, item);
var result = queryableData.Where(lambda);