I have a task when the same table is used to work with different data which similar content. I use dynamic linq expressions to get data from sql.
The most complicated task for me was to choose distinct rows based on one field. The one of the tables is Parts, and I will show it as the sample.
I need to select the first part from db with each Description.
Sql request is:
SELECT * FROM [Parts] WHERE [Id] IN (SELECT MIN([Id]) FROM[Parts] GROUP BY [Description])
LINQ request is:
queryable = queryable.GroupBy(r => r.Dewcription).Select(g => g.FirstOrDefault());
I should rewrite LINQ query to expressions. I have successed with GROUP BY:
var grouped = queryable.GroupBy(SelectedField, type); // using
public static IQueryable GroupBy([NotNull] this IQueryable source, string Property, [NotNull] Type type)
{
PropertyInfo property = type.GetProperty(Property);
ParameterExpression parameter = Expression.Parameter(type, "r");
MemberExpression member = Expression.MakeMemberAccess(parameter, property);
LambdaExpression lambda = Expression.Lambda(member, parameter);
MethodCallExpression resultExpression = GroupByCall(source, lambda, property.PropertyType, type);
return source.Provider.CreateQuery(resultExpression);
}
// to apply Group by method
public static MethodCallExpression GroupByCall([NotNull] IQueryable queryable, LambdaExpression predicate, Type typeKey, Type typeSource)
{
MethodInfo MethodGroupBy = typeof(Queryable).GetMethods(BindingFlags.Static | BindingFlags.Public)
.First(m => m.Name.Equals("GroupBy", StringComparison.OrdinalIgnoreCase) && m.GetParameters().Length == 2).MakeGenericMethod(typeSource, typeKey);
Type QueryableType = typeSource.QueryableType();
return Expression.Call(null, MethodGroupBy, Expression.Convert(queryable.Expression, QueryableType), Expression.Quote(predicate));
}
// to get type of Queryable to convert from Queryable
public static Type QueryableType(this Type type)
{
var list = Activator.CreateInstance(typeof(List<>).MakeGenericType(type));
MethodInfo Method = typeof(Queryable).GetMethod(nameof(Queryable.AsQueryable), new[] { typeof(IEnumerable) });
var listAsQueryable = Method.Invoke(null, new[] { list });
return listAsQueryable.GetType();
}
The next step is to make expression for .Select(g => g.FirstOrDefault()). It is not possible to apply it directly, because the type of grouped entity is not known, and the result of grouping is IGrouping<string, object>, where object in fact is the type which we know on execution.
I need something like it:
var grouped = queryable.GroupBy(SelectedField, type).SelectGroupFirst(type);
public static IQueryable SelectGroupFirst([NotNull] this IQueryable source, [NotNull] Type type)
{
ParameterExpression parameter = Expression.Parameter(type, "r");
MemberExpression member = Expression.MakeMemberAccess(parameter, ...get enumerable...);
...apply First() or FirstOrDefault()...
LambdaExpression lambda = Expression.Lambda(member, parameter);
MethodCallExpression resultExpression = SelectCall(source, lambda, ...IGrouping<string, type>..., type);
return source.Provider.CreateQuery(resultExpression);
}
public static MethodCallExpression SelectCall(IQueryable queryable, LambdaExpression predicate, Type typeKey, Type typeSource)
{
if (queryable == null) return null;
var methods = typeof(Queryable).GetMethods(BindingFlags.Static | BindingFlags.Public)
.Where(m => m.Name.Equals("Select", StringComparison.OrdinalIgnoreCase));
MethodInfo MethodSelect = typeof(Queryable).GetMethods(BindingFlags.Static | BindingFlags.Public)
.First(m => m.Name.Equals("Select", StringComparison.OrdinalIgnoreCase) && m.GetParameters().Length == 2).MakeGenericMethod(typeSource, typeKey);
return Expression.Call(null, MethodSelect, queryable.Expression, Expression.Quote(predicate));
}
I have not found the way to make type definition of IGrouping<string, type> dynamically, to define the group enumeration property and to get access to it.
Alternative could be the way to mix usage of SQL and LINQ. Is there the way to use the above mentioned sql request for first selection of rows, and to modify it with LINQ for paging and filtering? I have not found any.
Related
There's the problem I made an Extension method to IQueryable and I want to get property name and value to do a select on it the thing is that I am using Expression tree and try to search rows by the use of Contain() method which is needs a parameter of type string to do the job. my code is doing well while I do the search over string properties but I want this method to do the search on other types like int, decimal, datetime and even booleans so I need to get the property cast it to string and search for the matching values.
what I tried so far is:
private static readonly MethodInfo _tostring = typeof(Object).GetMethod("ToString") ?? throw new Exception("Cannot create Method");
public static IQueryable<T> Search<T>(this IQueryable<T> items, string propertyName, string filterValue)
{
MethodInfo _compare =
(((Expression<Func<string, bool>>)(s => s.Contains("aa"))).Body as MethodCallExpression ?? throw new Exception("Cannot Create Method"))
.Method;
var property = typeof(T).GetProperty(propertyName) ?? throw new Exception("Couldn't Get the property");
var row = Expression.Parameter(typeof(T), "row");
Expression prop = Expression.Property(row, property);
// now making sure if the type is string
if (property.PropertyType != typeof(string))
{
//Here I want to cast it to string but the problem is exactly here anything I try feels like a dead end
prop = Expression.Call(prop, _tostring);
}
var func =
Expression.Lambda<Func<T, bool>>
(
Expression.Call
(
prop,
_compare,
Expression.Constant(filterValue)
),
row
);
return items.Where(func);
}
I tried and tested it with some dummy objects; it seems to work.
I refined my answer to use ToString
var prop = Expression.Property(row, property);
Expression expression = prop;
if (prop.Type != typeof(string))
{
expression = Expression.Call(prop, "ToString", Type.EmptyTypes);
}
var func = Expression.Lambda<Func<T, bool>>
(
Expression.Call
(
expression,
_compare,
Expression.Constant(filterValue)
),
row
);
The above expression tree will be compiled to,
If the property type is not a string,
.Where(s => s.property.ToString().Contains("filter")))
If it's a string type,
.Where(s => s.property.Contains("filter")))
I have a linq query which is as follows:
var query = _appDbContext.Persons
.Where(p => p.Id == 123)
.Select(lambda)
.ToList();
I use lambda because i have a variable in my select statement. The lambda:
var wantedData = "Adress";
var x = Expression.Parameter(typeof(Person), "x");
var body = Expression.PropertyOrField(x, wantedData);
var lambda = Expression.Lambda<Func<Person, Adress>>(body, x);
The query will return the addresses for person with id 123. Let's say instead of the addresses I would like to recieve all subscriptions this person has. Only setting wantedData = "Subscription" will not work, because of the "Adress" in the lambda statement. So I'm looking for a way to use a variable to change "Adress" in the lambda statement.
I tried the following, which obviously does not work.
var lambda = Expression.Lambda<Func<Person, wantedData>>(body, x);
Is there a way to do this?
Ok, I managed to create a generic query method that takes an Expression as parameter.
EDIT: This will receive a string value and will try to match it with an existing property on your Person class.
private IEnumerable<TResult> GetPersonData<TResult>(Expression<Func<Person, TResult>> selectExpression)
{
return _dbContext.Persons
// .Where(filter)
.Select(selectExpression)
.ToList();
}
public IEnumerable<object> GetData(string dataPropertyName)
{
switch(dataPropertyName) {
case nameof(Person.Address): return GetPersonData(p => p.Address);
case nameof(Person.Subscription): return GetPersonData(p => p.Subscription);
// other cases
default: throw new InvalidArgumentException("Invalid property name");
}
}
note that this code is just an example written on the spot and it might not work directly with copy-paste
I've made something similar to an OrderBy and tweak it a little for your select. This only works if you know the return type.
You can create an extension method with the following code:
public static IQueryable<TResult> Select<T,TResult>(this IQueryable<T> source, string propertyName)
{
var prop = typeof(T).GetProperties()
.FirstOrDefault(x => x.Name.ToUpper() == propertyName.ToUpper());
ParameterExpression pe = Expression.Parameter(typeof(T), "x");
MemberExpression body = Expression.Property(pe, prop.Name);
LambdaExpression lambda = Expression.Lambda(body, new ParameterExpression[] { pe });
MethodCallExpression selectCallExpression = Expression.Call(
typeof(Queryable),
"Select",
new Type[] { source.ElementType, prop.PropertyType },
source.Expression,
lambda);
return source.Provider.CreateQuery<TResult>(selectCallExpression);
}
That way, you can use it like:
var query = _appDbContext.Persons
.Where(p => p.Id == 123)
.Select<Person,string>(wantedData)
.ToList();
I'm trying to implement a dynamic filter in a generic repository (.NET Core 3.1 + EF Core 3.1)
by building an Expression Tree, but the generated SQL query is never parameterized (I'm verifying the generated query via "Microsoft.EntityFrameworkCore.Database.Command": "Information" in appsettings.json and have EnableSensitiveDataLogging in Startup.cs)
The code to build an Expression Tree is the following (for sake of simplicity working with string values only here):
public static IQueryable<T> WhereEquals<T>(IQueryable<T> query, string propertyName, object propertyValue)
{
var pe = Expression.Parameter(typeof(T));
var property = Expression.PropertyOrField(pe, propertyName);
var value = Expression.Constant(propertyValue);
var predicateBody = Expression.Equal(
property,
value
);
var whereCallExpression = Expression.Call(
typeof(Queryable),
"Where",
new[] { typeof(T) },
query.Expression,
Expression.Lambda<Func<T, bool>>(predicateBody, new ParameterExpression[] { pe })
);
return query.Provider.CreateQuery<T>(whereCallExpression);
}
The approach works, but values are always incorporated inside a generated SQL query and I afraid that it could lead to SQL injections.
Here is an example of a generated query:
Microsoft.EntityFrameworkCore.Database.Command: Information: Executed DbCommand (33ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
SELECT [p].[Id], [p].[Name], [p].[FirstName], [p].[Created], [p].[CreatedBy], [p].[Updated], [p].[UpdatedBy]
FROM [Persons] AS [p]
WHERE [p].[Name] = N'smith'
Found a potential answer from a EF team member (#divega):
Force Entity Framework to use SQL parameterization for better SQL proc cache reuse,
managed it to work with Where method, but the generated SQL is still the same.
Tried to use System.Linq.Dynamic.Core,
but it has the same issue (generated SQL query is not parameterized).
Is there a way to force Entity Framework Core to generate a parameterized query from an Expression Tree?
The link you provided explains that EF uses a SQL parameter for variable values, so instead of creating an Expression.Constant for the value passed in, if you create a variable reference (which in C# is always a field reference), then you will get a parameterized query. The simplest solution seems to be to copy how the compiler handles a lambda outer scope variable reference, which is create a class object to hold the value, and reference that.
Unlike Expression.Constant, it isn't easy to get the actual type of the object parameter, so changing that to a generic type:
public static class IQueryableExt {
private sealed class holdPropertyValue<T> {
public T v;
}
public static IQueryable<T> WhereEquals<T, TValue>(this IQueryable<T> query, string propertyName, TValue propertyValue) {
// p
var pe = Expression.Parameter(typeof(T), "p");
// p.{propertyName}
var property = Expression.PropertyOrField(pe, propertyName);
var holdpv = new holdPropertyValue<TValue> { v = propertyValue };
// holdpv.v
var value = Expression.PropertyOrField(Expression.Constant(holdpv), "v");
// p.{propertyName} == holdpv.v
var whereBody = Expression.Equal(property, value);
// p => p.{propertyName} == holdpv.v
var whereLambda = Expression.Lambda<Func<T, bool>>(whereBody, pe);
// Queryable.Where(query, p => p.{propertyName} == holdpv.v)
var whereCallExpression = Expression.Call(
typeof(Queryable),
"Where",
new[] { typeof(T) },
query.Expression,
whereLambda
);
// query.Where(p => p.{propertyName} == holdpv.v)
return query.Provider.CreateQuery<T>(whereCallExpression);
}
}
If you need to pass in an object instead, it is simpler to add a conversion to the proper type (which won't affect the generated SQL), rather than dynamically create the right type of holdPropertyValue and assign it a value, so:
public static IQueryable<T> WhereEquals2<T>(this IQueryable<T> query, string propertyName, object propertyValue) {
// p
var pe = Expression.Parameter(typeof(T), "p");
// p.{propertyName}
var property = Expression.PropertyOrField(pe, propertyName);
var holdpv = new holdPropertyValue<object> { v = propertyValue };
// Convert.ChangeType(holdpv.v, p.{propertyName}.GetType())
var value = Expression.Convert(Expression.PropertyOrField(Expression.Constant(holdpv), "v"), property.Type);
// p.{propertyName} == Convert.ChangeType(holdpv.v, p.{propertyName}.GetType())
var whereBody = Expression.Equal(property, value);
// p => p.{propertyName} == Convert.ChangeType(holdpv.v, p.{propertyName}.GetType())
var whereLambda = Expression.Lambda<Func<T, bool>>(whereBody, pe);
// Queryable.Where(query, p => p.{propertyName} == Convert.ChangeType(holdpv.v, p.{propertyName}.GetType()))
var whereCallExpression = Expression.Call(
typeof(Queryable),
"Where",
new[] { typeof(T) },
query.Expression,
whereLambda
);
// query.Where(query, p => p.{propertyName} == Convert.ChangeType(holdpv.v, p.{propertyName}.GetType()))
return query.Provider.CreateQuery<T>(whereCallExpression);
}
I want to create a dynamic linq expression for sql IN clause in EF 6.0 with code first approch. Note that i am new to Expressions. What i want to achive is
select * from Courses where CourseId in (1, 2, 3, 4)
//CourseId is integer
The normal linq query looks like this. But i want to query it dynamically
string[] ids = new string[]{"1", "2", "3", "4"};
var courselist = DBEntities.Courses.Where(c => ids.Contains(SqlFunctions.StringConvert((decimal?)c.CourseId)))
There are two ways to make dynamic expression.
1) one ways is to loop through ids and make expressions
The below code will create the following expression in debug view
{f => ((StringConvert(Convert(f.CourseId)).Equals("23") Or StringConvert(Convert(f.CourseId)).Equals("2")) Or StringConvert(Convert(f.CourseId)).Equals("1"))}
Dynamic Expression is
var param = Expression.Parameters(typeof(Course), "f")
MemberExpression property = Expression.PropertyOrField(param, "CourseId");
MethodInfo mi = null;
MethodCallExpression mce = null;
if (property.Type == typeof(int))
{
var castProperty = Expression.Convert(property, typeof(double?));
var t = Expression.Parameter(typeof(SqlFunctions), "SqlFunctions");
mi = typeof(SqlFunctions).GetMethod("StringConvert", new Type[] { typeof(double?) });
mce = Expression.Call(null,mi, castProperty);
}
mi = typeof(string).GetMethod("Equals", new Type[]{ typeof(string)});
BinaryExpression bex = null;
if (values.Length <= 1)
{
return Expression.Lambda<Func<T, bool>>(Expression.Call(mce, mi, Expression.Constant(values[0]), param));
}
//var exp1 = Expression.Call(mce, mi, Expression.Constant(values[0]));
for (int i = 0; i < values.Length; i++)
{
if (bex == null)
{
bex = Expression.Or(Expression.Call(mce, mi, Expression.Constant(values[i])), Expression.Call(mce, mi, Expression.Constant(values[i + 1])));
i++;
}
else
bex = Expression.Or(bex, Expression.Call(mce, mi, Expression.Constant(values[i])));
}//End of for loop
return Expression.Lambda<Func<T, bool>>(bex, param);
2) The 2nd way that i tried (debug view)
{f => val.Contains("23")} //val is parameter of values above
The dynamic expression for above that i tried is
var param = Expression.Parameters(typeof(Course), "f")
MemberExpression property = Expression.PropertyOrField(param, "CourseId");
var micontain = typeof(Enumerable).GetMethods().Where(m => m.Name == "Contains" && m.GetParameters().Length == 2).Single().MakeGenericMethod(typeof(string));
var mc = Expression.Call(micontain, Expression.Parameter(values.GetType(), "val"), Expression.Constant("2"));//NOTE: I haven't use CourseId for now as i am getting conversion error
return Expression.Lambda<Func<T, bool>>(mc, param);
I get the following errors
LINQ to Entities does not recognize the method 'System.String StringConvert(System.Nullable`1[System.Double])' method, and this
method cannot be translated into a store expression when i use the
first methodology. I know i can't use ToString with EF thats why I used SqlFunctions but it is not working for me.
The parameter 'val' was not bound in the specified LINQ to Entities query expression using 2nd methodology
I am trying this from last 4 days. I googled it but didn't find any suitable solution. Please help me.
After a lot of struggle I found solution to my question.
I want to achieve this sql query
select * from Courses where CourseId in (1, 2, 3, 4)
Using Linq to Entities, but I want to pass in(1,2,3,4) list dynamically to linq query. I created an Extension class for that purpose.
public static class LinqExtensions
{
public static Expression<Func<T, bool>> False<T>() { return f => false; }
public static Expression<Func<T, bool>> In<T, TValue>(this Expression<Func<T, bool>> predicate,string propertyName, List<TValue> values)
{
var param = predicate.Parameters.Single();
MemberExpression property = Expression.PropertyOrField(param, propertyName);
var micontain = typeof(List<TValue>).GetMethod("Contains");
var mc = Expression.Call(Expression.Constant(values), micontain, property);
return Expression.Lambda<Func<T, bool>>(mc, param);
}
}
Use of LinqExtensions
var pred = LinqExtensions.False<Course>(); //You can chain In function like LinqExtensions.False<Course>().In<Course, int>("CourseId", inList);
var inList= new List<int>(){1, 2, 3}; //Keep in mind the list must be of same type of the Property that will be compared with. In my case CourseId is integer so the in List have integer values
pred =pred.In<Course, int>("CourseId", inList); //TValue is int. As CourseId is of type int.
var data = MyEntities.Courses.Where(pred);
I hope this might be beneficial for some one
have you seen the type of
var courselist = DBEntities.Courses.Where(c => ids.Contains(c.CourseId)))
above statement would not return actual list of courses. The query is not executed yet. It just returns IQuereable. The query is executed when you actually call .ToList() method on it
so, your solution is..
Create array of IDs using for loop and then simply run the below query
var courselist = DBEntities.Courses.Where(c => ids.Contains(c.CourseId))).ToList()
I started with the IQueryable extension methods from this example on CodePlex.
What i believe i need is an IQueryable extension method to "Where", where the method signature looks like:
public static IQueryable<T> Where<T>(this IQueryable<T> source, string columnName, string keyword)
and effectively does this (assuming T.columnName is of type string):
source.Where(p => p.ColumnName.Contains("keyword"))
using the above CodePlex example, i think i understand how he got the OrderBy method working, but my problem seems a bit more complex and I don't know how to get the Contains("keyword") part working.
Thanks in advance,
--Ed
Update: 9/13/2010 6:26pm PST
I thought the following would work, but end up getting a NotSupportedException (The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.) when I execute the expression via Count(). Any ideas?
public static IQueryable<T> Where<T>(this IQueryable<T> source, string columnName, string keyword)
{
var type = typeof(T);
var property = type.GetProperty(columnName);
if (property.PropertyType == typeof(string))
{
var parameter = Expression.Parameter(type, "p");
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
var sel = Expression.Lambda<Func<T, string>>(propertyAccess, parameter);
var compiledSel = sel.Compile();
return source.Where(item => compiledSel(item).Contains(keyword));
}
else
{
return source;
}
}
public static IQueryable<T> Where<T>(
this IQueryable<T> source, string columnName, string keyword)
{
var arg = Expression.Parameter(typeof(T), "p");
var body = Expression.Call(
Expression.Property(arg, columnName),
"Contains",
null,
Expression.Constant(keyword));
var predicate = Expression.Lambda<Func<T, bool>>(body, arg);
return source.Where(predicate);
}
Well a couple of years later, but if anybody still need this, here it is:
public static IQueryable<T> Has<T>(this IQueryable<T> source, string propertyName, string keyword)
{
if (source == null || propertyName.IsNull() || keyword.IsNull())
{
return source;
}
keyword = keyword.ToLower();
var parameter = Expression.Parameter(source.ElementType, String.Empty);
var property = Expression.Property(parameter, propertyName);
var CONTAINS_METHOD = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var TO_LOWER_METHOD = typeof(string).GetMethod("ToLower", new Type[] { });
var toLowerExpression = Expression.Call(property, TO_LOWER_METHOD);
var termConstant = Expression.Constant(keyword, typeof(string));
var containsExpression = Expression.Call(toLowerExpression, CONTAINS_METHOD, termConstant);
var predicate = Expression.Lambda<Func<T, bool>>(containsExpression, parameter);
var methodCallExpression = Expression.Call(typeof(Queryable), "Where",
new Type[] { source.ElementType },
source.Expression, Expression.Quote(predicate));
return source.Provider.CreateQuery<T>(methodCallExpression);
}
I called my method "Has" just to keep it short, sample usage:
filtered = filtered.AsQueryable().Has("City", strCity)
And you can concatenate to make it even more expressive:
filtered = filtered.AsQueryable().Has("Name", strName).Has("City", strCity).Has("State", strState);
By the way, the "IsNull()" attached to the strings is just another simple extension method:
public static Boolean IsNull(this string str)
{
return string.IsNullOrEmpty(str);
}
The .Contains("keyword") part is exactly right in your example.
It's the p.ColumnName part that's going to cause trouble.
Now, there are a number of ways of doing this, generally involving either reflection or Expression<>, neither of which is particularly efficent.
The problem here is by passing the column name as a string, you are doing to undo that exact thing LINQ was invented to allow.
However, there are probably better ways of accomplishing your overall task besides that way.
So, let's look at alternate ways:
You want to be able to say :
var selector = new Selector("Column1", "keyword");
mylist.Where(item => selector(item));
and have it was the equivalent of
mylist.Where(item=> item.Column1.Contains("keyword"));
How 'bout we go with:
Func<MyClass, string> selector = i => i.Column1;
mylist.Where(item => selector(item).Contains("keyword"));
or
Func<MyClass, bool> selector = i => i.Column1.Contains("keyword");
mylist.Where(item => selector(item));
These are easily expanded for alternatives:
Func<MyClass, string> selector;
if (option == 1)
selector = i => i.Column1;
else
selector = i => i.Column2;
mylist.Where(item => selector(item).Contains("keyword"));
See, in generics type of the object works dynamically. So when p.ColumnName is taken as string, the Contains of string is been executed.
Generally for any lambda expression you specify, it parses the thing into an Expression and ultimately produces the output at runtime.
If you see my post :
http://www.abhisheksur.com/2010/09/use-of-expression-trees-in-lamda-c.html
you will understand how it works actually.