Develop Reference

Dynamic GroupBy using expression in .net core

I wrote an extension method that makes an expression based on condition, but when the condition is groupby, the result is order by !!!
what am I wrong?
here is my method:
public static IQueryable<T> NewWhere<T, U>(this IQueryable<T> source, string prop, U value, string condition)
{
MethodInfo method;
Expression<Func<T, bool>> lambda = null;
Expression body = null;
string groupSelector = null;
var type = typeof(T);
var parameter = Expression.Parameter(type, "p");
var property = Expression.Property(parameter, prop);
var constant = Expression.Constant(value, typeof(U));
if (condition == "GreaterThan")
body = Expression.GreaterThan(property, constant);
else if (condition == "LessThan")
body = Expression.LessThan(property, constant);
else if (condition == "Equals")
body = Expression.Equal(property, constant);
//For implement sql like command we need them
else if (condition == "StartsWith") {
method = typeof(string).GetMethod("StartsWith", new[] { typeof(string) });
body = Expression.Call(property, method, constant);
}
else if (condition == "EndsWith")
{
method = typeof(string).GetMethod("EndsWith", new[] { typeof(string) });
body = Expression.Call(property, method, constant);
}
else if (condition == "Contains")
{
method = typeof(string).GetMethod("Contains", new[] { typeof(string) });
body = Expression.Call(property, method, constant);
}
//For implement sql like command we need them
//group by only one field
if (condition == "GroupBy")
groupSelector = prop;
else
lambda = Expression.Lambda<Func<T, bool>>(body, new[] { parameter });
//return the grouped by result or not grouped by
if (groupSelector != null)
{
var selectorExp = Expression.Lambda<Func<T, U>>(property, new ParameterExpression[] { parameter });
source = source.GroupBy(selectorExp).SelectMany(g => g);
//source.GroupBy(e => e.GetType().GetProperty(groupSelector)).SelectMany(gr => gr);
}
else
source = source.Where(lambda);
return source;
}
but when i run the mthod with GroupBy condition, the result is:
SELECT [e].[Id], [e].[Year]
FROM [org].[Data] AS [e]
ORDER BY [e].[Year]
i don't know why its happened?

TL;DR;: Entitity Framework uses the query because it is the most efficient way to get what you want.
What Entity Framework does is to translate your LINQ query into SQL. Check this line of your code:
source = source.GroupBy(selectorExp).SelectMany(g => g);
You are grouping (probably by year) and then you select all the items of the group. You are actually not requesting a grouped result set, you are expecting all items in all groups in a single flat result set. If EF would first request the groups and then request the group items, it would first have to select all groups:
SELECT [e].[Year]
FROM [org].[Data] AS [e]
GROUP BY [e].[Year]
Then it would have to get the group items in one query for each group:
SELECT [e].[Id]
FROM [org].[Data] AS [e]
WHERE [e].[Year] = --Value
That of course would be very inefficient (especially since you flatten the list anyway with SelectMany), so EF will just get the values ordered by your grouping predicate and group them during query execution (or in this case not group them at all, since you are requesting a flat list). Once the query is executed, EF can just start at the top of the result set and start a new group every time it encounters a new year.
When you are using EF queries, you have to accept that you won't have control over your SQL. If you want that, create stored procedures and run them from EF.

Can't use dynamic delegate parameter in lambda of expression tree

I am trying to apply a simple "Where" clause on a dynamically selected table. However, the table field the clause will be applied to is also dynamic and I can't figure out how to make that part work. Getting the dynamic table works correctly.
using (var context = new DBEntities())
{
var type = context.GetType();
var tableProperty = type.GetProperty("tableName");
var tableGet = tableProperty.GetMethod;
var tableContent = tableGet.Invoke(context, null);
var tableQuery = (IQueryable)tableContent;
var tableType = tableQuery.ElementType;
var pe = Expression.Parameter(tableType, "tableType");
var left = Expression.PropertyOrField(pe, "fieldName");
var right = Expression.Constant("fieldValue");
var predicateBody = Expression.Equal(left, right);
var whereCallExpression = Expression.Call(typeof(Queryable), "Where", new[] { tableType },
tableQuery.Expression, Expression.Lambda<Func<tableType, bool>>(predicateBody, pe));
IQueryable<string> results = tableQuery.Provider.CreateQuery<string>(whereCallExpression);
}
This block of code will not compile because of Expression.Lambda<Func<tableType, bool>>(predicateBody, pe). If I hard-code types for the Expression-related code, this sample runs and returns the expected results.

The code does not compile because tableType, a variable of type System.Type, cannot be used as a type parameter of a generic function.
However, you should be able to make this compile and run by replacing a call to generic Lambda<Func<...>> with a call to non-generic Lambda:
var whereCallExpression = Expression.Call(typeof(Queryable), "Where", new[] { tableType },
tableQuery.Expression, Expression.Lambda(predicateBody, pe));

Any shorter solution for select - from - where LINQ?

I have the search function. and I want to select all value in my Book table which contain _searchdata but I dont know how to express at "where" with the short code instead of listing all items of Table like this:
(I just get some items for example, it contains about 100 items like Booktitle, Author, Genre... i dont want to specify it because it's so long)
public void SearchAny(string _searchdata)
{
var searchAnyInDB = from Book x in BookDB.Books
where (x.BookTitle.Contains(_searchdata)
|| x.Author.Contains(_searchdata)
|| x.Genre.Contains(_searchdata))
select x;
DataSearch.Clear();
DataSearch = new ObservableCollection<Book>(searchAnyInDB);
}

Because LINQ to Entities and LINQ to SQL both use Expression<Func<TSource, bool>> as IQueryable.Where extension method parameter, you can use reflection to create that Expression during compile type and generate all there || instead of typing them into your source code.
Would be something like:
var searchAnyInDB = from Book x in BookDB.Books
where (GetWhereExpression<Book>(_searchdata))
select x;
And GetWhereExpression<TSource> method:
static Expression<Func<TSource, bool>> GetWhereExpression<TSource>(string value)
{
var param = Expression.Parameter(typeof(TSource));
var val = Expression.Constant(value);
var expression = Expression.Equal(Expression.Constant(1), Expression.Constant(1));
foreach(var prop in typeof(TSource).GetProperties(BindingFlags.Public | BindingFlags.Instance))
{
if(prop.PropertyType == typeof(string))
{
expression = Expression.OrElse(expression,
Expression.Call(
Expression.Property(param, prop),
"Contains",
null,
val
)
);
}
}
return Expression.Lambda<Func<TSource, bool>>(expression, param);
}
You still can do better then that, ex. remembering the expression for type to prevent using reflection every time you need to execute the query with different search texts, etc. But it should give you an idea where to go.

Dynamic linq query expression tree for sql IN clause using Entity framework

I want to create a dynamic linq expression for sql IN clause in EF 6.0 with code first approch. Note that i am new to Expressions. What i want to achive is
select * from Courses where CourseId in (1, 2, 3, 4)
//CourseId is integer
The normal linq query looks like this. But i want to query it dynamically
string[] ids = new string[]{"1", "2", "3", "4"};
var courselist = DBEntities.Courses.Where(c => ids.Contains(SqlFunctions.StringConvert((decimal?)c.CourseId)))
There are two ways to make dynamic expression.
1) one ways is to loop through ids and make expressions
The below code will create the following expression in debug view
{f => ((StringConvert(Convert(f.CourseId)).Equals("23") Or StringConvert(Convert(f.CourseId)).Equals("2")) Or StringConvert(Convert(f.CourseId)).Equals("1"))}
Dynamic Expression is
var param = Expression.Parameters(typeof(Course), "f")
MemberExpression property = Expression.PropertyOrField(param, "CourseId");
MethodInfo mi = null;
MethodCallExpression mce = null;
if (property.Type == typeof(int))
{
var castProperty = Expression.Convert(property, typeof(double?));
var t = Expression.Parameter(typeof(SqlFunctions), "SqlFunctions");
mi = typeof(SqlFunctions).GetMethod("StringConvert", new Type[] { typeof(double?) });
mce = Expression.Call(null,mi, castProperty);
}
mi = typeof(string).GetMethod("Equals", new Type[]{ typeof(string)});
BinaryExpression bex = null;
if (values.Length <= 1)
{
return Expression.Lambda<Func<T, bool>>(Expression.Call(mce, mi, Expression.Constant(values[0]), param));
}
//var exp1 = Expression.Call(mce, mi, Expression.Constant(values[0]));
for (int i = 0; i < values.Length; i++)
{
if (bex == null)
{
bex = Expression.Or(Expression.Call(mce, mi, Expression.Constant(values[i])), Expression.Call(mce, mi, Expression.Constant(values[i + 1])));
i++;
}
else
bex = Expression.Or(bex, Expression.Call(mce, mi, Expression.Constant(values[i])));
}//End of for loop
return Expression.Lambda<Func<T, bool>>(bex, param);
2) The 2nd way that i tried (debug view)
{f => val.Contains("23")} //val is parameter of values above
The dynamic expression for above that i tried is
var param = Expression.Parameters(typeof(Course), "f")
MemberExpression property = Expression.PropertyOrField(param, "CourseId");
var micontain = typeof(Enumerable).GetMethods().Where(m => m.Name == "Contains" && m.GetParameters().Length == 2).Single().MakeGenericMethod(typeof(string));
var mc = Expression.Call(micontain, Expression.Parameter(values.GetType(), "val"), Expression.Constant("2"));//NOTE: I haven't use CourseId for now as i am getting conversion error
return Expression.Lambda<Func<T, bool>>(mc, param);
I get the following errors
LINQ to Entities does not recognize the method 'System.String StringConvert(System.Nullable`1[System.Double])' method, and this
method cannot be translated into a store expression when i use the
first methodology. I know i can't use ToString with EF thats why I used SqlFunctions but it is not working for me.
The parameter 'val' was not bound in the specified LINQ to Entities query expression using 2nd methodology
I am trying this from last 4 days. I googled it but didn't find any suitable solution. Please help me.

After a lot of struggle I found solution to my question.
I want to achieve this sql query
select * from Courses where CourseId in (1, 2, 3, 4)
Using Linq to Entities, but I want to pass in(1,2,3,4) list dynamically to linq query. I created an Extension class for that purpose.
public static class LinqExtensions
{
public static Expression<Func<T, bool>> False<T>() { return f => false; }
public static Expression<Func<T, bool>> In<T, TValue>(this Expression<Func<T, bool>> predicate,string propertyName, List<TValue> values)
{
var param = predicate.Parameters.Single();
MemberExpression property = Expression.PropertyOrField(param, propertyName);
var micontain = typeof(List<TValue>).GetMethod("Contains");
var mc = Expression.Call(Expression.Constant(values), micontain, property);
return Expression.Lambda<Func<T, bool>>(mc, param);
}
}
Use of LinqExtensions
var pred = LinqExtensions.False<Course>(); //You can chain In function like LinqExtensions.False<Course>().In<Course, int>("CourseId", inList);
var inList= new List<int>(){1, 2, 3}; //Keep in mind the list must be of same type of the Property that will be compared with. In my case CourseId is integer so the in List have integer values
pred =pred.In<Course, int>("CourseId", inList); //TValue is int. As CourseId is of type int.
var data = MyEntities.Courses.Where(pred);
I hope this might be beneficial for some one

have you seen the type of
var courselist = DBEntities.Courses.Where(c => ids.Contains(c.CourseId)))
above statement would not return actual list of courses. The query is not executed yet. It just returns IQuereable. The query is executed when you actually call .ToList() method on it
so, your solution is..
Create array of IDs using for loop and then simply run the below query
var courselist = DBEntities.Courses.Where(c => ids.Contains(c.CourseId))).ToList()

LINQ Where in collection clause

I've been looking on google but not finding anything that does the trick for me.
as you know SQL has a "where x in (1,2,3)" clause which allows you to check against multiple values.
I'm using linq but I can't seem to find a piece of syntax that does the same as the above statement.
I have a collection of category id's (List) against which I would like to check
I found something that uses the .contains method but it doesn't even build.

You have to use the Contains method on your id list:
var query = from t in db.Table
where idList.Contains(t.Id)
select t;

The syntax is below:
IEnumerable<int> categoryIds = yourListOfIds;
var categories = _dataContext.Categories.Where(c => categoryIds.Contains(c.CategoryId));
The key thing to note is that you do the contains on your list of ids - not on the object you would apply the in to if you were writing sql.

Here's an article illustrating the approach. You should indeed use the Contains method over your collection which will be translated into IN clause.

Here is my realization of WhereIn() Method, to filter IQueryable collection by a set of selected entities:
public static IQueryable<T> WhereIn<T,TProp>(this IQueryable<T> source, Expression<Func<T,TProp>> memberExpr, IEnumerable<TProp> values) where T : class
{
Expression predicate = null;
ParameterExpression param = Expression.Parameter(typeof(T), "t");
bool IsFirst = true;
MemberExpression me = (MemberExpression) memberExpr.Body;
foreach (TProp val in values)
{
ConstantExpression ce = Expression.Constant(val);
Expression comparison = Expression.Equal(me, ce);
if (IsFirst)
{
predicate = comparison;
IsFirst = false;
}
else
{
predicate = Expression.Or(predicate, comparison);
}
}
return predicate != null
? source.Where(Expression.Lambda<Func<T, bool>>(predicate, param)).AsQueryable<T>()
: source;
}
And calling of this method looks like:
IQueryable<Product> q = context.Products.ToList();
var SelectedProducts = new List<Product>
{
new Product{Id=23},
new Product{Id=56}
};
...
// Collecting set of product id's
var selectedProductsIds = SelectedProducts.Select(p => p.Id).ToList();
// Filtering products
q = q.WhereIn(c => c.Product.Id, selectedProductsIds);