Is there a way to get the expression from known where clause, then pass it to other Where(<expression>) clause?
I'm on .NET Core 3.0 preview with EF Core preview.
Included linq and linq dynamic
public void myFunction ()
{
var expression = GetAllItems()
.Where(x => x.Id == 5 && x.Desc.Contains("foos"))
.AwesomeGetExpressioneMagicFunction();
var res = GenericBeforeSaveValidation(expression);
}
public IQueryable<T> GenericBeforeSaveValidation("delegate/expression" exp)
{
//some generic stuff before
return sourceItems.Where(exp);
}
Just save the lambda in a function variable and reuse it:
public void myFunction ()
{
Expression<Func<ItemType, bool>> expression = x => x.Id == 5 && x.Desc.Contains("foos");
var items = GetAllItems()
.Where(expression)
.ToList();
var res = GenericBeforeSaveValidation(expression);
}
public IQueryable<ItemType> GenericBeforeSaveValidation(Expression<Func<ItemType, bool>> exp)
{
//some generic stuff before
return sourceItems.Where(exp);
}
"itemType" is the actual type of your items.
Related
I am trying to write a custom Linq extension, which can also be performed within the DB.
Basically, what I want to do is collectionOfStrings.Where(x => !x.IsNullOrWhiteSpace) Unfortunately it's not supported.
=============== What I've tried so far =============
This part is only interesting to those, who might come up with another idea apart from the one below.
There is a workaround by going this way collection.Where(x => x != null && x.Trim() != string.Empty), but since I use it frequently, it's not the best solution.
The prettiest solution would be, to find a way to write a string extension IsNullOrWhiteSpaceDB, which works or to kind of add the IsNullOrWhiteSpace method to the database programmatically to ensure support.
That's been my attempt to create a working IsNullOrWhiteSpace method, but it's not supported too:
public static bool IsNullOrWhiteSpaceDB(this string? str) =>
str == null || str.Trim() == String.Empty;
So I've started writing a predicate, which is working fine:
public IQueryable<string> GetAll() =>
GetAll().Select(x => x.property).Where(StringIsNotNullOrWhiteSpace).Distinct();
private static Expression<Func<string?, bool>> StringIsNotNullOrWhiteSpace =>
x => x != null && x.Trim() != string.Empty;
=============== The current problem =============
Neverthless I'd actually like to be able to run it on another collection than on a collection of strings. So I tried to build a custom linq extension (inspired by this solution (https://stackoverflow.com/a/40924558/9487478)):
public class QueryVisitor : ExpressionVisitor
{
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (node.Method.IsStatic && node.Method.Name == "IsNullOrWhiteSpace")
{
//!(b.Diameter == null || b.Diameter.Trim() == string.Empty)
var arg = node.Arguments[0];
var argTrim = Expression.Call(arg, typeof(string).GetMethod("Trim", Type.EmptyTypes));
var exp = Expression.MakeBinary(ExpressionType.Or,
Expression.MakeBinary(ExpressionType.Equal, arg, Expression.Constant(null, arg.Type)),
Expression.MakeBinary(ExpressionType.Equal, argTrim, Expression.Constant(string.Empty, arg.Type))
);
return exp;
}
return base.VisitMethodCall(node);
}
}
public static class EfQueryableExtensions
{
public static IQueryable<T> WhereEx<T>(this IQueryable<T> queryable, Expression<Func<T, bool>> where)
{
var visitor = new QueryVisitor();
return queryable.Where(visitor.VisitAndConvert(where, "WhereEx"));
}
}
And that's what my custom extension actually looks like:
public static class QueryHelper
{
public static IQueryable<T> WhereIsNotNullOrWhiteSpace<T>(this IQueryable<T> query, Expression<Func<T, string?>> expression)
{
var arg = expression.Body;
var argTrim = Expression.Call(arg, typeof(string).GetMethod("Trim", Type.EmptyTypes));
var exp = Expression.MakeBinary(ExpressionType.And,
Expression.MakeBinary(ExpressionType.NotEqual, arg, Expression.Constant(null, arg.Type)),
Expression.MakeBinary(ExpressionType.NotEqual, argTrim, Expression.Constant(string.Empty, arg.Type))
);
var lambda = Expression.Lambda<Func<T, bool>>(exp, expression.Parameters);
var result = query.Where(lambda);
return result;
}
}
After the query.Where(lambda) is executed there is an inner exception within the result:
NHibernate.Hql.Ast.ANTLR.QuerySyntaxException: A recognition error occurred.
The "original version" throws the same error too, so I thought it might be the created expression ((x == null) Or (x.Trim() == "")) (copied from the debugger). For me it looks actually quite good and I don't understand the cause of the error.
Any ideas? I would be pleased!
You don't need to build an expression for this. You just need an extension method that takes and returns IQueryable<string>.
public static class Extensions
{
public static IQueryable<string> IsNullOrWhiteSpaceDB(this IQueryable<string> input)
{
return input.Where(x => x != null && x.Trim() != string.Empty);
}
}
I have created PR for LINQKit which should simplify your life https://github.com/scottksmith95/LINQKit/pull/127
Idea is to add ExpandableAttribute to such methods which points to static function with expression for substitution.
public static class Extensions
{
[Expandable(nameof(IsNotNullOrWhiteSpaceDBImpl))]
public static bool IsNotNullOrWhiteSpaceDB(string str)
=> throw new NotImplementedException();
public static Expression<Func<string, bool>> IsNotNullOrWhiteSpaceDBImpl()
=> x => x != null && x.Trim() != string.Empty;
}
So query should use AsExpandable() at least once. Put this call somewhere in repository.
db.Users.AsExpandable()
.Where(u => u.FirstName.IsNotNullOrWhiteSpaceDB() || u.MiddleName.IsNotNullOrWhiteSpaceDB())
I'm using Syncfusion's Grid component in ASP.NET Core project. When sorting, filtering and paginating the grid view it performs LINQ-operations to my IQueryable data source.
When searching text fields, it uses .Contains(string) method, which can't be translated to SQL query and will be evaluated locally.
Is there any way to force EF Core to alter the LINQ query (or to do it by myself) to use .EF.Functions.Like(column, string) instead, because it can be translated to SQL?
var dataSource = ...;
var operation = new QueryableOperation();
// Sorting
if (dm.Sorted != null)
{
dataSource = operation.PerformSorting(dataSource, dm.Sorted);
}
// Filtering
if (dm.Where != null)
{
// PerformFiltering uses .Contains(), which I don't want
dataSource = operation.PerformFiltering(dataSource, dm.Where, dm.Where[0].Operator);
}
// At this point, I want to alter LINQ to use EF.Functions.Like instead of Contains.
var count = dataSource.Count();
// Paging
if (dm.Skip != 0)
{
dataSource = operation.PerformSkip(dataSource, dm.Skip);
}
// Paging
if (dm.Take != 0)
{
dataSource = operation.PerformTake(dataSource, dm.Take);
}
return dm.RequiresCounts ? Json(new { result = dataSource, count }) : Json(dataSource);
You can modify ExpressionTree before execution and replace "".Contains() calls with EF.Functions.Like("", ""):
public static class LinqExtensions
{
public static IQueryable<T> FixQuery<T>(this IQueryable<T> query)
{
return query.Provider.CreateQuery<T>(
new FixQueryVisitor().Visit(query.Expression)
);
}
class FixQueryVisitor : ExpressionVisitor
{
private readonly MethodInfo _likeMethod = ExtractMethod(() => EF.Functions.Like(string.Empty, string.Empty));
private static MethodInfo ExtractMethod(Expression<Action> expr)
{
MethodCallExpression body = (MethodCallExpression)expr.Body;
return body.Method;
}
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (node.Method.DeclaringType == typeof(string) && node.Method.Name == "Contains")
{
return Expression.Call(this._likeMethod, Expression.Constant(EF.Functions), node.Object, node.Arguments[0]);
}
return base.VisitMethodCall(node);
}
}
}
[...]
dataSource = dataSource.FixQuery();
Does anyone know how I can parameterize following:
[Test]
void SelectTest(Expression<Func<MyType, bool>> where)
{
try
{
using (var db = new DataConnection("MyData"))
{
where = e => e.Status == Status.New;
var data = db.GetTable<MyType>()
.(where.Compile())
.Select(e => e);
Assert.IsNotEmpty(data);
}
}
catch (Exception)
{
Assert.False(true);
}
}
I tried adding a testcase like this:
[TestCase(e => e.Status == Status.New)]
But I'm getting the following error:
Expression cannot contain anonymous methods or lambda expressions.
What am I missing?
(I'm using linq2db and nunit)
Appearantly I can use NUnits TestCaseSource to pass Funcs.
See Pass lambda to parameterized NUnit test
My Solution:
public class SelectCollection
{
public static IEnumerable<Expression<Func<Evaluation, bool>>> Evaluation
{
get
{
yield return (e) => e.Status == Status.New;
yield return (e) => e.Id == 0;
}
}
}
Used as:
[Test]
[TestCaseSource(typeof(SelectCollection), "Evaluation")]
public void SelectTest(Expression<Func<Evaluation, bool>> where)
You cannot pass complex expressions as test arguments, only constant primitive types are supported.
public IEnumerable<Table1> GetMatchingTable1(string param, double[] Thicknesses)
{
return DBContext.Table1.Where(c => c.Field1 == param
&& Thicknesses.Any(Thickness => Thickness >= c.MinThickness && Thickness <= c.MaxThickness))
.ToList();
}
Above query return the following exception. "Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries."
So far, all my research on the web for this error pointed toward replacing "ANY" with "CONTAINS". Here is one site where they fix the problem using this solution : http://blog.hompus.nl/2010/08/26/joining-an-iqueryable-with-an-ienumerable/
But in my case, "CONTAINS" doesn't seem usable since I check a RANGE with Min and Max.
How should this query be written to have a proper SQL Statement generated by LinqToEntity?
Thanks
You could try to build the query dynamically:
public IEnumerable<Table1> GetAllCoilLengthSettingsWithChilds(string param, double[] Thicknesses)
{
// Base query
var query = LinqKit.Extensions.AsExpandable(DBContext.Table1.Where(c => c.Field1 == param));
// All the various || between the Thickness ranges
var predicate = LinqKit.PredicateBuilder.False<Table1>();
foreach (double th in Thicknesses)
{
// Don't want a closure around th
double th2 = th;
predicate = predicate.Or(c => th2 >= c.MinThickness && th2 <= c.MaxThickness);
}
// This is implicitly in && with the other Where
query = query.Where(predicate);
return query.ToList();
}
The PredicateBuilder helps you build an || query. Take it from the LinqKit (source available)
I've tested it with 1000 parameters (but they where DateTime, and I didn't have other query pieces), and it seems to work. Note that the program uses another extension of LinqPad, AsExpandable, used to make the PredicateBuilder "trick" work. Note that I'm using EF 6.1.3, so your mileage may vary.
If you don't want to use LinqKit, I'm appending my version of PredicateBuilder. It doesn't require the use of AsExpandable(), but its syntax is slightly different:
public class PredicateBuilder<T>
{
// We share a single parameter for all the PredicatBuilder<T>
// istances. This isn't a proble, because Expressions are immutable
protected static readonly ParameterExpression Parameter = Expression.Parameter(typeof(T), "x");
protected Expression Current { get; set; }
// Returns an empty PredicateBuilder that, if used, is true
public PredicateBuilder()
{
}
// Use it like this: .Where(predicate) or .Any(predicate) or
// .First(predicate) or...
public static implicit operator Expression<Func<T, bool>>(PredicateBuilder<T> predicate)
{
if (object.ReferenceEquals(predicate, null))
{
return null;
}
// Handling of empty PredicateBuilder
Expression current = predicate.Current ?? Expression.Constant(true);
Expression<Func<T, bool>> lambda = Expression.Lambda<Func<T, bool>>(current, Parameter);
return lambda;
}
public static implicit operator PredicateBuilder<T>(Expression<Func<T, bool>> expression)
{
var predicate = new PredicateBuilder<T>();
if (expression != null)
{
// Equivalent to predicate.Or(expression)
predicate.And(expression);
}
return predicate;
}
public void And(Expression<Func<T, bool>> expression)
{
if (expression == null)
{
throw new ArgumentNullException("expression");
}
var expression2 = new ParameterConverter(expression.Parameters[0], Parameter).Visit(expression.Body);
this.Current = this.Current != null ? Expression.AndAlso(this.Current, expression2) : expression2;
}
public void Or(Expression<Func<T, bool>> expression)
{
if (expression == null)
{
throw new ArgumentNullException("expression");
}
var expression2 = new ParameterConverter(expression.Parameters[0], Parameter).Visit(expression.Body);
this.Current = this.Current != null ? Expression.OrElse(this.Current, expression2) : expression2;
}
public override string ToString()
{
// We reuse the .ToString() of Expression<Func<T, bool>>
// Implicit cast here :-)
Expression<Func<T, bool>> expression = this;
return expression.ToString();
}
// Small ExpressionVisitor that replaces the ParameterExpression of
// an Expression with another ParameterExpression (to make two
// Expressions "compatible")
protected class ParameterConverter : ExpressionVisitor
{
public readonly ParameterExpression From;
public readonly ParameterExpression To;
public ParameterConverter(ParameterExpression from, ParameterExpression to)
{
this.From = from;
this.To = to;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (node == this.From)
{
node = this.To;
}
return base.VisitParameter(node);
}
}
}
public static class PredicateBuilder
{
// The value of source isn't really necessary/interesting. Its type
// is :-) By passing a query you are building to Create, the compiler
// will give to Create the the of the object returned from the query
// Use it like:
// var predicate = PredicateBuilder.Create<MyType>();
// or
// var predicate = PredicateBuilder.Create(query);
public static PredicateBuilder<T> Create<T>(IEnumerable<T> source = null)
{
return new PredicateBuilder<T>();
}
// Useful if you want to start with a query:
// var predicate = PredicateBuilder.Create<MyType>(x => x.ID != 0);
// Note that if expression == null, then a new PredicateBuilder<T>()
// will be returned (that by default is "true")
public static PredicateBuilder<T> Create<T>(Expression<Func<T, bool>> expression)
{
// Implicit cast to PredicateBuilder<T>
return expression;
}
}
Use it like:
var predicate = PredicateBuilder.Create(query);
and then everything is the same (but remove the LinqKit.Extensions.AsExpandable part)
We have implemented a security layer around our NHibernate persistence layer in a way that hopes to prevent a user from even receiving an object back from the database if he shouldn't have access to it. That security layer looks like this:
public static IQueryable<T> Secure<T>(this Queryable<T> query){
//if T does not implement ISecurable, then return query
//else
return query.Where(expressionFactory.GetExpression(securityKey));
}
We essentially restrict access to our ISession by wrapping it with a decorator that calls ISession.Query().Secure().
So we have numerous types that return an Expression<Func<T, bool>>, such that we can pass it to Where():
public class DocumentSecurityExpressionFactory : ISecurityExpressionFactory<Document> {
public Expression<Func<Document, bool>> GetExpression(SecurityKey key) {
return doc => doc.MasterDocument.Compartments.Where(c => c.AssociatedCompartment.Type != ProgramTypes.AccessGroup) //Look at non-access group compartments for access
.All(c => key.Compartments.Contains(c.AssociatedCompartment.ID))
&& (
//person has to be either NTK
doc.MasterDocument.NeedToKnowAccessList.Count() == 0
|| doc.MasterDocument.NeedToKnowAccessList.Any(p => p.PersonID == key.PersonID)
|| doc.MasterDocument.NeedToKnowAccessList.Any(p => key.AccessGroups.Contains(p.CompartmentID))
);
}
}
public class DocumentSummarySecurityExpressionFactory : ISecurityExpressionFactory<DocumentSummary> {
public Expression<Func<DocumentSummary, bool>> GetExpression(SecurityKey key) {
return doc => doc.MasterDocument.Compartments.Where(c => c.AssociatedCompartment.Type != ProgramTypes.AccessGroup)
.All(c => key.Compartments.Contains(c.AssociatedCompartment.ID))
&& (
doc.MasterDocument.NeedToKnowAccessList.Count() == 0
|| doc.MasterDocument.NeedToKnowAccessList.Any(p => p.PersonID == key.PersonID)
|| doc.MasterDocument.NeedToKnowAccessList.Any(p => key.AccessGroups.Contains(p.CompartmentID))
);
}
}
public class LatestDocumentVersionSecurityExpressionFactory : ISecurityExpressionFactory<LatestDocumentVersion> {
public Expression<Func<LatestDocumentVersion, bool>> GetExpression(SecurityKey key) {
return version => version.BaseDocument.MasterDocument.Compartments.Where(c => c.AssociatedCompartment.Type != ProgramTypes.AccessGroup)
.All(c => key.Compartments.Contains(c.AssociatedCompartment.ID))
&& (
version.BaseDocument.MasterDocument.NeedToKnowAccessList.Count() == 0
|| version.BaseDocument.MasterDocument.NeedToKnowAccessList.Any(p => p.PersonID == key.PersonID)
|| version.BaseDocument.MasterDocument.NeedToKnowAccessList.Any(p => key.AccessGroups.Contains(p.CompartmentID))
);
}
}
And there are actually several more for different types that look just like this.
The problem here should be clear: each of our entities that does this is essentially the same. They each have a reference to a MasterDocument object, on which all the logic is done. Repeating this code totally sucks (and it all sits in one file so they can all change together if they ever do).
I feel like I should be able to just tell a method how to get the MasterDocument from type T, and then have a generalized method that builds the expression. Something like this:
public static class ExpressionFactory {
public static Expression<Func<T, bool>> Get<T>(Expression<Func<T, MasterDocument>> mdSource, SecurityKey key) {
return t => {
var md = mdSource.Compile()(t);
return md.Compartments.Where(c => c.AssociatedCompartment)...
};
}
}
And call it like so:
public class DocumentSecurityExpressionFactory : ISecurityExpressionFactory<Document> {
public Expression<Func<Document, bool>> GetExpression(SecurityKey key) {
return ExpressionFactory.Get<Document>(doc => doc.MasterDocument, key);
}
}
Now, I understand why this code doesn't work. What I can't figure out is how to build up this expression tree correctly in order to vastly simplify our code. I imagine I could pass in the Expression<Func<T, MasterDocument>> mdSource like that and then use the Expression API to build it out with MemberAccessExpressions and such, but I'm anticipating the mess that would look like, and I'm not sure what would be the lesser evil.
Any help is greatly appreciated.
What you can do is use a Compose method that can compose one expression with another:
public static Expression<Func<TFirstParam, TResult>>
Compose<TFirstParam, TIntermediate, TResult>(
this Expression<Func<TFirstParam, TIntermediate>> first,
Expression<Func<TIntermediate, TResult>> second)
{
var param = Expression.Parameter(typeof(TFirstParam), "param");
var newFirst = first.Body.Replace(first.Parameters[0], param);
var newSecond = second.Body.Replace(second.Parameters[0], newFirst);
return Expression.Lambda<Func<TFirstParam, TResult>>(newSecond, param);
}
Which uses the following method to replace all instances of one expression with another:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
Now you can write:
public static class ExpressionFactory
{
public static Expression<Func<T, bool>> Get<T>(
Expression<Func<T, MasterDocument>> mdSource, SecurityKey key)
{
return mdSource.Compose(document =>
document.Compartments.Where(c => c.AssociatedCompartment.Type != ProgramTypes.AccessGroup)
.All(c => key.Compartments.Contains(c.AssociatedCompartment.ID))
&& (
doc.MasterDocument.NeedToKnowAccessList.Count() == 0
|| doc.MasterDocument.NeedToKnowAccessList.Any(p => p.PersonID == key.PersonID)
|| doc.MasterDocument.NeedToKnowAccessList.Any(p => key.AccessGroups.Contains(p.CompartmentID))
);
}
}