I want to achieve the following expression or as near as possible.
IRangePredicate range = new Range();
DbContext context = new dbModel();
context.Table1.Where(x => range.IsInRange(x.CreatedAt) && x.type == 1).ToList();
and range will produce a partial expression for the linq query, that can either be resolved as:
CreatedAt >= from && CreatedAt <= to
Or
CreatedAt >= from
Or
CreatedAt <= To
to be used in the linq query.
Eventually, I would like to extend this method to include the possibilities of less or more without equals as well.
and use it as a sort of "arguement dependency injection".
However, my attempts fail to even compile, as either
Expression<Func<DateTime, bool>> can't be used as a partial parameter, and I need to define the following query for these special filters. Which I don't want to do. I want it to read as "normal" Linq.
Or I need to simply insert them as Func only. Which might work, but as soon as I try to do that on a Context Linq Query, the thing explodes, because Entity Framework, does not play well if it is not formatted as an Expression
Can anyone guide me in the right direction?
Example of what I tried: (Please note this does not compile, because that is my entire issue :D )
EDIT From here:
-I have commented out the line of code that doesn't compile, so you have a compilable example. It just doesn't work if you try to do it on a DbContext set.
public interface IRangeFunctional
{
bool GetRange(DateTime param);
}
public interface IRange
{
Expression<Func<DateTime, bool>> GetRange(DateTime param);
}
public class RangeFunctional : IRangeFunctional
{
private DateTime _from;
private DateTime _to;
public RangeFunctional(DateTime from, DateTime to)
{
_from = from;
_to = to;
}
public bool GetRange(DateTime param)
{
return param >= _from && param <= _to;
}
}
public class Range : IRange
{
private DateTime _from;
private DateTime _to;
public Range(DateTime from, DateTime to)
{
_from = from;
_to = to;
}
public Expression<Func<DateTime, bool>> GetRange(DateTime param)
{
return (x => param >= _from && param <= _to);
}
}
public class Invoice
{
public DateTime CreatedAt { get; set; }
public int typeId { get; set; }
}
[TestClass]
public class TestRange
{
List<Invoice> list = new List<Invoice>()
{
new Invoice()
{
CreatedAt = new DateTime(2018,1,1,0,0,0), typeId = 1
},
new Invoice()
{
CreatedAt = new DateTime(2018,1,2,0,0,0), typeId = 1
},
new Invoice()
{
CreatedAt = new DateTime(2018,1,1,0,0,0), typeId = 2
},
new Invoice()
{
CreatedAt = new DateTime(2018,1,2,0,0,0), typeId = 2
}
};
[TestMethod]
public void RangeTest()
{
Range r = new Range(new DateTime(2018, 1, 1, 0, 0, 0), new DateTime(2018, 1, 2, 0, 0, 0));
RangeFunctional rf = new RangeFunctional(new DateTime(2018, 1, 1, 0, 0, 0), new DateTime(2018, 1, 2, 0, 0, 0));
List<Invoice> partialListFunc = list.Where(x => x.typeId == 2 && rf.GetRange(x.CreatedAt)).ToList();
//List<Invoice> partialList = list.Where(x => x.typeId == 2 && r.GetRange(x.CreatedAt)).ToList();
Assert.AreEqual(2, partialListFunc.Count);
}
}
Okay, So I added the base method that game me the idea, as a demo example, where I am just using ordinary "bool" to accomplish a pure search by link in generic collections.
However, I want to reuse this logic, or as close as possibe, to allow me to accomplish this towards a DbContext.
I have a base crud controller for any type of table towards the Db, however, I would like to enhance this bit, but letting the programmer to implement a strategy pattern over a partial classes generated from either code first, or db first models in C#.
However, in order to translate the Linq to SQL, I need to convert my "just bool" return type into expressions. I got that far. But How the heck do I make "subsets" of predicates, that can be unified over a single collection?
I see some code examples that require you to chain the queries. And that might end up being the solution. This just seems so... ugly.
I just can't get my brain to think up the syntax to do this. And it is frustrating me :D Forgive me if this cannot be done, because I am simply stupid. It just seems sort of intuitive to me that this should be possible.
Here is a sample implementation. It uses an extension method for modifying Expressions to build a new Expression:
public static class ExpressionExt {
/// <summary>
/// Replaces a sub-Expression with another Expression inside an Expression
/// </summary>
/// <param name="orig">The original Expression.</param>
/// <param name="from">The from Expression.</param>
/// <param name="to">The to Expression.</param>
/// <returns>Expression with all occurrences of from replaced with to</returns>
public static Expression Replace(this Expression orig, Expression from, Expression to) => new ReplaceVisitor(from, to).Visit(orig);
}
/// <summary>
/// Standard ExpressionVisitor to replace an Expression with another in an Expression.
/// </summary>
public class ReplaceVisitor : ExpressionVisitor {
readonly Expression from;
readonly Expression to;
public ReplaceVisitor(Expression from, Expression to) {
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node) => node == from ? to : base.Visit(node);
}
Now you can build a filter interface and some implementations to represent differing types of filters and an IQueryable extension that uses it to filter:
public interface IFilter<TMember> {
Expression<Func<TData, bool>> FilterFn<TData>(Expression<Func<TData, TMember>> memberFn);
}
public class FilterDateTimeRange : IFilter<DateTime?> {
public DateTime? from;
public DateTime? to;
public FilterDateTimeRange(DateTime? fromDT, DateTime? toDT) {
from = fromDT;
to = toDT;
}
public Expression<Func<T, bool>> FilterFn<T>(Expression<Func<T, DateTime?>> memberFn) {
Expression<Func<DateTime?, bool>> rangeBodyTemplate;
if (from.HasValue) {
if (to.HasValue)
rangeBodyTemplate = dt => from.Value <= dt && dt <= to.Value;
else
rangeBodyTemplate = dt => from.Value <= dt;
}
else if (to.HasValue) {
rangeBodyTemplate = dt => dt <= to.Value;
}
else
rangeBodyTemplate = dt => true;
return Expression.Lambda<Func<T, bool>>(rangeBodyTemplate.Body.Replace(rangeBodyTemplate.Parameters[0], memberFn.Body), memberFn.Parameters);
}
}
public class FilterDateRange : IFilter<DateTime?> {
public DateTime? from;
public DateTime? to;
public FilterDateRange(DateTime? fromDT, DateTime? toDT) {
from = fromDT?.Date;
to = toDT?.Date;
}
public Expression<Func<T, bool>> FilterFn<T>(Expression<Func<T, DateTime?>> memberFn) {
Expression<Func<DateTime?, bool>> rangeBodyTemplate;
if (from.HasValue) {
if (to.HasValue)
rangeBodyTemplate = dt => from <= (dt == null ? dt : dt.Value.Date) && (dt == null ? dt : dt.Value.Date) <= to;
else
rangeBodyTemplate = dt => from.Value <= (dt == null ? dt : dt.Value.Date);
}
else if (to.HasValue) {
rangeBodyTemplate = dt => (dt == null ? dt : dt.Value.Date) <= to.Value;
}
else
rangeBodyTemplate = dt => true;
return Expression.Lambda<Func<T, bool>>(rangeBodyTemplate.Body.Replace(rangeBodyTemplate.Parameters[0], memberFn.Body), memberFn.Parameters);
}
}
public class FilterStartsWith : IFilter<String> {
public string start;
public FilterStartsWith(string start) => this.start = start;
public Expression<Func<T, bool>> FilterFn<T>(Expression<Func<T, string>> memberFn) {
Expression<Func<string, bool>> rangeBodyTemplate;
if (!String.IsNullOrEmpty(start))
rangeBodyTemplate = s => s.StartsWith(start);
else
rangeBodyTemplate = s => true;
return Expression.Lambda<Func<T, bool>>(rangeBodyTemplate.Body.Replace(rangeBodyTemplate.Parameters[0], memberFn.Body), memberFn.Parameters);
}
}
public static class FilterExt {
public static IQueryable<TData> WhereFilteredBy<TData, TMember>(this IQueryable<TData> src, IFilter<TMember> r, Expression<Func<TData, TMember>> memberFn) => src.Where(r.FilterFn(memberFn));
}
Given all this, you use it like so:
var r1 = new FilterDateTimeRange(DateTime.Now.AddDays(-1).Date, DateTime.Now.AddDays(-1).Date);
var yesterdayFilter = new FilterDateRange(DateTime.Now.AddDays(-1), DateTime.Now.AddDays(-1));
var r1a = Accounts.Where(r1.RangeFilter<Accounts>(a => a.Modified_date));
var ya = Accounts.WhereFilteredBy(yesterdayFilter, a => a.Modified_date);
Since the C# type inference engine isn't as sophisticated as e.g. F# and won't infer through return expressions, you must specify the type when using the standard Where but an IQueryable extension replacement Where can infer the type from the first parameter (e.g. Accounts).
Since the IFilter is generic, you can use other types of filters such as FilterStartsWith to filter on other types of fields:
List<Table1> Table1InRangeWithName(IFilter<DateTime?> range, IFilter<string> name) => context.Table1.WhereFilteredBy(range, t1 => t1.Modified_date).WhereFilteredBy(name, t1 => t1.Name).ToList();
And then call it with a pre-created FilterDataRange and FilterStartsWith:
var nameFilter = new FilterStartsWith("TEST");
var ans = Table1InRangeWithName(yesterdayFilter, nameFilter);
Related
When using an entity with property of a custom type, the type cannot be translated into SQL.
I have created an example to explain my approach to solve it:
A class takes place in a certain semester. The semester is stored as a DateTime value in the database.
The semester itself is a custom type, with additional properties.
public class Semester
{
public enum HalfYear
{
First = 1,
Second = 7
}
DateTime _dateTime;
public Semester (HalfYear halfYear, int year)
{
_dateTime = new DateTime(year, (int) halfYear, 1)
}
public int Year => _dateTime.Year;
public HalfYear HalfYear => (HalfYear) _dateTime.Month;
public DateTime FirstDay => new DateTime(Year, _dateTime.Month, 1);
public DateTime LastDay => new DateTime(Year, _dateTime.Month + 5, DateTime.DaysInMonth(Year, _dateTime.Month + 5));
}
public class Class
{
int Id { get; set; }
string Title { get; set; }
Semester Semester { get; set; }
}
The Semester type can be mapped to a DateTime using value converters.
This does not work in Where clause such as
db.Classes.Where(c = c.Semester.FirstDay <= DateTime.Now &&
c.Semester.LastDay >= DateTime.Now)
When Entity Framework Core tries to translate the expression tree to SQL, it does not know how to translate Semester.FirstDay or Semester.LastDay.
This is a known limitation of value conversions as the documentation states
Use of value conversions may impact the ability of EF Core to translate expressions to SQL. A warning will be logged for such cases. Removal of these limitations is being considered for a future release.
How to solve this issue?
EntityFrameworkCore has 3 extension points that can be used to translate custom types to SQL.
IMemberTranslator
IMethodCallTranslator
RelationalTypeMapping
These translators and mapppings can be registered using the corresponding plugins:
IMemberTranslatorPlugin
IMethodCallTranslatorPlugin
IRelationalTypeMappingSourcePlugin
The plugins are registered with a IDbContextOptionsExtension
The following example illustrates how I have implemented these interfaces to register the custom type Semester:
IMemberTranslator
public class SqlServerSemesterMemberTranslator : IMemberTranslator
{
public Expression Translate(MemberExpression memberExpression)
{
if (memberExpression.Member.DeclaringType != typeof(Semester)) {
return null;
}
var memberName = memberExpression.Member.Name;
if (memberName == nameof(Semester.FirstDay)) {
return new SqlFunctionExpression(
"DATEFROMPARTS",
typeof(DateTime),
new Expression[] {
new SqlFunctionExpression( "YEAR", typeof(int),new[] { memberExpression.Expression }),
new SqlFunctionExpression( "MONTH", typeof(int),new[] { memberExpression.Expression }),
Expression.Constant(1, typeof(int))
});
}
if (memberName == nameof(Semester.LastDay)) {
return new SqlFunctionExpression(
"EOMONTH",
typeof(DateTime),
new Expression[] {
memberExpression.Expression
});
}
if (memberName == nameof(Semester.HalfYear)) {
return Expression.Convert(
new SqlFunctionExpression(
"MONTH",
typeof(int),
new Expression[] {
memberExpression.Expression
}),
typeof(HalfYear));
}
if (memberName == nameof(Semester.Year)) {
return new SqlFunctionExpression(
"YEAR",
typeof(int),
new Expression[] {
memberExpression.Expression
});
}
return null;
}
}
IMethodCallTranslator
public class SqlServerSemesterMethodCallTranslator : IMethodCallTranslator
{
public Expression Translate(MethodCallExpression methodCallExpression)
{
if (methodCallExpression.Method.DeclaringType != typeof(Period)) {
return null;
}
var methodName = methodCallExpression.Method.Name;
// Implement your Method translations here
return null;
}
}
RelationalTypeMapping
public class SqlServerSemesterTypeMapping : DateTimeTypeMapping
{
public SqlServerSemesterTypeMapping(string storeType, DbType? dbType = null) :
base(storeType, dbType)
{
}
protected SqlServerSemesterTypeMapping(RelationalTypeMappingParameters parameters) : base(parameters)
{
}
protected override RelationalTypeMapping Clone(RelationalTypeMappingParameters parameters) => new SqlServerSemesterTypeMapping(parameters);
}
IMemberTranslatorPlugin
public class SqlServerCustomMemberTranslatorPlugin : IMemberTranslatorPlugin
{
public IEnumerable<IMemberTranslator> Translators => new IMemberTranslator[] { new SqlServerSemesterMemberTranslator() };
}
public class SqlServerCustomMethodCallTranslatorPlugin : IMethodCallTranslatorPlugin
{
public IEnumerable<IMethodCallTranslator> Translators => new IMethodCallTranslator[] { new SqlServerSemesterMethodCallTranslator() };
}
IRelationalTypeMappingSourcePlugin
public class SqlServerCustomTypeMappingSourcePlugin : IRelationalTypeMappingSourcePlugin
{
public RelationalTypeMapping FindMapping(in RelationalTypeMappingInfo mappingInfo)
=> mappingInfo.ClrType == typeof(Semester) || (mappingInfo.StoreTypeName == nameof(DateTime))
? new SqlServerSemesterTypeMapping(mappingInfo.StoreTypeName ?? "datetime")
: null;
}
After you have defined and registered the translators, you have to confgure them in the DbContext.
IDbContextOptionsExtension
public class SqlServerCustomTypeOptionsExtension : IDbContextOptionsExtensionWithDebugInfo
{
public string LogFragment => "using CustomTypes";
public bool ApplyServices(IServiceCollection services)
{
services.AddEntityFrameworkSqlServerCustomTypes();
return false;
}
public long GetServiceProviderHashCode() => 0;
public void PopulateDebugInfo(IDictionary<string, string> debugInfo)
=> debugInfo["SqlServer:" + nameof(SqlServerCustomDbContextOptionsBuilderExtensions.UseCustomTypes)] = "1";
public void Validate(IDbContextOptions options)
{
}
}
Extension Methods
public static class SqlServerCustomDbContextOptionsBuilderExtensions
{
public static object UseCustomTypes(this SqlServerDbContextOptionsBuilder optionsBuilder)
{
if (optionsBuilder == null) throw new ArgumentNullException(nameof(optionsBuilder));
// Registere die SqlServerDiamantOptionsExtension.
var coreOptionsBuilder = ((IRelationalDbContextOptionsBuilderInfrastructure)optionsBuilder).OptionsBuilder;
var extension = coreOptionsBuilder.Options.FindExtension<SqlServerCustomTypeOptionsExtension>()
?? new SqlServerCustomTypeOptionsExtension();
((IDbContextOptionsBuilderInfrastructure)coreOptionsBuilder).AddOrUpdateExtension(extension);
// Configure Warnings
coreOptionsBuilder
.ConfigureWarnings(warnings => warnings
.Log(RelationalEventId.QueryClientEvaluationWarning) // Should be thrown to prevent only warnings if a query is not fully evaluated on the db
.Ignore(RelationalEventId.ValueConversionSqlLiteralWarning)); // Ignore warnings for types that are using a ValueConverter
return optionsBuilder;
}
}
public static class SqlServerServiceCollectionExtensions
{
public static IServiceCollection AddEntityFrameworkSqlServerCustomTypes(
this IServiceCollection serviceCollection)
{
if (serviceCollection == null) throw new ArgumentNullException(nameof(serviceCollection));
new EntityFrameworkRelationalServicesBuilder(serviceCollection)
.TryAddProviderSpecificServices(
x => x.TryAddSingletonEnumerable<IRelationalTypeMappingSourcePlugin, SqlServerCustomTypeMappingSourcePlugin>()
.TryAddSingletonEnumerable<IMemberTranslatorPlugin, SqlServerCustomTypeMemberTranslatorPlugin>()
.TryAddSingletonEnumerable<IMethodCallTranslatorPlugin, SqlServerCustomTypeMethodCallTranslatorPlugin>());
return serviceCollection;
}
}
Register the option in the DbContext
dbOptionsBuilder.UseSqlServer(connectionString, builder => builder.UseCustomTypes())
My project has many objects with date fields, and I often need to select everything where one such field is within a date range.
For example:
public class Contract
{
public DateTime SignDate { get; set; }
public DateTime ReleaseDate { get; set; }
}
public class PersonalCheck
{
public DateTime SignDate { get; set; }
public DateTime ProcessDate { get; set; }
public DateTime VoidDate { get; set; }
}
If I only cared about SignDate, it would be easy. I would declare an Interface...
public interface IObjectWithSignDate
{
DateTime SignDate { get; set; }
}
...change my other objects to inherit from it, then create a method like this:
public static IQueryable<T> SignedWithin<T>(this IQueryable<T> items, DateTime start, DateTime end) where T : IObjectWithSignDate
{
return items.Where(q => q.SignDate >= start && q.SignDate <= end);
}
How can I avoid rewriting this function for ReleaseDate, ProcessDate, VoidDate, etc.? Can I make this method take in an IQueryable of any object and a variable telling it which date field to run this selector against?
Note this would have to be able to a) execute in LinqToEntities to run against a database and b) not add a lot of overhead (as I'm fearful reflection might do)
Simple but specific
You can add an extension method like this:
public static class DateTimeExtensions
{
public static bool IsBetween(this DateTime thisDateTime, DateTime start, DateTime end)
{
return thisDateTime >= start && thisDateTime <= end;
}
}
which you can unit test in isolation.
Then you can use this on whichever DateTime field you want to check. For example:
var start = new DateTime(2017, 1, 1);
var end = new DateTime(2017, 12, 31, 23, 59, 59);
IList<Contract> contracts = new List<Contract>(); // or anything enumerable
var contractsSignedBetween = contracts.Where(x => x.SignDate.IsBetween(start, end));
var contractsReleasedBetween = contracts.Where(x => x.ReleaseDate.IsBetween(start, end));
(Notice how I set the start datetime to have 00:00:00 time, and the end datetime to have 23:59:59 time [feel free to include milliseconds as well], so that times within the last day are included.)
Making that reusable
If you find yourself needing to do that a lot, you could do an extension for that
public static class EnumerableContractsExtensions
{
public static IEnumerable<Contract> SignedBetween(this IEnumerable<Contract> contracts, DateTime start, DateTime end)
{
return contracts.Where(x => x.SignDate.IsBetween(start, end));
}
}
and use it like this
var contractsSignedBetween = contracts.SignedBetween(start, end);
which could also be unit tested in isolation.
More flexible but specific
Use an expression to say which date you want...
public static class EnumerableContractsExtensions
{
public static IEnumerable<Contract> Between(this IEnumerable<Contract> contracts, Func<Contract, DateTime> selector, DateTime start, DateTime end)
{
return contracts.Where(x => selector(x).IsBetween(start, end));
}
}
and then do:
var contractsSignedBetween = contracts.Between(x => x.SignDate, start, end);
var contractsReleasedBetween = contracts.Between(x => x.ReleaseDate, start, end);
Flexible and generic
Go the whole hog and do it generically (although you can't make it an extension method since it's generic):
public static class EnumerableExtensions
{
public static IEnumerable<T> Between<T>(IEnumerable<T> items, Func<T, DateTime> selector, DateTime start, DateTime end)
{
return items.Where(x => selector(x).IsBetween(start, end));
}
}
again, this is testable in its own right, and can be used like this:
IList<Contract> contracts = new List<Contract>();
IList<PersonalCheck> personalChecks = new List<PersonalCheck>();
var contractsSignedBetween = EnumerableExtensions.Between(contracts, x => x.SignDate, start, end);
var checksSignedBetween = EnumerableExtensions.Between(personalChecks, x => x.SignDate, start, end);
Making it IQueryable
To make this work as IQueryable the approach needs to shift to an expression tree, since LINQ to Entities does not know how to translate a method into SQL.
public static IQueryable<TSource> Between<TSource, TKey>(
this IQueryable<TSource> source,
Expression<Func<TSource, TKey>> keySelector,
TKey low,
TKey high)
where TKey : IComparable<TKey>
{
Expression key = keySelector.Body;
Expression lowerBound = Expression.LessThanOrEqual(Expression.Constant(low), key);
Expression upperBound = Expression.LessThanOrEqual(key, Expression.Constant(high));
Expression and = Expression.AndAlso(lowerBound, upperBound);
Expression<Func<TSource, bool>> lambda =
Expression.Lambda<Func<TSource, bool>>(and, keySelector.Parameters);
return source.Where(lambda);
}
which would still be used like this:
var contractsSignedBetween = contracts.Between(x => x.SignDate, start, end);
And this works for things other than DateTimes as well. Hope this helps.
is it possible to dynamically generate such a predicate using LambdaExpressions?
Expression<Func<Test, bool>> predicate = t =>
t.Levels.Any(l =>
l.LevelDetails.Any( ld =>
ld.LevelDate > DbFunctions.AddDays(t.TestDate, 1)
)
);
As long as the parameters in the inner BinaryExpression are identical or the right part of the expression is constant, there is no problem. But the example expressionld.LevelDate > DbFunctions.AddDays (t.TestDate, 1) contains two different ExpressionParameters which are independent from each other. What I am looking for is something like this:
Expression<Func<LevelDetail, DateTime?>> left =
ld => ld.LevelDate;
Expression<Func<Test, DateTime?>> right =
t => DbFunctions.AddDays(t.TestDate, 1);
BinaryExpression expr =
Expression.GreaterThan(
((LambdaExpression)left).Body,
((LambdaExpression)right).Body
);
Expression<Func<Test, bool>> predicate = t =>
t.Levels.Any(l =>
l.LevelDetails.Any( **expr** )
);
class Test {
public DateTime TestDate { get; set; }
public virtual ICollection<Level> Levels { get; set; }
}
class Level {
public virtual ICollection<LevelDetail> LevelDetails { get; set; }
}
class LevelDetail {
public DateTime LevelDate { get; set; }
}
Kind regards!
As pointed out in the answers by #Matt Warren if yow want to combine lambdas you will need to do it by hand and will need to set the correct expression parameters.
Firstlly, you will need a ExpressionVisitor that can replace node that you want:
private class SwapVisitor : ExpressionVisitor
{
public readonly Expression _from;
public readonly Expression _to;
public SwapVisitor(Expression from, Expression to)
{
_from = from;
_to = to;
}
public override Expression Visit(Expression node) => node == _from ? _to : base.Visit(node);
}
Secondly, you will need to combine lambdas by hand:
private static Expression<Func<Test, bool>> CreatePredicate()
{
Expression<Func<LevelDetail, DateTime?>> left = ld => ld.LevelDate;
// I didn't include EF, so I did test it just use directly Test.TestDate
//Expression<Func<Test, DateTime?>> right = t => t.TestDate;
Expression<Func<Test, DateTime?>> right = t => DbFunctions.AddDays(t.TestDate, 1);
var testParam = Expression.Parameter(typeof(Test), "test_par");
var levelParam = Expression.Parameter(typeof(Level), "level_par");
var detailParam = Expression.Parameter(typeof(LevelDetail), "detail_par");
// Swap parameters for right and left operands to the correct parameters
var swapRight = new SwapVisitor(right.Parameters[0], testParam);
right = swapRight.Visit(right) as Expression<Func<Test, DateTime?>>;
var swapLeft = new SwapVisitor(left.Parameters[0], detailParam);
left = swapLeft.Visit(left) as Expression<Func<LevelDetail, DateTime?>>;
BinaryExpression comparer = Expression.GreaterThan(left.Body, right.Body);
var lambdaComparer = Expression.Lambda<Func<LevelDetail, bool>>(comparer, detailParam);
// Well, we created here the lambda for ld => ld.LevelDate > DbFunctions.AddDays(t.TestDate, 1)
var anyInfo = typeof(Enumerable).GetMethods().Where(info => info.Name == "Any" && info.GetParameters().Length == 2).Single();
// Will create **l.LevelDetails.Any(...)** in the code below
var anyInfoDetail = anyInfo.MakeGenericMethod(typeof(LevelDetail));
var anyDetailExp = Expression.Call(anyInfoDetail, Expression.Property(levelParam, "LevelDetails"), lambdaComparer);
var lambdaAnyDetail = Expression.Lambda<Func<Level, bool>>(anyDetailExp, levelParam);
// Will create **t.Levels.Any(...)** in the code below and will return the finished lambda
var anyInfoLevel = anyInfo.MakeGenericMethod(typeof(Level));
var anyLevelExp = Expression.Call(anyInfoLevel, Expression.Property(testParam, "Levels"), lambdaAnyDetail);
var lambdaAnyLevel = Expression.Lambda<Func<Test, bool>>(anyLevelExp, testParam);
return lambdaAnyLevel;
}
And the code below contains usage of this:
var predicate = CreatePredicate();
var levelDetail = new LevelDetail { LevelDate = new DateTime(2017, 08, 19) };
var level = new Level { LevelDetails = new List<LevelDetail> { levelDetail } };
var test = new Test { TestDate = new DateTime(2027, 08, 19), Levels = new List<Level> { level } };
var result = predicate.Compile()(test);
I would recommend using nein-linq to combine, build and compose predicates (and many other expression puzzles),
or LinqKit
Both support Entity Framework
For example, using nein-linq
Given:
public static class TestExpressions
{
[InjectLambda]
public static bool IsTestDateEarlierThan(this Test test, DateTime? dateTime, int numberOfDays)
{
return dateTime > test.TestDate.AddDays(numberOfDays);
}
public static Expression<Func<Test, DateTime?, int, bool>> IsTestDateEarlierThan()
{
return (test, dateTime, numberOfDays) => dateTime > DbFunctions.AddDays(test.TestDate, numberOfDays);
}
// Simple caching...
private static readonly Func<Test, int, bool> _hasAnyLevelDateAfterTestDays = HasAnyLevelDateAfterTestDays().Compile();
[InjectLambda]
public static bool HasAnyLevelDateAfterTestDays(this Test test, int numberOfDays)
{
return _hasAnyLevelDateAfterTestDays(test, numberOfDays);
}
public static Expression<Func<Test, int, bool>> HasAnyLevelDateAfterTestDays()
{
return (test, numberOfDays) => test.Levels.Any(l => l.LevelDetails.Any(ld => test.IsTestDateEarlierThan(ld.LevelDate, numberOfDays)));
}
}
When:
var testList = new List<Test>
{
new Test {
Levels = new List<Level> {
new Level {
LevelDetails = new List<LevelDetail> {
new LevelDetail {
LevelDate = DateTime.Today
}
}
}
},
// Not matched
TestDate = DateTime.Today
},
new Test {
Levels = new List<Level> {
new Level {
LevelDetails = new List<LevelDetail> {
new LevelDetail {
LevelDate = DateTime.Today
}
}
}
},
// Not matched
TestDate = DateTime.Today.AddDays(-1)
},
new Test {
Levels = new List<Level> {
new Level {
LevelDetails = new List<LevelDetail> {
new LevelDetail {
LevelDate = DateTime.Today
}
}
}
},
// Matched
TestDate = DateTime.Today.AddDays(-2)
}
};
Then:
var testQuery = testList.AsQueryable();
// Alternative one
var result1 = testQuery
.ToInjectable() // Don't forget!!
.Where(test => test.Levels.Any(l => l.LevelDetails.Any(ld => test.IsTestDateEarlierThan(ld.LevelDate, 1))))
.ToList();
// Alternative two: You get the point :)
var result2 = testQuery
.ToInjectable() // Don't forget!!
.Where(test => test.HasAnyLevelDateAfterTestDays(1))
.ToList();
When you build an expression with nested lambda's the inner lambda's expressions will be able to access the outer lambda's parameters. It works the same way with Expression<T> lambdas as with regular C# lambdas.
If you are working with Expression<T> lambdas and trying to combine them, you'll need to work with them at the API level (do it by hand), and not expect the automatic C# language syntax to Expression<T> conversion to help you out.
One thing to note: when you created the two original lambdas (via conversion to Expression<T>), they each got their own ParameterExpression instances, which will make it impossible to combine them because both bodies will need to be referencing the same instance (unless you replace one for the other using an ExpressionVisitor.)
I am trying to build a LINQ query to query against a large SQL table (7M+ entries) of Documents.
Each document has many DocumentFields :
My goal is to apply successive filters (from 0 to ~10 filters) on the value field of DocumentField:
Here is an example of the filters I want to apply:
[
{fieldId: 32, value: "CET20533"},
{fieldId: 16, value: "882341"},
{fieldId: 12, value: "101746"}
]
What I want is to retrieve every document in my database that matches all of the filters. For the previous example, I want all documents that have a value of CET20533 for the field with the Id "32", the value 882341 for the field with the Id 16, and so on.
I had a first approach :
List<MyFilter> filters = ... // Json deserialization
db.Documents.Where(document =>
filters.All(filter =>
document.DocumentFields.Any(documentField =>
documentField.Id == filter.Id
&& documentField.Value == filter.Value)));
This approach doesn't work : my filters List isn't a primitive type, and therefore cannot be used in a LINQ query.
I had a second approach, which didn't throw an error at me, but only applied 1 filter :
var result = db.Documents.Select(d => d);
foreach (var filter in filters) {
var id = filter.Id;
var value = filter.Value;
result = result.Where(document => document.DocumentFields.Any(documentField =>
documentField.Id == id
&& documentField.Value == value));
}
The problem with this approach is, I believe, some sort of a concurrency problem. I applied a simple pause Thread.Sleep(2000) in each iteration of the foreach to test, and it seems to work.
Questions :
How to remove the pause and still not have concurrency problems ?
Is there a better way to build my query ?
EDIT :
For more clarity, here is an actual example of a document that matches the previous filters example :
You have to Build expression based on your filters and append each in where separately (or not if you can manage it)
db.Documents.Where(ex1).Where(ex2)...
see e.g and MSDN
Or simple case: Start from DocumentFields and retrieve Related Documents. operation Contains works for simple types. that will also simplier in case of building of expression
Quite convinced that your data model is too generic. It will hurt you in terms of program clarity and performance.
But let's go with it for this answer, which I took as a challenge in expression building. The goal is to get a nice queryable that honors the filters on the data server side.
Here's the data model I used, which I think closely matches yours:
public sealed class Document
{
public int Id { get; set; }
// ...
public ICollection<DocumentField> Fields { get; set; }
}
public sealed class DocumentField
{
public int Id { get; set; }
public int DocumentId { get; set; }
public string StringValue { get; set; }
public float? FloatValue { get; set; }
// more typed vales here
}
First, I implement conveniance functions to create predicates for individual fields of individual field types:
public static class DocumentExtensions
{
private static readonly PropertyInfo _piFieldId = (PropertyInfo)((MemberExpression)((Expression<Func<DocumentField, int>>)(f => f.Id)).Body).Member;
private static Expression<Func<DocumentField, bool>> FieldPredicate<T>(int fieldId, T value, Expression<Func<DocumentField, T>> fieldAccessor)
{
var pField = fieldAccessor.Parameters[0];
var xEqualId = Expression.Equal(Expression.Property(pField, _piFieldId), Expression.Constant(fieldId));
var xEqualValue = Expression.Equal(fieldAccessor.Body, Expression.Constant(value, typeof(T)));
return Expression.Lambda<Func<DocumentField, bool>>(Expression.AndAlso(xEqualId, xEqualValue), pField);
}
/// <summary>
/// f => f.<see cref="DocumentField.Id"/> == <paramref name="fieldId"/> && f.<see cref="DocumentField.StringValue"/> == <paramref name="value"/>.
/// </summary>
public static Expression<Func<DocumentField, bool>> FieldPredicate(int fieldId, string value) => FieldPredicate(fieldId, value, f => f.StringValue);
/// <summary>
/// f => f.<see cref="DocumentField.Id"/> == <paramref name="fieldId"/> && f.<see cref="DocumentField.FloatValue"/> == <paramref name="value"/>.
/// </summary>
public static Expression<Func<DocumentField, bool>> FieldPredicate(int fieldId, float? value) => FieldPredicate(fieldId, value, f => f.FloatValue);
// more overloads here
}
Usage:
var fieldPredicates = new[] {
DocumentExtensions.FieldPredicate(32, "CET20533"), // f => f.Id == 32 && f.StringValue == "CET20533"
DocumentExtensions.FieldPredicate(16, "882341"),
DocumentExtensions.FieldPredicate(12, 101746F) // f => f.Id == 12 && f.FloatValue == 101746F
};
Second, I implement an extension method HavingAllFields(also in DocumentExtensions) that creates an IQueryable<Document> where all of the field predicates are satisfied by at least one field:
private static readonly MethodInfo _miAnyWhere = ((MethodCallExpression)((Expression<Func<IEnumerable<DocumentField>, bool>>)(fields => fields.Any(f => false))).Body).Method;
private static readonly Expression<Func<Document, IEnumerable<DocumentField>>> _fieldsAccessor = doc => doc.Fields;
/// <summary>
/// <paramref name="documents"/>.Where(doc => doc.Fields.Any(<paramref name="fieldPredicates"/>[0]) && ... )
/// </summary>
public static IQueryable<Document> HavingAllFields(this IQueryable<Document> documents, IEnumerable<Expression<Func<DocumentField, bool>>> fieldPredicates)
{
using (var e = fieldPredicates.GetEnumerator())
{
if (!e.MoveNext()) return documents;
Expression predicateBody = Expression.Call(_miAnyWhere, _fieldsAccessor.Body, e.Current);
while (e.MoveNext())
predicateBody = Expression.AndAlso(predicateBody, Expression.Call(_miAnyWhere, _fieldsAccessor.Body, e.Current));
var predicate = Expression.Lambda<Func<Document, bool>>(predicateBody, _fieldsAccessor.Parameters);
return documents.Where(predicate);
}
}
Test:
var documents = (new[]
{
new Document
{
Id = 1,
Fields = new[]
{
new DocumentField { Id = 32, StringValue = "CET20533" },
new DocumentField { Id = 16, StringValue = "882341" },
new DocumentField { Id = 12, FloatValue = 101746F },
}
},
new Document
{
Id = 2,
Fields = new[]
{
new DocumentField { Id = 32, StringValue = "Bla" },
new DocumentField { Id = 16, StringValue = "882341" },
new DocumentField { Id = 12, FloatValue = 101746F },
}
}
}).AsQueryable();
var matches = documents.HavingAllFields(fieldPredicates).ToList();
Matches document 1, but not 2.
I usually do something like this: put all your desired Id's for your filter into a list, then use contains.
List<int> myDesiredIds = new List<int> { 1, 2, 3, 4, 5 };
db.documents.Where(x=>myDesiredIds.Contains(x.DocumentId));
I have picked up a project that uses the specification pattern, a pattern I have not used before, and I had to go and research the pattern. I have noticed it doesn't have OrderBy and Skip/Take functionality, and I can't find anywhere that shows how to implement this with the pattern.
I am struggling to think of how best to add this to the specification pattern. But I have hit issues, like the specification deals with "Expression<Func<T, bool>>" whereas I don't think I can store this along with orderby's etc
Basically there is a class like this:
public class Specification<T> : ISpecification<T>
{
public Expression<Func<T, bool>> Predicate { get; protected set; }
public Specification(Expression<Func<T, bool>> predicate)
{
Predicate = predicate;
}
public Specification<T> And(Specification<T> specification)
{
return new Specification<T>(this.Predicate.And(specification.Predicate));
}
public Specification<T> And(Expression<Func<T, bool>> predicate)
{
return new Specification<T>(this.Predicate.And(predicate));
}
public Specification<T> Or(Specification<T> specification)
{
return new Specification<T>(this.Predicate.Or(specification.Predicate));
}
public Specification<T> Or(Expression<Func<T, bool>> predicate)
{
return new Specification<T>(this.Predicate.Or(predicate));
}
public T SatisfyingItemFrom(IQueryable<T> query)
{
return query.Where(Predicate).SingleOrDefault();
}
public IQueryable<T> SatisfyingItemsFrom(IQueryable<T> query)
{
return query.Where(Predicate);
}
}
This allows to create a specification, passing in a where clause. It also allows chaining of rules with the "And", "Or". For example:
var spec = new Specification<Wave>(w => w.Id == "1").And(w => w.WaveStartSentOn > DateTime.Now);
How can I add a method for "OrderBy" and "Take"?
As this is existing code, I can't do any changes that would affect existing code, and it would be quite a job to refactor it. So any solution would need to play nicely with what is there.
How about
public class Specification<T> : ISpecification<T>
{
public Expression<Func<T, bool>> Predicate { get; protected set; }
public Func<IQueryable<T>, IOrderedQueryable<T>> Sort {get; protected set; }
public Func<IQueryable<T>, IQueryable<T>> PostProcess {get; protected set;
public Specification<T> OrderBy<TProperty>(Expression<Func<T, TProperty>> property)
{
var newSpecification = new Specification<T>(Predicate) { PostProcess = PostProcess } ;
if(Sort != null) {
newSpecification.Sort = items => Sort(items).ThenBy(property);
} else {
newSpecification.Sort = items => items.OrderBy(property);
}
return newSpecification;
}
public Specification<T> Take(int amount)
{
var newSpecification = new Specification<T>(Predicate) { Sort = Sort } ;
if(PostProcess!= null) {
newSpecification.PostProcess= items => PostProcess(items).Take(amount);
} else {
newSpecification.PostProcess= items => items.Take(amount);
}
return newSpecification;
}
public Specification<T> Skip(int amount)
{
var newSpecification = new Specification<T>(Predicate) { Sort = Sort } ;
if(PostProcess!= null) {
newSpecification.PostProcess= items => PostProcess(items).Skip(amount);
} else {
newSpecification.PostProcess= items => items.Skip(amount);
}
return newSpecification;
}
}
TODO:
similar construction for OrderByDescending
Update your other methods so the "Sort" value and "PostProcess" value is not lost when you call "And", for example
then your Satisfying methods become:
private IQueryable<T> Prepare(IQueryable<T> query)
{
var filtered = query.Where(Predicate);
var sorted = Sort(filtered);
var postProcessed = PostProcess(sorted);
return postProcessed;
}
public T SatisfyingItemFrom(IQueryable<T> query)
{
return Prepare(query).SingleOrDefault();
}
public IQueryable<T> SatisfyingItemsFrom(IQueryable<T> query)
{
return Prepare(query);
}
TODO: check if Sort & PostProcess are not null in the "Prepare" method
Usage:
var spec = new Specification<Wave>(w => w.Id == "1")
.And(w => w.WaveStartSentOn > DateTime.Now)
.OrderBy(w => w.WaveStartSentOn)
.Skip(20)
.Take(5);