I am using JQuery widgets's datatable to have a table control in an MVC application with server-side paging, sorting and filtering. When doing the filtering, I have the following method:
private IQueryable<ImportRfqViewModel> BuildLinqQuery(System.Collections.Specialized.NameValueCollection query)
{
var result = query.GetValues("filterslength");
var filtersCount = int.Parse(query.GetValues("filterslength")[0]);
if (result == null || filtersCount == 0)
{
return AllImportRfq();
}
Predicate<ImportRfqViewModel> orResultPredicate = PredicateExtensions.False<ImportRfqViewModel>();
for (var i = 0; i < filtersCount; i += 1)
{
var filterValue = query.GetValues("filtervalue" + i)[0].ToUpper();
var filterCondition = query.GetValues("filtercondition" + i)[0];
var filterDataField = query.GetValues("filterdatafield" + i)[0];
var filterOperator = query.GetValues("filteroperator" + i)[0];
if (filterDataField == "ImportRfqId")
{
Predicate<ImportRfqViewModel> predicate = p => p.ImportRfqId.ToString().Contains(filterValue);
orResultPredicate = orResultPredicate.Or(predicate);
}
else if (filterDataField == "DateCreated")
{
Predicate<ImportRfqViewModel> predicate = p => p.DateCreated.ToString("yyyy/MM/dd hh:mm:ss").Contains(filterValue);
orResultPredicate = orResultPredicate.Or(predicate);
}
...
}
Func<ImportRfqViewModel, bool> funcOr = l => orResultPredicate(l);
var allResearch = AllImportRfq().Where(funcOr).AsQueryable();
return allResearch;
}
I'm using the predicates in order to chain Or conditions. I obviously want to return an IQueryable so that the query is not run before I get to the part:
dbResult = dbResult.Skip(pagesize * pagenum).Take(pagesize);
The result of the predicate though, is an IEnumerable. This is why I call the .AsQueryable();
What I'm concerned about and don't know is whether the thing potentially is enumerating the query and then returning a IQueryable afterwards and throwing away the enumeration, if I'm making sense.
In all this I'm assuming that if I return an IEnumerable, it would execute the query.
Returning an IEnumerable by itself does not mean you are actually executing the query. It depends on the IEnumerable's creator, but normally IEnumerable is considered lazy and the underlying data source is touched only when enumerating an IEnumerator. Linq operators and extension methods behave nicely, so your AsQueryable() call does not enumerate the target, and you should be safe.
BTW, did you try changing your code from this:
Func<ImportRfqViewModel, bool> funcOr = l => orResultPredicate(l);
to this (as long as your query provider fully supports Where and the predicate you are building, of course)?
Expression<Func<ImportRfqViewModel, bool>> funcOr = ...;
This way you should be able to avoid going back and forth across IQueryable and IEnumerable, you would not need calling AsQueryable() at all and therefore you'd have no more doubts. If your query provider supports it, it would be much more efficient than what you are doing, which would execute the filtering after querying the underlying data source.
EDIT: just to make it clearer, the move from Func<> to Expression<> is not just a matter of changing the type of your variable, it actually means you'd have to review the whole process of building your predicates so that you always work by composing Expression instances, and not just Func ones, otherwise your resulting expression tree will hardly be understandable to your query provider.
Related
I need to get result from a function that it need to run in LINQ query. This result bind to grid but in run time I encounter with this error:
LINQ to Entities does not recognize the method 'System.String
GetName(System.Type, System.Object)' method, and this method cannot be
translated into a store expression.
This is my Code:
public IQueryable GetForRah_CapacityList(XQueryParam param)
{
var result = (from x in Data()
select new
{
Rah_CapacityId = x.Rah_CapacityId,
Rah_CapacityName = x.Rah_CapacityName,
Rah_St = Enum.GetName(typeof(Domain.Enums.CapacityState), x.Rah_St),
Rah_LinesId = x.Rah_LinesId
}).OrderByDescending(o => new { o.Rah_CapacityId });
return result;
}
GetName couldn't be translated to T-SQL, Linq to Entities couldn't recognize it. You can modify the code as below:
var result = (from x in Data().AsEnumerable()
select new
{
Rah_CapacityId = x.Rah_CapacityId,
Rah_CapacityName = x.Rah_CapacityName,
Rah_St = Enum.GetName(typeof(Domain.Enums.CapacityState), x.Rah_St),
Rah_LinesId = x.Rah_LinesId
}).OrderByDescending(o => new { o.Rah_CapacityId });
With .ToList() after data is loaded, any further operation (such as select) is performed using Linq to Objects, on the data already in memory.
EDIT: Also your method's return type is IQueryable while your query is IOrderedEnumerable of anonymous type, so you should either change the method's type to System.Object or as a better solution create a class, send the values into the class's properties, and then return it.
You can't use this method in Linq-To-Entities because LINQ does not know how to translate Enum.GetName to sql. So execute it in memory with Linq-To-Objects by using AsEnumerable and after the query use AsQueryable to get the desired AsQueryable:
So either:
var result = Data()
.OrderBy(x=> x.CapacityId)
.AsEnumerable()
.Select(x => new
{
Rah_CapacityId = x.Rah_CapacityId,
Rah_CapacityName = x.Rah_CapacityName,
Rah_St = Enum.GetName(typeof(Domain.Enums.CapacityState), x.Rah_St),
Rah_LinesId = x.Rah_LinesId
})
.AsQueryable();
You should first use OrderBy before you use AsEnumerable to benefit from database sorting performance. The same applies to Where, always do this before AsEnumerable(or ToList).
I have an app written with C#. I have a collection of objects and I need to perform tasks like sorting and jumping to a page of the objects. Currently, I have something that looks like this:
this.Items = model.Items
.Where(x => x.Id > 0)
.ToList();
if (SortBy == "Name")
{
if (Direction == "Descending")
this.Items = (this.Items.OrderByDescending(x => x.Name)).ToList();
else
this.Items = (this.Items.OrderBy(x => x.Name)).ToList();
}
else if (SortBy == "Description")
{
if (Direction == "Descending")
this.Items = (this.Items.OrderByDescending(x => x.Description)).ToList();
else
this.Items = (this.Items.OrderBy(x => x.Description)).ToList();
}
// I want to jump to the second page
this.Items = this.Items.Skip(26).ToList();
Is there a better way to do this? All of the ToList calls seem like that would be performance draining. Maybe I'm wrong though.
Thanks!
You can defer calling .ToList until your last line since based on your logic it always runs. Interim logic builds upon an IQueryable. Whether or not ToList has a significant impact largerly depends on whether this is Linq2Objects or it is Linq2SQL/Entitity Framework/Some other Database ORM. With database ORMs typically each .ToList hydrates the list and results in a database query executed. For Linq2Objects there is still a performance cost, but no where near the magnitude involved in a database call.
// itemsQuery will be of type IQueryable or IEnumerable depending on type of Items
var itemsQuery = model.Items.Where(x => x.Id > 0);
if (SortBy == "Description")
{
// we'll chain additional criteria or sorts to the IQueryable,
// and update the query reference by assigning over the previous itemsQuery variable
if (Direction == "Descending")
itemsQuery = itemsQuery.OrderByDescending(x => x.Description);
else
itemsQuery = itemsQuery.OrderBy(x => x.Description);
}
// I want to jump to the second page
this.Items = itemsQuery.Skip(26).ToList();
Essentially you are dynamically building up a query, but not actually running the query until the end where you call ToList, which will essentially build an SQL query from the expression tree and execute the entire chained sequence of .Where(...).OrderBy(...).Skip(26).ToList
You're quite right that all of the ToList calls are a major problem. Not only are they forcing all of the work to be done in the client if the underlying data source represents a database query provider, but even if it is inherently a LINQ to objects query you're needlessly creating, populating, and discarding lists. You're also removing any ability for deferred execution. Since you're not actually getting anything out of those calls, you can simply remove them all.
Next, you can write a method to create a selector, so that you don't need to duplicate all of that code for each column you sort on:
public static Expression<Func<T, TResult>> CreateSelector<T, TResult>(
string memberName)
{
var param = Expression.Parameter(typeof(T));
var body = Expression.PropertyOrField(param, memberName);
return Expression.Lambda<Func<T, TResult>>(body, param);
}
This allows you to write:
IQueryable<Item> items = model.Items;
if (Direction == "Descending")
items = items.OrderByDescending(CreateSelector<Item, string>(SortBy));
else
items = items.OrderBy(CreateSelector<Item, string>(SortBy));
items = items.Skip(26);
If the underlying data source that you have is an IEnumerable, not an IQueryable, just change the type of Items and Compile the selector expression.
Let say I have a function like this:
var filterValue = GetCurrentFilter(state);
And then an EF query:
var result = context.EntitySet.Where(x=> x.column > filterValue);
this works, but as soon as I try to inline that:
var result = context.EntitySet.Where(x=> x.column > GetCurrentFilter(state));
It does not because EF Linq tried to parse GetCurrentFilter into expression tree and is unable to do that. This is all quite understandable.
My question is, is there a way to let EF Linq know that in needs to execute the GetCurrentFilter function when it builds the tree and use its result in the tree?
Something like
var result = context.EntitySet.Where(x=> x.column > EfUtil.ResultOf(GetCurrentFilter(state)));
Since GetCurrentFilter does not have parameters that is a part of the query this should be technically possible to do that if EF Linq can support it that is. I'm suspecting that I'm just missing the correct syntax for that.
Make GetCurrentFilter a (read only) property instead of a method. EF will evaluate properties to their values, rather than trying to translate them into SQL, unlike methods.
The only other road that you have is to traverse the entire expression tree, search for usage of your ResultOf method, evaluate its parameter to a value, and then inline that value where the ResultOf call once was, rebuiding the query around that value.
In order for this to work it means you need to not only wrap the code you want to inline in a call to EfUtil.ResultOf, but it also means calling a method on the query itself to force it to go back and evaluate it:
public class EfUtil
{
public static T ResultOf<T>(T value)
{
return value;
}
}
//Note this could probably use a better name
public static IQueryable<T> EvaluateResults<T>(this IQueryable<T> query)
{
return query.Provider.CreateQuery<T>(
new ExpressionEvaluator().Visit(query.Expression));
}
internal class ExpressionEvaluator : ExpressionVisitor
{
protected override Expression VisitMethodCall(MethodCallExpression m)
{
if (m.Method.Name == "ResultOf" && m.Method.DeclaringType == typeof(EfUtil))
{
Expression target = m.Arguments[0];
object result = Expression.Lambda(target)
.Compile()
.DynamicInvoke();
return Expression.Constant(result, target.Type);
}
else
return base.VisitMethodCall(m);
}
}
This would allow you to write:
var result = context.EntitySet.Where(x=> x.column > EfUtil.ResultOf(GetCurrentFilter(state)))
.EvaluateResults();
It would then evaluate GetCurrentFilter(state) on the client side and inline the result as a constant into the query.
As a slightly simpler test, we can write the following:
var query = new[] { 1, 2, 3 }
.AsQueryable()
.Where(x => x > EfUtil.ResultOf(Math.Max(1, 2)))
.EvaluateResults();
Console.WriteLine(query.ToString());
And it will print out:
System.Int32[].Where(x => (x > 2))
Which is exactly what we want.
Note that the use of the lambda's parameter (x in these examples) cannot be used anywhere within the call to EfUtil.ResultOf or the code won't work, and couldn't possibly be made to work (although we could generate a better error message if we cared enough).
Is it possible using ASP.NET Web APi OData make similar:
List<string> customersTitles = Odata.OrdersService.Select(o=>o.CustomerTitle).Distinct().ToList();
List<Customer> customers = Odata.CustomerService.Where(m => customersTitles .Contains(m.CustomerTitle))
Get error:
Error translating Linq expression to URI: The expression
value(System.Collections.Generic.List`1[System.String]).Contains([10007].CustomerTitle)
is not supported.}
API:
public class CustomerController : EntitySetController<Customer, int>
{
[Queryable]
public override IQueryable<Customer> Get()
{
Expression filter = this.QueryOptions.Filter.ToExpression<Customer>();
return db.Query<Customer>(filter as Expression<Func<Customer, bool>>);
}
}
The Contains construct is not supported via URIs since the list of strings existing at the client side is not a Server side resource.
Linq2Sql Provider has an innate translation for Contains, which gets translated to IN clause for SQL.
With OData, such a translation is not supported. What you need to build is an expanded query list for your where clause using all the Title values:
Because this does not work:
List<Customer> customers = Odata.CustomerService.Where(m => customersTitles .Contains(m.CustomerTitle))
the expanded query option helps us in building a query like:
List<Customer> customers = Odata.CustomerService.Where(m => m.CustomerTitle == customerTitles[0] || m.CustomerTitle == customerTitles[1]); // and so on
Here is the code for the filter building:
var titleFilterList = customerTitles.Select(title => String.Format("(CustomerTitle eq {0})", title));
var titleFilter = String.Join(" or ", titleFilterList);
var customers = Odata.CustomerService.AddQueryOption("$filter", titleFilter).Execute().ToList(); // you may have to cast this.
There is another option to do the same in a strongly typed manner using a nice extension method and building a dynamic Expression based predicate. Follow the steps from here:
http://blogs.msdn.com/b/phaniraj/archive/2008/07/17/set-based-operations-in-ado-net-data-services.aspx
The following extension method can be used LINQ to perform OData queries that test if the value of a property is contained in a set similar to how Contains works with LINQ to EF. It is based on the link provided in Raja Nadar's answer, specifically the comment at the end by Nick.
public static IQueryable<T> WherePropertyIsIn<T, TSet>(
this IQueryable<T> query,
IEnumerable<TSet> set,
Expression<Func<T, TSet>> propertyExpression
) {
var filterPredicate = set.Select(value => Expression.Equal(propertyExpression.Body, Expression.Constant(value)))
.Aggregate<Expression, Expression>(Expression.Constant(false), Expression.Or);
var filterLambdaExpression = Expression.Lambda<Func<T, bool>>(filterPredicate, propertyExpression.Parameters.Single());
return query.Where(filterLambdaExpression);
}
Usage:
var allowed_states = getAllowedStates();
var maxPopulation = getMaxPopulation();
// Instead of...
var cities = context.Cities.Where(c => allowed_states.Contains(c.State) && c.Population <= maxPopulation);
// Use...
var cities = context.Cities.Where(c => c.Population <= maxPopulation).WherePropertyIsIn(allowed_states, c => c.Cities);
Note that you must have a separate Where call (as shown above) if you wish to filter by more than WherePropertyIsIn allows. It would be nice if this could be combined into a single Where but I couldn't figure out how.
I have run into issues with Timothy's solution when using it against Configuration Manager's OData service. CM utilises WML internally and therefore is even more limited in what can be parsed.
I have changed the code to generate a bit 'cleaner' query, which avoid using Expression.Constant(false) as aggregate base -> it produces 0 that is not accepted by WML
public static IQueryable<T> WherePropertyIsIn<T, TSet>(this IQueryable<T> query,
IEnumerable<TSet> valuesList, Expression<Func<T, TSet>> propertySelector)
{
if (valuesList == null) throw new ArgumentNullException(nameof(valuesList));
//if there are no values, no entities can fullfil the condition -> return empty
if (!valuesList.Any())
return Enumerable.Empty<T>().AsQueryable();
//create a check for each value
var filters = valuesList.Select(value => Expression.Equal(propertySelector.Body, Expression.Constant(value)));
//build an expression aggregating checks with OR, use first check as starter (could be '0', but doesn't get mapped to WML)
var firstCheck = filters.First();
//we could duplicate first check, but why not just skip it
var filterPredicate = filters.Skip(1).Aggregate(firstCheck, (Func<Expression, Expression, Expression>)Expression.Or);
var filterLambdaExpression = Expression.Lambda<Func<T, bool>>(filterPredicate, propertySelector.Parameters.Single());
return query.Where(filterLambdaExpression);
}
}
A side effect is that when there are no values passed as argument, the query will immediately return an empty result, which may save time comparing to processing it by external service. At the same time it may be undesired if query is meant to actually run on server.
I need to do some filtering on an ObjectSet to obtain the entities I need by doing this :
query = this.ObjectSet.Where(x => x.TypeId == 3); // this is just an example;
Later in the code (and before launching the deferred execution) I filter the query again like this :
query = query.Where(<another lambda here ...>);
That works quite well so far.
Here is my problem :
The entities contains a DateFrom property and a DateTo property, which are both DataTime types. They represent a period of time.
I need to filter the entities to get only those that are part of a collection of periods of time. The periods in the collection are not necessarily contiguous, so, the logic to retreive the entities looks like that :
entities.Where(x => x.DateFrom >= Period1.DateFrom and x.DateTo <= Period1.DateTo)
||
entities.Where(x => x.DateFrom >= Period2.DateFrom and x.DateTo <= Period2.DateTo)
||
... and on and on for all the periods in the collection.
I have tried doing that :
foreach (var ratePeriod in ratePeriods)
{
var period = ratePeriod;
query = query.Where(de =>
de.Date >= period.DateFrom && de.Date <= period.DateTo);
}
But once I launch the deferred execution, it translates this into SQL just like I want it (one filter for each of the periods of time for as many periods there is in the collection), BUT, it translates to AND comparisons instead of OR comparisons, which returns no entities at all, since an entity cannot be part of more than one period of time, obviously.
I need to build some sort of dynamic linq here to aggregate the period filters.
Update
Based on hatten's answer, I've added the following member :
private Expression<Func<T, bool>> CombineWithOr<T>(Expression<Func<T, bool>> firstExpression, Expression<Func<T, bool>> secondExpression)
{
// Create a parameter to use for both of the expression bodies.
var parameter = Expression.Parameter(typeof(T), "x");
// Invoke each expression with the new parameter, and combine the expression bodies with OR.
var resultBody = Expression.Or(Expression.Invoke(firstExpression, parameter), Expression.Invoke(secondExpression, parameter));
// Combine the parameter with the resulting expression body to create a new lambda expression.
return Expression.Lambda<Func<T, bool>>(resultBody, parameter);
}
Declared a new CombineWithOr Expression :
Expression<Func<DocumentEntry, bool>> resultExpression = n => false;
And used it in my period collection iteration like this :
foreach (var ratePeriod in ratePeriods)
{
var period = ratePeriod;
Expression<Func<DocumentEntry, bool>> expression = de => de.Date >= period.DateFrom && de.Date <= period.DateTo;
resultExpression = this.CombineWithOr(resultExpression, expression);
}
var documentEntries = query.Where(resultExpression.Compile()).ToList();
I looked at the resulting SQL and it's like the Expression has no effect at all. The resulting SQL returns the previously programmed filters but not the combined filters. Why ?
Update 2
I wanted to give feO2x's suggestion a try, so I have rewritten my filter query like this :
query = query.AsEnumerable()
.Where(de => ratePeriods
.Any(rp => rp.DateFrom <= de.Date && rp.DateTo >= de.Date))
As you can see, I added AsEnumerable() but the compiler gave me an error that it cannot convert the IEnumerable back to IQueryable, so I have added ToQueryable() at the end of my query :
query = query.AsEnumerable()
.Where(de => ratePeriods
.Any(rp => rp.DateFrom <= de.Date && rp.DateTo >= de.Date))
.ToQueryable();
Everything works fine. I can compile the code and launch this query. However, it doesn't fit my needs.
While profiling the resulting SQL, I can see that the filtering is not part of the SQL query because it filters the dates in-memory during the process. I guess that you already know about that and that is what you intended to suggest.
Your suggestion works, BUT, since it fetches all the entities from the database (and there are thousands and thousands of them) before filtering them in-memory, it's really slow to get back that huge amount from the database.
What I really want is to send the period filtering as part of the resulting SQL query, so it won't return a huge amount of entities before finishing up with the filtering process.
Despite the good suggestions, I had to go with the LinqKit one. One of the reasons is that I will have to repeat the same kind of predicate aggregation in many other places in the code. Using LinqKit is the easiest one, not to mention I can get it done by writing only a few lines of code.
Here is how I solved my problem using LinqKit :
var predicate = PredicateBuilder.False<Document>();
foreach (var submittedPeriod in submittedPeriods)
{
var period = period;
predicate = predicate.Or(d =>
d.Date >= period.DateFrom && d.Date <= period.DateTo);
}
And I launch deferred execution (note that I call AsExpandable() just before) :
var documents = this.ObjectSet.AsExpandable().Where(predicate).ToList();
I looked at the resulting SQL and it does a good job at translating my predicates into SQL.
You can use a method like the following:
Expression<Func<T, bool>> CombineWithOr<T>(Expression<Func<T, bool>> firstExpression, Expression<Func<T, bool>> secondExpression)
{
// Create a parameter to use for both of the expression bodies.
var parameter = Expression.Parameter(typeof(T), "x");
// Invoke each expression with the new parameter, and combine the expression bodies with OR.
var resultBody = Expression.Or(Expression.Invoke(firstExpression, parameter), Expression.Invoke(secondExpression, parameter));
// Combine the parameter with the resulting expression body to create a new lambda expression.
return Expression.Lambda<Func<T, bool>>(resultBody, parameter);
}
And then:
Expression<Func<T, bool>> resultExpression = n => false; // Always false, so that it won't affect the OR.
foreach (var ratePeriod in ratePeriods)
{
var period = ratePeriod;
Expression<Func<T, bool>> expression = (de => de.Date >= period.DateFrom && de.Date <= period.DateTo);
resultExpression = CombineWithOr(resultExpression, expression);
}
// Don't forget to compile the expression in the end.
query = query.Where(resultExpression.Compile());
For more information, you may want to check out the following:
Combining two expressions (Expression<Func<T, bool>>)
http://www.albahari.com/nutshell/predicatebuilder.aspx
Edit: The line Expression<Func<DocumentEntry, bool>> resultExpression = n => false; is just a placeholder. CombineWithOr method needs two methods to combine, if you write Expression<Func<DocumentEntry, bool>> resultExpression;', you can't use it in the call toCombineWithOrfor the first time in yourforeach` loop. It's just like the following code:
int resultOfMultiplications = 1;
for (int i = 0; i < 10; i++)
resultOfMultiplications = resultOfMultiplications * i;
If there's nothing in resultOfMultiplications to begin with, you cannot use it in your loop.
As to why the lambda is n => false. Because it doesn't have any effect in an OR statement. For example, false OR someExpression OR someExpression is equal to someExpression OR someExpression. That false doesn't have any effect.
How about this code:
var targets = query.Where(de =>
ratePeriods.Any(period =>
de.Date >= period.DateFrom && de.Date <= period.DateTo));
I use the LINQ Any operator to determine if there is any rate period that conforms to de.Date. Although I'm not quite sure how this is translated into efficient SQL statements by entity. If you could post the resulting SQL, that would be quite interesting for me.
Hope this helps.
UPDATE after hattenn's answer:
I don't think that hattenn's solution would work, because Entity Framework uses LINQ expressions to produce the SQL or DML that is executed against the database. Therefore, Entity Framework relies on the IQueryable<T> interface rather than IEnumerable<T>. Now the default LINQ operators (like Where, Any, OrderBy, FirstOrDefault and so on) are implemented on both interfaces, thus the difference is sometimes hard to see. The main difference of these interfaces is that in case of the IEnumerable<T> extension methods, the returned enumerables are continuously updated without side effects, while in the case of IQueryable<T> the actual expression is recomposed, which is not free of side effects (i.e. you are altering the expression tree that is finally used to create the SQL query).
Now Entity Framework supports the ca. 50 standard query operators of LINQ, but if you write your own methods that manipulate an IQueryable<T> (like hatenn's method), this would result in an expression tree that Entity Framework might not be able to parse because it simply doesn't know the new extension method. This might be the cause why you cannot see the combined filters after you composed them (although I would expect an exception).
When does the solution with the Any operator work:
In the comments, you told that you encountered a System.NotSupportedException: Unable to create a constant value of type 'RatePeriod'. Only primitive types or enumeration types are supported in this context. This is the case when the RatePeriod objects are in-memory objects and not tracked by the Entity Framework ObjectContext or DbContext. I made a small test solution that can be downloaded from here: https://dl.dropboxusercontent.com/u/14810011/LinqToEntitiesOrOperator.zip
I used Visual Studio 2012 with LocalDB and Entity Framework 5. To see the results, open the class LinqToEntitiesOrOperatorTest, then open Test Explorer, build the solution and run all tests. You will recognize that ComplexOrOperatorTestWithInMemoryObjects will fail, all the others should pass.
The context I used looks like this:
public class DatabaseContext : DbContext
{
public DbSet<Post> Posts { get; set; }
public DbSet<RatePeriod> RatePeriods { get; set; }
}
public class Post
{
public int ID { get; set; }
public DateTime PostDate { get; set; }
}
public class RatePeriod
{
public int ID { get; set; }
public DateTime From { get; set; }
public DateTime To { get; set; }
}
Well, it is as simple as it gets :-). In the test project, there are two important unit test methods:
[TestMethod]
public void ComplexOrOperatorDBTest()
{
var allAffectedPosts =
DatabaseContext.Posts.Where(
post =>
DatabaseContext.RatePeriods.Any(period => period.From < post.PostDate && period.To > post.PostDate));
Assert.AreEqual(3, allAffectedPosts.Count());
}
[TestMethod]
public void ComplexOrOperatorTestWithInMemoryObjects()
{
var inMemoryRatePeriods = new List<RatePeriod>
{
new RatePeriod {ID = 1000, From = new DateTime(2002, 01, 01), To = new DateTime(2006, 01, 01)},
new RatePeriod {ID = 1001, From = new DateTime(1963, 01, 01), To = new DateTime(1967, 01, 01)}
};
var allAffectedPosts =
DatabaseContext.Posts.Where(
post => inMemoryRatePeriods.Any(period => period.From < post.PostDate && period.To > post.PostDate));
Assert.AreEqual(3, allAffectedPosts.Count());
}
Notice that the first method passes while the second one fails with the exception mentioned above, although both methods do exactly the same thing, except that in the second case I created rate period objects in memory the DatabaseContext does not know about.
What can you do to solve this problem?
Do your RatePeriod objects reside in the same ObjectContext or DbContext, respectively? Then use them right from it like I did in the first unit test mentioned above.
If not, can you load all your posts at once or would this result in an OutOfMemoryException? If not, you could use the following code. Notice the AsEnumerable() call that results in the Where operator being used against the IEnumerable<T> interface instead of IQueryable<T>. Effectively, this results in all posts being loaded into memory and then filtered:
[TestMethod]
public void CorrectComplexOrOperatorTestWithInMemoryObjects()
{
var inMemoryRatePeriods = new List<RatePeriod>
{
new RatePeriod {ID = 1000, From = new DateTime(2002, 01, 01), To = new DateTime(2006, 01, 01)},
new RatePeriod {ID = 1001, From = new DateTime(1963, 01, 01), To = new DateTime(1967, 01, 01)}
};
var allAffectedPosts =
DatabaseContext.Posts.AsEnumerable()
.Where(
post =>
inMemoryRatePeriods.Any(
period => period.From < post.PostDate && period.To > post.PostDate));
Assert.AreEqual(3, allAffectedPosts.Count());
}
If the second solution is not possible, then I would recommend to write a TSQL stored procedure where you pass in your rate periods and that forms the correct SQL statement. This solution is also the most performant one.
Anyways, I think dynamic LINQ query creation was not as simple as I thought. Try using Entity SQL, similar to the way below:
var filters = new List<string>();
foreach (var ratePeriod in ratePeriods)
{
filters.Add(string.Format("(it.Date >= {0} AND it.Date <= {1})", ratePeriod.DateFrom, ratePeriod.DateTo));
}
var filter = string.Join(" OR ", filters);
var result = query.Where(filter);
This may not be exactly correct (I haven't tried it), but it should be something similar to this.