Is it possible using ASP.NET Web APi OData make similar:
List<string> customersTitles = Odata.OrdersService.Select(o=>o.CustomerTitle).Distinct().ToList();
List<Customer> customers = Odata.CustomerService.Where(m => customersTitles .Contains(m.CustomerTitle))
Get error:
Error translating Linq expression to URI: The expression
value(System.Collections.Generic.List`1[System.String]).Contains([10007].CustomerTitle)
is not supported.}
API:
public class CustomerController : EntitySetController<Customer, int>
{
[Queryable]
public override IQueryable<Customer> Get()
{
Expression filter = this.QueryOptions.Filter.ToExpression<Customer>();
return db.Query<Customer>(filter as Expression<Func<Customer, bool>>);
}
}
The Contains construct is not supported via URIs since the list of strings existing at the client side is not a Server side resource.
Linq2Sql Provider has an innate translation for Contains, which gets translated to IN clause for SQL.
With OData, such a translation is not supported. What you need to build is an expanded query list for your where clause using all the Title values:
Because this does not work:
List<Customer> customers = Odata.CustomerService.Where(m => customersTitles .Contains(m.CustomerTitle))
the expanded query option helps us in building a query like:
List<Customer> customers = Odata.CustomerService.Where(m => m.CustomerTitle == customerTitles[0] || m.CustomerTitle == customerTitles[1]); // and so on
Here is the code for the filter building:
var titleFilterList = customerTitles.Select(title => String.Format("(CustomerTitle eq {0})", title));
var titleFilter = String.Join(" or ", titleFilterList);
var customers = Odata.CustomerService.AddQueryOption("$filter", titleFilter).Execute().ToList(); // you may have to cast this.
There is another option to do the same in a strongly typed manner using a nice extension method and building a dynamic Expression based predicate. Follow the steps from here:
http://blogs.msdn.com/b/phaniraj/archive/2008/07/17/set-based-operations-in-ado-net-data-services.aspx
The following extension method can be used LINQ to perform OData queries that test if the value of a property is contained in a set similar to how Contains works with LINQ to EF. It is based on the link provided in Raja Nadar's answer, specifically the comment at the end by Nick.
public static IQueryable<T> WherePropertyIsIn<T, TSet>(
this IQueryable<T> query,
IEnumerable<TSet> set,
Expression<Func<T, TSet>> propertyExpression
) {
var filterPredicate = set.Select(value => Expression.Equal(propertyExpression.Body, Expression.Constant(value)))
.Aggregate<Expression, Expression>(Expression.Constant(false), Expression.Or);
var filterLambdaExpression = Expression.Lambda<Func<T, bool>>(filterPredicate, propertyExpression.Parameters.Single());
return query.Where(filterLambdaExpression);
}
Usage:
var allowed_states = getAllowedStates();
var maxPopulation = getMaxPopulation();
// Instead of...
var cities = context.Cities.Where(c => allowed_states.Contains(c.State) && c.Population <= maxPopulation);
// Use...
var cities = context.Cities.Where(c => c.Population <= maxPopulation).WherePropertyIsIn(allowed_states, c => c.Cities);
Note that you must have a separate Where call (as shown above) if you wish to filter by more than WherePropertyIsIn allows. It would be nice if this could be combined into a single Where but I couldn't figure out how.
I have run into issues with Timothy's solution when using it against Configuration Manager's OData service. CM utilises WML internally and therefore is even more limited in what can be parsed.
I have changed the code to generate a bit 'cleaner' query, which avoid using Expression.Constant(false) as aggregate base -> it produces 0 that is not accepted by WML
public static IQueryable<T> WherePropertyIsIn<T, TSet>(this IQueryable<T> query,
IEnumerable<TSet> valuesList, Expression<Func<T, TSet>> propertySelector)
{
if (valuesList == null) throw new ArgumentNullException(nameof(valuesList));
//if there are no values, no entities can fullfil the condition -> return empty
if (!valuesList.Any())
return Enumerable.Empty<T>().AsQueryable();
//create a check for each value
var filters = valuesList.Select(value => Expression.Equal(propertySelector.Body, Expression.Constant(value)));
//build an expression aggregating checks with OR, use first check as starter (could be '0', but doesn't get mapped to WML)
var firstCheck = filters.First();
//we could duplicate first check, but why not just skip it
var filterPredicate = filters.Skip(1).Aggregate(firstCheck, (Func<Expression, Expression, Expression>)Expression.Or);
var filterLambdaExpression = Expression.Lambda<Func<T, bool>>(filterPredicate, propertySelector.Parameters.Single());
return query.Where(filterLambdaExpression);
}
}
A side effect is that when there are no values passed as argument, the query will immediately return an empty result, which may save time comparing to processing it by external service. At the same time it may be undesired if query is meant to actually run on server.
Related
How to have the select query projection to a lambda expression in Entity framework..
In repo class
database.Employees.Where(s => s.EmpID == curEmp.ID)
.Select(s => new EmployeeModel
{
Value = s.Name,
Text = s.ID
// I have 40 fields and 10 join table projections here so much code
});
Since I have many fields to project, so created a separate class with the lambda expressions for those long select projections.
public class ProjectionQueries
{
public static readony Fun<Employee, EmployeeModel> GetEmployeeAsModel = (emp)
=> new EmployeeModel
{
Value = s.Name,
Text = s.ID...
...
..
..
Address = new AddressModel{.....},
Country = new CountryModel{.....}
};
}
Then called the above lambda expression from the select clause of the repository method.
database.Employees.Where(s => s.EmpID == curEmp.ID)
.Select(x => ProjectionQueries.GetEmployeeAsModel(x));
This way it is well segregated the code. But when I do this I get an error saying
The client projection contains a reference to a constant expression of the System.Func<*,*>. This could potentially cause a memory leak; consider assigning this constant to a local variable and using the variable in the query instead.
Can't use this way to make my code less in the repository class. ?
You could separate this out as a static extension method.
public static class EmployeeExtensions{
public static IQueryable<EmployeeModel> AsEmployeeModels(this IQueryable<Employee> emps){
return emps.Select(emp=> new EmployeeModel
{
Value = emp.Name,
Text = emp.ID...
});
}
}
and use it as:
database.Employees.Where(s => s.EmpID == curEmp.ID).AsEmployeeModels();
The #gokul-panigrahi answer show a solution with extension method, that I advice to use because I found the syntax smoothest.
The error is :
The client projection contains a reference to a constant expression of the System.Func<,>. This could potentially cause a memory leak; consider assigning this constant to a local variable and using the variable in the query instead.
The error indicate EF Core can't translate System.Func<*,*> to SGBD language. EF Core client evaluation. A solution is to execute the untranslatable part in client :
database.Employees
// 1) Prepare Query
.Where(s => s.EmpID == curEmp.ID)
// 2) Execute query on server
.AsEnumerable()
// 3) Next is execute in client
.Select(x => ProjectionQueries.GetEmployeeAsModel(x));
See the ms-doc for more information about Client vs Server Evaluation
But client evaluation could result in poor performance. In your case, if the projection can limit the number of column read in database (and limit the data transmit on network).
A other solution, it is to modify the type of GetEmployeeAsModel :
public static readony Expression<Fun<Employee, EmployeeModel>> GetEmployeeAsModel = ...
Thus EF Core can try translate the expression to SGBD's language. Try because the expression can use some term untranslatable like Func<*,*>.
What is the difference between Func and Expression<Func>?
A Func is delegate (trivially a pointer to a zone code in memory). EF Core don't know how translate a delegate.
A Expression is a specific structure of instruction, that is possible to explore :
Expression<Func<int,int>> expression = (x) => x % 2;
// Explore the expression
var operation = expression.Body as BinaryExpression;
var leftPart = operation.Left as ParameterExpression;
var rightPart = operation.Right as ConstantExpression;
// Translate
Console.WriteLine("#" + leftPart.Name);
// #x
Console.WriteLine(operation.NodeType);
// modulo
Console.WriteLine(rightPart.Value);
// 2
EF Core do this to translate the expression to SGBD's language, but sometime the expression contains a term unknown by EF Core. Then this produce a error like above.
Can someone explain to me why the EF Engine is failing in the following scenario?
It works fine with the following expression:
var data = context.Programs
.Select(d => new MyDataDto
{
ProgramId = d.ProgramId,
ProgramName = d.ProgramName,
ClientId = d.ClientId,
Protocols = d.Protocols.Where(p => p.UserProtocols.Any(u => u.UserId == userId))
.Count(pr => pr.Programs.Any(pg => pg.ProgramId == d.ProgramId))
})
.ToList();
But if I encapsulate some into an extension method:
public static IQueryable<Protocol> ForUser(this IQueryable<Protocol> protocols, int userId)
{
return protocols.Where(p => p.UserProtocols.Any(u => u.UserId == userId));
}
The resulting query:
var data = context.Programs
.Select(d => new MyDataDto
{
ProgramId = d.ProgramId,
ProgramName = d.ProgramName,
ClientId = d.ClientId,
Protocols = d.Protocols.ForUser(userId)
.Count(pr => pr.Programs.Any(pg => pg.ProgramId == d.ProgramId))
})
.ToList();
Fails with the exception: LINQ to Entities does not recognize the method 'System.Linq.IQueryable1[DAL.Protocol] ForUser(System.Linq.IQueryable1[DAL.Protocol], Int32)' method, and this method cannot be translated into a store expression.
I would expect the EF Engine to build the entire expression tree, chaining the necessary expressions and then generate the SQL. Why doesn't it do that?
This is happening because the call to ForUser() is being made inside of the expression tree that the C# compiler builds when it sees the lambda you pass into Select. Entity Framework tries to figure out how to convert that function into SQL, but it can't invoke the function for a few reasons (e.g. d.Protocols does not exist at the moment).
The simplest approach that works for a case like this is to have your helper return a criteria lambda expression, and then pass that into the .Where() method yourself:
public static Expression<Func<Protocol, true>> ProtocolIsForUser(int userId)
{
return p => p.UserProtocols.Any(u => u.UserId == userId);
}
...
var protocolCriteria = Helpers.ProtocolIsForUser(userId);
var data = context.Programs
.Select(d => new MyDataDto
{
ProgramId = d.ProgramId,
ProgramName = d.ProgramName,
ClientId = d.ClientId,
Protocols = d.Protocols.Count(protocolCriteria)
})
.ToList();
More information
When you invoke a LINQ method outside of an expression tree (like you do with context.Programs.Select(...)), the Queryable.Select() extension method actually gets invoked, and its implementation returns an IQueryable<> that represents the extension method getting called on the original IQueryable<>. Here's the implementation of Select, for instance:
public static IQueryable<TResult> Select<TSource,TResult>(this IQueryable<TSource> source, Expression<Func<TSource, TResult>> selector) {
if (source == null)
throw Error.ArgumentNull("source");
if (selector == null)
throw Error.ArgumentNull("selector");
return source.Provider.CreateQuery<TResult>(
Expression.Call(
null,
GetMethodInfo(Queryable.Select, source, selector),
new Expression[] { source.Expression, Expression.Quote(selector) }
));
}
When the queryable's Provider has to generate actual data from the IQueryable<>, it analyzes the expression tree and tries to figure out how to interpret those method calls. Entity Framework has built-in knowledge of many LINQ-related functions like .Where() and .Select(), so it knows how to translate those method calls into SQL. However, it doesn't know what to do for methods that you write.
So why does this work?
var data = context.Programs.ForUser(userId);
The answer is that your ForUser method is not implemented like the Select method above: you are not adding an expression to the queryable to represent calling ForUser. Instead, you are returning the result of a .Where() call. From the IQueryable<>'s perspective, it's as if Where() was called directly, and the call to ForUser() never happened.
You can prove this by capturing the Expression property on the IQueryable<>:
Console.WriteLine(data.Expression.ToString());
... which will produce something like this:
Programs.Where(u => (u.UserId == value(Helpers<>c__DisplayClass1_0).userId))
There's no call to ForUser() anywhere in that expression.
On the other hand, if you include the ForUser() call inside of an expression tree like this:
var data = context.Programs.Select(d => d.Protocols.ForUser(id));
... then the .ForUser() method never actually gets invoked, so it never returns an IQueryable<> that knows the .Where() method got called. Instead, the expression tree for the queryable shows .ForUser() getting invoked. Outputting its expression tree would look something like this:
Programs.Select(d => d.Protocols.ForUser(value(Repository<>c__DisplayClass1_0).userId))
Entity Framework has no idea what ForUser() is supposed to do. As far as it's concerned, you could have written ForUser() to do something that's impossible to do in SQL. So it tells you that's not a supported method.
As I mentioned in my comment above, I can't tell why the EF Engine is working the way it is. Therefore, I've tried to find a way to re-write the query so I'll be able to make use of my extension methods.
The tables are:
Program -> 1..m -> ProgramProtocol -> m..1 -> Protocol
ProgramProtocol is just a join table and is not mapped in the model by Entity Framework.
The idea is simple: select "from left", select "from right" and then join the resulted sets for proper filtering:
var data = context.Programs.ForUser(userId)
.SelectMany(pm => pm.Protocols,
(pm, pt) => new {pm.ProgramId, pm.ProgramName, pm.ClientId, pt.ProtocolId})
.Join(context.Protocols.ForUser(userId), pm => pm.ProtocolId,
pt => pt.ProtocolId, (pm, pt) => pm)
.GroupBy(pm => new {pm.ProgramId, pm.ProgramName, pm.ClientId})
.Select(d => new MyDataDto
{
ProgramName = d.Key.ProgramName,
ProgramId = d.Key.ProgramId,
ClientId = d.Key.ClientId,
Protocols = d.Count()
})
.ToList();
I have got the following expression that works with mockup data - hereby not using Entity Framework:
public static Expression<Func<Resource, bool>> FilterResourcesByUserCriteria(IEnumerable<FilterValue> filterValuesForUser)
{
Expression<Func<Resource, bool>> filter = (resource) =>
// Get filter values for the current resource in the loop
resource.ResourceFilterValues
// Group filter values for user
.GroupBy(filterValue => filterValue.FilterValue.FilterGroup.Id)
// Each group must fulfill the following logic
.All(filterGroup =>
// For each filter group, only select the user values from the same group
filterValuesForUser
.Where(filterValueForUser => filterValueForUser.FilterGroup.Id == filterGroup.Key)
.Select(filterValueForUser => filterValueForUser.FilterValue1)
// Each group must at least one value in the sublist of filter values of the current user
.Any(filterValueForUser => filterGroup
.Select(resourceFilterValue => resourceFilterValue.FilterValue.FilterValue1)
.Any(x => x == filterValueForUser))
);
}
However, I get this famous exception when I try to insert this expression in the where clause of my repository method (using Entity Framework):
Unable to create a constant value of type. Only primitive types or enumeration types are supported in this context.
I suspect this has something to do with a parameter called filterValuesForUser, which is a collection of a complex (i.e. custom) type.
Is this behavior even possible in Entity Framework where I do a subquery that is not directly related to Entity Framework? What I want to achieve here is to query on a subset of a custom list for each group in the query.
Any solutions for this or other workarounds? I'd like to minimize the amount of database calls, preferrably limit it to just one.
The exact query you are asking for is impossible with LinqToEF (due to limitation of SQL). But fear not. It is possible to salvage your problem with a slight tweaking.
public static Expression<Func<Resource, bool>> FilterResourcesByUserCriteria(FilterValue filterValueForUser)
{
//I assume you can write this part yourself.
}
public IQueryable<Resource> GetResources()
{
IQueryable<Resource> resources = _context.Resources;
IEnumerable<FilterValue> filterValuesForUser = GetFilterValues();
IEnumerable<IQueryable<Resource>> queries = from filter in filterValuesForUser
let filterExp = FilterResourcesByUserCriteria(filter)
select resources.Where(filterExp);
return Enumerable.Aggregate(queries, (l, r) => Queryable.Concat(l, r));
}
Types and Extension methods expanded for clarity.
In addition to Aron's answer, I used the PredicateBuilder utility in the LinqKit assembly to generate 1 expression rather than multiple and separate expresssions. This also avoids doing multiple database calls.
Here is how you can achieve this (pseudo-code):
public IQueryable<Resource> GetResources()
{
MyContext ctx = new MyContext ();
IEnumerable<Expression<Func<Resource, bool>>> queries =
filterValuesForUser.GroupBy(x => x.FilterGroup)
.Select(filter => SecurityFilters.FilterResourcesByUserCriteriaEF(filter.Select(y => y.FilterValue1)))
.Select(filterExpression => { return filterExpression; });
Expression<Func<Resource, bool>> query = PredicateBuilder.True<Resource>();
foreach (Expression<Func<Resource, bool>> filter in queries)
{
query = query.And(filter);
}
return ctx.Resources.AsExpandable().Where(query);
}
public static Expression<Func<Resource, bool>> FilterResourcesByUserCriteriaEF(IEnumerable<string> filterValuesForUser)
{
// From the resource's filter values, check if there are any present in the user's filter values
return (x) => x.ResourceFilterValues.Any(y => filterValuesForUser.Contains(y.FilterValue.FilterValue1));
}
I'm still having issues with getting this working in my repository but that has something do with something blocking AsExpandable() from working properly.
I need to do some filtering on an ObjectSet to obtain the entities I need by doing this :
query = this.ObjectSet.Where(x => x.TypeId == 3); // this is just an example;
Later in the code (and before launching the deferred execution) I filter the query again like this :
query = query.Where(<another lambda here ...>);
That works quite well so far.
Here is my problem :
The entities contains a DateFrom property and a DateTo property, which are both DataTime types. They represent a period of time.
I need to filter the entities to get only those that are part of a collection of periods of time. The periods in the collection are not necessarily contiguous, so, the logic to retreive the entities looks like that :
entities.Where(x => x.DateFrom >= Period1.DateFrom and x.DateTo <= Period1.DateTo)
||
entities.Where(x => x.DateFrom >= Period2.DateFrom and x.DateTo <= Period2.DateTo)
||
... and on and on for all the periods in the collection.
I have tried doing that :
foreach (var ratePeriod in ratePeriods)
{
var period = ratePeriod;
query = query.Where(de =>
de.Date >= period.DateFrom && de.Date <= period.DateTo);
}
But once I launch the deferred execution, it translates this into SQL just like I want it (one filter for each of the periods of time for as many periods there is in the collection), BUT, it translates to AND comparisons instead of OR comparisons, which returns no entities at all, since an entity cannot be part of more than one period of time, obviously.
I need to build some sort of dynamic linq here to aggregate the period filters.
Update
Based on hatten's answer, I've added the following member :
private Expression<Func<T, bool>> CombineWithOr<T>(Expression<Func<T, bool>> firstExpression, Expression<Func<T, bool>> secondExpression)
{
// Create a parameter to use for both of the expression bodies.
var parameter = Expression.Parameter(typeof(T), "x");
// Invoke each expression with the new parameter, and combine the expression bodies with OR.
var resultBody = Expression.Or(Expression.Invoke(firstExpression, parameter), Expression.Invoke(secondExpression, parameter));
// Combine the parameter with the resulting expression body to create a new lambda expression.
return Expression.Lambda<Func<T, bool>>(resultBody, parameter);
}
Declared a new CombineWithOr Expression :
Expression<Func<DocumentEntry, bool>> resultExpression = n => false;
And used it in my period collection iteration like this :
foreach (var ratePeriod in ratePeriods)
{
var period = ratePeriod;
Expression<Func<DocumentEntry, bool>> expression = de => de.Date >= period.DateFrom && de.Date <= period.DateTo;
resultExpression = this.CombineWithOr(resultExpression, expression);
}
var documentEntries = query.Where(resultExpression.Compile()).ToList();
I looked at the resulting SQL and it's like the Expression has no effect at all. The resulting SQL returns the previously programmed filters but not the combined filters. Why ?
Update 2
I wanted to give feO2x's suggestion a try, so I have rewritten my filter query like this :
query = query.AsEnumerable()
.Where(de => ratePeriods
.Any(rp => rp.DateFrom <= de.Date && rp.DateTo >= de.Date))
As you can see, I added AsEnumerable() but the compiler gave me an error that it cannot convert the IEnumerable back to IQueryable, so I have added ToQueryable() at the end of my query :
query = query.AsEnumerable()
.Where(de => ratePeriods
.Any(rp => rp.DateFrom <= de.Date && rp.DateTo >= de.Date))
.ToQueryable();
Everything works fine. I can compile the code and launch this query. However, it doesn't fit my needs.
While profiling the resulting SQL, I can see that the filtering is not part of the SQL query because it filters the dates in-memory during the process. I guess that you already know about that and that is what you intended to suggest.
Your suggestion works, BUT, since it fetches all the entities from the database (and there are thousands and thousands of them) before filtering them in-memory, it's really slow to get back that huge amount from the database.
What I really want is to send the period filtering as part of the resulting SQL query, so it won't return a huge amount of entities before finishing up with the filtering process.
Despite the good suggestions, I had to go with the LinqKit one. One of the reasons is that I will have to repeat the same kind of predicate aggregation in many other places in the code. Using LinqKit is the easiest one, not to mention I can get it done by writing only a few lines of code.
Here is how I solved my problem using LinqKit :
var predicate = PredicateBuilder.False<Document>();
foreach (var submittedPeriod in submittedPeriods)
{
var period = period;
predicate = predicate.Or(d =>
d.Date >= period.DateFrom && d.Date <= period.DateTo);
}
And I launch deferred execution (note that I call AsExpandable() just before) :
var documents = this.ObjectSet.AsExpandable().Where(predicate).ToList();
I looked at the resulting SQL and it does a good job at translating my predicates into SQL.
You can use a method like the following:
Expression<Func<T, bool>> CombineWithOr<T>(Expression<Func<T, bool>> firstExpression, Expression<Func<T, bool>> secondExpression)
{
// Create a parameter to use for both of the expression bodies.
var parameter = Expression.Parameter(typeof(T), "x");
// Invoke each expression with the new parameter, and combine the expression bodies with OR.
var resultBody = Expression.Or(Expression.Invoke(firstExpression, parameter), Expression.Invoke(secondExpression, parameter));
// Combine the parameter with the resulting expression body to create a new lambda expression.
return Expression.Lambda<Func<T, bool>>(resultBody, parameter);
}
And then:
Expression<Func<T, bool>> resultExpression = n => false; // Always false, so that it won't affect the OR.
foreach (var ratePeriod in ratePeriods)
{
var period = ratePeriod;
Expression<Func<T, bool>> expression = (de => de.Date >= period.DateFrom && de.Date <= period.DateTo);
resultExpression = CombineWithOr(resultExpression, expression);
}
// Don't forget to compile the expression in the end.
query = query.Where(resultExpression.Compile());
For more information, you may want to check out the following:
Combining two expressions (Expression<Func<T, bool>>)
http://www.albahari.com/nutshell/predicatebuilder.aspx
Edit: The line Expression<Func<DocumentEntry, bool>> resultExpression = n => false; is just a placeholder. CombineWithOr method needs two methods to combine, if you write Expression<Func<DocumentEntry, bool>> resultExpression;', you can't use it in the call toCombineWithOrfor the first time in yourforeach` loop. It's just like the following code:
int resultOfMultiplications = 1;
for (int i = 0; i < 10; i++)
resultOfMultiplications = resultOfMultiplications * i;
If there's nothing in resultOfMultiplications to begin with, you cannot use it in your loop.
As to why the lambda is n => false. Because it doesn't have any effect in an OR statement. For example, false OR someExpression OR someExpression is equal to someExpression OR someExpression. That false doesn't have any effect.
How about this code:
var targets = query.Where(de =>
ratePeriods.Any(period =>
de.Date >= period.DateFrom && de.Date <= period.DateTo));
I use the LINQ Any operator to determine if there is any rate period that conforms to de.Date. Although I'm not quite sure how this is translated into efficient SQL statements by entity. If you could post the resulting SQL, that would be quite interesting for me.
Hope this helps.
UPDATE after hattenn's answer:
I don't think that hattenn's solution would work, because Entity Framework uses LINQ expressions to produce the SQL or DML that is executed against the database. Therefore, Entity Framework relies on the IQueryable<T> interface rather than IEnumerable<T>. Now the default LINQ operators (like Where, Any, OrderBy, FirstOrDefault and so on) are implemented on both interfaces, thus the difference is sometimes hard to see. The main difference of these interfaces is that in case of the IEnumerable<T> extension methods, the returned enumerables are continuously updated without side effects, while in the case of IQueryable<T> the actual expression is recomposed, which is not free of side effects (i.e. you are altering the expression tree that is finally used to create the SQL query).
Now Entity Framework supports the ca. 50 standard query operators of LINQ, but if you write your own methods that manipulate an IQueryable<T> (like hatenn's method), this would result in an expression tree that Entity Framework might not be able to parse because it simply doesn't know the new extension method. This might be the cause why you cannot see the combined filters after you composed them (although I would expect an exception).
When does the solution with the Any operator work:
In the comments, you told that you encountered a System.NotSupportedException: Unable to create a constant value of type 'RatePeriod'. Only primitive types or enumeration types are supported in this context. This is the case when the RatePeriod objects are in-memory objects and not tracked by the Entity Framework ObjectContext or DbContext. I made a small test solution that can be downloaded from here: https://dl.dropboxusercontent.com/u/14810011/LinqToEntitiesOrOperator.zip
I used Visual Studio 2012 with LocalDB and Entity Framework 5. To see the results, open the class LinqToEntitiesOrOperatorTest, then open Test Explorer, build the solution and run all tests. You will recognize that ComplexOrOperatorTestWithInMemoryObjects will fail, all the others should pass.
The context I used looks like this:
public class DatabaseContext : DbContext
{
public DbSet<Post> Posts { get; set; }
public DbSet<RatePeriod> RatePeriods { get; set; }
}
public class Post
{
public int ID { get; set; }
public DateTime PostDate { get; set; }
}
public class RatePeriod
{
public int ID { get; set; }
public DateTime From { get; set; }
public DateTime To { get; set; }
}
Well, it is as simple as it gets :-). In the test project, there are two important unit test methods:
[TestMethod]
public void ComplexOrOperatorDBTest()
{
var allAffectedPosts =
DatabaseContext.Posts.Where(
post =>
DatabaseContext.RatePeriods.Any(period => period.From < post.PostDate && period.To > post.PostDate));
Assert.AreEqual(3, allAffectedPosts.Count());
}
[TestMethod]
public void ComplexOrOperatorTestWithInMemoryObjects()
{
var inMemoryRatePeriods = new List<RatePeriod>
{
new RatePeriod {ID = 1000, From = new DateTime(2002, 01, 01), To = new DateTime(2006, 01, 01)},
new RatePeriod {ID = 1001, From = new DateTime(1963, 01, 01), To = new DateTime(1967, 01, 01)}
};
var allAffectedPosts =
DatabaseContext.Posts.Where(
post => inMemoryRatePeriods.Any(period => period.From < post.PostDate && period.To > post.PostDate));
Assert.AreEqual(3, allAffectedPosts.Count());
}
Notice that the first method passes while the second one fails with the exception mentioned above, although both methods do exactly the same thing, except that in the second case I created rate period objects in memory the DatabaseContext does not know about.
What can you do to solve this problem?
Do your RatePeriod objects reside in the same ObjectContext or DbContext, respectively? Then use them right from it like I did in the first unit test mentioned above.
If not, can you load all your posts at once or would this result in an OutOfMemoryException? If not, you could use the following code. Notice the AsEnumerable() call that results in the Where operator being used against the IEnumerable<T> interface instead of IQueryable<T>. Effectively, this results in all posts being loaded into memory and then filtered:
[TestMethod]
public void CorrectComplexOrOperatorTestWithInMemoryObjects()
{
var inMemoryRatePeriods = new List<RatePeriod>
{
new RatePeriod {ID = 1000, From = new DateTime(2002, 01, 01), To = new DateTime(2006, 01, 01)},
new RatePeriod {ID = 1001, From = new DateTime(1963, 01, 01), To = new DateTime(1967, 01, 01)}
};
var allAffectedPosts =
DatabaseContext.Posts.AsEnumerable()
.Where(
post =>
inMemoryRatePeriods.Any(
period => period.From < post.PostDate && period.To > post.PostDate));
Assert.AreEqual(3, allAffectedPosts.Count());
}
If the second solution is not possible, then I would recommend to write a TSQL stored procedure where you pass in your rate periods and that forms the correct SQL statement. This solution is also the most performant one.
Anyways, I think dynamic LINQ query creation was not as simple as I thought. Try using Entity SQL, similar to the way below:
var filters = new List<string>();
foreach (var ratePeriod in ratePeriods)
{
filters.Add(string.Format("(it.Date >= {0} AND it.Date <= {1})", ratePeriod.DateFrom, ratePeriod.DateTo));
}
var filter = string.Join(" OR ", filters);
var result = query.Where(filter);
This may not be exactly correct (I haven't tried it), but it should be something similar to this.
I have the following code:
public OTestTable GetTestCode(Func<TestTable, bool> whereClause)
{
return CoreContext.TestTables.Where(whereClause).Select(TestTableMap.DataToObject).FirstOrDefault();
}
CoreContext is my data context (which is initialized in a base class)
My TestTableMap is as follows:
public class TestTableMap
{
public static readonly Func<TestTable, OTestTable> DataToObject = mapper =>
new OTestTable
{
Code = mapper.mycode
};
}
Then in my business method i have the following:
public OTestTable GetTestCode(string code)
{
return QueryEngine.GetTestCode(id => id.mycode == code);
}
From my main program, i am calling GetTestCode with a string value.
When I watch SQL profiler, I get the following:
SELECT [t0].[mycode]
FROM [dbo].[TestTable] AS [t0]
It does not have the where clause appended to the SQL query. If i add the where clause to the LINQ as var query = from c in DataContext.TestTable where c.mycode == '' select c;
It will add the where clause.
However, when I run my code, it will return the correct record, but it seems like I am pulling back all records from the database and filtering in my code (which should not happen).
Any thoughts with what I am doing wrong?
Thanks
In order to construct SQL statements, LINQ to SQL requires an expression tree. Func<TestTable, bool> does not represent an expression tree, it is a "black box" function pointer. LINQ cannot do anything intelligent with this apart from blindly execute it on an in-memory collection.
You need to do this instead:
public OTestTable GetTestCode(Expression<Func<TestTable, bool>> whereClause) {
return CoreContext.TestTables.Where(whereClause).Select(TestTableMap.DataToObject).FirstOrDefault();
}
This code compiles using the Queryable.Where extension method, which does accept an expression tree, rather than the Enumerable.Where extension method, which only accepts a raw delegate.
Try creating your where clause as:
Expression<Func<T, bool>> whereClause
Where the T parameter is your source type Table<T> source
Also see the PredicateBuilder here: http://www.albahari.com/nutshell/predicatebuilder.aspx
It provides you convenient extension methods to predicate IQueryable<T>. like this:
var predicate = PredicateBuilder.True<Family>();
predicate = predicate.And(o => o.Birthday < new DateTime(1980, 1, 1));
.Or(o => o.Name.Contains("ke"));
var result = Source.Where(predicate).ToList();