Develop Reference

Create generic .Any extention method for IQueryable [duplicate]

I'm building a SQL "WHERE" clause dynamically using the System.Linq.Expressions.Expression class. It works well for simple clauses, e.g. to add "PhaseCode = X" clause, I do the following:
var equalTarget = Expression.Constant(phaseCode, typeof(int?));
var phaseEquals = Expression.Equal(Expression.PropertyOrField(projParam, "PhaseCode"), equalTarget);
However, now I'm trying to build an expression that will return the record if a project has been assigned to a particular group. Project and Group has many-to-many relationship.
Without the expression trees, I would do it as follows:
db.Projects.Where(p => .... && p.GroupsAssigned.Any(g => g.ID == groupId))
However, I can't seem to find a way to express that with the Expression class.
There are actually two things I can't figure out:
How to traverse the relationships between tables
How to do x.Any()
Any help is greatly appreciated.

Calling an extension method, like Enumerable.Any or Queryable.Any, is simply a static method call on the sequence and the lambda expression you created for the WHERE clause. You can use Expression.Call to do this:
// for Enumerable.Any<T>(IEnumerable<T>,Predicate<T>)
var overload = typeof(Enumerable).GetMethods("Any")
.Single(mi => mi.GetParameters().Count() == 2);
var call = Expression.Call(
overload,
Expression.PropertyOrField(projParam, "GroupsAssigned"),
anyLambda);
For Queryable.Any<T>, you'll need to roll this up into a method:
static Expression BuildAny<TSource>(Expression<Func<TSource, bool>> predicate)
{
var overload = typeof(Queryable).GetMethods("Any")
.Single(mi => mi.GetParameters().Count() == 2);
var call = Expression.Call(
overload,
Expression.PropertyOrField(projParam, "GroupsAssigned"),
predicate);
return call;
}
Although this seems odd that you're unable to do this through a normal query.

EF + LINQ: How do I define and utilize a variable (maybe a Func variable) to define part of a Select statement?

Before try to use a variable (this has sql server do the calculation):
IQueryable<MyEntity> query = _dbSet; // IQueryable<TEntity>
var results = query.Select(m => new MyViewModel
{
MyCalculation = m.Column1 * m.Column2
}).ToList();
What I want to do (dynamically create part of my select statement from a Func variable or some other kind of variable to allow this):
IQueryable<MyEntity> query = _dbSet; // IQueryable<TEntity>
Func<MyEntity, decimal> funcVariableAttempt = m => m.Column1 * m.Column2;
var results = query.Select(m => new MyViewModel
{
MyCalculation = funcVariableAttempt.Invoke(m) // My foolish attempt does not work.
}).ToList();
The error I get when I try what I want (aka my foolish attempt):
LINQ to Entities does not recognize the method 'System.Decimal Invoke(MyProject.Repository.Models.MyEntity)' method, and this method cannot be translated into a store expression.
How do I define and utilize a variable (maybe a Func variable) to define part of a Select statement?

It's a completely valid question that I stumbeled across earlier and found a solution that works well for me.
The problem with your Func<MyEntity, decimal> is that it is a delegate and that the O/R mapper has to have access to the internal expression (the multiplication of two properties in your case). But this information is compiled into the delegate and hidden forever.
If you however start off with a Expression<Func<MyEntity, decimal>> customCalculation, things look more promising as you have the internal logic as an expression tree.
Problem with this is: you cannot invoke an expression. customCalculation(m) doesn't compile.
The compiler would let you write
m => new MyViewModel { MyCalculation = customCalculation.Compile()(m) }
, but this wouldn't be understood by most O/R mappers.
But you see that it at least somehow contains the customCalculation lambda expression and also how it relates to its surrounding expressions.
Getting from here to the expression tree as in your original working version involves some expression manipulation:
We have to replace customCalculation.Compile()(m) with the body of the lambda that is to be Compile()d, but with the lambda's parameter(s) replaced with the respective expression(s) of the delegate invocation. So if customCalculation were x => x.Column1 * x.Column2, customCalculation.Compile()(m) would have to be replaced with m.Column1 * m.Column2
Doing so is not trivial, since the lambda itself has to be dug out of a field inside an instance of a compiler generated closure class.
I've posted my implementation of this expression manipulator in another similar question . Hope that helps.
With that, you should be able to:
var customCalculation = (Expression<Func<MyEntity, decimal>>)(x => x.Column1 * x.Column2);
var selector = Express.Prepare((Expression<Func<MyEntity, MyViewModel>>)(m => new MyViewModel { MyCalculation = customCalculation.Compile()(m) }));
var result = query.Select(selector).ToList();

As you already know, your funcVariableAttempt makes no sense to your database, so you have to call your method in the linq-to-object context. i.e. for instance first fetch the data as an Enumerable, then call your method:
var results = query.Select(m => new {
Column1= col1,
Column2= col2
}).AsEnumerable()
.Select(m => new MyViewModel
{
MyCalculation = Foo(m.Column1, m.Column2)
});
*Note: Code is not tested.

You should call ToList() first and perform the Select() on the result in memory.
var results = query.ToList()
.Select(m => new MyViewModel {
MyCalculation = Foo(m.Column1, m.Column2)
});
You're trying to perform the Select as part of the query. You should just use a regular function for the mapping calculation. Lambda functions are useful with LINQ but in this case they're not needed.

How to reuse where clauses in Linq To Sql queries

I have users searching records of type Record. They type a search term in a textbox and then I search records by matching several fields with the search term.
My query looks like:
var results = from record in DataContext.Records
where
record.Field1.ToLower().Contains(term) ||
record.Field2.ToLower().Contains(term) ||
record.Field3.ToLower().Contains(term)
select record;
I have a number of queries that all use the same filter and thus I would like to extract the filtering so it can be reused. Something like:
var filter = new Func<Record, string, bool>(
(record, term) =>
record.Field1.ToLower().Contains(term) ||
record.Field2.ToLower().Contains(term) ||
record.Field3.ToLower().Contains(term)
);
var results = from record in DataContext.Records
where filter(record, term)
select record;
However, it does not work because:
Method 'System.Object DynamicInvoke(System.Object[])' has no supported translation to SQL.
How can I reuse my where condition across queries?

You need to build an expression instead of a function:
Expression<Func<Record, bool>> filter =
record => record.Field1.ToLower().Contains(term); // rest omitted
The lambda expression remains the same, but you need to return it into a variable of type Expression<Func<Record, bool>> -- that will make the C# compiler compile it as an expression instead of a delegate, allowing it to be passed to LINQ to SQL.
However, you won't be able to use an expression variable with a C#-syntax where clause: you'll need to use the Where extension method:
var results = DataContext.Records.Where(filter);
Edited to add: If you want to be able to create filters on different terms, you just need a method to produce an expression from a term:
private static Expression<Func<Record, bool>> Filter(string term)
{
return r => r.Field1.ToLower().Contains(term);
}
var results = DataContext.Records.Where(Filter(term));
If you prefer to keep filter as a lambda as you have at the moment, you can do so, but the generics get a bit nested:
Func<string, Expression<Func<Record, bool>>> filter =
term => (r => r.Field1.ToLower().Contains(term));
var results = DataContext.Records.Where(filter(term));
Regardless, the important thing is that what goes in the Where clause must be an Expression<Func<Record, bool>> -- but as shown above you can make the expression depend on term by building a suitable expression on the fly. Which is exactly what LINQ to SQL would be doing if you spelled out the filter longhand in the Where clause.

Use a CompiledQuery!
var filter = CompiledQuery.Compile(
(DatabaseDataContext dc, Record record, string term) =>
record.Field1.ToLower().Contains(term) ||
record.Field2.ToLower().Contains(term) ||
record.Field3.ToLower().Contains(term)
);
var results = from record in DataContext.Records
where filter(DataContext, record, term)
select record;
For more information, see How to: Store and Reuse Queries.

In addition to the Expression<Func<Record, bool>> issue that others have pointed out, I suggest looking into PredicateBuilder. It's very good for dynamically combining lambda expressions.

I think you need to make it an Expression<Func<Record, bool>>. Otherwise it's trying to translate the actual C# method call to SQL rather than the description of it. This is not a guarantee that this version will work; I'm not sure which string functions are translatable to SQL.

How to Expression.Invoke an arbitrary LINQ to SQL Query

Say I take an arbitrary LINQ to SQL query's Expression, is it possible to invoke it somehow?
MyContext ctx1 = new MyContext("...");
var q = from t in ctx1.table1 where t.id = 1 select t;
Expression qe = q.Expression;
var res = Expression.Invoke(qe);
This throws:
ArgumentException "Expression of type System.Linq.IQueryable`1[...]' cannot be invoked".
My ultimate goal is to evaluate the same query on several different data contexts.

Queries are not Expressions. A Query has an ExpressionTree.
Queries are not Methods to be invoked.
Queries may be Enumerated, yielding their results. This code will Enumerate any IQueryable:
List<object> result = query.Cast<object>().ToList();
My ultimate goal is to evaluate the
same query on several different data
contexts.
Then you should write your queries as query generators, that accept DataContext as a parameter.
Func<MyDataContext, IQueryable<Customer>> queryGen =
(dc) => dc.Customers.Where(c => c.Name == "Bob");
//now we can get some queries
IQueryable<Customer> query1 = queryGen(new MyDataContext());
IQueryable<Customer> query2 = queryGen(new MyDataContext());

If you goal is to run an Expression across different contexts, why not create just the expression like so:-
Expression<Func<MyClass, bool>> myExpression = x => x.id == 1;
Then you can do whatever you like with it, including using it in .Where() clauses.

LINQ to SQL expressions are parsed by the LINQ to SQL Provider and converted into T-SQL. My guess is that exception is being raised explicitly - that Microsoft did not intend for those expressions to be Invoked directly (you could confirm this using .NET Reflector.)

LINQ, Large filter on a query

I am building a section of an application that revolves around pulling information about transactions out of the database. Due to the nature of the data, there are many columns in the table that I want to filter on. I have a filter selection box with 15 fields that I want to be able to build up a where clause for the LINQ statement. The interesting part comes when I want certain fields to be null. For example I want to be able to filter on any or all of:
Transaction Type
Response Code
Transaction Amount
Many more
I can build up a predicate that looks like
Func<Transaction, bool> pred = t => t.ResponseCode == ResponseCode && t.TransactionType == TransactionType && t.TransactionAmount > 100.00;
But in order to be able to choose which fields to include in the predicate I am concatenating the predicates together:
Func<Transaction, bool> pred = t => true;
if(ResponseCode != null)
pred.AndAlso(t => t.ResponseCode == ResponseCode);
// Rinse and repeat
And then passing that predicate to the where clause of the LINQ statement.
This works exactly the way I want it, but is rather complicated. Are there any other ways of doing this?
UPDATE:
Thanks Justice for the comments. I'm not using LINQ to SQL, I'm using LINQ on a collection of objects from a repository. How would you programatically build an Expression filter?

In dynamic SQL... Since you only have one WHERE clause - you must concatenate predicates with AND.
In linq query construction... you get as many WHERE clauses as you want. Linq will AND them together for you when it translates the query.
Example:
IQueryable<Transaction> query = db.Transactions;
if (filterByTransactionType)
{
query = query.Where(t => t.TransactionType == theTransactionType);
}
if (filterByResponseCode)
{
query = query.Where(t => t.ResponseCode == theResponseCode);
}
if (filterByAmount)
{
query = query.Where(t => t.TransactionAmount > theAmount);
}
Another Example:
List<Expression<Func<Transaction, bool>>> filters = GetFilterExpressions();
IQueryable<Transaction> query = db.Transactions;
filters.ForEach(f => query = query.Where(f));

First, you would need to use Expression<Func<Transaction, bool>> for LINQ-to-SQL (that's what you're trying to use, and it's not the same thing as LINQ).
Second, you can programmatically build up an Expression<Func<Transaction, bool>> using the System.Linq.Expression namespace.
You will not be able to use LINQ per se to query the database using programmatically built-up expressions. Instead of using the query operators, you will need to use the query extension methods: for example, instead of from p in db.People where p.Age > 50 select p.Name you will need to use db.People.Where(p => p.Age > 50). You can use this style to add filters: db.People.Where(myFilter), where myFilter = new Expression<Func<Person, bool>>(p => p.Age > 50). In your case, myFilter would be your programmatically built-up filter, not one created using lambda-expression syntax.