Develop Reference

How to make LINQ-to-Objects handle projections?

I have implemented a basic (naive?) LINQ provider that works ok for my purposes, but there's a number of quirks I'd like to address, but I'm not sure how. For example:
// performing projection with Linq-to-Objects, since Linq-to-Sage won't handle this:
var vendorCodes = context.Vendors.ToList().Select(e => e.Key);
My IQueryProvider implementation had a CreateQuery<TResult> implementation looking like this:
public IQueryable<TResult> CreateQuery<TResult>(Expression expression)
{
return (IQueryable<TResult>)Activator
.CreateInstance(typeof(ViewSet<>)
.MakeGenericType(elementType), _view, this, expression, _context);
}
Obviously this chokes when the Expression is a MethodCallExpression and TResult is a string, so I figured I'd execute the darn thing:
public IQueryable<TResult> CreateQuery<TResult>(Expression expression)
{
var elementType = TypeSystem.GetElementType(expression.Type);
if (elementType == typeof(EntityBase))
{
Debug.Assert(elementType == typeof(TResult));
return (IQueryable<TResult>)Activator.CreateInstance(typeof(ViewSet<>).MakeGenericType(elementType), _view, this, expression, _context);
}
var methodCallExpression = expression as MethodCallExpression;
if(methodCallExpression != null && methodCallExpression.Method.Name == "Select")
{
return (IQueryable<TResult>)Execute(methodCallExpression);
}
throw new NotSupportedException(string.Format("Expression '{0}' is not supported by this provider.", expression));
}
So when I run var vendorCodes = context.Vendors.Select(e => e.Key); I end up in my private static object Execute<T>(Expression,ViewSet<T>) overload, which switches on the innermost filter expression's method name and makes the actual calls in the underlying API.
Now, in this case I'm passing the Select method call expression, so the filter expression is null and my switch block gets skipped - which is fine - where I'm stuck at is here:
var method = expression as MethodCallExpression;
if (method != null && method.Method.Name == "Select")
{
// handle projections
var returnType = method.Type.GenericTypeArguments[0];
var expType = typeof (Func<,>).MakeGenericType(typeof (T), returnType);
var body = method.Arguments[1] as Expression<Func<T,object>>;
if (body != null)
{
// body is null here because it should be as Expression<Func<T,expType>>
var compiled = body.Compile();
return viewSet.Select(string.Empty).AsEnumerable().Select(compiled);
}
}
What do I need to do to my MethodCallExpression in order to be able to pass it to LINQ-to-Objects' Select method? Am I even approaching this correctly?

(credits to Sergey Litvinov)
Here's the code that worked:
var method = expression as MethodCallExpression;
if (method != null && method.Method.Name == "Select")
{
// handle projections
var lambda = ((UnaryExpression)method.Arguments[1]).Operand as LambdaExpression;
if (lambda != null)
{
var returnType = lambda.ReturnType;
var selectMethod = typeof(Queryable).GetMethods().First(m => m.Name == "Select");
var typedGeneric = selectMethod.MakeGenericMethod(typeof(T), returnType);
var result = typedGeneric.Invoke(null, new object[] { viewSet.ToList().AsQueryable(), lambda }) as IEnumerable;
return result;
}
}
Now this:
var vendorCodes = context.Vendors.ToList().Select(e => e.Key);
Can look like this:
var vendorCodes = context.Vendors.Select(e => e.Key);
And you could even do this:
var vendors = context.Vendors.Select(e => new { e.Key, e.Name });
The key was to fetch the Select method straight from the Queryable type, make it a generic method using the lambda's returnType, and then invoke it off viewSet.ToList().AsQueryable().

Finding reference to DbFunction within expression tree and replacing with a different function

I would like to have some somewhat complex logic kept in a single lambda expression, which can be compiled and therefore used in Linq-To-Objects, or used as an expression to run against a database in Linq-To-Entities.
it involves date calculations, and I have hitherto been using something like (hugely simplified)
public static Expression<Func<IParticipant, DataRequiredOption>> GetDataRequiredExpression()
{
DateTime twentyEightPrior = DateTime.Now.AddDays(-28);
return p=> (p.DateTimeBirth > twentyEightPrior)
?DataRequiredOption.Lots
:DataRequiredOption.NotMuchYet
}
And then having a method on a class
public DataRequiredOption RecalculateDataRequired()
{
return GetDataRequiredExpression().Compile()(this);
}
There is some overhead in compiling the expression tree. Of course I cannot simply use
public static Expression<Func<IParticipant, DataRequiredOption>> GetDataRequiredExpression(DateTime? dt28Prior=null)
{
return p=> DbFunctions.DiffDays(p.DateTimeBirth, DateTime.Now) > 28
?DataRequiredOption.Lots
:DataRequiredOption.NotMuchYet
}
Because this will only run at the database (it will throw an error at execution of the Compile() method).
I am not very familiar with modifying expressions (or the ExpressionVisitor class). Is it possible, and if so how would I find the DbFunctions.DiffDays function within the expression tree and replace it with a different delegate? Thanks for your expertise.
Edit
A brilliant response from svick was used - a slight modification beecause difdays and date subtraction have their arguments switched to produce a positive number in both cases:
static ParticipantBaseModel()
{
DataRequiredExpression = p =>
((p.OutcomeAt28Days >= OutcomeAt28DaysOption.DischargedBefore28Days && !p.DischargeDateTime.HasValue)
|| (DeathOrLastContactRequiredIf.Contains(p.OutcomeAt28Days) && (p.DeathOrLastContactDateTime == null || (KnownDeadOutcomes.Contains(p.OutcomeAt28Days) && p.CauseOfDeath == CauseOfDeathOption.Missing))))
? DataRequiredOption.DetailsMissing
: (p.TrialArm != RandomisationArm.Control && !p.VaccinesAdministered.Any(v => DataContextInitialiser.BcgVaccineIds.Contains(v.VaccineId)))
? DataRequiredOption.BcgDataRequired
: (p.OutcomeAt28Days == OutcomeAt28DaysOption.Missing)
? DbFunctions.DiffDays(p.DateTimeBirth, DateTime.Now) < 28
? DataRequiredOption.AwaitingOutcomeOr28
: DataRequiredOption.OutcomeRequired
: DataRequiredOption.Complete;
var visitor = new ReplaceMethodCallVisitor(
typeof(DbFunctions).GetMethod("DiffDays", BindingFlags.Static | BindingFlags.Public, null, new Type[]{ typeof(DateTime?), typeof(DateTime?)},null),
args =>
Expression.Property(Expression.Subtract(args[1], args[0]), "Days"));
DataRequiredFunc = ((Expression<Func<IParticipant, DataRequiredOption>>)visitor.Visit(DataRequiredExpression)).Compile();
}

Replacing a call to a static method to something else using ExpressionVisitor is relatively simple: override VisitMethodCall(), in it check if it's the method that you're looking for and if it, replace it:
class ReplaceMethodCallVisitor : ExpressionVisitor
{
readonly MethodInfo methodToReplace;
readonly Func<IReadOnlyList<Expression>, Expression> replacementFunction;
public ReplaceMethodCallVisitor(
MethodInfo methodToReplace,
Func<IReadOnlyList<Expression>, Expression> replacementFunction)
{
this.methodToReplace = methodToReplace;
this.replacementFunction = replacementFunction;
}
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (node.Method == methodToReplace)
return replacementFunction(node.Arguments);
return base.VisitMethodCall(node);
}
}
The problem is that this won't work well for you, because DbFunctions.DiffDays() works with nullable values. This means both its parameters and its result are nullable and the replacementFunction would have to deal with all that:
var visitor = new ReplaceMethodCallVisitor(
diffDaysMethod,
args => Expression.Convert(
Expression.Property(
Expression.Property(Expression.Subtract(args[0], args[1]), "Value"),
"Days"),
typeof(int?)));
var replacedExpression = visitor.Visit(GetDataRequiredExpression());
To make it work better, you could improve the visitor to take care of the nullability for you by stripping it from the method arguments and then readding it to the result, if necessary:
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (node.Method == methodToReplace)
{
var replacement = replacementFunction(
node.Arguments.Select(StripNullable).ToList());
if (replacement.Type != node.Type)
return Expression.Convert(replacement, node.Type);
}
return base.VisitMethodCall(node);
}
private static Expression StripNullable(Expression e)
{
var unaryExpression = e as UnaryExpression;
if (unaryExpression != null && e.NodeType == ExpressionType.Convert
&& unaryExpression.Operand.Type == Nullable.GetUnderlyingType(e.Type))
{
return unaryExpression.Operand;
}
return e;
}
Using this, the replacement function becomes much more reasonable:
var visitor = new ReplaceMethodCallVisitor(
diffDaysMethod,
args => Expression.Property(Expression.Subtract(args[0], args[1]), "Days"));

Replace parameter value in Expression Tree

After having searched for quite a while, I am still not finding the answer I am looking for. I have found answers about adding and removing parameters from a tree, but not anything about replacing specific parameters.
My first method is working as I would like it to, I need to replace the partitionKey value with the Uri escaped value and then return the results un-escaped.
public override IList<T> GetRowEntityList(string partitionKey)
{
IList<T> rowEntities = base.GetRowEntityList(Uri.EscapeDataString(partitionKey));
return rowEntities.Select(UnEscapeRowEntity).ToList();
}
The problem I am having is overriding this method to behave the same way. I already know that type T has the properties PartitionKey and RowKey but can also have any other number of properties.
For a sample predicate:
x => x.RowKey == "foo/bar" && x.SomeValue == "test"
I would expect it to become
x => x.RowKey == Uri.EscapeDataString("foo/bar") && x.SomeValue == "test"
Is there a way to do this?
My base class uses this predicate to do a table lookup on a table containing entities of type T using a Where(predicate) call
public override IList<T> GetRowEntityList(System.Linq.Expressions.Expression<Func<T, bool>> predicate)
{
//modify predicate value here
return base.GetRowEntityList(predicate);
}

You need to implement an ExpressionVisitor:
class MyVisitor : ExpressionVisitor
{
protected override Expression VisitBinary(BinaryExpression node)
{
if(CheckForMatch(node.Left))
return Expression.Equal(node.Left, Rewrite(node.Right));
if(CheckForMatch(node.Right))
return Expression.Equal(Rewrite(node.Left), node.Right);
return Expression.MakeBinary(node.NodeType, Visit(node.Left), Visit(node.Right));
}
private bool CheckForMatch(Expression e)
{
MemberExpression me = e as MemberExpression;
if(me == null)
return false;
if(me.Member.Name == "RowKey" || me.Member.Name == "PartitionKey")
return true;
else
return false;
}
private Expression Rewrite(Expression e)
{
MethodInfo mi = typeof(Uri).GetMethod("EscapeDataString");
return Expression.Call(mi, e);
}
}
I think that's right. It's a bit hard to test. Please note this will only work for the limited case of (x => x.RowKey == "some string"). It won't work for (x => x.RowKey.Equals("somestring"). It also won't work for (x => x.RowKey() == "some string").
You then use the implemented visitor to re-write the predicate:
Expression<Func<T, bool>> predicate = (s => s.RowKey == "1 2");
ExpressionVisitor v = new MyVisitor();
Expression<Func<T, bool>> rewrittenPredicate = v.Visit(predicate);
//rewrittenPredicate then tests if s.RowKey == "1%202"

linq to entities and store expression

I work on project that use some dynamic linq query to an entities.
i have huge amount of case and to avoid code duplication i refactoring to a method.
But using method which isn't in store expression will result to throw an exception.
One of solutions is to encapsulate method result into an expression which can be interpreted by linq to entitie query.
Consider that code :
parentExpression = x => x.child.Any(y=>IsGoodChild(y,childType, childSize));
private bool IsGoodChild(child c, int childType, int childSize){
return c.type == childType && c.size == childSize;
}
"parentExpression " is predicate of type "Parent" of my EF.
This code throw an exception, "IsGoodChild" method return a boolean and can't be interpreted by linq to Entities.
So, i would like something like this :
parentExpression = x => x.child.AsQueryable().Any(IsGoodChild(childType, childSize));
private System.Linq.Expression.Expression<Func<child, bool>> IsGoodChild(int childType, int childSize){
return ????
}
So how can i do "IsGoodChild(...)" can work even if which not take x.child attribute ?
Thx for advance
Re,
I try something, when i write lambda directly in expression like this :
parentExpression = x => x.child.Any(y=>y.type == childType && y.size == childSize);
i used extract method from resharper and generate it this :
private Expression<Func<child,Boolean>> IsGoodChildFunctional(Int32 childType, Int32 childSize)
{
return c => c.type == childType && c.size == childSize;
}
But i also have .NET Framework Data Provider error 1025' error ...

In this case the compiler is clever, given an anonymous method it will switch between an expression tree or a compiled lambda depending on the declared type. The following should work:
private Expression<Func<child,Boolean>>
IsGoodChildFunctional(Int32 childType, Int32 childSize)
{
return c => c.type == childType && c.size == childSize;
}
which would be used like so:
parentExpression = x => x.child
.AsQueryable()
.Any(IsGoodChildFunctional(childType,childSize));

Create a static generic method which will return an Expression. The Expression is built by using factory methods.
public static Expression<Func<TTargetObject,Boolean>> IsGoodChildFunctional<TTargetObject>(Int32 childType, Int32 childSize)
{
var e = Expression.Parameter(typeof(TTargetObject), "e");
var childTypeMember = Expression.MakeMemberAccess(e, typeof(TTargetObject).GetProperty("childType"));
var childSizeMember = Expression.MakeMemberAccess(e, typeof(TTargetObject).GetProperty("childSize"));
var childTypeConstant = Expression.Constant(childType, childType.GetType());
var childSizeConstant = Expression.Constant(childSize, childSize.GetType());
BinaryExpression b;
BinaryExpression bBis;
Expression<Func<TTargetObject, bool>> returnedExpression;
b = Expression.Equal(childTypeMember , childTypeConstant );
bBis2 = Expression.Equal(childSizeMember, c2);
var resultExpression = Expression.AndAlso(b, bBis);
returnedExpression = Expression.Lambda<Func<TTargetObject, bool>>(resultExpression , e);
return returnedExpression;
}
It is called like this:
var predicat = IsGoodChildFunctional<child>(childType, childSize);
parentExpression = x => x.child.Any(predicat);

Dynamically building queries

Hi and thanks for taking the time to answer my question.
After a year and a half of working with Java, I've decided to switch back to .NET. I must say that I feel at home in VS2012.
While working with Java I came across an implementation of hibernate that enabled for creating dynamic queries easily.
Imagine I had a form with 5 fields of which only one, any one, must be populated in order for me to filter the results by.
is there a way I can do the following in C#:
if(txtMunicipality.text.length > 0){
(x => x.municipality == txtMunicipality.text)
}
if(chkboxIsFinished){
(x => x.isfinished == true)
}
etc..
So I ccheck for every field and if the value has been populated then add that criteria to the query.. and after i'm done with the checks i execute the query. Is there a way to do this in C#?

The simplest way to do this is to compose two queries, i.e.
IQueryable<Foo> query = ... // or possibly IEnumerable<Foo>
if(!string.IsNullOrEmpty(txtMunicipality.text)) {
query = query.Where(x => x.municipality == txtMunicipality.text);
}
if(chkboxIsFinished) {
query = query.Where(x.isfinished);
}
You can also directly compose expression-trees and delegates; if you need that, please indicate which you have: an expression-tree vs a delegate.
Edit: here's how you would do that composing expressions rather than queries:
static class Program
{
static void Main()
{
Expression<Func<int, bool>> exp1 = x => x > 4;
Expression<Func<int, bool>> exp2 = x => x < 10;
Expression<Func<int, bool>> exp3 = x => x == 36;
var combined = (exp1.AndAlso(exp2)).OrElse(exp3);
// ^^^ equiv to x => (x > 4 && x < 10) || x == 36
}
static Expression<Func<T, bool>> OrElse<T>(this Expression<Func<T, bool>> x, Expression<Func<T, bool>> y)
{ // trivial cases
if (x == null) return y;
if (y == null) return x;
// rewrite using the parameter from x throughout
return Expression.Lambda<Func<T, bool>>(
Expression.OrElse(
x.Body,
SwapVisitor.Replace(y.Body, y.Parameters[0], x.Parameters[0])
), x.Parameters);
}
static Expression<Func<T, bool>> AndAlso<T>(this Expression<Func<T, bool>> x, Expression<Func<T, bool>> y)
{ // trivial cases
if (x == null) return y;
if (y == null) return x;
// rewrite using the parameter from x throughout
return Expression.Lambda<Func<T, bool>>(
Expression.AndAlso(
x.Body,
SwapVisitor.Replace(y.Body, y.Parameters[0], x.Parameters[0])
), x.Parameters);
}
class SwapVisitor : ExpressionVisitor
{
public static Expression Replace(Expression body, Expression from, Expression to)
{
return new SwapVisitor(from, to).Visit(body);
}
private readonly Expression from, to;
private SwapVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
}

Yes, it is possible. The simplest way is with delegates, especially the anonymous ones.
For example:
Func<YourEntity, bool> filter = (_ => true); // Default value.
if (txtMunicipality.text.length > 0)
{
filter = (x => x.municipality == txtMunicipality.text);
}
else if (chkboxIsFinished)
{
filter = (x => x.isfinished == true);
}
Then you can use the filter delegate in a query, for example in a Where statement (which I suppose was your intent - if not, the example is still relevant, just not directly applicable)
/ LINQ syntax.
var entities = from e in context
where filter(e)
select e;
// Method syntax.
var entities = context.Where(x => filter(x));
// Or simply:
var entities = context.Where(filter);

In this article you can find some useful extension methods that allows you to combine predicates (it should work for NHibernate as well):
LINQ to Entities: Combining Predicates
You can then build a lambda expression like this:
Expression<Func<MyObject, bool>> predicate = x => true;
if(txtMunicipality.text.length > 0){
predicate = predicate.And(x => x.municipality == txtMunicipality.text);
}
if(chkboxIsFinished){
predicate = predicate.And(x => x.isfinished == true);
}