Develop Reference

using expression trees to compare objects by a single property nets InvalidOperationException

I am trying to use Expression Trees because based on description, that seems to be the most correct (performant, configurable) approach.
I expect to be able to craft a statement that gets the first item from the existingItems collection that matches the propertyNameToCompareOn value of the incomingItem.
I have a method with the following signature and simulated code body...
DetectDifferences<T>(List<T> incomingItems, List<T> existingItems)
{
var propertyNameToCompareOn = GetThisValueFromAConfigFile(T.FullName());
//does this belong outside of the loop?
var leftParam = Expression.Parameter(typeof(T), "left");
var leftProperty = Expression.Property(leftParam, identField);
var rightParam = Expression.Parameter(typeof(T), "right");
var rightProperty = Expression.Property(rightParam, identField);
//this throws the error
var condition = Expression.Lambda<Func<T, bool>>(Expression.Equal(leftProperty, rightProperty));
foreach (var incomingItem in incomingItems) //could be a parallel or something else.
{
// also, where am I supposed to provide incomingItem to this statement?
var existingItem = existingItems.FirstOrDefault(expression/condition/idk);
// the statement for Foo would be something like
var existingFoos = exsistingItems.FirstOrDefault(f => f.Bar.Equals(incomingItem.Bar);
//if item does not exist, consider it new for persistence
//if item does exist, compare a configured list of the remaining properties between the
// objects. If they are all the same, report no changes. If any
// important property is different, capture the differences for
// persistence. (This is where precalculating hashes seems like the
// wrong approach due to expense.)
}
}
At the marked line above, I get an "Incorrect number of parameters supplied for lambda declaration" InvalidOperationException. At this point I am just hacking crap together from the web and I really dont know what this wants. There are a bunch of overloads that VS can full my screen with, and none of the examples make sense from the articles on MSDN/SO.
PS - I dont really want an IComparer or similar implementation if it can be helped. I can do that with reflection. I do need to make this as rapid as possible, but allow it to be called for multiple types, hence the choice of expression trees.

When working with expression trees, it's important to first understand, in real code, what you want to do.
I always begin by first writing out (in static code) what the resulting expression looks like with real C# lambda syntax.
Based on your description, your stated goal is that you should be able to (dynamically) look up some property of the type T that gives some sort of quick comparison. How would you write this if both T and TProperty were both known at compile time?
I suspect it would look something like this:
Func<Foo, Foo, bool> comparer = (Foo first, Foo second) =>
first.FooProperty == second.FooProperty;
Right away we can see that your Expression is wrong. You don't need one input T, you need two!
It should also be obvious why you're getting the InvalidOperationException as well. You never supplied any parameters to your lambda expression, only the body. Above, 'first' and 'second' are the parameters provided to the lambda. You'll need to provide them to the Expression.Lambda()call as well.
var condition = Expression.Lambda<Func<T,T, bool>>(
Expression.Equal(leftProperty, rightProperty),
leftParam,
rightParam);
This simply uses the Expression.Lambda(Expression, ParameterExpression[]) overload for Expression.Lambda. Each ParameterExpression is the parameter that is used in the body. That's it. Don't forget to .Compile() your expression into a delegate if you want to actually invoke it.
Of course this doesn't mean that your technique will be necessarily fast. If you're using fancy expression trees to compare two lists with a naive O(n^2) approach, it won't matter.

Here's a method to make a property access expression;
public static Expression<Func<T, object>> MakeLambda<T>(string propertyName)
{
var param = Expression.Parameter(typeof(T));
var propertyInfo = typeof(T).GetProperty(propertyName);
var expr = Expression.MakeMemberAccess(param, propertyInfo);
var lambda = Expression.Lambda<Func<T, object>>(expr, param);
return lambda;
}
which you can use like this;
var accessor = MakeLambda<Foo>("Name").Compile();
accessor(myFooInstance); // returns name
Making your missing line
var existingItem = existingItems.FirstOrDefault(e => accessor(e) == accessor(incomingItem));
Be aware the == only works well for value types like ints; careful of comparing objects.
Here's proof the lambda approach is much faster;
static void Main(string[] args)
{
var l1 = new List<Foo> { };
for(var i = 0; i < 10000000; i++)
{
l1.Add(new Foo { Name = "x" + i.ToString() });
}
var propertyName = nameof(Foo.Name);
var lambda = MakeLambda<Foo>(propertyName);
var f = lambda.Compile();
var propertyInfo = typeof(Foo).GetProperty(nameof(Foo.Name));
var sw1 = Stopwatch.StartNew();
foreach (var item in l1)
{
var value = f(item);
}
sw1.Stop();
var sw2 = Stopwatch.StartNew();
foreach (var item in l1)
{
var value = propertyInfo.GetValue(item);
}
sw2.Stop();
Console.WriteLine($"{sw1.ElapsedMilliseconds} vs {sw2.ElapsedMilliseconds}");
}
As someone's also pointed out, though, the double-loop in the OP is O(N^2) and that should probably be the next consideration if efficiency is the driver here.

Pass expression parameter as argument to another expression

I have a query which filters results:
public IEnumerable<FilteredViewModel> GetFilteredQuotes()
{
return _context.Context.Quotes.Select(q => new FilteredViewModel
{
Quote = q,
QuoteProductImages = q.QuoteProducts.SelectMany(qp => qp.QuoteProductImages.Where(qpi => q.User.Id == qpi.ItemOrder))
});
}
In the where clause I'm using the parameter q to match a property against a property from the parameter qpi.
Because the filter will be used in several places I'm trying to rewrite the where clause to an expression tree which would look like something like this:
public IEnumerable<FilteredViewModel> GetFilteredQuotes()
{
return _context.Context.Quotes.Select(q => new FilteredViewModel
{
Quote = q,
QuoteProductImages = q.QuoteProducts.SelectMany(qp => qp.QuoteProductImages.AsQueryable().Where(ExpressionHelper.FilterQuoteProductImagesByQuote(q)))
});
}
In this query the parameter q is used as a parameter to the function:
public static Expression<Func<QuoteProductImage, bool>> FilterQuoteProductImagesByQuote(Quote quote)
{
// Match the QuoteProductImage's ItemOrder to the Quote's Id
}
How would I implement this function? Or should I use a different approach alltogether?

If I understand correctly, you want to reuse an expression tree inside another one, and still allow the compiler to do all the magic of building the expression tree for you.
This is actually possible, and I have done it in many occasions.
The trick is to wrap your reusable part in a method call, and then before applying the query, unwrap it.
First I would change the method that gets the reusable part to be a static method returning your expression (as mr100 suggested):
public static Expression<Func<Quote,QuoteProductImage, bool>> FilterQuoteProductImagesByQuote()
{
return (q,qpi) => q.User.Id == qpi.ItemOrder;
}
Wrapping would be done with:
public static TFunc AsQuote<TFunc>(this Expression<TFunc> exp)
{
throw new InvalidOperationException("This method is not intended to be invoked, just as a marker in Expression trees!");
}
Then unwrapping would happen in:
public static Expression<TFunc> ResolveQuotes<TFunc>(this Expression<TFunc> exp)
{
var visitor = new ResolveQuoteVisitor();
return (Expression<TFunc>)visitor.Visit(exp);
}
Obviously the most interesting part happens in the visitor.
What you need to do, is find nodes that are method calls to your AsQuote method, and then replace the whole node with the body of your lambdaexpression. The lambda will be the first parameter of the method.
Your resolveQuote visitor would look like:
private class ResolveQuoteVisitor : ExpressionVisitor
{
public ResolveQuoteVisitor()
{
m_asQuoteMethod = typeof(Extensions).GetMethod("AsQuote").GetGenericMethodDefinition();
}
MethodInfo m_asQuoteMethod;
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (IsAsquoteMethodCall(node))
{
// we cant handle here parameters, so just ignore them for now
return Visit(ExtractQuotedExpression(node).Body);
}
return base.VisitMethodCall(node);
}
private bool IsAsquoteMethodCall(MethodCallExpression node)
{
return node.Method.IsGenericMethod && node.Method.GetGenericMethodDefinition() == m_asQuoteMethod;
}
private LambdaExpression ExtractQuotedExpression(MethodCallExpression node)
{
var quoteExpr = node.Arguments[0];
// you know this is a method call to a static method without parameters
// you can do the easiest: compile it, and then call:
// alternatively you could call the method with reflection
// or even cache the value to the method in a static dictionary, and take the expression from there (the fastest)
// the choice is up to you. as an example, i show you here the most generic solution (the first)
return (LambdaExpression)Expression.Lambda(quoteExpr).Compile().DynamicInvoke();
}
}
Now we are already half way through. The above is enough, if you dont have any parameters on your lambda. In your case you do, so you want to actually replace the parameters of your lambda to the ones from the original expression. For this, I use the invoke expression, where I get the parameters I want to have in the lambda.
First lets create a visitor, that will replace all parameters with the expressions that you specify.
private class MultiParamReplaceVisitor : ExpressionVisitor
{
private readonly Dictionary<ParameterExpression, Expression> m_replacements;
private readonly LambdaExpression m_expressionToVisit;
public MultiParamReplaceVisitor(Expression[] parameterValues, LambdaExpression expressionToVisit)
{
// do null check
if (parameterValues.Length != expressionToVisit.Parameters.Count)
throw new ArgumentException(string.Format("The paraneter values count ({0}) does not match the expression parameter count ({1})", parameterValues.Length, expressionToVisit.Parameters.Count));
m_replacements = expressionToVisit.Parameters
.Select((p, idx) => new { Idx = idx, Parameter = p })
.ToDictionary(x => x.Parameter, x => parameterValues[x.Idx]);
m_expressionToVisit = expressionToVisit;
}
protected override Expression VisitParameter(ParameterExpression node)
{
Expression replacement;
if (m_replacements.TryGetValue(node, out replacement))
return Visit(replacement);
return base.VisitParameter(node);
}
public Expression Replace()
{
return Visit(m_expressionToVisit.Body);
}
}
Now we can advance back to our ResolveQuoteVisitor, and hanlde invocations correctly:
protected override Expression VisitInvocation(InvocationExpression node)
{
if (node.Expression.NodeType == ExpressionType.Call && IsAsquoteMethodCall((MethodCallExpression)node.Expression))
{
var targetLambda = ExtractQuotedExpression((MethodCallExpression)node.Expression);
var replaceParamsVisitor = new MultiParamReplaceVisitor(node.Arguments.ToArray(), targetLambda);
return Visit(replaceParamsVisitor.Replace());
}
return base.VisitInvocation(node);
}
This should do all the trick.
You would use it as:
public IEnumerable<FilteredViewModel> GetFilteredQuotes()
{
Expression<Func<Quote, FilteredViewModel>> selector = q => new FilteredViewModel
{
Quote = q,
QuoteProductImages = q.QuoteProducts.SelectMany(qp => qp.QuoteProductImages.Where(qpi => ExpressionHelper.FilterQuoteProductImagesByQuote().AsQuote()(q, qpi)))
};
selector = selector.ResolveQuotes();
return _context.Context.Quotes.Select(selector);
}
Of course I think you can make here much more reusability, with defining expressions even on a higher levels.
You could even go one step further, and define a ResolveQuotes on the IQueryable, and just visit the IQueryable.Expression and creating a new IQUeryable using the original provider and the result expression, e.g:
public static IQueryable<T> ResolveQuotes<T>(this IQueryable<T> query)
{
var visitor = new ResolveQuoteVisitor();
return query.Provider.CreateQuery<T>(visitor.Visit(query.Expression));
}
This way you can inline the expression tree creation. You could even go as far, as override the default query provider for ef, and resolve quotes for every executed query, but that might go too far :P
You can also see how this would translate to actually any similar reusable expression trees.
I hope this helps :)
Disclaimer: Remember never copy paste code from anywhere to production without understanding what it does. I didn't include much error handling here, to keep the code to minimum. I also didn't check the parts that use your classes if they would compile. I also don't take any responsability for the correctness of this code, but i think the explanation should be enough, to understand what is happening, and fix it if there are any issues with it.
Also remember, that this only works for cases, when you have a method call that produces the expression. I will soon write a blog post based on this answer, that allows you to use more flexibility there too :P

Implementing this your way will cause an exception thrown by ef linq-to-sql parser. Within your linq query you invokes FilterQuoteProductImagesByQuote function - this is interpreted as Invoke expression and it simply cannot be parsed to sql. Why? Generally because from SQL there is no possibility to invoke MSIL method. The only way to pass expression to query is to store it as Expression> object outside of the query and then pass it to Where method. You can't do this as outside of the query you will not have there Quote object. This implies that generally you cannot achieve what you wanted. What you possibly can achieve is to hold somewhere whole expression from Select like this:
Expression<Func<Quote,FilteredViewModel>> selectExp =
q => new FilteredViewModel
{
Quote = q,
QuoteProductImages = q.QuoteProducts.SelectMany(qp => qp.QuoteProductImages.AsQueryable().Where(qpi => q.User.Id == qpi.ItemOrder)))
};
And then you may pass it to select as argument:
_context.Context.Quotes.Select(selectExp);
thus making it reusable. If you would like to have reusable query:
qpi => q.User.Id == qpi.ItemOrder
Then first you would have to create different method for holding it:
public static Expression<Func<Quote,QuoteProductImage, bool>> FilterQuoteProductImagesByQuote()
{
return (q,qpi) => q.User.Id == qpi.ItemOrder;
}
Application of it to your main query would be possible, however quite difficult and hard to read as it will require defining that query with use of Expression class.

Syntax to refer a method returning an Expression to another method?

I found a piece of code of the following form:
public static Expression<Func<Invoice, CustomerContact>> GetCustomerContact()
{
return i => new CustomerContact {
FirstName = i.Customer.FirstName,
LastName = i.Customer.LastName,
Email = i.Customer.Email,
TelMobile = i.Customer.TelMobile,
};
}
In other parts of the code, I want to get the same lightweight CustomerContact object, only not from the Invoice, but from the Customer itself. So the obvious thing to do would be to have:
public static Expression<Func<Customer, CustomerContact>> GetCustomerContact()
{
return c => new CustomerContact {
FirstName = c.FirstName,
LastName = c.LastName,
Email = c.Email,
TelMobile = c.TelMobile,
};
}
and then change the Expression taking Invoice as input to refer to this method, i.e. something like this:
public static Expression<Func<Invoice, CustomerContact>> GetCustomerContact()
{
return i => GetCustomerContact(i.Customer); // doesn't compile
}
What's the correct syntax for this?

You can use Expression.Invoke:
var paramExpr = Expression.Parameter(typeof(Invoice), "i");
var propertyEx = Expression.Property(paramExpr, "Customer");
var body = Expression.Invoke(GetCustomerContactFromCustomer(), propertyEx);
return Expression.Lambda<Func<Invoice, CustomerContact>>(body, paramExpr);
Do note that some LINQ providers have problems with such invocation-expressions.
The easiest way to work around this (and to give you more convenient syntax) is to use LINQKit:
var expr = GetCustomerContactFromCustomer();
Expression<Func<Invoice, CustomerContact>> result = i => expr.Invoke(i.Customer);
return result.Expand();

Are you sure you need to use an Expression? If you don't need different Linq providers to convert code trees into queries, then consider using just Func, instead. If you just use Func so that the method signatures are:
public static Func<Customer, CustomerContact> GetCustomerContact();
and
public static Func<Customer, CustomerContact> GetCustomerContact();
Then your syntax would be fine for constructing the second Func off of the first one. Of course, this will only work for in-memory objects (with Linq-to-objects).
The problem is that in order to build an Expression, you have to explicitely build the evaluation tree, which can be quite hairy (using the various static methods on Expression). Because of this hairiness, there are several helper packages, including LINQKit.

Dude, where's my object? or, Why does Linq not return my object?

Once I have the results of my Linq query, I am not always happy. There could be a result that I was expecting to be there but wasn't. For example, my client was expecting that a customer was in a customer list, but it wasn't. It is my client saying "Dude, where's my customer?", not me. I am the Dude, and to remain a dude, I have to give my client the reason.
Is there a simple way to take a given object instance and a Linq query and determine which expressions within the query excluded that instance?
Edit Ok, here is a better example
Output should be something along the lines:
Your Customer was excluded for 2 reasons:
Customer FirstName is Carl but it should be Daniel
Customer Age is 18 but it should be > 20
public class Customer
{
public string FirstName { get; set; }
public int Age { get; set; }
}
[Test]
public void Dude_wheres_my_object_test1()
{
var daniel = new Customer { FirstName = "Daniel", Age = 41 };
var carl = new Customer { FirstName = "Carl", Age= 18 };
var Customers = new List<Customer>() { daniel, carl };
// AsQueryable() to convert IEnumerable<T> to IQueryable<T> in
//the case of LinqtoObjects - only needed for this test, not
//production code where queies written for LinqToSql etc normally
//return IQueryable<T>
var query = from c in Customers.AsQueryable()
where c.Age > 20
where c.FirstName == "Daniel"
select c;
//query would return Daniel as you'd expect, but not executed here.
//However I want to explain why Carl was not in the results
string[] r = DudeWheresMyObject(query, carl);
Assert.AreEqual("Age is 18 but it should be > 20", r[0]);
Assert.AreEqual("FirstName is Carl but it should be Daniel", r[1]);
//Should even work for a Customer who is not
//in the original Customers collection...
var ficticiousCustomer = new Customer { FirstName = "Other", Age = 19};
string[] r2= DudeWheresMyObject(query,
ficticiousCustomer);
Assert.AreEqual("Age is 19 but it should be > 20", r2[0]);
Assert.AreEqual("FirstName is Other but it should be Daniel", r2[1]);
}
public string[] DudeWheresMyObject<T>(IQueryable<T> query, T instance)
{
//Do something here with the query.Expression and the instance
}
First of all, before I attempt to write some fancy Fluent framework, Has anyone done this already?
So far, I have considered navigating the expression tree and executing each branch against an IQueryable that only contains my object. Now I don't have a great deal of experience using raw expression trees, so I would like those who have to suggest any pitfalls or even explain whether this is a dead end and why.
I am anxious that anything that results from this should:
Be Reusable - Should be applicable to any object compared against a Linq query returning objects of the same class.
Not affect the performance of the original query (this should just be standard Linq).
Should be Linq-implementation agnostic.
If there are multiple property values set on the missing instance that excluded it from the results, then all of those reasons should be reported.
Edit
I am not suggesting that I keep executing LinqToSql against the database multiple times with different permutations of the query and comparing the results. Rather, I am looking for a way to take a single instance and compare it to the expression tree (without executing the query directly again)
Also, I would like an indication of whether others might find this useful. If so, I would consider starting an open source project to solve it.

I think you'd have to re-create the query as linq-to-objects and deal with the subtle differences between linq-to-sql/entities/whatever and linq-to-objects, accepting that some providers just won't work realistically.
You have your object you want to find in an in memory IEnumerable<T> or something.
You'd have to walk the expression tree somehow and snip out the leaves, so say you had:
where obj.foo == true && obj.bar == "yes"
you'd have to figure out that obj.foo == true and obj.bar == "yes" are leaves and start there. It'd be a sort of depth first search of the expression tree.
So, construct linq to objects queries that only had those leaves. See if the object is included in the results. If not then we've found out why it's excluded, if not then go up the tree (i.e. make the where query include more clauses, getting closer to the orignal one until the object disappears from the results).
As I see it the tough parts would be handling the differences between original linq to 'whatever' and link to objects, figuring out where to split the where claues, dealing with things like joins which can also exclude things and dealing with things like SqlMethods.Like that don't work in linq to objects.

For a one-off exploration of what's filtering out the result, it's hard to beat the Dump method in LINQPad. Here's an extract from one of their samples that shows it in action:
// Dump returns exactly what it was given, so you can sneakily inject
// a Dump (or even many Dumps) *within* an expression. This is useful
// for monitoring a query as it progresses:
new[] { 11, 5, 17, 7, 13 } .Dump ("Prime numbers")
.Where (n => n > 10) .Dump ("Prime numbers > 10")
.OrderBy (n => n) .Dump ("Prime numbers > 10 sorted")
.Select (n => n * 10) .Dump ("Prime numbers > 10 sorted, times 10!");
This gives nicely formatted tables of results:

With some fun expression hacking, you can see the results of each stage of the evaluation for each item in the set. Inspect the local result after the breakpoint has been hit to see the results of the evaluation. To actually use the results of the evaluation, just append .Where(x => x.IsIncludedInResult).Select(x => x.EvaluationTarget) to the line where the report is generated.
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Linq.Expressions;
namespace ConsoleApplication4
{
[DebuggerDisplay("{Condition} on {EvaluationTarget} is {EvaluationResult}")]
public class ReportItem<T>
{
public string Condition { get; private set; }
public IEnumerable<ReportItem<T>> NestedReports { get; private set; }
public object EvaluationResult { get; private set; }
public T EvaluationTarget { get; private set; }
public ReportItem(Expression condition, IEnumerable<ReportItem<T>> nestedReports, T evaluationTarget, object evaluationResult)
{
Condition = condition.ToString();
NestedReports = nestedReports;
EvaluationTarget = evaluationTarget;
EvaluationResult = evaluationResult;
}
public override string ToString()
{
return string.Format("{0} on {1} is {2}", Condition, EvaluationTarget, EvaluationResult);
}
}
[DebuggerDisplay("Included: {IsIncludedInResult} \n{Summary}")]
public class Report<T>
{
public ReportItem<T> Contents { get; private set; }
public T EvaluationTarget { get; private set; }
public Report(T source, Expression<Func<T, bool>> predicate)
{
EvaluationTarget = source;
IsIncludedInResult = predicate.Compile()(source);
Contents = Recurse(predicate.Parameters.Single(), predicate.Body, source);
}
private object Evaluate(Expression expression, ParameterExpression parameter, T source)
{
var expr = Expression.Lambda(expression, parameter);
var #delegate = expr.Compile();
var value = #delegate.DynamicInvoke(source);
return value;
}
private ReportItem<T> Recurse(ParameterExpression parameter, Expression sourceExpression, T source)
{
var constantExpression = sourceExpression as ConstantExpression;
if(constantExpression != null)
{
return new ReportItem<T>(sourceExpression, null, source, Evaluate(constantExpression, parameter, source));
}
var unaryExpression = sourceExpression as UnaryExpression;
if(unaryExpression != null)
{
var content = Recurse(parameter, unaryExpression.Operand, source);
var result = Evaluate(sourceExpression, parameter, source);
return new ReportItem<T>(sourceExpression, new[]{content}, source, result);
}
var binaryExpression = sourceExpression as BinaryExpression;
if(binaryExpression != null)
{
var left = Recurse(parameter, binaryExpression.Left, source);
var right = Recurse(parameter, binaryExpression.Right, source);
var item = new ReportItem<T>(sourceExpression, new[] {left, right}, source, Evaluate(sourceExpression, parameter, source));
return item;
}
var methodCallExpression = sourceExpression as MethodCallExpression;
if(methodCallExpression != null)
{
var args = methodCallExpression.Arguments.Select(x => Evaluate(x, parameter, source)).ToArray();
var result = methodCallExpression.Method.Invoke(Expression.Lambda(methodCallExpression.Object, parameter).Compile().DynamicInvoke(source), args);
return new ReportItem<T>(sourceExpression, null, source, result);
}
throw new Exception("Unhandled expression type " + sourceExpression.NodeType + " encountered");
}
public bool IsIncludedInResult { get; private set; }
public string Summary
{
get { return Contents.ToString(); }
}
public override string ToString()
{
return Summary;
}
}
public static class PredicateRunner
{
public static IEnumerable<Report<T>> Report<T>(this IEnumerable<T> set, Expression<Func<T, bool>> predicate)
{
return set.Select(x => new Report<T>(x, predicate));
}
}
class MyItem
{
public string Name { get; set; }
public int Value { get; set; }
public override int GetHashCode()
{
return Value % 2;
}
public override string ToString()
{
return string.Format("Name: \"{0}\" Value: {1}", Name, Value);
}
}
class Program
{
static void Main()
{
var items = new MyItem[3];
items[0] = new MyItem
{
Name = "Hello",
Value = 1
};
items[1] = new MyItem
{
Name = "Hello There",
Value = 2
};
items[2] = new MyItem
{
Name = "There",
Value = 3
};
var result = items.Report(x => !x.Name.Contains("Hello") && x.GetHashCode() == 1).ToList();
Debugger.Break();
}
}
}

It's kind of a tricky one, as in, from your example you could always code something to check for specifics and report 'I searched for term x and the object i returned was not in term x'.
I would have though as others suggested though, that this would have been along the lines for 'return me x' then in code, run a query for 'x where x.property = y' and report non matches.
Following this through, I'd imagine the issue would be that in order to generate the list of non matches, your query or object graph would become pretty massive as you'd need your original object either initially include (or to be expanded via lazy loading to include) many permutations to determine the matches or not.
This is kind of the inverse of running a query on the first place where you'd start with an object and sub select based on conditions, you'd want to select and then selectively super select, catching non conditions.
It's an interesting problem, and one that I'd normally address either client side or code wise before getting to a point where and object was returned or not. But I guess the perfect solution would be to return a single solution and perhaps examine it's associations for links.
The links wouldn't be too hard to find a generic "I've not got one of these" type reason, but to give a 'I've got this link, not that link' response would be harder.
You'd need to maybe provide a method based on some form of predicate builder which took a field and search term and returned an appropriate message if things didn't match. In my mind seems like two slightly different problems.
Slightly rambling now, but would be curious to hear any answers to this!...

I think I follow what you mean. What I think you would want to do is perform two queries, one with selection criteria, and one without, then perform a Linq Except on them to determine which items were excluded, then walk that list and determine what criteria caused them to be excluded.
I can't really think of a better way to do it.
Something like this:
var a = db.Trades.Where(z => z.user == x && z.date == y);
var b = a.Where(z => z.TradeCurrency != null && z.TradeUnderlying.Index != null);
var c = a.Except(b);
List<string> reasons;
foreach(var d in c) {
if (d.TradeCurrency == null)
// add reason
... etc..
}
This would perform a single query (which would have several sub-queries) and only return the results that were excluded (rather than trying to return all results which could be quite large). Unless of course you have a million excluded records and only a few included ones.
Not sure how efficient this is, though compared to a way I can't think of.
EDIT:
I think you are forgetting that Linq queries do not execute until you call an operation that realizes them. In this example, the database is only hit once, even though there are several linq query objects here. The expression tree is modified without executing the queries.
So in this example, when the foreach() occurs, a single database query (with several sub-queries) is executed.

LINQ: How to force a value based reference?

I want to provide a set of filters for a user to pick from, and each filter will correspond to an Expression<Func<X, bool>>. So, I might want to take a dynamic list of available items ('Joe', 'Steve', 'Pete', etc), and create a collection of "hard-coded" filters based on those names, and let the user select which filter(s) he wants to use. My problem is that even when I try to "hard-code" my expression based on a string value from the dynamic list, the expression is still storing the value as, what looks to me, a property hanging off of an anonymous type (and I don't know how to serialize the anon. type). Sorry if this is confusing, I'm not quite sure how to articulate this.
Here's my sample code:
public class Foo
{
public string Name { get; set; }
}
static void Main(string[] args)
{
Foo[] source = new Foo[]
{
new Foo() { Name = "Steven" } ,
new Foo() { Name = "John" } ,
new Foo() { Name = "Pete" },
};
List<Expression<Func<Foo, bool>>> filterLst = new List<Expression<Func<Foo, bool>>>();
foreach (Foo f in source)
{
Expression<Func<Foo, bool>> exp = x => x.Name == f.Name;
filterLst.Add(exp);
}
}
}
My problem is that when I look at when I look at the body of my expression, it reads as follows:
(x.Name = value(ConsoleApplication1.Program+<>c__DisplayClass3).value)
When what I really want is for the first one to read like this:
(x.Name = "Steven")
(if I change my code to this, instead, that's exactly what I get:
Expression<Func<Foo, bool>> exp = x => x.Name == "Steven";
)
I've tried forcing my value to a local string value before sticking it into the Expression, but it doesn't seem to help:
List<Expression<Func<Foo, bool>>> filterLst = new List<Expression<Func<Foo, bool>>>();
foreach (Foo f in source)
{
string value = f.Name;
Expression<Func<Foo, bool>> exp = x => x.Name == value;
filterLst.Add(exp);
}
I don't understand why (or really even how) it's still looking at some anonymous type even once I'm using a local variable that is declared to a string. Is there a way to make this work as I want it to?

The anon-type is actually the compiler-generated type it is using to perform capture of the variables. With delegates you can hack around this by implementing the capture by hand, but not with lambda expressions compiled to expression-trees.
Two choices:
build the expression tree explicitely on code via Expression.Constant etc
learn how to handle the anon types
The latter isn't too bad actually; they are just MemberExpression typically, although I have some code kicking around that covers this in full detail. I can also provide examples of buildin the expression tree, but I'm not at a PC at the moment and it doesn't lend itself well to iPod typing...
From a brief read of the question I'd look at the first option more than the second.
Oh, and watch out; the first foreach code in the question looks susceptible to the notorious l-value capture issue ;)
Edit: I found a PC ;p
var param = Expression.Parameter(typeof(Foo), "x");
var body = Expression.Equal(
Expression.PropertyOrField(param, "Name"),
Expression.Constant(f.Name, typeof(string)));
var exp = Expression.Lambda<Func<Foo, bool>>(body, param);
filterLst.Add(exp);

Marc Gravell's answer is correct, and here's how you'd implement his first choice:
var filters =
from f in source
let param = Expression.Parameter(typeof(Foo),"x")
select Expression.Lambda<Func<Foo, bool>>(
Expression.Equal(
Expression.Property(param, "Name"),
Expression.Constant(f.Name)), param);
But this typically isn't necessary. Your second example of the for loop should work with all major LINQ providers. Is there a reason you need the expression to use constants?