Develop Reference

Encapsulation and use of LINQ logic in functions using dot notation

I tried my best to explain in the title, however I am trying to achieve giving linq statements an 'alias' and still use them in dot notation. Allow me to explain further.
below we have a list that has a linq statement applied:
private List<string> _matches;
var output = _matches.Where(x => x.EntityScore == 100).ToList();
I agree that this is simple to read. However I wish to simplify it further especially when the statements start to get bigger. This is an example of linq getting longer than I care for:
private List<string> _matches;
var matchAddressList = _matches.Where(x => x.EntityDetails.Addresses.Any(x => x.Street.Equals(inputObject.Address)
&& x.StateProvinceDistrict.Equals(inputObject.State)
&& x.City.Equals(inputObject.City))).ToList();
What I am trying to do is alias certain groups of LINQ and then call that linq as a dot operator
for example:
var finalOutput = _matches.perfectMatches().addressMatches(inputObject).someOtherMatchCondition(inputObject)
I think the above line is clear and easily readable. Future devs dont necessarily have to look into the logic. They can read the business domain name and understand what it does.
I want to avoid the following line, as I believe the previous code is more clean:
var finalOutput = someOtherMatchCondition(addressMatches(perfectMatches(_matches)));
the previous line is how I feel you would go about it using functions at a basic level. However I am struggling to find a way to create an alias or encapsulate the linq logic into a business domain name and then use that as a dot operator.
I have tried expression body definitions:
public List<string> perfectMatches => _matches.Where(x => x.EntityScore == 100).ToList();
is this going to require extensions of another class? or the writing of generics? or am I perhaps unaware of a standard way of doing this?

Update: maybe this is helpfull too:
How to add custom methods for LINQ queries (C#)
It has to be an extension method to make use of the dot notation.
Do you mean something like that. It is rather pseudo code than working. You may have to play around with the types or try out some kind of generic approach:
public class ProductionCode
{
public void MyMain()
{
var myList = new List<EntityThingType>() { .... };
var newList = myList.PerfectMatches().AddressMatches(myInputObject).ToList();
}
}
public static class test
{
public static IEnumerable<EntityThingType> PerfectMatches(this IEnumerable<EntityThingType> myList)
{
return myList.Where(x => x.EntityScore == 100);
}
public static IEnumerable<EntityThingType> AddressMatches(this IEnumerable<EntityThingType> myList, MyObjectType inputObject)
{
return myList.Where(x => x.EntityDetails.Addresses.Any(x => x.Street.Equals(inputObject.Address)
&& x.StateProvinceDistrict.Equals(inputObject.State)
&& x.City.Equals(inputObject.City)));
}
}

I think what you are looking for is Extension Methods. You can have the perfectMatches() method be an extension method that takes an IEnumerable<string> and return the same. Then you can chain those together.

Pass expression parameter as argument to another expression

I have a query which filters results:
public IEnumerable<FilteredViewModel> GetFilteredQuotes()
{
return _context.Context.Quotes.Select(q => new FilteredViewModel
{
Quote = q,
QuoteProductImages = q.QuoteProducts.SelectMany(qp => qp.QuoteProductImages.Where(qpi => q.User.Id == qpi.ItemOrder))
});
}
In the where clause I'm using the parameter q to match a property against a property from the parameter qpi.
Because the filter will be used in several places I'm trying to rewrite the where clause to an expression tree which would look like something like this:
public IEnumerable<FilteredViewModel> GetFilteredQuotes()
{
return _context.Context.Quotes.Select(q => new FilteredViewModel
{
Quote = q,
QuoteProductImages = q.QuoteProducts.SelectMany(qp => qp.QuoteProductImages.AsQueryable().Where(ExpressionHelper.FilterQuoteProductImagesByQuote(q)))
});
}
In this query the parameter q is used as a parameter to the function:
public static Expression<Func<QuoteProductImage, bool>> FilterQuoteProductImagesByQuote(Quote quote)
{
// Match the QuoteProductImage's ItemOrder to the Quote's Id
}
How would I implement this function? Or should I use a different approach alltogether?

If I understand correctly, you want to reuse an expression tree inside another one, and still allow the compiler to do all the magic of building the expression tree for you.
This is actually possible, and I have done it in many occasions.
The trick is to wrap your reusable part in a method call, and then before applying the query, unwrap it.
First I would change the method that gets the reusable part to be a static method returning your expression (as mr100 suggested):
public static Expression<Func<Quote,QuoteProductImage, bool>> FilterQuoteProductImagesByQuote()
{
return (q,qpi) => q.User.Id == qpi.ItemOrder;
}
Wrapping would be done with:
public static TFunc AsQuote<TFunc>(this Expression<TFunc> exp)
{
throw new InvalidOperationException("This method is not intended to be invoked, just as a marker in Expression trees!");
}
Then unwrapping would happen in:
public static Expression<TFunc> ResolveQuotes<TFunc>(this Expression<TFunc> exp)
{
var visitor = new ResolveQuoteVisitor();
return (Expression<TFunc>)visitor.Visit(exp);
}
Obviously the most interesting part happens in the visitor.
What you need to do, is find nodes that are method calls to your AsQuote method, and then replace the whole node with the body of your lambdaexpression. The lambda will be the first parameter of the method.
Your resolveQuote visitor would look like:
private class ResolveQuoteVisitor : ExpressionVisitor
{
public ResolveQuoteVisitor()
{
m_asQuoteMethod = typeof(Extensions).GetMethod("AsQuote").GetGenericMethodDefinition();
}
MethodInfo m_asQuoteMethod;
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (IsAsquoteMethodCall(node))
{
// we cant handle here parameters, so just ignore them for now
return Visit(ExtractQuotedExpression(node).Body);
}
return base.VisitMethodCall(node);
}
private bool IsAsquoteMethodCall(MethodCallExpression node)
{
return node.Method.IsGenericMethod && node.Method.GetGenericMethodDefinition() == m_asQuoteMethod;
}
private LambdaExpression ExtractQuotedExpression(MethodCallExpression node)
{
var quoteExpr = node.Arguments[0];
// you know this is a method call to a static method without parameters
// you can do the easiest: compile it, and then call:
// alternatively you could call the method with reflection
// or even cache the value to the method in a static dictionary, and take the expression from there (the fastest)
// the choice is up to you. as an example, i show you here the most generic solution (the first)
return (LambdaExpression)Expression.Lambda(quoteExpr).Compile().DynamicInvoke();
}
}
Now we are already half way through. The above is enough, if you dont have any parameters on your lambda. In your case you do, so you want to actually replace the parameters of your lambda to the ones from the original expression. For this, I use the invoke expression, where I get the parameters I want to have in the lambda.
First lets create a visitor, that will replace all parameters with the expressions that you specify.
private class MultiParamReplaceVisitor : ExpressionVisitor
{
private readonly Dictionary<ParameterExpression, Expression> m_replacements;
private readonly LambdaExpression m_expressionToVisit;
public MultiParamReplaceVisitor(Expression[] parameterValues, LambdaExpression expressionToVisit)
{
// do null check
if (parameterValues.Length != expressionToVisit.Parameters.Count)
throw new ArgumentException(string.Format("The paraneter values count ({0}) does not match the expression parameter count ({1})", parameterValues.Length, expressionToVisit.Parameters.Count));
m_replacements = expressionToVisit.Parameters
.Select((p, idx) => new { Idx = idx, Parameter = p })
.ToDictionary(x => x.Parameter, x => parameterValues[x.Idx]);
m_expressionToVisit = expressionToVisit;
}
protected override Expression VisitParameter(ParameterExpression node)
{
Expression replacement;
if (m_replacements.TryGetValue(node, out replacement))
return Visit(replacement);
return base.VisitParameter(node);
}
public Expression Replace()
{
return Visit(m_expressionToVisit.Body);
}
}
Now we can advance back to our ResolveQuoteVisitor, and hanlde invocations correctly:
protected override Expression VisitInvocation(InvocationExpression node)
{
if (node.Expression.NodeType == ExpressionType.Call && IsAsquoteMethodCall((MethodCallExpression)node.Expression))
{
var targetLambda = ExtractQuotedExpression((MethodCallExpression)node.Expression);
var replaceParamsVisitor = new MultiParamReplaceVisitor(node.Arguments.ToArray(), targetLambda);
return Visit(replaceParamsVisitor.Replace());
}
return base.VisitInvocation(node);
}
This should do all the trick.
You would use it as:
public IEnumerable<FilteredViewModel> GetFilteredQuotes()
{
Expression<Func<Quote, FilteredViewModel>> selector = q => new FilteredViewModel
{
Quote = q,
QuoteProductImages = q.QuoteProducts.SelectMany(qp => qp.QuoteProductImages.Where(qpi => ExpressionHelper.FilterQuoteProductImagesByQuote().AsQuote()(q, qpi)))
};
selector = selector.ResolveQuotes();
return _context.Context.Quotes.Select(selector);
}
Of course I think you can make here much more reusability, with defining expressions even on a higher levels.
You could even go one step further, and define a ResolveQuotes on the IQueryable, and just visit the IQueryable.Expression and creating a new IQUeryable using the original provider and the result expression, e.g:
public static IQueryable<T> ResolveQuotes<T>(this IQueryable<T> query)
{
var visitor = new ResolveQuoteVisitor();
return query.Provider.CreateQuery<T>(visitor.Visit(query.Expression));
}
This way you can inline the expression tree creation. You could even go as far, as override the default query provider for ef, and resolve quotes for every executed query, but that might go too far :P
You can also see how this would translate to actually any similar reusable expression trees.
I hope this helps :)
Disclaimer: Remember never copy paste code from anywhere to production without understanding what it does. I didn't include much error handling here, to keep the code to minimum. I also didn't check the parts that use your classes if they would compile. I also don't take any responsability for the correctness of this code, but i think the explanation should be enough, to understand what is happening, and fix it if there are any issues with it.
Also remember, that this only works for cases, when you have a method call that produces the expression. I will soon write a blog post based on this answer, that allows you to use more flexibility there too :P

Implementing this your way will cause an exception thrown by ef linq-to-sql parser. Within your linq query you invokes FilterQuoteProductImagesByQuote function - this is interpreted as Invoke expression and it simply cannot be parsed to sql. Why? Generally because from SQL there is no possibility to invoke MSIL method. The only way to pass expression to query is to store it as Expression> object outside of the query and then pass it to Where method. You can't do this as outside of the query you will not have there Quote object. This implies that generally you cannot achieve what you wanted. What you possibly can achieve is to hold somewhere whole expression from Select like this:
Expression<Func<Quote,FilteredViewModel>> selectExp =
q => new FilteredViewModel
{
Quote = q,
QuoteProductImages = q.QuoteProducts.SelectMany(qp => qp.QuoteProductImages.AsQueryable().Where(qpi => q.User.Id == qpi.ItemOrder)))
};
And then you may pass it to select as argument:
_context.Context.Quotes.Select(selectExp);
thus making it reusable. If you would like to have reusable query:
qpi => q.User.Id == qpi.ItemOrder
Then first you would have to create different method for holding it:
public static Expression<Func<Quote,QuoteProductImage, bool>> FilterQuoteProductImagesByQuote()
{
return (q,qpi) => q.User.Id == qpi.ItemOrder;
}
Application of it to your main query would be possible, however quite difficult and hard to read as it will require defining that query with use of Expression class.

BuildPredicate comparing two fields

I am trying to map comparisons between two fields, where previously I had only been doing field to Expression.Constant comparisons.
private static Expression<Func<TDomainModel, bool>> BuildPredicate<TDomainModel>(string leftPropName, string rightPropName, TypeMap map)
{
PropertyMap leftPropMap = map.GetPropertyMaps().FirstOrDefault(pro => pro.DestinationProperty.Name == leftPropName);
Expression leftParam = leftPropMap.CustomExpression.Body;
PropertyMap rightPropMap = map.GetPropertyMaps().FirstOrDefault(pro => pro.DestinationProperty.Name == rightPropName);
Expression rightParam = rightPropMap.CustomExpression.Body;
Expression operatorBody = Expression.GreaterThanOrEqual(leftParam, rightParam);
return Expression.Lambda<Func<TDomainModel, bool>>(operatorBody, leftPropMap.CustomExpression.Parameters[0]);
}
However I always get an exception...
The parameter 's' was not bound in the specified LINQ to Entities query expression.
Both properties are on the same entity, and even use 's' in their mappings. No idea how to fix this. I've tried putting both CustomExpression.Parameters values into the returned Lambda, but it complains of too many overloads then.
Thanks for any help.

You need to rewrite the right expression using the same parameter as the left expression. They're different now.
I used to do this creating a Expression.Invoke node to the right side, then expanding invocations with a ExpressionVisitor.

How to define anonymous method types to build dynamic queries with LINQ?

I'm busy with a LINQ to SQL project that basically creates multiple threads for each entity type in my database, which constantly queries information from the DB in a thread.
Here's a pseudo example:
streamer.DefineDataExpression<Contacts>(x => x.FirstName == "Bob");
while(true)
{
List<Contacts> MyContactsResult = streamer.ResultList;
// do whatever with MyContactsResult
}
The above code doesn't exist, but this is what I have so far for the 'streamer' class (it obviously doesn't work, but you can see what I'm trying to achieve above):
public void DefineExpression(System.Linq.Expressions.Expression<System.Func<T, bool>> expression)
{
using (var db = new LINQDataContext())
{
ResultList = db.GetTable<T>().Where(expression);
}
}
How do I go about creating a method like 'DefineExpression' that will allow me to query a LINQ type dynamically?

Why not use the Dynamic LINQ provider, as mentioned by Scott Guthrie. I think that would give you everything you are looking for, because you can define the query as a string. Therefore, you can more easily build a string representation of your query, and execute on the fly.

when and in what scenario to use Expression Tree

I was reading about Expression Tree feature and how you can create delegates using lambda expressions. I still can't get as to in what scenario it is useful and in what real world example should I use it.

The primary use for expression trees is for out-of-process LINQ providers such as LINQ to SQL.
When you write something like this:
var query = people.Where(x => x.Age > 18)
.Select(x => x.Name);
those lambda expressions can either be converted to delegates, which can then be executed (as they are in LINQ to Object) or they can be converted to expression trees, which can be analyzed by the query source and acted on accordingly (e.g. by turning them into SQL, web service calls etc). The difference is that expression trees represent the code as data. They can be compiled into delegates if necessary, but usually (within LINQ anyway) they're never executed directly - just examined to find out the logic they contain.
Expression trees are also used extensively in the Dynamic Language Runtime, where they represent the code which should execute when a dynamic expression is evaluated. Expression trees are well suited for this as they can be composed and broken down again, and after they're compiled the resulting IL is JIT-compiled as normal.
Most developers will never need to mess with the expression tree API, although it has a few other uses.

Aside from LINQ, another very simple use case is to extract both the name and the value of a property. I use this in a fluent API for validating data transfer objects. It's safer to pass one lambda parameter to define both name and value rather than have a second string parameter for the name, and run the risk of developers getting it wrong.
Here's an example (minus all the safety checks and other housekeeping):
public Validator<T> Check<T>(Expression<Func<T>> expr) {
// Analyse the expression as data
string name = ((MemberExpression) expr.Body).Member.Name;
// Compile and execute it to get the value
T value = (expr.Compile())();
return new Validator<T>(name, value);
}
Example of use:
Check(() => x.Name).NotNull.MinLength(1);
Check(() => x.Age).GreaterThan(18);

I used expression trees to make a null-safe evaluator:
string name = myObject.NullSafeEval(x => x.Foo.GetBar(42).Baz.Name, "Default");
This methods analyzes and rewrites the expression tree to insert null checks before each property or method call along the "path" to Name. If a null is encountered along the way, the default value is returned.
See implementation here
Expression trees are also commonly used to avoid referring to a property by hard-coding its name in a string:
private string _foo;
public string Foo
{
get { return _foo; }
set
{
_foo = value;
OnPropertyChanged(() => Foo);
// Rather than:
// OnPropertyChanged("Foo");
}
}
static string GetPropertyName<T>(Expression<Func<T>> expr)
{
var memberExpr = expr.Body as MemberExpression;
if (memberExpr == null)
throw new ArgumentException("expr", "The expression body must be a member expression");
return memberExpr.Member.Name;
}
protected void OnPropertyChanged<T>(Expression<Func<T>> expr)
{
OnPropertyChanged(GetPropertyName(expr));
}
This enables compile time checking and name refactoring