So, I'm getting the error that's in the title. I'll jump straight into the relevant code. First, the manual creation of the where statement (which does work) and then the dynamic creation which does not work. Both create the same queryExpression.
First, the queryExpression that is generated with both methods:
{Table(Products).Select
(s => new SalesData() {
ProductID = s.ProductID,
StoreName = s.StoreName,
Count = s.Count
}).Where(s => (s.Count > 500))}
Now, the manual method that does work:
IQueryable<SalesData> data = ( from s in sales.Products
select new SalesData
{
ProductID = s.ProductID,
StoreName = s.StoreName,
Count = s.Count
} ).AsQueryable<SalesData>();
data = data.Where(s => s.Count > 500);
this.dataGridView1.DataSource = null;
this.dataGridView1.DataSource = (from d in data
select new
{
d.ProductID,
d.StoreName,
d.Count
} );
This works as expected; my DataGridView is populated with the data.
Now, to create the where clause dynamically:
IQueryable<SalesData> data = ( from s in sales.Products
select new SalesData
{
ProductID = s.ProductID,
StoreName = s.StoreName,
Count = s.Count
} ).AsQueryable<SalesData>();
if (this.filter.Predicate != null)
{
Expression<Func<SalesData, bool>> lambda =
Expression.Lambda<Func<SalesData, bool>>(this.filter.Predicate,
new ParameterExpression[] { this.filter.PE });
data = data.Where(lambda);
}
this.dataGridView1.DataSource = null;
this.dataGridView1.DataSource = (from d in data <---- fails here
select new
{
d.ProductID,
d.StoreName,
d.Count
} );
The only thing that is different is that in the second snippet I'm creating the lambda expression dynamically. I've noted where it fails in the second snippet.
Using the debugger I can see the queryExpression for data and it is the same with both methods, so I don't think my problem is with my actual expression creation. If that code is needed I can post it here.
My question is, what am I doing wrong and how do I fix it?
Edit:
Here is the function that gives filter.Predicate it's value:
public static Expression GetPredicate(List<FilterItem> itemList, ParameterExpression pe)
{
List<Expression> expressions = new List<Expression>();
List<string> combiners = new List<string>();
foreach (FilterItem item in itemList)
{
Expression left = Expression.PropertyOrField(pe, item.Field);
Expression right = Expression.Constant(Convert.ToInt32(item.Value), typeof(int));
expressions.Add(Expression.GreaterThan(left, right));
combiners.Add(item.Combiner);
}
int expressionCount = expressions.Count();
Expression predicateBody = expressions[0];
if (expressionCount > 1)
{
for (int x = 1; x <= expressionCount; x++)
{
switch (combiners[x - 1])
{
case "AND":
predicateBody = Expression.And(predicateBody, expressions[x]);
break;
case "OR":
predicateBody = Expression.Or(predicateBody, expressions[x]);
break;
default:
break;
}
}
}
return predicateBody;
}
Inside filter I have:
this.Predicate = BuildPredicate.GetPredicate(this.filterList, this.PE);
this.PE is:
public ParameterExpression PE
{
get
{
return Expression.Parameter(typeof(SalesData), "s");
}
}
From the father of C#, Anders Hejlsberg:
Parameters are referenced in expressions through their object identity, not by comparing their names. In fact, from an expression tree's point of view, the name of a parameter is purely informational. The reason for this design is the same reason that types are referenced through their System.Type objects and not their names--expression trees are fully bound and are not in the business of implementing name lookup rules (which may differ from language to language).
From MSDN Forums
The short and long of it is that filter.Predicate has a value of {(s.Count > 500)} but this doesn't mean that s has meaning here by sharing a name with the initial LINQ expression.
I was able to solve my problem thanks to the dialog on this page and using Predicate Builder.
A better method for what I'm trying to do can be found at Scott Gu's site. This is better for my purposes because I need to determine the field and the value dynamically, plus allow for grouping (easier with this method).
Related
I have a request query using linq. The query has multiple where clause say return list of items matching name and city.
Below is the piece of code I used for multiple where clause, but it returns empty set of items.
wherefield contains list of field names like name;city
wherefieldValue contains list of field values like james;delhi
var where = FilterLinq<T>.GetWherePredicate(wherefield, wherefieldvalue).Compile();
items = items.Where(where).OrderByDescending(a => a.GetType().GetProperty(field).GetValue(a, null)).Skip
public class FilterLinq<T>
{
public static Expression<Func<T, Boolean>> GetWherePredicate(string whereFieldList, string whereFieldValues)
{
//the 'IN' parameter for expression ie T=> condition
ParameterExpression pe = Expression.Parameter(typeof(T), typeof(T).Name);
//combine them with and 1=1 Like no expression
Expression combined = null;
if (whereFieldList != null)
{
string[] field = whereFieldList.Split(';');
string[] fieldValue = whereFieldValues.Split(';');
for (int i = 0; i < field.Count(); i++)
{
//Expression for accessing Fields name property
Expression columnNameProperty = Expression.Property(pe, field[i]);
//the name constant to match
Expression columnValue = Expression.Constant(fieldValue[i]);
//the first expression: PatientantLastName = ?
Expression e1 = Expression.Equal(columnNameProperty, columnValue);
if (combined == null)
{
combined = e1;
}
else
{
combined = Expression.And(combined, e1);
}
}
}
//create and return the predicate
return Expression.Lambda<Func<T, Boolean>>(combined, new ParameterExpression[] { pe });
}
}
Your example works. Sure it works only for string properties, but i assume, that is your use case.
Perhaps it doesn't do what you want to achieve, becasue you combine your clause with and, but actually you do want to do it with Or, jus change this code line :
combined = Expression.And(combined, e1);
To
combined = Expression.Or(combined, e1);
I'm in a situation where I need to specify queries on a EF-context at runtime. Our consultants configure these queries at a customer site for customer specific situations.
In order to facilitate that I was thinking of using linq to build the queries, based on a criteria-list which the consultants specify in a front-end of some kind (right now, winforms). The consultants basically specify a property from an object, specify the operator and then the value. For example: give me all clients where the [status] [equals] [1].
At the moment I have an expression builder which creates the where clause at runtime, and so far I can manage one group by clause. Where I'm running against the wall here, is when a consultant configures multiple group by clauses of different types (f.e. a string and a datetime property).
For example, I must be able to process this query: select bsn as a, dateofbirth as b from clients where status = 1 group by bsn, dateofbirth (where bsn = string and dateofbirth = datetime).
At the moment, this is the code which "glues" the query together:
public List<ClientV2> ExportClients(List<CriteriaV2> criteriaList)
{
var whereExpression = BuildWhereExpressionChain(criteriaList.Where(c => c.Operator != CriteriaOperatorV2.GROUPBY).ToList());
var groupByExpression = BuildGroupByExpression(criteriaList.Where(c => c.Operator == CriteriaOperatorV2.GROUPBY).ToList());
var sourceClients = _context.Clients.Where(whereExpression).GroupBy(groupByExpression).ToList();
IEnumerable<Client> resultClients = sourceClients.SelectMany(group => group);
return ClientToClientV2.MapList(resultClients.ToList());
}
This is the where-clause builder:
private Expression<Func<Client, bool>> BuildWhereExpressionChain(List<CriteriaV2> criteriaList)
{
var expressionList = new List<Expression<Func<Client, bool>>>();
var paramExp = Expression.Parameter(typeof(Client));
foreach (var crit in criteriaList)
{
var propertyItem = PropertyTranslator.GetPropertyItem(crit.Property);
if (propertyItem == null) throw new InvalidFilterCriteriaException("Property " + crit.Property + " niet toegestaan als filter criterium");
var propInfo = typeof(Client).GetProperty(propertyItem.InternalName);
var left = Expression.Property(paramExp, propInfo);
Expression right;
if (propInfo.PropertyType.IsEnum)
right = Expression.Constant(Enum.ToObject(propInfo.PropertyType, PropertyTranslator.TranslateEnum(propertyItem.Type, crit.Value)));
else if (propInfo.PropertyType == typeof(DateTime) || propInfo.PropertyType == typeof(DateTime?))
right = Expression.Constant(DateTime.Parse(crit.Value), propInfo.PropertyType);
else
right = Expression.Constant(crit.Value, typeof(string));
var exp = BuildExpression(left, right, crit.Operator);
expressionList.Add(Expression.Lambda<Func<Client, bool>>(exp, new ParameterExpression[] { paramExp }));
}
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.And(ex); });
return firstExpression;
}
And this is the part where I am stuck (it does work for one clause of type string):
private Expression<Func<Client, string>> BuildGroupByExpression(List<CriteriaV2> criteriaList)
{
var expressionList = new List<Expression<Func<Client, string>>>();
var paramExp = Expression.Parameter(typeof(Client));
foreach (var crit in criteriaList)
{
var propertyItem = PropertyTranslator.GetPropertyItem(crit.Property);
if (propertyItem == null) throw new InvalidFilterCriteriaException("Property " + crit.Property + " niet toegestaan als group by criterium");
var propInfo = typeof(Client).GetProperty(propertyItem.InternalName);
var body = Expression.Property(paramExp, propInfo);
var lambda = Expression.Lambda<Func<Client, string>>(body, paramExp);
expressionList.Add(lambda);
}
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.And(ex); });
return firstExpression;
}
Would it be possible to make the BuildGroupByExpression() in such way that it results in an expression which contains multiple clauses of different types that I can use directly in .GroupBy(expression);?
I'm not that much of an expert on linq, but I have a feeling that what I want could be possible. If I do stupid things here, please point them out and I'll work on it.
I dont think so. At least using the approach where the Expression is built they way you have. At least I ended up building expressions per type.
The main reason is
var lambda = Expression.Lambda<Func<Client, string>>(body, paramExp);
I dont know how you can make this Lambda defintion dynamic or generic.
There is a different approach, and a library that uses a string interpretation at runtime to build expressions. See
Install-Package System.Linq.Dynamic.Library
see also https://msdn.microsoft.com/en-US/vstudio/bb894665.aspx
I was looking at a simple rule engine http://netmatze.wordpress.com/2012/01/22/building-a-rule-engine-in-c/ and I'm doing something very similar to this. I have two classes that look like:
class A
{
public List<B> ListB { get; set; }
}
Class B
{
public int ID { get; set; }
}
With my rule set looking like:
List<Rule> rules = new List<Rule>{
new Rule("listB", ExpressionType.Loop, 1, "ID")
};
I'm trying to build the expression to basically look at class A property listB, loop it camparing each item's ID property to see if at least one equals 1. I'm having trouble on how to do this. I currently have something like (I have hard coded values set in this, but it will eventually be changed to be generic as much as possible). This expression does not work, I get compile exceptions:
var parameterExpression = Expression.Parameter(typeof(A));
var listB = MemberExpression.Property(parameterExpression, "ListB");
var leftOperand = MemberExpression.Property(Expression.Parameter(typeof(B)), "ID");
var rightOperand = Expression.Constant(1); //1
var found = Expression.Variable(typeof(bool), "found");
return Expression.Lambda<Func<T, bool>>(
Expression.Block(
listB,
found,
Expression.Loop(
Expression.Block(
Expression.IfThen(
Expression.Equal(
leftOperand,
rightOperand
),//equal
Expression.Assign(
found,
Expression.Constant(true)
)//set to true
)
)//block
)//loop
),
A
).Compile();
I'll end up calling the rule set against my object like so:
Engine ruleEngine = new Engine();
var compiledRules = rules.Select(r => ruleEngine.CompileRule<A>(r)).ToList();
var result = compiledRules.All(rule => rule(objA));
My questions are:
How do I get this function to return true/false if any of the list
items matched the condition.
How do you prevent the Expression.Loop
to stop looping once all list items are compared (and none of them
matched)?
Thanks for any help.
Why use a loop? You wouldn't use a loop if you were coding the check in C#. You'd use Enumerable.Any. So generate the following expression:
A a;
return a.ListB.Any(b => b.ID == 1);
This is translated to:
A a;
return Enumerable.Any(a.ListB, b => b.ID == 1);
This is easily translatable to expression trees.
Following your last comment it sounds like you could use the approach i suggested for another question. Replace this part:
var childProperty = parameter.Type.GetProperty(properties[0]);
var left = Expression.Property(parameter, childProperty);
var right = Expression.Constant(test, typeof(int));
navigationPropertyPredicate = Expression.Equal(left, right);
resultExpression = MakeLambda(parameter, navigationPropertyPredicate);
with something using your ruleOperator and value
I have a method in my data layer in which I pass in a set of search parameters, dynamically build up the required 'where' clause using the PredicateBuilder class from LINQKit and then execute the LINQ query to return a list of objects.
In some cases the search criteria may contain a category name which doesn't exist in the Library table, I then need to join to this table to perform the necessary query. The reason it's been done this way is because there are potentially hundreds of categories per book and for optimisation reasons I only want the join to be performed if actually required.
Now my question is, is it possible to dynamically append this join to the LINQ query?
After a few attempts at trying to get this to work I've unfortunately had to resort to the 'cut-and-paste' pattern which I dislike immensely but needed something that worked for it to get 'out-the-door'.
The code below is an extract with variables renamed of what I've currently got (i.e. this isn't really a library application!):
public IEnumerable<ILibrarySearchResultsDTO> SearchLibrary(ISearchLibrary searchValues)
{
var whereStatement = PredicateBuilder.True<Library>();
bool categorySearch = false;
if (!string.IsNullOrEmpty(searchValues.AuthorFirstName))
whereStatement = whereStatement.And(q => q.AuthorFirstName == searchValues.AuthorFirstName);
if (!string.IsNullOrEmpty(searchValues.AuthorLastName))
whereStatement = whereStatement.And(q => q.AuthorLastName == searchValues.AuthorLastName);
if (!string.IsNullOrEmpty(searchValues.CategoryName))
categorySearch = true;
var libraryObjectSet = Context.CreateObjectSet<Library>();
LibraryObjectSet.MergeOption = MergeOption.NoTracking;
var categoriesObjectSet = Context.CreateObjectSet<Categories>();
categoriesObjectSet.MergeOption = MergeOption.NoTracking;
if (!categorySearch)
{
var query = from lib in libraryObjectSet
.Where(whereStatement)
.Take(ConfigHelper.MaxQueryRecords)
.AsExpandable()
select new LibrarySearchResultsDTO()
{
BookName = lib.BookName,
AuthorFirstName = lib.AuthorFirstName,
AuthorLastName = lib.AuthorLastName,
ISBN = lib.ISBN
};
}
else
{
var query = from lib in LibraryObjectSet
.Where(whereStatement)
.Take(ConfigHelper.MaxQueryRecords)
.AsExpandable()
join categories_LKP in categoriesObjectSet on new { CategoryID = lib.CategoryID, CategoryName = searchValues.CategoryName }
equals new { CategoryID = categories_LKP.CategoryID, CategoryName = categories_LKP.CategoryName }
select new LibrarySearchResultsDTO()
{
BookName = lib.BookName,
AuthorFirstName = lib.AuthorFirstName,
AuthorLastName = lib.AuthorLastName,
ISBN = lib.ISBN
};
}
return query.ToList();
}
I've had to create the sample code in Notepad++ and because it's a contrived example I haven't been able to check if it compiles. Should do though (I hope!).
If there is a navigation property from Library to Category you can just dynamically add another predicate:
if (!string.IsNullOrEmpty(searchValues.CategoryName))
{
whereStatement = whereStatement
.And(q => q.Categories
.Any(c => c.CategoryName == searchValues.CategoryName));
}
If the navigation property is not present, you can still use a join in the predicate without having to duplicate the whole query. But it could be a good reason to add a navigation property.
You may use Reflection API like a following generic function...which compiles a dynamic query with a unknown type...
IQueryable<T> getQuery<T>(T myTableEntity, string[] arrayOfQueryTerms, Expression<Func<T, bool>> predicate)
{
var fieldOrProperty = getMemberInfo(predicate);
}
MemberInfo getmemberInfo<T>(Expression<Func<T,bool> expr)
{
var memberExpr = expr as MemberExpression;
if (memberExpr != null) return memberExpr.Member;
throw new ArgumentException();
}
var q = getQuery<FooTable>(foo, new[]{"Bar","Baz"}, x=>x.FieldName);
How do I write a dynamic query for Linq, if I have say Customer class which holds the fields:
string name
string address
int phoneno
I have to query based on information given similar to
query = string.Empty;
if(!string.IsNullorEmpty(name))
{
query += "#name = name";
}
if(!string.IsNullorEmpty(address))
{
query += "#address = address";
}
if(!string.IsNullorEmpty(phoneno))
{
query += "#phoneno = phoneno";
}
var result = from condition in customer
where(query)
select condition;
Edit #1:
the items are changeable at run time like
private Customer[] GetCustomers(Dictionary<string,string> attributes)
{
here the attribute may be, name alone, or name and address, or name address and phoneno
foreach(string field in attributes.key)
{
query += field == attributes[key];
}
Customers[] =ExecuteQuery(query);
}
Is this kind of query supported by LINQ?
Edit #2:
Hi Mouk,
As I am new to C#, I am still struggling, this is not working for me.
var query = _ConfigFile.ConnectionMasterSection;
for(int i = 0; i < filter.count; i++)
{
query = result.Where(p => typeof(ConnectionMaster).GetProperty(filter[i].Attribute).Name == filter[i].Value);
}
This yeilds Empty, where as i used this
var query = _ConfigFile.ConnectionMasterSection;
//Hard coded
res.Where(q => q.category == filter[0].Value);
And it worked as I expected.
Hi Bryan Watts,
I tried your code also and I getting this error: "Lambda Parameter not in scope".
for(int i = 0; i < filter.count; i++)
{
Field item = filter[i];
MemberExpression param = Expression.MakeMemberAccess(Expression.Parameter(typeof(Connection), "p"), typeof(Connection).GetProperty(item.Attribute));
MemberExpression constant = Expression.MakeMemberAccess(Expression.Constant(item), typeof(Field).GetProperty("Value"));
}
try
{
var myquery = Queryable.Where(coll, Expression.Lambda<Func<Connection, bool>>(
Expression.Equal(param, constant), Expression.Parameter(typeof(Connection),"p")));
}
What is the mistake here?
Check out this http://www.albahari.com/nutshell/predicatebuilder.aspx, it allows for strongly typed predicate building, it can be really nice. If you want actually dynamic string built predicates than you can use the LINQ Dynamic Query Library provided by ScottGu.
Both will accomplish what you want although I would recommend the first option before the second.
Allowing you to do:
var predicate = PredicateBuilder.True<MyLinqType>();
if(!string.IsNullOrEmpty(name))
predicate = predicate.And(p => p.name == name);
...
var myResults = Context.MyLinTypeQueryTable.Where(predicate);
And more.
Here you go:
var result = from customer in Customers
where string.IsNullOrEmpty(phoneNo) || customer.PhoneNo == phoneNo
where string.IsNullOrEmpty(address) || customer.Address == address
select customer;
If you're concerned that this generate the optimal SQL query underneath, as always you should attach a SQL Query Analyzer and check. But I believe the expression parser in Linq To Sql will collapse down the where clauses as appropriate based on the value of the arguments.
You can use the fluent interface and add a new Where clause fpr each condition. Something like:
var result = from cus in customers select cus;
if(!string.IsNullOrEmpty(name))
result= result.Where(p => p.Name == name);
EDIT upon hte comment:
if you are querying over a collection in memory, you could retrieve the properties using reflection.
private Customer[] GetCustomers(Dictionary<string,string> attributes)
{
var result = from cus in customers select cus;
foreach(string key in attributes.Keys)
result= result.Where(p => GetProperty(p, key )== attributes[key]);
return result.ToList();
}
Supposing GetProperty retrieve the property by reflection.
Using Linq2Sql this method will result in retrieving all record an then iterating over them using reflection.
I've had good experience with Dynamic LINQ.
I used it for a rich HTML table that could be filtered and sorted server side. The server receives a request containing a request parameter where the key is the name of the property (for example 'Lastname') and the value is the value that the property needs to be sorted on (for example 'Smith'). Using that information I built a query string that I passed to the Dynamic LINQ's Where method.
Roughly, you could think of something like the following:
public static IQueryable<T> Filter<T>(this IQueryable<T> query, Dictionary<string, string> dictionary)
{
Type t = typeof(T);
StringBuilder sb = new StringBuilder();
PropertyInfo[] properties = t.GetProperties();
foreach(string key in dictionary.Keys)
{
PropertyInfo property = properties.Where(p => p.Name == key).SingleOrDefault();
if(property != null)
{
if (sb.Length > 0) sb.Append(" && ");
string value = dictionary[key];
sb.Append(string.Format(#"{0}.ToString().Contains(""{1}"")", key, value));
}
}
if (sb.Length > 0)
return query.Where(sb.ToString());
else
return query;
}
The code is out of the top of my head and thus untested.
Of course, this is the most basic version: it does a simple string comparison. If you want to have numerical comparison (meaning you want for example the User where UserID is exactly 100, not where the UserID.ToString().Contains("100")), or query nested Properties (Customer.Company.CompanyAddress for example), or query Collections this gets more complicated. You should also think about security: while Dynamic LINQ is not vulnerable to SQL injection, you shouldn't let it blindly parse all user input.
It sounds like you need to dynamically compose queries.
See my answer to this question.
It explains how queries against an IQueryable<T> are composed by the compiler, and what you can do to add dynamic elements.
Edit
Here is an example of how you would dynamically build a set of Where conditions on top of an IQueryable<Customer>:
// This method ANDs equality expressions for each property, like so:
//
// customers.Where(c => c.Property1 == value1 && c.Property2 == value2 && ...);
private IQueryable<Customer> FilterQuery(IQueryable<Customer> customers, IDictionary<string, string> filter)
{
var parameter = Expression.Parameter(typeof(Customer), "c");
Expression filterExpression = null;
foreach(var filterItem in filter)
{
var property = typeof(Customer).GetProperty(filterItem.Key);
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
var equality = Expression.Equal(propertyAccess, Expression.Constant(filterItem.Value));
if(filterExpression == null)
{
filterExpression = equality;
}
else
{
filterExpression = Expression.And(filterExpression, equality);
}
}
if(filterExpression != null)
{
var whereBody = Expression.Lambda<Func<Customer, bool>>(filterExpression, parameter);
customers = customers.Where(whereBody);
}
return customers;
}