I have this LINQ Expression that finds all the historical changes to a given customer's CreditBalance:
var history = GetHistory(id);
var changes = history.Where(x => history.Where(y => y.AuditId < x.AuditId)
.OrderByDescending(y => y.AuditId)
.Select(y => y.CreditBalance)
.FirstOrDefault() != x.CreditBalance);
This function works as expected. What I want to do is change this function to allow the user to query changes to any historical field. The way I chose to tackle this was with expression trees.
So far I have come up with this solution:
var history = GetHistory(id);
var c = Expression.Parameter(typeof(Customer_history), "c");
var d = Expression.Parameter(typeof(Customer_history), "d");
var cAudit = Expression.Property(c, typeof(Customer_history).GetProperty("AuditId"));
var dAudit = Expression.Property(d, typeof(Customer_history).GetProperty("AuditId"));
var whereBody = Expression.LessThan(dAudit, cAudit);
var whereLambda = Expression.Lambda(whereBody, d);
var where = Methods.QueryableWhere.MakeGenericMethod(typeof(Customer_history));
var whereCall = Expression.Call(null, where, **Expression.Constant(history)**, whereLambda);
var orderByLambda = Expression.Lambda(dAudit, d);
var orderBy = Methods.QueryableOrderByDescending.MakeGenericMethod(typeof(Customer_history), orderByLambda.Body.Type);
var orderByCall = Expression.Call(null, orderBy, whereCall, orderByLambda);
var dProp = Expression.Property(d, typeof(Customer_history).GetProperty(field));
var selectLambda = Expression.Lambda(dProp, d);
var select = Methods.QueryableSelect.MakeGenericMethod(typeof(Customer_history), selectLambda.Body.Type);
var selectCall = Expression.Call(null, select, orderByCall, selectLambda);
var firstOrDefault = Methods.QueryableFirstOrDefault.MakeGenericMethod(selectLambda.Body.Type);
var firstOrDefaultCall = Expression.Call(null, firstOrDefault, selectCall);
var cProp = Expression.Property(c, typeof(Customer_history).GetProperty(field));
var comparison = Expression.NotEqual(firstOrDefaultCall, cProp);
var lambda = Expression.Lambda<Func<Customer_history, bool>>(comparison, c);
var changes = history.Where(lambda);
The problem is, I get this exception when the query is executed:
Unable to create constant value of type
'Namespace.Customer_history'. Only primitive types or enumeration
types are supported in this context.
Now I am assuming that the issue is the Expression.Constant(history) statement based on the exception message. The problem is, I don't know how to rewrite it to allow the query provider to handle it appropriately. I know it works because of the original query, I just don't know how to do it in an expression tree.
Can anyone provide any direction?
As suspected, it seems that a ConstantExpression is not the way to obtain the value from a local variable.
What I needed to do was create a private class to store the variable in, and then I was able to access it with a field MemberExpression
private class ValueHolder<T>
{
public IQueryable<T> History;
}
Then in my method I was able to have the expression evaluated using this:
var valueHolder = new ValueHolder<T>
{
History = data
};
var c = Expression.Parameter(typeof(T), "c");
var constantEx = Expression.Constant(valueHolder);
var fieldEx = Expression.Field(constantEx, valueHolder.GetType().GetField("History"));
You can always try using dynamic linq, which allows you to use strings as expressions instead of lambda.
Example:
var query = history.Where("MyField = MyFilter");
https://www.nuget.org/packages/System.Linq.Dynamic.Library/
https://github.com/NArnott/System.Linq.Dynamic
Related
I'm trying to convert the following lambda expression into an Expression tree
var datatable = "<DataTable retrieved and filled with SQLCommand>"
var aggregate = datatable.AsEnumerable().GroupBy(x =>
new {
MeasureCode = x.Field<string>("CODMEASURE"),
FieldName = x.Field<string>("FIELDNAME"),
Formula = x.Field<string>("FORMULA"),
TableCode = x.Field<string>("CODTAB")})
.Select(x => new
{
Organization = String.Join(",",x.Select(z=>z.CODDIV)),
x.Key.MeasureCode,
x.Key.FieldName,
x.Key.Formula,
x.Key.TableCode
});
and thanks to various articles I've been able to write the following code that performs the first group by
.GroupBy(x =>
new {
MeasureCode = x.Field<string>("CODMEASURE"),
FieldName = x.Field<string>("FIELDNAME"),
Formula = x.Field<string>("FORMULA"),
TableCode = x.Field<string>("CODTAB")})
Below the code that builds the expression tree:
1. Declare the "x" input parameter
var dataRowParameter = Expression.Parameter(typeof(DataRow), "x");
var dataRowType = typeof(DataRow);
2. Create an anonymous type with one property for each datatable columns using the TypeBuilder class
Dictionary<string, Expression> GroupByFields = new Dictionary<string, Expression>();
var helper = new ReflectionHelper();
var myTypeBuilder = helper.GetTypeBuilder();
foreach (DataColumn column in table.Columns)
{
if (column.ColumnName.Equals("CODDIV")) continue;
// Add the column as dynamic object property
myTypeBuilder.DefineField(column.ColumnName, column.DataType, FieldAttributes.Public);
3. While looping through the columns I also create the Expression for accessing the DataRow's Field method like x.Field("columnName")
// Get the expression for dynamically call the DataRow field method
MethodInfo mi = typeof(DataRowExtensions).GetMethod("Field", new Type[] { typeof(DataRow), typeof(string) });
mi = mi.MakeGenericMethod(column.DataType);
var columnName = Expression.Constant(column.ColumnName, typeof(string));
var valueGetter = Expression.Call(mi, new Expression[] { dataRowParameter, columnName });
//Add method definition to the list of fields
GroupByFields.Add(column.ColumnName, valueGetter);
}
// Create the type and assign it to a "new" expression
var dynamicType = myTypeBuilder.CreateType();
4. When I get all the columns, I create the association between the anonymous type property and the DataRowField
var bindingArray = dynamicType.GetFields()
.Select(p => Expression.Bind(p, GroupByFields[p.Name]))
.OfType<MemberBinding>()
.ToArray();
var GroupedRow = Expression.New(dynamicType.GetConstructor(Type.EmptyTypes));
var memberInit = Expression.Convert(Expression.MemberInit(GroupedRow, bindingArray),typeof(object));
var predicate = Expression.Lambda<Func<DataRow, object>>(memberInit, dataRowParameter);
5. Then to test the result I do the GroupBy using lambda from my Expression tree and one written as normal lambda expression.
var groupByResult = table.AsEnumerable().AsQueryable().GroupBy(predicate);
var aggregate = table.AsEnumerable().AsQueryable().GroupBy(x =>
new
{
MeasureCode = x.Field<string>("CODMEASURE")
});
/*.Select(x => new
{
Organization = String.Join(",",x.Select(z=>z.Field<string>("CODDIV"))),
});*/
6. The correctResult is the count from the groupby using the LINQ Lambda Expression and is giving the right result, the result variable instead is returning the wrong result the is simply all the record without grouping
var correctresult = aggregate.ToArray().Count();
var result = groupByResult.ToArray().Count();
return table;
I've tried to use the debug view to understand the difference between the two lambda but i see only a small difference.
The one returning the correct result is
.Call System.Linq.Queryable.GroupBy(
.Constant<System.Linq.EnumerableQuery`1[System.Data.DataRow]>(System.Data.EnumerableRowCollection`1[System.Data.DataRow]),
'(.Lambda #Lambda1<System.Func`2[System.Data.DataRow,<>f__AnonymousType2`4[System.String]]>))
.Lambda #Lambda1<System.Func`2[System.Data.DataRow,<>f__AnonymousType2`4[System.String,System.String,System.String,System.String]]>(System.Data.DataRow $x)
{
.New <>f__AnonymousType2`4[System.String](
.Call System.Data.DataRowExtensions.Field(
$x,
"CODMEASURE"),
)
}
The one NOT GROUPING is
.Call System.Linq.Queryable.GroupBy(
.Constant<System.Linq.EnumerableQuery`1[System.Data.DataRow]>(System.Data.EnumerableRowCollection`1[System.Data.DataRow]),
'(.Lambda #Lambda1<System.Func`2[System.Data.DataRow,System.Object]>))
.Lambda #Lambda1<System.Func`2[System.Data.DataRow,System.Object]>(System.Data.DataRow $x) {
(System.Object).New 13d459a0-b85d-4c0d-8ab1-8f0e9af36ad7(){
CODMEASURE = .Call System.Data.DataRowExtensions.Field(
$x,
"CODMEASURE")
}
}
So I suppose that the reason for which is not grouping is because the one built with expression tree is using a System.Object instead of an anonymous type with the right properties mapped
But I don't know how to make it using an anomymous type, any help would be appreciated.
Thanks
I'm in a situation where I need to specify queries on a EF-context at runtime. Our consultants configure these queries at a customer site for customer specific situations.
In order to facilitate that I was thinking of using linq to build the queries, based on a criteria-list which the consultants specify in a front-end of some kind (right now, winforms). The consultants basically specify a property from an object, specify the operator and then the value. For example: give me all clients where the [status] [equals] [1].
At the moment I have an expression builder which creates the where clause at runtime, and so far I can manage one group by clause. Where I'm running against the wall here, is when a consultant configures multiple group by clauses of different types (f.e. a string and a datetime property).
For example, I must be able to process this query: select bsn as a, dateofbirth as b from clients where status = 1 group by bsn, dateofbirth (where bsn = string and dateofbirth = datetime).
At the moment, this is the code which "glues" the query together:
public List<ClientV2> ExportClients(List<CriteriaV2> criteriaList)
{
var whereExpression = BuildWhereExpressionChain(criteriaList.Where(c => c.Operator != CriteriaOperatorV2.GROUPBY).ToList());
var groupByExpression = BuildGroupByExpression(criteriaList.Where(c => c.Operator == CriteriaOperatorV2.GROUPBY).ToList());
var sourceClients = _context.Clients.Where(whereExpression).GroupBy(groupByExpression).ToList();
IEnumerable<Client> resultClients = sourceClients.SelectMany(group => group);
return ClientToClientV2.MapList(resultClients.ToList());
}
This is the where-clause builder:
private Expression<Func<Client, bool>> BuildWhereExpressionChain(List<CriteriaV2> criteriaList)
{
var expressionList = new List<Expression<Func<Client, bool>>>();
var paramExp = Expression.Parameter(typeof(Client));
foreach (var crit in criteriaList)
{
var propertyItem = PropertyTranslator.GetPropertyItem(crit.Property);
if (propertyItem == null) throw new InvalidFilterCriteriaException("Property " + crit.Property + " niet toegestaan als filter criterium");
var propInfo = typeof(Client).GetProperty(propertyItem.InternalName);
var left = Expression.Property(paramExp, propInfo);
Expression right;
if (propInfo.PropertyType.IsEnum)
right = Expression.Constant(Enum.ToObject(propInfo.PropertyType, PropertyTranslator.TranslateEnum(propertyItem.Type, crit.Value)));
else if (propInfo.PropertyType == typeof(DateTime) || propInfo.PropertyType == typeof(DateTime?))
right = Expression.Constant(DateTime.Parse(crit.Value), propInfo.PropertyType);
else
right = Expression.Constant(crit.Value, typeof(string));
var exp = BuildExpression(left, right, crit.Operator);
expressionList.Add(Expression.Lambda<Func<Client, bool>>(exp, new ParameterExpression[] { paramExp }));
}
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.And(ex); });
return firstExpression;
}
And this is the part where I am stuck (it does work for one clause of type string):
private Expression<Func<Client, string>> BuildGroupByExpression(List<CriteriaV2> criteriaList)
{
var expressionList = new List<Expression<Func<Client, string>>>();
var paramExp = Expression.Parameter(typeof(Client));
foreach (var crit in criteriaList)
{
var propertyItem = PropertyTranslator.GetPropertyItem(crit.Property);
if (propertyItem == null) throw new InvalidFilterCriteriaException("Property " + crit.Property + " niet toegestaan als group by criterium");
var propInfo = typeof(Client).GetProperty(propertyItem.InternalName);
var body = Expression.Property(paramExp, propInfo);
var lambda = Expression.Lambda<Func<Client, string>>(body, paramExp);
expressionList.Add(lambda);
}
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.And(ex); });
return firstExpression;
}
Would it be possible to make the BuildGroupByExpression() in such way that it results in an expression which contains multiple clauses of different types that I can use directly in .GroupBy(expression);?
I'm not that much of an expert on linq, but I have a feeling that what I want could be possible. If I do stupid things here, please point them out and I'll work on it.
I dont think so. At least using the approach where the Expression is built they way you have. At least I ended up building expressions per type.
The main reason is
var lambda = Expression.Lambda<Func<Client, string>>(body, paramExp);
I dont know how you can make this Lambda defintion dynamic or generic.
There is a different approach, and a library that uses a string interpretation at runtime to build expressions. See
Install-Package System.Linq.Dynamic.Library
see also https://msdn.microsoft.com/en-US/vstudio/bb894665.aspx
I'm attempting to build an expression tree to perform a LINQ to Entities query in .NET 4.0 with EF4. When I attempt to execute the query I've built, I get a NotSupportedException with the following message:
LINQ to Entities does not recognize the method
'System.Data.Objects.ObjectQuery`1[TestWpf.Customer]
Where(System.String, System.Data.Objects.ObjectParameter[])' method,
and this method cannot be translated into a store expression.
I'm querying against the Northwind database. My entities were generated from the database. In my code below, I have the query I'm trying to build in the method GetQuery1(), and I'm attempting to build it in the GetQuery2() method.
If I set a breakpoint and inspect the query1 variable, it's Expression property is:
Convert(value(System.Data.Objects.ObjectSet`1[TestWpf.Customer])).MergeAs(AppendOnly).Where(c => c.CompanyName.Contains("z"))
What is this Convert().MergeAs(AppendOnly) doing? I attempted to search on MSDN but couldn't locate what I need (at least I don't think I could find it...). Additionally, what am I doing wrong?
I think that perhaps I'm calling an incorrect Where() method, as Intellisense says there is another, which is an extension method. I have not tried to update the whereMethod variable to fetch that one, but I'm not sure how, either.
private static IQueryable<Customer> GetQuery1(NorthEntities context) {
return context.Customers.Where(c => c.CompanyName.Contains("z"));
}
private static IQueryable<Customer> GetQuery2(NorthEntities context) {
var custParam = Expression.Parameter(typeof(Customer), "c");
var custCollection = Expression.Constant(context.Customers);
var companyNamePropValue = Expression.Property(custParam, typeof(Customer).GetProperty("CompanyName"));
var containsParameter = Expression.Constant("z");
var containsMethod = Expression.Call(companyNamePropValue, typeof(string).GetMethod("Contains"), containsParameter);
var whereMethod = context.Customers.GetType().GetMethod("Where", new Type[] { typeof(string), typeof(ObjectParameter[]) });
var param2 = Expression.Constant(new ObjectParameter[] { });
var where = Expression.Call(custCollection, whereMethod, companyNamePropValue, param2);
return ((IQueryable<Customer>)context.Customers).Provider.CreateQuery<Customer>(where);
}
private static void Main(string[] args) {
using (var context = new NorthEntities()) {
var query1 = GetQuery1(context);
var query2 = GetQuery2(context);
foreach (var c in query1)
Console.WriteLine(c.CompanyName);
foreach (var c in query2)
Console.WriteLine(c.CompanyName);
}
Console.ReadLine();
}
To construct the specific query you're working with, try the following:
private static IQueryable<Customer> GetQuery2(NorthEntities context) {
IQueryable<Customer> customers = context.Customers;
var custParam = Expression.Parameter(typeof(Customer), "c");
var companyNamePropValue = Expression.Property(custParam, typeof(Customer).GetProperty("CompanyName"));
var containsParameter = Expression.Constant("z");
var containsCall = Expression.Call(companyNamePropValue, typeof(string).GetMethod("Contains"), containsParameter);
var wherePredicate = Expression.Lambda<Func<Customer, bool>>(containsCall, custParam);
return customers.Where(wherePredicate);
}
In general, to get access to the LINQ extension methods (e. g. Where), you'll have to look in the Queryable class:
var genericWhereMethod = typeof(Queryable).GetMethods()
.Single(m => m.Name == "Where"
// distinguishes between Where((T, int) => bool) and Where(T => bool)
&& m.GetParameters()[1].ParameterType
.GetGenericArguments()[0].GetGenericTypeDefinition() == typeof(Func<,>));
// make the Where method that applies to IQueryable<Customer>
var whereMethod = genericWhereMethod.MakeGenericMethod(typeof(Customer));
For a certain column name I want to calculate the SUM. I just found out how to use reflection but don't know if i'm doing it the right way..
string columnName = "days";
ObjectQuery<TableX> objectSet = context.CreateObjectSet<TableX>();
Func<TDomainEntity, object> fieldGetter;
var type = typeof(TDomainEntity);
var prop = type.GetProperty(columnName);
fieldGetter = e => prop.GetValue(e,null);
//this wouldn't work because the fieldGetter should be Func<TDomainEntity, decimal> and not object
var total = objectSet.Sum(fieldGetter);
I can't get this working because the fieldgetter should be of the type decimal but the the type.GetProperty will fail and casting it didn't work. So what am I missing or is there another way?
I could make a giant switch statement for the columnname but that's not nice is it :)
If you know the type should be decimal, then tell that to the getter:
Func<TDomainEntity, decimal> fieldGetter = e => (decimal)prop.GetValue(e,null);
Which should then work with LINQ-to-Objects as:
var total = objectSet.Sum(fieldGetter);
Note that if you intend to use LINQ-to-EF, you'll have to construct an expression tree instead:
var p = Expression.Parameter(typeof(TDomainEntity));
var lambda = Expression.Lambda<Func<TDomainEntity, decimal>>(
Expression.PropertyOrField(p, columnName), p);
var total = query.Sum(lambda);
Of course, you can use that approach either way:
var p = Expression.Parameter(typeof(TDomainEntity));
var lambda = Expression.Lambda<Func<TDomainEntity, decimal>>(
Expression.PropertyOrField(p, columnName), p);
var total = objectSet.Sum(lambda.Compile());
I'll start with piece of code:
var objectType = typeof(Department); // Department is entity class from linqdatacontext
using (var dataContext = new DataModel.ModelDataContext())
{
var entity = Expression.Parameter(objectType, "model");
var keyValue = Expression.Property(entity, "Id");
var pkValue = Expression.Constant(reader.Value);
var cond = Expression.Equal(keyValue, pkValue);
var table = dataContext.GetTable(objectType);
... // and here i don't how to proceed
}
I am not even sure if i am building that expression correctly. However simply put, i need to call dynamically SingleOrDefault() on that table to find entity by primary key. Every example i had found is using generic variant of GetTable<>(), but i cannot use that obviously. I am probably overlooking something...
Whenever I build expression trees, I like to start off with an example of what I'm building:
() => dataContext.GetTable<TEntity>().SingleOrDefault(entity => entity.Id == 1);
From that, we can easily dissect the target expression. You are partway there; you just need to include a call to the GetTable method in the expression tree and then build an outer lambda expression to call the whole thing:
using(var dataContext = new DataModel.ModelDataContext())
{
var getTableCall = Expression.Call(
Expression.Constant(dataContext),
"GetTable",
new[] { entityType });
var entity = Expression.Parameter(entityType, "entity");
var idCheck = Expression.Equal(
Expression.Property(entity, "Id"),
Expression.Constant(reader.Value));
var idCheckLambda = Expression.Lambda(idCheck, entity);
var singleOrDefaultCall = Expression.Call(
typeof(Queryable),
"SingleOrDefault",
new[] { entityType },
getTableCall,
Expression.Quote(idCheckLambda));
var singleOrDefaultLambda = Expression.Lambda<Func<object>>(
Expression.Convert(singleOrDefaultCall, typeof(object)));
var singleOrDefaultFunction = singleOrDefaultLambda.Compile();
return singleOrDefaultFunction();
}
We have to convert the SingleOrDefault call to have a return type of object so it can serve as the body of the Func<object> function.
(Untested)
Edit: Parameterizing the data context and value
Now we are building this function:
(dataContext, value) => dataContext.GetTable<TEntity>().SingleOrDefault(entity => entity.Id == value);
You would change the constants to parameters and add those parameters to the function you compile:
var dataContextParameter = Expression.Parameter(typeof(ModelDataContext), "dataContext");
var valueParameter = Expression.Parameter(typeof(object), "value");
var getTableCall = Expression.Call(
dataContextParameter,
"GetTable",
new[] { entityType });
var entity = Expression.Parameter(entityType, "entity");
var idCheck = Expression.Equal(
Expression.Property(entity, "Id"),
valueParameter);
var idCheckLambda = Expression.Lambda(idCheck, entity);
var singleOrDefaultCall = Expression.Call(
typeof(Queryable),
"SingleOrDefault",
new[] { entityType },
getTableCall,
Expression.Quote(idCheckLambda));
var singleOrDefaultLambda =
Expression.Lambda<Func<ModelDataContext, object, object>>(
Expression.Convert(singleOrDefaultCall, typeof(object)),
dataContextParameter,
valueParameter);
var singleOrDefaultFunction = singleOrDefaultLambda.Compile();
// Usage
using(var dataContext = new DataModel.ModelDataContext())
{
return singleOrDefaultFunction(dataContext, reader.Value);
}
If you are using .NET 4, you could try casting your returned objects as dynamic, so you could then query like this.
using (var dataContext = new DataModel.ModelDataContext())
{
var entity = Expression.Parameter(objectType, "model");
var keyValue = Expression.Property(entity, "Id");
var pkValue = Expression.Constant(reader.Value);
var cond = Expression.Equal(keyValue, pkValue);
var table = dataContext.GetTable(objectType);
var result = table.Where(ent => ((dynamic)ent).SomeField == "SomeValue");
}
I'm still not entirely sure as to the whole of your problem (and I suspect the answer about dynamic is going to solve part of what will come up too). Still, just to answer:
Every example i had found is using generic variant of GetTable<>(), but i cannot use that obviously
For any T, Table<T> implements (among other interfaces) ITable<T> and ITable. The former is generically typed, the latter not.
The form GetTable<T>() returns such a Table<T>. However, the form GetTable(Type t) returns an ITable. Since ITable inherits from IQueryable you can query it. If you need to do something with that query that would normally require knowledge of the type (such as comparing on a given property) then dynamic as per the previous answer given by Steve Danner allows that to happen.
I'd do it using Reflection and the LINQ dynamic query library, personally.
You can get the list of all the keys for a table with dataContext.Mapping.GetMetaType(objectType).IdentityMembers, then access the data with something along the lines of dataContext.GetTable(objectType).Where(key.Name + "==#0", id).
Obviously, I left out a few steps in there - if you have multiple keys, you'll need to build a fuller predicate with a loop over .IdentityMembers, and if you always just have the one key, you can use .First() on it. I haven't tested it either, but it should be pretty close. It'd probably be 6-7 lines of code total - I can write it up (and test) if you need it.
Edit: The LINQ Dynamic Query Library can be downloaded from Microsoft at http://msdn.microsoft.com/en-us/vcsharp/bb894665.aspx - just include DynamicLINQ.cs in your project and you're good.