Develop Reference

Add new lambda expressions using Expression Tree

I have been looking at many posts here and on the web but none of them seem to be helping.
I have a table with about 2 million records, it has over 200 columns.
A simple web service allow the user to pull a specific number of columns out of the table, the user has the option to choose which column to pull.
The result needs to be string of comma separated values, so my query needs to pull the requested columns and return a concatenate string.
I have done this using ADO.NET and pure SQL queries it works fine however I was asked to do it in Entity Framework.
Here is what I have and have done already.
I get the requested columns list as an array of strings.
The following is my query, not sure if it's the best solution or idea hence I'm asking for help here.
var valueList2 = ctx.mytable.Where(x => x.pcds == comValue).Select(x => new{temp = x.column1 +", "+ x.column2}).Select(x => x.temp).ToList();
The above gives me string of two columns separated by commas, I just need to somehow push my array of column names into the lambda part of it.
I did the following but then realised that it only works with a specific type of a class not anonymous, also I can't figure out how I can use it for a multiple columns and not make it so complex.
var createdType = typeof(mytable);
var Param = Expression.Parameter(typeof(string), "pr");
var obj = Expression.New(createdType);
var ValueProperty = createdType.GetProperty("long");
var ValueAssignment = Expression.Bind(ValueProperty, Param);
var memberInit = Expression.MemberInit(obj, ValueAssignment);
var lm = Expression.Lambda<Func<string, mytable>>(memberInit, Param);
Thank you

I'm using Dynamic Linq (source code). Sadly there is little documentation about how to use it :-) In a fun boomerang effect, there is an "evolved" version. The boomerang effect is because the code for generating the dynamic class is based on one of my responses :-) The remaining code seems to be very beautiful... And there is a full suit of unit tests with code samples!!! Note that this second library is a superset of the first library, so you can probably apply many examples to the first one! :-)
I'm adding some static methods to translate the result of a a Dynamic Linq query to a IEnumerable<object[]>.... Example code:
using (var ctx = new Model1())
{
var result = ctx.MyTable
.Take(100)
.SimpleSelect(new[] { "ID", "Col1", "Col2" })
.ToObjectArray();
foreach (var row in result)
{
Console.WriteLine(string.Join(", ", row));
}
}
More complex example:
var columnsNames = new[] { "SomeNullableInt32", "SomeNonNullableDateTimeColumn" };
// One for each column!
var formatters = new Func<object, string>[]
{
x => x != null ? x.ToString() : null,
x => ((DateTime)x).ToShortDateString()
};
var result = ctx.MyTable.Take(100).SimpleSelect(columnsNames).ToObjectArray();
foreach (var row in result)
{
var stringRow = new string[row.Length];
for (int i = 0; i < row.Length; i++)
{
stringRow[i] = formatters[i](row[i]);
}
Console.WriteLine(string.Join(", ", stringRow));
}
And the classes... One (SimpleSelect) produces the Dynamic SQL Select, and "anonymizes" the field names. I do this because for each type of return the Dynamic Linq will generate at runtime a class. This class won't be unloaded until the program ends. By anonymizing the columns (I rename them to Item1, Item2, Item3...) I increase the possibility that the same class will be reused. Note that different type of columns will generate different classes! (int Item1, string Item2 will be a different class from int Item1, DateTime Item2), the other (ToObjectArray) returns a IEnumerable<object[]>, something easier to parse.
public static class DynamicLinqTools
{
private static ConcurrentDictionary<Type, Func<object, object[]>> Converters = new ConcurrentDictionary<Type, Func<object, object[]>>();
public static IQueryable SimpleSelect(this IQueryable query, string[] fields)
{
// With a little luck, "anonymizing" the field names we should
// reduce the number of types created!
// new (field1 as Item1, field2 as Item2)
return query.Select(string.Format("new ({0})", string.Join(", ", fields.Select((x, ix) => string.Format("{0} as Item{1}", x, ix + 1)))));
}
public static IEnumerable<object[]> ToObjectArray(this IQueryable query)
{
Func<object, object[]> converter;
Converters.TryGetValue(query.ElementType, out converter);
if (converter == null)
{
var row = Expression.Parameter(typeof(object), "row");
// ElementType row2;
var row2 = Expression.Variable(query.ElementType, "row2");
// (ElementType)row;
var cast = Expression.Convert(row, query.ElementType);
// row2 = (ElementType)row;
var assign = Expression.Assign(row2, cast);
var properties = query.ElementType.GetProperties(BindingFlags.Public | BindingFlags.Instance)
.Where(x => x.CanRead && x.GetIndexParameters().Length == 0)
.ToArray();
// (object)row2.Item1, (object)row2.Item2, ...
var properties2 = Array.ConvertAll(properties, x => Expression.Convert(Expression.Property(row2, x), typeof(object)));
// new object[] { row2.Item1, row2.Item2 }
var array = Expression.NewArrayInit(typeof(object), properties2);
// row2 = (ElementType)row; (return) new object[] { row2.Item1, row2.Item2 }
var body = Expression.Block(typeof(object[]), new[] { row2 }, assign, array);
var exp = Expression.Lambda<Func<object, object[]>>(body, row);
converter = exp.Compile();
Converters.TryAdd(query.ElementType, converter);
}
foreach (var row in query)
{
yield return converter(row);
}
}
}

This is a short and easy answer for whoever needs a different answer, but per our discussion with #xanatos, it's not the best as it also returns all the columns which need to be cut off before adding to a list of strings.
List<string> valueList = new List<string>();
using (var ctx = new DataEntities1())
{
var query = ctx.myTable.Where(x => x.pcds == scode).SingleOrDefault();
foreach (var item in columnsArray)
{
valueList.Add(typeof(myTable).GetProperty(onsColumns[Convert.ToInt32(item)]).GetValue(query).ToString());
}
}

LINQ - Where Clause using Contains

I have a LINQ statement that I need to do a "contains" with, but also need some sort of loop.
The format of the data is as follows:
x.Product_Name = "product[x], product[y], product[z]"
As user selects multiple items from a list to search on.
I need to find anything within Product_Name that was selected from the user.
var names = JsonConvert.DeserializeObject<IEnumerable<string>>
(criteria.value).ToArray();
This line gets the items a user selected from the list and stores them in an array.
query = query.Where(x => names.contains(x.Product_Name))
Doesn't work because Product_Name is a flattened out version of products, so I can't do this.
What I need is something like the following:
foreach (string s in names)
{
projectsQuery = projectsQuery.Where(x => x.Product_Name.Contains(s));
}
But when the SQL is created for the above, it uses an AND conditional instead of an OR conditional. I need to find any instances where string s is contained within the Product_Name.

You can achieve it by creating Expression tree manually. Although it is kind of hard to manage code.
var containsMethod = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var xParameter = Expression.Parameter(typeof(x), "x");
var searchexpression = new stack<expression>();
foreach (string s in names)
{
var containsmethodexp = expression.call(Expression.Property(xParameter, "Product_Name", containsMethod, expression.constant(s));
if (searchexpression.count == 0)
{
searchexpression.push(containsmethodexp);
}
else
{
searchexpression.push(expression.orelse(containsmethodexp, searchexpression.pop()));
}
}
var finalResult = projectsQuery.Where(Expression.Lambda<Func<x, bool>>(searchexpression.pop(), new ParameterExpression[] { xParameter }));
here x is your Entity Name

Using a local variable in an expression tree

I have this LINQ Expression that finds all the historical changes to a given customer's CreditBalance:
var history = GetHistory(id);
var changes = history.Where(x => history.Where(y => y.AuditId < x.AuditId)
.OrderByDescending(y => y.AuditId)
.Select(y => y.CreditBalance)
.FirstOrDefault() != x.CreditBalance);
This function works as expected. What I want to do is change this function to allow the user to query changes to any historical field. The way I chose to tackle this was with expression trees.
So far I have come up with this solution:
var history = GetHistory(id);
var c = Expression.Parameter(typeof(Customer_history), "c");
var d = Expression.Parameter(typeof(Customer_history), "d");
var cAudit = Expression.Property(c, typeof(Customer_history).GetProperty("AuditId"));
var dAudit = Expression.Property(d, typeof(Customer_history).GetProperty("AuditId"));
var whereBody = Expression.LessThan(dAudit, cAudit);
var whereLambda = Expression.Lambda(whereBody, d);
var where = Methods.QueryableWhere.MakeGenericMethod(typeof(Customer_history));
var whereCall = Expression.Call(null, where, **Expression.Constant(history)**, whereLambda);
var orderByLambda = Expression.Lambda(dAudit, d);
var orderBy = Methods.QueryableOrderByDescending.MakeGenericMethod(typeof(Customer_history), orderByLambda.Body.Type);
var orderByCall = Expression.Call(null, orderBy, whereCall, orderByLambda);
var dProp = Expression.Property(d, typeof(Customer_history).GetProperty(field));
var selectLambda = Expression.Lambda(dProp, d);
var select = Methods.QueryableSelect.MakeGenericMethod(typeof(Customer_history), selectLambda.Body.Type);
var selectCall = Expression.Call(null, select, orderByCall, selectLambda);
var firstOrDefault = Methods.QueryableFirstOrDefault.MakeGenericMethod(selectLambda.Body.Type);
var firstOrDefaultCall = Expression.Call(null, firstOrDefault, selectCall);
var cProp = Expression.Property(c, typeof(Customer_history).GetProperty(field));
var comparison = Expression.NotEqual(firstOrDefaultCall, cProp);
var lambda = Expression.Lambda<Func<Customer_history, bool>>(comparison, c);
var changes = history.Where(lambda);
The problem is, I get this exception when the query is executed:
Unable to create constant value of type
'Namespace.Customer_history'. Only primitive types or enumeration
types are supported in this context.
Now I am assuming that the issue is the Expression.Constant(history) statement based on the exception message. The problem is, I don't know how to rewrite it to allow the query provider to handle it appropriately. I know it works because of the original query, I just don't know how to do it in an expression tree.
Can anyone provide any direction?

As suspected, it seems that a ConstantExpression is not the way to obtain the value from a local variable.
What I needed to do was create a private class to store the variable in, and then I was able to access it with a field MemberExpression
private class ValueHolder<T>
{
public IQueryable<T> History;
}
Then in my method I was able to have the expression evaluated using this:
var valueHolder = new ValueHolder<T>
{
History = data
};
var c = Expression.Parameter(typeof(T), "c");
var constantEx = Expression.Constant(valueHolder);
var fieldEx = Expression.Field(constantEx, valueHolder.GetType().GetField("History"));

You can always try using dynamic linq, which allows you to use strings as expressions instead of lambda.
Example:
var query = history.Where("MyField = MyFilter");
https://www.nuget.org/packages/System.Linq.Dynamic.Library/
https://github.com/NArnott/System.Linq.Dynamic

Dynamic linq query expression tree for sql IN clause using Entity framework

I want to create a dynamic linq expression for sql IN clause in EF 6.0 with code first approch. Note that i am new to Expressions. What i want to achive is
select * from Courses where CourseId in (1, 2, 3, 4)
//CourseId is integer
The normal linq query looks like this. But i want to query it dynamically
string[] ids = new string[]{"1", "2", "3", "4"};
var courselist = DBEntities.Courses.Where(c => ids.Contains(SqlFunctions.StringConvert((decimal?)c.CourseId)))
There are two ways to make dynamic expression.
1) one ways is to loop through ids and make expressions
The below code will create the following expression in debug view
{f => ((StringConvert(Convert(f.CourseId)).Equals("23") Or StringConvert(Convert(f.CourseId)).Equals("2")) Or StringConvert(Convert(f.CourseId)).Equals("1"))}
Dynamic Expression is
var param = Expression.Parameters(typeof(Course), "f")
MemberExpression property = Expression.PropertyOrField(param, "CourseId");
MethodInfo mi = null;
MethodCallExpression mce = null;
if (property.Type == typeof(int))
{
var castProperty = Expression.Convert(property, typeof(double?));
var t = Expression.Parameter(typeof(SqlFunctions), "SqlFunctions");
mi = typeof(SqlFunctions).GetMethod("StringConvert", new Type[] { typeof(double?) });
mce = Expression.Call(null,mi, castProperty);
}
mi = typeof(string).GetMethod("Equals", new Type[]{ typeof(string)});
BinaryExpression bex = null;
if (values.Length <= 1)
{
return Expression.Lambda<Func<T, bool>>(Expression.Call(mce, mi, Expression.Constant(values[0]), param));
}
//var exp1 = Expression.Call(mce, mi, Expression.Constant(values[0]));
for (int i = 0; i < values.Length; i++)
{
if (bex == null)
{
bex = Expression.Or(Expression.Call(mce, mi, Expression.Constant(values[i])), Expression.Call(mce, mi, Expression.Constant(values[i + 1])));
i++;
}
else
bex = Expression.Or(bex, Expression.Call(mce, mi, Expression.Constant(values[i])));
}//End of for loop
return Expression.Lambda<Func<T, bool>>(bex, param);
2) The 2nd way that i tried (debug view)
{f => val.Contains("23")} //val is parameter of values above
The dynamic expression for above that i tried is
var param = Expression.Parameters(typeof(Course), "f")
MemberExpression property = Expression.PropertyOrField(param, "CourseId");
var micontain = typeof(Enumerable).GetMethods().Where(m => m.Name == "Contains" && m.GetParameters().Length == 2).Single().MakeGenericMethod(typeof(string));
var mc = Expression.Call(micontain, Expression.Parameter(values.GetType(), "val"), Expression.Constant("2"));//NOTE: I haven't use CourseId for now as i am getting conversion error
return Expression.Lambda<Func<T, bool>>(mc, param);
I get the following errors
LINQ to Entities does not recognize the method 'System.String StringConvert(System.Nullable`1[System.Double])' method, and this
method cannot be translated into a store expression when i use the
first methodology. I know i can't use ToString with EF thats why I used SqlFunctions but it is not working for me.
The parameter 'val' was not bound in the specified LINQ to Entities query expression using 2nd methodology
I am trying this from last 4 days. I googled it but didn't find any suitable solution. Please help me.

After a lot of struggle I found solution to my question.
I want to achieve this sql query
select * from Courses where CourseId in (1, 2, 3, 4)
Using Linq to Entities, but I want to pass in(1,2,3,4) list dynamically to linq query. I created an Extension class for that purpose.
public static class LinqExtensions
{
public static Expression<Func<T, bool>> False<T>() { return f => false; }
public static Expression<Func<T, bool>> In<T, TValue>(this Expression<Func<T, bool>> predicate,string propertyName, List<TValue> values)
{
var param = predicate.Parameters.Single();
MemberExpression property = Expression.PropertyOrField(param, propertyName);
var micontain = typeof(List<TValue>).GetMethod("Contains");
var mc = Expression.Call(Expression.Constant(values), micontain, property);
return Expression.Lambda<Func<T, bool>>(mc, param);
}
}
Use of LinqExtensions
var pred = LinqExtensions.False<Course>(); //You can chain In function like LinqExtensions.False<Course>().In<Course, int>("CourseId", inList);
var inList= new List<int>(){1, 2, 3}; //Keep in mind the list must be of same type of the Property that will be compared with. In my case CourseId is integer so the in List have integer values
pred =pred.In<Course, int>("CourseId", inList); //TValue is int. As CourseId is of type int.
var data = MyEntities.Courses.Where(pred);
I hope this might be beneficial for some one

have you seen the type of
var courselist = DBEntities.Courses.Where(c => ids.Contains(c.CourseId)))
above statement would not return actual list of courses. The query is not executed yet. It just returns IQuereable. The query is executed when you actually call .ToList() method on it
so, your solution is..
Create array of IDs using for loop and then simply run the below query
var courselist = DBEntities.Courses.Where(c => ids.Contains(c.CourseId))).ToList()

How to make query without having generic type?

I'll start with piece of code:
var objectType = typeof(Department); // Department is entity class from linqdatacontext
using (var dataContext = new DataModel.ModelDataContext())
{
var entity = Expression.Parameter(objectType, "model");
var keyValue = Expression.Property(entity, "Id");
var pkValue = Expression.Constant(reader.Value);
var cond = Expression.Equal(keyValue, pkValue);
var table = dataContext.GetTable(objectType);
... // and here i don't how to proceed
}
I am not even sure if i am building that expression correctly. However simply put, i need to call dynamically SingleOrDefault() on that table to find entity by primary key. Every example i had found is using generic variant of GetTable<>(), but i cannot use that obviously. I am probably overlooking something...

Whenever I build expression trees, I like to start off with an example of what I'm building:
() => dataContext.GetTable<TEntity>().SingleOrDefault(entity => entity.Id == 1);
From that, we can easily dissect the target expression. You are partway there; you just need to include a call to the GetTable method in the expression tree and then build an outer lambda expression to call the whole thing:
using(var dataContext = new DataModel.ModelDataContext())
{
var getTableCall = Expression.Call(
Expression.Constant(dataContext),
"GetTable",
new[] { entityType });
var entity = Expression.Parameter(entityType, "entity");
var idCheck = Expression.Equal(
Expression.Property(entity, "Id"),
Expression.Constant(reader.Value));
var idCheckLambda = Expression.Lambda(idCheck, entity);
var singleOrDefaultCall = Expression.Call(
typeof(Queryable),
"SingleOrDefault",
new[] { entityType },
getTableCall,
Expression.Quote(idCheckLambda));
var singleOrDefaultLambda = Expression.Lambda<Func<object>>(
Expression.Convert(singleOrDefaultCall, typeof(object)));
var singleOrDefaultFunction = singleOrDefaultLambda.Compile();
return singleOrDefaultFunction();
}
We have to convert the SingleOrDefault call to have a return type of object so it can serve as the body of the Func<object> function.
(Untested)
Edit: Parameterizing the data context and value
Now we are building this function:
(dataContext, value) => dataContext.GetTable<TEntity>().SingleOrDefault(entity => entity.Id == value);
You would change the constants to parameters and add those parameters to the function you compile:
var dataContextParameter = Expression.Parameter(typeof(ModelDataContext), "dataContext");
var valueParameter = Expression.Parameter(typeof(object), "value");
var getTableCall = Expression.Call(
dataContextParameter,
"GetTable",
new[] { entityType });
var entity = Expression.Parameter(entityType, "entity");
var idCheck = Expression.Equal(
Expression.Property(entity, "Id"),
valueParameter);
var idCheckLambda = Expression.Lambda(idCheck, entity);
var singleOrDefaultCall = Expression.Call(
typeof(Queryable),
"SingleOrDefault",
new[] { entityType },
getTableCall,
Expression.Quote(idCheckLambda));
var singleOrDefaultLambda =
Expression.Lambda<Func<ModelDataContext, object, object>>(
Expression.Convert(singleOrDefaultCall, typeof(object)),
dataContextParameter,
valueParameter);
var singleOrDefaultFunction = singleOrDefaultLambda.Compile();
// Usage
using(var dataContext = new DataModel.ModelDataContext())
{
return singleOrDefaultFunction(dataContext, reader.Value);
}

If you are using .NET 4, you could try casting your returned objects as dynamic, so you could then query like this.
using (var dataContext = new DataModel.ModelDataContext())
{
var entity = Expression.Parameter(objectType, "model");
var keyValue = Expression.Property(entity, "Id");
var pkValue = Expression.Constant(reader.Value);
var cond = Expression.Equal(keyValue, pkValue);
var table = dataContext.GetTable(objectType);
var result = table.Where(ent => ((dynamic)ent).SomeField == "SomeValue");
}

I'm still not entirely sure as to the whole of your problem (and I suspect the answer about dynamic is going to solve part of what will come up too). Still, just to answer:
Every example i had found is using generic variant of GetTable<>(), but i cannot use that obviously
For any T, Table<T> implements (among other interfaces) ITable<T> and ITable. The former is generically typed, the latter not.
The form GetTable<T>() returns such a Table<T>. However, the form GetTable(Type t) returns an ITable. Since ITable inherits from IQueryable you can query it. If you need to do something with that query that would normally require knowledge of the type (such as comparing on a given property) then dynamic as per the previous answer given by Steve Danner allows that to happen.

I'd do it using Reflection and the LINQ dynamic query library, personally.
You can get the list of all the keys for a table with dataContext.Mapping.GetMetaType(objectType).IdentityMembers, then access the data with something along the lines of dataContext.GetTable(objectType).Where(key.Name + "==#0", id).
Obviously, I left out a few steps in there - if you have multiple keys, you'll need to build a fuller predicate with a loop over .IdentityMembers, and if you always just have the one key, you can use .First() on it. I haven't tested it either, but it should be pretty close. It'd probably be 6-7 lines of code total - I can write it up (and test) if you need it.
Edit: The LINQ Dynamic Query Library can be downloaded from Microsoft at http://msdn.microsoft.com/en-us/vcsharp/bb894665.aspx - just include DynamicLINQ.cs in your project and you're good.