Develop Reference

EF Core dynamically filter from list

I am working on creating a generic dynamic filter on a front-end table. The idea is each column will have a filter, where you can select values of that column. The column names must be completely dynamic and selected values. We are using .NET 5 and EF Core 5.
The SQL query I want is:
SELECT *
FROM Users
WHERE externalId IN ('1234', '5678');
Models:
public class ColumnFilter
{
[Required]
public string Name { get; set; } // column name
public List<string> SelectedValues { get; set; } = new List<string>(); // values selected in this column
}
public class User {
public string Name { get; set; }
public string ExternalId { get; set; }
}
Simplified code:
//columnfilter really comes from API request
var columnFilter = new ColumnFilter{
Name = "ExternalId",
SelectedValues = new List<string>{ "1234", "5678" }
};
var queryable = _context.Users.AsNoTracking();
queryable = queryable.Where(f => columnFilter.SelectedValues.Contains(EF.Property<string>(f, columnFilter.Name)));
var values = await queryable.ToListAsync(); // 0 results
EF Core generates this query:
SELECT *
FROM [Users] AS [u]
WHERE 0 = 1
How do I get this to work dynamically?

It is a sample using string values:
var queryable = _context.Users.AsNoTracking();
queryable = queryable.FilterDynamic(columnFilter.Name, columnFilter.SelectedValues);
public static class QueryableExtensions
{
public static IQueryable<T> FilterDynamic<T>(this IQueryable<T> query, string fieldName, ICollection<string> values)
{
var param = Expression.Parameter(typeof(T), "e");
var prop = Expression.PropertyOrField(param, fieldName);
var body = Expression.Call(typeof(Enumerable), "Contains", new[] {typeof(string)},
Expression.Constant(values), prop);
var predicate = Expression.Lambda<Func<T, bool>>(body, param);
return query.Where(predicate);
}
}

The solution I had actually works. I had some other code involved that was causing the issue.

Build an lambda Expression tree with a specific field in a linked entity

Data MODEL
public class TABLE
{
[Key]
public int ID { get; set; }
public int ONEFIELD { get; set; }
// -------------------- ForeignKey --------------------
[ForeignKey("People")]
public long PeopleID { get; set; }
public virtual People People { get; set; }
}
public class People
{
[Key]
public long PeopleID { get; set; }
public int CountryID { get; set; }
}
I need to build a lambda to query this MODEL :
Get TABLE.ONEFIELD = 1 AND TABLE.PEOPLE.COUNTRYID = 6
LINQ equivalent
_context.TABLEs
.Where(e => e.ONEFIELD == 1)
.Include(e => e.People)
.Where(i=>i.People.CountryID == 6);
My try
public static Expression<Func<TEntity, bool>> BuildLambda<TEntity>(OBJTYPE obj)
{
var item = Expression.Parameter(typeof(TEntity), "table");
Expression query = null;
// 1
var prop1 = Expression.Property(item, "ONEFIELD");
var value1 = Expression.Constant(1);
var equal1 = Expression.Equal(prop1, value1);
var lambdaFIELDONE = Expression.Lambda<Func<TEntity, bool>>(equal1, item);
query = lambdaFIELDONE.Body;
// 2
var prop2 = Expression.Property(item, typeof(People).Name + ".CountryID");
var value2 = Expression.Constant(6);
var equal2 = Expression.Equal(prop2, value2);
var lambdaCOUNTRYID = Expression.Lambda<Func<TEntity, bool>>(equal2, item);
query = Expression.And(query, lambdaCOUNTRYID);
}
but I receive this error
System.ArgumentException: Instance property 'People.CountryID' is not defined for type 'SOLUTION.Models.TABLE'
I don't need Generic, just a fixed lambda (and I couldn't use LINQ).
I tried several things to catch People.CountryID like
Expression.Property(item1, typeof(People).GetProperty("CountryID"));
Expression.Property(item, typeof(People).Name+"." + typeof(People).GetProperty("CountryID"));
Expression.Property(item, typeof(People).Name + "." + typeof(People).GetProperties().Where(x => x.Name == "CountryID").FirstOrDefault().Name);
no success
Any ideas ? thanks

So, to build a nested property access, you must nest the Expressions that access each level. Then you can combine the tests into a body and finally create the lambda for the result:
public static Expression<Func<TEntity, bool>> BuildLambda<TEntity>(OBJTYPE obj) {
// (TEntity table)
var parmTable = Expression.Parameter(typeof(TEntity), "table");
// table.ONEFIELD
var prop1 = Expression.Property(parmTable, "ONEFIELD");
// table.ONEFIELD == 1
var equal1 = Expression.Equal(prop1, Expression.Constant(1));
// table.People
var prop2_1 = Expression.Property(parmTable, nameof(People));
// table.People.CountryID
var prop2_2 = Expression.Property(prop2_1, "CountryID");
// table.People.CountryID == 6
var equal2 = Expression.Equal(prop2_2, Expression.Constant(6));
// table.ONEFIELD == 1 && table.People.CountryID == 6
var finalBody = Expression.AndAlso(equal1, equal2);
// table => table.ONEFIELD == 1 && table.People.CountryID == 6
return Expression.Lambda<Func<TEntity, bool>>(finalBody, parmTable);
}
Using LINQPad, you could create a sample lambda and then use the Dump method and you would see that a nested FieldExpression is created, which is what gets created when you call Expression.Property:
Expression<Func<TEntity, int>> f = t => t.People.CountryID;
f.Dump();

Linq expression filter in IQueryable

I want to create generic expression filtering in IQueryable
public class Vehicle
{
public int Id { get; set; }
public string VehicleNO { get; set; }
public int DriverId { get; set; }
public Driver Driver {get;set;}
}
public class Driver
{
public int Id { get; set; }
public string Name { get; set; }
}
operator = "Contain", field name = "Driver.Name", Value filter =
"Micheal"
I don't know how to filter driver name.
Here is my full code
IQueryable<SysClientSiteUser> query = entity.SysClientSiteUsers.Include(i => i.SysClientSiteRole);
Dictionary<string, string> dtFilter = new Dictionary<string, string>();
dtFilter.Add("VehicleNo", "A123");
Dictionary<Type, Func<string, object>> lookup = new Dictionary<Type, Func<string, object>>();
lookup.Add(typeof(string), x => { return x; });
lookup.Add(typeof(long), x => { return long.Parse(x); });
lookup.Add(typeof(int), x => { return int.Parse(x); });
lookup.Add(typeof(double), x => { return double.Parse(x); });
var paramExpr = Expression.Parameter(typeof(Vehicle), "VehicleNo");
var keyPropExpr = Expression.Property(paramExpr, "VehicleNo");
if (!lookup.ContainsKey(keyPropExpr.Type))
throw new Exception("Unknown type : " + keyPropExpr.Type.ToString());
var typeDelegate = lookup[keyPropExpr.Type];
var constantExp = typeDelegate("A123");
var eqExpr = Expression.Equal(keyPropExpr, Expression.Constant(constantExp));
var condExpr = Expression.Lambda<Func<SysClientSiteUser, bool>>(eqExpr, paramExpr);
query = query.Where(condExpr);
for normal field, it's working. But if I want to call Driver name. it's not work. How to call "Driver.Name"?

You can use a helper function to convert a nested property name string to an Expression that accesses that property for a given ParameterExpression and type:
private static Expression MakePropertyExpression<T>(string propertyName, Expression baseExpr) =>
propertyName.Split('.').Aggregate(baseExpr, (b, pname) => Expression.Property(b, pname));

Building a dynamic expression tree to filter on a collection property 2

Maybe this is a duplicate thread, but I am going to try, because there is a tiny difference.
I am trying to build a dynamic expression to filter a collection property.
The code:
public class TestEntity
{
public int ID { get; set; }
public string Name { get; set; }
public IEnumerable<string> Values { get; set; }
}
public class TestFilter
{
public TestFilter()
{
var itens = new List<TestEntity>();
itens.Add(new TestEntity { ID = 1, Name = "Test1", Values = new List<string> { "V1", "V2" } });
itens.Add(new TestEntity { ID = 2, Name = "Test2", Values = new List<string> { "V6", "V3" } });
itens.Add(new TestEntity { ID = 3, Name = "Test3", Values = new List<string> { "V4", "V5" } });
itens.Add(new TestEntity { ID = 4, Name = "Test4", Values = new List<string> { "V2", "V3" } });
itens = itens.Where(e => e.Values.Any(c => c.Equals("V2"))).ToList();
**//Result IDs: 1, 4**
}
}
The filter above will give me IDs 1 and 4 as result.
I want to filter entities where exists a certain value in the collection "Values".
So far, I have tried this thread, but didnt realize how it can be done.
Any help would be apreciated.

So you are seeking for a method which given a collection property name and value will produce Where predicate like e => e.Collection.Any(c => c == value).
You can use the following extension method (hope the code is self explanatory):
public static class QueryableExtensions
{
public static IQueryable<T> WhereAnyEquals<T>(this IQueryable<T> source, string collectionName, object value)
{
var e = Expression.Parameter(typeof(T), "e");
var collection = Expression.PropertyOrField(e, collectionName);
var itemType = (collection.Type.IsIEnumerableT() ? collection.Type :
collection.Type.GetInterfaces().Single(IsIEnumerableT))
.GetGenericArguments()[0];
var c = Expression.Parameter(itemType, "c");
var itemPredicate = Expression.Lambda(
Expression.Equal(c, Expression.Constant(value)),
c);
var callAny = Expression.Call(
typeof(Enumerable), "Any", new Type[] { itemType },
collection, itemPredicate);
var predicate = Expression.Lambda<Func<T, bool>>(callAny, e);
return source.Where(predicate);
}
private static bool IsIEnumerableT(this Type type)
{
return type.IsInterface && type.IsConstructedGenericType &&
type.GetGenericTypeDefinition() == typeof(IEnumerable<>);
}
}
like this:
itens = itens.AsQueryable().WhereAnyEquals("Values", "V2").ToList();
If you step through the code, the variable predicate contains the expression you are asking for.

How do I select specific columns from IQueryable<T> in C#

I need select some columns dynamically from a IQueryble.
Example:
IQueryable<Car> query = (from a in _dbContext.Table1
select new Car
{
Prop1 = a.Prop1,
Prop2 = a.Prop2,
Prop3 = a.Prop3
});
I need to do something like:
var lambda = new Test().CreateNewStatement<Car>("Prop1,Prop2");
I am using the function:
public Func<T, T> CreateNewStatement<T>(string fields)
{
var xParameter = Expression.Parameter(typeof(T), "o");
var xNew = Expression.New(typeof(T));
var bindings = fields.Split(',').Select(o => o.Trim())
.Select(o =>
{
var mi = CustomGetType<T>(o);
var xOriginal = Expression.Property(xParameter, mi);
return Expression.Bind(mi, xOriginal);
}
);
var xInit = Expression.MemberInit(xNew, bindings);
var lambda = Expression.Lambda<Func<T, T>>(xInit, xParameter);
return lambda.Compile();
}
And the Call:
var lambda = new Test().CreateNewStatement<Car>("Prop1");
var ax = query.Select(lambda);
var result = ax.ToList();
But in database, the query is wrong:
SELECT
[Limit1].[C1] AS [C1],
[Limit1].[Prop1] AS [Prop1],
[Limit1].[Prop2] AS [Prop2],
[Limit1].[Prop3] AS [Prop3],
FROM ( ...
The correct search in database should be:
SELECT
[Limit1].[C1] AS [C1],
[Limit1].[Prop1] AS [Prop1]
FROM ( ...
I need manipulate the IQueryable< T > before the search in database.

The issue you're having is that you're passing a Func<User, User> into Select. This essentially means that the method is executed in memory, not in the database.
Entity Framework cannot generate SQL code from a compiled function.
When you write this:
var users = Users.Select(u => new User { a = 1 });
You're passing it an Expression<Func<User, User>> which entity framework CAN turn into SQL.
So, what you need to change is the end of your function. Instead of this:
var lambda = Expression.Lambda<Func<T, T>>(xInit, xParameter);
return lambda.Compile()
You want to simply do
return Expression.Lambda<Func<T, T>>(xInit, xParameter);
And change the return type of your function to be Expression<Func<T,T>>
HOWEVER
For whatever reason, EntityFramework does not allow you to construct objects which are entities. Linq-To-Sql does allow this, and running the code with my above changes will indeed work, and only select the one column.
Essentially this means that we need to change the expression to return a class which is not an entity used by entity framework.
So, instead of writing this:
var users = Users.Select(u => new User { a = 1 });
We need to write this:
var users = Users.Select(u => new NotAnEntityUser { a = 1 });
We then need to change our method to return a NotAnEntityUser instead of a User.
public class Test
{
public Expression<Func<TEntityType, TModelType>> CreateNewStatement<TEntityType, TModelType>(string fields)
{
var xParameter = Expression.Parameter(typeof (TEntityType), "o");
var xNew = Expression.New(typeof (TModelType));
var bindings = fields.Split(',').Select(o => o.Trim())
.Select(paramName =>
{
var xOriginal = Expression.Property(xParameter, typeof(TEntityType).GetProperty(paramName));
return Expression.Bind(typeof(TModelType).GetProperty(paramName), xOriginal);
}
);
var xInit = Expression.MemberInit(xNew, bindings);
var lambda = Expression.Lambda<Func<TEntityType, TModelType>>(xInit, xParameter);
return lambda;
}
}
And define our entity and model:
public class User
{
public virtual string Id { get; set; }
public virtual string UserName { get; set; }
public virtual string Salt { get; set; }
public static implicit operator User(NotAnEntityUser o)
{
return new User { Id = o.Id, UserName = o.UserName, Salt = o.Salt };
}
}
public class NotAnEntityUser
{
public virtual string Id { get; set; }
public virtual string UserName { get; set; }
public virtual string Salt { get; set; }
}
Which then lets you write:
var lam = new Test().CreateNewStatement<User, NotAnEntityUser>("Id");
var c = new MyContext().Users.Select(lam).AsEnumerable().Select(u => (User)u).ToList();
And indeed, the SQL generated from this is:
SELECT
1 AS [C1],
[Extent1].[Id] AS [Id]
FROM [dbo].[User] AS [Extent1]
Instead of writing the implicit cast operator, you could also use a tool like automapper, etc.
With automapper, you'd end up with something like:
var c = AutoMapper.Map<List<User>>(new MyContext().Users.Select(lam).ToList());