I've got a collection of movies which have various properties (title, release year, rating, etc) that I need to search for using a LINQ query as follows:
public BindingList<Movie> SearchByTitle(string title)
{
var matches = from movies in movieCollection
where movies.Title == title
select movies;
// do some other stuff with the matches
}
But I don't want a separate method to search for each property since the only thing that changes between searches is the where section. For example where movies.Rating == rating or where movies.ReleaseYear == releaseYear. How do I make the search method reusable for all different kinds of searches by passing in some sort of Expression or Func as the where section?
How do I make the search method reusable for all different kinds of searches by passing in some sort of Expression or Func as the where section?
Your query really isn't anything other than the where clause. But you can easily make the where part configurable... just not using query expressions.
public BindingList<Movie> SearchByTitle(Expression<Func<Movie, bool>> predicate)
{
var matches = movies.Where(predicate);
// Do common stuff with the matches.
}
EDIT: I was assuming that movies was an IQueryable<T>, given that you were talking about Expression. If it's just an IEnumerable<T>, you want:
public BindingList<Movie> SearchByTitle(Func<Movie, bool> predicate)
{
var matches = movies.Where(predicate);
// Do common stuff with the matches.
}
You can use an extension method (define this in a static class)
public static IQueryable<T> AddSearchParameter<T>(this IQueryable<T> query, bool condition, System.Linq.Expressions.Expression<Func<T, bool>> predicate)
{
if (condition)
{
query = query.Where(predicate);
}
return query;
}
So for example:
public BindingList<Movie> Search(string title, int? year, int? rating)
{
var matches = movieCollection.AddSearchParameter(!string.IsNullorEmpty(title), m=>m.Title == title);
matches = matches.AddSearchParameter(year.HasValue, m=>m.Year == year.Value);
matches = matches.AddSearchParameter(rating.HasValue, m=>m.rating >= rating.Value);
// do some other stuff with the matches
}
If you're using this against a database it will not actually execute the query until you enumerate so this will not make multiple calls to your database.
You could use a CompiledQuery.
Check this very interesting answer on SO.
Hope it helps.
Related
I've been using LinqKit to create generic queries for quite some time.
One thing that has always bothered me is the fact that you always have to test whether the value sent in the filter is valid.
For example: Suppose I have a string filter. Conditions can be Equal, StartsWith, EndsWith and Contains.
My method would look something like this:
public List<MyModel> Get(MyModelFilter filter)
{
if (string.IsNullOrEmpty(filter.prop))
{
predicate = predicate.And(_myModel => myModel.Prop.Contains(filter.prop));
}
// Plus a giant amount of if's with multiple filters
return DbSet.AsExpandable()
.Where(predicate)
.ToList();
}
To end this bunch of If's, I decided to create a generic method to apply the filter to the properties.
My idea is to pass the property where the filter will be applied, and the filter definition, and encapsulate the Expression creation logic
It would be something of the type:
public List<MyModel> Get(MyModelFilter filter)
{
predicate = predicate.And(_myModel => myModel.Prop, filter.PropFilterDefinition);
// Goodnye If's, Only others filter impl
return DbSet.AsExpandable()
.Where(predicate)
.ToList();
}
For this, I've created some extension methods to handle this
public static Expression<Func<TPredicate, bool>> And<TPredicate>(
this ExpressionStarter<TPredicate> predicate,
Func<TPredicate, string> property, StringFilterDefinition filter,
bool ignoreNull = true)
{
if (InvalidStringFilter(filter, ignoreNull))
{
return predicate;
}
// This is LinqKit's And Extension Method
return predicate.And(BuildPredicate(property, filter));
}
private static Expression<Func<TPredicate, bool>> BuildPredicate<TPredicate>(
Func<TPredicate, string> property,
StringFilterDefinition filter)
{
if (filter.Filter == StringFilterComparators.Equal)
{
return x => property.Invoke(x) == filter.Value;
}
if (filter.Filter == StringFilterComparators.BeginsWith)
{
return x => property.Invoke(x).StartsWith(filter.Value);
}
if (filter.Filter == StringFilterComparators.EndsWith)
{
return x => property.Invoke(x).EndsWith(filter.Value);
}
return x => property.Invoke(x).Contains(filter.Value);
}
private static bool InvalidStringFilter(
StringFilterDefinition filter,
bool ignoreNullValue = true)
{
if (filter?.Filter == null)
{
return true;
}
return ignoreNullValue && string.IsNullOrEmpty(filter.Value);
}
The problem is that the filter is not applied, and the answer is in Invoke right up there. EF can not translate the above expression to SQL.
The EF error is
Microsoft.EntityFrameworkCore.Query.Internal.SqlServerQueryCompilationContextFactory[8]
The LINQ expression '(__property_0.Invoke([x]) == __filter_Value_1)'
could not be translated and will be evaluated locally. To configure
this warning use the DbContextOptionsBuilder.ConfigureWarnings API
(event id 'RelationalEventId.QueryClientEvaluationWarning').
ConfigureWarnings can be used when overriding the
DbContext.OnConfiguring method or using AddDbContext on the
application service provider.
The question is:
How can I make this construction work?
Also, any suggestions on how best this?
You seem to forgot that besides the PredicateBuilder, the really useful feature provided by LINQKit AsExpandable, Expand and Invoke custom extension methods is to be able to correctly embed expressions inside the expression tree.
In order to utilize that feature, you should use Expression<Func<...>> instead of Func<...>. In the posted code, replace all occurrences of Func<TPredicate, string> with Expression<Func<TPredicate, string>> and the issue should be solved.
I am using a generic repository, like this:
itemsList = (from myrow in UoW.FileRepository.Get()
select new FileModel()
{record_id = myrow.type_id,
descr = myrow.descr}).ToList();});
And this is the Get method:
public virtual IEnumerable<TEntity> Get()
{
// _aQuery = _theDbContext.Set<TEntity>();
IEnumerable<TEntity> query = _aQuery;
return query;
}
How would I implement a generic linq lambda expression if I wanted to create a similar query to search for a particular string in a particular field? In my viewmodel I would like to call something like:
from myrow in UoW.FileRepository.Srch(nameofFieldToSearch, searchString).
The query would look something like this?
public IEnumerable<TEntity> Srch(Expression<Func<TEntity, bool>> expression)
{
IEnumerable<TEntity> srchList = _aQuery.ToList();
return srchList.Where(????);
}
Thank you for your suggestions.
EDIT-----------------------
I have all my queries like Get and Srch in a general repository class and for now just need to know how to declare the query in the repository class and how to call it with the search string from my viewmodel. I am not sure if there is a consensus as to where/when to materialize and compile? I saw another discussion http://www.fascinatedwithsoftware.com/blog/post/2012/01/10/More-on-Expression-vs-Func-with-Entity-Framework.aspx and I quote from it below to inquire whether that is the same approach being suggested here? Thank you again.
"The profiler told us that LoadMyEntities was being called many, many times and it was taking a large fraction of our CPU time. The simple change below solved the problem. Can you guess why?"
public IEnumerable<MyEntity> LoadMyEntities(Func<MyEntity, bool> predicate)
{return Context.MyEntities.Where(predicate);}
"The parameter is now a Func<> instead of an Expression>. The reason this makes a difference is that a predicate that's in the form of an Expression is passed to SQL server, but a predicate that's passed as a Func is not. Normally, you'd want SQL Server to do as much for you as possible, and an Expression would be the right choice, but in this case we'd like to pre-load the entire table in the context -- which is exactly what a Func will do.
The Where extension method has two flavors. One extends IQueryable and takes an Expression parameter. The other extends IEnumerable and takes a Func.
Because 'predicate' is now a Func, the Where that extends IEnumerable is used.
The Entity Framework's fluent interface for constructing SQL queries is based on IQueryables, not IEnumerables. Therefore, the fluency stops just before the Where. The part of the statement that gets passed to the Entity Framework is just Context.MyEntities.
Context.MyEntities therefore returns the entire table to the context.
The entire table is now filtered with the predicate, and the value we really want is returned.
The next time the method is called, the Entity Framework realizes that the record we want is already in the context. (In my case, we were querying by the primary key, and EF is apparently smart enough to know that if there's a record in the context with that ID, it won't find an additional such record in the database.) Since we don't go out to SQL Server, we save lots of time. Obviously there are occasions when you would not want this, but in our case it was exactly what we wanted. The table was relatively small, and the same context was queried hundreds of times.
In the original version, the predicate was an Expression, so the compiler used the Where that extends IQueryable. The predicate was thus passed to SQL Server, which dutifully returned just one row to the context. The next time we called LoadMyEntities, Entity Framework had to call SQL Server again."
Please take a look at msdn repository pattern
public virtual IEnumerable<TEntity> Get(
Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
string includeProperties = "")
{
IQueryable<TEntity> query = dbSet;
if (filter != null)
{
query = query.Where(filter);
}
foreach (var includeProperty in includeProperties.Split
(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty);
}
if (orderBy != null)
{
return orderBy(query).ToList();
}
else
{
return query.ToList();
}
}
This will provide more complete Generic Get method.
For search you could Try
public IEnumerable<TEntity> GlobalSearch(Expression<Func<TEntity, bool>> expression)
{
return Get(expression);
}
In DataProvider you could try something like this
Note This is not may be exact implementation, but will give you basic idea, how to invoke.
public List<Users> Search(List<string> userList)
{
Expression<Func<User, bool>> expression = x=>UserList.Contains(x.UserName);
return GlobalSearch(expression);
}
Example of Repository pattern
This would be a simple implementation using the Expression trees. Following is the complete solution:
Method to fetch the Expression for Srch method:
Public Expression<Func<TEntity, bool>> SrchExpression(string nameofFieldToSearch, string searchString)
{
var parameterType = Expression.Parameter(typeof(TEntity), "obj");
var memberExpression = Expression.Property(typeof(string), nameofFieldToSearch)
// Calls Extension method created underneath
var filtersMethodInfo = typeof(StringExtensions).GetMethod("Contains", new[] { typeof(string), typeof(string) });
var filtersConstantExpression = Expression.Constant(searchString, typeof(string));
var finalExpression = Expression.Call(null, filtersMethodInfo, memberExpression, filtersConstantExpression)
return Expression.Lambda<Func<TEntity, bool>>(finalExpression, parameterType)
}
// Create a String extension method for Contains
public static class StringExtensions
{
public static bool Contains(this string source, string searchString)
{
return source?.IndexOf(subString, StringComparison.OrdinalIgnoreCase) >= 0;
}
}
Now your Srch method shall look like:
public IEnumerable<TEntity> Srch(Expression<Func<TEntity, bool>> expression)
{
Func<TEntity, bool>> func = expression.Compile();
IEnumerable<TEntity> srchList = _aQuery.Where(o => func(o));
return srchList;
}
I'm building my own reflection functions for certain types of searches.
The problem is that I want to search a group of IDs within a list of IDs and filter my search/select query to have only these specific objects.
This is the same as using "IN()" in Linq-Entity framework. But I can't use a.objid.
return query.Where(a => ObjectsToFind.Contains(a.objid));
However, "a.objid" is causing errors because I use T Template.
So a is "T a" instead of "MyTable a" so that I can call it's "objid" property.
I know there is a way to do this with parameter expressions. However, I can't figure it out.
Here's what I tried to replace that above line with:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query, string contains)
{
var ObjectsToFind = new List<int>(); // I want to search IN() this function that gets filled in here.
ObjectsToFind = FillObjectsToFind(); // just the object id integers I want to filter
var parameter = Expression.Parameter(typeof(T), "type");
var propertyExpression = Expression.Property(parameter, "objid"); // I look for this
MethodInfo method = typeof(string).GetMethod("Contains", new[] { typeof(int) }); // int type
var vars = Expression.Variable(List<int>,); // I can't use "constant", but what do I use here?
var containsExpression = Expression.Call(propertyExpression, method, vars);
return query.Where(Expression.Lambda<Func<T, bool>>(containsExpression, parameter));
}
Replacing "T" with the actual table entity, causes a lot of problems, so I decided to keep the T.
else if (s.ST == "function")
{ // search func
myTable a;
DC2 = MyUtility.WhereFunctionContains(DC2, a => a.objid, s.S);
// s.S is my search query string.
// s.ST means I'm searching for functions (from another associated table)
// I don't understand how a.objid gets called, I wanted to use Template/Reflections.
}
Here's how I call Zev's function.
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query, Expression<Func<T, int>> converter, string contains)
{
FunctionsClass fc = new FunctionsClass();
var ObjectsToFind = new List<int>();
ObjectsToFind = fc.SearchContainFunction(contains); // I grabbed my list of IDs to search
return query.Where(t => ObjectsToFind.Contains(converter(t)));
}
If I understand you correctly, you have a number of queries on different types:
IQueryable<Person> personsQry = ...
IQueryable<Sale> salesQry = ...
IQueryable<Order> ordersQry = ...
and you have a method that generates a List<int>, called FillObjectsToFind:
public List<int> FillObjectsToFind()
{
//code here
}
You want to be able to limit each of the above queries to only have the id in the returned list. As an added bonus, this should be an extension method, so you can call it like this:
var personsFiltered = personsQry.WhereFunctionContains(...);
var salesFiltered = salesQry.WhereFunctionContains(...);
var ordersFiltered = ordersQry.WhereFunctionContains(...);
The problem is each query is of a separate type, and you would prefer to write one method that covers all of them.
The first part of the solution is to define a generic method:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query)
{
//code here
}
but there is still a problem: the only type we know of is type T which is not a real type, but a placeholder for actual types. Since these types could be anything -- string, System.Random, SqlConnection, an ASP.NET Label, a WPF TextBlock -- there is no way of knowing how to compare each object to a List of ints.
The most straightforward solution is to define an interface:
interface IHasObjID
{
int ObjID {get;set;}
}
Then each type should implement this interface:
class Person : IHasObjID
{
int objID;
int ObjID
{
get {return objID;}
set {objID = value;}
}
}
//implement sales and orders similarly
Once that is done, you can define a constraint on the types allowed by the method. Now that the type definitely has an ObjID property, we can query on that:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query) where T : IHasObjID
{
var intsToFind = FillObjectsToFind();
return query.Where(t => intsToFind.Contains(t.ObjID));
}
This is what King King was telling you in this comment.
I propose that when calling this function, you also pass in how to get at the integer from the type:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query, Expression<Func<T,int>> converter)
{
var intsToFind = FillObjectsToFind();
return query.Where(t => intsToFind.Contains(converter(t)));
}
However, I haven't tested this code, and since we are working with Entity Framework and expressions I suspect there is still an issue: An expression cannot be "called" within an expression.
I wanted to suggest the above, but it doesn't compile with the following error -- 'converter' is a 'variable' but used like a 'method'.
After all that, the solution is straightforward, using Join:
public static IQueryable<T> WhereFunctionContains<T>(this IQueryable<T> query, Expression<Func<T,int>> converter)
{
var ints = new List<int>() { 1, 2, 3, 4, 5 };
return query.Join(ints,converter,i=>i,(t,i) => t);
}
to be called like this:
var filteredPersons = query.WhereFunctionContains(p => p.PersonID);
If this is only used with a single type MyTable:
public static IQueryable<MyTable> WhereFunctionContains(this IQueryable<MyTable> query)
{
var ints = new List<int>() { 1, 2, 3, 4, 5 };
return query.Join(ints, mt=>mt.objid, i=>i, (t,i) => t);
}
Some links from the C# Programming Guide (in order of the answer):
Extension methods
Generics
Generic methods
Generic constraints
Interfaces
LINQ inner joins
Also, see here for a nice overview of LINQ operators, such as Select, Where, Count and ToList.
I want to take some elements by checking them with my custom function.
I have Person table:
public class Person
{
public int Id { get; set; }
public DateTime BirthDay { get; set; }
public string Name { get; set; }
...
}
I should to use my GetAge() and other functions to filter Persons list.
My following code doesnt work:
public List<Person> FilterPersons(int ageFrom, int ageTo...etc..)
{
var all = Database.Persons.AsQueryable();
all = from item in all
where GetAge(item.BirthDay) > ageFrom
select item;
all = from item in all
where GetAge(item.BirthDay) < ageTo
select item;
// other operations
...
}
I think I can write so. In every step to do this:
List<Person> newList = new List<Person>();
foreach (var item in all)
{
var itemAge = Common.GetAge(item.BirthDay);
if (itemAge > AgeFrom)
{
newList.Add(item);
}
}
all = newList.List();
But this is not best way I think, because I should do filter by many criteries. It will work with low speed.
How can I use my functions in Linq query?
Edit:
I showed GetAge() function for example. I have many functions like that. I wanted to know how to use my function.
Well, you can't.
If you want to have criteria used in Where clause of your SQL query, you need to write them directly as a linq.Expression so that entity may parse it and transform it into SQL, not an external function.
Somthing like this works :
DateTime date = DateTime.Now.AddDays(ageFrom);
all = from item in all
where item.BirthDay > date
select item;
Query Expressions are built in to the C# compiler and as such, it only understands the expression that are built in to the compiler.
For example, when you use the where keyword, it converts that to a call to the Where<TSource>(this IQueryable<TSource> source, Func<TSource, bool> predicate) method.
This is true of Linq To Objects and Linq To SQL. What's more, with Linq To SQL, the compiler then has to convert the Query Expression to SQL, which has no way of knowing the definition of your GetAge method.
Or you can use this syntax:
DateTime date = DateTime.Now.AddDays(ageFrom);
all = item.Where(x => x.BirthDay > date).ToList();
Why not use a List<Person>.FindAll() method and pass in a method filter as the predicate?
You would use the method like this.
List<Person> filteredPersons = allPersons.FindAll(FilterPersons);
Below is the a sample method you would use as your filter.
bool FilterPersons(Person p)
{
if(//enter criteria here to determine if you want to select the person)
return true;
else
return false;
}
To do what you want this may be the code you need.
bool FilterPersons(Person p)
{
var itemAge = Common.GetAge(item.BirthDay);
if( itemAge > AgeFrom )
return true;
else
return false;
}
Assuming you can apply filters on the result:
You can apply normal filters ( in linq expressions ) and than apply your functions on the result. Of course, you need to refactor your methods.
Something like this :
var result= Users.Where(s=>s.Name).ToList();
result= MyFilter(result);
I am trying to build a generic query mechanism to access my repository. I wish to use Lambda expressions to filter and sort the query. I am struggling to find a way to pass a list of generic Lambda expressions in, specifically for the order-by, and would appreciate help in doing so.
EDIT: 2 requirements I am trying to meet is, not expose IQueryable beyond the repository, but still be able to carry out some filtering and sorting at database level.
To better illustrate this let me show you the code
public class Query<T>
{
public class OrderBy<T>
{
public Expression<Func<T, **int**>> Clause { set; get; } // Order By clause
public bool Descending = true;
}
public Expression<Func<T, bool>> Where { set; get; } // Where clause
public IList<OrderBy<T>> OrderBys { set; get; } // Where clause
public Query()
{
OrderBys = new List<OrderBy<T>>();
}
}
public IEnumerable<Person> FindBy(Query<Person> query)
{
IQueryable<Person> Temp = GetObjectSet();
if (query.Where != null)
Temp = Temp.Where(query.Where);
foreach (var OrderByThis in query.OrderBys)
{
if (OrderByThis.Descending)
Temp = Temp.OrderByDescending(OrderByThis.Clause);
else
Temp = Temp.OrderBy(OrderByThis.Clause);
}
return Temp.ToList<Person>();
}
This is all very nice, BUT Expression< Func< T, int>> is not generic. I need to be able to do something like:
Query<Person> query = new Query<Person>();
Query<Person>.OrderBy<Person> clause1 = new Query<Person>.OrderBy<Person>();
clause1.Clause = m => m.Username;
Query<Person>.OrderBy<Person> clause2 = new Query<Person>.OrderBy<Person>();
clause2.Clause = m => m.DateOfBirth;
query.OrderBys.Add(clause1);
query.OrderBys.Add(clause2);
i.e. adding a number of different fields of different types.
I imagine there must be a way to store these as generic Lambda functions, and then in the repository convert then to the strongly typed Lambda function it needs.
How can I do this?
As I noted in my answer to your other question, I would discourage this approach in general. It makes more sense just to expose IQueryable<T>/IOrderedQueryable<T>.
That being said, there is a solution along the lines of your intention available in the selected answer to How to pass multiple Expressions to OrderBy for EF? .
It allows you to use a syntax like:
var query = context.Users ... ;
var queryWithOrderBy = ApplyOrderBy(query,
new OrderByExpression<User, string>(expression: u => u.UserName, descending: false), // a string, asc
new OrderByExpression<User, int>(expression: u => u.UserId, descending: true)); // an int, desc
var result = queryWithOrderBy.ToList(); // didn't throw an exception for me
Elaborating on my comment, I don't see why you need to construct your own intermediate query object out of Expressions and then reconstruct Expressions from that intermediate object, when you could just skip that translation altogether.
Given your example query:
repository.FindBy(people => people.OrderBy(p => p.Username).ThenBy(p => p.DateOfBirth));
Take note that you can still build up the queries incrementally, if it is being done based on user selections, for example. The following query is equivalent to the above:
Func<IEnumerable<Person>, IEnumerable<Person>> query = people => people.OrderBy(p => p.Username);
query = query.ThenBy(p => p.DateOfBirth);
I understand that you don't want to expose IQueryable beyond the repository, but you can still use LINQ with a signature such as:
public IEnumerable<Person> FindBy(Func<IEnumerable<Person>, IEnumerable<Person>> query)
{
return query(GetObjectSet()).ToList();
}
Speaking to your actual question, however, you can achieve your OrderBy task by using Expression<Func<T, object>> for the Clause property type, or if that unsettles you, you could constrain it a bit more by using IComparable instead of object, as it is really all you need for ordering, and strings and numeric types all implement it.