Develop Reference

How can I build a linq query at runtime with multiple group by clauses with different types

I'm in a situation where I need to specify queries on a EF-context at runtime. Our consultants configure these queries at a customer site for customer specific situations.
In order to facilitate that I was thinking of using linq to build the queries, based on a criteria-list which the consultants specify in a front-end of some kind (right now, winforms). The consultants basically specify a property from an object, specify the operator and then the value. For example: give me all clients where the [status] [equals] [1].
At the moment I have an expression builder which creates the where clause at runtime, and so far I can manage one group by clause. Where I'm running against the wall here, is when a consultant configures multiple group by clauses of different types (f.e. a string and a datetime property).
For example, I must be able to process this query: select bsn as a, dateofbirth as b from clients where status = 1 group by bsn, dateofbirth (where bsn = string and dateofbirth = datetime).
At the moment, this is the code which "glues" the query together:
public List<ClientV2> ExportClients(List<CriteriaV2> criteriaList)
{
var whereExpression = BuildWhereExpressionChain(criteriaList.Where(c => c.Operator != CriteriaOperatorV2.GROUPBY).ToList());
var groupByExpression = BuildGroupByExpression(criteriaList.Where(c => c.Operator == CriteriaOperatorV2.GROUPBY).ToList());
var sourceClients = _context.Clients.Where(whereExpression).GroupBy(groupByExpression).ToList();
IEnumerable<Client> resultClients = sourceClients.SelectMany(group => group);
return ClientToClientV2.MapList(resultClients.ToList());
}
This is the where-clause builder:
private Expression<Func<Client, bool>> BuildWhereExpressionChain(List<CriteriaV2> criteriaList)
{
var expressionList = new List<Expression<Func<Client, bool>>>();
var paramExp = Expression.Parameter(typeof(Client));
foreach (var crit in criteriaList)
{
var propertyItem = PropertyTranslator.GetPropertyItem(crit.Property);
if (propertyItem == null) throw new InvalidFilterCriteriaException("Property " + crit.Property + " niet toegestaan als filter criterium");
var propInfo = typeof(Client).GetProperty(propertyItem.InternalName);
var left = Expression.Property(paramExp, propInfo);
Expression right;
if (propInfo.PropertyType.IsEnum)
right = Expression.Constant(Enum.ToObject(propInfo.PropertyType, PropertyTranslator.TranslateEnum(propertyItem.Type, crit.Value)));
else if (propInfo.PropertyType == typeof(DateTime) || propInfo.PropertyType == typeof(DateTime?))
right = Expression.Constant(DateTime.Parse(crit.Value), propInfo.PropertyType);
else
right = Expression.Constant(crit.Value, typeof(string));
var exp = BuildExpression(left, right, crit.Operator);
expressionList.Add(Expression.Lambda<Func<Client, bool>>(exp, new ParameterExpression[] { paramExp }));
}
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.And(ex); });
return firstExpression;
}
And this is the part where I am stuck (it does work for one clause of type string):
private Expression<Func<Client, string>> BuildGroupByExpression(List<CriteriaV2> criteriaList)
{
var expressionList = new List<Expression<Func<Client, string>>>();
var paramExp = Expression.Parameter(typeof(Client));
foreach (var crit in criteriaList)
{
var propertyItem = PropertyTranslator.GetPropertyItem(crit.Property);
if (propertyItem == null) throw new InvalidFilterCriteriaException("Property " + crit.Property + " niet toegestaan als group by criterium");
var propInfo = typeof(Client).GetProperty(propertyItem.InternalName);
var body = Expression.Property(paramExp, propInfo);
var lambda = Expression.Lambda<Func<Client, string>>(body, paramExp);
expressionList.Add(lambda);
}
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.And(ex); });
return firstExpression;
}
Would it be possible to make the BuildGroupByExpression() in such way that it results in an expression which contains multiple clauses of different types that I can use directly in .GroupBy(expression);?
I'm not that much of an expert on linq, but I have a feeling that what I want could be possible. If I do stupid things here, please point them out and I'll work on it.

I dont think so. At least using the approach where the Expression is built they way you have. At least I ended up building expressions per type.
The main reason is
var lambda = Expression.Lambda<Func<Client, string>>(body, paramExp);
I dont know how you can make this Lambda defintion dynamic or generic.
There is a different approach, and a library that uses a string interpretation at runtime to build expressions. See
Install-Package System.Linq.Dynamic.Library
see also https://msdn.microsoft.com/en-US/vstudio/bb894665.aspx

How to inner join(filtering for nested properties) with expression tree?

My users should be able to configure a filter to get a result from the database.
I decided to use an expression tree, to be flexible with the query. But I have problems with the expression tree for filtering by ReferenceCode.
Entities:
PumpStartUp
Duration (TimeSpan)
DutDbId (Guid/FK)
TimeStamp (DateTime)
DUT (DeviceUnderTest)
DeviceUnderTest
AdminId (string/unique)
DbId (Guid/PK)
ReferenceCode (string)
StartUps (List)
Here is a part of the filter in equivalent linq. But linq I can't use, because I don't know how many ReferenceCodes will be defined by the user.:
//-------------------reference code filter---------------------------
var query = context.PumpStartUps.Where((p => p.DUT.ReferenceCode == "HKR566" ||
p.DUT.ReferenceCode == "HSH967" ||
.
.
.));
startUps = query.ToList();
For filtering by DeviceUnderTest part of the filter, my solution is:
// --------------------duts filter-----------------------------------
Expression dutsExpression = null;
Expression psuExpression = Expression.Parameter(typeof(PumpStartUp), "psu");
Expression psuDutIdExpression = Expression.Property(psuExpression, "DutDbId");
foreach (var dut in filter.Duts)
{
DeviceUnderTest deviceUnderTest = context.DevicesUnderTest.Where(d => d.AdminId == dut.Id).Single();
Expression dutIdExpression = Expression.Constant(deviceUnderTest.DbId);
Expression dutExpression = Expression.Equal(pumpStartUpDutIdExpression, dutIdExpression);
if (dutsExpression == null)
{
dutsExpression = dutExpression;
}
else
{
dutsExpression = Expression.Or(dutsExpression, dutExpression);
}
}
How can I filter by ReferenceCode in that manner:

Use this:
var dutExpression = Expression.Property(psuExpression, "DUT");
var referenceCodeExp = = Expression.Property(dutExpression, "ReferenceCode ");
var constExpr = Expression.Constant("HKR566");
var eqExp = Expression.Equal(referenceCodeExp , constExpr);
dutsExpression = Expression.Or(dutsExpression, eqExp);

If you have a limited amount of codes, you can always say
var query = context.PumpStartUps.Where(p => codes.Contains(p.DUT.ReferenceCode))
This works up until about 2000 parameters. If you need more, then you should probably send the codes somehow to a temp table (or rather a function returning a table, since ef does not support temp tables), and join on that since constructing an expression with more than 2000 ors is not gonna perform well.

Construct LINQ query using variables in asp.net WebAPI

I am trying to build a method in my asp.net WebAPI to grab data based on the arguments passed on the method. The method is used to perform a search on restaurant data. I have a variable called 'type' that determines the type of data search performed. The second variable 'keyword' is the keyword searched by the user. The WHERE condition in my LINQ query depends on the type and needs to be dynamic, so I have used a separate variable outside the LINQ query to define the condition. I have tried assigning this variable to my WHERE statement on the LINQ query but it doesn't seem to work. Can someone help with it please? I have been stuck on this for a few days now
public IQueryable<RestaurantView> GetRestaurantsForSearch(string keyword, int type, string location)
{
//
var condition = "";
if(type == 1)
{
condition = "x.RestaurantName.Contains(keyword)";
} else if(type == 2){
condition = "x.Cuisine.Equals(keyword)";
}
else {
condition = "x.Rating.Equals(keyword)";
}
var query = from x in db.Restaurants
join y in db.Cuisine on x.RestaurantCuisine equals y.CuisineID
where condition
select new RestaurantView
{
RestaurantID = x.RestaurantID,
RestaurantName = x.RestaurantName,
RestaurantCuisine = y.CuisineName,
RestaurantDecription = x.RestaurantDecription
};
return query;
}

Try this:
Predicate<Restaurant> pred;
if (type == 1) pred = x => x.RestaurantName.Contains(keyword);
else if (type == 2) pred = x => x.Cuisine.Equals(keyword);
else pred = x => x.Rating.Equals(keyword);
var query = from x in db.Restaurants
join y in db.Cuisine on x.RestaurantCuisine equals y.CuisineID
where pred(x)
select new RestaurantView
{
RestaurantID = x.RestaurantID,
RestaurantName = x.RestaurantName,
RestaurantCuisine = y.CuisineName,
RestaurantDecription = x.RestaurantDecription
};
return query;

You need to look a dynamic linq library i think then you can execute string statements inside your linq
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
or you can execute direct query
http://msdn.microsoft.com/en-us/library/system.data.linq.datacontext.executequery.aspx

If you are ok with dropping your comprehensive LINQ query in favour of the extension method syntax, it's pretty simple (I'm on a netbook without VS, so I apologize that this is untested but should give you the idea):
var query = db.Restaurants
.Include("Cuisine")
if(type == 1)
{
query= query.Where(x => x.RestaurantName.Contains(keyword));
}
else if(type == 2)
{
query = query.Where(x => x.Cuisine == keyword);
}
else {
query = query.Where(x => x.Rating == keyword);
}
This builds out your expression tree differently based on your logic checks, which will result in a different SQL query being generated based on the value of type.
I notice that in your join, Cuisine appears to be an Entity, but in your logic checks, you attempt to filter by comparing Cuisine to a string so I think there is some disconnect.

var query = from x in db.Restaurants
join y in db.Cuisine on x.RestaurantCuisine equals y.CuisineID
where condition
select new RestaurantView
{
RestaurantID = x.RestaurantID,
RestaurantName = x.RestaurantName,
RestaurantCuisine = y.CuisineName,
RestaurantDecription = x.RestaurantDecription
};
return query;
}
how to get the return query value in client side to assign for grid view binding

Dynamically Append LINQ Join

I have a method in my data layer in which I pass in a set of search parameters, dynamically build up the required 'where' clause using the PredicateBuilder class from LINQKit and then execute the LINQ query to return a list of objects.
In some cases the search criteria may contain a category name which doesn't exist in the Library table, I then need to join to this table to perform the necessary query. The reason it's been done this way is because there are potentially hundreds of categories per book and for optimisation reasons I only want the join to be performed if actually required.
Now my question is, is it possible to dynamically append this join to the LINQ query?
After a few attempts at trying to get this to work I've unfortunately had to resort to the 'cut-and-paste' pattern which I dislike immensely but needed something that worked for it to get 'out-the-door'.
The code below is an extract with variables renamed of what I've currently got (i.e. this isn't really a library application!):
public IEnumerable<ILibrarySearchResultsDTO> SearchLibrary(ISearchLibrary searchValues)
{
var whereStatement = PredicateBuilder.True<Library>();
bool categorySearch = false;
if (!string.IsNullOrEmpty(searchValues.AuthorFirstName))
whereStatement = whereStatement.And(q => q.AuthorFirstName == searchValues.AuthorFirstName);
if (!string.IsNullOrEmpty(searchValues.AuthorLastName))
whereStatement = whereStatement.And(q => q.AuthorLastName == searchValues.AuthorLastName);
if (!string.IsNullOrEmpty(searchValues.CategoryName))
categorySearch = true;
var libraryObjectSet = Context.CreateObjectSet<Library>();
LibraryObjectSet.MergeOption = MergeOption.NoTracking;
var categoriesObjectSet = Context.CreateObjectSet<Categories>();
categoriesObjectSet.MergeOption = MergeOption.NoTracking;
if (!categorySearch)
{
var query = from lib in libraryObjectSet
.Where(whereStatement)
.Take(ConfigHelper.MaxQueryRecords)
.AsExpandable()
select new LibrarySearchResultsDTO()
{
BookName = lib.BookName,
AuthorFirstName = lib.AuthorFirstName,
AuthorLastName = lib.AuthorLastName,
ISBN = lib.ISBN
};
}
else
{
var query = from lib in LibraryObjectSet
.Where(whereStatement)
.Take(ConfigHelper.MaxQueryRecords)
.AsExpandable()
join categories_LKP in categoriesObjectSet on new { CategoryID = lib.CategoryID, CategoryName = searchValues.CategoryName }
equals new { CategoryID = categories_LKP.CategoryID, CategoryName = categories_LKP.CategoryName }
select new LibrarySearchResultsDTO()
{
BookName = lib.BookName,
AuthorFirstName = lib.AuthorFirstName,
AuthorLastName = lib.AuthorLastName,
ISBN = lib.ISBN
};
}
return query.ToList();
}
I've had to create the sample code in Notepad++ and because it's a contrived example I haven't been able to check if it compiles. Should do though (I hope!).

If there is a navigation property from Library to Category you can just dynamically add another predicate:
if (!string.IsNullOrEmpty(searchValues.CategoryName))
{
whereStatement = whereStatement
.And(q => q.Categories
.Any(c => c.CategoryName == searchValues.CategoryName));
}
If the navigation property is not present, you can still use a join in the predicate without having to duplicate the whole query. But it could be a good reason to add a navigation property.

You may use Reflection API like a following generic function...which compiles a dynamic query with a unknown type...
IQueryable<T> getQuery<T>(T myTableEntity, string[] arrayOfQueryTerms, Expression<Func<T, bool>> predicate)
{
var fieldOrProperty = getMemberInfo(predicate);
}
MemberInfo getmemberInfo<T>(Expression<Func<T,bool> expr)
{
var memberExpr = expr as MemberExpression;
if (memberExpr != null) return memberExpr.Member;
throw new ArgumentException();
}
var q = getQuery<FooTable>(foo, new[]{"Bar","Baz"}, x=>x.FieldName);

LINQ Where in collection clause

I've been looking on google but not finding anything that does the trick for me.
as you know SQL has a "where x in (1,2,3)" clause which allows you to check against multiple values.
I'm using linq but I can't seem to find a piece of syntax that does the same as the above statement.
I have a collection of category id's (List) against which I would like to check
I found something that uses the .contains method but it doesn't even build.

You have to use the Contains method on your id list:
var query = from t in db.Table
where idList.Contains(t.Id)
select t;

The syntax is below:
IEnumerable<int> categoryIds = yourListOfIds;
var categories = _dataContext.Categories.Where(c => categoryIds.Contains(c.CategoryId));
The key thing to note is that you do the contains on your list of ids - not on the object you would apply the in to if you were writing sql.

Here's an article illustrating the approach. You should indeed use the Contains method over your collection which will be translated into IN clause.

Here is my realization of WhereIn() Method, to filter IQueryable collection by a set of selected entities:
public static IQueryable<T> WhereIn<T,TProp>(this IQueryable<T> source, Expression<Func<T,TProp>> memberExpr, IEnumerable<TProp> values) where T : class
{
Expression predicate = null;
ParameterExpression param = Expression.Parameter(typeof(T), "t");
bool IsFirst = true;
MemberExpression me = (MemberExpression) memberExpr.Body;
foreach (TProp val in values)
{
ConstantExpression ce = Expression.Constant(val);
Expression comparison = Expression.Equal(me, ce);
if (IsFirst)
{
predicate = comparison;
IsFirst = false;
}
else
{
predicate = Expression.Or(predicate, comparison);
}
}
return predicate != null
? source.Where(Expression.Lambda<Func<T, bool>>(predicate, param)).AsQueryable<T>()
: source;
}
And calling of this method looks like:
IQueryable<Product> q = context.Products.ToList();
var SelectedProducts = new List<Product>
{
new Product{Id=23},
new Product{Id=56}
};
...
// Collecting set of product id's
var selectedProductsIds = SelectedProducts.Select(p => p.Id).ToList();
// Filtering products
q = q.WhereIn(c => c.Product.Id, selectedProductsIds);