Develop Reference

Inner method in Linq query that return IQueryable

I would like to do following thing in Entity Framework code first:
Data.GroupBy(x => x.Person.LastName)
.Select(x => x.OtherGrouping(...)).Where(...).GroupBy(...)...etc
public static class Extensions
{
public static IQueryable<IGrouping<T, T>> OtherGrouping<T>(this IQueryable<T> collection /**** Here I want to pass consts and lambdas\expressions*****/)
{
// Grouping, filtration and other stuff here
return collection.GroupBy(x => x);
}
}
The problem is .net compiles OtherGrouping method, it doesn't represent like an expression, and couldn't be transformed to the SQL.
I found LinqKit library which suppose to help in such cases, but I coudn't figure out how to apply it in my specific case. It works fine for simple cases, when I just have expression like x => x+2, but I get stucked with return IQueryable. Probably it is possible to write expression tree completely by hand but I don't want to go so deep.
Any ideas how it might be done or where I can read about it?
Here is what I tried to do based on Rob's comment
void Main()
{
AddressBases.GroupBy(x => x.ParentId).Select(x => new {x.Key, Items = x.AsQueryable().OtherGrouping(a => a.Country) }).Dump();
}
public static class Extensions
{
public static IQueryable<IGrouping<TKey, TSource>> OtherGrouping<TSource, TKey>(this IQueryable<TSource> source, Expression<Func<TSource, TKey>> keySelector)
{
return source.Provider.CreateQuery<IGrouping<TKey, TSource>>(
source.GroupBy(keySelector).Expression);
}
}
And I got an exception:
NotSupportedException: LINQ to Entities does not recognize the method 'System.Linq.IQueryable`1[System.Linq.IGrouping`2[System.String,InsuranceData.EF.AddressBase]] OtherGrouping[AddressBase,String](System.Linq.IQueryable`1[InsuranceData.EF.AddressBase], System.Linq.Expressions.Expression`1[System.Func`2[InsuranceData.EF.AddressBase,System.String]])' method, and this method cannot be translated into a store expression.
Couldn't figure out yet why I have OtherGrouping method in expression tree? OtherGrouping method should just attach another grouping to the expression and pass it to the provider, but not put itself to the tree.

Your current code after the edit is correct - where you're going wrong is a common issue with the syntactic sugar we're given by C#.
If you write the following:
void Main()
{
var res = Containers.OtherGrouping(c => c.ContainerID);
res.Expression.Dump();
res.Dump();
}
public static class Extensions
{
public static IQueryable<IGrouping<TKey, TSource>> OtherGrouping<TSource, TKey>(this IQueryable<TSource> source, Expression<Func<TSource, TKey>> keySelector)
{
return source.Provider.CreateQuery<IGrouping<TKey, TSource>>(
source.GroupBy(keySelector).Expression
);
}
}
You'll see that we get the output:
Table(Container).GroupBy(c => c.ContainerID)
And the result executes with an issue, properly grouping by our predicate. However, you're invoking OtherGrouping while inside an expression, when you're passing it to Select. Let's try a simple case without OtherGrouping:
var res = Containers.OtherGrouping(c => c.ContainerID)
.Select(x => new
{
x.Key,
Items = x.GroupBy(c => c.ContainerID),
});
res.Expression.Dump();
What you're providing to Select is an expression tree. That is, the inner GroupBy's method is never actually invoked. If you change the query to x.AsQueryable().OtherGrouping and put a breakpoint in OtherGrouping, it will only be hit the first time.
In addition to that, x is IEnumerable<T>, not IQueryable<T>. Invoking AsQueryable() gives you an IQueryable, but it also gives you a new query provider. I'm not 100% sure as to the inner workings of entity framework, but I'd wager to say that this is invalid - as we no longer have the database as a target for the provider. Indeed, with linq2sql (using LINQPad), we get an error saying .AsQueryable() is not supported.
So, what can you do? Unfortunately there's no clean way to do this. Since the expression tree is already built before OtherGrouping has a chance, it doesn't matter what the body of OtherGrouping is. We'll need to change the expression tree after it's built, but before it's executed.
For that, you'll need to write an expression visitor which will look for .OtherGrouping expression calls, and replace it with Queryable.GroupBy

EF6: Order by using anonymous type + Include

I am trying to execute the following query with EF6:
var contracts = ctx.Orders
.Include(c => c.OrderLines)
.Where(c => c.IdCompany == idCompany)
.OrderBy(x => new {OrderByColumn}) //OrderByColumn is a string
.ToPagedQuery(pageSize, pageNumber) //ordinary Skip and Take
.ToList();
but it throws this SqlException:
A column has been specified more than once in the order by list. Columns in the order by list must be unique.
The problem happens when I use Include together with the anonymous type in the OrderBy clause.
I know that anonymous types don't work well with Include on projections (Projecting to an Anonymous Type), but I don't know if this applies to OrderBy clauses.
Is there a way of doing that? If not, why it doesn't work?
Thanks

I think your order by part should be like this:
.OrderBy(x => x.OrderByColumn)

I've found an extension method that receives a string instead of an anonymous type (dynamic-linq-orderby-using-string-names). Problem solved.

Did you mix projection and sort.
Orderby doesnt not expect an anonymous type. The Select Can use an anonymous type
// context DbSet.....
.Where(w=>w.Id == "return true") // an expression that returns true
.OrderBy(o=>o.Name) // expression that returns a prop Name not anonymous type
.Select(s=> new {s.Id, s.Name}); // New type with 2 fields
The Orderby Signature
public static IOrderedQueryable<TSource> OrderBy<TSource, TKey>(this IQueryable<TSource> source, Expression<Func<TSource, TKey>> keySelector)
The Select Signature
public static IQueryable<TResult> Select<TSource,TResult>(this IQueryable<TSource> source, Expression<Func<TSource, TResult>> selector)

Building a Linq Query from a List<string> using dynamic linq

I am using dynamic linq to preocess some user requests. The way it was set up is that I would gather the data into Var data
var data = Project.Model.Adhoc.GetData().AsQueryable();
This is basically a select * from a view
Then from there I would loop through all of the options that I have to filter that the user selected
foreach (Filters filter in filters.OrderBy(x=>x.strOrderNumber))
{
along with some checks and permutations, I get down to this
data = data.Where(filter.strFilter + FormatOperator(filter.strOperator) + "#0", filter.strValue).
Select(x => x).ToList().AsQueryable();
This is working pretty well, however the datasource is starting to grow, so what I would like to do is something like this:
data = data.select(get all items that were selected) and then do my checks and permutations. This would allow me to only pull what is needed, not the entire datasource. What is the best way in C# using linq to accomplish this.
Ex.
datasource = {Name, Age, Race, Gender, Hair Color, Eye Color, height, weight, etc}
user selected = {Name, Age, Race, Gender}
Instead of querying against that whole datasource, I want to limit the datasource to only what is brought in by the user off the bat, and then I can filter based on that as teh datasource

Take a look at Dynamic Linq
You can use the DynamicQuery library against any LINQ data provider
(including LINQ to SQL, LINQ to Objects, LINQ to XML, LINQ to
Entities, LINQ to SharePoint, LINQ to TerraServer, etc). Instead of
using language operators or type-safe lambda extension methods to
construct your LINQ queries, the dynamic query library provides you
with string based extension methods that you can pass any string
expression into.

Remove the .ToList() call in the foreach loop.
data = data.Where() will build a query expression with ANDs. So after the loop you can finally invoke .ToList() to finally hit the database.
Update
And the .Select() isn't necessary.
data = data.Where(filter.strFilter + FormatOperator(filter.strOperator) + "#0", filter.strValue);
Update2
Oh, after reading your question again I get that you need to build the query using OR.
This is a little bit more difficult using the standard library. If you don't mind pulling in a extra dependency then it can (probably) be done using LinqKit
IQueryable<Product> SearchProducts (params string[] keywords)
{
var predicate = PredicateBuilder.False<Product>();
foreach (string keyword in keywords)
{
string temp = keyword;
predicate = predicate.Or (p => p.Description.Contains (temp));
}
return dataContext.Products.Where (predicate);
}
Although I am not sure how well that works together with Dynamic Linq.
Otherwise you'll have to handcraft the expression, which might end up looking similar to this:
public static class IQueryableExtensions
{
public static IQueryable<T> WhereIn<T, TValue>(
this IQueryable<T> source,
Expression<Func<T, TValue>> propertySelector,
IEnumerable<TValue> values)
{
return source.Where(GetWhereInExpression(propertySelector, values));
}
private static Expression<Func<T, bool>> GetWhereInExpression<T, TValue>(
Expression<Func<T, TValue>> propertySelector, IEnumerable<TValue> values)
{
if (!values.Any())
return c => false;
ParameterExpression p = propertySelector.Parameters.Single();
// You'll have to adjust this:
var equals = values.Select(value => (Expression)Expression.Equal(
propertySelector.Body, Expression.Constant(value, typeof(TValue))));
var body = equals.Aggregate<Expression>(
(accumulate, equal) => Expression.Or(accumulate, equal));
return Expression.Lambda<Func<T, bool>>(body, p);
}
}

Intersect LINQ query

If I have an IEnumerable where ClassA exposes an ID property of type long.
Is it possible to use a Linq query to get all instances of ClassA with ID belonging to a second IEnumerable?
In other words, can this be done?
IEnumerable<ClassA> = original.Intersect(idsToFind....)?
where original is an IEnumerable<ClassA> and idsToFind is IEnumerable<long>.

Yes.
As other people have answered, you can use Where, but it will be extremely inefficient for large sets.
If performance is a concern, you can call Join:
var results = original.Join(idsToFind, o => o.Id, id => id, (o, id) => o);
If idsToFind can contain duplicates, you'll need to either call Distinct() on the IDs or on the results or replace Join with GroupJoin (The parameters to GroupJoin would be the same).

I will post an answer using Intersect.
This is useful if you want to intersect 2 IEnumerables of the same type.
First we will need an EqualityComparer:
public class KeyEqualityComparer<T> : IEqualityComparer<T>
{
private readonly Func<T, object> keyExtractor;
public KeyEqualityComparer(Func<T, object> keyExtractor)
{
this.keyExtractor = keyExtractor;
}
public bool Equals(T x, T y)
{
return this.keyExtractor(x).Equals(this.keyExtractor(y));
}
public int GetHashCode(T obj)
{
return this.keyExtractor(obj).GetHashCode();
}
}
Secondly we apply the KeyEqualityComparer to the Intersect function:
var list3= list1.Intersect(list2, new KeyEqualityComparer<ClassToCompare>(s => s.Id));

You can do it, but in the current form, you'd want to use the Where extension method.
var results = original.Where(x => yourEnumerable.Contains(x.ID));
Intersect on the other hand will find elements that are in both IEnumerable's. If you are looking for just a list of ID's, you can do the following which takes advantage of Intersect
var ids = original.Select(x => x.ID).Intersect(yourEnumerable);

A simple way would be:
IEnumerable<ClassA> result = original.Where(a => idsToFind.contains(a.ID));

Use the Where method to filter the results:
var result = original.Where(o => idsToFind.Contains(o.ID));

Naming things is important. Here is an extension method base on the Join operator:
private static IEnumerable<TSource> IntersectBy<TSource, TKey>(
this IEnumerable<TSource> source,
IEnumerable<TKey> keys,
Func<TSource, TKey> keySelector)
=> source.Join(keys, keySelector, id => id, (o, id) => o);
You can use it like this var result = items.IntersectBy(ids, item => item.id).

I've been tripping up all morning on Intersect, and how it doesn't work anymore in core 3, due to it being client side not server side.
From a list of items pulled from a database, the user can then choose to display them in a way that requires children to attached to that original list to get more information.
What use to work was:
itemList = _context.Item
.Intersect(itemList)
.Include(i => i.Notes)
.ToList();
What seems to now work is:
itemList = _context.Item
.Where(item => itemList.Contains(item))
.Include(i => i.Notes)
.ToList();
This seems to be working as expected, without any significant performance difference, and is really no more complicated than the first.

ascending/descending in LINQ - can one change the order via parameter?

I have a method which is given the parameter "bool sortAscending". Now I want to use LINQ to create sorted list depending on this parameter. I got then this:
var ascendingQuery = from data in dataList
orderby data.Property ascending
select data;
var descendingQuery = from data in dataList
orderby data.Property descending
select data;
As you can see, both queries differ only in "ascending" resp. "descending". I'd like to merge both queries, but I don't know how. Does anyone have the answer?

You can easily create your own extension method on IEnumerable or IQueryable:
public static IOrderedEnumerable<TSource> OrderByWithDirection<TSource,TKey>
(this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector,
bool descending)
{
return descending ? source.OrderByDescending(keySelector)
: source.OrderBy(keySelector);
}
public static IOrderedQueryable<TSource> OrderByWithDirection<TSource,TKey>
(this IQueryable<TSource> source,
Expression<Func<TSource, TKey>> keySelector,
bool descending)
{
return descending ? source.OrderByDescending(keySelector)
: source.OrderBy(keySelector);
}
Yes, you lose the ability to use a query expression here - but frankly I don't think you're actually benefiting from a query expression anyway in this case. Query expressions are great for complex things, but if you're only doing a single operation it's simpler to just put that one operation:
var query = dataList.OrderByWithDirection(x => x.Property, direction);

In terms of how this is implemented, this changes the method - from OrderBy/ThenBy to OrderByDescending/ThenByDescending. However, you can apply the sort separately to the main query...
var qry = from .... // or just dataList.AsEnumerable()/AsQueryable()
if(sortAscending) {
qry = qry.OrderBy(x=>x.Property);
} else {
qry = qry.OrderByDescending(x=>x.Property);
}
Any use? You can create the entire "order" dynamically, but it is more involved...
Another trick (mainly appropriate to LINQ-to-Objects) is to use a multiplier, of -1/1. This is only really useful for numeric data, but is a cheeky way of achieving the same outcome.

What about ordering desc by the desired property,
blah = blah.OrderByDescending(x => x.Property);
And then doing something like
if (!descending)
{
blah = blah.Reverse()
}
else
{
// Already sorted desc ;)
}
Is it Reverse() too slow?

In addition to the beautiful solution given by #Jon Skeet, I also needed ThenBy and ThenByDescending, so I am adding it based on his solution:
public static IOrderedEnumerable<TSource> ThenByWithDirection<TSource, TKey>(
this IOrderedEnumerable<TSource> source,
Func<TSource, TKey> keySelector,
bool descending)
{
return descending ?
source.ThenByDescending(keySelector) :
source.ThenBy(keySelector);
}