Develop Reference

Linq: Extension method on IEnumerable to automatically do null-checks when performing selects

When performing a Select on an IEnumerable I believe it is good practice to check for null references, so I often have a Where before my Select like this:
someEnumerable.Where(x => x != null).Select(x => x.SomeProperty);
This gets more complicated when accessing sub-properties:
someEnumerable.Where(x => x != null && x.SomeProperty != null).Select(x => x.SomeProperty.SomeOtherProperty);
To follow this pattern I need to do a lof of calls to Where. I would like to create an extension method on IEnumerable that perform such null-checks automatically depending on what is referenced in the Select. Like this:
someEnumerable.SelectWithNullCheck(x => x.SomeProperty);
someEnumerable.SelectWithNullCheck(x => x.SomeProperty.SomeOtherProperty);
Can this be done? Is it fx. possible to retrieve the selected properties from the selector parameter when creating an extension method such as this?
public static IEnumerable<TResult> SelectWithNullCheck<TSource, TResult>(this IEnumerable<TSource> source, Func<TSource, TResult> selector)
{
return source.Where(THIS IS WHERE THE AUTOMATIC NULL-CHECKS HAPPEN).Select(selector);
}
EDIT: I use C# 5.0 with .NET Framework 4.5

As you are using C# 5.0 you can write your extension method the following way:
public static IEnumerable<TResult> SelectWithNullCheck<TSource, TResult>(
this IEnumerable<TSource> source,
Func<TSource, TResult> selector)
{
return source.Where(x => x != null).Select(selector).Where(x => x != null);
}
Before and after the projection (Select call) to apply a check that the result is not null.
Then usage will be:
someEnumerable.SelectWithNullCheck(x => x.SomeProperty)
.SelectWithNullCheck(y => y.SomeOtherProperty);
Notice that the type of item in each call is different.
If you do want it similar to this:
someEnumerable.SelectWithNullCheck(x => x.SomeProperty.SomeOtherProperty);
Then you'd need to go with #Treziac's suggestion and use the ?. operator (introduced in C# 6.0) and then filter nulls out:
public static IEnumerable<TResult> SelectWithNullCheck<TSource, TResult>(this IEnumerable<TSource> source, Func<TSource, TResult> selector)
{
return source.Select(selector).Where( x=> x != null);
}
someEnumerable.SelectWithNullCheck(x => x?.SomeProperty?.SomeOtherProperty);

You can use Expression based solution. Below is basic and workable solution for field / property chain call. It will work for very deep call chains. It is not perfect. For example it won't work if there is method call in the chain (obj.Prop1.MethodCall().Prop2).
Expression based solutions are in general slower, because of the need to compile the lambda expression to delegate, which should be taken in consideration.
Performance stats:
Testes with collection of 200k objects with nested call level of 2 (obj.Prop1.Prop2) where all objects fail for the condition.
LINQ Where with C# 6 ?. operator : 2 - 4 ms
Exception based (try / catch): 14,000 - 15,000 ms
Expression based: 4 - 10 ms
NOTE: The expression based solution will add overhead of several ms for every call, this number won't depend on the collection size, because the expression will be compiled for every call which is an expensive operation. You can think for cache mechanism if you are interested.
Source for expression based solution::
public static IEnumerable<T> IgnoreIfNull<T, TProp>(this IEnumerable<T> sequence, Expression<Func<T, TProp>> expression)
{
var predicate = BuildNotNullPredicate(expression);
return sequence.Where(predicate);
}
private static Func<T, bool> BuildNotNullPredicate<T, TProp>(Expression<Func<T, TProp>> expression)
{
var root = expression.Body;
if (root.NodeType == ExpressionType.Parameter)
{
return t => t != null;
}
var pAccessMembers = new List<Expression>();
while (root.NodeType == ExpressionType.MemberAccess)
{
var mExpression = root as MemberExpression;
pAccessMembers.Add(mExpression);
root = mExpression.Expression;
}
pAccessMembers.Reverse();
var body = pAccessMembers
.Aggregate(
(Expression)Expression.Constant(true),
(f, s) =>
{
if (s.Type.IsValueType)
{
return f;
}
return Expression.AndAlso(
left: f,
right: Expression.NotEqual(s, Expression.Constant(null))
);
});
var lambda = Expression.Lambda<Func<T, bool>>(body, expression.Parameters[0]);
var func = lambda.Compile();
return func;
}
This is how is used:
var sequence = ....
var filtered = sequence.IgnoreIfNull(x => x.Prop1.Prop2.Prop3 ... etc);

Why not using ?. operator ?
someEnumerable.Where(x => x?.SomeProperty != null).Select(x => x.SomeProperty.SomeOtherProperty);
(note that this may return null values)
or
someEnumerable.Select(x => x?.SomeProperty?.SomeOtherProperty).Where(x => x != null);
(this will not return any null values)
It's not really good or bad practice, it depends of what you want in your return

Another option is to split the selection null check into a custom operator (e.g. WhereNotNull). Combine this with the ?. operator, solves your problem in an imho very expressive way.
public static IEnumerable<TSource> WhereNotNull<TSource>(this IEnumerable<TSource> source)
{
return source.Where(x=> x != null);
}
This allows you to write:
someEnumerable.Select(x => x?.SomeProperty?.SomeOtherProperty)
.WhereNotNull();
If not you could always chain the selects (for version prior to C# 6):
someEnumerable.Select(x => x.SomeProperty)
.Select(x => x.SomeOtherProperty)
.WhereNotNull();
Given you absolutely want to access x.SomeProperty.SomeOtherProperty the last option would be to catch the NullReferenceException.
public static IEnumerable<TResult> SelectWithNullCheck<TSource, TResult>(this IEnumerable<TSource> source, Func<TSource, TResult> selector)
{
return source.Select(x =>
{
try
{
return selector(x);
}
catch(NullReferenceException ex)
{
return default(TResult);
}
})
.Where(x=> default(TResult) != x);
}

Why doesn't IOrderedEnumerable retain order after where filtering

I've created a simplification of the issue. I have an ordered IEnumerable, I'm wondering why applying a where filter could unorder the objects
This does not compile while it should have the potential to
IOrderedEnumerable<int> tmp = new List<int>().OrderBy(x => x);
//Error Cannot Implicitly conver IEnumerable<int> To IOrderedEnumerable<int>
tmp = tmp.Where(x => x > 1);
I understand that there would be no gaurenteed execution order if coming from an IQueryable such as using linq to some DB Provider.
However, when dealing with Linq To Object what senario could occur that would unorder your objects, or why wasn't this implemented?
EDIT
I understand how to properly order this that is not the question. My Question is more of a design question. A Where filter on linq to objects should enumerate the give enumerable and apply filtering. So why is that we can only return an IEnumerable instead of an IOrderedEnumerable?
EDIT
To Clarify the senario in when this would be userful. I'm building Queries based on conditions in my code, I want to reuse as much code as possible. I have a function that is returning an OrderedEnumerable, however after applying the additional where I would have to reorder this even though it would be in its original ordered state

Rene's answer is correct, but could use some additional explanation.
IOrderedEnumerable<T> does not mean "this is a sequence that is ordered". It means "this is a sequence that has had an ordering operation applied to it and you may now follow that up with a ThenBy to impose additional ordering requirements."
The result of Where does not allow you to follow it up with ThenBy, and therefore you may not use it in a context where an IOrderedEnumerable<T> is required.
Make sense?
But of course, as others have said, you almost always want to do the filtering first and then the ordering. That way you are not spending time putting items into order that you are just going to throw away.
There are of course times when you do have to order and then filter; for example, the query "songs in the top ten that were sung by a woman" and the query "the top ten songs that were sung by a woman" are potentially very different! The first one is sort the songs -> take the top ten -> apply the filter. The second is apply the filter -> sort the songs -> take the top ten.

The signature of Where() is this:
public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate)
So this method takes an IEnumerable<int> as first argument. The IOrderedEnumerable<int> returned from OrderBy implements IEnumerable<int> so this is no problem.
But as you can see, Where returns an IEnumerable<int> and not an IOrderedEnumerable<int>. And this cannot be casted into one another.
Anyway, the object in that sequence will still have the same order. So you could just do it like this
IEnumerable<int> tmp = new List<int>().OrderBy(x => x).Where(x => x > 1);
and get the sequence you expected.
But of course you should (for performance reasons) filter your objects first and sort them afterwards when there are fewer objects to sort:
IOrderedEnumerable<int> tmp = new List<int>().Where(x => x > 1).OrderBy(x => x);

The tmp variable's type is IOrderedEnumerable.
Where() is a function just like any other with a return type, and that return type is IEnumerable. IEnumerable and IOrderedEnumerable are not the same.
So when you do this:
tmp = tmp.Where(x => x > 1);
You are trying to assign the result of a Where() function call, which is an IEnuemrable, to the tmp variable, which is an IOrderedEnumerable. They are not directly compatible, there is no implicit cast, and so the compiler sends you an error.
The problem is you are being too specific with the tmp variable's type. You can make one simple change that will make this all work by being just be a little less specific with your tmp variable:
IEnumerable<int> tmp = new List<int>().OrderBy(x => x);
tmp = tmp.Where(x => x > 1);
Because IOrderedEnumerable inherits from IEnumerable, this code will all work. As long as you don't want to call ThenBy() later on, this should give you exactly the same results as you expect without any other loss of ability to use the tmp variable later.
If you really need an IOrderedEnumerable, you can always just call .OrderBy(x => x) again:
IOrderedEnumerable<int> tmp = new List<int>().OrderBy(x => x);
tmp = tmp.Where(x => x > 1).OrderBy(x => x);
And again, in most cases (not all, but most) you want to get your filtering out of the way before you start sorting. In other words, this is even better:
var tmp = new List<int>().Where(x => x > 1).OrderBy(x => x);

why wasn't this implemented?
Most likely because the LINQ designers decided that the effort to implement, test, document etc. isn't worth enough compared to the potential use cases. In fact your are the first one I hear complaining about that.
But if it's so important to you, you can add that missing functionality yourself (similar to #Jon Skeet MoreLINQ extension library). For instance, something like this:
namespace MyLinq
{
public static class Extensions
{
public static IOrderedEnumerable<T> Where<T>(this IOrderedEnumerable<T> source, Func<T, bool> predicate)
{
return new WhereOrderedEnumerable<T>(source, predicate);
}
class WhereOrderedEnumerable<T> : IOrderedEnumerable<T>
{
readonly IOrderedEnumerable<T> source;
readonly Func<T, bool> predicate;
public WhereOrderedEnumerable(IOrderedEnumerable<T> source, Func<T, bool> predicate)
{
if (source == null) throw new ArgumentNullException(nameof(source));
if (predicate == null) throw new ArgumentNullException(nameof(predicate));
this.source = source;
this.predicate = predicate;
}
public IOrderedEnumerable<T> CreateOrderedEnumerable<TKey>(Func<T, TKey> keySelector, IComparer<TKey> comparer, bool descending) =>
new WhereOrderedEnumerable<T>(source.CreateOrderedEnumerable(keySelector, comparer, descending), predicate);
public IEnumerator<T> GetEnumerator() => Enumerable.Where(source, predicate).GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
}
}
And putting it into action:
using System;
using System.Collections.Generic;
using System.Linq;
using MyLinq;
var test = Enumerable.Range(0, 100)
.Select(n => new { Foo = 1 + (n / 20), Bar = 1 + n })
.OrderByDescending(e => e.Foo)
.Where(e => (e.Bar % 2) == 0)
.ThenByDescending(e => e.Bar) // Note this compiles:)
.ToList();

What is faster in finding element with property of maximum value [duplicate]

This question already has answers here:
How to use LINQ to select object with minimum or maximum property value
(20 answers)
Closed 10 months ago.
Commonly, to find element with property of max value I do like this
var itemWithMaxPropValue = collection.OrderByDescending(x => x.Property).First();
But is it good way from performance point of view? Maybe I should do something like this?
var maxValOfProperty = collection.Max(x => x.Property);
var itemWithMaxPropValue = collection
.Where(x => x.Property == maxValueOfProperty).First();

Sorting is N * log (N) while Max has N only time complexity, so Max is faster. What you're looking for is ArgMax function which Linq doesn't provide, so I suggest implementing it, e.g:
public static class EnumerableExtensions {
public static T ArgMax<T, K>(this IEnumerable<T> source,
Func<T, K> map,
IComparer<K> comparer = null) {
if (Object.ReferenceEquals(null, source))
throw new ArgumentNullException("source");
else if (Object.ReferenceEquals(null, map))
throw new ArgumentNullException("map");
T result = default(T);
K maxKey = default(K);
Boolean first = true;
if (null == comparer)
comparer = Comparer<K>.Default;
foreach (var item in source) {
K key = map(item);
if (first || comparer.Compare(key, maxKey) > 0) {
first = false;
maxKey = key;
result = item;
}
}
if (!first)
return result;
else
throw new ArgumentException("Can't compute ArgMax on empty sequence.", "source");
}
}
So you can put it simply
var itemWithMaxPropValue = collection
.ArgMax(x => x.Property);

Both solutions are not very efficient. First solution involves sorting whole collection. Second solution requires traversing collection two times. But you can find item with max property value in one go without sorting collection. There is MaxBy extension in MoreLINQ library. Or you can implement same functionality:
public static TSource MaxBy<TSource, TProperty>(this IEnumerable<TSource> source,
Func<TSource, TProperty> selector)
{
// check args
using (var iterator = source.GetEnumerator())
{
if (!iterator.MoveNext())
throw new InvalidOperationException();
var max = iterator.Current;
var maxValue = selector(max);
var comparer = Comparer<TProperty>.Default;
while (iterator.MoveNext())
{
var current = iterator.Current;
var currentValue = selector(current);
if (comparer.Compare(currentValue, maxValue) > 0)
{
max = current;
maxValue = currentValue;
}
}
return max;
}
}
Usage is simple:
var itemWithMaxPropValue = collection.MaxBy(x => x.Property);

I will go with Max since it is specifically designed for that purpose. Sorting to find Max value seems to be too much.
Also, I wouldn't use Where for finding the max, but Single - since what we need here is but a Single value.
var maxValOfProperty = collection.Max(x => x.Property);
var itemWithMaxPropValue = collection
.Single(x => x.Property == maxValueOfProperty);
Or alternatively using First (if the collection contains duplicates of max value)
var maxValOfProperty = collection.Max(x => x.Property);
var itemWithMaxPropValue = collection
.First(x => x.Property == maxValueOfProperty);
Or, using MoreLINQ (as suggested by Kathi), you could do it with MaxBy:
var itemWithMaxPropValue = collection.MaxBy(x => x.Property);
Check this post, on answer by Jon Skeet.

The maximum element under some specified function can also be found by means of the following two functions.
static class Tools
{
public static T ArgMax<T, R>(T t1, T t2, Func<T, R> f)
where R : IComparable<R>
{
return f(t1).CompareTo(f(t2)) > 0 ? t1 : t2;
}
public static T ArgMax<T, R>(this IEnumerable<T> Seq, Func<T, R> f)
where R : IComparable<R>
{
return Seq.Aggregate((t1, t2) => ArgMax<T, R>(t1, t2, f));
}
}
The solution above works as follows; the first overload of ArgMax takes a comparator as an argument which maps both instances of T to a type which implements comparability; a maximum of these is returned. The second overload takes a sequence as an argument and simply aggregates the first function. This is the most generic, framework-reusing and structurally sound formulation for maximum search I am aware of; searching the minimum can be implemented in the same way by changing the comparison in the first function.

I'm a little bit surprised that no one mentioned the Aggregate method. Aggregate lets you iterate a collection and return an aggregate value.
An ArgMax function can be implemented in this way:
var maxItem = collection.Aggregate((max, next) => next.Property.CompareTo(max.Property) > 0 ? next : max);
This function will iterate all over the collection and aggregate the item that has the largest Property. This implementation is O(N) which is good.
Please note that the Property getter (or the compared value in general) is called 2N times so don't do this when the value computation is heavy. You can avoid this with another iteration over the array or use the #Sergey Berezovskiy answer which suits all the cases.
But if you need it for simple values, this is a one-line efficient solution

LINQ to Entities does not recognize the method 'Boolean Contains[Int32]

I have the following extension methods in which I am using to do a Contains on LINQ-To-Entities:
public static class Extensions
{
public static IQueryable<TEntity> WhereIn<TEntity, TValue>
(
this ObjectQuery<TEntity> query,
Expression<Func<TEntity, TValue>> selector,
IEnumerable<TValue> collection
)
{
if (selector == null) throw new ArgumentNullException("selector");
if (collection == null) throw new ArgumentNullException("collection");
if (!collection.Any())
return query.Where(t => false);
ParameterExpression p = selector.Parameters.Single();
IEnumerable<Expression> equals = collection.Select(value =>
(Expression)Expression.Equal(selector.Body,
Expression.Constant(value, typeof(TValue))));
Expression body = equals.Aggregate((accumulate, equal) =>
Expression.Or(accumulate, equal));
return query.Where(Expression.Lambda<Func<TEntity, bool>>(body, p));
}
//Optional - to allow static collection:
public static IQueryable<TEntity> WhereIn<TEntity, TValue>
(
this ObjectQuery<TEntity> query,
Expression<Func<TEntity, TValue>> selector,
params TValue[] collection
)
{
return WhereIn(query, selector, (IEnumerable<TValue>)collection);
}
}
When I call the extenion method to check if a list of ids is in a particular table, it works and I get back the List of ids, like this:
List<int> Ids = _context.Persons
.WhereIn(x => x.PersonId, PersonIds)
.Select(x => x.HeaderId).ToList();
When I execute the next statement, it complains that LINQ-To-Entities does not recogonize Contains(int32), but I thought I am not going against the entity anymore, but a collection of ints.
predicate = predicate.And(x=> Ids.Contains(x.HeaderId));
If I have a comma separated string such as "1,2,3", then the following works:
predicate = predicate.And(x=>x.Ids.Contains(x.HeaderId));
I am trying to take the List returned and create comma separated list of strings, the problem here is that now when I do predicate = predicate.And(x=>sb.Contains(x.HeaderId.ToString());, it complains that it does not like ToString().
I also tried doing:
predicate = predicate.And(x=>Extensions.WhereIn(Ids, x.id));, but it can't resolve WhereIn. It says I must add `<>`, but I am not sure what to add here and how implement it.

Where is nothing wrong with your WhereIn, and you are correct: when you use Ids, you are not going against the entity anymore, but a collection of ints.
Problem is when you're using .And on predicate: LINQ-To-Entities tries to convert everything inside those brackets into Entities methods, and there is no corresponding Contains method.
Solution:
Instead of
predicate = predicate.And(x=> Ids.Contains(x.HeaderId));
use
predicate = predicate.And(Contains<XClassName, int>(x.HeaderId));
where Contains defined as follows:
private static Expression<Func<TElement, bool>> Contains<TElement, TValue>(Expression<Func<TElement, TValue>> valueSelector, List<TValue> values)
{
if (null == valueSelector) { throw new ArgumentNullException("valueSelector"); }
if (null == values) { throw new ArgumentNullException("values"); }
if (!values.Any())
return e => false;
var equals = values.Select(value => (Expression)Expression.Equal(valueSelector.Body, Expression.Constant(value, typeof(TValue))));
return Expression.Lambda<Func<TElement, bool>>(#equals.Aggregate(Expression.Or), valueSelector.Parameters.Single());
}
and XClassName is the name of the class of your x

You cant use array like that, you need to previsit this lambda in order to expand it to primitives. Alternatively you can change underlying provider so it knows how to generate IN statement , as it doesnt by default.
Didnt find post where one guys actually implement it, will updated once I did.
Basically when you use your extension method it is like
x=>arr.Contains(x)
So if you try to execute such lambda agains your entityset etc it will throw you exception saying that parameters can only be primitives.
The reason is that underlying provider doesnt know how to convert .Contains method for array as function parameter into sql query. And in order to solve that you have two options
teach it how to use T[] as parameter and use Contains with this parameter
update your extension method in order to generate new lamda which will use 'allowed' building blocks, ie expressions using primitive types like int, string, guid etc.
Check this article
http://msdn.microsoft.com/en-us/library/bb882521(v=vs.90).aspx

Replace your:
List<int> Ids = _context.Persons
.WhereIn(x => x.PersonId, PersonIds)
.Select(x => x.HeaderId).ToList();
with
var Ids = _context.Persons
.WhereIn(x => x.PersonId, PersonIds)
.Select(x => x.HeaderId).ToList();
and then try.

Intersect LINQ query

If I have an IEnumerable where ClassA exposes an ID property of type long.
Is it possible to use a Linq query to get all instances of ClassA with ID belonging to a second IEnumerable?
In other words, can this be done?
IEnumerable<ClassA> = original.Intersect(idsToFind....)?
where original is an IEnumerable<ClassA> and idsToFind is IEnumerable<long>.

Yes.
As other people have answered, you can use Where, but it will be extremely inefficient for large sets.
If performance is a concern, you can call Join:
var results = original.Join(idsToFind, o => o.Id, id => id, (o, id) => o);
If idsToFind can contain duplicates, you'll need to either call Distinct() on the IDs or on the results or replace Join with GroupJoin (The parameters to GroupJoin would be the same).

I will post an answer using Intersect.
This is useful if you want to intersect 2 IEnumerables of the same type.
First we will need an EqualityComparer:
public class KeyEqualityComparer<T> : IEqualityComparer<T>
{
private readonly Func<T, object> keyExtractor;
public KeyEqualityComparer(Func<T, object> keyExtractor)
{
this.keyExtractor = keyExtractor;
}
public bool Equals(T x, T y)
{
return this.keyExtractor(x).Equals(this.keyExtractor(y));
}
public int GetHashCode(T obj)
{
return this.keyExtractor(obj).GetHashCode();
}
}
Secondly we apply the KeyEqualityComparer to the Intersect function:
var list3= list1.Intersect(list2, new KeyEqualityComparer<ClassToCompare>(s => s.Id));

You can do it, but in the current form, you'd want to use the Where extension method.
var results = original.Where(x => yourEnumerable.Contains(x.ID));
Intersect on the other hand will find elements that are in both IEnumerable's. If you are looking for just a list of ID's, you can do the following which takes advantage of Intersect
var ids = original.Select(x => x.ID).Intersect(yourEnumerable);

A simple way would be:
IEnumerable<ClassA> result = original.Where(a => idsToFind.contains(a.ID));

Use the Where method to filter the results:
var result = original.Where(o => idsToFind.Contains(o.ID));

Naming things is important. Here is an extension method base on the Join operator:
private static IEnumerable<TSource> IntersectBy<TSource, TKey>(
this IEnumerable<TSource> source,
IEnumerable<TKey> keys,
Func<TSource, TKey> keySelector)
=> source.Join(keys, keySelector, id => id, (o, id) => o);
You can use it like this var result = items.IntersectBy(ids, item => item.id).

I've been tripping up all morning on Intersect, and how it doesn't work anymore in core 3, due to it being client side not server side.
From a list of items pulled from a database, the user can then choose to display them in a way that requires children to attached to that original list to get more information.
What use to work was:
itemList = _context.Item
.Intersect(itemList)
.Include(i => i.Notes)
.ToList();
What seems to now work is:
itemList = _context.Item
.Where(item => itemList.Contains(item))
.Include(i => i.Notes)
.ToList();
This seems to be working as expected, without any significant performance difference, and is really no more complicated than the first.