Quick way to get the difference between two List<> objects - c#

How do I get itemsToRemove to only contain "bar one", and itemsToAdd to only contain "bar five"?
I'm trying to use "Except", but obviously I'm using it incorrectly.
var oldList = new List<Foo>();
oldList.Add(new Foo(){ Bar = "bar one"});
oldList.Add(new Foo(){ Bar = "bar two"});
oldList.Add(new Foo(){ Bar = "bar three"});
oldList.Add(new Foo(){ Bar = "bar four"});
var newList = new List<Foo>();
newList.Add(new Foo(){ Bar = "bar two"});
newList.Add(new Foo(){ Bar = "bar three"});
newList.Add(new Foo(){ Bar = "bar four"});
newList.Add(new Foo(){ Bar = "bar five"});
var itemsToRemove = oldList.Except(newList); // should only contain "bar one"
var itemsToAdd = newList.Except(oldList); // should only contain "bar one"
foreach(var item in itemsToRemove){
Console.WriteLine(item.Bar + " removed");
// currently says
// bar one removed
// bar two removed
// bar three removed
// bar four removed
}
foreach(var item in itemsToAdd){
Console.WriteLine(item.Bar + " added");
// currently says
// bar two added
// bar three added
// bar four added
// bar five added
}

Except will use the default Equals and GetHashCode method of the objects in question to define "equality" for the objects, unless you provide a custom comparer (you have not). In this case, that will compare the references of the objects, not their Bar value.
One option would be to create an IEqualityComparer<Foo> that compares the Bar property, rather than references to the object itself.
public class FooComparer : IEqualityComparer<Foo>
{
public bool Equals(Foo x, Foo y)
{
if (x == null ^ y == null)
return false;
if (x == null && y == null)
return true;
return x.Bar == y.Bar;
}
public int GetHashCode(Foo obj)
{
if (obj == null)
return 0;
return obj.Bar.GetHashCode();
}
}
Another option is to create an Except method that accepts a selector to compare the values on. We can create such a method and then use that:
public static IEnumerable<TSource> ExceptBy<TSource, TKey>(
this IEnumerable<TSource> first,
IEnumerable<TSource> second,
Func<TSource, TKey> keySelector,
IEqualityComparer<TKey> comparer = null)
{
comparer = comparer ?? EqualityComparer<TKey>.Default;
var set = new HashSet<TKey>(second.Select(keySelector), comparer);
return first.Where(item => set.Add(keySelector(item)));
}
This allows us to write:
var itemsToRemove = oldList.ExceptBy(newList, foo => foo.Bar);
var itemsToAdd = newList.ExceptBy(oldList, foo => foo.Bar);

Your logic is sound, but Except default behaviour for comparing two classes is to go by references. Since you are effectively create two lists with 8 differents objets (regardless of their content), there will be no two equal objects.
You can, however, use the Except overload that takes an IEqualityComparer. For example:
public class FooEqualityComparer : IEqualityComparer<Foo>
{
public bool Equals(Foo left, Foo right)
{
if(left == null && right == null) return true;
return left != null && right != null && left.Bar == right.Bar;
}
public int GetHashCode(Foo item)
{
return item != null ? item.Bar.GetHashcode() : 0;
}
}
// In your code
var comparer = new FooEqualityComparer();
var itemsToRemove = oldList.Except(newList, comparer );
var itemsToAdd = newList.Except(oldList, comparer);

This is mostly a riff on Servy's answer to give a more general approach to this:
public class PropertyEqualityComparer<TItem, TKey> : EqualityComparer<Tuple<TItem, TKey>>
{
readonly Func<TItem, TKey> _getter;
public PropertyEqualityComparer(Func<TItem, TKey> getter)
{
_getter = getter;
}
public Tuple<TItem, TKey> Wrap(TItem item) {
return Tuple.Create(item, _getter(item));
}
public TItem Unwrap(Tuple<TItem, TKey> tuple) {
return tuple.Item1;
}
public override bool Equals(Tuple<TItem, TKey> x, Tuple<TItem, TKey> y)
{
if (x.Item2 == null && y.Item2 == null) return true;
if (x.Item2 == null || y.Item2 == null) return false;
return x.Item2.Equals(y.Item2);
}
public override int GetHashCode(Tuple<TItem, TKey> obj)
{
if (obj.Item2 == null) return 0;
return obj.Item2.GetHashCode();
}
}
public static class ComparerLinqExtensions {
public static IEnumerable<TSource> Except<TSource, TKey>(this IEnumerable<TSource> first, IEnumerable<TSource> second, Func<TSource, TKey> keyGetter)
{
var comparer = new PropertyEqualityComparer<TSource, TKey>(keyGetter);
var firstTuples = first.Select(comparer.Wrap);
var secondTuples = second.Select(comparer.Wrap);
return firstTuples.Except(secondTuples, comparer)
.Select(comparer.Unwrap);
}
}
// ...
var itemsToRemove = oldList.Except(newList, foo => foo.Bar);
var itemsToAdd = newList.Except(oldList, foo => foo.Bar);
This should work fine for any classes without unusual equality semantics, where it's incorrect to call the object.Equals() override instead of IEquatable<T>.Equals().Notably, this will work fine for anonymous types.

This is because you're comparing objects of type Foo, and not property Bar of type string. Try:
var itemsToRemove = oldList.Select(i => i.Bar).Except(newList.Select(i => i.Bar));
var itemsToAdd = newList.Select(i => i.Bar).Except(oldList.Select(i => i.Bar));

Implement IComparable on your data objects; I think you're being bitten by reference comparison. If you change Foo to just string, your code works.
var oldList = new List<string>();
oldList.Add("bar one");
oldList.Add("bar two");
oldList.Add("bar three");
oldList.Add("bar four");
var newList = new List<string>();
newList.Add("bar two");
newList.Add("bar three");
newList.Add("bar four");
newList.Add("bar five");
var itemsToRemove = oldList.Except(newList); // should only contain "bar one"
var itemsToAdd = newList.Except(oldList); // should only contain "bar one"
foreach (var item in itemsToRemove)
{
Console.WriteLine(item + " removed");
}
foreach (var item in itemsToAdd)
{
Console.WriteLine(item + " added");
}

Related

OrderBy list by a nested list

I have a objectA list, each one contain an another objectB list.
I'm trying to OrderBy each objectB list with an (int) Id :
var sorted = objectA.OrderBy(a => a.ObjectB.OrderBy(b => b.Id)).ToList();
Of course, this doesn't work. Someone have an advice ?
If you want to sort each ObjectB list in place (i.e. modify it), then simply use the List<T>.Sort method. You will need to specify a custom Comparison<T> delegate:
foreach (var a in objectA)
{
a.ObjectB.Sort((x, y) => x.Id - y.Id)
}
If objectA is a List<ObjectA>, then you can use the ForEach method and pass a delegate:
objectA.ForEach(a => a.ObjectB.Sort((x, y) => x.Id - y.Id));
If you don't want to modify your original ObjectA instances, then you will have to project each ObjectA instance into a new instance (by cloning it) and then assign sorted ObjectB lists. It would look something like (presuming that all properties have public setters):
var newList = objectA
.Select(x => new ObjectA()
{
Id = x.Id,
SomethingElse = x.SomethingElse,
ObjectB = x.ObjectB.OrderBy(b => b.Id).ToList()
})
.ToList();
You can prepare extension method and use it at a generic way. For first, helpers for Expression:
public static void SetProperty<T, B>(
this Expression<Func<T, B>> propertySelector,
T target,
B value)
{
SetObjectProperty(target, propertySelector, value);
}
public static void SetObjectProperty<T, B>(
T target,
Expression<Func<T, B>> propertySelector,
object value)
{
if (target == null)
{
throw new ArgumentNullException("target");
}
if (propertySelector == null)
{
throw new ArgumentNullException("propertySelector");
}
var memberExpression = propertySelector.Body as MemberExpression;
if (memberExpression == null)
{
throw new NotSupportedException("Cannot recognize property.");
}
var propertyInfo = memberExpression.Member as PropertyInfo;
if (propertyInfo == null)
{
throw new NotSupportedException(
"You can select property only."
+ " Currently, selected member is: "
+ memberExpression.Member);
}
propertyInfo.SetValue(target, value);
}
Then write this extension:
public static IEnumerable<TSource> OrderInnerCollection<TSource, TInner, TKey>(
this IEnumerable<TSource> source,
Expression<Func<TSource, IEnumerable<TInner>>> innerSelector,
Func<TInner, TKey> keySelector)
{
var innerSelectorDelegate = innerSelector.Compile();
foreach (var item in source)
{
var collection = innerSelectorDelegate(item);
collection = collection.OrderBy(keySelector);
innerSelector.SetProperty(item, collection);
yield return item;
}
}
And usage:
var result = objectA.OrderInnerCollection(
aObj => aObj.ObjectB,
objB => objB.Id).ToList();

Sort a List<T> by enum where enum is out of order

I have a List of messages.
Each message has a type.
public enum MessageType
{
Foo = 0,
Bar = 1,
Boo = 2,
Doo = 3
}
The enum names are arbitrary and cannot be changed.
I need to return the list sorted as: Boo, Bar, Foo, Doo
My current solution is to create a tempList, add the values in the order I want, return the new list.
List<Message> tempList = new List<Message>();
tempList.AddRange(messageList.Where(m => m.MessageType == MessageType.Boo));
tempList.AddRange(messageList.Where(m => m.MessageType == MessageType.Bar));
tempList.AddRange(messageList.Where(m => m.MessageType == MessageType.Foo));
tempList.AddRange(messageList.Where(m => m.MessageType == MessageType.Doo));
messageList = tempList;
How can I do this with an IComparer?
An alternative to using IComparer would be to build an ordering dictionary.
var orderMap = new Dictionary<MessageType, int>() {
{ MessageType.Boo, 0 },
{ MessageType.Bar, 1 },
{ MessageType.Foo, 2 },
{ MessageType.Doo, 3 }
};
var orderedList = messageList.OrderBy(m => orderMap[m.MessageType]);
So, let's write our own comparer:
public class MyMessageComparer : IComparer<MessageType> {
protected IList<MessageType> orderedTypes {get; set;}
public MyMessageComparer() {
// you can reorder it's all as you want
orderedTypes = new List<MessageType>() {
MessageType.Boo,
MessageType.Bar,
MessageType.Foo,
MessageType.Doo,
};
}
public int Compare(MessageType x, MessageType y) {
var xIndex = orderedTypes.IndexOf(x);
var yIndex = orderedTypes.IndexOf(y);
return xIndex.CompareTo(yIndex);
}
};
How to use:
messages.OrderBy(m => m.MessageType, new MyMessageComparer())
There is a easier way: just create ordereTypes list and use another overload of OrderBy:
var orderedTypes = new List<MessageType>() {
MessageType.Boo,
MessageType.Bar,
MessageType.Foo,
MessageType.Doo,
};
messages.OrderBy(m => orderedTypes.IndexOf(m.MessageType)).ToList();
Hm.. Let's try to take advantages from writing our own IComparer. Idea: write it like our last example but in some other semantic. Like this:
messages.OrderBy(
m => m.MessageType,
new EnumComparer<MessageType>() {
MessageType.Boo,
MessageType.Foo }
);
Or this:
messages.OrderBy(m => m.MessageType, EnumComparer<MessageType>());
Okay, so what we need. Our own comparer:
Must accept enum as generic type (how to solve)
Must be usable with collection initializer syntax (how to)
Must sort by default order, when we have no enum values in our comparer (or some enum values aren't in our comparer)
So, here is the code:
public class EnumComparer<TEnum>: IComparer<TEnum>, IEnumerable<TEnum> where TEnum: struct, IConvertible {
protected static IList<TEnum> TypicalValues { get; set; }
protected IList<TEnum> _reorderedValues;
protected IList<TEnum> ReorderedValues {
get { return _reorderedValues.Any() ? _reorderedValues : TypicalValues; }
set { _reorderedValues = value; }
}
static EnumComparer() {
if (!typeof(TEnum).IsEnum)
{
throw new ArgumentException("T must be an enumerated type");
}
TypicalValues = new List<TEnum>();
foreach (TEnum value in Enum.GetValues(typeof(TEnum))) {
TypicalValues.Add(value);
};
}
public EnumComparer(IList<TEnum> reorderedValues = null) {
if (_reorderedValues == null ) {
_reorderedValues = new List<TEnum>();
return;
}
_reorderedValues = reorderedValues;
}
public void Add(TEnum value) {
if (_reorderedValues.Contains(value))
return;
_reorderedValues.Add(value);
}
public int Compare(TEnum x, TEnum y) {
var xIndex = ReorderedValues.IndexOf(x);
var yIndex = ReorderedValues.IndexOf(y);
// no such enums in our order list:
// so this enum values must be in the end
// and must be ordered between themselves by default
if (xIndex == -1) {
if (yIndex == -1) {
xIndex = TypicalValues.IndexOf(x);
yIndex = TypicalValues.IndexOf(y);
return xIndex.CompareTo(yIndex);
}
return -1;
}
if (yIndex == -1) {
return -1; //
}
return xIndex.CompareTo(yIndex);
}
public void Clear() {
_reorderedValues = new List<TEnum>();
}
private IEnumerable<TEnum> GetEnumerable() {
return Enumerable.Concat(
ReorderedValues,
TypicalValues.Where(v => !ReorderedValues.Contains(v))
);
}
public IEnumerator<TEnum> GetEnumerator() {
return GetEnumerable().GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator() {
return GetEnumerable().GetEnumerator();
}
}
So, well, let's make sorting more faster. We need to override default OrderBy method for our enums:
public static class LinqEnumExtensions
{
public static IEnumerable<TSource> OrderBy<TSource, TEnum>(this IEnumerable<TSource> source, Func<TSource, TEnum> selector, EnumComparer<TEnum> enumComparer) where TEnum : struct, IConvertible
{
foreach (var enumValue in enumComparer)
{
foreach (var sourceElement in source.Where(item => selector(item).Equals(enumValue)))
{
yield return sourceElement;
}
}
}
}
Yeah, that's lazy. You can google how yield works. Well, let's test speed. Simple benchmark: http://pastebin.com/P8qaU20Y. Result for n = 1000000;
Enumerable orderBy, elementAt: 00:00:04.5485845
Own orderBy, elementAt: 00:00:00.0040010
Enumerable orderBy, full sort: 00:00:04.6685977
Own orderBy, full sort: 00:00:00.4540575
We see, that our own orderBy by is more lazy that standart order by (yeah, it doesn't need to sort everything). And faster even for fullsort.
Problems in this code: it doesn't support ThenBy(). If you need this, you can write your own linq extension that returns IOrderedEnumerable There are a blog post series by Jon Skeet which goes into LINQ to Objects in some depth, providing a complete alternative implementation. The basis of IOrderedEnumerable is covered in part 26a and 26b, with more details and optimization in 26c and 26d.
Instead of using an IComparer, you could also use a SelectMany approach, which should have better performance for large message lists, if you have a fixed number of message types.
var messageTypeOrder = new [] {
MessageType.Boo,
MessageType.Bar,
MessageType.Foo,
MessageType.Doo,
};
List<Message> tempList = messageTypeOrder
.SelectMany(type => messageList.Where(m => m.MessageType == type))
.ToList();
You may avoid writing a completely new type just to implement IComparable. Use the Comparer class instead:
IComparer<Message> comparer = Comparer.Create<Message>((message) =>
{
// lambda that compares things
});
tempList.Sort(comparer);
You can build a mapping dictionary dynamically from the Enum values with LINQ like this:
var mappingDIctionary = new List<string>((string[])Enum.GetNames(typeof(Hexside)))
.OrderBy(label => label )
.Select((i,n) => new {Index=i, Label=n}).ToList();
Now any new values added to the Enum n future will automatically get properly mapped.
Also, if someone decides to renumber, refactor, or reorder the enumeration, everything is handled automatically.
Update:
As pointed out below, Alphabetical ordering was not called for; rather a semi- alphabetical ordering, so essentially random. Although not an answer to this particular question, this technique might be useful to future visitors, so I will leave it standing.
No need to have the mapping. This should give you the list and order based on the enum. You don't have to modify anything even when you change the enum's order or and new items...
var result = (from x in tempList
join y in Enum.GetValues(typeof(MessageType)).Cast<MessageType>()
on x equals y
orderby y
select y).ToList();
If you are about to get this working with Entity Framework (EF), you would have to spread out your enum in your OrderBy as such:
messageList.OrderBy(m =>
m.MessageType == MessageType.Boo ? 0 :
m.MessageType == MessageType.Bar ? 1 :
m.MessageType == MessageType.Foo ? 2 :
m.MessageType == MessageType.Doo ? 3 : 4
);
This creates a sub select with CASE WHEN, then ORDER BY on that temporary column.

Using IEqualityComparer for Union

I simply want to remove duplicates from two lists and combine them into one list. I also need to be able to define what a duplicate is. I define a duplicate by the ColumnIndex property, if they are the same, they are duplicates. Here is the approach I took:
I found a nifty example of how to write inline comparers for the random occassions where you need em only once in a code segment.
public class InlineComparer<T> : IEqualityComparer<T>
{
private readonly Func<T, T, bool> getEquals;
private readonly Func<T, int> getHashCode;
public InlineComparer(Func<T, T, bool> equals, Func<T, int> hashCode)
{
getEquals = equals;
getHashCode = hashCode;
}
public bool Equals(T x, T y)
{
return getEquals(x, y);
}
public int GetHashCode(T obj)
{
return getHashCode(obj);
}
}
Then I just have my two lists, and attempt a union on them with the comparer.
var formatIssues = issues.Where(i => i.IsFormatError == true);
var groupIssues = issues.Where(i => i.IsGroupError == true);
var dupComparer = new InlineComparer<Issue>((i1, i2) => i1.ColumnInfo.ColumnIndex == i2.ColumnInfo.ColumnIndex,
i => i.ColumnInfo.ColumnIndex);
var filteredIssues = groupIssues.Union(formatIssues, dupComparer);
The result set however is null.
Where am I going astray?
I have already confirmed that the two lists have columns with equal ColumnIndex properties.
I've just run your code on a test set.... and it works!
public class InlineComparer<T> : IEqualityComparer<T>
{
private readonly Func<T, T, bool> getEquals;
private readonly Func<T, int> getHashCode;
public InlineComparer(Func<T, T, bool> equals, Func<T, int> hashCode)
{
getEquals = equals;
getHashCode = hashCode;
}
public bool Equals(T x, T y)
{
return getEquals(x, y);
}
public int GetHashCode(T obj)
{
return getHashCode(obj);
}
}
class TestClass
{
public string S { get; set; }
}
[TestMethod]
public void testThis()
{
var l1 = new List<TestClass>()
{
new TestClass() {S = "one"},
new TestClass() {S = "two"},
};
var l2 = new List<TestClass>()
{
new TestClass() {S = "three"},
new TestClass() {S = "two"},
};
var dupComparer = new InlineComparer<TestClass>((i1, i2) => i1.S == i2.S, i => i.S.GetHashCode());
var unionList = l1.Union(l2, dupComparer);
Assert.AreEqual(3, unionList);
}
So... maybe go back and check your test data - or run it with some other test data?
After all - for a Union to be empty - that suggests that both your input lists are also empty?
A slightly simpler way:
it does preserve the original order
it ignores dupes as it finds them
Uses a link extension method:
formatIssues.Union(groupIssues).DistinctBy(x => x.ColumnIndex)
This is the DistinctBy lambda method from MoreLinq
public static IEnumerable<TSource> DistinctBy<TSource, TKey>
(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
HashSet<TKey> knownKeys = new HashSet<TKey>();
foreach (TSource element in source)
{
if (knownKeys.Add(keySelector(element)))
{
yield return element;
}
}
}
Would the Linq Except method not do it for you?
var formatIssues = issues.Where(i => i.IsFormatError == true);
var groupIssues = issues.Where(i => i.IsGroupError == true);
var dupeIssues = issues.Where(i => issues.Except(new List<Issue> {i})
.Any(x => x.ColumnIndex == i.ColumnIndex));
var filteredIssues = formatIssues.Union(groupIssues).Except(dupeIssues);

Filtering duplicates out of an IEnumerable

I have this code:
class MyObj {
int Id;
string Name;
string Location;
}
IEnumerable<MyObj> list;
I want to convert list to a dictionary like this:
list.ToDictionary(x => x.Name);
but it tells me I have duplicate keys. How can I keep only the first item for each key?
I suppose the easiest way would be to group by key and take the first element of each group:
list.GroupBy(x => x.name).Select(g => g.First()).ToDictionary(x => x.name);
Or you could use Distinct if your objects implement IEquatable to compare between themselves by key:
// I'll just randomly call your object Person for this example.
class Person : IEquatable<Person>
{
public string Name { get; set; }
public bool Equals(Person other)
{
if (other == null)
return false;
return Name == other.Name;
}
public override bool Equals(object obj)
{
return base.Equals(obj as Person);
}
public override int GetHashCode()
{
return Name.GetHashCode();
}
}
...
list.Distinct().ToDictionary(x => x.Name);
Or if you don't want to do that (maybe because you normally want to compare for equality in a different way, so Equals is already in use) you could make a custom implementation of IEqualityComparer just for this case:
class PersonComparer : IEqualityComparer<Person>
{
public bool Equals(Person x, Person y)
{
if (x == null)
return y == null;
if (y == null)
return false;
return x.Name == y.Name;
}
public int GetHashCode(Person obj)
{
return obj.Name.GetHashCode();
}
}
...
list.Distinct(new PersonComparer()).ToDictionary(x => x.Name);
list.Distinct().ToDictionary(x => x.Name);
You could also create your own Distinct extension overload method that accepted a Func<> for choosing the distinct key:
public static class EnumerationExtensions
{
public static IEnumerable<TSource> Distinct<TSource,TKey>(
this IEnumerable<TSource> source, Func<TSource,TKey> keySelector)
{
KeyComparer comparer = new KeyComparer(keySelector);
return source.Distinct(comparer);
}
private class KeyComparer<TSource,TKey> : IEqualityComparer<TSource>
{
private Func<TSource,TKey> keySelector;
public DelegatedComparer(Func<TSource,TKey> keySelector)
{
this.keySelector = keySelector;
}
bool IEqualityComparer.Equals(TSource a, TSource b)
{
if (a == null && b == null) return true;
if (a == null || b == null) return false;
return keySelector(a) == keySelector(b);
}
int IEqualityComparer.GetHashCode(TSource obj)
{
return keySelector(obj).GetHashCode();
}
}
}
Apologies for any bad code formatting, I wanted to reduce the size of the code on the page. Anyway, you can then use ToDictionary:
var dictionary = list.Distinct(x => x.Name).ToDictionary(x => x.Name);
Could make your own perhaps? For example:
public static class Extensions
{
public static IDictionary<TKey, TValue> ToDictionary2<TKey, TValue>(
this IEnumerable<TValue> subjects, Func<TValue, TKey> keySelector)
{
var dictionary = new Dictionary<TKey, TValue>();
foreach(var subject in subjects)
{
var key = keySelector(subject);
if(!dictionary.ContainsKey(key))
dictionary.Add(key, subject);
}
return dictionary;
}
}
var dictionary = list.ToDictionary2(x => x.Name);
Haven't tested it, but should work. (and it should probably have a better name than ToDictionary2 :p)
Alternatively, you can implement a DistinctBy method, for example like this:
public static IEnumerable<TSubject> DistinctBy<TSubject, TValue>(this IEnumerable<TSubject> subjects, Func<TSubject, TValue> valueSelector)
{
var set = new HashSet<TValue>();
foreach(var subject in subjects)
if(set.Add(valueSelector(subject)))
yield return subject;
}
var dictionary = list.DistinctBy(x => x.Name).ToDictionary(x => x.Name);
The problem here is that the ToDictionary extension method does not support multiple values with the same key. One solution is to write a version which does and use that instead.
public static Dictionary<TKey,TValue> ToDictionaryAllowDuplicateKeys<TKey,TValue>(
this IEnumerable<TValue> values,
Func<TValue,TKey> keyFunc) {
var map = new Dictionary<TKey,TValue>();
foreach ( var cur in values ) {
var key = keyFunc(cur);
map[key] = cur;
}
return map;
}
Now converting to a dictionary is straight forward
var map = list.ToDictionaryAllowDuplicateKeys(x => x.Name);
The following will work if you have different instances of MyObj with the same value for the Name property. It will take the first instance found for each duplicate (sorry for the obj - obj2 notation, it is just sample code):
list.SelectMany(obj => new MyObj[] {list.Where(obj2 => obj2.Name == obj.Name).First()}).Distinct();
EDIT: Joren's solution is better as it does not create unnecessary arrays in the process.

Distinct() with lambda?

Right, so I have an enumerable and wish to get distinct values from it.
Using System.Linq, there's, of course, an extension method called Distinct. In the simple case, it can be used with no parameters, like:
var distinctValues = myStringList.Distinct();
Well and good, but if I have an enumerable of objects for which I need to specify equality, the only available overload is:
var distinctValues = myCustomerList.Distinct(someEqualityComparer);
The equality comparer argument must be an instance of IEqualityComparer<T>. I can do this, of course, but it's somewhat verbose and, well, cludgy.
What I would have expected is an overload that would take a lambda, say a Func<T, T, bool>:
var distinctValues = myCustomerList.Distinct((c1, c2) => c1.CustomerId == c2.CustomerId);
Anyone know if some such extension exists, or some equivalent workaround? Or am I missing something?
Alternatively, is there a way of specifying an IEqualityComparer inline (embarrass me)?
Update
I found a reply by Anders Hejlsberg to a post in an MSDN forum on this subject. He says:
The problem you're going to run into is that when two objects compare
equal they must have the same GetHashCode return value (or else the
hash table used internally by Distinct will not function correctly).
We use IEqualityComparer because it packages compatible
implementations of Equals and GetHashCode into a single interface.
I suppose that makes sense.
IEnumerable<Customer> filteredList = originalList
.GroupBy(customer => customer.CustomerId)
.Select(group => group.First());
It looks to me like you want DistinctBy from MoreLINQ. You can then write:
var distinctValues = myCustomerList.DistinctBy(c => c.CustomerId);
Here's a cut-down version of DistinctBy (no nullity checking and no option to specify your own key comparer):
public static IEnumerable<TSource> DistinctBy<TSource, TKey>
(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
HashSet<TKey> knownKeys = new HashSet<TKey>();
foreach (TSource element in source)
{
if (knownKeys.Add(keySelector(element)))
{
yield return element;
}
}
}
To Wrap things up . I think most of the people which came here like me want the simplest solution possible without using any libraries and with best possible performance.
(The accepted group by method for me i think is an overkill in terms of performance. )
Here is a simple extension method using the IEqualityComparer interface which works also for null values.
Usage:
var filtered = taskList.DistinctBy(t => t.TaskExternalId).ToArray();
Extension Method Code
public static class LinqExtensions
{
public static IEnumerable<T> DistinctBy<T, TKey>(this IEnumerable<T> items, Func<T, TKey> property)
{
GeneralPropertyComparer<T, TKey> comparer = new GeneralPropertyComparer<T,TKey>(property);
return items.Distinct(comparer);
}
}
public class GeneralPropertyComparer<T,TKey> : IEqualityComparer<T>
{
private Func<T, TKey> expr { get; set; }
public GeneralPropertyComparer (Func<T, TKey> expr)
{
this.expr = expr;
}
public bool Equals(T left, T right)
{
var leftProp = expr.Invoke(left);
var rightProp = expr.Invoke(right);
if (leftProp == null && rightProp == null)
return true;
else if (leftProp == null ^ rightProp == null)
return false;
else
return leftProp.Equals(rightProp);
}
public int GetHashCode(T obj)
{
var prop = expr.Invoke(obj);
return (prop==null)? 0:prop.GetHashCode();
}
}
Shorthand solution
myCustomerList.GroupBy(c => c.CustomerId, (key, c) => c.FirstOrDefault());
No there is no such extension method overload for this. I've found this frustrating myself in the past and as such I usually write a helper class to deal with this problem. The goal is to convert a Func<T,T,bool> to IEqualityComparer<T,T>.
Example
public class EqualityFactory {
private sealed class Impl<T> : IEqualityComparer<T,T> {
private Func<T,T,bool> m_del;
private IEqualityComparer<T> m_comp;
public Impl(Func<T,T,bool> del) {
m_del = del;
m_comp = EqualityComparer<T>.Default;
}
public bool Equals(T left, T right) {
return m_del(left, right);
}
public int GetHashCode(T value) {
return m_comp.GetHashCode(value);
}
}
public static IEqualityComparer<T,T> Create<T>(Func<T,T,bool> del) {
return new Impl<T>(del);
}
}
This allows you to write the following
var distinctValues = myCustomerList
.Distinct(EqualityFactory.Create((c1, c2) => c1.CustomerId == c2.CustomerId));
Here's a simple extension method that does what I need...
public static class EnumerableExtensions
{
public static IEnumerable<TKey> Distinct<T, TKey>(this IEnumerable<T> source, Func<T, TKey> selector)
{
return source.GroupBy(selector).Select(x => x.Key);
}
}
It's a shame they didn't bake a distinct method like this into the framework, but hey ho.
This will do what you want but I don't know about performance:
var distinctValues =
from cust in myCustomerList
group cust by cust.CustomerId
into gcust
select gcust.First();
At least it's not verbose.
From .NET 6 or later, there is a new build-in method Enumerable.DistinctBy to achieve this.
var distinctValues = myCustomerList.DistinctBy(c => c.CustomerId);
// With IEqualityComparer
var distinctValues = myCustomerList.DistinctBy(c => c.CustomerId, someEqualityComparer);
Something I have used which worked well for me.
/// <summary>
/// A class to wrap the IEqualityComparer interface into matching functions for simple implementation
/// </summary>
/// <typeparam name="T">The type of object to be compared</typeparam>
public class MyIEqualityComparer<T> : IEqualityComparer<T>
{
/// <summary>
/// Create a new comparer based on the given Equals and GetHashCode methods
/// </summary>
/// <param name="equals">The method to compute equals of two T instances</param>
/// <param name="getHashCode">The method to compute a hashcode for a T instance</param>
public MyIEqualityComparer(Func<T, T, bool> equals, Func<T, int> getHashCode)
{
if (equals == null)
throw new ArgumentNullException("equals", "Equals parameter is required for all MyIEqualityComparer instances");
EqualsMethod = equals;
GetHashCodeMethod = getHashCode;
}
/// <summary>
/// Gets the method used to compute equals
/// </summary>
public Func<T, T, bool> EqualsMethod { get; private set; }
/// <summary>
/// Gets the method used to compute a hash code
/// </summary>
public Func<T, int> GetHashCodeMethod { get; private set; }
bool IEqualityComparer<T>.Equals(T x, T y)
{
return EqualsMethod(x, y);
}
int IEqualityComparer<T>.GetHashCode(T obj)
{
if (GetHashCodeMethod == null)
return obj.GetHashCode();
return GetHashCodeMethod(obj);
}
}
All solutions I've seen here rely on selecting an already comparable field. If one needs to compare in a different way, though, this solution here seems to work generally, for something like:
somedoubles.Distinct(new LambdaComparer<double>((x, y) => Math.Abs(x - y) < double.Epsilon)).Count()
Take another way:
var distinctValues = myCustomerList.
Select(x => x._myCaustomerProperty).Distinct();
The sequence return distinct elements compare them by property '_myCaustomerProperty' .
You can use LambdaEqualityComparer:
var distinctValues
= myCustomerList.Distinct(new LambdaEqualityComparer<OurType>((c1, c2) => c1.CustomerId == c2.CustomerId));
public class LambdaEqualityComparer<T> : IEqualityComparer<T>
{
public LambdaEqualityComparer(Func<T, T, bool> equalsFunction)
{
_equalsFunction = equalsFunction;
}
public bool Equals(T x, T y)
{
return _equalsFunction(x, y);
}
public int GetHashCode(T obj)
{
return obj.GetHashCode();
}
private readonly Func<T, T, bool> _equalsFunction;
}
You can use InlineComparer
public class InlineComparer<T> : IEqualityComparer<T>
{
//private readonly Func<T, T, bool> equalsMethod;
//private readonly Func<T, int> getHashCodeMethod;
public Func<T, T, bool> EqualsMethod { get; private set; }
public Func<T, int> GetHashCodeMethod { get; private set; }
public InlineComparer(Func<T, T, bool> equals, Func<T, int> hashCode)
{
if (equals == null) throw new ArgumentNullException("equals", "Equals parameter is required for all InlineComparer instances");
EqualsMethod = equals;
GetHashCodeMethod = hashCode;
}
public bool Equals(T x, T y)
{
return EqualsMethod(x, y);
}
public int GetHashCode(T obj)
{
if (GetHashCodeMethod == null) return obj.GetHashCode();
return GetHashCodeMethod(obj);
}
}
Usage sample:
var comparer = new InlineComparer<DetalleLog>((i1, i2) => i1.PeticionEV == i2.PeticionEV && i1.Etiqueta == i2.Etiqueta, i => i.PeticionEV.GetHashCode() + i.Etiqueta.GetHashCode());
var peticionesEV = listaLogs.Distinct(comparer).ToList();
Assert.IsNotNull(peticionesEV);
Assert.AreNotEqual(0, peticionesEV.Count);
Source:
https://stackoverflow.com/a/5969691/206730
Using IEqualityComparer for Union
Can I specify my explicit type comparator inline?
If Distinct() doesn't produce unique results, try this one:
var filteredWC = tblWorkCenter.GroupBy(cc => cc.WCID_I).Select(grp => grp.First()).Select(cc => new Model.WorkCenter { WCID = cc.WCID_I }).OrderBy(cc => cc.WCID);
ObservableCollection<Model.WorkCenter> WorkCenter = new ObservableCollection<Model.WorkCenter>(filteredWC);
A tricky way to do this is use Aggregate() extension, using a dictionary as accumulator with the key-property values as keys:
var customers = new List<Customer>();
var distincts = customers.Aggregate(new Dictionary<int, Customer>(),
(d, e) => { d[e.CustomerId] = e; return d; },
d => d.Values);
And a GroupBy-style solution is using ToLookup():
var distincts = customers.ToLookup(c => c.CustomerId).Select(g => g.First());
IEnumerable lambda extension:
public static class ListExtensions
{
public static IEnumerable<T> Distinct<T>(this IEnumerable<T> list, Func<T, int> hashCode)
{
Dictionary<int, T> hashCodeDic = new Dictionary<int, T>();
list.ToList().ForEach(t =>
{
var key = hashCode(t);
if (!hashCodeDic.ContainsKey(key))
hashCodeDic.Add(key, t);
});
return hashCodeDic.Select(kvp => kvp.Value);
}
}
Usage:
class Employee
{
public string Name { get; set; }
public int EmployeeID { get; set; }
}
//Add 5 employees to List
List<Employee> lst = new List<Employee>();
Employee e = new Employee { Name = "Shantanu", EmployeeID = 123456 };
lst.Add(e);
lst.Add(e);
Employee e1 = new Employee { Name = "Adam Warren", EmployeeID = 823456 };
lst.Add(e1);
//Add a space in the Name
Employee e2 = new Employee { Name = "Adam Warren", EmployeeID = 823456 };
lst.Add(e2);
//Name is different case
Employee e3 = new Employee { Name = "adam warren", EmployeeID = 823456 };
lst.Add(e3);
//Distinct (without IEqalityComparer<T>) - Returns 4 employees
var lstDistinct1 = lst.Distinct();
//Lambda Extension - Return 2 employees
var lstDistinct = lst.Distinct(employee => employee.EmployeeID.GetHashCode() ^ employee.Name.ToUpper().Replace(" ", "").GetHashCode());
The Microsoft System.Interactive package has a version of Distinct that takes a key selector lambda. This is effectively the same as Jon Skeet's solution, but it may be helpful for people to know, and to check out the rest of the library.
Here's how you can do it:
public static class Extensions
{
public static IEnumerable<T> MyDistinct<T, V>(this IEnumerable<T> query,
Func<T, V> f,
Func<IGrouping<V,T>,T> h=null)
{
if (h==null) h=(x => x.First());
return query.GroupBy(f).Select(h);
}
}
This method allows you to use it by specifying one parameter like .MyDistinct(d => d.Name), but it also allows you to specify a having condition as a second parameter like so:
var myQuery = (from x in _myObject select x).MyDistinct(d => d.Name,
x => x.FirstOrDefault(y=>y.Name.Contains("1") || y.Name.Contains("2"))
);
N.B. This would also allow you to specify other functions like for example .LastOrDefault(...) as well.
If you want to expose just the condition, you can have it even simpler by implementing it as:
public static IEnumerable<T> MyDistinct2<T, V>(this IEnumerable<T> query,
Func<T, V> f,
Func<T,bool> h=null
)
{
if (h == null) h = (y => true);
return query.GroupBy(f).Select(x=>x.FirstOrDefault(h));
}
In this case, the query would just look like:
var myQuery2 = (from x in _myObject select x).MyDistinct2(d => d.Name,
y => y.Name.Contains("1") || y.Name.Contains("2")
);
N.B. Here, the expression is simpler, but note .MyDistinct2 uses .FirstOrDefault(...) implicitly.
Note: The examples above are using the following demo class
class MyObject
{
public string Name;
public string Code;
}
private MyObject[] _myObject = {
new MyObject() { Name = "Test1", Code = "T"},
new MyObject() { Name = "Test2", Code = "Q"},
new MyObject() { Name = "Test2", Code = "T"},
new MyObject() { Name = "Test5", Code = "Q"}
};
I'm assuming you have an IEnumerable<T>, and in your example delegate, you would like c1 and c2 to be referring to two elements in this list?
I believe you could achieve this with a self join:
var distinctResults = from c1 in myList
join c2 in myList on <your equality conditions>
I found this as the easiest solution.
public static IEnumerable<TSource> DistinctBy<TSource, TKey>
(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
return source.GroupBy(keySelector).Select(x => x.FirstOrDefault());
}

Categories

Resources