Develop Reference

Why do I need to cast a List<T> of a declared List<T> when passing through a Lambda OrderBy expression? [duplicate]

This question already has answers here:
OrderBy and List vs. IOrderedEnumerable
(6 answers)
Closed 2 years ago.
So I'm trying to sort a list of my objects using lambda like so:
List<DirectoryObjects> sort = dirObjList.OrderBy(ms => ms.name);
Visual Studio though says that is wrong and it says this is correct:
List<DirectoryObjects> sort = (List<DirectoryObjects>)dirObjList.OrderBy(ms => ms.name);
I've been exposed to lambda before but I've never really understood it. My way of doing it seems correct to me because I'm under the impression lambda is just sorting my list for me but still retaining it's type. Is it wrong to think that? The second instance looks...redundant to me. Why would I need to cast a sorted List when it already is that? Or should I think of lambdas as something that converts Enum's to a different type?

The lambda in itself is irrelevant here, the key is LINQ behavior.
The OrderBy extension method does NOT order your list as you think. Instead, it creates a new, independent IEnumerable<T> with the result of the sorting, leaving the original untouched. So it's not a List<T>, but a completely different type, whose only capability is to be enumerated, following the order you asked for.
A List is that and also a couple more things (irrelevant for now), but the important thing is that the return value of OrderBy is not a list. The proposed cast is also wrong just because of that, as you're trying to cast an IEnumerable<T> to List<T>. In general, those casts may or may not work, depending on the real underling type (as IEnumerable<T> is an interface, compared to List<T> being a concrete class), so the compiler allows it. In the particular case of LINQ's OrderBy it'll certainly fail, as it doesn't returns a list at all.
To actually get a list, you can use yet another LINQ method, ToList:
List<DirectoryObjects> sort = dirObjList.OrderBy(ms => ms.name).ToList();
This will iterate the original collection, sort it as you want, and store the result in a new List<T> object.

Does usage of contains on IEnumerable cast it to a List?

I'm using Linq to filter Data I get from the database. Due to design choices made 1 method returns me an IEnumerable<int> which I then use for a linq statement to see which IDs are permitted to be returned (code follows below). My question here is as I'm not seeing anything there in the documentation: Does the Contains method implicitly cast the IEnumerable to a List for the statement to be executed? (If so the question is if using List in the first place instead of IEnumerable is better).
Code Example
private List<MyData> GetAllPermittedData()
{
IEnumerable<int> permitteddIds = GetPermittedIDs();
return (from a in MyDataHandler.GetAllData() where permittedIds.Contains(a.Id)
select a);
}
Like I asked above I'm not sure if the Contains part implicitly converts permittedIds into a List<int> (for/inside the use of the Contains statement). If this is the case then a followup question would be if it is not better to already use the following statement instead (performance-wise):
private List<MyData> GetAllPermittedData()
{
List<int> permitteddIds = GetPermittedIDs().ToList();
return (from a in MyDataHandler.GetAllData() where permittedIds.Contains(a.Id)
select a);
}

The LINQ operator will attempt to cast it to ICollection<T> first. If the cast succeeds, it uses that method. Since List<T> implements this interface, it will use the list's contain method.
Note that if you use the overload that accepts an IEqualityComparer, it must iterate over the enumerable and the ICollection shortcut is not taken.
You can see this implementation in the .NET Framework reference source:
public static bool Contains<TSource>(this IEnumerable<TSource> source, TSource value) {
ICollection<TSource> collection = source as ICollection<TSource>;
if (collection != null) return collection.Contains(value);
return Contains<TSource>(source, value, null);
}
Jon Skeet also has a good (and lengthy) blog series called "Reimplementing LINQ" where he discusses the implementation in depth. He specifically covers Contains in part 32 of his blog.

The Contains method may try to cast the passed IEnumerable<T> to IList<T> or to ICollection<T>. If the cast succeeds, it may directly use the methods of IList<T>, otherwise it will enumerate over the full sequence.
Note that I am writing may because this is implementation-specific and it is not specified in the docs. As such, it could be different across .NET versions and also in alternative implementations such as Mono.
Your advantage by providing only an IEnumerable<T> is that you have more freedom to exchange the object returned from that property without changing the public interface. The performance cost of the attempted cast to IList<T> or similar should be negligible.
In any case, this way is more performant than your suggestion of calling ToList, as that will actually create a new List<T> and copy all items from the enumeration into it.

Contains exists as an extension method for IEnumerable<T>. But you con't need to convert your IEnumerable to a List<T> with ToList(), you could simply use that IEnumerable<T> to fill a HashSet<T>:
var permitteddIds = new HashSet<int>(GetPermittedIDs());

Generic method for both IEnumerable<T> and IQueryable<T>

I was trying to make a generic method for paging both IEnuemrable<T> and IQueryable<T>
Like this:
public T Paginate<T, TS>(T list) where T : IEnumerable<TS>
{
CheckValidityAndClamp();
return (T)(list.Skip(Page*PageSize).Take(PageSize));
}
When I passed in a List<int> instance it compiles fine.
But when run it gives a cast exception:
System.InvalidCastException : An object of type '<TakeIterator>d__3a`1[System.Int32]' can not be converted to the type 'System.Collections.Generic.List`1[System.Int32]'.
Why is that? A List<int> implements IEnumerable<int> so why the cast exception?

You're not returning a list. Take is implemented with an iterator block, which means that the actual type is a type with no compile time identifier (which is why it looks so weird) and it implements IEnumerable. It is not a List, which you're trying to cast it to.
On top of that, your method can't actually work for any IQueryable objects. It's calling the implementation of Skip that accepts an IEnumerable (because that's what the generic constraint tells the compiler it must implement) so if it were an IQueryable (even if you resolved the messy cast issue) you wouldn't be using the query provider to translate the Skip call. You need to have two overloads here, one for IQueryable and one for IEnumerable; this is pretty much inherent to the problem given that there are two Skip and Take methods that you need to call, one for each of IEnumerable and IQueryable.

To expand on Servy's (correct) answer - another issue is that Skip and Take are extension methods, which is syntactic sugar for calling static methods. Since static methods are bound at compile time, the best information that the compiler has is that T is an IEnumerable<TS>, so the compiler binds Skip and Take to the static methods of Enumerable. There's no way to dynamically bind static methods at run-time. That is why there are two different classes for the extension methods on IEnumerable and IQueryable (Enumerable and Queryable)
You need two overloads.

You get this as you cannot cast the IEnumerable to List (This is why LINQ has .ToList() extension). I think what you want is something like this:
public IEnumerable<T> Paginate<T>(IEnumerable<T> list)
{
return list.Skip(Page * PageSize).Take(PageSize);
}
Which will work with any source that implements IEnumerable (List, T[], IQueryable)
You can then call it like this:
IEnumerable<int> list = someQueryResult;
IEnumerable<int> page = class.Paginate<int>(list);
As pointed out in other answers. IQueryable has it's own set of extension methods. So all though this will work, the paging will not be done as the data-source, as it will use the IEnumerable extensions

How to cast List<object> to List<SomethingElse>

How can i cast a List<object> to List<SomethingElse>?
(where SomethingElse is known to descend from object)
Bonus Chatter
Casting the list:
List<Object> first = ...;
List<SomethingElse> second = (List<SomethingElse>)first;
doesn't work:
Cannot convert type 'System.Collections.Generic.List' to 'System.Collections.Generic.List'
Casting the list:
List<SomethingElse> second = first.Cast<SomethingElse>();
doesn't work:
Cannot implicitely convert type 'System.Collections.Generic.List' to 'System.Collections.Generic.List'
i don't actually need the full List<T> object, just an ICollection<T> will do:
ICollection<SomethingElse> second = first;
ICollection<SomethingElse> second = (ICollection<SomethingElse>)first;
ICollection<SomethingElse> second = first.Cast<SomethingElse>();
don't work.

LINQ, as implemented through the extension methods within the Enumerable class, relies on deferred execution:
Methods that are used in a query that returns a sequence of values do not consume the target data until the query object is enumerated. This is known as deferred execution.
Cast<T> does not create a new list immediately, but rather stores all the information that is required to perform the action. The list would only get enumerated when required (for example, through a foreach statement).
In your case, if you simply intend to iterate over the sequence, you should consider sticking to the IEnumerable<T> interface, which is the declared return type of Cast<T>:
IEnumerable<SomethingElse> second = first.Cast<SomethingElse>();
foreach (SomethingElse se in second)
{
// ...
}
This is efficient, since it only performs the cast as each item is iterated.
If you’re convinced you want a new list to be created immediately, use ToList:
List<SomethingElse> second = first.Cast<SomethingElse>().ToList();
Edit: Replying to point posted in comment:
It depends on what you mean by “a list that can be modified”. There are several LINQ query operators that will allow you to alter the definition of your query further. For example, if you want to remove all SomethingElse elements whose IsDeleted property is true, you can use the Where operator:
IEnumerable<SomethingElse> second = first.Cast<SomethingElse>();
second = second.Where(element => !element.IsDeleted);
If you want to add a sequence of new elements, you can use the Concat operator:
second = second.Concat(anotherCollectionOfSomethingElse);
If you want to sort your sequence in ascending order of ID, use the OrderBy operator:
second = second.OrderBy(element => element.ID);
Each time, we’re applying a query operator over the former definition of our query, and assigning the new (composite) query to our second variable. LINQ would store all your operators in the query definition. Then, when the sequence is actually enumerated (for example, through a foreach or ToList), it would give you the composite result of your sequence, with all the query operators applied in order.
As with all cases of deferred execution / lazy evaluation, be careful not to go overboard with this. If, for example, you’re going to apply a Where operator which will reduce the size of your sequence drastically, it might make sense to execute the query eagerly and store the enumerated list instead.

You have the option of using either Cast or OfType. Cast will throw an exception if you cannot cast to the type specified. OfType on the other hand will return only those items in the list that can be cast to the specified type. I would recommend using OfType in your situation.
List<Foo> fooList = myList.OfType<Foo>().ToList();

I think you're close with the Cast<T> expression. The difference is that Cast<T> returns an IEnumerable<T>, not a List<T>.
Try this:
IEnumerable<SomethingElse> second = first.Cast<SomethingElse>();
You can get a list by doing something similar:
List<SomethingElse> second = first.Cast<SomethingElse>().ToList();

How to know which type to use when use foreach on an IEnumerable type?

I have an object which implements IEnumerable interface. In C#, I can use foreach to iterate all its element. However I am wondering how to determine the type in the foreach loop?

If you have an old style IEnumerable (not IEnumerable<T>) you can usually call .Cast<T> or .OfType<T> to get a strongly typed one to .ForEach<T> over...
If you don't know T then you can call Debug.Write(item.GetType())

If you are using only IEnumerable you have to use object. If you know the real type then you can cast that object.
You can also use the generic IEnumberable and then you won't need to do the cast.

Why don't you try var?
foreach(var item in myEnumerable)
{
...
}

Without having the generic variant of IEumerable there is no nice way of knowing.
You could use the is operator to check first item in enumerable along with different foreach loops but I wouldn't recomend it.

If you are using the non-generic IEnumerable you cannot infer the type of items returned through the interface at compile-time. If you know enough about the object, you may know it only contains a specific type, which means you can (with relative safety) construct a foreach loop with the known type. For example, if you know it only returns strings, you can iterate over the object like so:
foreach(string s in myObject)
{
// Do some operation.
}
Unfortunately, you can return any item from a non-generic IEnumerable. If the IEnumerable returns a float, you will receive an InvalidCastException on the first line.
Worse still, there's nothing to prevent the elements being a variety of types within the same sequence. The object could legally return the following types within one foreach loop.
string
int
float
DateTime
If you don't know from the object's documentation, the only other certain way to know the type of any one item in the loop is to check at runtime.
These are the reasons why it's always preferable to use the generic IEnumerable<T> over IEnumerable when you know your object will only really return one type of value.