Develop Reference

Does usage of contains on IEnumerable cast it to a List?

I'm using Linq to filter Data I get from the database. Due to design choices made 1 method returns me an IEnumerable<int> which I then use for a linq statement to see which IDs are permitted to be returned (code follows below). My question here is as I'm not seeing anything there in the documentation: Does the Contains method implicitly cast the IEnumerable to a List for the statement to be executed? (If so the question is if using List in the first place instead of IEnumerable is better).
Code Example
private List<MyData> GetAllPermittedData()
{
IEnumerable<int> permitteddIds = GetPermittedIDs();
return (from a in MyDataHandler.GetAllData() where permittedIds.Contains(a.Id)
select a);
}
Like I asked above I'm not sure if the Contains part implicitly converts permittedIds into a List<int> (for/inside the use of the Contains statement). If this is the case then a followup question would be if it is not better to already use the following statement instead (performance-wise):
private List<MyData> GetAllPermittedData()
{
List<int> permitteddIds = GetPermittedIDs().ToList();
return (from a in MyDataHandler.GetAllData() where permittedIds.Contains(a.Id)
select a);
}

The LINQ operator will attempt to cast it to ICollection<T> first. If the cast succeeds, it uses that method. Since List<T> implements this interface, it will use the list's contain method.
Note that if you use the overload that accepts an IEqualityComparer, it must iterate over the enumerable and the ICollection shortcut is not taken.
You can see this implementation in the .NET Framework reference source:
public static bool Contains<TSource>(this IEnumerable<TSource> source, TSource value) {
ICollection<TSource> collection = source as ICollection<TSource>;
if (collection != null) return collection.Contains(value);
return Contains<TSource>(source, value, null);
}
Jon Skeet also has a good (and lengthy) blog series called "Reimplementing LINQ" where he discusses the implementation in depth. He specifically covers Contains in part 32 of his blog.

The Contains method may try to cast the passed IEnumerable<T> to IList<T> or to ICollection<T>. If the cast succeeds, it may directly use the methods of IList<T>, otherwise it will enumerate over the full sequence.
Note that I am writing may because this is implementation-specific and it is not specified in the docs. As such, it could be different across .NET versions and also in alternative implementations such as Mono.
Your advantage by providing only an IEnumerable<T> is that you have more freedom to exchange the object returned from that property without changing the public interface. The performance cost of the attempted cast to IList<T> or similar should be negligible.
In any case, this way is more performant than your suggestion of calling ToList, as that will actually create a new List<T> and copy all items from the enumeration into it.

Contains exists as an extension method for IEnumerable<T>. But you con't need to convert your IEnumerable to a List<T> with ToList(), you could simply use that IEnumerable<T> to fill a HashSet<T>:
var permitteddIds = new HashSet<int>(GetPermittedIDs());

Generic method for both IEnumerable<T> and IQueryable<T>

I was trying to make a generic method for paging both IEnuemrable<T> and IQueryable<T>
Like this:
public T Paginate<T, TS>(T list) where T : IEnumerable<TS>
{
CheckValidityAndClamp();
return (T)(list.Skip(Page*PageSize).Take(PageSize));
}
When I passed in a List<int> instance it compiles fine.
But when run it gives a cast exception:
System.InvalidCastException : An object of type '<TakeIterator>d__3a`1[System.Int32]' can not be converted to the type 'System.Collections.Generic.List`1[System.Int32]'.
Why is that? A List<int> implements IEnumerable<int> so why the cast exception?

You're not returning a list. Take is implemented with an iterator block, which means that the actual type is a type with no compile time identifier (which is why it looks so weird) and it implements IEnumerable. It is not a List, which you're trying to cast it to.
On top of that, your method can't actually work for any IQueryable objects. It's calling the implementation of Skip that accepts an IEnumerable (because that's what the generic constraint tells the compiler it must implement) so if it were an IQueryable (even if you resolved the messy cast issue) you wouldn't be using the query provider to translate the Skip call. You need to have two overloads here, one for IQueryable and one for IEnumerable; this is pretty much inherent to the problem given that there are two Skip and Take methods that you need to call, one for each of IEnumerable and IQueryable.

To expand on Servy's (correct) answer - another issue is that Skip and Take are extension methods, which is syntactic sugar for calling static methods. Since static methods are bound at compile time, the best information that the compiler has is that T is an IEnumerable<TS>, so the compiler binds Skip and Take to the static methods of Enumerable. There's no way to dynamically bind static methods at run-time. That is why there are two different classes for the extension methods on IEnumerable and IQueryable (Enumerable and Queryable)
You need two overloads.

You get this as you cannot cast the IEnumerable to List (This is why LINQ has .ToList() extension). I think what you want is something like this:
public IEnumerable<T> Paginate<T>(IEnumerable<T> list)
{
return list.Skip(Page * PageSize).Take(PageSize);
}
Which will work with any source that implements IEnumerable (List, T[], IQueryable)
You can then call it like this:
IEnumerable<int> list = someQueryResult;
IEnumerable<int> page = class.Paginate<int>(list);
As pointed out in other answers. IQueryable has it's own set of extension methods. So all though this will work, the paging will not be done as the data-source, as it will use the IEnumerable extensions

Concise syntax to invoke instance methods after invoking LINQ methods

I have a type that wraps an IEnumerable<T> and contains some additional logic, implementing an interface which we'll call IMyIterable<T>. On one hand, I want people to use LINQ methods to filter and process IMyIterable<T> (by actually applying them to the underlying object), but on the other hand, I still want to fluently expose IMyIterable<T>'s custom instance functionality (e.g state information) after the LINQ methods have been applied. e.g.
var query = myIterable.Select(x => x.whatever);
//query is now of a completely different type than IMyIterable<T>
var result = query.MyCustomThingy();
//But I really want MyCustomThingy to work too.
I know a solution would be:
var query = myIterable.Linq(x => x.Select(y => y.Whatever));
But the extra brackets are unseemly to me. Are there any other ways I could do this? I'm not actually asking for the method Select() to return a custom class (though if there are neat ways to do this, e.g. involving code generation, I'm definitely aboard), just some neater syntax.

LINQ is not tied to IEnumerable. You can implement the LINQ methods you need on IMyIterable itself (either as instance methods or as extension methods). This way you can even use the linq syntax directly on a variable of type IMyIterable.
E.g.:
public interface IMyIterable<T>
{
IMyIterable<TOut> Select<TOut>(Func<T, TOut> selector);
IMyIterable<T> Where(Func<T, bool> predicate);
// ...
}
This would allow you to write something like this:
IMyIterable<T> source = ...;
var query = from item in source
where item.X == Y
select new {...}

Could you implement the methods on IMyIterable as extension methods to IEnumerable instead ? Not knowing exactly what this interface does makes it hard to come up with a solution.

Instead of deriving from IEnumerable, you could create a bunch of equivalent methods on IMyIterable (returning IMyIterable) and forward them to the inner IEnumerable instance.
However this seems like a lot of work to go around a cast. I guess your suggested .Linq() solution would work OK.

Why can Enumerable.Except be used on a string array in C#?

Example
Here is a code example I found at the Pete on Software blog:
var listThree = new string[] { "Pete", "On", "Software" };
var listFour = new string[] { "Joel", "On", "Software" };
stringExcept = listThree.Except(listFour);
The code compiles and runs. So far so good.
Question
However, I don't understand why it works.
So, can anyone explain why I can use Enumerable.Except on a string array?
Perhaps, it will be clear to me if someone could explain how to read the signature of Enumerable.Except and give me a code example:
public static IEnumerable<TSource> Except<TSource>(
this IEnumerable<TSource> first,
IEnumerable<TSource> second
)
What I know
I know the concepts of generics and extension methods. But obviously not good enough to understand the code example above. I have also already used some basic Linq queries.

Except is an extension method which extends any type that implements IEnumerable<T>. This includes the System.Array type which implements IEnumerable<T>.
The note on the linked page explains why the docs don't show System.Array implementing IEnumerable<T>
In the .NET Framework version 2.0, the Array class implements the System.Collections.Generic.IList<T>, System.Collections.Generic.ICollection<T>, and System.Collections.Generic.IEnumerable<T> generic interfaces. The implementations are provided to arrays at run time, and therefore are not visible to the documentation build tools. As a result, the generic interfaces do not appear in the declaration syntax for the Array class, and there are no reference topics for interface members that are accessible only by casting an array to the generic interface type (explicit interface implementations). The key thing to be aware of when you cast an array to one of these interfaces is that members which add, insert, or remove elements throw NotSupportedException.

It just says that if you have an IEnumerable of a given type TSource in this case string you can Except it with another IEnumerable of the same type and get a third IEnumerable of the same type back. The key point is that the two IEnumerable inputs have to be the same (and obviously the return will be of the same type).

An array of T (or say a T[]), is also an IEnumerable<T>. In your question, T is System.String. And Enumerable.Except is an extension method on IEnumerable<T>, so it's also working for a string[]. And stringExcept = listThree.Except(listFour); equals to
stringExcept = Enumerable.Except(listThree, listFour).

The compiler will match the TSource argument to string as a string array implements the IEnumerable<string> interface and thus matches the first argument of the extension method. So the answer is two things:
string[] implements IEnumerable<string>
The compiler is intelligent enough to infer the generic arguments

The Except method returns the elements in the first enumerable that do not also appear in the
second enumerable. So in the case you specified, the result would be {"Pete", "Joel"}.
In this case, thinking in terms of string arrays is perhaps a red herring. It might be more advantageous to think in terms of object equality (http://msdn.microsoft.com/en-us/library/system.object.equals.aspx).
The Microsoft documentation is here: http://msdn.microsoft.com/en-us/library/system.linq.enumerable.except.aspx

IEnumerable.ToArray<T>() vs. IEnumerable.Cast<T>().ToArray()

When trying to obtain an array of objects from an IEnumerable collection of objects (cast differently than the array I want), I know I can first cast the source collection to the proper type, and then obtain an array from that, but the method ToArray<T>() gives me the impression that it can handle both of those operations in one step. From my experience, though, I have never been able to find a case where the ToArray<T>() method works for any T except the original source's T (which, in my mind, makes ToArray<T>() silly, since it does the same thing that the non-generic ToArray() already does).
So my question is, am I missing the point of the ToArray<T>() method, and I'm trying to make it do something it was never intended for, or is there something silly I'm missing with regard to the method, and what I'm trying to do does generally follow its intent?
Here is a concrete example to illustrate my issue:
public interface IFoo { }
public class Foo : IFoo { }
static void Main(string[] args)
{
// Suppose a list of Foos was created
List<Foo> src = new List<Foo>();
// I would be safe obtaining an array of IFoos from that list, but
// This is not supported (although intellisense shows the method is there, the compiler balks):
// IFoo[] results = src.ToArray<IFoo>();
// Whereas this works just fine:
IFoo[] results = src.Cast<IFoo>().ToArray();
}

The reason ToArray<T>() is generic is so that it can operate on any IEnumerable<T>, not so that you can supply a different T:
public static T[] ToArray<T>(this IEnumerable<T> self) { ... }
You should never need to provide the T yourself. If you did, as in your example, the method would expect to receive, for example, an IEnumerable<IFoo>, which you are not supplying.
FYI, there is no "non-generic ToArray()". The compiler is inferring the T generic argument based on the type of enumerable you call ToArray() on.