Develop Reference

Invalid cast from IEnumerable to List

I'm working on a C# script within a Unity3D Project where I'm trying to take a list of strings and get a 2D list of the permutations. Using this answer's GetPermutations() in the following fashion:
List<string> ingredientList = new List<string>(new string[] { "ingredient1", "ingredient2", "ingredient3" });
List<List<string>> permutationLists = GetPermutations(ingredientList, ingredientList.Count);
But it throws an implicit conversion error:
IEnumerable<IEnumerable<string>> to List<List<string>> ... An explicit conversion exists (are you missing a cast)?
So I looked at a few places, such as here and came up with the following modification:
List<List<string>> permutationLists = GetPermutations(ingredientList, ingredientList.Count).Cast<List<string>>().ToList();
But it breaks at runtime, gets handled internally, and allows it to continue without indicating a failure – probably because it's running in Unity3D.
Here is what I see in Unity3D after I stop debugging the script:
InvalidCastException: Cannot cast from source type to destination type.
System.Linq.Enumerable+<CreateCastIterator>c__Iterator0`1[System.Collections.Generic.List`1[System.String]].MoveNext ()
System.Collections.Generic.List`1[System.Collections.Generic.List`1[System.String]].AddEnumerable (IEnumerable`1 enumerable) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Collections.Generic/List.cs:128)
System.Collections.Generic.List`1[System.Collections.Generic.List`1[System.String]]..ctor (IEnumerable`1 collection) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Collections.Generic/List.cs:65)
System.Linq.Enumerable.ToList[List`1] (IEnumerable`1 source)
Which I interpret as still casting incorrectly, so I also attempted the following approaches and more that I can't remember:
List<List<string>> permutationLists = GetPermutations(ingredientList, ingredientList.Count).Cast<List<List<string>>>();
List<List<string>> permutationLists = GetPermutations(ingredientList.AsEnumerable(), ingredientList.Count);
as well as explicitly casting with parenthesis before the method call like you would in C or Java, still to no avail.
So how should I be casting the results from the GetPermutations() function to get a List<List<string>>? Or alternatively, how could I modify the function to only return List<List<string>> since I don't need it to work for a generic type? I tried to modify the method myself to be the following:
List<List<string>> GetPermutations(List<string> items, int count)
{
int i = 0;
foreach(var item in items)
{
if(count == 1)
yield return new string[] { item };
else
{
foreach(var result in GetPermutations(items.Skip(i + 1), count - 1))
yield return new string[] { item }.Concat(result);
}
++i;
}
}
However, having removed the <T> from the function name it breaks stating that the body cannot be an iterator block. I have no prior experience with C# and I'm rusty with template functions in strongly typed languages, so any explanation/help is appreciated.
I wasn't sure how to look this issue up, so if this is a duplicate just post it here and I'll delete this post immediately.

So how should I be casting the results from the GetPermutations() function to get a List<List<string>>
Best solution: don't. Why do you need to turn the sequence into a list in the first place? Keep it as a sequence of sequences.
If you must though:
GetPermutations(...).Select(s => s.ToList()).ToList()
If you want to modify the original method, just do the same thing:
IEnumerable<List<string>> GetPermutations(List<string> items, int count)
{
int i = 0;
foreach(var item in items)
{
if(count == 1)
yield return new List<T>() { item };
else
{
foreach(var result in GetPermutations(items.Skip(i + 1), count - 1))
yield return (new string[] {item}.Concat(result)).ToList();
}
++i;
}
}
And then do GetPermutations(whatever).ToList() and you have a list of lists. But again, do not do this. Keep everything in sequences if you possibly can.
I want to turn the sequence into a list so that I can sort the elements alphabetically and re-join them as a sorted, single comma-delimited string.
OK, then do that. Let's rewrite your method as an extension method Permute(). And let's make some new one-liner methods:
static public string CommaSeparate(this IEnumerable<string> items) =>
string.Join(",", items);
static public string WithNewLines(this IEnumerable<string> items) =>
string.Join("\n", items);
static public IEnumerable<string> StringSort(this IEnumerable<string> items) =>
items.OrderBy(s => s);
Then we have the following -- I'll annotate the types as we go:
string result =
ingredients // List<string>
.Permute() // IEnumerable<IEnumerable<string>>
.Select(p => p.StringSort()) // IEnumerable<IEnumerable<string>>
.Select(p => p.CommaSeparate())// IEnumerable<string>
.WithNewLines(); // string
And we're done. Look at how clear and straightforward the code is when you make methods that do one thing and do it well. And look at how easy it is when you keep everything in sequences, as it should be.

Your question is related to several aspects of C# and .net types system. I will try to provide simple explanation and will provide links as more formal answers.
So, according to your description it looks like GetPermutations(ingredientList, ingredientList.Count); returns IEnumerable<IEnumerable<string>> but you are trying to assign this result to the variable of another type, in pseudo code:
List<List<string>> = IEnumerable<IEnumerable<string>>;
List<T> implements IEnumerable<T>, so in general it is possible to make this assignment:
IEnumerable<T> = List<T>;
but the problem is that in your case T on the left side differs from the T on the right side.
for IEnumerable<IEnumerable<string>> T is IEnumerable<string>.
for List<List<string>> T is List<string>
To fix your problem we should change the code to have the same T on the left and right sides i.e. convert T to either List<string> or IEnumerable<string>.
You can convert T to the List<string> this way:
IEnumerable<List<string> GetPermutationsList(List<string> items, int count)
{
return GetPermutations(items, count).Select(x=>x.ToList())
}
IEnumerable<List<string>> permutationLists = GetPermutations(ingredientList.AsEnumerable(), ingredientList.Count);
// or
List<List<string>> permutationLists = GetPermutations(ingredientList.AsEnumerable(), ingredientList.Count).ToList();
but in general it is not good idea to use List in all places. Use lists only where you really need it. Important points here:
IEnumerable<T> provides minimum functionality (enumeration only) that should be enougth for your goals.
IList <T> (List implements it ) provides maximum functionality (Add, Remove ,“random” access by index). Do you really need maximum functionality?
Also using ToList() can cause memory shortage problem for big data.
ToList() just forces immediate query evaluation and returns a List<T>
Some useful information: covariance-contr-variance, List, casting

Comparing two lists of nested lists and returning the added/changed/removed items

I've looked at many similar questions on stackoverflow, but I haven't seen an exact match for my problem.
I need to compare two "lists of nested lists" and capture the differences. One is an "old" list and the other is a "new" list. When comparing the nested lists, they can be considered equal if all of the NESTED list items (the MyObject.Ids) are present in both lists in order (you can assume that the nested MyObject.Ids lists are already sorted and that there are no duplicates). The MyObject.Id and MyObject.Name properties are not considering in the equality comparison, but they are still important metadata for MyObject's which should not get lost.
I am not looking for a boolean indicator of equality. Instead I need to create three new lists which capture the differences between the old and new lists (e.g. a list of items which were Added, a list of items which were Removed, and a list of items which were present in both lists).
Below is an example of some code which does exactly what I want! What I would like to know is how to make this shorter/better/simpler (cutting out one of the for loops would be a good start). To make things trickier, please assume that you cannot make any changes to the MyObject class or use any custom Equals/IEqualityComparer etc implementations.
public class MyObject
{
public Guid Id { get; set; }
public string Name { get; set; }
public List<Guid> Ids { get; set; }
}
...
// Get the list of existing objects (assume this returns some populated list)
List<MyObject> existingObjects = GetExistingObjects();
// Create a list of updated objects
List<MyObject> updatedObjects = new List<MyObject>()
{
new MyObject()
{
Ids = new List<Guid>() { new Guid("48af3cb9-945a-4ab9-91e4-7ee5765e5304"), new Guid("54b5128a-cf53-436c-9d88-2ef7abd15140") }
},
new MyObject()
{
Ids = new List<Guid>() { new Guid("0485382f-8f92-4a71-9eba-09831392ceb9"), new Guid("3d8b98df-caee-41ce-b802-2f0c5f9742de") }
}
};
// Do the comparison and capture the differences
List<MyObject> addedObjects = new List<MyObject>();
List<MyObject> removedObjects = new List<MyObject>();
List<MyObject> sameObjects = new List<MyObject>();
foreach (MyObject obj in updatedObjects)
{
if (existingObjects.Any(list => list.Ids.SequenceEqual(obj.Ids)))
{
sameObjects.Add(obj);
continue;
}
addedObjects.Add(obj);
}
foreach (MyObject obj in existingObjects)
{
if (!updatedObjects.Any(list => list.Ids.SequenceEqual(obj.Ids)))
{
removedObjects.Add(obj);
}
}

Here is a little shorter (due to elimination of the second loop) and little better (due to elimination of the ineffective search contained in the second loop). Still O(N^2) time complexity due to ineffective search contained in the loop though.
var addedObjects = new List<MyObject>();
var removedObjects = new List<MyObject>(existingObjects);
var sameObjects = new List<MyObject>();
foreach (var newObject in updatedObjects)
{
int index = removedObjects.FindIndex(oldObject => oldObject.Ids.SequenceEqual(newObject.Ids));
if (index < 0)
addedObjects.Add(newObject);
else
{
removedObjects.RemoveAt(index);
sameObjects.Add(newObject);
}
}
Update: A shorter, but IMO definitely not better (in fact worse performance wise) version
var addedObjects = updatedObjects.Where(newObject => !existingObjects.Any(oldObject => oldObject.Ids.SequenceEqual(newObject.Ids))).ToList();
var removedObjects = existingObjects.Where(oldObject => !updatedObjects.Any(newObject => newObject.Ids.SequenceEqual(oldObject.Ids))).ToList();
var sameObjects = updatedObjects.Where(newObject => !addedObjects.Any(addedObject => addedObject.Ids.SequenceEqual(newObject.Ids))).ToList();
If MyObject does not define custom equality comparison, i.e. uses default reference equality, the last line could be replaced with shorter and better performing
var sameObjects = updatedObjects.Except(addedObjects);

You can use Intersect and Except function in Linq
With Intersect you will get existing object,
and with Except you will get new objects.
Example of Except from MSDN:
double[] numbers1 = { 2.0, 2.1, 2.2, 2.3, 2.4, 2.5 };
double[] numbers2 = { 2.2 };
IEnumerable<double> onlyInFirstSet = numbers1.Except(numbers2);
foreach (double number in onlyInFirstSet)
Console.WriteLine(number);

What is the best way to trim a list?

I have a List of strings. Its being generated elsewhere but i will generate it below to help describe this simplified example
var list = new List<string>();
list.Add("Joe");
list.Add("");
list.Add("Bill");
list.Add("Bill");
list.Add("");
list.Add("Scott");
list.Add("Joe");
list.Add("");
list.Add("");
list = TrimList(list);
I would like a function that "trims" this list and by trim I want to remove all items at the end of the array that are blank strings (the final two in this case).
NOTE: I still want to keep the blank one that is the second item in the array (or any other one that is just not at the end) so I can't do a .Where(r=> String.isNullOrEmpty(r))

I would just write it without any LINQ, to be honest- after all, you're modifying a collection rather than just querying it:
void TrimList(List<string> list)
{
int lastNonEmpty = list.FindLastIndex(x => !string.IsNullOrEmpty(x));
int firstToRemove = lastNonEmpty + 1;
list.RemoveRange(firstToRemove, list.Count - firstToRemove);
}
If you actually want to create a new list, then the LINQ-based solutions are okay... although potentially somewhat inefficient (as Reverse has to buffer everything).

Take advantage of Reverse and SkipWhile.
list = list.Reverse().SkipWhile(s => String.IsNullOrEmpty(s)).Reverse().ToList();

List<T> (not the interface) has a FindLastIndex method. Therefore you can wrap that in a method:
static IList<string> TrimList(List<string> input) {
return input.Take(input.FindLastIndex(x => !string.IsNullOrEmpty(x)) + 1)
.ToList();
}
This produces a copy, whereas Jon's modifies the list.

The only solution I can think of is to code a loop that starts at the end of the list and searches for an element that is not an empty string. Don't know of any library functions that would help. Once you know the last good element, you know which ones to remove.
Be careful not to modify the collection while you are iterating over it. Tends to break the iterator.

I always like to come up with the most generic solution possible. Why restrict yourself with lists and strings? Let's make an algorithm for generic enumerable!
public static class EnumerableExtensions
{
public static IEnumerable<T> TrimEnd<T>(this IEnumerable<T> enumerable, Predicate<T> predicate)
{
if (predicate == null)
{
throw new ArgumentNullException("predicate");
}
var accumulator = new LinkedList<T>();
foreach (var item in enumerable)
{
if (predicate(item))
{
accumulator.AddLast(item);
}
else
{
foreach (var accumulated in accumulator)
{
yield return accumulated;
}
accumulator.Clear();
yield return item;
}
}
}
}
Use it like this:
var list = new[]
{
"Joe",
"",
"Bill",
"Bill",
"",
"Scott",
"Joe",
"",
""
};
foreach (var item in list.TrimEnd(string.IsNullOrEmpty))
{
Console.WriteLine(item);
}

Adding to a mutable type as if it was immutable (orderlessly)

I'm building an object on form:
return table.Rows.Cast<DataRow>()
.Select(row => new Something
{
Field = row["field1"] as int?,
Bunch = GetBunch(index)
});
with GetBunch() as follows.
private IList GetBunch(int index) { ... }
It works as supposed to. Now, we've noticed that we need to add an additional element to the array that we put into Bunch. For a range of reasons, it's not an option to change the signature of the method, nor is it feasible to add the extra element inside it.
I tried to add the extra thingy like so:
return table.Rows.Cast<DataRow>()
.Select(row => new Something
{
Field = row["field1"] as int?,
Bunch = GetBunch(index).Add(new Thingy() { ... })
});
but it didn't work because Add() doesn't return the original array with the new element in it. It returns an int. If the original object would be immutable, the result of Add() would be what I wish but, apparently, it isn't.
Is it possible to immutabilize an mutable object? If not (because I'm fairly certain it's not), how can I handle this easily? (Easily = without storing the created thing and then accessing it and adding an element to Bunch property's array.)
NB, the order of the result is of no importance in this case. The proper type would be Bag of some kind but I'm hand-tied by the pre-existing design.

Something like this maybe:
return table.Rows.Cast<DataRow>()
.Select(row => {
var list = GetBunch(index);
list.Add(new Thingy() { ... });
return new SomeThing
{
Field = row["field1"] as int?,
Bunch = list
};
});
As #Chris noted in the comments you can use a lambda statement, you don't necessarily need to use lamda expression.You can do whatever you want inside of the blocks since it's just a method that takes a DataRow and returns SomeThing.

Alternative to good Selman22's answer: create helper extension method that would let you add to result of GetBunch() without need to wrap it in lambda inline:
static class MyListExtenstions
{
static IList AddToList<T>(this IList list, T item)
{
list.Add(item);
return list;
}
}
And use it inline:
return table.Rows.Cast<DataRow>()
.Select(row => new Something
{
Field = row["field1"] as int?,
Bunch = GetBunch(index).AddToList(new Thingy() { ... })
});
One more LINQ approach that creates more intermediate objects, but would work on immutable (meaning adding items throws exception) list:
If you want to have method that can be used for chaining you can use IEnumerable<T>.Concat() (and than .ToList() in your case). This approach will give you new list and would be useful if result of GetBunch is :
GetBunch(index)
.Concat(Enumerable.Repeat(new Thingy() { ... }, 1))
.ToList()
NOTE: If your GetBunch would return generic IList<T> or IEnumerable<T> than you can use helper methods of Enumerable (like .Concat()). Since it returns non-generic version you need to convert it to generic variant with something like Enumerable.Cast or cast to generic interface to use LINQ approach. You need to know what actual type is there OR what type of items should be returned.
GetBunch(index).Cast<object>()
GetBunch(index) as IList<Thingy>

return table.Rows.Cast<DataRow>()
.Select(row => new Something
{
Field = row["field1"] as int?,
Bunch = GetBunch(index).Union(x =>
new[] { new Thingy() { ... } }
).ToList()
});

Adding items to an IEnumerable through an extension method does not work?

In most of the methods I use that return some kind of collection I return IEnumerable rather than the specific type (e.g. List). In many cases I have another collection that I want to combine with the result IEnumerable, this would be exactly like taking a List and adding another List to it using the AddRange method. I have the following example, in it I have created an extension method that should take a collection of items to add and adds them to a base collection, while debugging this appears to works but in the original collection the items are never added. I don't understand this, why aren't they added, is there something about the implementation of the IEnumerable that I am missing? I understand that IEnumerable is a read only interface, but Iam not adding to this list in the example below, I am replacing it, but the original IEnumerable does not change.
class Program
{
static void Main(string[] args)
{
var collectionOne = new CollectionContainerOne();
var collectionTwo = new CollectionContainerTwo();
// Starts at 1- 50 //
collectionOne.Orders.AddRange(collectionTwo.Orders);
// Should now be 100 items but remains original 50 //
}
}
public class CollectionContainerOne
{
public IEnumerable<Order> Orders { get; set; }
public CollectionContainerOne()
{
var testIds = Enumerable.Range(1, 50);
var orders = new List<Order>();
foreach (int i in testIds)
{
orders.Add(new Order() { Id = i, Name = "Order #" + i.ToString() });
}
this.Orders = orders;
}
}
public class CollectionContainerTwo
{
public IEnumerable<Order> Orders { get; set; }
public CollectionContainerTwo()
{
var testIds = Enumerable.Range(51, 50);
var orders = new List<Order>();
foreach (int i in testIds)
{
orders.Add(new Order() { Id = i, Name = "Order #" + i.ToString() });
}
this.Orders = orders;
}
}
public class Order
{
public int Id { get; set; }
public string Name { get; set; }
public override string ToString()
{
return this.Name;
}
}
public static class IEnumerable
{
public static void AddRange<T>(this IEnumerable<T> enumerationToAddTo, IEnumerable<T> itemsToAdd)
{
var addingToList = enumerationToAddTo.ToList();
addingToList.AddRange(itemsToAdd);
// Neither of the following works //
enumerationToAddTo.Concat(addingToList);
// OR
enumerationToAddTo = addingToList;
// OR
enumerationToAddTo = new List<T>(addingToList);
}
}

You are modifying the parameter enumerationToAddTo, which is a reference. However, the reference is not itself passed by reference, so when you modify the reference, the change is not observable in the caller. Furthermore, it is not possible to use ref parameters in extension methods.
You are better off using Enumerable.Concat<T>. Alternatively, you can use ICollection, which has an Add(T) method. Unfortunately, List<T>.AddRange isn't defined in any interface.
Here is an example to illustrate the passing of reference types by reference. As Nikola points out, this is not really useful code. Don't try this at home!
void Caller()
{
// think of ss as a piece of paper that tells you where to find the list.
List<string> ss = new List<string> { "a", "b" };
//passing by value: we take another piece of paper and copy the information on ss to that piece of paper; we pass that to the method
DoNotReassign(ss);
//as this point, ss refers to the same list, that now contains { "a", "b", "c" }
//passing by reference: we pass the actual original piece of paper to the method.
Reassign(ref ss);
// now, ss refers to a different list, whose contents are { "x", "y", "z" }
}
void DoNotReassign(List<string> strings)
{
strings.Add("c");
strings = new List<string> { "x", "y", "z" }; // the caller will not see the change of reference
//in the piece of paper analogy, we have erased the piece of paper and written the location
//of the new list on it. Because this piece of paper is a copy of SS, the caller doesn't see the change.
}
void Reassign(ref List<string> strings)
{
strings.Add("d");
//at this point, strings contains { "a", "b", "c", "d" }, but we're about to throw that away:
strings = new List<string> { "x", "y", "z" };
//because strings is a reference to the caller's variable ss, the caller sees the reassignment to a new collection
//in the piece of paper analogy, when we erase the paper and put the new object's
//location on it, the caller sees that, because we are operating on the same
//piece of paper ("ss") as the caller
}
EDIT
Consider this program fragment:
string originalValue = "Hello, World!";
string workingCopy = originalValue;
workingCopy = workingCopy.Substring(0, workingCopy.Length - 1);
workingCopy = workingCopy + "?";
Console.WriteLine(originalValue.Equals("Hello, World!"); // writes "True"
Console.WriteLine(originalValue.Equals(workingCopy); // writes "False"
If your assumption about reference types were true, the output would be "False" then "True"

Calling your extensions method like this:
collectionOne.Orders.AddRange(collectionTwo.Orders);
Is essentially the same as:
IEnumerable.AddRange(collectionOne.Orders, collectionTwo.Orders);
Now what happens there, is you pass copy of reference to the collectionOne.Orders to the AddRange method. In your AddRange implementation you try to assign new value to the copy. It gets "lost" inside. You are not assigning new value to collectionOne.Orders, you assign it to its local copy - which scope is only within the method body itself. As a result of all modifications happenining inside AddRange, outside world notices no changes.
You either need to return new enumerable, or work on lists directly. Having mutating methods on IEnumerable<T> is rather counterintuitive, I'd stay away from doing that.

What you want exists and is called Concat. Essentially, when you do this in your Main:
var combined = collectionOne.Orders.Concat(collectionTwo.Orders);
Here, combined will refer to an IEnumerable that will traverse both source collections when enumerated.

IEnumerable does not support adding. What you in essence did in your code is create new collection from your enumerable, and add items to that new collection. Your old collection still has same items.
E.g., you create a collection of numbers like this
Collection1 = [ 1, 2, 3, 4, 5 ]
when you do Collection1.ToList().Add(...) you will get new collection with same members, and add new members like so:
Collection1 = [ 1, 2, 3, 4, 5, 6, 7, ... ]
your old collection will however still hold old members, as ToList creates new collection.
Solution #1:
Instead of using IEnumerable use IList which supports modification.
Solution #2 (bad):
Cast your IEnumerable back to it's derived type and add members to it. This is quite bad though, in fact it's better to just return List in the first place
IEnumerable<Order> collectionOne = ...;
List<Order> collectionOneList = (List<Order>)collectionOne;
collectionOneList.Add(new Order());
General guideline (best):
If you are returning collections which are standard in .NET there is no reason to return their interfaces. In this case it's best to use original type. If you are however returning collection which you implemented yourself, then you should return an interface
It's a completely different case when you are thinking about input parameters. If your method is asking to enumerate over items, then you should ask for IEnumerable. This way you can do what you need over it, and you are placing least constraint on person who is calling it. They can send any enumerable. If you need to add to that collection, you may require IList so that you can also modify it in your method.

Basically the problem is that you can't assign a value to enumerationToAddTo partially because it isn't a reference parameter. Also as phoog mentions ToList() creates a new list and does not cast the existing IEnumerable to a list.
This isn't really a good use of a extension. I would recommend that you add a method to your container collection that allows you add add new items to the IEnumerable instance. This would better encapsulate the logic that's particular to that class.