Develop Reference

Can I convert Dictionary loop inside of a foreach loop into a LINQ statement

If possible please help me convert these nested loops into a LINQ statement
Thank you very much for your help!
public static List<Term> GetTermsByName(this IEnumerable<Term> terms, Dictionary<string, string> termInfo)
{
List<Term> termList = new List<Term>();
foreach (Term term in terms)
{
foreach (var value in termInfo.Values)
{
if (term.Name == value)
{
termList.Add(term);
}
}
}
return termList;
}

Maybe Contains method is what you are after
Determines whether a sequence contains a specified element.
The following can be read as, Filter all Terms where the Term.Name exists in the dictionary Values
var values = termInfo.Values;
var result = terms.Where(term => values.Contains(term.Name));
.ToList();
// or
var result = terms.Where(term => termInfo.Values.Contains(term.Name));
.ToList();

You're losing the plot of the dictionary a bit here, don't you think? The speediness is in using the keys. However, you can still do better than a nested foreach or an inline linq equivalent with where and contains. Use a join to at least improve your efficiency.
var termList = (from term in terms
join value in termInfo.Values
on term.Name equals value
select term)
.Distinct() // if there are duplicates in either set
.ToList();

querying list within a list using linq

I'm trying to learn linq, but struggling with some concepts. How would I transform this double foreach loop into a linq query please?
foreach (var l1 in list1)
{
foreach (var l2 in list2)
{
if (l1 == l2)
{
list1.Remove(l1);
}
}
}

var list3 = list1.Except(list2);
LINQ does not mutate lists.

If list1 is declared as a List<T>, then you can do this:
list1.RemoveAll(list2.Contains);
You might find that a little difficult to read. The above is essentially equivalent to:
list1.RemoveAll(item1 => list2.Contains(item1));
Note that this solution is not based on LINQ. However, if list2's type does not have a Contains method, then LINQ can help you out with its .Contains extension method; add a using System.Linq; directive to your code in that case.)
P.S.: Please make sure that you have read and understood my above comment: LINQ's purpose is querying for data, not modifying it.

There is number of ways to do it, one example is to use Intersect:
var inBothLists = list1.Intersect(list2);
inBothLists.ToList().ForEach(i => list1.Remove(i));

Linq is for querying, not updating, but you could query for the items that need to be removed and then remove them in a loop:
var itemsToRemove = list1.Where(l2.Contains(l1));
foreach(var item in itemsToRemove)
{
list1.Remove(item)
}

Memory optimized OrderBy and Take?

I have 9 GB of data, and I want only 10 rows. When I do:
data.OrderBy(datum => datum.Column1)
.Take(10)
.ToArray();
I get an OutOfMemoryException. I would like to use an OrderByAndTake method, optimized for lower memory consumption. It's easy to write, but I guess someone already did. Where can I find it.
Edit: It's Linq-to-objects. The data comes from a file. Each row can be discarded if its value for Column1 is smaller than the current list of 10 biggest values.

I'm assuming you're doing this in Linq to Objects. You could do something like...
var best = data
.Aggregate(new List<T>(), (soFar, current) => soFar
.Concat(new [] { current })
.OrderBy(datum => datum.Column1)
.Take(10)
.ToList());
In this way, not all the items need to be kept in a new sorted collection, only the best 10 you're interested in.
This was the least code way. Since you know the soFar list is sorted, testing where/if to insert current could be optimized. I didn't feel like doing ALL the work for you. ;-)
PS: Replace T with whatever your type is.
EDIT: Thinking about it, the most efficient way would actually be a plain old foreach that compares each item to the running list of best 10.

It figures: OrderBy is a Sort and that requires storing all the elements (deferred execution is cancelled).
It ought to work efficiently when data is an IQueryable, then it's up to the database.
// just 4 fun
public static IEnumerable<T> TakeDistinctMin<T, TKey>(this IEnumerable<T> #this,
int n, Func<T, TKey> selector)
where TKey: IComparable<TKey>
{
var tops = new SortedList<TKey, T>(n+1);
foreach (var item in #this)
{
TKey k = selector(item);
if (tops.ContainsKey(k))
continue;
if (tops.Count < n)
{
tops.Add(k, item);
}
else if (k.CompareTo(tops.Keys[tops.Count - 1]) < 0)
{
tops.Add(k, item);
tops.RemoveAt(n);
}
}
return tops.Values;
}

To order a set of unordered objects you have to look at all of them, no?
I don't see how you'd be able to avoid parsing all 9 GB of data to get the first 10 ordered in a certain way unless the 9 GB of data was already ordered in that fashion or if there were indexes or other ancillary data structures that could be utilized.
Could you provide a bit more background on your question. Are you querying a database using LINQ to SQL or Entity Framework or some other O/RM?

You can use something like this together with a projection comparer:
public static IEnumerable<T> OrderAndTake<T>(this IEnumerable<T> seq,int count,IComparer<T> comp)
{
var resultSet=new SortedSet<T>(comp);
foreach(T elem in seq)
{
resultSet.Add(elem);
if(resultSet.Count>count)
resultSet.Remove(resultSet.Max);
}
return resultSet.Select(x=>x);
}
Runtime should be O(log(count)*seq.Count()) and space O(min(log(count),seq.Count()))
One issue is that it will break if you have two elements for which comp.Compare(a,b)==0 since the set doesn't allow duplicate entries.

What's the most (performance) efficient and readable way to 'split' a generic list based on a condition?

(highly simplified example)
I have a generic list of strings:
var strings = new List<string> { "abc", "owla", "paula", "lala", "hop" };
I'm looking for the most efficient way to split this list into a list with elements that meet a condition and a list of elements that don't meet that same condition.
Func<string, bool> condition = s => s.IndexOf("o") > -1;
Predicate<string> kickOut = s => s.IndexOf("o") > -1;
var stringsThatMeetCondition = strings.Where(condition);
strings.RemoveAll(kickOut);
var stringsThatDontMeetCondition = strings;
Is there a way to do this with looping only once through the original list?

Use some linq:
var matches = list.Select(s => s.IndexOf("o") > -1).ToList();
var notMatches = list.Except(matches).ToList();
list.Clear();
list.AddRange(matches);
Update: as has been mentioned in the comments, be careful mutating the list as linq methods try to be on-demand, they will not iterate the list until you start looking into the IEnumerable. However in my case, I call ToList, which effectively causes it to run through the entire list of items.

This would do it:
IEnumerable<T> FilterAndRemove(this List<T> list, Func<T, bool> pred)
{
List<T> filtered = new List<T>();
int i = 0;
while(i < list.Count)
{
if (pred(list[i]))
{
filtered.Add(list[i]);
list.RemoveAt(i);
}
else
{
++i;
}
}
return list;
}
But am sure you have already thought of something similar. Can you please update your answer with the kind of efficiency that you seek?
Note that two filtering runs with pred and !pred over the original list would still be O(n) and not at all inefficient. Especially considering that you'd get the full benefit of lazy evaluation for both result sets. See also Rob's answer.
This algorithm is in O(n^2).
Instead removing each element, you can also collect them in a new list and copy them over to the input list before returning. This will also get you O(n).
One more option for O(n) would be switching to a linked list.

Why not just use
var stringsThatMeetCondition = strings.Where(condition);
var stringsThatDontMeetCondition = strings.Where(x => !condition(x));
Of course, you end up applying the condition to each element in the list twice. To avoid this you might want to write a generic splitting function, which wouldn't be as neat.

Func<string, bool> condition = ...;
var groupedStrings = strings.GroupBy(condition)
var stringsMeetingCondition = groupedStrings.FirstOrDefault(g => g.Key);
var stringsNotMeetingCondition = groupedStrings.FirstOrDefault(g => !g.Key);

LINQ query to select based on property

IEnumerable<MyClass> objects = ...
foreach(MyClass obj in objects)
{
if(obj.someProperty != null)
SomeFunction(obj.someProperty);
}
I get the feeling I can write a smug LINQ version using a lambda but all my C# experience is 'classical' i.e more Java-like and all this Linq stuff confuses me.
What would it look like, and is it worth doing, or is this kind of Linq usage just seen as showing off "look I know Linq!"

LINQ itself doesn't contain anything for this - I'd would use a normal foreach loop:
foreach (var value in objects.Select(x => x.someProperty)
.Where(y => y != null))
{
SomeFunction(value);
}
Or if you want a query expression version:
var query = from obj in objects
let value = obj.SomeProperty
where value != null
select value;
foreach (var value in query)
{
SomeFunction(value);
}
(I prefer the first version, personally.)
Note that I've performed the selection before the filtering to avoid calling the property twice unnecessarily. It's not for performance reasons so much as I didn't like the redundancy :)
While you can use ToList() and call ForEach() on that, I prefer to use a straight foreach loop, as per Eric's explanation. Basically SomeFunction must incur a side-effect to be useful, and LINQ is designed with side-effect-free functions in mind.

objects.where(i => i.someProperty != null)
.ToList()
.ForEach(i=> SomeFunction(i.someProperty))

Although it can be done with Linq, sometimes its not always necessary. Sometimes you lose readability of your code. For your particular example, I'd leave it alone.

One option is to use the pattern outlined in the book Linq In Action which uses an extension method to add a ForEach operator to IEnumerable<>
From the book:
public static void ForEach<T> (this IEnumerable<T> source, Action<T> func)
{
foreach (var item in source)
func(item)
}
Then you can use that like this:
(from foo in fooList
where foo.Name.Contains("bar")
select foo)
.ForEach(foo => Console.WriteLine(foo.Name));

LINQ is used to create a result, so if you use it to call SomeFunction for each found item, you would be using a side effect of the code to do the main work. Things like that makes the code harder to maintain.
You can use it to filter out the non-null values, though:
foreach(MyClass obj in objects.Where(o => o.someProperty != null)) {
SomeFunction(obj.someProperty);
}

You can move the if statement into a Where clause of Linq:
IEnumerable<MyClass> objects = ...
foreach(MyClass obj in objects.Where(obj => obj.someProperty != null)
{
SomeFunction(obj.someProperty);
}
Going further, you can use List's ForEach method:
IEnumerable<MyClass> objects = ...
objects.Where(obj => obj.someProperty != null).ToList()
.ForEach(obj => SomeFunction(obj.someProperty));
That's making the code slightly harder to read, though. Usually I stick with the typical foreach statement versus List's ForEach, but it's entirely up to you.