Develop Reference

What is the best way to take a small portion of items from a large RealmCollection?

I'm currently working on a Xamarin.Forms project using Realm 5.1.2. I need to take a collection of RealmObjects, filter them into a smaller set, and convert them to a UI element for displaying data based on each item. On smaller data sets, the implementation I have works well. However, when attempting to do this with a large data set (1,000,000+ items to start, filtered down to ~1000), the app slows down drastically when attempting to convert the filtered objects.
So far, I've attempted to fix this by containing my filtering process to IQueryables and IRealmCollection. I'm aware that instantiating all the objects in my final result is causing the slow down. I'm looking to add a lazy loading method, since not every element will be visible at a time (only loading 10 at a time, and request more as needed). My current implementation is:
Creating the RealmCollection once:
var filteredRealmCollection = localRealm.All<Field>()
.Where(field => field.IndexedId == input.TargetIndexId) //Use indexed parameter for initial filter
.Where(field => field.Value == input.TargetValue) //Take all fields that match the requested filter value
.AsRealmCollection();
Calling for new items as needed:
IEnumerable<FieldItemViewModel> GetFieldItemViewModels(int numberToSkip, int numberToTake)
{
return filteredRealmCollection
.Skip(numberToSkip)
.Take(numberToTake)
.Select(field => new FieldItemViewModel(field)
}
The creation of the RealmCollection runs just fine, and if I wait for the whole list to instantiate, it has the correct values. Calling GetFieldItemViewModels also works just fine. But when I try to actually use the returned IEnumerable, the app freezes up until it instantiates the entirety of filteredRealmCollection. I've also attempted to use RealmCollection[index] to get a subset of items, with similar results.
Is there a way to only instantiate the items I've requested using RealmCollection, so that I'm only loading 10 items instead of ~1000? If not, is there a different way to accomplish what I'm trying to do here?

Selecting all objects from a list that return true on a member function, as a list?

I'm new to LINQ and trying to get a hold of it.
It's been useful so far for various things such as cutting down the code required, like when using .ForEach() to run a function on every object.
Now I'm trying to get a list of all objects from a master list, when their IsMouseOver() function returns true.
As a standard foreach it looks like this:
this.m_EntHovered.Clear();
foreach (EntEditor ent in this.m_EntAll)
{
if (ent.IsMouseOver(mousePos))
this.m_EntHovered.Add(ent);
}
But I wanted to shortern this using LINQ, however the shortest I could get it wasn't much shorter:
this.m_EntHovered = (from ent in this.m_EntAll
where ent.IsMouseOver(input)
select ent).ToList<EntEditor>();
Is there a better way to achieve what I'm after or is what I'm doing fine?

There isn't necessarily a better way to do it, but you can write it more succinctly via:
this.m_EntHovered = this.m_EntAll.Where(ent => ent.IsMouseOver(input)).ToList();
Note that this is not the same as your original, however, as you're assigning a new list, instead of adding items to the existing list. To get the same behavior (which may not be needed), you could do:
this.m_EntHovered.Clear();
this.m_EntHovered.AddRange(this.m_EntAll.Where(ent => ent.IsMouseOver(input)));

DbSortClause expressions must have a type that is order comparable parameter Name :Key

I am using Linq to entity and have the following query
IQueryable<DomainModel.User> userResult =
userResult.OrderBy(u => u.UserClientRoles.OrderBy(r => r.Role.RoleName));
But I am getting this error
DbSortClause expressions must have a type that is order comparable
parameter Name :Key
and it returns an empty collection.
Any idea what's going on?

.OrderBy(), when working with databases, is supposed to take in a delegate that returns only a single property that represents a column in your database. I'm not sure what you're trying to do, but it looks like
u.UserClientRoles.OrderBy(r => r.Role.RoleName)
Will return an enumeration of values, which can't be ordered.

I had the same problem, I solved it using this:
your code:
IQueryable<DomainModel.User> userResult = userResult.OrderBy(u => u.UserClientRoles.OrderBy(r => r.Role.RoleName));
my code:
List<Membership> results = new List<Membership>();
results.AddRange(memberships.OrderBy(m => m.Roles));
memberships = results.AsQueryable();
coincidences:
*.OrderBy(m => m.Roles)
solution:
*.OrderBy(m => m.Roles.Select(r => r.RoleId).FirstOrDefault())
possible problem's reason:
Maybe, you did what I did, and cause that 1 user/member could have more than 1 role in the same membership. That made a conflict with/to OrderBy() because the application can just "order" a single element at the time, when she call the Role (which is an ICollection of elements) the instead receive more than 1 element with no kind of priority's levels (even when we could assume that the application will take the role's index as priority's base level, actually its don't).
solution's explaination:
When you add the *.Select(r => r.RoleId), you are specifying to the application which element will be used to OrderBy(). But, as you shall see when you maybe reached at this point, just by using the *.Select(r => r.RoleId) could be not enough, because the application is still receiving multiple results with the same priority's level. Adding *.Select(r => r.RoleId).FirstOrDefault() you are basically saying: "...I don't care how many results you received from that element, just the focus on the first result, or order them by its default..." (default normally means EMPTY or NULL).
additional information:
I used non-official's simple concepts/meanings to explain a complex solution with simple words, which means that you could maybe have problems to find similar posts in the web by using the words/concepts used in this "answer". Otherwise, the code itself works and you shouldn't not have any problem by applying it and/or modifying it by yourself. GOOD LUCK!!! (^_^)

In my case, I was accidentally trying to order by an object instead of ordering by one of it's properties.
You should you use
var query = from Foo in Bar
orderby Foo.PropertyName
select Foo;
Instead of
var query = from Foo in Bar
orderby Foo
select Foo;
Note: you will get the same behaviour event if there is an override on Foo's ToString() method.

Error with SortedList: "Does not contain a definition for Select."

The following link suggests using this syntax:
var result = SortedList.Select(x => new {
x.Value.MyProperty,
x.Value.AnotherProperty
});
However, when I attempt to test it, I receive the error:
System.Collections.SortedList does not contain a definition for Select.
I have the following references:
using System;
using System.Collections;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Windows.Forms.DataVisualization.Charting;
using System.IO;
Despite extensive Internet searching, I cannot figure out how to fix this. Can anyone help?
EDIT:
perhaps I am going about this wrong. my goal is to return a subset of a SortedList as a SortedList.
var subset = data.Where(x => x.Key >= startDate && x.Key <= endDate);
where 'data' is a
SortedList<DateTime, BarData>
'barData' is a class

Note: You've recently edited your question, stating the real issue, which probably renders this answer obsolete. I've posted a new one to address the "real" issue.
Make sure your project has a reference to System.Core.dll. That's the .NET assembly which contains LINQ's extension methods for just about any collection type.
(This includes Select which you couldn't find with SortedList. Select is actually a static method found in the static class System.Linq.Enumerable.)
Make sure you have at least the following two using directives (you already do):
using System.Collections; // makes referring to SortedList easier
using System.Linq; // required for the Select method
Since SortedList is a non-generic and untyped collection, ie. it contains only objects, in order to use LINQ's Select method and/or access the elements' Value property, you first need to invoke LINQ's Cast<T> operator. Try this:
var result = sortedList.Cast<TItem>().Select(x => new {
x.Value.MyProperty,
x.Value.AnotherProperty
});
where TItem is the name of the type of the items in your SortedList.
P.S.: I'm assuming that SortedList in your example refers to a local variable, or field, or property, and is not used as the type name. That's why I changed the capitalization to sortedList.
P.P.S.: Unless you have a good reason not to, life might be a little easier if you used the System.Collections.Generic.SortedList<TKey, TValue> class, as some others here have already suggested before me.

Note: I'm posting this as a completely new answer because the recent edit to your question has made it clear that the real problem is quite different from what it first seemed.
Select the subset from data that you want in your resulting SortedList<…>:
var subset = data.Where(x => x.Key >= startDate && x.Key <= endDate);
Build a new SortedList from the filtered items:
var sortedListSubset = new SortedList<DateTime, BarData>();
foreach (var subsetItem in subset)
{
sortedListSubset.Add(subsetItem.Key, subsetItem.Value);
}
I don't see an easier solution. The problem is that Enumerable.Where will return an IEnumerable<KeyValuePair<DateTime, BarData>>, and there's no extension method to convert that back to a SortedList<…>. Thus the need to iterate through the IEnumerable<…> item by item and add them to a fresh SortedList<…>.
When to use Where and when to use Select:
Use Where whenever you want to find a subset, ie. when you want to "filter" a collection. (If you know SQL well, Where corresponds to WHERE, hence the name.)
Use Select when you want to change/transform, or "project", each element of a collection into something else. In functional programming, this operation is often called "map". (Again, if you know SQL well, then Select — unsurprisingly — corresponds to SELECT.)
Finally, remember that instead of data.Where(x => x.Key >= …), you could have written:
var subset = from x in data
where x.Key >= …
select x
This is some syntactic sugar that C# allows, but essentially means the same thing (however in this form, the select cannot be omitted, even if it doesn't really do any projection). Note again the similarity to SQL!

It looks like you're using SortedList as a static class, or your object is named the same as the class. Anyway, this should work:
var myList = new SortedList<int, int>();
var mySelect = myList.Select(x => x.Value);

The LINQ Select method is receives an IEnumerable. But SortedList doesn´t implement this interface.

C# How can I tell if an IEnumerable is Mutable?

I want a method to update certain entries of an IEnumerable. I found that doing a foreach over the entries and updating the values failed as in the background I was cloning the collection. This was because my IEnumerable was backed by some LINQ->SQL queries.
By changing the method to take a List I have changed this behavior, Lists are always mutable and hence the method changes the actual objects in the list. Rather than demand a List is passed is there a Mutable interface I can use?
// Will not behave as consistently for all IEnumerables
public void UpdateYesterday (IEnumerable<Job> jobs) {
foreach (var job in jobs.Where(x => x.date == Yesterday)) {
job.done = true;
}
}
...
public class Job {
...
public DateTime Date { get; set; }
}

You're not changing the list at all here - the collection itself could be immutable and this code would still "work". You're changing the data within the items in the collection.
Imagine a row of houses - that's pretty immutable (creating or destroying a house is tricky) but you can still change the contents of those houses.
So, what you really need to know is whether the elements of the collection are going to be cloned or not... whether you'll get a collection of "new" jobs each time you perform the query, or whether it'll use the existing objects.

Here's one hypothetical way this might happen. Say you have a method that takes an existing data source and creates Job objects from that source.
Something like this:
public IEnumerable<Job> GetJobs() {
var rows = GetRowsFromDatabaseTable();
foreach (var row in rows)
yield return new Job(row.Name, row.Date, row.Etc);
}
Then if you had code that did this:
var jobs = GetJobs();
UpdateYesterday(jobs);
foreach (var job in jobs)
Console.WriteLine(job);
You would find that the jobs you printed using Console.WriteLine did not reflect the updates you performed in UpdateYesterday. This is because UpdateYesterday would have enumerated over the new objects created by GetJobs, and then your foreach loop would have enumerated over a new set of new objects created (again) by GetJobs.
On the other hand, if you simply changed that first line to this:
var jobs = GetJobs().ToList();
Then you would have put all those new objects created by GetJobs into a list, where they would persist, and thus your UpdateYesterday and foreach loop would be referring to the same objects (the ones in the list you created).
Does that make sense? I think this could very well be the problem you're encountering (in which case, the answer is indeed to construct a collection such as a List<Job> in memory from which you can access members to manipulate).
In answer to your question about being able to tell if an IEnumerable is mutable or not, though... well, I really don't think that is possible.
I've removed the old answer, as the new one seems to be what the OP needed.