I have an entity set that is a Session for a survey. There is a navigation property to a collection of Responses that are also keyed to an Question entity (using a foreign key relationship in the DB).
It is easy for me to call up the collection of responses by simply doing
session.Responses
That returns an enumerated list, which for most cases is fine.
However, for large datasets I'm running into a conceptual problem.
If I want to select a particular response from a Session's Response collection based on an Item, given that it is a collection would it be a seek or a scan operation? Does the FK relationship between Response and Question get utilized at all?
If not, would it be wise to create a Keyed Dictionary in a Session Partial class that takes it's Response collection and pairs it with Question Keys? That way, it would be a direct seek to yield the exact Response per requested Question.
LINQ over objects uses Enumerators, which "scan" over the collection.
EDIT: Your best bet is to do as much logical joining and filtering in the database as possible. When you load a Dictionary with the results of an EF query expression, the query will be executed on the database and the results placed into memory. At this moment you are no longer dealing with an IQueryable expression, but an IEnumerable set of objects. Further LINQ expressions on the Dictionary are LINQ over objects.
Related
I'm looking to get a better understanding on when we should look to use IEnumerable over IQueryablewith LINQ to Entities.
With really basic calls to the database, IQueryable is way quicker, but when do i need to think about using an IEnumerable in its place?
Where is an IEnumerable optimal over an IQueryable??
Basically, IQueryables are executed by a query provider (for example a database) and some operations cannot be or should not be done by the database. For example, if you want to call a C# function (here as an example, capitalize a name correctly) using a value you got from the database you may try something like;
db.Users.Select(x => Capitalize(x.Name)) // Tries to make the db call Capitalize.
.ToList();
Since the Select is executed on an IQueryable, and the underlying database has no idea about your Capitalize function, the query will fail. What you can do instead is to get the correct data from the database and convert the IQueryable to an IEnumerable (which is basically just a way to iterate through collections in-memory) to do the rest of the operation in local memory, as in;
db.Users.Select(x => x.Name) // Gets only the name from the database
.AsEnumerable() // Do the rest of the operations in memory
.Select(x => Capitalize(x)) // Capitalize in memory
.ToList();
The most important thing when it comes to performance of IQueryable vs. IEnumerable from the side of EF, is that you should always try to filter the data using an IQueryable to get as little data as possible to convert to an IEnumerable. What the AsEnumerable call basically does is to tell the database "give me the data as it is filtered now", and if you didn't filter it, you'll get everything fetched to memory, even data you may not need.
IEnumerable represents a sequence of elements which you enumerate one by one until you find the answer you need, so for example if I wanted all entities that had some property greater than 10, I'd need to go through each one in turn and return only those that matched. Pulling every row of a database table into memory in order to do this would not maybe be a great idea.
IQueryable on the other hand represents a set of elements on which operations like filtering can be deferred to the underlying data source, so in the filtering case, if I were to implement IQueryable on top of a custom data source (or use LINQ to Entities!) then I could give the hard work of filtering / grouping etc to the data source (e.g. a database).
The major downside of IQueryable is that implementing it is pretty hard - queries are constructed as Expression trees which as the implementer you then have to parse in order to resolve the query. If you're not planning to write a provider though then this isn't going to hurt you.
Another aspect of IQueryable that it's worth being aware of (although this is really just a generic caveat about passing processing off to another system that may make different assumptions about the world) is that you may find things like string comparison work in the manner they are supported in the source system, not in the manner they are implemented by the consumer, e.g. if your source database is case-insensitive but your default comparison in .NET is case-sensitive.
I'm using Entity Framework but I've cached some application wide entities, so now they're List rather than entities.
Two of the objects I've cached are Products and Stores - a Product has a store. In a table they are linked by StoreId, but since I'm using EF the Product can also include the Store entity.
I don't cache them together simply because they are also used independently. What I would like to do is join them at some point, so I might subselect the Product object then want to populate Stores by joining the Stores object.
Is the most efficient way of joining 2 lists using linq - if so whats the most efficient syntax, or is there's a better way converting lists back to entities.
is there a better way of converting lists back to entities
This is a bit misleading as this isn't what your trying to do. You pull down a list of Products/Stores which are entities - no need to convert them (even if your storing them in a list).
If I understand you correctly what you want to do is basically map each Product to it's appropriate Store from the cached entities (to save going back to the DB by accessing the Store property on the Product entity). I am pretty sure all you need to do is map this across, so something like:
foreach (var product in cachedProducts)
{
product.Store = cachedStores.SingleOrDefault(s => s.Id == product.StoreId);
}
Your entities are connected via the underlying database context, so removing them from this context prevents you from joining them back later or even saving them without attaching context.
the most efficient design is to use EF's IQueryable syntax to retrieve them whenever you need them and let the database context do any caching required / possible.
I have written a page which uses Linq to query the database and bind the resulting IQueryable to a datagrid. I have a partial class which contains extra properties which derive their values based on other values brought in from the database.
Sorting works fine on fields that are actually in the database but not for the derived fields. When I attempt to sort on such a field I get an error saying "The member 'Trip.Difference' has no supported translation to SQL.
Any idea how to allow sorting on these derived fields?
The problem is that you are binding to an IQueryable, so every time you enumerate it, you are translating the LINQ expression on the IQueryable to a SQL statement and going back to the database to execute it.
If you are trying to sort on properties that are not bound to the database model then you will get the error mentioned, as those properties only exist once an object has been created from a data row.
The simplest solution is to call ToList() on the IQueryable before using it for sorting and data-binding, so that you sort on the in-memory objects where the properties are actually available. ToList() converts your IQueryable into an IEnumerable (via List<T>), and stops it from going to the database again via LINQ to SQL.
This is generally a good design pattern to follow - the last thing you want is consumers of your business layer being able to unwittingly execute arbitrary queries against your database, simply because you returned IQueryable where you should have returned IEnumerable.
Call ToEnumerable() first, and then add the OrderBy:
var q = (from a in mDataContext.Somethings()
select a).ToEnumerable().OrderBy...
I'm using NHibernate as a persistency layer and I have many places in my code where I need to retrieve all the columns of a specific table (to show in a grid for example) but i also need a fast way to get specific item from this collection.
The ICriteria API let me get the query result either as a unique value of T or a IList of T.
I wonder if there is a way to make NHibernate give me those objects as an IDictionary where the key in the object's Id and the value is the object itself. doing it myself will make me iterate all over the original list which is not very scalable.
Thank you.
If you are working with .NET 3.5, You could use the Enumerable() method from IQuery, then use the IEnumerable<T>.ToDictionary() extension method :
var dictionary = query.Enumerable().ToDictionary(r => r.Id);
This way, the list would not be iterated twice over.
You mention using ICriteria, but it does not provide a way to lazily enumerate over items, whereas IQuery does.
However, if the number of items return by your query is too big, you might want to consider querying the database with the key you'd have used against the IDictionary instance.
Using LINQ to Entities sounds like a great way to query against a database and get actual CLR objects that I can modify, data bind against and so forth. But if I perform the same query a second time do I get back references to the same CLR objects or an entirely new set?
I do not want multiple queries to generate an ever growing number of copies of the same actual data. The problem here is that I could alter the contents of one entity and save it back to the database but another instance of the entity is still in existence elsewhere and holding the old data.
Within the same DataContext, my understanding is that you'll always get the same objects - for queries which return full objects instead of projections.
Different DataContexts will fetch different objects, however - so there's a risk of seeing stale data there, yes.
In the same DataContext you would get the same object if it's queried (DataContext maintains internal cache for this).
Be aware that that the objects you deal are most likely mutable, so instead of one problem (data duplication) you can get another (concurrent access).
Depending on business case it may be ok to let the second transaction with stale data to fail on commit.
Also, imagine a good old IDataReader/DataSet scenario. Two queries would return two different readers that would fill different datasets. So the data duplication problem isn't ORM specific.
[oops; note that this reply applies to Linq-to-SQL, not Entity Framework.]
I've left it here (rather than delete) because it is partly on-topic, and might be useful.
Further to the other replies, note that the data-context also has the ability to avoid doing a round-trip for simply "by primary key" queries - it will check the cache first.
Unfortunately, it was completely broken in 3.5, and is still half-broken in 3.5SP1, but it works for some queries. This can save a lot of time if you are getting individual objects.
So basically, IIRC you need to use:
// uses object identity cache (IIRC)
var obj = ctx.Single(x=>x.Id == id);
But not:
// causes round-trip (IIRC)
var obj = ctx.Where(x=>x.Id == id).Single();