Develop Reference

Separate out two identical collections except for different ids with LINQ

I am trying to produce a method which feeds in two database (.db) files, loads them into an object which contains various string properties and a unique ID.
The db files were created from a LiteDB instance, with a unique ID assigned under the LiteDB library method, like so:
Id = ObjectID.NewObjectId();
I'm trying to obtain all items that are different between the two db collections, but unfortunately any LINQ query I try falls down because the IDs are always unique, despite all other values contained within the object being identical.
I would like to perform a LINQ query that will ignore the ID field in the object but do a comparison on all other properties within that object but am struggling to do so
My initial idea was to use
var differences = items1.Except(items2)
But this populates all items into the differences collection, due to the differing IDs.
Any advice on how to exclude the comparison of the ids would be appreciated, the tool may be used on millions of records so needs to be efficient.

Except has an overload that accepts an IEqualityComparer<T>. You can implement this interface in a class in a way that ignores the ID and then pass an instance of that class to the method. See this example if you need a reference on how to implement IEqualityComparer<T>.

How do you build extensible data model

I'm thinking of building a ecommerce application with an extensible data model using NHibernate and Fluent NHibernate. By having an extensible data model, I have the ability to define a Product entity, and allow a user in the application to extend it with new fields/properties with different data types including custom data types.
Example:
Product can have an addition fields like:
Size - int
Color - string
Price - decimal
Collection of ColoredImage - name, image (e.g. "Red", red.jpg (binary file))
An additional requirement is to be able to filter the products by these additional/extended fields. How should I implement this?
Thanks in advance.

I think this link describes kind of what you want...
http://ayende.com/Blog/archive/2009/04/11/nhibernate-mapping-ltdynamic-componentgt.aspx
More info on dynamic-component:
http://www.mattfreeman.co.uk/2009/01/nhibernate-mapping-with-dynamic-component/
http://bartreyserhove.blogspot.com/2008/02/dynamic-domain-mode-using-nhibernate.html
The idea behind dynamic-component is that you can build your data model by not having a one to one mapping of databse columns with properties. Instead you have only a dictionary property that can contain data from as many properties as you like. This way when you fetch the entity, the dictionary gets the data of all columns configured to belong in there. You can extend the database table's schema to include more columns and that will be reflected to the databse model if you update the mapping file accordingly (manually or though code at application start).
To be honest I do not know you can query such entity using the "attributes" property but if I had to guess I would do an IN statement to it.

One of the options is EAV model (Entity-Attribute-Value).
This model is good to apply if you have a single class in your domain, which table representation would result in a wide table (large number of columns, many null values)
It's originally designed for medical domain, where objects may have thousands of columns (sympthoms).
Basically you have
Entity (Id) (for example your Product table)
Attribute(Id, ColumnName)
Value(EntityId, AttributeId, value)
You can have some additional metadata tables.
Value should better be multiple tables, one for a type.
For example:
ShortStringValue(EntityId, AttributeId, Value nvarchar(50));
LongStringValue(EntityId, AttributeId, Value nvarchar(2048));
MemoValue(EntityId, AttributeId, Value nvarchar(max));
IntValue(EntityId, AttributeId, Value int);
or even a comple type:
ColorComponentsValue(EntityId, AttributeId, R int, G int, B int );
One of the things from my experience is that you should not have EAV for everything. Just have EAV for a single class, Product for example.
If you have to use extensibility for different base classes, let it be a separate set of EAV tables.
Onother thing is that you have to invent a smart materialization strategy for your objects.
Do not pivot these values to a wide row set, pivot just a small number of collumns for your query criteria needs, then return a narrow collection of Value rows for each of the selected objects. Otherwise pivoting would involve massive join.
There are some points to consider:
. Each value takes storage space for foreign keys
. For example row-level locking will behave different for such queries, which may result in performance degradation.
. May result in larger index sizes.
Actually in a shallow hellow world test my EAV solution outperformed it's static counterpart on a 20 column table in a query with 4 columns involved in criteria.

Possible option would be to store all extra fields in an XML structure and use XPath/XQuery to retrieve them from the database.
Each extensible entity in your application will have an XML field, like ExtendedData, which will contain all extra properties.

Another option is to use Non-relationnal Databases which are typically suited for this kind of things.
NOSQL databases(couchDB, mongoDB, cassandre...) let you define dynamically your propretyfields, you could add fields to your product class whenever you want.

I'm searching for similar thing and just found N2 CMS (http://n2cms.com) which implements domain extensibility in quite usable way. It also supports querying over extension fields which is important. The only downside I find out is that it's implemented using HQL so it would take some time to reimplement it to be able to query using QueryOver/Linq, but the main idea and mappings are there. Take a look on ContentItem, DetailCollection, ContentDetail classes, their mappings and QueryBuilder/DetailCriteria.

How do I sort a gridview of Linq objects based on a derived field?

I have written a page which uses Linq to query the database and bind the resulting IQueryable to a datagrid. I have a partial class which contains extra properties which derive their values based on other values brought in from the database.
Sorting works fine on fields that are actually in the database but not for the derived fields. When I attempt to sort on such a field I get an error saying "The member 'Trip.Difference' has no supported translation to SQL.
Any idea how to allow sorting on these derived fields?

The problem is that you are binding to an IQueryable, so every time you enumerate it, you are translating the LINQ expression on the IQueryable to a SQL statement and going back to the database to execute it.
If you are trying to sort on properties that are not bound to the database model then you will get the error mentioned, as those properties only exist once an object has been created from a data row.
The simplest solution is to call ToList() on the IQueryable before using it for sorting and data-binding, so that you sort on the in-memory objects where the properties are actually available. ToList() converts your IQueryable into an IEnumerable (via List<T>), and stops it from going to the database again via LINQ to SQL.
This is generally a good design pattern to follow - the last thing you want is consumers of your business layer being able to unwittingly execute arbitrary queries against your database, simply because you returned IQueryable where you should have returned IEnumerable.

Call ToEnumerable() first, and then add the OrderBy:
var q = (from a in mDataContext.Somethings()
select a).ToEnumerable().OrderBy...

Using Unmapped Class with NHibernate Named Query

I'm using a custom named query with NHibernate which I want to return a collection of Person objects. The Person object is not mapped with an NHibernate mapping which means I'm getting the following exception:
System.Collections.Generic.KeyNotFoundException:
The given key was not present in the
dictionary.
It's getting thrown when the Session gets created because it can't find the class name when it calls NHibernate.Cfg.Mappings.GetClass(String className). This is all fairly understandable but I was wondering if there was any way to tell NHibernate to use the class even though I haven't got a mapping for it?

Why don't you use:
query.SetResultTransformer(Transformers.AliasToBean(typeof(Person)));
It will insert data from each column in your query into Person object properties using column alias as a property name.

How can you create a query which would return instances of a type that is not mapped ?
I think Michal has a point here, and maybe you should have a look at projections. (At least, this is what I think you're looking for).
You create a query on some mapped type, and then, you can 'project' that query to a 'DTO'.
In order to do this, you'll have to 'import' your Person class, so that it is known to NHibernate, and you'll have to use a ResultTransformer.
Something like this:
ICriteria crit = session.CreateCriteria (typeof(Person));
// set some filter criteria
crit.SetProjection (Projections.ProjectionList()
.Add (Property("Name"), "Name")
.Add (Property( ... )
);
crit.SetResultTransformer(Transformers.AliasToBean(typeof(PersonView));
return crit.List<PersonView>();
But, this still means you'll have to import the class, so that NHibernate knows about it.

By using the class, NHibernate would basically be guessing about everything involved including which table you meant to use for Person, and the field mappings. NHibernate could probably be hacked to do dynamic binding based on matching the names or something, but the whole idea is to create the mappings from plain old data object to the database fields using the xml files.

If there's not a really good reason not to map the class, simply adding the mapping will give you the best results...
That said, you can't use a named query to directly inject results into an unmapped class. You would need to tell it which columns to put into which fields or in other words, a mapping. ;) However, you can return scalar values from a named query and you could take those object arrays and build your collection manually.

To solve this, I ended up using the TupleToPropertyResultTransformer and providing the list of property values. There are a few limitations to this, the main one being that the SQL query must return the results in the same order as you provide your properties to the TupleToPropertyResultTransformer constructor.
Also the property types are inferred so you need to be careful with decimal columns returning only integer values etc. Apart from that using the TupleToPropertyResultTransformer provided a reasonably easy way to use an SQL query to return a collection of objects without explicitly mapping the objects within NHibernate.

NHibernate - Getting the results as an IDictionary instead of IList

I'm using NHibernate as a persistency layer and I have many places in my code where I need to retrieve all the columns of a specific table (to show in a grid for example) but i also need a fast way to get specific item from this collection.
The ICriteria API let me get the query result either as a unique value of T or a IList of T.
I wonder if there is a way to make NHibernate give me those objects as an IDictionary where the key in the object's Id and the value is the object itself. doing it myself will make me iterate all over the original list which is not very scalable.
Thank you.

If you are working with .NET 3.5, You could use the Enumerable() method from IQuery, then use the IEnumerable<T>.ToDictionary() extension method :
var dictionary = query.Enumerable().ToDictionary(r => r.Id);
This way, the list would not be iterated twice over.
You mention using ICriteria, but it does not provide a way to lazily enumerate over items, whereas IQuery does.
However, if the number of items return by your query is too big, you might want to consider querying the database with the key you'd have used against the IDictionary instance.