there has been quite bit of discussion on this topic ( Modifying an Entity Framework Model at Run-Time), but so far I haven't been able to find a nice solution to the problem. We currently have a model built with EF 4.0 and need to allow customers to add custom fields to various tables. We have currently solved the problem with a few generic tables that describe the new properties, but we're finding that the solution is not performing well.
Does anyone know of a good way to dynamically add columns to the database and also update the ORM to reflect that at run-time?
There is no good, nice or manageable way to update EF at runtime to reflect changes in the database. If you really have a database which must change at runtime EF is not good tool for you. EF is strongly typed - every change to database must be reflected not only in mapping but also in entity classes used for loading and persisting data.
Changing entity classes at runtime always goes to area of emitting IL code at runtime. If you pass the process of creating dynamic assembly with dynamic module and dynamic entity types you will face a lot of new challenges:
You will not change your existing types - you will generate a new type every time the user adds or removes some property.
A new type generation brings a new performance cost for rebuilding context's metadata workspace. You must also make sure that this is correctly synchronized if you use it on a server. Another problem is that all existing instances of the old type (before adding or removing property) are now unknown to new instances of EF context and cannot be persisted. If you want to persist them as well you need EF context instances with old metadata workspace as well.
A lot of your code will probably use dynamic instead of real type = no compile time checking. Inheritance and interfaces will not be useful when interacting directly with EF because inheritance must be mapped (you don't want it) and interfaces are not accepted by EF.
The new type is unknown at design time - you cannot use it for design time code and compilation.
EF doesn't like dynamic or ExpandoObject because it uses reflection for mapping -> at runtime your dynamic instance must be correct type otherwise reflection will not work.
How to write queries for dynamic type? The query always starts at generic instance of DbSet or ObjectSet created for concrete type - you must be able to make those instances dynamically as well. Generic argument must be a type mapped to current context - dynamic will not help in this case because it is not a mapped type.
.NET has a nice behavior for this case. It cannot unload assembly. So every time you generate a new type you will have an old time loaded as well.
Do you still want to change EF at runtime? Your current approach is correct one. Simply tune it for better performance but beware that these requirements always come with performance costs - especially with EF.
Alternatively use the last approach mentioned with linked table - fixed number of predefined custom fields directly in the main entity.
Related
from what i've seen so far WCF Data Services are pretty easy to setup when using then in combination with EF.
That's kinda what i'm after out of the box but I also need the ability for the EF model to change at runtime.
I'm building an app where the app users will be able to specify the database structure and then begin populating it ... the relevant UI components needed are then generated with MVC using some clever rule based trickery.
So for example the user will be given a "Create new Object" button, which will let them specify field names.
Once that part is complete the user submits that and it generates a new table in the db.
From there the UI components are generated that allow that table to be managed within the app.
The problem of course is getting that new table in to the EF model without a recompile of the back end data service.
The concept being that this builds the database and the pages required to manage the various parts of it (there's a bigger picture in mind here but i don't want to confuse matters by trying to explain it all).
I'm thinking that maybe EF is not the right tool to use at the moment .. because it needs a strongly typed set of entities in order to work ... that may not be possible in this case.
I'm toying with the idea of passing this service Dynamic objects ... (e.g. objects of type Something : dynamic )
i'd suggest not only that entity framework is not right for this, but neither is a relational database. document database or key-value store would probably be a better fit than trying to create tables on demand to shove this into a relational structure.
WCF Data Services can be used without Entity Framework. Using either the "Reflection Provider" or a custom provider, which you will have to implement (the Reflection provider requires you to have actual .NET classes, which you don't).
Basically, you implement the DataService class and the IServiceProvider interface, which will provide instances of the IDataServiceQueryProvider, IDataServiceMetadataProvider and IDataServiceUpdateProvider. This might involve a lot of work, so be sure that you actually do want to do this.
See http://msdn.microsoft.com/en-us/library/ee960143.aspx for more information.
OMG ...
Apparently this is supported (mostly) out the box with EF 4.2
http://blogs.msdn.com/b/adonet/archive/2011/09/28/ef-4-2-release-candidate-available.aspx
WOW !!!
When using EF 4.1 Code First, is it possible to create User-Defined Data Types for your schema?
Simple answer is no.
Longer answer:
Current EF implementation leads to multiple issues when trying to use user defined types:
The type must be defined prior to its usage in table's DDL definition. Because of that the type cannot be defined in Seed method of database initializer (as often used for other database constructs like triggers or indexes). To make this work you must create whole new initializer by implementing IDatabaseInitializer and separate database creation and table creation because custom type definitions must be between them. Here is some example how to create database initializer (this one will recreate tables every time you run the application).
Creating database initializer which reflects model changes from sketch is more complicated because this logic is probably internal to EF
You must also add check that type is not already created
Even if you have initializer you still face the biggest blocker. EF is not ready to allow you defining user defined types for columns - neither ColumnAttribute and HasColumnType in fluent API will accept custom type. So your mapping must specify basic primitive SQL types and these custom types will be used in tables DDL produced by EF. So unless custom database initializer post-process those generated SQL scripts and replaces basic types with custom types (really ugly solution) tables will not use them.
I'm exploring Mongo as an alternative to relational databases but I'm running into a problem with the concept of schemaless collections.
In theory it sounds great, but as soon as you tie a model to a collection, the model becomes your defacto schema. You can no longer just add or remove fields from your model and expect it to continue to work. I see the same problems here managing changes as you have with a relational database in that you need some sort of script to migrate from one version of the database schema to the other.
Am I approaching this from the wrong angle? What approaches do members here take to ensure that their collection items stay in sync with their domain model when making updates to their domain model?
Edit: It's worth noting that these problems obviously exist in relational databases as well, but I'm asking specifically for strategies in mitigating the problem using schemaless databases and more specifically Mongo. Thanks!
Schema migration with MongoDB is actually a lot less painful than with, say, SQL server.
Adding a new field is easy, old records will come in with it set to null or you can use attributes to control the default value [BsonDefaultValue("abc", SerializeDefaultValue = false)]
The [BsonIgnoreIfNull] attribute is also handy for omitting objects that are null from the document when it is serialized.
Removing a field is fairly easy too, you can use [BSonExtraElements] (see docs) to collect them up and preserve them or you can use [BsonIgnoreExtraElements] to simply throw them away.
With these in place there really is no need to go convert every record to the new schema, you can do it lazily as needed when records are updated, or slowly in the background.
PS, since you are also interested in using dynamic with Mongo, here's an experiment I tried along those lines. And here's an updated post with a complete serializer and deserializer for dynamic objects.
My current thought on this is to use the same sort of implementation I would using a relational database. Have a database version collection which stores the current version of the database.
My repositories would have a minimum required version which they require to accurately serialize and deserialize the collections items. If the current db version is lower than the required version, I just throw an exception. Then use migrations which would do all the conversion necessary to update the collections to the required state to be deserialized and update the database version number.
With statically-typed languages like C#, whenever an object gets serialised somewhere, then its original class changes, then it's deserialised back into the new class, you're probably going to run in problems somewhere along the line. It's fairly unavoidable whether it's MongoDB, WCF, XmlSerializer or whatever.
You've usually got some flexibility with serialization options, for example with Mongo you can change a class property name but still have its value map to the same field name (e.g. using the BsonElement attribute). Or you can tell the deserializer to ignore Mongo fields that don't have a corresponding class property, using the BsonIgnoreExtraElements attribute, so deleting a property won't cause an exception when the old field is loaded from Mongo.
Overall though, for any structural schema changes you'll probably need to reload the data or run a migration script. The other alternative is to use C# dynamic variables, although that doesn't really solve the underlying problem, you'll just get fewer serialization errors.
I've been using mongodb for a little over a year now though not for very large projects. I use hugo's csmongo or the fork here. I like the dynamic approach it introduces. This is especially useful for projects where the database structure is volatile.
I am new to Entity Framework 4.0, using it with C#, and currently experimenting with its features.
What I noticed is that, like with most similar ORMs, it relies on an Context object to deal with the data-manipulation and CRUD statements generation done on the generated entities.
This means that if I want to save the changes back to the database, I always need to be able to have access to a reference to the ObjectContext that has instanciated the entities.
It is fine and all if the context has been created in an accessable scope (same method, for example), but what if I pass an entity or and entity set to a method and want this method to save the changes? It looks like the only easy way is to pass the ObjectContext along with the parameters.
Another solution would be placing the ObjectContext in some sort of global variable.
Needless to say, I find styling and maintainability issues to both of these approaches.
In short, the best way I can imagine is getting a reference to the ObjectContext from the entity or entity set.
I know that it isn't possible by default.
I have found a method showing adding an extension method to get the ObjectContext from the entity. However, it only works for entities with relationships and calling this method is expensive according to the author.
I was thinking about maybe modifying the T4 template to add a Context property to all my entities and fill it automatically on entities' instanciation.
I have already modified T4 template once to have Entity Framework enforce Max Length on my generated classes (by following Julie Lerman's Programming Entity Framework 4 book).
I can't say I really enjoy the T4 syntax so far, but if that's the best/only way, so be it...
Has anyone already done so and what would be the best way to handle this and willing to share his T4 template or explain what are the best partial methods or events to hook into to get this done?
Is there any major downside in using such an approach?
I am thinking that having so many references to the ObjectContext may hinder/delay its ability to be recollected by the GC if some of my entities remain in scope but I actually have no use anymore for the ObjectContext.
Many thanks.
If you need to pass object context as parameter together with your entities you are doing something wrong.
Usually context is needed only in well defined layer. All classes from this layer which requires context to their logic can receive the context through some specialized class - context provider (it can also be called service locator). Context provider will hold current context instance in some storage - you can create your own or you can store it per thread, per http request, etc.
If they need more then one context instance in your classes you can modify your provider to work also as factory.
Another common approach is combined with dependency injection. You will pass the context to your classes through the constructor (or property) and you will have some bootstraper code which will do all necessary initialization for you (create required instances and pass all dependencies into them). Again you can pass a context or a factory. This is usually used together with IoC containers which will do the plumbing for you.
Once you have this infrastructure prepared you can pass your entity to the any initialized class from that layer and it will have the context available.
I know we can add an external XML map file to dynamically alter the table names in a data context. But according to what i have seen on most of the posts, that can be done only if the table structures are same (same columns and primary key).
What i want to know is that, is there a way to dynamically add a new table to the data context when its different from the tables currently in the context.
Or is it possible to achieve this from entity framework?
I'm gonna go a head and say no because the ORM doesn't provide a dynamic run-time model it's a prebuilt static model of the data set that you query against.
Then, all you really need to do to make it dynamic is to rebuilt it as you need it. That will require you to invoke a build when you add new tables or columns but that's not out of the question. And this can be done in a verity of ways.
However, if this is your problem you shouldn't be trying to solve this with an ORM like linq-to-sql. It's was never built for that. I would also recommend entity framework as it has superceeded linq-to-sql in every way (and linq-to-sql is no longer being developed).
Another way would be to use EF 4.1 code first where you can generate and compile code dynamically and load them in your app domain. Well I think you can use dynamic compilation for EF 4.0 as well but Edmx and mappings are complicated.
And it will be new context, you won't be able to change existing context but you may merge in new class library. But this all will have to happen on reflection because you will not have generic types available at compile time for linq.