I have read several articles on applying migration in production. I Understand it quite well but i think i want to know detailed about it. I have also read microsoft's documentations but it does not give me what i am looking for.
The Database.Migrate() method is being used in almost everything I read online about it. How does it work? no one explains it.
Assuming in a CMS platform, i want to apply new migrations, a new module is added on runtime and I have to run the migrations automatically because you won't want your users to be involved in the complexity of database operations just as wordpress does.
New modules may contain migration scripts if needed, they will contain a class which inherits from my base DbContext which they can add their module models and everything related to database.
I will then have to merge or load that class or assembly with the core.
How does Database.Migrate work?
Does it produce a migration script?
Does it add a migrations to the migration table as when performed online?
Do I need to pass a migration script to it or can I?
Database.Migrate only applies migrations that have been generated. This is the same as command dotnet ef database update from CLI.
No, it does not. This is the thing that you need to do on your own or automate it with tool / script - documentation
As mentioned before it does not add any NEW migrations, only applies them.
I don' think you can do it. An easier way would be to create some script on deploy that will do the thing you want and revert the database in case of failure. I also advise you to think about migration that will affect data. What if someone will generate migration that will cause the lost of data?
Related
We are building an Asp.Net core web API that must use MongoDb on the backend. According to what I found, it is recommended to use directly MongoDb.Driver because it does already most of the job and it makes less sense to use an ORM(EF Core) with a NoSql DB.
One thing that I'm not sure:
Is there a way to make "migration" as we would with Entity Framework? Same thing for the data seeding? I could imagine some way to doing it myself, but it feels like re-inventing the wheel.
So, how should we handle the potentials data migrations?
Ps, I understand that if we just add a property, we might not do an update, but there might be some occasion where you have a real structure changes
Some of the changes can be handled without explicit migrations; for some examples, see this link.
For others, we executed some code at startup that created indexes, set up collections or performed migrations. In the best case, this code is idempotent, so that it can be run multiple times. Otherwise we stored a migration marker in the database so that only the necessary migrations are run and parallel executions of the code are avoided.
If you need a more sophisticated approach with documents storing their version and the option to perform an on-the-fly-migration on a per-document-basis, you could check out this package.
Up to now, it was sufficient for us to perform the migrations at startup so that we haven't used this package yet.
I have a service which uses EF Core runtime migrations on startup:
var migrator = dbContext.Database.GetService<IMigrator>();
await migrator.MigrateAsync("targetMigration", cancellationToken);
To generate the migrations I first update the DbContext class, then perform "dotnet ef migrations add" to generate the migration code.
It may happen that a deployed upgrade will be automatically rolled back to the previous version after a migration has occurred. For example if health checks or tests fail. In this case I want the previous version of the application to be able to automatically roll back the migration. I know MigrateAsync can revert migrations, but in my current workflow the migration code will not be in the previous version of the code, so I am not sure whether it will be able to revert the migration.
I can think of a workflow like:
Change DbContext and run "dotnet ef add migration" to generate the migration code
Revert the DbContext change and deploy the application so that the code for migration 'n' exists, but the target migration in MigrateAsync and the version of the DbContext is 'n-1'
Re-apply the DbContext change, change MigrateAsync to target migration 'n', and deploy the application
But this seems awkward and I am not sure whether it is necessary, and whether it would definitely work.
What is a good strategy for deploying code first runtime migrations using EF Core such that if the previous version is deployed, the migration can be automatically rolled back?
From what I know, EF does not have anything prepared for you case, it is really hard to make migrations back and forth in prod, also, you must consider that many migrations can create data loss.
First you need to create a process when a change must be tested very well, when the change gets to production you should be 99% sure that there wont be rollbacks in prod.
As you say, you would need the last version of the code, if not, EF wont know what "down" should do.
In our current system, we analyse each migration, if is a new table or something simple, we just run the migration from the CI. If it is something more complex, or we need more complex movements (table modifications with millions of rows), we just do it by hand, so we can send data to temp tables, fill empty data or work with special functions we just generate the script and work with it.
dotnet ef migrations script 20190725054716_Add_new_tables
It is a really hard problem, Java and JPA shares the same problem to generate a history.
These migrations generators are great for development, but hard for production, changing environments, specially when you need to go back and forth like you, another option is to use other tools to handle the migrations that are better prepared for this scenario, like liquibase
Another idea can be found here:
I ended up creating a custom tool that executed migrations
intelligently and automatically determined which model (context)
assembly to use for the migration.
I am new to Entity Framework 6 Code First and am trying to perform what I thought would be a simple task. I want to create a SQL View and then have an Entity in my database context that I can use to query the view.
I have tried articles such as this but the key difference in my case is that the SQL View is not an existing view coming from another existing database.
I examined the proposition made in this article but it seems like overkill to me that I would need to create some extension methods to do something as simple as create a view/entity combo and use it in my database context.
Am I missing something? I know it would be much easier if I weren't using Code First but please keep in mind it's Code First and I am trying to create a view, not reuse one from an existing database.
Colin and Kevin, Thank you for the link to your answer on the other post and your concise answer. I have used several resources to finally create a queryable entity based on a new SQL view. Just in case anyone else is new to EF 6.0 Code First and is just getting their feet wet, I do have a few steps that will hopefully benefit others in the future.
It may seem obvious to more seasoned Entity Framework developers, but in order to execute the 'Migration' approach you need to disable automatic migrations and actually dive into the guts of the Code First Migrations inner workings. Since automatic migrations is turned on out of the box, I had already created a fairly complex database with seed scripts all relying on automatic migrations and rebuilding the database on every run of my application. This post helped me wipe my migrations history and get to square 1 with automatic migrations turned off (I went with the web.config approach in case you were wondering)
After I had cleared my migrations information, I deleted the mdf from within solution explorer. That guaranteed that I wouldn't run into any problems when running Update-Database (further down the list of steps).
In the Package Manger console, I then executed Add-Migration Initial to generate an "Initial" migration. The result of this was the editable Up and Down methods as described in Colin's answer. I then followed the steps in Colin's answer by commenting out the table create statement (Entity Framework tries to create a table but we really want to create a view and map it to the Entity) and inserting my own view create sql statement at the end of the Up method. It's important to put the create statement after the creation of any tables that it may depend on. I also performed my Seed activities in the Configuration.Seed method instead of in my Context's Seed method. I see how this would be important if you were dealing with multiple migrations. Finally, as Colin suggested I added the table mapping to my context's OnModelCreating event.
The final step in this was to actually apply the migration to the database. In order to do that, in the Package Manager console you execute the Update-Database command. That statement will rebuild the database with the "Initial" migration you created and edited in earlier steps.
It still surprises me that I need to do all of this custom work to create a view and map it to an entity with Code First, but at the end of the day it was helpful in getting me started on migrations as you can only rely on the "automatic migrations" for so long anyways.
You can manually add the sql to create the view to a migration then consume it as per your first link.
The answer in the link provided by Colin does the job.
In case there are lots of views to be created, it can be a good idea to save the view queries in separate files and add them in a resource (.resx) file instead of hard-coding the sql queries in the Migration Up() method.
For e.g.
public override void Up()
{
Sql("ResourceFileName.ResourceName");
}
instead of hard coding like
{
Sql("EXEC ('CREATE View [dbo].[ClientStatistics] AS --etc");
}
I started a new project C#, and I used the "enable-migrations" command in the package console window. This naturally added migrations to my project. I then set automatic migrations to true, so that as I call "update-database" it will create my tables for me with all keys and that.
The only problem is that I have multiple websites where want to do this, which all use the ASP.NET membership provider to login. Which through automatic code migrations create a bunch of account tables for me to use. But the tables are all called the same, so if I do this targeting the same database for different sites they will overwrite eachother. So the question I got is this: How can I specify a prefix for my tables created by the entity framework?
I've seen several ideas on how to do this while searching, but they didn't work for me (the necessary properties wasn't there for some reason and so on.)
Thank you
Xenoxsis
I'm not sure how do you plan to do just that - if I'm getting it right you'd want to keep one database (shared) in between number of web sites - yet, have each site has its own membership tables, named differently, with different prefixes, right?
First problem is that for each Db/table name change - you need a 'code to match' - i.e. code first entities and code, the 'migration table' in the Db - and tables are all in sync - so it could all work together as it should. In that sense, just changing script or table names in Db won't work. It has to be done at the level of attributes (as #Steven suggested) or fluent configuration.
Which in your case, it means that somehow you'd need to 'build' separate configurations for each site, deploy them separately (code) to each site - and build one mega Db that contains all the small variants of each merged together.
That's going to be tough to manage - but you could try (what I described above) - I have no idea if it'd work (as this requires lot of 'infrastructure' to try this one) - but maybe along these lines...
put Table attributes (or via fluent config)
Build code - 'vary' the Table names for each - and rebuild (ideally you might need to employ some tool, code-generator to do this automatically in a batch - i.e. you build, copy files externally, change names and repeat)
Build 'migrations' for each case (Table name) also - save migrations
as files - and also do Update-Database -Script to save the actual
scripts for each case (important).
Save each migration - or we can
say a 'script' to represent.
Once done - you'd need to merge the
migrations - scripts - into one big master script - i.e. remove the
identical set of tables (leave just one of course) - and copy all
different sets for membership tables.
Remove the migration table
from the database - as that'd surely be out of sync and won't let you
do anything (or there is also a flag in code I think to just ignore
that, don't have it right now). For details see below in my other
post.
Deploy one master Db - using script you created
Deploy the
specific code - to each of the sites.
Pray it'd all work :)
There must be something smarter - but on the other hand, migrations are not made to work for such scenarios, so it's going to be hard if not impossible to pull this off.
Some general info that might help...
How to synchronize migrations with existing databases - geared toward production scenarios, maintaining Db-s and CF to match. It's not exactly what you need but has a detailed description, possible ways to resolve this which I wrote a while ago...
MVC3 and Code First Migrations - "model backing the 'blah' context has changed since the database was created"
To summarize...
What works for me is to use Update-Database -Script
That creates a script with a 'migration difference', which you can
manually apply as an SQL script on the target server database (and you
should get the right migration table rows inserted etc.).
If that still doesn't work - you can still do two things...(more inside)...
I don't know of anyway to make Entity Framework do this automatically across all entities. But you could force a table name, or schema using attributes or fluent API to get the desired effect. For example:
[Table("[put prefix here]_Users", Schema = "[put schema here]")]
public class User {
// ...
}
I wonder what you are using for updating a client database when your program is patched?
Let's take a look at this scenario:
You have a desktop application (.net, entity framework) which is using sql server compact database.
You release a new version of your application which is using extended database.
The user downloads a patch with modified files
How do you update the database?
I wonder how you are doing this process. I have some conception but I think more experienced people can give me better and tried solutions or advice.
You need a migration framework.
There are existing OSS libraries like FluentMigrator
project page
wiki
long "Getting started" blogpost
Entity Framework Code First will also get its own migration framework, but it's still in beta:
Code First Migrations: Beta 1 Released
Code First Migrations: Beta 1 ‘No-Magic’ Walkthrough
Code First Migrations: Beta 1 ‘With-Magic’ Walkthrough (Automatic Migrations)
You need to provide explicitly or hidden in your code DB upgrade mechanism, and - thus implement something like DB versioning chain
There are a couple of aspects to it.
First is versioning. You need some way of tying teh version of teeh db to the version of the program, could be something as simple as table with a version number in it. You need to check it on executing the application as well.
One fun scenario is you 'update' application and db successfully, and then for some operational reason the customer restores a previous version of the db, or if you are on a frequent patch cycle, do you have to do each patch in order or can thay catch up. Do you want to deal with application only or database only upgrades differently?
There's no one right way for this, you have to look at what sort of changes you make, and what level of complexity you are prepared to maintain in order to cope with everything that could go wrong.
A couple a of things worth looking at.
Two databases, one for static 'read-only' data, and one for more dynamic stuff. Upgrading the static data, can then simply be a restore from a resource within the upgrade package.
The other is how much can you do with meta-data, stored in db tables. For instance a version based xsd to describe your objects instead of a concrete class. That's goes in your read only db, now you've updated code and application with a restore and possibly some transforms.
Lots of ways to go, just remember
'users' will always find some way of making you look like an eejit, by doing something you never thought they would.
The more complex you make the system, the more chance of the above.
And last but not least, don't take short cuts on data version conversions, if you lose data integrity, everything else you do will be wasted.