We have multiple developers working on a project that uses Entity Framework 5.0. Every developer uses his own local SQL 2012 database so he can develop and test without impeding others.
At first, we used a hybrid of automatic migrations and code-based migrations. That didn't work well at all so we decided to disable automatic migrations and to only allow code-based. I should add that we started again with a clean database without a 'corrupted' _MigrationsHistory from all the automatic migrations.
So now the workflow is:
Developer changes his datamodel
Does add-migration <Name> and applies it to his database with update-database.
Checks in the datamodel change and the migration into Git.
Another developer pulls, receives the changes and applies it to his database.
So far, this worked well. However before today it was usually just me who made the migrations and the others applied them. But today there were migrations from three developers. I just pulled those migrations, did an update-database which went fine.
I also had a change to my own datamodel however so at the end of the update-database it gave me a warning that I still wasn't up to date so I did add-migration <my migration>. However when it scaffolded the migration, it gave me the changes of all the migrations I had already applied to the database. So: it tried to drop columns that had already been dropped, tried to create a table that already existed, etc.
How can that be? My assumption was that EF would just check the _MigrationsHistory table and find out which migrations weren't present in the table yet and apply those one by one ordered by the timestamp that's part of the name. But apparently not, because even when I undo my own changes and I have a clean environment it still complains my database isn't in sync with the model. But I just pulled those changes and applied them to my database. It is in sync. I can see the migrations that I just applied in the _MigrationsHistory table too.
The only thing I can think of is that I added a property to a datamodel that wouldn't result in a database change (I added a List<X> to datamodel Y where X is the many in the one-to-many relationship. This wouldn't result in a database change as X already had a foreign key to Y). Could that be it? If so, that's really fragile because there's no way to add a migration for that since there's no database change and I'm not sure how to fix this either.
I'm not sure how to deal with this, because I can of course just edit what it scaffolded and remove everything that has already been applied to my database. But then what? I check it in and then some other developer gets the same message that his database isn't up to date even after applying my new changes, scaffolds his own changes, gets the same nonsense scaffolding, edits it, checks it in and then the next developer gets it. It becomes a vicious circle and a similar one to what we had when we used automatic migrations and I thought we had fixed that by switching to code-based only. I can't trust it right now to do the right thing and it's a nightmare to work with like this.
What I also tried is adding the migrations I pulled from my coworkers one by one with update-database -t:201211091112102_<migrationname> but to no avail. It still gives me the erroneous scaffold.
So what did we do wrong here, or is EF simply not built for collaboration like this?
UPDATE
I created a reproducible test case, it's a bit of a lengthy dance though in order to simulate this multi user/multi database scenario.
https://github.com/JulianR/EfMigrationsTest/
Steps to reproduce when you have the above project (these steps are also present in the code):
add-migration Init
update-database (on database 'TestDb')
Change connection string to point to TestDb1
update-database on TestDb1
Uncomment property Foo on class Test
add-migration M1 to add property Foo to TestDb1
Comment out Test.Foo again
Change connection string to point to TestDb2
Exclude migration M1 from project so it doesn't get applied to TestDb2
Uncomment property Bar on class Test
update-database to apply Init migration to TestDb2
add-migration M2 to add property Bar to TestDb2
Change connection string to point to the original TestDb again
Include migration M1 into the project again
Uncomment property Foo on class Test
Uncomment property SomeInt on class Test
update-database
add-migration M3
update-database, get an error because M3 tries to add column Foo to database TestDb which was already just added by migration M1.
The above is to simulate three users, where user 1 inits his database, the other two use his initialization to create their database as well. Then user 2 and user 3 both make their own change to the datamodel and add it to source control together with the migrations needed to apply the changes. Then user 1 pulls the changes of user 2 and 3 while user 1 has also made a change to the database himself. Then user 1 calls update-database to apply the changes of user 2 and 3. He then scaffolds his own migration which then erroneously adds a change from user 2 or 3 to the scaffolded migration which causes an error when applied to user 1's database.
You need to add a blank "merge" migration that will reset the snapshot of the latest migration in the .resx file. Do this using the IgnoreChanges switch:
Add-Migration <migration name> -IgnoreChanges
See here for an explanation
You need to manually resolve migration conflicts just like you would code conflicts. If you update and there are new migrations, you need to ensure that the metadata behind the last migration matches the current model. To update the metadata of the migration, re-issue the Add-Migration command for it.
For example, before step 17 (Update-Database) in your scenario, you should issue the following command
Add-Migration M2
This will update the metadata to bring it in sync with your current model. Now when you try and add M3, it should be blank since you have not made any further model changes.
Option 1: Add a blank ‘merge’ migration
Ensure any pending model changes in your local code base have been
written to a migration. This step ensures you don’t miss any
legitimate changes when it comes time to generate the blank
migration.
Sync with source control.
Run Update-Database to apply
any new migrations that other developers have checked in. **
Note:****if you don’t get any warnings from the Update-Database
command then there were no new migrations from other developers and
there is no need to perform any further merging.
Run Add-Migration
–IgnoreChanges (e.g. Add-Migration Merge
–IgnoreChanges). This generates a migration with all the metadata
(including a snapshot of the current model) but will ignore any
changes it detects when comparing the current model to the snapshot
in the last migrations (meaning you get a blank Up and Down method).
Continue developing, or submit to source control (after running your
unit tests of course).
Option 2: Update the model snapshot in the last migration
Ensure any pending model changes in your local code base have been
written to a migration. This step ensures you don’t miss any
legitimate changes when it comes time to generate the blank
migration.
Sync with the source control.
Run Update-Database to
apply any new migrations that other developers have checked in. **
Note:****if you don’t get any warnings from the Update-Database
command then there were no new migrations from other developers and
there is no need to perform any further merging.
Run Update-Database
–TargetMigration (in the example we’ve been
following this would be Update-Database –TargetMigration AddRating).
This roles the database back to the state of the second last
migration – effectively ‘un-applying’ the last migration from the
database. ** Note:****This step is required to make it safe to edit
the metadata of the migration since the metadata is also stored in
the __MigrationsHistoryTable of the database. This is why you should
only use this option if the last migration is only in your local
code base. If other databases had the last migration applied you
would also have to roll them back and re-apply the last migration to
update the metadata.
Run Add-Migration
(in the example
we’ve been following this would be something like Add-Migration
201311062215252_AddReaders). ** Note:****You need to include the
timestamp so that migrations knows you want to edit the existing
migration rather than scaffolding a new one. This will update the
metadata for the last migration to match the current model. You’ll
get the following warning when the command completes, but that’s
exactly what you want. “Only the Designer Code for migration
'201311062215252_AddReaders' was re-scaffolded. To re-scaffold the
entire migration, use the -Force parameter.”
Run Update-Database to
re-apply the latest migration with the updated metadata.
Continue
developing, or submit to source control (after running your unit
tests of course).
MSDN have a great article on this. Please go through it.
Entity Framework Code First Migrations in Team Environments
We are having similar issues in our environment, here is what we've figured out so far and how we got around it:
When you have changes that you have applied (update-database) but not checked in, and then you receive changes from another developer who doesn't have your changes, this is where things seem to get out of sync. In our experience, it seems like the meta data that is saved for your own changes get over written by the meta-data from the other developer when you do the update-database process. The other developer doesn't have your changes, so the meta-data that gets saved is no longer a real reflection of your database. When EF does a comparison after that, it 'thinks' that your changes are actually new again because of the meta data change.
A simple, admittedly ugly workaround is to do another migration, and wipe out it's contents so you have empty up() and empty down() methods. Apply that migration and check it into source control and let everyone sync to that. This simply syncs up all of the meta data so everyone has all of the changes accounted for.
I have added an issue on codeplex, this issue causes many a head scratching in our team too.
The link is https://entityframework.codeplex.com/workitem/1670
I have put some thought into this and I hope I will contribute to the different opinions and practices presented here.
Consider what your local migrations actually represent. When working locally with a dev database, I use migrations to update the database in the most convenient way possible when adding columns etc to tables, adding new entities etc.
So, Add-Migration checks my current model (let's call it model b) against my previous model (model a) and generates a migration to go from a => b in the database.
To me it makes very little sense to try and merge my migrations with anyone elses migrations, if everyone indeed has their own database and there then exists some kind of stage / test / dev / production database servers in the organization. This all depends on how the team has it set up, but it makes sense to insulate each other from changes that other people make if you want to truly work in a distributed manner.
Well, if you work distributed and have some entity, Person, for example, that you work on. For some reason, lots of other people are also working on it. So, you add and remove properties on Person as needed for your particular story in the sprint (we're all working agile here, aren't we?), like Social Security number that you first made into an integer because you aren't that bright and then to a string etc.
You add FirstName And LastName.
You are then done and you have ten weird up and down migrations (you probably removed some of them while working since they were just crap) and you fetch some changes from the central Git repo. Wow. Your colleague Bob also needed some names, maybe you should've talked to each other?
Anyways, he has added NameFirst and NameLast, I guess... so what do you do? Well, you merge, refactor, change so it has more sane names... like FirstName and LastName, you run your tests and check his code, and then you push to the central.
But what about the migrations? Well, now would be the time to make a migration moving the central repo, or the branch "test" more specifically, contain a nice little migration from its model a => model b. This migration will be one and only one migration, not ten weird ones.
Do you see what I'm getting at? We are working with nice little pocos and the comparisons of them constitute the actual migrations. So, we shouldn't merge migrations at all, in my opinion, we should have migrations-per-branch or something like that.
In fact, do we even need to create the migration in the branch after merge? Yes, if this database is updated automatically, we need to.
Another thing to consider is to never actually creating a migration before doing a pull from the central repo. That means you will both get the other team members' migration code and their changes to the model before creating your migration.
Gotta work some more, those are my thoughts on this, at least.
The solution I was able to come up with (at least for 2 users, haven't tested for 3) is:
merging migrations to sync up the meta-data run update-database (this should fail), then
add-database and then
delete all of the generated code in up() and down() methods
this will still be run by update database but won't do anything, just bringing the metadata up to sync.
I agree with #LavaEater. The core of the issue, it would seem, is that migration scaffolding should be centralised. Perhaps as part of some automated/integrated build process each time a push occurs? Thereafter the resulting migrations can be pulled from the server by team-members.
This means that their own migration scripts should not be pushed to the server.
There is an easy way to have no merge conflicts/errors with migrations.
Work on your branch as you would do at any time.
If you merge to master and have merge errors then:
remove all *.cs files from migrations folder.
do git checkout master ./* inside migrations folder.
Recreate your migration.
Your snapshot is up2date and there is no merge conflict.
Also just before merging pull request to master you need to merge with master and do steps 3-6 ALWAYS.
Below is simple Powershell script that does steps 3-6:
function Write-Info($text)
{
Write-Color "$pwd", "> ", "$text" -Colour "Yellow", "Blue", "White"
}
function Create-Migration($project, $migrationName, $referenceBranch)
{
Set-Location "$SolutionPath\$project"
Write-Info "Going to migrations"
Set-Location "Migrations"
Write-Info "Removing ./*.cs"
Remove-Item ./*.cs
Write-Info "git fetch --all"
git fetch --all
Write-Info "git checkout origin/$referenceBranch ./*"
git checkout origin/$referenceBranch ./*
Set-Location ..
Write-Info "Creating migration $migrationName "
dotnet ef migrations add "$migrationName"
}
I am working with that method for last half year. 0 merge conflicts to resolve when it comes to migrations 8).
Related
The title is a bit confusing but I'm trying to specify that the problem is not initiating a code first model and migration for an existing database, but comes after that.
I needed to activate automatic migration because we switched to a code first model for our system. So, here's what has been done:
I created an empty InitialCreate for the existing database
I did some other scripts because there were some changes, those worked OK and the scripts were created and run on database
The problem happen when I want to use those script and migrate another database that was not yet initialized this way. I don't know what to do.
When I try to run Update-database I get the error:
Unable to update database to match the current model because there are pending changes and automatic migration is disabled. Either write the pending model changes to a code-based migration or enable automatic migration. Set DbMigrationsConfiguration.AutomaticMigrationsEnabled to true to enable automatic migration.
You can use the Add-Migration command to write the pending model changes to a code-based migration.
If I do a Add-Migration it creates a migration with everything in it again, all the create tables, it's like it ignores my currents scripts, the fact that there is a InitialCreate empty and the other scripts.
OK, you have 2 databases - let's say DEV and PROD. Both are in an identical state and have the same schema before migrations have been enabled. This is what to do:
1 - Add migrations to your DEV environment and set your database initializer to MigrateDatabaseToLatestVersion. Another option is to programatically run migrations.
enable-migrations
// take a snapshot of current state. -IgnoreChanges prevents recreate of existing objects.
add-migration InitialBaseline -IgnoreChanges
update-database
2 - There are several ways to keep the other database(s) in sync:
A) Maintain migrations in parallel by changing the connection string. So point at PROD, and run update-database to create the __MigrationHistory table and apply the initial, blank, baseline. I don't recommend this option for PROD databases (see below).
B) Sync with scripts. Many organizations don't want EF applying changes and instead require DBAs to apply scripts. For this option, you may want to set your database initializer to NULL. In this case you can do an update-database -Script to generate changes. This would be done on migrations subsequent to your initial baseline since they are already in sync. See here for more info on this technique.
C) Use database projects or a diff tool to keep things in sync.
Now when you go and change your models in DEV:
add-migration Changes1
update-database
For option A, change connect string and repeat. For option B, use update-database -Script. For option C, resync with tool.
NOTE: "I needed to activate automatic migrations..." - Automatic migrations are a whole different matter and can complicate the process. See here.
I've done this kind of procedure multiple times successfully, but for the first time, it seems to not work. I want to merge two branches of my application, they have different Code First Migration states. I'm aware of the fact that during development of two versions, the schema changed in both, resulting in a different snapshots, so I need to re-scaffold my merged Migrations to match the database snapshots, basically following this guide, Option 2. In the given Project I have dozens of Migrations and multiple of them were merged and re-scaffolded successfully as described.
So what I'm doing is bringing the database state to the database version before Migration (e.g. Update-Database -targetMigration 201511261243087_BeforeMigration, Step 4 of the Guide) and re-scaffold the merged Migrations in the correct order, so they can re-scaffold the metadata.
When I use the Command Add-Migration 201512170931290_FirstMergedMigration, I get the following error message:
Unable to generate an explicit migration because the following
explicit migrations are pending:
[201512170931290_FirstMergedMigration,
201512140935307_SecondMergedMigration]. Apply the pending explicit
migrations before attempting to generate a new explicit migration.
What? I wasn't expecting this. I'm expecting the command to re-scaffold the existing migration's metadata and update the Schema snapshot. But it's assuming that I want to add a new Migration. The -force flag doesn't help either.
When I follow the recommended step and Update-Database, EF will create a new Migration, called: 201512210931290_201512170931290_FirstMergedMigration. Now THATs really crazy.
I wish there was a separate command to Re-Scaffold Metadata so it would maybe show a clearer error message.
Anybody seen this before?
UPDATE:
See my own answer.
OK so it seems I found the issue here... It seems like the first migration I attempted to re-scaffold, didn't contain any schema changes, because it was only adding Indexes etc. It seems that EF will try to add a new migration if it doesn't detect the change in the given Migration. What I was able to do was to skip that Migration, continue with the next one, which resulted in the expected:
Only the Designer Code for migration
'201512140935307_SecondMergedMigration' was re-scaffolded. To
re-scaffold the entire migration, use the -Force parameter.
I checked the database and the __MigrationHistory looks good, so do the Tables. It would mean less trouble if EF had generated a better error message instead of trying to add a new migration where I clearly was trying to re-scaffold.
I am not even sure how to phrase this one. I have migrations applied to the db: Migration One. Migration Two. Migration Three. I just pulled a "Migration 2.5" - if I try to revert to migration Two and `Update-Database', the db updates successfully and says there are pending changes. These "pending changes" are Migration 2.5, even though it's been applied - EF wants me to add them again.
Now, normally, I get around the problem like this - revert to Migration One or Two, delete Migration 2.5, apply changes, and create a new "Migration 4" that comes LAST. This way the problem is avoided. However, I cannot change around the order of the migrations now. How do I solve this?
Edit: I think it's important to note that my db does work properly. If there is a db/code mismatch, EF throws an error stating so - this isn't happening. The problem is that the moment I do need to Add-Migration with a legitimate change, it will try to duplicate the "2.5" changes along with the actual legitimate changes.
This isn't really an answer to the original question - I have not found a way to make EF recognize that it has applied an "out of order" migration. However, here are two ways to get around the problem, which, more specifically, is getting the quoted error when trying to pull an "out of order" migration:
Unable to update database to match the current model because there are pending changes and automatic migration is disabled. Either write the pending model changes to a code-based migration or enable automatic migration. Set DbMigrationsConfiguration.AutomaticMigrationsEnabled to true to enable automatic migration.
Create an empty "Fix" migration - this is a fairly simple, but dirty solution. You simply run Add-Migration. You then delete migration actions, so you should be left with empty blocks like so:
public override void Up()
{
}
public override void Down()
{
}
You then run Update-Database. The reason this works is simple - as pointed out earlier, EF the "out of order" migration was not applied, despite the fact that it was actually applied the first time you ran Update Database- note that you should be able to verify that it was applied with Get-Migrations. Anyway, for some reason EF wants to duplicate it, but will be placated by an empty migration instead. Makes no sense to me.
This solution is undesirable as it leaves a pointless migration, but may be your only route if you cannot easily get to the environments which the out of order migration has already been pushed to.
a. Update-Database Target-Migration "The Migration Before The Out of Order One" to revert to the migration before this problem began.
b. Now manual delete the "Out of Order" migration file from your migrations folder.
c. Run Update-Database again to bring your db current with all changes EXCEPT the "Out of Order" migration.
d.Run Add-Migration - this will recreate the "Out of Order" migration you deleted in the beginning, but in its proper,or at least functional, place.
e. Run Update-Database and you are done.
Note that this will create problems unless synchronized with your team for reasons I am too lazy to go into. #2 is better, but #1 is easier and faster.
P.S. Stackoverflow nested list syntax is too much for me to handle.
I'm learning to use EF Code First Migrations from https://msdn.microsoft.com/en-us/data/jj591621.aspx
Somewhere it said:
Code First Migrations has two primary commands that you are going to
become familiar with.
Add-Migration will scaffold the next migration based on changes you
have made to your model since the last migration was created.
Update-Database will apply any pending migrations to the database.
I don't understand what's Add-Migration doing exactly. To more precise, my problem is with:
since the last migration was created
In order to create a migration, it should pick two database structure to compare.
Obviously, one side is the current structure of models in the code. But what is the other side? The options are:
Populating a database structure by unifying all migrations from initial to the last migration before this?
Comparing it to a database which has the old structure?
Check the code behind files of your migrations - they contain a lot of metadata, including a snapshot of the model from when it was created.
So, when you run Add-Migration the process is approximately this:
Build a model based on your code
Find the previous model from your last migration (if applicable)
Compare the two models
Generate a migration based on the difference
There's a useful article with some information and videos that cover this in more detail.
I have a Web project which has already been published to the production server. On my development machine, I made some changes on the model class and ran Update-Database without run Add-Migration. I tried to
Update-Database -TargetMigration:"201304020555457_previous_migration"
but I got the error of
Automatic migration was not applied because it would result in data loss.
How to roll back so I can get the full SQL script for applying on the production server?
Update:
I just published the code, start the page in browser, and the database changes (adding columns) were done after it's executed. I don't need to do anything. Does it means it's really not necessary to run Add-Migration every time?
You could undo the changes you made , then run Update-Database.
Then re-make the changes and this time run Add-Migration
If your initializer is MigrateDatabaseToLatestVersion and your configuration has AutomaticMigrationsEnabled set to true then it will update your db without the need for migrations, if it can.
If you really wanted you could only add migrations for changes that your really want documented or you want to modify, You could skip adding simple columns and add migrations for renaming columns for example.
But if i where you I'd keep migrations complete to avoid issues further down the line.