I need to map to a view when using EF6 with migrations.
The view pivots 2 other tables to enable a simple summary view of the underlying data, the idea being it allows us to use this in a summary index view.
The issue I have is I am unable create a migration that either deploys the view (ideal goal) or deploys the DB without the view for later manual deployment.
In most attempts, following other SO questions, I end up either deadlocking the Add-Migration and Update-Database commands or generally causing an error that breaks one or the other.
What is the current best way to use EF6 to access views, even if I lose the ability to automatically deploy them with the migrations, and not cause errors with migrations.
Further detail
The Db contains 2 tables Reports and ReportAnswers. The view ReportView combines these two and pivots ReportAnswers to allow some of the rows to become columns in this summary view.
Reports and ReportAnswers were depolied via EF Migrations. The view is currently a script that needs be added to the deployment somehow.
Reports, ReportAnswers & ReportView are accessible from the db Context
public virtual DbSet<ReportAnswer> ReportAnswers { get; set; }
public virtual DbSet<Report> Reports { get; set; }
public virtual DbSet<ReportView> ReportView { get; set; }
I have tried using Add-Migration Name -IgnoreChanges to create a blank migration and then manually adding the view to the Up() and Down() methods but this just deadlocks the migration and update commands, each wanting the other to run first.
I have also tried using modelBuilder.Ignore<ReportView>(); to ignore the type when running the migrations but this proved incredibly error prone, even though it did seem to work at least once.
I just walked around interesting article about using views with EF Core few days ago, but I found also the very same using EF 6.
You may want to use Seed method instead of migration Up and Down methods.
protected override void Seed({DbContextType} context)
{
string codeBase = Assembly.GetExecutingAssembly().CodeBase;
UriBuilder uri = new UriBuilder(codeBase);
string path = Uri.UnescapeDataString(uri.Path);
var baseDir = Path.GetDirectoryName(path) + "\\Migrations\\{CreateViewSQLScriptFilename}.sql";
context.Database.ExecuteSqlCommand(File.ReadAllText(baseDir));
}
Your SQL command should look like sample below.
IF NOT EXISTS (SELECT * FROM sys.views WHERE object_id = OBJECT_ID(N'[dbo].[{ViewName}]'))
EXEC dbo.sp_executesql #statement = N'CREATE VIEW [dbo].[{ViewName}]
AS
SELECT {SelectCommand}
It is not perfect, but I hope at least helpful.
I found another blog post about this topic and the writer says to use Sql(#"CREATE VIEW dbo.{ViewName} AS...") in Up method and Sql(#"DROP VIEW dbo.{ViewName};") in Down method. I added it as you didn't supplied the code from Up and Down migration methods. Maybe good idea will be to add SqlFile instead of Sql method.
There is also option to create customized code or sql generator and plug it in to migrations, but I guess it is not the things you are looking for.
Let me know in comment in case you need additional help.
Related links:
Using Views with Entity Framework Code First
EF CODE FIRST - VIEWS AND STORED PROCEDURES
Leveraging Views in Entity Framework
DbMigration.Sql Method (String, Boolean, Object)
DbMigration.SqlFile Method (String, Boolean, Object)
Related
I have a specific EF Core 6.x question.
If the SQL table has a column removed. Then EF Core will throw a SqlException saying that it's an invalid column name unless I also update the C# model.
For example,
Create Table User
(
FirstName varchar(200)
,MiddleName varchar(200) null -- tried to remove this column after table is created
,LastName varchar(200)
)
I tried deleting the MiddleName column from the SQL Table. When I run a simple read call using EF Core 6, I get the error.
c# model
public class User
{
public virtual string FirstName { get; set; }
public virtual string? MiddleName { get; set; }
public virtual string LastName { get; set; }
}
var db = new EFDbContext(connectionString);
var data = db.Users.ToList(); // SqlException here after column removal
Is there any way to remove columns from the table without needing to update the c# class as well?
Tried making the C# property MiddleName not virtual.
Update:
In the event that I have an existing application. I would need to modify the c# model even if the codebase doesn't refer to the removed column anywhere. Alternatively, I can decorate the property with [NotMapped] or use the Ignore() method in the modelbuilder.
Both approaches means a rebuild of the assembly is needed and downtime during deployment.
NHibernate's mapping can be done using an XML file and thus all it takes would be a simple config file update.
I can't seem to find anything in EF Core that will reduce the headache of maintaining older codebases when schema changes occur.
EF creates a data model mapping internally to track the database schema and your code models. By removing the column in your database table, your code model no longer matches the database. Hence, the exception occurs.
This is definitely not be the answer you're looking for, but as far as I know EF Core need consistency between the models and the DB schemas to work.
I can think of 2 things here:
Maybe you could benefit from using a different ORM (Did you give Dapper a cahnce)
You might be facing an architectural issue, if there's more than one team working with the same database, and more than one system calling that database, the best way to avoid headaches in the future would be to isolate the data access layer and expose an API that serves all the involved systems.
That way, if the database changes, you just need to re-build the data access layer, no downtime for your clients.
And finally... in my opinion the ideal solution is a combination of both, create a decoupled data access layer, map things there and expose an API with the models your application needs.
Just ramping up on .NET Core and running into some unsolvable problems.
I have a stored procedure called GetLeads whose schema basically returns
LEADID (INT)
SALENO (INT)
OFFICE (INT)
PACKAGE (NVARCHAR(4)
SALEDATE (DATETIME)
For sake of an example, there is a table in the same database where that Stored Procedure resides called Prospects. Its schema is
LEADID (INT)
FNAME (NVARCHAR(50))
LNAME (NVARCHAR(50))
ADDR1 (NVARCHAR(150))
ADDR2 (NVARCHAR(150))
EMAIL (NVARCHAR(200))
PHONENO (NVARCHAR(200))
CONTACTDATE (DATETIME)
According to the docs, I can use the FromSql method of a context model to call a stored procedure. But what if the stored procedure returns a schema having a only a few (if any) of the fields that match the model of the context from which the call is made?
If I construct a Query such as:
var _ctx = new DbContext();
var pkgs = _ctx.Prospects.FromSql("EXEC dbo.GetPackages").ToList();
This obviously will fail due the fact that returned recordset doesn't contain fields relevant to the Prospects model.
Conversely, using something like:
var pkgs = _ctx.Database.ExecuteSqlCommand("EXEC dbo.GetPackages");
Is pointless as well because it will only return the number of records affected by the command execution.
So the FromSql method available in .NET Core obviously has its set of limitations although it's documented as available for making stored procedure calls.
Ideally, I'd like to see the Core Framework have the ability to make direct calls to Stored Procedures and Functions in much the same manner as is with EF within the .NET Framework and simplify things. But I realize that functionality isn't yet available in Core; though I hope it eventually does.
Anyway, enough rambling and to the point.
How or what currently is the most practical and widely accepted way to call a stored procedure under these set of circumstances in .NET Core? I know ADO.NET is an option but I'm speaking strictly within the paradigm of the Core Framework using EF
I'm going to give some credit to Selvin for his last comment as it did prompt me down the path for a solution. But I really needed an explicit answer.
I'm going to also acknowledge the much appreciated comment from Rahul for using Snickler as an alternative. But I wanted a solution that solely involved nothing more than EF and .NET Core. Quite honestly, all the comments are greatly appreciated
So here are the actual steps required to always be able to get the FromSql method to work with Stored Procedures within the Entity Framework. I am using the Database First approach using MVC but the overall methodology for the solution shouldn't matter.
Note: If the EF context has already been created, Step 1 can be skipped.
Step 1: Generate the EntityFramework context and associated Entity Models from the database using the dotnet utility.
dotnet ef dbcontext scaffold "Server=[YourDBServerName];Initial Catalog=[YourDatabaseName];Integrated Security=False;Persist Security Info=False;User ID=[YourUserId];Password=[YourUserPassword];" Microsoft.EntityFrameworkCore.SqlServer -o Models
Step 2: Add a ViewModel that defines all the fields that your stored procedure will return. Note: You could alternately add this as a Model instead, but I prefer to keep the stored procedure entities distinctly separated from the Models generated directly from the database for brevity and a clear separation of the architecture.
Equally important, the entity class MUST have a primary key or it won't work. EF requires that all entities have a primary key.
public partial class Lead
{
[Key]
public int leadid { get; set; }
public int saleno { get; set; }
public int office { get; set; }
public string package { get; set; }
public datetime saledate { get; set; }
}
Step 3: Add the stored procedure entity Model created in Step 2 to the Database context as a virtual DbSet. This should be located in the Models directory created by Step 1 and would have the word "Context" prepended in the name.
public virtual DbSet<Lead> Lead { get; set; }
After performing these steps 2 and 3 you can all any stored procedure using the FromSql method within the Core Framework
var _ctx = new DbContext();
var pkgs = _ctx.Prospects.FromSql("EXEC dbo.GetPackages").ToList();
Note: Since record sets returned from stored procedures don't require the presence of a primary key you could either:
Modify the Stored Procedure to include a Primary Key such as an identity field
Identify a couple of fields within the returned record set that would constitute as being viable primary keys and create a composite key within the Model
However the 2nd option requires some additional work and considerations because you'll have to use fluent API to set composite primary keys. EF Core won't allow Data Annotations to set composite keys.
Hopefully this shared information will help others who journey the same path with the similar question
I've always been a database oriented programmer, so up to this day, I've always used a database-driven approach to programming and I feel pretty confident in T-SQL and SQL Server.
I'm trying to wrap my head around the Entity Framework 6 code-first approach - and frankly - I'm struggling.
I have an existing database - so I did a Add New Item > ADO.NET Entity Data Model > Code-First from Database and I get a bunch of C# classes representing my existing database. So far so good.
What I'm trying to do now is explore how to handle ongoing database upgrades - both in schema as well as "static" (pre-populated) lookup data. My first gripe is that the entities that were reverse-engineered from the database are being configured with the Fluent API, while it seems more natural to me to create the new tables I want to have created as a C# class with data annotations. Is there any problems / issues with "mixing" those two approaches? Or could I tell the reverse-engineering step to just use data annotation attributes instead of the Fluent API altogether?
My second and even bigger gripe: I'm trying to create nice and small migrations - one each for each set of features I'm trying to add (e.g. a new table, a new index, a few new columns etc.) - but it seems I cannot have more than a single "pending" migration...... when I have one, and I modify my model classes further, and I try to get a second migration using add-migration (name of migration), I'm greeted with:
Unable to generate an explicit migration because the following explicit migrations are pending: [201510061539107_CreateTableMdsForecast]. Apply the pending explicit migrations before attempting to generate a new explicit migration.
Seriously ?!?!? I cannot have more than one, single pending migration?? I need to run update-database after every single tiny migration I'm adding?
Seems like a rather BIG drawback! I'd much rather create my 10, 20 small, compact, easy-to-understand migrations, and then apply them all in one swoop - no way to do this!?!? This is really hard to believe..... any way around this??
It is true that you can only have one pending migration open at a time during development. To understand why, you have to understand how the migrations are generated. The generator works by comparing the current state of your database (the schema) with the current state of your model code. It then effectively creates a "script" (a C# class) which changes the schema of the database to match the model. You would not want to have more than one of these pending at the same time or else the scripts would conflict with each other. Let's take an simple example:
Let's say I have a class Widget:
class Widget
{
public int Id { get; set; }
public string Name { get; set; }
}
and a matching table Widgets in the database:
Widgets
-------
Id (int, PK, not null)
Name (nvarchar(100), not null)
Now I decide to add a new property Size to my class.
class Widget
{
public int Id { get; set; }
public string Name { get; set; }
public int Size { get; set; } // added
}
When I create my migration, the generator looks at my model, compares it with the database and sees that my Widget model now has a Size property while the corresponding table does not have a Size column. So the resulting migration ends up looking like this:
public partial class AddSizeToWidget : DbMigration
{
public override void Up()
{
AddColumn("dbo.Widgets", "Size", c => c.Int());
}
public override void Down()
{
DropColumn("dbo.Widgets", "Size");
}
}
Now, imagine that it is allowed to create a second migration while the first is still pending. I haven't yet run the Update-Database command, so my baseline database schema is still the same. Now I decide to add another property Color to Widget.
When I create a migration for this change, the generator compares my model to the current state of the database and sees that I have added two columns. So it creates the corresponding script:
public partial class AddColorToWidget : DbMigration
{
public override void Up()
{
AddColumn("dbo.Widgets", "Size", c => c.Int());
AddColumn("dbo.Widgets", "Color", c => c.Int());
}
...
}
So now I have two pending migrations, and both of them are going to try to add a Size column to the database when they are ultimately run. Clearly, that is not going to work. So that is why there is only one pending migration allowed to be open at a time.
So, the general workflow during development is:
Change your model
Generate a migration
Update the database to establish a new baseline
Repeat
If you make a mistake, you can roll back the database to a previous migration using the –TargetMigration parameter of the Update-Database command, then delete the errant migration(s) from your project and generate a new one. (You can use this as a way to combine several small migrations into a larger chunk if you really want to, although I find in practice it is not worth the effort).
Update-Database –TargetMigration PreviousMigrationName
Now, when it comes time to update a production database, you do not have to manually apply each migration one at a time. That is the beauty of migrations -- they are applied automatically whenever you run your updated code against the database. During initialization, EF looks at the target database and checks the migration level (this is stored in the special __MigrationHistory table which was created when you enabled migrations on the database). For any migration in your code which has not yet been applied, it runs them all in order for you, to bring the database up to date.
Hope this helps clear things up.
Is there any problems / issues with "mixing" those two approaches?
No, there is no problem to mix them.
You can do more with fluent config than with data annotations.
Fluent config overrides data annotation when constructing the migration script.
You can use data annotations to generate DTOs and front-end/UI constraints dynamically - saves a lot of code.
Fluent API has class EntityTypeConfiguration which allows you to make domains (in DDD sense) of objects dynamically and store them - speeds up work with DbContext a lot.
I cannot have more than a single "pending" migration
Not 100% true. ( Maybe 50% but this is not a showstopper )
Yes, the DbMigrator compares your model "hash" to the database model "hash" when it generates the Db - so it blocks you before you make your new small migration. But this is not a reason to think you can not make small migration steps. I do only small migration steps all the time.
When you develop an app and you use your local db you apply the small migrations one by one as you develop functionality - gradually. At the end you deploy to staging/production all your small migrations in one dll with all the new functionality - and they are applied one by one.
I am currently working towards implementing a charting library with a database that contains a large amount of data. For the table I am using, the raw data is spread out across 148 columns of data, with over 1000 rows. As I have only created models for tables that contain a few columns, I am unsure how to go about implementing a model for this particular table. My usual method of creating a model and using the Entity Framework to connect it to a database doesn't seem practical, as implementing 148 properties for each column does not seem like an efficient method.
My questions are:
What would be a good method to implement this table into an MVC project so that there are read actions that allow one to pull the data from the table?
How would one structure a model so that one could read 148 columns of data from it without having to declare 148 properties?
Is the Entity Framework an efficient way of achieving this goal?
Entity Framework Database First sounds like the perfect solution for your problem.
Data first models mean how they sound; the data exists before the code does. Entity Framework will create the models as partial classes for you based on the table you direct it to.
Additionally, exceptions won't be thrown if the table changes (as long as nothing is accessing a field that doesn't exist), which can be extremely beneficial in a lot of cases. Migrations are not necessary. Instead, all you have to do is right click on the generated model and click "Update Model from Database" and it works like magic. The whole process can be significantly faster than Code First.
Here is another tutorial to help you.
yes with Database First you can create the entites so fast, also remember that is a good practice return onlye the fiedls that you really need, so, your entity has 148 columns, but your app needs only 10 fields, so convert the original entity to a model or viewmodel and use it!
One excelent tool that cal help you is AutoMapper
Regards,
Wow, that's a lot of columns!
Given your circumstances a few thoughts come to mind:
1: If your problem is the leg work of creating that many properties you could look at Entity Framework Power Tools. EF Tools is able to reverse engineer a database and create the necessary models/entity relation mappings for you, saving you a lot of the grunt work.
To save you pulling all of that data out in one go you can then use projections like so:
var result = DbContext.ChartingData.Select(x => new PartialDto {
Property1 = x.Column1,
Property50 = x.Column50,
Property109 = x.Column109
});
A tool like AutoMapper will allow you to do this with ease via simply configurable mapping profiles:
var result = DbContext.ChartingData.Project().To<PartialDto>().ToList();
2: If you have concerns with the performance of manipulating such large entities through Entity Framework then you could also look at using something like Dapper (which will happily work alongside Entity Framework).
This would save you the hassle of modelling the entities for the larger tables but allow you to easily query/update specific columns:
public class ModelledDataColumns
{
public string Property1 { get; set; }
public string Property50 { get; set; }
public string Property109 { get; set; }
}
const string sqlCommand = "SELECT Property1, Property50, Property109 FROM YourTable WHERE Id = #Id";
IEnumerable<ModelledDataColumns> collection = connection.Query<ModelledDataColumns>(sqlCommand", new { Id = 5 }).ToList();
Ultimately if you're keen to go the Entity Framework route then as far as I'm aware there's no way to pull that data from the database without having to create all of the properties one way or another.
This may be a bit silly, but all the applications I've built have always utilized the EF Code-First approach to generate the database. When using this method, I've always accessed the database through the Context:
public class RandomController : Controller
{
public CombosContext db = new CombosContext();
//
// GET: /Home/
public ActionResult Index()
{
var rows = db.Combos.OrderBy(a => a.Id).ToList();
However, what if the database is already created for me, OR I create one by adding entities to the schema/design surface and then generate the database from that. How would I access the db without the
public CombosContext db = new ComboxContext();
If the DB is already created, you can use the Database First approuch: http://blogs.msdn.com/b/adonet/archive/2011/09/28/ef-4-2-model-amp-database-first-walkthrough.aspx
A basic setup would by to rightclick the project in the solution explorer and click Add > new item. On the dialog, select Data on the left pane and ADO.net Entity Data Model and follow the wizard to create your model based on the database. This way, you will have a context object exactly the way you have with code first (with some minor changes, but works almost the same).
You can still do this with Code first and is the better approach IMHO. Use the Entity Framework Power Tools to reverse engineer your existing database into a code-first model.
http://visualstudiogallery.msdn.microsoft.com/72a60b14-1581-4b9b-89f2-846072eff19d/
See my demo on using it at:
http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/DEV215